dim_STAT User's Guide

^<<

^[up]

^>>

^{dim_STAT User's Guide. by Dimitri}

Report Tool

This User's Guide is completely written using Report Tool!! And as so often, this tool was mainly created to cover my own day to day needs.
Quite often I have to write reports to show performance findings, to present the observed system / application activity, etc., etc. Yes, etc. because sometime we have to write too much to make things work or simply to protect people from doing stupid things. :))
OK, you've started to write your document for a French customer, so you write it in French, and then it appears that the majority of the development team only speaks English. You start to keep two copies in parallel for the same document: FR/EN. Then you discover something very important but you can not say it yet your customer, but you absolutely need to communicate it internally. So you split the document once again: FR/EN and Customer/Internal, which means four different documents. The next split will give you eight version of the document. But it is still based on the same source of information. The result is a lot of hours spent doing copy-paste of activity graphs from the browser, teamquest, best1, patrol, etc. into your wordprocessor. It makes me cry... :))
I was really tired of this situation and tried to imagine something different.

Overview

The first issue was the choice of format: At least everybody on any platform is able to read HTML. So that's an easy one. If needed you can easily convert HTML into other formats, like PDF, etc.
The next problem is harder to solve. It was my idea to find a solution for generating different kinds of documents from the same main data source. When you take a look at any document, how is its content organized? You'll see:

Document = N x Chapters
Chapter = M x Sections
Section = P x Paragraphs
and so on ...
Smallest part = Smallest part :-)

It all depends on what is your smallest part. So, I've named my smallest part a Note and a Document or Report is presented simply as an ordered tree of Notes.
The main points :

the position of each Note in a Report is decided by its parent-ID (level + 1) and order number (same level)

Note : each Note has/contains:
     - a Data Type
     - a Title
     - text comments
     - possibly an attachment (depends on Data Type)
     - a list of attributes

Attributes : any Note may have zero, one or several attributes on:
     - Language (French, English, ...)
     - Confidentiality (Personal, Customer, ...)
     - ... (any other can be easily added into the system)

Data Type : the list of Data Types is fixed (but may be extended):
     - Text
     - HTML
     - Image
     - Binary
     - dim_STAT collect
     - SysINFO
     - HTML.tar.Z archive

Any Note can be created/edited/deleted at any time. During Report generation you only need to choose the right criteria for your requirements to create a valid document with all parts corresponding to the criteria.

Datatype: Text, HTML, Image, Binary

These data types are quite similar, you can create any note with any text, html, image or binary file in an attachment, with or without your comments. Except binary, any other file may be presented "In-Line" or "Linked".
In-Line means your file will be part of the main document page and the visible contents, ex: text directly included, image showed, etc.
Linked means linked :)), meaning that the main document page will only include a link to your attachment. However, this attachment will be always included with document.
Note: the same idea is applied to other types of Notes as well.

Datatype: SysINFO

This is a special type, with the purpose to get on-line system information from any host on the network that runs STAT-service. Of course only if you have permission to access this service and SysINFO.

Datatype: HTML.tar.Z

A special type in case you want to integrate into your Report any other documents, already written, that are converted to HTML and archived into a single tar.Z file. As you may have several files in your archive, the tool will ask you for the name of the 'main' file, which will maintain references to all other files.

Datatype: dim_STAT-Snapshot

A type for when you've saved graph pages based on Java applets during dim_STAT analyzis. You may integrate them 'as is', the tool will extract the applet data and insert them as Note contents.
Probably this should be deprecated, as any graph can be saved in PNG format, or you could simply convert it to PNG or GIF.

Datatype: dim_STAT-Collect

This is a very special type, it helps you to generate all STAT graphs automatically and it will save you a lot of time. Follow the example below.

Preview / Generate / Publish

At any moment you can 'Preview' your Report or 'Generate' a current/final version to be accessed on-line, or saved and shared as tar.Z archive, or as a single PDF file. Also, your document may be published on another site (actually, this part is limited to the same physical host).

Export / Import

These features explains why Report Tool is called 'Mobile'. At any time you can export your Report and import it into any other dim_STAT server. This means: you edit/prepare everything on your laptop, and from time to time you synchronize your work with a central repository. Also, it gives you a simple way to prepare your own templates! Instead of starting a new report every time, just import your template (old report) and continue.

Let's try! New Report

Now relax, take your coffee, be sure you've 20 minutes of free time (while nobody is stressing you), your GSM is off, you're ready to listen ... go to the dim_STAT Main page and click on 'Report Tool'.

Click on Report Tool

As you could have expected, nothing yet here for the moment.
Let's click on the "New Report" button.

New Report

All you need to do here is just to fill in the new report form:

ID: unique digital number
Title: the main title
Owner: owner information
Chart: any additional comments to be present on the cover page
Use: choose a pre-configured Report template

and click on "Create"

Edit Report

Wow! It works! :))
With the 'big' buttons, you may now:

Hide/Show Note comments
Preview your report
Generate the report
go Home (back to the main Report page)

But if you'll hover your mouse over the pre-generated notes you'll see pop-ups explaining each action.

Edit Actions

And now:

click on the 'down' icon to create a new note 'after' the current one (same parent level)
click on the 'right' icon to create new 'child' note 'under' the current one (parent level+1)
click on the 'cut' icon to cut and then paste (may go to 'trash' if need to be deleted (end of screen))
click on the 'data' to edit/view the Note

Let's edit 'General Information' (click on 'data' icon).

Edit Note

From here you may see the current Note preview and edit the Note comments or attributes. If you change only attributes, then click on the corresponding button to apply the changes. If you want to modify the Note comments, click on 'Edit Note'. BTW, you can also do that with any external editor.

Edit Note, continue...

Add what you want in the text fields (you may use any HTML tags, etc.)

Edit Note, continue2...

Note: if you choose Text-format option your text is auto-formatted.

an empty line is seen as a 'new paragraph'
three spaces at the start of the line are replaced by a "blanked-tabulation"
some kind of limited wiki-like syntax is supported (see below example of input text containing wiki-like tags and its output result)..

Save the Note.

Wiki-Like syntax: INPUT

Here is a =!Big BOLD Header!=

Here is just a text +!with INCREASED font size!+

*!Here!* or **Here** will be a bold text

/!Here!/ is text in italic

_!Here!_ or __Here__ will by underlined test

__Simple TEXT List__ :
   - one
   - two
   - three

__Simple HTML List__ :
 * one
 * two
 * three


__Simple code or text formatted__ :
[code]
$ ls -l /usr/sfw/bin/*
...
$ ps -ef
...
$ pkill -9 oracle
[/code]

__Simple Table__ :

| **System/Performance** | **TPS** | **Resp.Time(ms)** |
| M5000  |  4.500 | 10.0 |
| M8000  |  8.000 | 10.0 |
| M9000  | 15.000 | 9.2 |

Wiki-Like syntax: OUTPUT

Here is a
Big BOLD Header

Here is just a text with INCREASED font size

Here or Here will be a bold text
Here is text in italic
Here or Here will by underlined test
Simple TEXT List :
     - one
     - two
     - three
Simple HTML List :

one
two
three

Simple code or text formatted :
$ ls -l /usr/sfw/bin/*
...
$ ps -ef
...
$ pkill -9 oracle
Simple Table :

System/Performance TPS Resp.Time(ms)

M5000 4.500 10.0

M8000 8.000 10.0

M9000 15.000 9.2

Edit Note, continue3...

You may re-edit again or open the door :))

Edit Report, continue...

Let's fill other notes in the same way...

Edit Report, continue2...

So far so good :))
Now, I want to add a SysINFO Note for both hosts 'tahiti' and 'java'. SysINFO data is collected on-line, at the moment you're asking for and it's an easy way to keep your document updated at the moment you're writing. BTW, look into the STAT-service package to know how it is configured on the host side. You may extend it with any other information you need.
So, a new SysINFO note under 'Software Configuration'... (right icon)

Add Note

New Note -- SysINFO

As the tool has no idea what kind of Note you want to add, it will ask you to choose one before it can continue. Also, I did not want to add too much complexity to the interface.
So, just click on 'SysINFO' here...

New Note -- SysINFO Form

Here you will need to fill in the SysINFO form: the usual data (title/comments/attributes) and SysINFO specific ones:

the host name
the host's STAT-service port

As SysINFO output is usually quite wide, it's preferred to keep it as an 'External Link'.
Save the Note. If you gave the right hostname, port and the STAT-service is up and running on this host, You'll receive your data in a few seconds, in our example from the 'tahiti' domain :))

New Note -- SysINFO Result

Because I asked for 'Linked' contents, there is only a link to SysINFO data from 'tahiti'. Let's click on it to see if it works correctly.

New Note -- SysINFO Link Contents

Edit Report, continue3...

As you see, I've my new SysINFO note under 'Software Configuration'. Let's get SysINFO from 'java' host now and place it 'under' current tahiti SysINFO...

Edit Report, continue4...

Now, under 'Hardware Configuration' I want to add an image representing my platform diagram (a very simple image, just for those who are not able to imagine two hosts with one storage device :)), but "a picture says more than a thousand words". :))
So: 'Hardware Configuration' -> Image -> ...

Add New Note -- Image

Once again, similar info to fill, except you may give a name of your image file to upload [Browse]. Let's fill it and save as 'In-Line' attachment.

Add New Note -- Image Inline

Oops, it's TOO BIG! And that's not so you can see it better!! I prefer to keep all big images 'linked'.
So, [Edit Note] -> 'As External Link' (no more need to give image file again) -> [Save Note]

Add New Note -- Image Linked

That's better!!
Now, let's add a 'dim_STAT Collect' note!
Leave this page [Door], go to the end of Report and click on the [Right] icon on 'Report' note, then choose 'dim_STAT Collect'.

Add New Note -- dim_STAT Collect, Step1

The dim_STAT Collect Note needs several steps to be created:

1. setup dim_STAT server database parameters, [Next]
2. select STAT collect you want to use, [Next]
3. select STATs you want to see and time interval, [Next]
4. [Finish] or select STATs you want to see and time interval, [Next] (goto 4)
5. graph titles, choose graph parameters, [Save]

We are on the Step-1 here, and if you don't have any data collected, you may get them from the 'Default' demo collect:

Server : localhost
Port : Default
Database : Default

[Next]...
NOTE: the interface becomes more optimized and more extended with each new release, so screen shots are probably not everywhere up to date.

Add New Note -- dim_STAT Collect, Step2

Choose STAT collect here and Search mode. We have already the log messages from the 'java' host, each message was added before any of the tests started, so it's quite easy to find them, corresponding to the time interval for each test. Otherwise we can always do a 'Date and Time' search, but you'll quickly understand thatt that is much more painful compared to LOG messages.

NOTE: with version 8.0, more options added to simplify reporting:

replay the same time slices for N days (in Date and Time)

auto include time/date into generated graph titles

replace on-the-fly some part (max 5) of the LOG messages

Add New Note -- dim_STAT Collect, Step3

Now we need to choose the type of graphs we want to see and the time interval of them.
NOTE:

All the Per-Host STATs are Bookmarks. The more you created Bookmarks during analyzing, the more data you can generate in for report.
When I selected the two hosts, the tool gave me also Multi-Host STATs, depending on stat commands to be present or not. Each STAT (like in Multi-Host Analyze) will put all requested hosts onto single graph.

Add New Note -- dim_STAT Collect, Step3 continue

Here we're choosing:

per host : CPU busy%, Run queue, Mutex spin, System calls/s
multi-host : CPU busy%, Network load bytes/s and packets/s

Time interval: as we know each test run for ~15min, we can choose a time interval of '15 min. After each LOG message.
[Next]...

Add New Note -- dim_STAT Collect, Step4

So, this looks OK. I've got my STATs selected with a pre-populated graph title (from the LOG message). BTW, you may see that all your previously selected STATs are pre-selected here (the selection is saved via cookies and specific to each database name).
[Finish] ...

Add New Note -- dim_STAT Collect, Step5

Here you have to specify the graphs parameters:

Main title
per graph title
order generation
graph mode, style, size, etc.
Auto-AVG: good to select if you have too large time intervals and your graph become too dense
Show LOG/TASK (as during analyze)
Show processing - get generation output on the browser. Not all browsers work correctly with this feature, some are waiting for an EOF before they show something. If you don't choose this option, processing output is always printed into a /tmp/.report.log file on the Report Tool server side.

[Save]...
Now you're free to start doing something else, because your machine is working for you and all you have to do is sit back and relax. Once you get use to the report tool, you'll ask it to generate A LOT OF graphs at the same time and you've time on your hands to do something else.

Add New Note -- dim_STAT Collect Result

Here is the final result after all the graphs are generated!
Click on a link to see the graph results.
NOTE: If you remember, I've selected generating order by Collect , and what I see now is a list of collects first, and each collect link will show me all selected STAT graphs for the same given STAT collect.
Now, if I select the by STATs order generation - I'll see here a STAT list, and each link will show me the same STAT metric for different collects on single page...

Add New Note -- dim_STAT Collect Contents, ordered by:Collect

Add New Note -- dim_STAT Collect Result per STATs

As you see here, the single STAT link contains all given collects, so if you want to compare the network usage in different cases, just click on either the bytes/sec or the packets/sec link.

Add New Note -- dim_STAT Collect Contents, ordered by:STATS

Edit Report, next...

Edit Report -- Cut

Last thing now: I don't want to see my 'per STAT' first in Report section, just let's move it at the end...
Click on [Cut] icon, then [Paste] where you want ([Trash] icon does delete operation!)

Edit Report -- Paste!

Edit Report -- Pasted...

Edit Report -- Preview

Edit Report -- Preview Output

Edit Report -- Preview Output2

Generate Report

Generated Report documents

Report Tool Home

THAT'S ALL, folks! :))
The export file of this demonstration report may be found within dim_STAT distribution as 'ExpReport_15.tar.Z'. You may import and play with it as long as you want! :))
Also, for good first exercise you may try to generate your first graphs from 'Demo collect' giving by default in your dim_STAT database!...

^<<

^[up]

^>>


From here you may see the current Note preview and edit the Note comments or attributes. If you change only attributes, then click on the corresponding button to apply the changes. If you want to modify the Note comments, click on 'Edit Note'. BTW, you can also do that with any external editor.


Add what you want in the text fields (you may use any HTML tags, etc.)

System/Performance	TPS	Resp.Time(ms)
M5000	4.500	10.0
M8000	8.000	10.0
M9000	15.000	9.2


Because I asked for 'Linked' contents, there is only a link to SysINFO data from 'tahiti'. Let's click on it to see if it works correctly.


As you see, I've my new SysINFO note under 'Software Configuration'. Let's get SysINFO from 'java' host now and place it 'under' current tahiti SysINFO...


Once again, similar info to fill, except you may give a name of your image file to upload [Browse]. Let's fill it and save as 'In-Line' attachment.


As you see here, the single STAT link contains all given collects, so if you want to compare the network usage in different cases, just click on either the bytes/sec or the packets/sec link.