|dim_STAT User's Guide. by Dimitri|
This User's Guide is completely written using Report Tool!! And as so often, this tool was mainly created to cover my own day to day needs. Quite often I have to write reports to show performance findings, to present the observed system / application activity, etc., etc. Yes, etc. because sometime we have to write too much to make things work or simply to protect people from doing stupid things. :)) OK, you've started to write your document for a French customer, so you write it in French, and then it appears that the majority of the development team only speaks English. You start to keep two copies in parallel for the same document: FR/EN. Then you discover something very important but you can not say it yet your customer, but you absolutely need to communicate it internally. So you split the document once again: FR/EN and Customer/Internal, which means four different documents. The next split will give you eight version of the document. But it is still based on the same source of information. The result is a lot of hours spent doing copy-paste of activity graphs from the browser, teamquest, best1, patrol, etc. into your wordprocessor. It makes me cry... :)) I was really tired of this situation and tried to imagine something different.
The first issue was the choice of format: At least everybody on any platform is able to read HTML. So that's an easy one. If needed you can easily convert HTML into other formats, like PDF, etc. The next problem is harder to solve. It was my idea to find a solution for generating different kinds of documents from the same main data source. When you take a look at any document, how is its content organized? You'll see:
It all depends on what is your smallest part. So, I've named my smallest part a Note and a Document or Report is presented simply as an ordered tree of Notes. The main points :
- Document = N x Chapters
- Chapter = M x Sections
- Section = P x Paragraphs
- and so on ...
- Smallest part = Smallest part :-)
Any Note can be created/edited/deleted at any time. During Report generation you only need to choose the right criteria for your requirements to create a valid document with all parts corresponding to the criteria.
- the position of each Note in a Report is decided by its parent-ID (level + 1) and order number (same level)
- Note : each Note has/contains:
- a Data Type
- a Title
- text comments
- possibly an attachment (depends on Data Type)
- a list of attributes
- Attributes : any Note may have zero, one or several attributes on:
- Language (French, English, ...)
- Confidentiality (Personal, Customer, ...)
- ... (any other can be easily added into the system)
- Data Type : the list of Data Types is fixed (but may be extended):
- dim_STAT collect
- HTML.tar.Z archive
|Datatype: Text, HTML, Image, Binary|
These data types are quite similar, you can create any note with any text, html, image or binary file in an attachment, with or without your comments. Except binary, any other file may be presented "In-Line" or "Linked". In-Line means your file will be part of the main document page and the visible contents, ex: text directly included, image showed, etc. Linked means linked :)), meaning that the main document page will only include a link to your attachment. However, this attachment will be always included with document. Note: the same idea is applied to other types of Notes as well.
This is a special type, with the purpose to get on-line system information from any host on the network that runs STAT-service. Of course only if you have permission to access this service and SysINFO.
A special type in case you want to integrate into your Report any other documents, already written, that are converted to HTML and archived into a single tar.Z file. As you may have several files in your archive, the tool will ask you for the name of the 'main' file, which will maintain references to all other files.
A type for when you've saved graph pages based on Java applets during dim_STAT analyzis. You may integrate them 'as is', the tool will extract the applet data and insert them as Note contents. Probably this should be deprecated, as any graph can be saved in PNG format, or you could simply convert it to PNG or GIF.
This is a very special type, it helps you to generate all STAT graphs automatically and it will save you a lot of time. Follow the example below.
|Preview / Generate / Publish|
At any moment you can 'Preview' your Report or 'Generate' a current/final version to be accessed on-line, or saved and shared as tar.Z archive, or as a single PDF file. Also, your document may be published on another site (actually, this part is limited to the same physical host).
|Export / Import|
These features explains why Report Tool is called 'Mobile'. At any time you can export your Report and import it into any other dim_STAT server. This means: you edit/prepare everything on your laptop, and from time to time you synchronize your work with a central repository. Also, it gives you a simple way to prepare your own templates! Instead of starting a new report every time, just import your template (old report) and continue.
|Let's try! New Report|
Now relax, take your coffee, be sure you've 20 minutes of free time (while nobody is stressing you), your GSM is off, you're ready to listen ... go to the dim_STAT Main page and click on 'Report Tool'.
|Click on Report Tool|
As you could have expected, nothing yet here for the moment. Let's click on the "New Report" button.
All you need to do here is just to fill in the new report form:
and click on "Create"
- ID: unique digital number
- Title: the main title
- Owner: owner information
- Chart: any additional comments to be present on the cover page
- Use: choose a pre-configured Report template
Wow! It works! :)) With the 'big' buttons, you may now:
But if you'll hover your mouse over the pre-generated notes you'll see pop-ups explaining each action.
- Hide/Show Note comments
- Preview your report
- Generate the report
- go Home (back to the main Report page)
Let's edit 'General Information' (click on 'data' icon).
- click on the 'down' icon to create a new note 'after' the current one (same parent level)
- click on the 'right' icon to create new 'child' note 'under' the current one (parent level+1)
- click on the 'cut' icon to cut and then paste (may go to 'trash' if need to be deleted (end of screen))
- click on the 'data' to edit/view the Note
From here you may see the current Note preview and edit the Note comments or attributes. If you change only attributes, then click on the corresponding button to apply the changes. If you want to modify the Note comments, click on 'Edit Note'. BTW, you can also do that with any external editor.
|Edit Note, continue...|
Add what you want in the text fields (you may use any HTML tags, etc.)
|Edit Note, continue2...|
Note: if you choose Text-format option your text is auto-formatted.
Save the Note.
- an empty line is seen as a 'new paragraph'
- three spaces at the start of the line are replaced by a "blanked-tabulation"
- some kind of limited wiki-like syntax is supported (see below example of input text containing wiki-like tags and its output result)..
|Wiki-Like syntax: INPUT|
Here is a =!Big BOLD Header!= Here is just a text +!with INCREASED font size!+ *!Here!* or **Here** will be a bold text /!Here!/ is text in italic _!Here!_ or __Here__ will by underlined test __Simple TEXT List__ : - one - two - three __Simple HTML List__ : * one * two * three __Simple code or text formatted__ : [code] $ ls -l /usr/sfw/bin/* ... $ ps -ef ... $ pkill -9 oracle [/code] __Simple Table__ : | **System/Performance** | **TPS** | **Resp.Time(ms)** | | M5000 | 4.500 | 10.0 | | M8000 | 8.000 | 10.0 | | M9000 | 15.000 | 9.2 |
|Wiki-Like syntax: OUTPUT|
Here is a
Big BOLD HeaderHere is just a text with INCREASED font size
Here or Here will be a bold text Here is text in italic Here or Here will by underlined test Simple TEXT List :
- three Simple HTML List :
Simple code or text formatted :
Simple Table :$ ls -l /usr/sfw/bin/* ... $ ps -ef ... $ pkill -9 oracle
System/Performance TPS Resp.Time(ms) M5000 4.500 10.0 M8000 8.000 10.0 M9000 15.000 9.2
|Edit Note, continue3...|
You may re-edit again or open the door :))
|Edit Report, continue...|
Let's fill other notes in the same way...
|Edit Report, continue2...|
So far so good :)) Now, I want to add a SysINFO Note for both hosts 'tahiti' and 'java'. SysINFO data is collected on-line, at the moment you're asking for and it's an easy way to keep your document updated at the moment you're writing. BTW, look into the STAT-service package to know how it is configured on the host side. You may extend it with any other information you need. So, a new SysINFO note under 'Software Configuration'... (right icon)
|New Note -- SysINFO|
As the tool has no idea what kind of Note you want to add, it will ask you to choose one before it can continue. Also, I did not want to add too much complexity to the interface. So, just click on 'SysINFO' here...
|New Note -- SysINFO Form|
Here you will need to fill in the SysINFO form: the usual data (title/comments/attributes) and SysINFO specific ones:
As SysINFO output is usually quite wide, it's preferred to keep it as an 'External Link'. Save the Note. If you gave the right hostname, port and the STAT-service is up and running on this host, You'll receive your data in a few seconds, in our example from the 'tahiti' domain :))
- the host name
- the host's STAT-service port
|New Note -- SysINFO Result|
Because I asked for 'Linked' contents, there is only a link to SysINFO data from 'tahiti'. Let's click on it to see if it works correctly.
|New Note -- SysINFO Link Contents|
|Edit Report, continue3...|
As you see, I've my new SysINFO note under 'Software Configuration'. Let's get SysINFO from 'java' host now and place it 'under' current tahiti SysINFO...
|Edit Report, continue4...|
Now, under 'Hardware Configuration' I want to add an image representing my platform diagram (a very simple image, just for those who are not able to imagine two hosts with one storage device :)), but "a picture says more than a thousand words". :)) So: 'Hardware Configuration' -> Image -> ...
|Add New Note -- Image|
Once again, similar info to fill, except you may give a name of your image file to upload [Browse]. Let's fill it and save as 'In-Line' attachment.
|Add New Note -- Image Inline|
Oops, it's TOO BIG! And that's not so you can see it better!! I prefer to keep all big images 'linked'. So, [Edit Note] -> 'As External Link' (no more need to give image file again) -> [Save Note]
|Add New Note -- Image Linked|
That's better!! Now, let's add a 'dim_STAT Collect' note! Leave this page [Door], go to the end of Report and click on the [Right] icon on 'Report' note, then choose 'dim_STAT Collect'.
|Add New Note -- dim_STAT Collect, Step1|
The dim_STAT Collect Note needs several steps to be created:
We are on the Step-1 here, and if you don't have any data collected, you may get them from the 'Default' demo collect:
- 1. setup dim_STAT server database parameters, [Next]
- 2. select STAT collect you want to use, [Next]
- 3. select STATs you want to see and time interval, [Next]
- 4. [Finish] or select STATs you want to see and time interval, [Next] (goto 4)
- 5. graph titles, choose graph parameters, [Save]
[Next]... NOTE: the interface becomes more optimized and more extended with each new release, so screen shots are probably not everywhere up to date.
- Server : localhost
- Port : Default
- Database : Default
|Add New Note -- dim_STAT Collect, Step2|
Choose STAT collect here and Search mode. We have already the log messages from the 'java' host, each message was added before any of the tests started, so it's quite easy to find them, corresponding to the time interval for each test. Otherwise we can always do a 'Date and Time' search, but you'll quickly understand thatt that is much more painful compared to LOG messages.
NOTE: with version 8.0, more options added to simplify reporting:
- replay the same time slices for N days (in Date and Time)
- auto include time/date into generated graph titles
- replace on-the-fly some part (max 5) of the LOG messages
|Add New Note -- dim_STAT Collect, Step3|
Now we need to choose the type of graphs we want to see and the time interval of them. NOTE:
- All the Per-Host STATs are Bookmarks. The more you created Bookmarks during analyzing, the more data you can generate in for report.
- When I selected the two hosts, the tool gave me also Multi-Host STATs, depending on stat commands to be present or not. Each STAT (like in Multi-Host Analyze) will put all requested hosts onto single graph.
|Add New Note -- dim_STAT Collect, Step3 continue|
Here we're choosing:
Time interval: as we know each test run for ~15min, we can choose a time interval of '15 min. After each LOG message. [Next]...
- per host : CPU busy%, Run queue, Mutex spin, System calls/s
- multi-host : CPU busy%, Network load bytes/s and packets/s
|Add New Note -- dim_STAT Collect, Step4|
So, this looks OK. I've got my STATs selected with a pre-populated graph title (from the LOG message). BTW, you may see that all your previously selected STATs are pre-selected here (the selection is saved via cookies and specific to each database name). [Finish] ...
|Add New Note -- dim_STAT Collect, Step5|
Here you have to specify the graphs parameters:
[Save]... Now you're free to start doing something else, because your machine is working for you and all you have to do is sit back and relax. Once you get use to the report tool, you'll ask it to generate A LOT OF graphs at the same time and you've time on your hands to do something else.
- Main title
- per graph title
- order generation
- graph mode, style, size, etc.
- Auto-AVG: good to select if you have too large time intervals and your graph become too dense
- Show LOG/TASK (as during analyze)
- Show processing - get generation output on the browser. Not all browsers work correctly with this feature, some are waiting for an EOF before they show something. If you don't choose this option, processing output is always printed into a /tmp/.report.log file on the Report Tool server side.
|Add New Note -- dim_STAT Collect Result|
Here is the final result after all the graphs are generated! Click on a link to see the graph results. NOTE: If you remember, I've selected generating order by Collect , and what I see now is a list of collects first, and each collect link will show me all selected STAT graphs for the same given STAT collect. Now, if I select the by STATs order generation - I'll see here a STAT list, and each link will show me the same STAT metric for different collects on single page...
|Add New Note -- dim_STAT Collect Contents, ordered by:Collect|
|Add New Note -- dim_STAT Collect Result per STATs|
As you see here, the single STAT link contains all given collects, so if you want to compare the network usage in different cases, just click on either the bytes/sec or the packets/sec link.
|Add New Note -- dim_STAT Collect Contents, ordered by:STATS|
|Edit Report, next...|
|Edit Report -- Cut|
Last thing now: I don't want to see my 'per STAT' first in Report section, just let's move it at the end... Click on [Cut] icon, then [Paste] where you want ([Trash] icon does delete operation!)
|Edit Report -- Paste!|
|Edit Report -- Pasted...|
|Edit Report -- Preview|
|Edit Report -- Preview Output|
|Edit Report -- Preview Output2|
|Generated Report documents|
|Report Tool Home|
THAT'S ALL, folks! :)) The export file of this demonstration report may be found within dim_STAT distribution as 'ExpReport_15.tar.Z'. You may import and play with it as long as you want! :)) Also, for good first exercise you may try to generate your first graphs from 'Demo collect' giving by default in your dim_STAT database!...