<< | [up] | >> |
dim_STAT User's Guide. by Dimitri |
BatchLOAD |
The idea of BatchLOAD came (as all other things) from day to day needs: sometime you are facing customers/users who want to know what happens on their machines, but they don't agree to install any additional software on them... (very constructive approach :)). So, all you can do is to ask them to run some stat commands on their systems and send you the output files. And every day loading their files via Web interface you'll think harder and harder if there is any way to do it automatically... Are you ready for BatchLOAD? :))
Once decided to add a new component into dim_STAT, I've kept in mind also some other tools already existing/coming around and collecting output from stat commands on the machine. All such of tools keeping data in their own format, so I've tried to design the input format for BatchLOAD to be easily adaptable. Of course, I did not think to create something universal :)), but hope it should be not too hard to write a script converting from already existing format to BatchLOAD...
Some words about BatchLOAD internals: there is no dependency or something else on the name of loaded files. All needed information is given by command options and inside of the loaded file. Loaded file must have special TAGs, at least two: to give STAT name and confirm the END.USAGE:
Usage: /apps/ADMIN/BatchLOAD -cmd NEW/ADD options Options [NEW]: -- force new collect creation -base DBname -- database name -ID id -- Collect ID, if 0 use max+1 id automatically -title Title -- Collect Title -host Hostname -- Collect Host Name -isec sec -- Collect STATs Interval (sec) -start datetime -- Collect Start DateTime in format YYYYMMDDHHMISS -skip1 yes/no -- Yes/No skip first STAT measurement (often wrong values) -file Filename -- Full path to file with STATs outputs -verbose on/off -- verbose output on/off Options [ADD]: -- add to existing collect whenever possible -base DBname -- database name -host Hostname -- Collect Host Name (optional) -ID id -- Collect ID, if 0 : -- if host is given - use max id used by host -- otherwise, use max (last) id automatically -skip1 yes/no -- Yes/No skip first STAT measurement (often wrong values) -file Filename -- File with STATs outputs -verbose on/off -- verbose output on/off
Example:$ /apps/ADMIN/BatchLOAD -cmd NEW -ID 0 -base ANT -file `pwd`/vmstat.out -skip1 no -title "Test BatchLOAD" -host V880 -isec 20 -start 20031024100000 $ /apps/ADMIN/BatchLOAD -cmd ADD -ID 0 -base ANT -file `pwd`/iostat.out -skip1 no $ /apps/ADMIN/BatchLOAD -cmd ADD -ID 0 -base ANT -file `pwd`/mpstat.out -skip1 no -verbose on
in this example first line will create new STAT Collect using automatically new ID (max+1) with title "Test BatchLOAD" and load first file: "vmstat.out" second & third lines just load into newly created Collect next data: "iostat.out" and "mpstat.out"; once it's finished - we may connect dim_STAT web server and start analyze.Note: several "-file" options may be used on the same time, for ex:
$ /apps/ADMIN/BatchLOAD -cmd NEW -ID 0 -base ANT -skip1 no -title "Test BatchLOAD" -host V880 -isec 20 -start 20031024100000 -file `pwd`/vmstat.out -file `pwd`/mpstat.out -file `pwd`/iostat.out
File Format of STAT output
File format is designed in way to give as more possible flexibility on data grouping + processing.
Main TAGs are STAT and END:
==> STAT StatName -- after this point all following data corresponds to given STAT command (StatName) Actually supported STAT names: VMSTAT MPSTAT IOSTAT (iostat -x) IOSTAT-xn (iostat -xn) VXSTAT (vxstat -v) psSTAT And all other Add-On STAT you are able to create! :)) like some already shipped: netLOAD T3stat oraEXEC oraIO ... ==> END -- end of STAT data
At any time the following TAGs may also be inserted:==> DTSET yyyy-mm-dd hh:mi:ss -- set date+time point for next STAT data ==> LOGMSG message -- add log message into database corresponding to the currently loading dataOutside of "STAT" - "END" blocks any other lines are ignored.
Note: TAGs are exactly as it shown: "==> STAT", "==> END", "==> DTSET", "==> LOGMSG". Don't miss any characters, please :))
BatchLOAD Example |
Small example, let's say you have 3 vmstat and 3 iostat files corresponding to, say: "morning", "day" and "night" activity during some special tasks. So, you can make 6 load files each one containing its own "STAT", "DTSET", "END" TAGs, OR! put ALL in one:
... ==> DTSET 2004-01-19 10:30:00 -- set "morning" point ==> LOGMSG Morning workload ==> STAT VMSTAT -- load vmstat ... output of vmstat.out1 ==> LOGMSG Strange CPU activity -- marking time period to analyze (example) ... continue ... ==> END -- end of first vmstat ==> STAT IOSTAT-xn ... output of iostat.out1 ==> END ==> DTSET 2004-01-19 14:30:00 -- set "day" point ==> LOGMSG Day workload ==> STAT VMSTAT ... output of vmstat.out2 ==> END ==> STAT IOSTAT-xn ... output of iostat.out2 ==> END ==> DTSET 2004-01-19 23:30:00 -- set "night" point ==> LOGMSG Night workload ==> STAT VMSTAT ... output of vmstat.out3 ==> END ==> STAT IOSTAT-xn ... output of iostat.out3 ==> ENDSo, ALL information is placed in one single file ready to load:
$ /apps/ADMIN/BatchLOAD -cmd NEW -ID 0 -base ANT -skip1 no -title "Customer Workload" -host V880 -isec 20 -start 20040119100000 -file `pwd`/all_stat.outIn the same way you may group by single file all data of the same STAT command, or all outputs corresponding to the same collecting time period...
NOTE: don't forget to create your database _before_ starting any load! (in this example database name is 'ANT').
Special NOTE |
Please, take care - there is no option to give a name of loaded stat command! That's why "STAT" and "END" tags are mandatory!. Even you want to load just one vmstat file, tool have no idea about your file contents till it'll meet a "STAT" tag inside!
GUDs integration |
If you already worked with Sun support or you're Sun employe - you may know or already used GUDs (shell script collecting various system information + stats and saving them into special case archive). GUDs was created by Sun France engineer, and another French engineer made an integration script to load GUDs data into dim_STAT via BatchLOAD - 'guds2dim.sh'. This script is shipped now with dim_STAT and may be found in /apps/ADMIN directory. To obtain GUDs script - please, contact directly Sun support.
<< | [up] | >> |