Version 14 (modified by shetrone, 5 years ago) (diff) |
---|
Creating a system monitor
A monitor can be run from inside the TCS gui or from the command line in htcs, but to avoid cross purposes with TO, run it preferably from /home/mcs/astronomer.
In the following example on is run for the TCS (note the -T) but it excludes the debug commands (note the d):
monitor --verbose -T --log-print --key-filter='log_[^d].*'
In the following example you can get the WFS seeing estimate from pas:
monitor --verbose -p --key-filter='metrology_data' --source='WFS*' | grep "time\|seeing\|\.metrology"
Log files
Perhaps more useful than the db are the log files. These can be accessed during the night and include the current night a usually about a month of order logs. There are a number of different logs on different machines:
- TCS logs are located on htcs in /var/log/tcs_logs/subsystem_name/
- TCS_GUI logs are located on jove or mcs in /var/tcs_logs/
- VIRUS and LRS logs are located on vdas and lrs2 in /var/log/tcs_logs/instrument_name/
- Htopx logs are located in /data1/nossy/www/html/noss/htopx3/log/htopx.log
The TCS logs are split into the different sub-systems: legacy, pfip, pas, tcs, and tracker. There are several ways one might use these. Note that to run this command you must be logged into htcs (e.g. ssh -X sco@htcs).
- For example to look at the pfip probe motions to check for the "move complete" before making any further moves. This would be done as tail -F tcs_server.log | grep pfip
- One can also look for a "go_next" or "load_trajectory" inside of the tcs_server logs.
- Be sure to be logged into htcs.
One could look at the TCS GUI logs to see what buttons were pushed to get an idea if a problem was a human error or a software problem. The htopx_logs are useful for looking to see when/if targets were incremented.
Nightly Databases
Nightly databases can be found in /opt/het/hetdex/logs/Mon/YYYY/MM/ where YYYY and MM are the year and month. Data files are written at around 18 UT and are off by one UT date. For example /opt/het/hetdex/logs/Mon/2017/05/20170513T180005.db is the database for the night of 20170514.
You can extract a text version of the db with the python code: db2log.py
Plotter
Here is an example of plotting guider1 metrology x vs. y: tcs_plotter --named-route "tcp://192.168.66.99:30000" --plotter-route "" --endpoint-system tcs --system tcs --source Guider1 --key measurement -x 'focal_plane.dx_asec' -y 'focal_plane.dy_asec'
Here is another example tcs_plotter --named-route "tcp://192.168.66.99:30000" --plotter-route "" --endpoint-system tcs --system tcs --source root --key ra_dec -x 'ra' -y 'dec'
Other useful monitors
Looking at the DMI and WFS data:
monitor --verbose -T --source-filter 'WFS*|DMI' --key-filter measurement
working with the htopx and night report databases
It is useful to know what fields can be queried with a mysql query in these databases. To get a list try: esql htopx -e "describe fullqueue"