• TCS comms with VIRUS/LRS2
• TCS comms with HPF

If we have the situation where TCS information (STRUCTAZ*, etc) is not being communicated to LRS2/VIRUS (i.e., those values are -9999s in the FITS file headers), try the following.

• restart the VIRUS (or LRS2) server only - check if it is fixed
• restart both the instrument server and the TCS server

As of Nov 2021, Jason says that syscmd -V 'reset_tcs()' (as described below) cannot possibly help. Restarting the VIRUS or LRS2 server (not muxes or controllers) should be enough to restore TCS communications.

If the instruments are still not receiving TCS information, try running:

  syscmd -V 'reset_tcs()'

and

  syscmd -l 'reset_tcs()'

in the LRS2 and VIRUS servers. That may fix the problem.

This is the help statement (from JR):

camra::reset_tcs (10228)

Flags the socket connection to tcs for resetting. The reset occurs on the next subsequent failed ACK from tcs.

HPF TCS communications problems

HPF has a TCS telemetry relay running on its side to listen to the TCS events and encode those in HPF data file headers. Sometimes this connection is broken and causes serious problems with missing metadata. The solution we have been using (as recommended by Chad Bender in Feb 2019) is to restart HPF's TCS relay and the HPF's TCS client, as described below:

1. First, if necessary, restart the HET TCS server (if you suspect the problem originated there)

2. Second, restart the HPF TCS relay:
   • Navigate to the TIMS Screen session:
     • Option 1: run ssh hpf@hpf and then screen -x TIMS
     • Option 2: use the top right window of the 'rasession' view
   • Change to the "HPF HET Relay" window (currently #3, but not guaranteed to always be so), by pressing ctrl+a n (next) or ctrl+a p (previous) until you find it. (Press Ctrl+a (a lower case "a" is fine), then release, then press "n")
   • CTRL-C the script running there (this is a script, not a TIMS client). You will need to CTRL-C twice (It took 3x for me once -SJ) to actually kill it.
   • Restart the script (up arrow or ./bashwrapperfor_HETTCSRebroadcast.sh)
   • Go back to the normal window (ctrl-a-n / ctrl-a-p), which should say "HPF HET Client" (currently screen 4).

3. Restart the TIMS HET Client. Two options:
   • During OPS, use the OPS web page to click "TCS Client Restart"
   • OR, on in the "Scripts" screen session, run: tims het shutdown, and then in the HPF HET Client screen session, run: timshet

It may also help to review our Screens Summary Page