wiki:HetProcedures/RA/overview

Context Navigation

Version 220 (modified by shetrone, 5 years ago) (diff)
--

Overview of the basic night time procedures

Afternoon checkout

suggested order of OPS procedure.

run syscmd -l 'get_cryo_pressure( update_state="true" )' to check LRS2 VG pressure. If got 999999 that means that VG need to be enabled and one needs to wait 10 minutes before read out pressure
turn off ION PUMP, VG and VIRUS V309_IONPump using the command ipvg off -V confirm IP is off on APC GUI. The VG is only disabled not turned off with this command. If you need to do this with low level commands you can turn off the IP on the APC directly but Important!!!! Leave LRS2 VacuumGauge? on on APC GUI. To disable or enable VG use commands
```
syscmd -l 'disable_pressure_gauges()'  # should be run during OPS after pressure reading
syscmd -l 'enable_pressure_gauges()'   # should be run RA in the morning
```
check instruments temperatures
check IFU's percentage power (Left and Right Pwr %). Adjust any temperatures as needed.
take lrs2 and virus test cals
restart HPF HET client and make sure the observation numbering has not been reset.
do test move of the guide probes (home4 or home5)
do the acam stability test
add the LRS2 and virus uptime getUptime --lrs2 --virus
take any long cals that can be done

When servers are DOWN

On TO Launcher on TO console > Operations > TCS Servers > All Servers > Restart
A window will pop up showing each server being started/restarted

Detailed list

Check the temperatures and voltages of the instruments

LRS2 temperatures:
- For LRS2, use the RA launcher application (Operations drop-down menu) and select Display LRS2 Temperatures
  - CCD temps (L,R for Red, & Blue) should be around -115.0
  - Cryo temps should be around -169.0
  - the temp check window remains open for 30 seconds
  - check to make sure that the % power are all between 5 and 90%.
- Sometimes the RAlauncher approach does not work, so you might try
```
   $ getSpectTemperatures --virus
   $ getSpectTemperatures --lrs2
```
- Frequently temps will not report for LRS2 and it is useful to restart the server:
```
    [astronomer@mcs ~]$ ssh -X sco@lrs2
    [sco@lrs2 ~]$ sudo service ldas restart
```
  Wait for several minutes until the green LRS2 light appears in the TCS gui.
VIRUS temperatures:
- For VIRUS, use the RA launcher application (Operations drop-down menu) and select Display VIRUS Temperatures
  - NOTE: If you just restarted VIRUS within the last hour the temperatures and power might not be stable
  - check to make sure all power percentages (last two columns labeled Left Pwr % and Right Pwr %) are between 5 and 90%.
- If below 5% AND the Set Point is < -90 then follow the procedure below. If above 90% AND the Set Point is > -115 then follow the procedure below.
  - Bring up the TCS gui and go to the Virus Environment tab
  - Within the Virus Environment tab go to the Warm Up/Cool? Down tab
  - Enter the Virus spectrograph ID number into the Spec entry form. Example for V320 enter 320.
  - Hit the Update Temps button and then look at the Set Point values on the listed under Left CCD and Right CCD
  - Enter BOTH your new and any old target temperatures under Target L and Target R. NOTE: Both of these need to be filled out as both are set when you hit Execute.
    - If the power is below 5% then raise the temperature by 5 degrees. If the power is above 90% then lower the temperature by 5 degrees. For example if the Left and Right set points are -110 and -105 and the Left and Right Pwr % are 21.12 and 4.99 then you will be entering the Target set points as -110 and -100 (the left did not change while the right is now 5 degrees warmer).
  - Hit Execute when BOTH fields are entered.
  - Look at the Left CCD Set Point and Right CCD Set Point on the Gui to see that they changed.
  - NOTE: It may take several tens of minutes for the actual Temp to get to the Set Point and during that period the Pwr % may go to 100% or 0% depending upon which way the set point moved.

Check LRS2 vacuum pressure

run syscmd -l 'get_cryo_pressure( update_state="true" )' to check LRS2 VG pressure. If got 999999 that means that VG need to be enabled and one needs to wait 10 minutes before reading out the pressure

If the servers are not up you will need to ask the TO or day staff to do this through the pull down menu of the TOlauncher on jove.

Turn off the LRS2 ion pump and Vacuum Gauge

In an mcs window do a ipvg off -V

Take test exposures:

Take an initial bias frame with VIRUS and LRS2 to make sure everything is ok
```
cal lrs2 -b 1 -pb -o 9990 -O 9990 -B
```

The following check of LRS2 should take about 11 minutes to complete:

 cal lrs2 -l fear -L 1 -c qth -C 1 -d 1 -de 300  -b 0 -o 9991 -B

The following check of VIRUS should take about 6 minutes to complete:
```
 cal virus -l hg cd-a -L 1 -c ldls_long -C 1 -b 0 -o 9991 -B 
```

Recall that you can use the same log name for multiple runs (the results will be append each time you run cal.py)
Examples of images: Sample cal images for VIRUS
Run vhc on the VIRUS images: see How to run vhc.
Make a summary of the VHC output for the report by using viruserr

Make sure the /mnt/camra_ramdisk disks on lrs2 and virus are empty (ssh to 'lrs2' and 'vdas'). If they are not that may indicate a problem with data transfer.

Check that the HPF numbering has not been reset (if before 0 UT):

in the HPF scripts window run tims detect pyhxrg:GetObsnum

Restart HPF HET client:

find the 'HET Client' window of the TIMS screen (currently screen 7).
shutdown the het communication client: tims het shutdown from the HPF Scripts window or a free xterm on hpf.
in the 'HET Client' window of the TIMS screen restart the communication client with the command timshet

Test guide probe motions:

run shoprobes for current positions

Then run:

[astronomer@mcs ~]$ home6  (or home5, home4, etc...)

Watch probe display ("Probe Assembly" window) on tcs gui to insure that probes move

Test the ACAM position:

Make sure the RA and TO are not doing anything with the PFIP or PAS
run command acqm.py <tracker temp> (e.g., acqm.py 25.0 getting tracker temp from HET weather GUI)
or try the following manual steps
- execute the script /home/jove/guider/chevo/shell/acamtest/acqmirror (be sure to not let this span the UT date roll over)
- do a ls -t -1 /hetdata/data/`cwd`/acm/20* | head -10 > /home/mcs/astronomer/iraf/ilist
- start iraf
- run xregister on the ilist with /hetdata/data/20180327/acm/20180327T024656.8_acm_sci.fits as the reference with the region set to [160:360,333:600]
- calculate average and standard deviations for both X and Y

report in the RA OPS report along with the Tracker temperature:

For acam mirror test (Tracker Temp = 16.9 C):
                     X(pixels)       Y(pixels)
Mean shift            3.173             3.193
StanDev               0.047             0.024

Get the LRS2 and virus uptimes via:

getUptime --lrs2 --virus

If you can make the dome dark:

Start a set of HPF Part A+C cals (if time. As of 13 Sep 2018 PartA takes 44m and PartC takes 43m)

e.g., /home/hpf/Scripts/InstConfig/CalScripts/hpf_evening_cals_PartAC.sh on HPF Scripts window

Record the start and estimated stop time for the Part A+C cals in the report.

Problems you might encounter

get_hardware_status does not return values: restart camra server
camra server does not restart: See problem solutions for camra

If there will be time in dark dome conditions (or when hpf PartC is running) perform LRS2+VIRUS cals:

If this might span 0 UT OR if you want to run both LRS2 and VIRUS lamp cals use sir script, e.g. /home/mcs/astronomer/sir/evcals.sh <-l> <-V> [-b|-pb] [-v] [--dry-run]
- Use the "-h" flag to get more details
- /home/mcs/astronomer/sir/evcals.sh -l -v takes LRS2 lamp cals in 44 minutes plus any wait time around 0 UT.
- /home/mcs/astronomer/sir/evcals.sh -V -v take VIRUS lamp cals in 38 minutes plus any wait time around 0 UT.
- /home/mcs/astronomer/sir/evcals.sh -pb -v takes combined bias frames in 14 minutes plus any wait time around 0 UT.
- /home/mcs/astronomer/sir/evcals.sh -l -V -pb -v takes all LRS2 and VIRUS cals in 1 hour 35 minutes plus any wait time around 0 UT.
If the cal of interest will not span 0 UT OR is not both LRS2 and VIRUS lamp cals then use normal cal script, e.g. cal lrs2 -l hg cd-a fear -c qth ldls_long -B , cal -pb -B or cal virus -l hg cd-a -c ldls_long -b 0 -d 1 -de 360 .
To run the parallel biases at a certain time (i.e., if you know when PartA finishes and PartC starts), you can use:

at 23:40

/home/mcs/astronomer/sir/evcals.sh -pb -v

[ctrl-D]

OR

at 23:20 (note how this cal of interest would complete before the 0 UT roll-over)

cal lrs2 -l hg cd-a fear -c qth ldls_long -B

[ctrl-D]

Start of the Night

Start an RA log named YYYYMMDD_RAlog.txt (can do it in /data1/mcs/guider/YYYYMMDD/ )

Check the temperatures of the instruments (via RA Launcher-->Operations)

Check on status of the LRS2 ion pump and VG (in TCS)

Check e-mails to see if PV or other engineering sky flats are required (file unneeded; flag necessary)

Check latest commissioning plan located here or Science plan located here

Use chksys to see if any defunct processes such as the APC or Gui exist. Kill those processes.

Start Monitors for instruments and TCS see LRS2 and VIRUS procedures.

Walk around dome area to make sure lights and mothenators are off.

NEW PROCEDURE FOR TWILIGHTS
- Take twilight spectra (start GC2 with "VIEW IMAGE" = raw for B filter and exp 0.1s
- IF no clouds start lsky.sh when GC2 flux is 42k
- ELSE if there are clouds then start lsky.sh when GC2 flux is 35k
- Start VIRUS script vsky.sh when GC2 flux is 6.6k

IF required (NOT usually), can take HPF twilight spectra (last taken, Oct 2018).
- open the PFIP shutter, make sure ACAM and FCU head are retracted
- AT sunset, run hpframp twilight 28 1
- this should produce nice twilight flats for HPF, but may eliminate your chance of getting LRS2 sky flats that night

Take twilight spectra (start GC2 and check flux level - ~22k LRS2 and 4.7k virus for B filter and exp 0.1s); NOTE: 30k for LRS2 and 5.5k for VIRUS seemed to work better in the summer. If you end up using GC1 on filter B then you should use ~36k for LRS2 and ~6.9k for VIRUS.
- First any imaging cameras
- LRS2 spectra
- VIRUS spectra

Evening twilights are lsky.sh and vsky.sh
Morning cals. ~3k virus vsky-morning.sh and ~15k lrs2 lrs2-morning.sh

For LRS2:
vlexp -i lrs2 -texp 15 -t twi -pobj skyflat
or
iexp lrs2 -skyflat

For VIRUS:
vlexp -i virus -texp 15 -t twi -pobj skyflat
OR
specialtwilight

While stacking take the LRS2 and VIRUS biases.

Start DIMM if weather permits. For directions see DIMM wiki page

End of the Night

Data should transfer itself to the TACC so no action from the RA should be required.

End of the night e-mails should be done automatically so no action from the RA should be required.

Take end of night calibrations:

Take any cals needed for science targets observed.
At the moment the only required end of night cals are for VIRUS (just make sure that the LRS2 VG and IP are still off).

Park the instruments

Turn on LRS2 ion pump and VG see LRS2 park procedure (an easy way to do this is to run SIR's ipvg script: ipvg on stat)
- take end-of-night VIRUS bias before the IP/VG.
Turn on the VIRUS ion pump see VIRUS park procedure
record the final LRS2 temperatures, LRS2 VG reading, and VIRUS temperatures into the RA log.
Make sure the LRS2 and VIRUS servers, pivot and monitors are running
Use chksys to see if any defunct processes such as the APC or Gui exist. Kill those processes.

Move the RA log into the Guider nightly working directory /data1/mcs/guider/YYYYMMDD/

Shut down the DIMM software and close the DIMM. See DIMM wiki page

Attachments (3)

template_email.txt (3.5 KB) - added by stevenj 4 years ago.
YYYYMMDD_RAlog.txt (3.3 KB) - added by stevenj 4 years ago.
ops_template.txt (3.4 KB) - added by stevenj 4 years ago.

Download all attachments as: .zip

Download in other formats:

Plain Text