= Overview of the basic night time procedures = == Afternoon checkout == #OPSENG === suggested order of OPS procedure. === * run {{{ syscmd -l 'get_cryo_pressure( update_state="true" )' }}} to check LRS2 VG pressure. If got 999999 that means that VG need to be enabled and one needs to wait 10 minutes before read out pressure * turn off ION PUMP, VG and VIRUS V309_IONPump using the command {{{ ipvg off -V }}} confirm IP is off on APC GUI. The VG is only disabled not turned off with this command. If you need to do this with low level commands you can turn off the IP on the APC directly but Important!!!! Leave LRS2 VacuumGauge on on APC GUI. To disable or enable VG use commands {{{ syscmd -l 'disable_pressure_gauges()' # should be run during OPS after pressure reading syscmd -l 'enable_pressure_gauges()' # should be run RA in the morning }}} * check instruments temperatures * check IFU's percentage power (Left and Right Pwr %). Adjust any temperatures as needed. * take lrs2 and virus test cals * restart HPF HET client and make sure the observation numbering has not been reset. * do test move of the guide probes (home4 or home5) * do the acam stability test * add the LRS2 and virus uptime {{{getUptime --lrs2 --virus}}} * take any long cals that can be done === When servers are DOWN === * On TO Launcher on TO console > Operations > TCS Servers > All Servers > Restart * A window will pop up showing each server being started/restarted === Detailed list === Check the temperatures and voltages of the instruments * LRS2 temperatures: * For LRS2, use the RA launcher application (Operations drop-down menu) and select ''Display LRS2 Temperatures'' * CCD temps (L,R for Red, & Blue) should be around -115.0 * Cryo temps should be around -169.0 * the temp check window remains open for 30 seconds * check to make sure that the % power are all between 5 and 90%. * Sometimes the RAlauncher approach does not work, so you might try {{{ $ getSpectTemperatures --virus $ getSpectTemperatures --lrs2 }}} * Frequently temps will not report for LRS2 and it is useful to restart the server: {{{ [astronomer@mcs ~]$ ssh -X sco@lrs2 [sco@lrs2 ~]$ sudo service ldas restart }}} Wait for several minutes until the green LRS2 light appears in the TCS gui. [[br]] * VIRUS temperatures: * For VIRUS, use the RA launcher application (Operations drop-down menu) and select ''Display VIRUS Temperatures'' * '''NOTE: If you just restarted VIRUS within the last hour the temperatures and power might not be stable''' * check to make sure all power percentages (last two columns labeled Left Pwr % and Right Pwr %) are between 5 and 90%. * If below 5% AND the Set Point is < -90 then follow the procedure below. If above 90% AND the Set Point is > -115 then follow the procedure below. * Bring up the TCS gui and go to the ''Virus Environment'' tab * Within the ''Virus Environment'' tab go to the ''Warm Up/Cool Down'' tab * Enter the Virus spectrograph ID number into the ''Spec'' entry form. Example for V320 enter 320. * Hit the ''Update Temps'' button and then look at the Set Point values on the listed under ''Left CCD'' and ''Right CCD'' * Enter '''BOTH ''' your new and any old target temperatures under Target L and Target R. NOTE: Both of these need to be filled out as both are set when you hit Execute. * If the power is below 5% then raise the temperature by 5 degrees. If the power is above 90% then lower the temperature by 5 degrees. For example if the Left and Right set points are -110 and -105 and the Left and Right Pwr % are 21.12 and 4.99 then you will be entering the Target set points as -110 and -100 (the left did not change while the right is now 5 degrees warmer). * Hit ''Execute'' when '''BOTH ''' fields are entered. * Look at the Left CCD Set Point and Right CCD Set Point on the Gui to see that they changed. * '''NOTE:''' It may take several tens of minutes for the actual Temp to get to the Set Point and during that period the Pwr % may go to 100% or 0% depending upon which way the set point moved. Check LRS2 vacuum pressure * run {{{ syscmd -l 'get_cryo_pressure( update_state="true" )' }}} to check LRS2 VG pressure. If got 999999 that means that VG need to be enabled and one needs to wait 10 minutes before reading out the pressure * If the servers are not up you will need to ask the TO or day staff to do this through the pull down menu of the TOlauncher on jove. Turn off the LRS2 ion pump and Vacuum Gauge * In an mcs window do a {{{ipvg off -V }}} Take test exposures: * Take an initial bias frame with VIRUS and LRS2 to make sure everything is ok {{{ cal lrs2 -b 1 -pb -o 9990 -O 9990 -B }}} * The following check of LRS2 should take about 11 minutes to complete: {{{ cal lrs2 -l fear -L 1 -c qth -C 1 -d 1 -de 300 -b 0 -o 9991 -B }}} * The following check of VIRUS should take about 6 minutes to complete: {{{ cal virus -l hg cd-a -L 1 -c ldls_long -C 1 -b 0 -o 9991 -B }}} * Recall that you can use the same log name for multiple runs (the results will be append each time you run cal.py) * Examples of images: [wiki:HetProcedures/RA/overview/image_examples Sample cal images for VIRUS] * Run vhc on the VIRUS images: see [wiki:HetProcedures/RA/virus#vhc How to run vhc]. * Make a summary of the VHC output for the report by using {{{ viruserr }}} Make sure the /mnt/camra_ramdisk disks on lrs2 and virus are empty (ssh to 'lrs2' and 'vdas'). If they are not that may indicate a problem with data transfer. {{{#!comment Restart the HPF communications server. For full notes on this see: [wiki:HetProcedures/RA/hpf#Startup HPF restart instructions] * Check that the OBS number has not been set back to 1 {{{tims detect pyhxrg:GetObsnum}}} from the HPF Scripts window or a free xterm on hpf. }}} Check that the HPF numbering has not been reset (if before 0 UT): * in the HPF scripts window run {{{ tims detect pyhxrg:GetObsnum }}} Restart HPF HET client: * find the 'HET Client' window of the TIMS screen (currently screen 7). * shutdown the het communication client: {{{tims het shutdown}}} from the HPF Scripts window or a free xterm on hpf. * in the 'HET Client' window of the TIMS screen restart the communication client with the command {{{timshet}}} Test guide probe motions: * run shoprobes for current positions * Then run: {{{ [astronomer@mcs ~]$ home6 (or home5, home4, etc...) }}} * Watch probe display ("Probe Assembly" window) on tcs gui to insure that probes move Test the ACAM position: * Make sure the RA and TO are not doing anything with the PFIP or PAS * run command {{{ acqm.py }}} (e.g., {{{acqm.py 25.0}}} getting tracker temp from HET weather GUI)[[br]] or try the following manual steps [[br]] * execute the script /home/jove/guider/chevo/shell/acamtest/acqmirror (be sure to not let this span the UT date roll over) * do a {{{ls -t -1 /hetdata/data/`cwd`/acm/20* | head -10 > /home/mcs/astronomer/iraf/ilist}}} * start iraf * run xregister on the ilist with /hetdata/data/20180327/acm/20180327T024656.8_acm_sci.fits as the reference with the region set to [160:360,333:600] * calculate average and standard deviations for both X and Y * report in the RA OPS report along with the Tracker temperature: {{{ For acam mirror test (Tracker Temp = 16.9 C): X(pixels) Y(pixels) Mean shift 3.173 3.193 StanDev 0.047 0.024 }}} Get the LRS2 and virus uptimes via: * {{{getUptime --lrs2 --virus}}} If you can make the dome dark: * Start a set of HPF Part A+C cals (if time. As of 13 Sep 2018 PartA takes 44m and PartC takes 43m) e.g., {{{/home/hpf/Scripts/InstConfig/CalScripts/hpf_evening_cals_PartAC.sh}}} on HPF Scripts window * Record the start and estimated stop time for the Part A+C cals in the report. Problems you might encounter * get_hardware_status does not return values: restart camra server * camra server does not restart: See [wiki:HetProcedures/RA/overview/problems problem solutions for camra] If there will be time in dark dome conditions (or when hpf PartC is running) perform LRS2+VIRUS cals: * If this might span 0 UT OR if you want to run both LRS2 and VIRUS lamp cals use sir script, e.g. {{{/home/mcs/astronomer/sir/evcals.sh <-l> <-V> [-b|-pb] [-v] [--dry-run] }}} * Use the "-h" flag to get more details * {{{ /home/mcs/astronomer/sir/evcals.sh -l -v }}} takes LRS2 lamp cals in 44 minutes plus any wait time around 0 UT. * {{{ /home/mcs/astronomer/sir/evcals.sh -V -v }}} take VIRUS lamp cals in 38 minutes plus any wait time around 0 UT. * {{{/home/mcs/astronomer/sir/evcals.sh -pb -v}}} takes combined bias frames in 14 minutes plus any wait time around 0 UT. * {{{ /home/mcs/astronomer/sir/evcals.sh -l -V -pb -v }}} takes all LRS2 and VIRUS cals in 1 hour 35 minutes plus any wait time around 0 UT. * If the cal of interest will not span 0 UT OR is not both LRS2 and VIRUS lamp cals then use normal cal script, e.g. {{{ cal lrs2 -l hg cd-a fear -c qth ldls_long -B }}}, {{{ cal -pb -B }}} or {{{ cal virus -l hg cd-a -c ldls_long -b 0 -d 1 -de 360 }}}. * To run the parallel biases at a certain time (i.e., if you know when PartA finishes and PartC starts), you can use: {{{at 23:40}}} {{{/home/mcs/astronomer/sir/evcals.sh -pb -v}}} {{{[ctrl-D]}}} OR {{{at 23:20}}} (note how this cal of interest would complete before the 0 UT roll-over) {{{ cal lrs2 -l hg cd-a fear -c qth ldls_long -B }}} {{{[ctrl-D]}}} == Start of the Night == #startup * Start an RA log named YYYYMMDD_RAlog.txt (can do it in /data1/mcs/guider/YYYYMMDD/ ) * Check the temperatures of the instruments (via RA Launcher-->Operations) * Check on status of the LRS2 ion pump and VG (in TCS) * Check e-mails to see if PV or other engineering sky flats are required (file unneeded; flag necessary) * Check latest commissioning plan located [wiki:NightOperations/Commissioning/Plans here] or Science plan located [wiki:NightOperations/SciencePlans here] * Use {{{chksys}}} to see if any defunct processes such as the APC or Gui exist. Kill those processes. * Start Monitors for instruments and TCS see [wiki:HetProcedures/RA/lrs2#startup LRS2] and [wiki:HetProcedures/RA/virus#startup VIRUS] procedures. * Walk around dome area to make sure lights and mothenators are off. * NEW PROCEDURE FOR TWILIGHTS * Take twilight spectra (start GC2 with "VIEW IMAGE" = raw for B filter and exp 0.1s * IF no clouds start {{{lsky.sh }}} when GC2 flux is 42k * ELSE if there are clouds then start {{{lsky.sh }}} when GC2 flux is 35k * Start VIRUS script {{{vsky.sh }}} when GC2 flux is 6.6k * IF required (NOT usually), can take HPF twilight spectra (last taken, Oct 2018). * open the PFIP shutter, make sure ACAM and FCU head are retracted * AT sunset, run {{{hpframp twilight 28 1}}} * this should produce nice twilight flats for HPF, but may eliminate your chance of getting LRS2 sky flats that night * Take twilight spectra (start GC2 and check flux level - ~22k LRS2 and 4.7k virus for B filter and exp 0.1s); NOTE: 30k for LRS2 and 5.5k for VIRUS seemed to work better in the summer. If you end up using GC1 on filter B then you should use ~36k for LRS2 and ~6.9k for VIRUS. * First any imaging cameras * LRS2 spectra * VIRUS spectra[[BR]] * Evening twilights are {{{lsky.sh}}} and {{{vsky.sh}}} * Morning cals. ~3k virus {{{vsky-morning.sh}}} and ~15k lrs2 {{{lrs2-morning.sh}}} {{{ For LRS2: vlexp -i lrs2 -texp 15 -t twi -pobj skyflat or iexp lrs2 -skyflat For VIRUS: vlexp -i virus -texp 15 -t twi -pobj skyflat OR specialtwilight }}} * While stacking take the LRS2 and VIRUS biases. * Start DIMM if weather permits. For directions see [wiki:HetProcedures/RA/DIMM/#startdimm DIMM wiki page] == End of the Night == #park Data should transfer itself to the TACC so no action from the RA should be required. End of the night e-mails should be done automatically so no action from the RA should be required. Take end of night calibrations: * Take any cals needed for science targets observed. * At the moment the only required end of night cals are for VIRUS (just make sure that the LRS2 VG and IP are still off). Park the instruments * Turn on LRS2 ion pump and VG see [wiki:HetProcedures/RA/lrs2#park LRS2 park procedure] (an easy way to do this is to run SIR's ipvg script: ipvg on stat) * take end-of-night VIRUS bias before the IP/VG. * Turn on the VIRUS ion pump see [wiki:HetProcedures/RA/virus#park VIRUS park procedure] * record the final LRS2 temperatures, LRS2 VG reading, and VIRUS temperatures into the RA log. * Make sure the LRS2 and VIRUS servers, pivot and monitors are running * Use chksys to see if any defunct processes such as the APC or Gui exist. Kill those processes. Move the RA log into the Guider nightly working directory /data1/mcs/guider/YYYYMMDD/ Shut down the DIMM software and close the DIMM. See [wiki:HetProcedures/RA/DIMM/#shutdowndimm DIMM wiki page]