HPF

You can find the HPF team's web site with their man at ​http://psuastro.github.io/HPF/Operators-Manual/ and another useful site is their ​exposure time calculator

HPF Observing

Suggested terminals setup

Easiest method to start up is to

• click "RA HPF session" icon on central desktop
• run "rasession"

This will bring four terminal screen session

Or

Note that if you open new xterms you should select mcs as the host machine (not local).

----------------------------------------------
- HPF terminal -       | - HET client -      |
ssh -X hpf@hpf         | ssh -X hpf@hpf      |
                       | screen -x TIMS      |
----------------------------------------------
- HPF monitor -        | - HPF Scipts -      |
ssh -X hpf@hpf         |ssh -X hpf@hpf       |
screen -x hpf          |screen -x sc         |
----------------------------------------------

In the rare event that these screens sessions are not running (or you accidentally "exit"ed out of the screen session), you can create them with the following command, e.g., for the Scripts screen:

screen -S scripts

and then connect to them as usual.

Possible script shortcut to setting up with the regular mode on the HR fiber for HPF

• Send the target from shuffle using either the IFU id 600 for regular setup or IFU id 603 for direct setups. Note if you need to use the side CB IFU use id 610.
• make sure the pfip shutter is open
• open a connection to htcs in an xterm or console window
• in the htcs window as the structure sets down run the script ./regularhpfsetup or ./directhpfsetup
• follow the steps as prompted
• in the htcs window when the regularhpfsetup concludes run either ./guider1setup or ./guider2setup
• tell the RA when done so they can start the exposure.

The step by step details

Setting up for an HPF target follows the following procedures:

• use target_setup and either IFU 600 (normal setup) or 603 (direct setup) Note if you need to use the side CB IFU use id 610.
  • Example target_setup 1183 E -to -ifu 600
• Change the acam filter to icd to /home/hpf/HPFics
• Change the guider guide probe to i
• Change the metrology guide probe to g
• Offset to the correct position on the ACAM based on the current best positions marker positions.
• Save an ACAM image.
• open the PFIP shutter
• retract ACAM and turn off the ACAM pipeline
• start guiding on the correct position on the HPFACAM based on the current best positions marker positions.

NOTE: If you have trouble getting the star to stay on the HPFACAM try increasing the integral term on the PID loop syscmd -T 'HPF_ACQ_tune(kd=0,ki=0.07,kp=1)' if you want to set it back to default syscmd -T 'HPF_ACQ_tune(kd=0,ki=0,kp=1)' to see the current values do syscmd -T 'HPF_ACQ_get_tuning()'

• Center the guide star in the desired guider probe (gc1 or gc2)
• Offset the desired guider probe by the correct amount based on the desired target science fiber: HE offsets or HR offsets
• Do handshake from HPFACAM to the guider
• Offset the fiducial by the correct amount based on the desired target science fiber: HE offsets or HR offsets
• start HPF exposure

HPF offsets to fibers

HPF HR mode

• syscmd -T 'Guider1_offset_probe (dx_ang=5,dy_ang=4)'
• syscmd -T 'Guider1_offset_fiducial (dx_asec=-9.6,dy_asec=-5.5,compensate="true")'

OR

• syscmd -T 'Guider2_offset_probe (dx_ang=5,dy_ang=4)'
• syscmd -T 'Guider2_offset_fiducial (dx_asec=-9.6,dy_asec=-5.5,compensate="true")'

Blind offsets from ACAM without using HPFACQ in the HR mode

Run shuffle with the fiber ID of 603 instead of 600. Use the HPF_HRfiber position to setup on in the ACAM. If there is uncertainty about the position then measure it at the end of a regular HPF setup to confirm the position. Do the handshake from the ACAM to the best guider choice. pull out the ACAM and start the science exposure.

Instrument Control

Here are the old instrument control commands. Many of which are valid but we have new wrappers that are easier to use and more elegant.

To get to the HPF machine: ssh -X hpf@hpf

We communicate with HPF via screens to see more about screens see our Screens Summary Page. To bring up the HPF monitor do a:

(HPFpython) [hpf@hpfserver ~]$ screen -x hpf

That screen is simply for monitoring the state of the instrument. To end the screen session close the xterm/tab or do a <cntrl>a d. Do not do a <cntrl> d

You will need another ssh window to make commands to the instrument ssh -X hpf@hpf . In this window you will control the lamps, the comparison lamp shutters, exposure times etc... grab the scripts screen session screen -x scripts session so that any failures can be diagnosed by the HPF team.

Data is stored in /HPFData/LinuxGigE/YYYYMMDD/hpf from other machines, e.g. mcs, or /home/hpf/HPFData/LinuxGigE/YYYYMMDD/hpf on the hpf machine. Scripts for changing the instrument settings are located in /home/hpf/Scripts/InstConfig

Controlling the Laser Frequency Comb (lfc)

Some targets, usually the brighter ones and the precision radial velocity targets, will want to use the lfc to get the best possible radial velocities. The lfc remains on at all times but to include it in the observation you need to open its internal shutter. To turn it off and on use the following commands:

lfc_open

lfc_close

In some cases, the laser comb may be down and we must use the following:

etalon_open

etalon_close

Note that each of these command sets are executed in the Scripts window. The open command will usually take a few seconds to complte and you will see a variety of messages coming to the window.

After PartA cals are done, I turn on the etalon with: ./hpf_TurnOn.sh Etalon

That takes 10 minutes to warm up.

Then, throughout the nights, I try to take the etalon cals at least once an hour: ./hpf_takeEtalonHRCalExposure.sh 3

Doing 3 of those takes about 9 minutes. You can run: ./hpf_takeEtalonHRCalExposure.sh 1 to run a single ramp (takes 3 minutes).

For targets which request LFC, I think our approach right now is to: --keep cal_closed --run etalon cals before observing the target (while slewing/setting up) --run etalon cals after the target

At the end of the night turn off the Etalon ./hpf_TurnOff.sh Etalon

Engineering observing

The command to execute any basic observation is hpframp targetname #reads_per_ramp #ramps [obs#]. Each read is 10.5 seconds so a 30 read observation would be 315 seconds. There is a 2 read reset at the beginning of the exposure that adds to the overhead. You can include a optional observing number, but if the software has not been restarted and you do not include this observing number it will use the next observing number. If you restart the software the count starts again at 1 and the next target will be included in directory 0001 even if there is data in that directory, however, due to the time stamp no data will be overwritten.

Example:

hpframp AD_Leo 30 2

You can set the type to Eng so that they are not science exposures and sent to the TACC by doing a tims detect pyhxrg:SetObstype:Eng

Science observing

run all of our science frames and cals in the screen -x scripts session so that any failures can be diagnosed by the HPF team.

The command to execute any basic observation is hpfseconds targetname exposuretime #exposures [obs#]. If the target name has a space we should use double quotes around the name, e.g. "please do not use spaces in target names" Wtail -f OCDlog_20181210_C.txt | grep DEXhere the exposure time is in seconds. You can include a optional observing number, but if the software has not been restarted and you do not include this observing number it will use the next observing number. If you restart the software the count starts again at 1 and the next target will be included in directory 0001 even if there is data in that directory, however, due to the time stamp no data will be overwritten.

Examples:

hpfseconds GJ_3470 315 5 - 5 exposures of 315 seconds each.

hpfseconds GJ_3470 315 5 100 - 5 exposures of 315 seconds each to be saved in directory 0100.

You can look at any spectrum using showhpfspec YYYYMMDD obs#. If you leave off any date and observation number it will show the last exposure.

Examples:

showhpfspec

showhpfspec 20180428 2

Aborting normal science exposures

To stop after the current exposure (before the next ramp/exposure starts) you will need another hpf xterm and in there do a

sc

If you need to stop one of the observing scripts, like PartA, then try a cntl-C. This will stop the script and at the end of the current exposure (which you can monitor from the HPF monitor screen) you should be able to resume working with HPF.

Standard stars

For telluric standards you should not observe anything brighter than V=4.5 (our lists have been culled to remove these and emission line objects). If you find that any telluric is in a visual binary with a brighter star please please feel free to remove that from the list and let astronomer@… know so that we can edit the online list.

For exposure times please use:

• 315 sec exposure (30 reads) for anything fainter than V = 5.8
• 105 sec exposure (10 reads) for anything between 4.5 and 5.8 in V

Be sure to edit the night report to change the Data Type to "telluric"

NB Telluric standards must be taken with LFC closed

Closing the HPF terminal windows before an mcs reboot

Starting the HPF windows (method 1)

• From a gnome-terminal windo hit the File menu and select "Open Terminal" then "HPF".
• At the prompt enter "$ ssh -X hpf".
• At tthe prompt enter "$ rasession".

Starting the HPF windows (method 2)

• Click the icon named "RA HPF session" on the upper right of the middle RA monitor.
• In the window that comes up (which is already in full screen mode), ender "rasession".
• The four hpf windows appear

Stopping the terminal (essential before mcs reboot)

• Click the small X in the upper-right corner of the terminal window.

Or Just press Alt-F4 to kill terminal window

Stopping and starting the client

restarting the the HPF client

Note that the HPF client is restarted automatically every day 16:45 UT just after the LN2 refill. The act of restarting the HPF makes a small change to the instrument temperature which impacts RV stability.

restart_detectorclient is called daily as part of the crontab at 16:45 UT to restart the HPF client daily. It can also be called on demand.

There are two new executable scripts:

1) start_hpfdetectorInfloop.sh - this is running in the 'hpf' screen session, and takes care of automatically restarting the client after it is shutdown

2) restart_detectorclient - can be called from any hpf terminal (but preferably the 'scripts' session. This will trigger a shutdown sequence. After the shutdown, 'start_hpfdetectorInfloop.sh' will automatically take care of bringing things back up.

Check that the observation number is correct with pyhxrg:GetObsnum

if it is not then use the following command to set it.

tims Detect pyhxrg:SetObsnum:nn

where nn - number from previous GetObsnum? command

How to stop the client

If you need to completely power off the detector and prevent a restart, there are two options:

• tims detect pyhxrg:GetObsnum
• touch ~/stopdetectorloop
• tims detect pyhxrg:PowerDown
• tims detect shutdown

This will create a file in $HOME called stopdetectorloop. If that file exists, the infinite loop will not continue following the normal powerdown/shutdown sequence.

• There is a 10 s pause between completing the power down and auto-

restarting. You can press CTRL+c in the 'hpf' screen session during that time to abort the script.

You should see something like

Configured HxRG
Sending Cli15380: Detect Cli15380 PowerDown:OK
Resetting Asic for JADE1 ... Success
Powering Down Analog Supply Voltages:
   VDDA   ... Success
Powering Down Digital Supply Voltages:
   VDD3p3 ... Success
   VDD2p5 ... Success
   VDDIO  ... Success
   VSSIO  ... Success
Powering Down Voltage Regulators:
   VDDIO  ... Success
   VDD2p5 ... Success
   VDD2p5 ... Success
Powering Down Analog Main:
   5V  ... Success
De-Initializing Hal

Powered Down.

Shutting down this client.
Stopping scheduled loops:
    ...lakeshore336
Shutting down subclients
    ...pyhxrg
    ......Failed
    ...lakeshore336
    ......Success
Sending Cli16420: Detect Cli1642 * tims detect pyhxrg:SetObstype:Sci0 message:Detect_shuttingdown
Shutting down...
Connection lost to: lakeshore336
Reason : [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionDone'>: Connection was closed cleanly.
]
Closing connection to server
Client stopped.

When you restart the system you should check to see if it has the correct observation number (assuming you have not rolled through 0 UT).

Restarting communications with the HET

1) Find the 'HET Client' window in the 'TIMS' Screen session by doing a CTRL-a n (multiple times, until the window title says 'HET Client'). If a TIMS screen session is not open open one in an xterm with (HPFpython) [hpf@hpfserver ~]$ screen -x TIMS

2) In the HPF Scripts window or a new hpf xterm execute : tims het shutdown . This will stop the dictionary python stuff in the 'HET Client' window.

3) If you don't have a command prompt in the HET Client window, CTRL-c the client. If you do end up doing this you might want to run another tims het shutdown just to make sure everything is down.

4) Run the command 'timshet' in the HET Client screen window timshet

Note that this is the only TIMS client startup command that will not run with the normal startup syntax (e.g. python -m TIMS.clients.tims_het). This is because the library environment needs to be changed to access the het system libraries. We only want to do this for the HET client, so I've wrapped the normal startup command in a bash script that lives in ~/bin/ .

HPF twilight sky flats (evening)

These are not required nightly, but desired when possible. Only take HPF twilights if the sky is clear. Brightly lit clouds can cause HPF to saturate.

Run, on MCS, 2 minutes after sunset: htwi

Can view current incoming data with showhpfspec (after 3rd frame is captured)

Can also view fits files in:

~/HPFData/LinuxGigE/YYYYMMDD/hpf/OBSNUM/fits/

(note there is a 11k bias level)

HPF Darks and checking for saturation

If you need to take a dark exposure (i.e., to check for saturation), this is the procedure (in the CalScripts folder). First, make sure PFIP shutter is closed. Then, run:

./hpf_setup_ToTakeDark.sh

./hpf_expose.sh Dark 30 1 &

HPF cals in the evening and morning

run all of our science frames and cals in the screen -x scripts session so that any failures can be diagnosed by the HPF team.

ATTENTION. If you need to pause cals use:

• pausehpf (playhpf to continue) if you want to pause the script before the next hpf exposure stars
• pausecal (playcal to continue) is the command to pause the cals before it stars a next set of observations before using the FCU head. Historically this command was made to pause the cals temporally for anybody on side to switch on the lights in the dome, or use fcu head for something else, etc.

Evening

The evening 'A' calibration sequence includes the LFC and Etalon frames (and flats) that are necessary to start the nightly drift calibration. The 'C' calibrations contain the UNe lamps, which are only present as an emergency backup source in case the LFC is offline. Taking the 'A' sequence is critical; skipping the 'C' sequence if you are crunched for time is OK, although that sequence has been timed to fit within a typical stack time.

• The RA performing Ops in the afternoon will typically start the eveening HPS cal set. This will take about 44m to complete.

  [hpf@hpfserver CalScripts] cd /home/hpf/Scripts/InstConfig/CalScripts
  [hpf@hpfserver CalScripts] ./hpf_evening_cals_PartA.sh
  

If you don't have the 44m minutes for the full PartA cals just run the
./matt_shortflats.sh 22 minutes
or
./matt_wave.sh 14 minutes
depending no how long you have before sunset. The most critical cals are the wavelength (wave) cals.

• The night RA will check that the job is complete by confirming that the command prompt has returned in the HPF command window. If the job is still in execution, we'll see a sequence of integers being written to this window. Also, if the script is still executing, we'll see update message appearing in the HPF monitor window.

• The night RA will start the last set of HPF cals during stacking (or when possible). This last step, referred to as the "C" script will take about 43m. Note that we can take VIRUS or LRS2 cals during this (internal) part C calibration run.

  [hpf@hpfserver CalScripts] cd /home/hpf/Scripts/InstConfig/CalScripts
  [hpf@hpfserver CalScripts] ./hpf_evening_cals_PartC.sh
  

• Combining A+C cals takes 1h 27m, and could be initiated as part of the afternoon OPS if dome stays closed.

• *NOTE on order of A,C cals*:
  • Part C cals must be run AFTER Part A for lamp warmup/shutdown reasons. When Part A is run, it leaves the UNe lamp on. This lamp is needed by Part C, and Part C turns it off when it finishes. However, if Part C is not run after Part A, you must turn off the lamp manually with:

    [hpf@hpfserver CalScripts] /home/hpf/Scripts/InstConfig/CalScripts/hpf_TurnOff.sh UNeS
    

  • Similarly, if you are running Part C without first running Part A, you must turn on the UNe lamp via:

    [hpf@hpfserver CalScripts] /home/hpf/Scripts/InstConfig/CalScripts/hpf_TurnOn.sh UNeS
    

We have added an additional script (hpf_takeLFCHRCalExposure.sh) that will take a LFC exposure through the calibration fiber (~2 minutes to complete). This should not interfere with any other telescope operations, as it is entirely internal to HPF. If you find yourself in a situation similar to last night, you can run this stand alone script a few times while a different primary operation is happening (e.g. LRS or VIRUS exposure, slew, or other telescope setup). That will at least provide a few LFC reference points to tie the drift calculation to.

The script accepts an optional argument - N - where N is the number of ramps you wish to take. e.g. ./hpf_takeLFCHRCalExposure.sh 3

Morning

• The night RA will start a morning calibration for HPF. This will take about 50m to complete the portion requiring the FCUhead and a dark dome. An additional 1h10m with the pfip shutter closed is required to complete the full calibration script run.

  [hpf@hpfserver CalScripts] cd /home/hpf/Scripts/InstConfig/CalScripts
  [hpf@hpfserver CalScripts] ./hpf_morning_cals.sh
  

Stopping Cals

If you need to stop the cals then you just do a cntrC in the scripts window which will stop the script but will NOT stop the current exposure. Watch the HPF monitor screen to see when the instrument stops the exposure and goes idle.

Trouble shooting and restarting

Arrowing up to see what happened

If you missed something that has scrolled off the top of the screen and you want to see what it was you can do a <cnrl>a followed by an <esc> to get into "copy" mode. Then use the arrows to move the cursor up to get to the message of interest. To leaves copy mode hit two <esc> in a row. To see the full set of screens commands please see our Screens Summary Page.

Observing if the LFC is down

If the LFC is down but was requested, you can use the Etalon as a calibrator.

Instead of running the LFC setup scripts, run the following script:

hpf_setup_ToTakeHRCalFib.sh Etalon

This will turn on the etalon and open the shutter

To turn off the etalon, run the following two scripts:

hpf_TurnOff.sh Etalon

hpf_CloseCalShutter.sh HRCal Etalon

Joe has created two new scripts for the etalon:

etalon_open

etalon_close

These can be used just as you would have used the equivalent LFC scripts.

Fits Key words

See Restarting communications with the HET

Restart Client when it stop working

If see 'Error' in hpf session of client is not operation normally you can try to restart client

If you get the QSocketNotifier: in the detector client window you can try restarting it but if it comes back then the instrument is down and you should call Chad.

Here is restarting procedure:

HPF Starting and Stopping the Client

Problem with LFC

If part A script hang on message:

WARNING: Setting LFC flux to High ..

means absence of communication to LFC computer. It possible to abort script with Ctrl-C and report to Chand, Joe and Connor

Until problem fixed it is better to assume that LFC is down and use Etalon

Use the following replacement scripts

• hpf_evening_cals_PartA_withoutLFC.sh NOTE this takes 53 minutes to complete which is longer than the old Part A
• ./hpf_evening_cals_PartC.sh This one has no change because it does not interact with the LFC.
• hpf_evening_cals_PartAC_withoutLFC.sh This is just like the usual PartAC but calling the LFC-less partA.
• hpf_morning_cals_withoutLFC.sh NOTE this takes 26 minutes of dark dome to complete and 1 hour 15 minutes pfip restriction

The etallon is brighter than the LFC so if you take an exposure longer than 150 seconds with the etallon open you will likely saturate the detector. Thus, do not use the etallon during an observation unless the exposures are less than 150 seconds. Instead, taken an single etallon before and after each HPF observation.

Fixing the calibration connection

If the etalon turn off command hangs or any of the cal lamps try the following:

tims Calib shutdown in any hpf window with a prompt

In the TIMS HET window do a cntrl-a n five times to move to screen 2 which is the TIMS Calib screen. Note that we must hit cntl-a then hit the n (for next srccen). This full sequence is repeated to go to the next screen. The screen we usually operate in has the heading name "Screen 4: HPF HET Client". To fix the probelm of Q-data missing from the HPF headers (for example) we need the screen that is labeled "Screen 3: HPF TCS Relay". The screen we start from is named "Screen 4: HPF HET Client". The screen we want to end up at is It should have a prompt. If it does not you may need to do a few cntrl-c to kill any hanging processes.

In the TIMS Calib screen you should do an up arrow and see the python -m TIMS.clients.tims_calib command which restarted the Calib client. Wait a few minutes for it to finish.

Take a quick lfc image and perhaps a quick sky exposure to make sure all of the shutters and baffles are working.

Fixing the loss of tcs info in hpf headers

From Chad Bender (Feb17,2019):
Here is what I did to restart both the TCS relay and the TCS Client.
In the TIMS Screen session:
  1) Change to HPF TCS Relay window (currently #3, but not guaranteed to always be so)
  2) CTRL-C the script running there (this is a script, not a TIMS client).  You will need to CTRL-C twice to actually kill it.
  3) Restart the script (up arrow or ./bashwrapperfor_HETTCSRebroadcast.sh)
  4) Restart the TIMS HET Client in the normal manner
For this last step, we use:
     tims het shutdown         # in the Scripts window
     timshet                   # in the HPF HET Client window  (usually upper-right screen)

Listening to the fiber scrambler

As of Feb 2019, you can use the "RA" knob on the audio box (under the UT clock) to control the volume of the audio feed from HPF calibration enclosure which plays on the speaker under the RA desk. You can also connect to this audio feed with VLC media player on MCS via:

cvlc rtp://@:4444 --sout-rtp-caching 40

(drop the leading "c" if you want a UI)