OPS: Sergey Rostopchin / Nathan McReynolds HET released for OPS at 2024-04-18 21:20:20 UT SUMMARY: All good, ACAM stability is great today. One issue is PFIP did not w Calibrations: HPF cals ETA Start time 21:55 UT PartA 22:38 UT PartC 23:24 UT at jobs check @ 2024-04-18 21:56:05 #1196 scheduled @ 2024-04-18 22:38 cal lrs2 -l qth ldls_long cd-a fear hg cal virus -l ldls_long cd-a hg cal lrs2 -pb -b 11 -pd -de 360 -d 3 V/L cals ETA LRS2 23:04 UT VIRUS 23:23 UT V/L PARALLEL DARKS AND BIASES 23:56 UT Sunset tonight 01:33 UT Still needed cals: NONE ========================================================================================= QUEUE STATE 2024-04-18 21:19:37 Checked for new target submissions and update 2024-04-18 20:26:44 New allocation(s) has been accepted and PIs settings were updated 2024-04-18 21:20:14 Ran the queue updates 2024-04-18 21:29:35 OCD targets for 2460419.51 +-------------+----------------------+-------+-------+--------+-------+-----+ | UT | Field | QIDX | PIpri | Az | illum | Pri | +-------------+----------------------+-------+-------+--------+-------+-----+ | 10:23-10:48 | NEP_row11_p12c | 2076 | 51 | 3.3 | 56.7 | 0 | +-------------+----------------------+-------+-------+--------+-------+-----+ Non-HETDEX targets +-----+------------+---------------+-----+-------+-------+----------+---------+ | idx | program | object | pri | Eopt | Wopt | sky | syndate | +-----+------------+---------------+-----+-------+-------+----------+---------+ | 570 | UT24-2-003 | VDFI_EGS_D3.4 | 0 | 04:24 | -- | 21.5 / S | | | 568 | UT24-2-003 | VDFI_EGS_C3.4 | 0 | 04:24 | -- | 21.5 / S | | | 569 | UT24-2-003 | VDFI_EGS_D1.4 | 0 | 04:24 | -- | 21.5 / S | | | 564 | UT24-2-003 | VDFI_EGS_A3.4 | 0 | 04:24 | -- | 21.5 / S | | | 566 | UT24-2-003 | VDFI_EGS_B3.4 | 0 | 04:24 | -- | 21.5 / S | | | 687 | UT24-2-006 | J1427+2632_p5 | 0 | -- | 09:58 | 21.5 / S | | | 688 | UT24-2-006 | J1427+2632_p5 | 0 | -- | 09:58 | 21.5 / S | | | 689 | UT24-2-006 | J1427+2632_p6 | 0 | -- | 09:58 | 21.5 / S | | | 690 | UT24-2-006 | J1427+2632_p6 | 0 | -- | 09:58 | 21.5 / S | | | 691 | UT24-2-006 | J1552+5705_p3 | 0 | -- | 10:56 | 21.5 / S | | | 692 | UT24-2-006 | J1552+5705_p3 | 0 | -- | 10:56 | 21.5 / S | | +-----+------------+---------------+-----+-------+-------+----------+---------+ ========================================================================================= STATE OF NOSS VITAL COMPONENTS REPLICATION DEPOT 2024-04-18 21:20:13: Replication up and running HYDRA 2024-04-18 21:20:13: Replication up and running RA interface checked 2024-04-18 21:21:21 PI interface checked 2024-04-18 21:21:23 HTOPX3 checked 2024-04-18 21:21:25 Night Report checked 2024-04-18 21:56:21 ========================================================================================= STATE OF VARIOUS PRODUCTION COMPUTERS and TCS COMMUNICATION 2024-04-18 21:22:05 HET OPS calendar checked 2024-04-18 21:21:41 RA station sound checked 2024-04-18 21:22:12 DIMM data mounted 2024-04-18 21:22:08 State of the RA station file systems / 80% HPF TCS COMM /HPFData/LinuxGigE/20240418/hpf/0088/fits/hpf_20240418T212442_R01_F010.fits RTFX = -0.4768 / Tracker Real Coord X STRTAZ = 179.897095 / Structure Actual Azimuth LRS2 2024-04-18 21:25:25 LRS2 server running Pivot server running Data transfer server (RAM->LOCAL->NFS) running RAMDISK state 0% Used LRS2 TCS COMM /hetdata/data/20240418/lrs2/lrs20009904/exp01/lrs2/20240418T215251.9_066LL_cmp.fits STRUCTAZ= 0.007917 / Azimuth of the structure X_STRT = -0.443871 / X pos of tracker at start of exposure (mm) VIRUS 2024-04-18 21:26:38 VIRUS server running Pivot server running Data transfer server (RAM->LOCAL>NFS) running Data transfer server (NFS->TACC) running RAMDISK state 0% Used VIRUS TCS COMM /hetdata/data/20240418/virus/virus0009902/exp01/virus/20240418T214857.2_046RL_drk.fits STRUCTAZ= 0.008818 / Azimuth of the structure X_STRT = -90.672024 / X pos of tracker at start of exposure (mm) ========================================================================================= HPF Detect says "GetObsnum:87" 2024-04-18 21:23:28 Het says "message:Het_shuttingdown" 2024-04-18 21:23:33 timshet started 2024-04-18 21:24:45 fiber scrambler sounds normal ========================================================================================= LRS2 Uptime: 022:02:25:17 2024-04-18 21:54:50 LRS2 data and Qth/FeAr lamps checked IP/VG turn off 2024-04-18T21:20:33 Cryo Cryo Left Right Left Right Left Right Left Right IHMP Unit Cntl Encl Mux Temp Press SetPt SetPt Temp Temp Htr Htr Pwr Pwr Snnn Vnnn Cnnn Exyz Mnnn C mB C C C C V V % % 1 056 503 082 111 000 -182.7 1.2e-06 -110.0 -110.0 -110.0 -110.0 0.835 1.234 23.20 50.70 2 066 502 100 121 000 -184.0 1.5e-06 -110.0 -110.0 -110.0 -110.0 1.164 1.273 45.13 53.92 ========================================================================================= VIRUS VHC report: 20240418 / 9901:1 S015-V317LL: FFT 1.9 > 1.4 S015-V317LU: FFT 1.6 > 1.4 S026-V318RL: FFT 1.7 > 1.4 S040-V419LL: FFT 1.5 > 1.4 S085-V307LL: FFT 1.5 > 1.4 S085-V307LU: FFT 1.6 > 1.4 S085-V307RL: FFT 1.5 > 1.4 S085-V307RU: FFT 1.5 > 1.4 S087-V204LU: bias line stdev 0.6 > 0.5 S091-V205LL: zero pixels 1065024 > 5000 S095-V426RU: zero pixels 1065024 > 5000 Uptime: 027:01:24:40 Virus Ion Pumps: OFF: ok STATUS: All ion pumps are off since 2024-04-17 21:25:54Z 2024-04-18 21:51:56 VIRUS data and LDLS/CdA/FeAr lamps checked 2024-04-18T21:20:34 Cryo Left Right Left Right Left Right Left Right Left Right IHMP Unit Cntl Encl Mux Temp SetPt SetPt Temp Temp Htr Htr Pwr Pwr Id Id Snnn Vnnn Cnnn Exyz Mnnn C C C C C V V % % 1 013 412 021 241 008 -182.5 -110.0 -110.0 -110.0 -110.0 1.208 1.107 48.60 40.83 26932 26960 2 014 411 008 221 006 -184.6 -110.0 -110.0 -110.0 -110.0 1.348 1.391 60.49 64.43 26904 27016 3 015 317 049 234 007 -184.5 -110.0 -110.0 -110.0 -110.0 1.177 1.376 46.09 63.04 21133 21181 4 016 409 058 242 008 -180.3 -110.0 -110.0 -110.0 -110.0 1.171 1.141 45.69 43.34 26937 26988 5 021 315 068 218 005 -180.8 -110.0 -110.0 -110.0 -110.0 1.206 1.231 48.45 50.46 21176 21177 6 022 316 102 247 008 -182.0 -110.0 -110.0 -110.0 -110.0 1.326 1.253 58.49 52.24 21188 21195 7 023 313 024 255 009 -182.9 -110.0 -110.0 -110.0 -110.0 1.138 1.145 43.12 43.66 21254 21265 8 024 410 070 224 006 -181.9 -110.0 -110.0 -110.0 -110.0 1.241 1.058 51.28 37.30 27004 27010 9 025 303 028 254 009 -184.5 -110.0 -110.0 -110.0 -110.0 1.284 1.146 54.93 43.75 21091 21208 10 026 318 057 244 008 -182.3 -110.0 -110.0 -110.0 -110.0 1.361 1.151 61.65 44.09 21150 21225 11 027 314 018 253 009 -184.4 -110.0 -110.0 -110.0 -110.0 1.226 1.296 50.07 55.90 21113 21227 12 028 413 014 252 009 -178.2 -110.0 -110.0 -110.0 -110.0 1.020 0.959 34.62 30.59 27031 27044 13 030 420 061 212 005 -179.9 -110.0 -110.0 -110.0 -110.0 1.019 1.075 34.59 38.49 27040 26967 14 031 408 033 211 005 -185.5 -110.0 -110.0 -110.0 -110.0 1.325 1.199 58.47 47.85 26915 26951 15 032 320 059 236 007 -184.1 -110.0 -110.0 -110.0 -110.0 1.230 1.190 50.39 47.18 21172 21262 16 033 310 069 257 009 -183.3 -110.0 -110.0 -110.0 -110.0 0.991 1.256 32.73 52.53 21182 21213 17 034 305 078 248 008 -183.6 -110.0 -110.0 -110.0 -110.0 1.297 1.191 56.02 47.24 21132 21165 18 035 202 038 238 007 -171.1 -110.0 -110.0 -110.0 -110.0 0.616 1.189 12.64 47.05 15278 18066 19 036 302 053 235 007 -185.5 -110.0 -110.0 -110.0 -110.0 1.162 1.250 44.93 52.03 21102 21264 20 037 306 042 228 006 -182.2 -110.0 -110.0 -110.0 -110.0 1.357 1.008 61.27 33.86 21248 21252 21 038 414 072 251 009 -180.1 -110.0 -110.0 -110.0 -110.0 1.141 0.955 43.36 30.37 26927 26959 22 039 424 046 217 005 -184.0 -110.0 -110.0 -110.0 -110.0 1.179 1.257 46.25 52.57 27033 26851 23 040 419 085 225 006 -183.7 -110.0 -110.0 -110.0 -110.0 1.140 1.135 43.28 42.89 26873 26897 24 041 407 104 226 006 -181.1 -110.0 -110.0 -110.0 -110.0 1.148 1.170 43.90 45.59 26902 26954 25 042 321 077 243 008 -183.2 -110.0 -110.0 -110.0 -110.0 1.301 1.354 56.34 61.00 21155 21191 26 043 323 041 231 007 -186.4 -110.0 -110.0 -110.0 -110.0 1.184 1.449 46.67 69.88 21153 21190 27 044 304 083 256 009 -183.7 -110.0 -110.0 -110.0 -110.0 0.941 1.134 29.50 42.82 21120 21126 28 045 312 084 237 007 -183.5 -110.0 -110.0 -110.0 -110.0 1.085 0.956 39.19 30.46 21216 21218 29 046 309 025 258 009 -184.2 -110.0 -110.0 -110.0 -110.0 1.170 1.393 45.60 64.60 21202 21148 30 047 308 081 227 006 -183.5 -110.0 -110.0 -110.0 -110.0 1.061 1.317 37.50 57.75 21129 21229 31 048 415 031 223 006 -184.4 -110.0 -110.0 -110.0 -110.0 1.286 1.203 55.06 48.20 26910 26911 32 049 425 052 222 006 -185.0 -110.0 -110.0 -110.0 -110.0 1.323 1.336 58.30 59.40 26990 27012 33 050 504 067 213 005 -186.1 -110.0 -110.0 -110.0 -110.0 1.255 1.238 52.42 51.04 27726 27725 34 051 503 016 214 005 -182.4 -110.0 -110.0 -110.0 -110.0 0.942 1.170 29.57 45.55 27702 27693 35 052 301 087 233 007 -182.7 -110.0 -110.0 -110.0 -110.0 1.075 1.445 38.47 69.50 21123 21185 36 053 311 027 245 008 -184.6 -110.0 -110.0 -110.0 -110.0 1.154 1.071 44.30 38.18 21214 21215 37 057 418 055 216 005 -180.9 -110.0 -110.0 -110.0 -110.0 1.081 0.985 38.88 32.32 26876 26984 38 058 416 034 215 005 -183.2 -110.0 -110.0 -110.0 -110.0 1.155 1.179 44.38 46.26 27032 27039 39 059 508 043 232 007 -185.4 -110.0 -110.0 -110.0 -110.0 1.340 1.309 59.81 57.05 27772 27650 40 060 206 039 246 008 -182.7 -110.0 -110.0 -110.0 -110.0 1.226 1.203 50.07 48.21 18201 18265 41 061 427 106 116 000 -181.0 -110.0 -110.0 -110.0 -110.0 1.041 1.038 36.08 35.86 26888 26985 42 062 319 044 137 002 -181.8 -110.0 -110.0 -110.0 -110.0 1.099 1.313 40.20 57.44 21166 21200 43 063 511 062 133 002 -183.6 -110.0 -110.0 -110.0 -110.0 1.153 1.109 44.27 40.94 27661 27668 44 067 025 092 154 004 -186.8 -110.0 -110.0 -110.0 -110.0 1.293 1.381 55.66 63.46 12841 12807 45 068 417 032 126 001 -183.3 -110.0 -110.0 -110.0 -110.0 1.182 1.165 46.50 45.17 26989 27013 46 069 421 076 127 001 -179.0 -110.0 -110.0 -110.0 -110.0 0.943 0.950 29.58 30.02 26912 26952 47 070 502 080 125 001 -182.8 -110.0 -110.0 -110.0 -110.0 1.149 1.173 43.92 45.84 27785 27627 48 071 406 088 117 000 -183.2 -110.0 -110.0 -110.0 -110.0 1.224 1.295 49.89 55.87 27018 27025 49 072 405 023 115 000 -180.8 -110.0 -110.0 -110.0 -110.0 1.232 1.256 50.55 52.52 26948 26907 50 073 404 029 143 003 -185.5 -110.0 -110.0 -110.0 -110.0 1.216 1.246 49.21 51.65 26866 26939 51 074 403 063 142 003 -186.1 -110.0 -110.0 -110.0 -110.0 1.410 1.360 66.19 61.57 26956 27061 52 075 324 048 156 004 -183.2 -110.0 -110.0 -110.0 -110.0 1.386 1.185 63.99 46.78 21212 21253 53 076 325 030 147 003 -183.6 -110.0 -110.0 -110.0 -110.0 0.987 1.141 32.41 43.33 21221 21263 54 077 402 054 141 003 -182.2 -110.0 -110.0 -110.0 -110.0 1.280 1.221 54.51 49.61 26856 26928 55 078 422 096 124 001 -180.9 -110.0 -110.0 -110.0 -110.0 1.056 0.983 37.10 32.19 26900 26894 56 079 401 037 123 001 -184.8 -110.0 -110.0 -110.0 -110.0 1.317 1.291 57.78 55.50 26892 26913 57 080 501 066 148 003 -182.0 -110.0 -110.0 -110.0 -110.0 1.157 1.051 44.57 36.77 27789 27630 58 081 327 073 118 000 -183.1 -110.0 -110.0 -110.0 -110.0 0.864 1.134 24.84 42.84 21121 21168 59 082 326 060 146 003 -183.9 -110.0 -110.0 -110.0 -110.0 1.076 1.046 38.52 36.40 21194 21267 60 083 328 091 112 000 -184.2 -110.0 -110.0 -110.0 -110.0 1.209 1.317 48.65 57.73 21143 21243 61 084 322 026 128 001 -184.1 -110.0 -110.0 -110.0 -110.0 1.342 1.084 59.93 39.08 21135 21151 62 085 307 105 131 002 -185.9 -110.0 -110.0 -110.0 -110.0 1.302 1.115 56.41 41.39 21220 21240 63 086 203 040 151 004 -185.5 -110.0 -110.0 -110.0 -110.0 1.387 1.463 64.03 71.23 18092 18110 64 087 204 056 145 003 -183.0 -110.0 -110.0 -110.0 -110.0 1.373 1.418 62.77 66.93 18181 18264 65 088 329 071 144 003 -184.3 -110.0 -110.0 -110.0 -110.0 1.306 1.328 56.77 58.72 21173 21203 66 089 423 095 114 000 -183.9 -110.0 -110.0 -110.0 -110.0 1.175 1.121 45.97 41.85 27024 26931 67 091 205 094 113 000 -184.8 -110.0 -110.0 -110.0 -110.0 1.346 1.260 60.28 52.81 18244 18249 68 092 330 103 132 002 -182.8 -110.0 -110.0 -110.0 -110.0 1.202 1.294 48.14 55.71 21223 21224 69 093 507 020 157 004 -182.5 -110.0 -110.0 -110.0 -110.0 1.238 1.182 51.01 46.55 27735 27779 70 094 509 012 155 004 -180.9 -110.0 -110.0 -110.0 -110.0 1.113 1.064 41.21 37.70 27771 27679 71 095 426 107 158 004 -182.0 -110.0 -110.0 -110.0 -110.0 1.184 1.152 46.69 44.21 26940 26983 72 096 510 051 134 002 -185.1 -110.0 -110.0 -110.0 -110.0 1.155 1.223 44.40 49.83 27680 27691 73 097 505 064 135 002 -181.5 -110.0 -110.0 -110.0 -110.0 1.259 1.103 52.81 40.48 27743 27756 74 098 027 090 122 001 -182.3 -110.0 -110.0 -110.0 -110.0 1.335 1.157 59.34 44.54 12808 12849 75 103 514 075 152 004 -183.5 -110.0 -110.0 -110.0 -110.0 1.315 1.268 57.59 53.54 27800 27732 76 104 512 098 153 004 -178.0 -110.0 -110.0 -110.0 -110.0 0.872 0.666 25.30 14.78 27741 27671 77 105 513 036 138 002 -178.5 -110.0 -110.0 -110.0 -110.0 1.049 0.942 36.61 29.55 27748 27734 78 106 506 013 136 002 -182.7 -110.0 -110.0 -110.0 -110.0 1.119 1.129 41.71 42.46 27796 27737 ========================================================================================= PFIP Notes: PFIP server issue during ACQ stability test: You have requested a stability test of the acq with 10 cycles, 0.7s exposures, and a pause of 15s between each move Please confirm the following: 1) Tracker is aborted (optionally, at center) 2) No other cals using the FCU are running, including with HPF You have used the -f option, so I will not ask for your confirmation! here we go... 2024-04-18 21:27:32.988278 connecting to TCS systems... stopping ACQ pipeline and ACQ metrology loop (just in case) configuring FCU/PFIP... unexpected error setting up the PFIP/FCU hardware. I will cleanup and return to a neutral state. Please try again. Traceback (most recent call last): File "/usr/local/bin/stability_test.py", line 801, in run_test() File "/usr/local/bin/stability_test.py", line 361, in run_test cleanup() File "/usr/local/bin/stability_test.py", line 447, in cleanup pfip.FCUCancelExposure(shutter="SliderB") File "", line 3, in FCUCancelExposure_method pytcs.error: remote exception: Error writing to MODBUS registers ret=32: Broken pipe Check for probes move Before: GC1 0.14993 -0.05999 GC2 -0.15121 0.05708 WF1 0.06054 0.14972 WF2 -0.05923 -0.15027 After home5: Commanded positions: GC1 0.16 -0.00 GC2 -0.16 0.00 WF1 0.00 0.16 WF2 0.00 -0.16 Final positions: GC1 0.15995 -0.00003 GC2 -0.15994 0.00002 WF1 -0.00000 0.16002 WF2 0.00001 -0.16004 ACQ camera mirror insertion test: averages on 9 comparisons: ------------------------------- dx = -0.0676 +/- 0.0194 [pixels] dy = 0.0043 +/- 0.0040 [pixels]