Jim Fowler's notes to himself about procedures and processes to fix issues and problems at HET.

  * PowerOutage Procedures

  * TrackerProcedures Procedures

  * '''To clear out the ram disk on lrs2 or vdas'''
    * insure data directory in /hetdata has owner hetdex:hetdex and group write permission
    * run as hetdex on lrs2 or vdas  $HET_SRC_ROOT/camra/testing/proxy-pivot-[virus|lrs2]-data.sh !`find /mnt/camra_ramdisk/ -name "*.fits"`

  * '''To change the IP address on Tcs'''
    * change hostname in /etc/sysconfig/network to match /etc/hosts table
    * change IPADDR from 192.168.66.9 to 192.168.66.31 in /etc/sysconfig/network-scripts/ifcfg-em1
    * shutdown virtual htcs and wait a few minutes to let arp tables reset
    * restart hardware htcs
    * possibly restart dSpace

  * '''To look up something in tcsnamed'''
    * syscmd -v -n 'lookup(key="something")'
    * tcsnamed database is in /opt/het/hetdex/etc/conf/tcsnamed_db.conf

  * '''Restart vdas Magma if'''
    * the ccd array failed during an exposure
    * if vdas hangs during reboots at the Initializing PCIe step

  * '''LDAP admin'''
    * add new user (example .ldif files can be found in het-ldap:/root)
      * ldapadd -x -W -D "cn=manager,dc=as,dc=utexas,dc=edu" -f secook.ldif
      * ldappasswd -S -W -D "cn=manager,dc=as,dc=utexas,dc=edu" -x "uid=secook,ou=People,dc=as,dc=utexas,dc=edu"
      * ldapmodify -x -W -D "cn=manager,dc=as,dc=utexas,dc=edu" -f secook_users.ldif

  * '''Move previous month's weather data to the weather archives'''
    * change directories to `/weather/archive/het/`
    * move any previous month's files `mmmYYYY.dat` to the directory `YYYY`, where mmm is the lower case month name and YYYY is the year.
    * move any previous month's files `enclosure_temps_mmmYYYY.dat` to `enclosure_temps/YYYY`
    * the data should now be available through Darrin's weather archive
    * Note, need to automate this action someday
    * Note that the `enclosure_temps_*.dat` really don't contain any data, should shutdown the collection process someday.

  * '''Move truss temperature data to weather archives'''
    * `rdesktop trusst`
    * login guider, standard password.
    * open a finder window and change to the folder to `C:\Users\guider\Temperatures`
    * open a second finder window change to the folder `Z:\jrfwin\trusst`
    * copy the relevant months from the first folder to the second folder
    * move older files in C:\Users\guider\Temperature to the relevant year folder
    * on a Redhat machine, login and change directory `/data1/het/common/jrfwin/trusst`
    * run the command `python convert.py nnnn`, where `nnnn` is the year of interest. This program will read the *nnnn_Temps.dat file, convert the data to a format understandable by the weather archive system, and write the reformatted data to `/weather/archive/het/trusst/nnnn`.
    * you can then remove the data file in `/data1/het/common/jrfwin/trusst` 

  * '''Restart Truss Temperatures Computer'''
    * `rdesktop trusst`
    * login as guider
    * Start a command prompt from the Start->Accessories->Command Prompt
    * run the command `shutdown -r -f -t 0`  (-r reboot instead of shutdown) (-f force kill all running applications) (-t 0 zero second delay)

  * '''Enable core dumps on Redhat'''
    * see the instructions at https://www.unixmen.com/how-to-enable-core-dumps-in-rhel6/

  * '''To clear a port on the terminal server'''
    * telnet hetts2
      * enable
      * tunnel <n> # Where <n> is the channel number 1-16
      * accept
      * kill connection
      * exit (do this command three times)
    * you should be able to connect to the port now.

  * '''To reset a locked out Temperature Setpoint on Trane'''
    * if you are unable to set an Temperature set point 
    * select the Applications Tree
    * select the Equipment item
    * select the Application tab on the Equipment Summary
    * select the Configuration tab 
    * find AHU 1 Truss Temp Setpoint
    * select the Override on the right hand side
    * under Override Details click the checkbox on "Release all Manual Overrides"
    * click the Apply button 

Bugzilla Notes:

  * Work flow
    * user submits bug
    * responsible is assigned automatically or by JRF/Herman, state changed to assigned
    * assignee selects proper product/category for bug
    * assignee looks in to bug and makes comments (comment by assignee or status is duplicate)
    * assignee resolved bug status (status changed to resolved *)
    * Weekly cadmin or jrf can verify bugs
    * Weekly cadmin or jrf can close bugs

  * Available products and assignees
    * Camra jason ramsey
    * Ccas  jrf/herman
    * Dimm  jrf/gdamm/herman
    * Dmi   gdamm/herman
    * Doc   jrf/herman
    * Dome  gdamm/herman
    * Ecs   gdamm/herman
    * Enclosure gdamm/herman
    * ...

Have categories General, Hardware, Software for all Products. Only General category is open for new bugs.  Assignee selects proper category for the bug. Could have further subdivision for Dome-Hardware or Shutter-Hardware or Das-Software or Dome-Pmac or Shutter-Pmac under the Dome Product for example.

  * To Do List
    * Close, reassign, duplicate existing open bugs so that only open bugs are real problems
    * Reassign or close bug assigned to people who are no longer here.
    * For future
      * Need to resolve if new bugs should only go to Jim-Herman or if they can be assigned automatically.
      * Need to determine Product, Categories 
      * Consider PI categories
      * Need to write documentation for users
      * Need to generate weekly report somehow