Changes between Version 25 and Version 26 of HetProcedures/DevelInstall


Ignore:
Timestamp:
Feb 7, 2019 9:17:48 PM (5 years ago)
Author:
admin
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • HetProcedures/DevelInstall

    v25 v26  
    1 '''Board Directive''':
    21
    3   * At the behest of the Board, HET will start full time science without any routine engineering periods
    4   * HET may request engineering time with the HET Scientist before the call for proposals for a trimester if engineering tests will preclude science.  Once granted this engineering time may be scheduled by the operations team as needed.
    52
    63'''Goals'''
     4  * At the behest of the Board, HET will start full time science without any routine engineering periods. This requires that we be able to install trackable changes.
    75  * install small, incremental changes without interrupting science or causing long down times.
    86  * deploy larger changes in a controlled fashion
    97  * be able to switch back to a working version of software if a change does not work.
    10     * this allows use to do short engineering runs of 1-2 hours and then revert back to working code for science
     8    * this allows us to do short engineering runs of 1-2 hours and then revert back to working code for science
    119  * test changes as completely as possible during the day time
    1210  * keep track of which version is running at which time
    … …  
    1614'''Development'''
    1715
    18   You can work in your own branch or trunk. For mid-sized or extensive modifications that may require intermediate commits, a branch is recommended. Once you have tested as fully as possible the branch can be deployed in order to test on the hardware during the day or on the sky at night.  If your changes don't work, redeploy the previous tag (see '''Deployment''' below). Once you are confident that your modification works, then do a single merge with trunk.  In this manner, only one revision needs to be merged or back out in order to add or remove your modification. When working in a branch, you should merge trunk  frequently to pick up the current working version
     16  The environment variable `$(HET_SRC_ROOT)` is defined as the location of the working copy of the software`$(HET_DEPLOY_DIR)` is the location of the deployment directory.
     17 
     18  You can work in your own branch or trunk. For mid-sized or extensive modifications that may require intermediate commits, a branch is recommended. Once you have tested as fully as possible the branch can be deployed in order to test on the hardware during the day or on the sky at night.  If your changes don't work, redeploy the previous tag (see '''Deployment''' below). Once you are confident that your modification works, then do a single merge with trunk.  In this manner, only one revision needs to be merged or back out in order to add or remove your modification. When working in a branch, you should merge from trunk frequently to pick up the current working version. This will insure that changes made by others will not cause conflicts with your changes.
    1919
    2020  If you are making only small changes that need only one or two commits, you can work in trunk... but don't break the build.
    2121
    22   Testing is important prior to releasing code on unsuspecting RA/TOs.  With a new deployment scheme it is easy to revert to a prior working edition.
     22  Testing is important prior to releasing code on unsuspecting RA/TOs. Test as many of your modifications as possible prior to testing at night. How you do testing is up to you, but with a new deployment scheme it is easy to revert to a prior working edition if your testing reveals fatal problems.
    2323
    24   If you need to test only your modification at night, then create a branch, make your changes, commit, and then install only the package(s) with the changes using the '-p' flag from `het_release`. Allow your version to run at night but be prepared to reinstall the current build (by running `make install [packages]` in the current build directory).
     24  If you need to test your modification at night, then create a branch, make your changes, commit, and then install only the package(s) with the changes using the `-p` flag from `het_release`. Allow your version to run at night but be prepared to re-install the current build (by running `make install [packages]` in the current build directory).
    2525
    26   If testing during daytime only, then you can run the packages that have the changes. Replace the existing code before night operations are reinstall the existing version by doing `make install [packages]` in the current `build` directory. In this way we can keep track of what version are running at night during science operations.
     26  If testing during daytime only, then you can run the packages that have the changes. Replace the modified code before night operations by re-install the existing version with `make install [packages]` in the current `build` directory. In this way we can keep track of what version are running at night during science operations.
    2727
    2828
    2929'''Deployment'''
    3030
    31   * Old Deployment method
    32     * roughly once per month create a new branch from trunk with the name branches/deployment/science/YYYYMMDD
    33     * build and install the branch
    34     * merge important bug fixes into the branch as necessary
     31Old Deployment method
     32  * roughly once per month create a new branch from trunk with the name branches/deployment/science/YYYYMMDD
     33  * build and install the branch
     34  * merge important bug fixes into the branch as necessary
    3535
    36   * New Deployment method (still being debugged)
    37     * is done via the script `het_release`, for example I would use the following
    38       * `het_release -U fowler -a jrf@utexas.edu -b /opt/het/hetdex/src/tcs -r het-operations@het.as.utexas.edu -i /opt/het/hetdex -u hetdex:hetdex -P trunk`
    39       * the command line flags for het_release are shown below. Note that -i is required for installation and -b is highly recommended. I use -U because my user name on hetdex-pr is different that my user name on the machines at Het. The -P flag defaults to trunk but must be used if you want to install a branch or a tag.
     36The current deployment method is done via the script `het_release`, for example I would use the following
     37
     38{{{ het_release -U fowler -a jrf@utexas.edu -b /opt/het/hetdex/src/tcs -r het-operations@het.as.utexas.edu -i /opt/het/hetdex -u hetdex:hetdex -P trunk -p all }}}
     39
     40The command line flags for het_release are shown below. Note that -i is required for installation and -b is highly recommended. I use -U because my user name on hetdex-pr is different that my user name on the machines at Het. The -P flag defaults to trunk but must be used if you want to install a branch or a tag.
    4041
    4142    {{{
    4243$ ./het_release -h
    43 Usage: ./het_release [-b <build dir>] [-c <changelog>] [-d] [-h] [-i <install dir>] [-l <log file>]
    44            [-p <packages>] [-P <repo path>] [-r <recipients>] [-R <repo>] [-t <tag>]
    45            [-u <user>[:<group>]] [-U <repo user>]
     44Usage: ./het_release [-b <build dir>] [-c <changelog>] [-d] [-h]
     45           [-i <install dir>] [-l <log file>] [-p <packages>]
     46           [-P <repo path>] [-r <recipients>] [-R <repo>]
     47           [-t <tag>] [-u <user>[:<group>]] [-U <repo user>]
    4648
    47 Tags the HET <source repository path> at <tag> then checks the tag out, builds and
    48 optionally installs the HET software <packages>.
     49Tags the HET <source repository path> at <tag>,
     50then checks the tag out, builds and optionally
     51installs the HET software <packages>.
    4952
    50 NOTE: This script likely doesn't support file paths with special characters and/or spaces.
     53NOTE: This script likely doesn't support file paths with
     54      special characters and/or spaces.
    5155
    5256Options are:
    5357
    54   -a <email author> Sets the author for the email notification for the build.
    55   -b <build dir>    Specifies the path where the <tag> will be checked out and built.
     58  -a <email author> Sets the author for the email notification
     59                    for the build.
     60  -b <build dir>    Specifies the path where the <tag> will be
     61                    checked out and built. Defaults to /tmp.
     62  -c <changelog>    Copies the <changelog> to the given file.
     63                    The temporary file used for the changelog
     64                    is removed without copying by default.
     65  -d                Enable debug output. To disable system
     66                    and svn commands, give '-dd'.
     67  -h                Print this message and exit.
     68  -i <install dir>  Specifies the directory where the build will
     69                    be installed.  If omitted or given an empty
     70                    string, the installation is skipped. Defaults
     71                    to "".
     72  -l <log path>     Writes the script console log and build log
     73                    under <log path>/. If empty, the log is
     74                    written to the current working directory.
    5675                    Defaults to /tmp.
    57   -c <changelog>    Copies the <changelog> to the given file.  The temporary file used
    58                     for the changelog is removed without copying by default.
    59   -d                Enable debug output. To disable system and svn commands, give '-dd'.
    60   -h                Print this message and exit.
    61   -i <install dir>  Specifies the directory where the build will be installed.  If omitted
    62                     or given an empty string, the installation is skipped.  Defaults to
    63                     "".
    64   -l <log path>     Writes the script console log and build log under <log path>/.  If empty,
    65                     the log is written to the current working directory.  Defaults to /tmp.
    66   -p <packages>     List of systems to build.  Packages supported are determined by top
    67                     level directories and makefile targets.  Defaults to all.
    68   -P <repo path>    Path of the installation source in the repository.  Defaults to trunk.
    69   -r <addresses>    Specify a comma separated list of email <addresses> that will
    70                     be sent a copy of the build and changelog.
    71   -R <repo>         Remote location of the repository.
    72                     Defaults to svn+ssh://hetdex-pr.as.utexas.edu/repos/het.
    73   -t <tag>          Specify the repository path for the tag.  This is the relative
    74                     path from the root given by <repo>.
    75                     Defaults to tags/deployment/liberty_20180904T145132.
    76   -u <user>:<group> Sets ownership of the insallation to <user>:<group>.
    77                     Defaults to leaving original installation permissions set.
    78   -U <user>         Username for all svn transactions, defaults to the current user.
     76  -p <packages>     List of systems to build. Packages supported
     77                    are determined by top level directories and
     78                    makefile targets. Available group targets are:
     79                    all, telescope, instruments, all_config,
     80                    telescope_config, or instruments_config.
     81                    Defaults to all.
     82  -P <repo path>    Path of the installation source in the
     83                    repository. Defaults to trunk.
     84  -r <addresses>    Specify a comma separated list of email
     85                    <addresses> that will be sent a copy of
     86                    the build and changelog.
     87  -R <repo>         Remote location of the repository. Defaults
     88                    to svn+ssh://hetdex-pr.as.utexas.edu/repos/het.
     89  -t <tag>          Specify the repository path for the tag.
     90                    This is the relative path from the root given
     91                    by <repo>. Defaults to
     92                    tags/deployment/<hostname>_<ISO8601_time>.
     93  -u <user>:<group> Sets ownership of the installation
     94                    to <user>:<group>.
     95                    Defaults to leaving original installation
     96                    permissions set.
     97  -U <user>         Username for all svn transactions,
     98                    defaults to the current user.
    7999}}}
    80100
    … …  
    94114    * `generate-changelog svn+ssh://fowler@hetdex-pr.as.utexas.edu/repos/het/trunk .`
    95115
     116'''Requirements and Caveats'''
     117  * The build directory must be writable by the user running the `het_release` script.
     118  * The installation directory must be writable by the user running the `het_release` script or by the user designated by the `-u` flag on the command line.
     119  * Currently the Makefile enforces the installation user to be `hetdex`. This may change in the future.
     120  * The `het_release` script is normally run on the host 'htcs'.  This may also change if we do separate installation of instruments and telescope packages.
     121  * Installing `telescope` or `instruments` does not install the configuration files.
     122  * The target `make install_configs` is defined in makefile.commonTargets. If invoked in `$(HET_SRC_ROOT)`, it will install all the files in `$(HET_SRC_ROOT)` into `$(HET_DEPLOY_DIR)/etc`. You do not want to do this.
     123  * `$(HET_SRC_ROOT)` must be defined if you invoke `make` directly.  `$(HET_DEPLOY_DIR)` will default to `/opt/het/hetdex` unless explicitly defined in the environment.
     124
     125'''Examples'''
     126
     127Some reasons for doing different types of installation and examples of how to do them
     128
     129  * You are working in trunk in the pas system on a bug fix. You have thoroughly tested your change and are ready to install it.  You should use the command `het_release ... -P trunk -p telescope`
     130  * You are working in trunk in the camra system on a bug fix. You have thoroughly tested your change and are ready to install it.  You should use the command `het_release ... -P trunk -p instruments`
     131  * You are working in trunk on a bug fix in het_lib. You have thoroughly tested your change and are ready to install it.  You should use the command `het_release ... -P trunk -p all`
     132  * You are working in trunk in the pas system on change to the configuration files. You have thoroughly tested your change and are ready to install it.  You should use the command `het_release ... -P trunk -p telescope_config`
     133  * You are working in trunk and adding a new spectrograph configuration file. You have thoroughly tested your change and are ready to install it.  You should use the command `het_release ... -P trunk -p instruments_config`
     134  * You have made changes to the pas configuration as well as adding a new spectrograph configuration.  You should use the command `het_release ... -P trunk -p all_config`
     135
     136  * You have made extensive changes to the executables in a new branch, you have tested it, and are ready to install your changes.  You should use the command `het_release ... -P branches/my_neat_branch -p all`
     137    * if you need to revert for some reason use the command `het_release ... -P tags/deployment/<last_good_tag> -p all`
     138
     139
    96140'''Decisions Required'''
    97   * decide whether or not to clean out the directory before installing new files
     141  * decide whether or not to clean out the directory before installing new files. This will require that no other files are in the directory.
    98142  * decide whether to split the build/installation of the camra code and the scripts from main tcs servers
    99143    * if so, create deployment directories for each, e.g. /opt/het/tcs, /opt/het/camra with ./bin, ./etc, ./lib as required