VHC provides a few extension hooks to allow to add custom drivers without need to edit the VHC code.
We provide these three access points:
For the sake of this documentation we assume that the custom code is implemented into a package with the following structure:
vhc_extra
├── setup.py # make the package installable
├── vhc_extra # the actual code
│ ├── __init__.py # this is a package
│ ├── common.py # module implementing the driver
│ ├── ref_file_parsers.py # module implementing new file parsers
│ └── hooks.py # vhc modifications
├── ...
...
Before we start, we need to let VHC know where the extra code, vhc_extra is.
We recommend to install the package, so that ends up in the standard path known by Python.
Alternatively you can do one of the following two:
put the path to your package into the PYTHONPATH before running vhc
export PYTHONPATH=/path/to/vhc_extra:$PYTHONPATH
add the path into the vhc_settings.cfg file
[extension] # list of comma separated paths to add to the python path extra_paths = /path/to/vhc_extra
vhc will add the extra paths, unless it’s already present, just after setting up the configuration, the logging objects and, if required, the multiprocessing capability.
We strongly suggest against adding vhc_extra/vhc_extra to the PYTHONPATH in order to avoid name clashes, e.g. between the common module in vhc_extra and libvhc.common
If you implement a new recipe, add it to the list of known recipes in the configuration file:
[recipes]
# comma separated list of recipes and of paths where the drivers are
# implemented. The `recipes_paths` are prepended to the `libvhc` and the system
# packages.
recipes = hetdex_dithers, flat, arc, bias, new_recipe
When it read the recipe file, vhc compare it with this list and abort if the recipe is not known.
If you add a new recipe and use one of the factories described in Create and use a drivers, you must add a section with at least the expected number of exposures; e.g.:
[new_recipe]
# expected number of exposures
n_exposures = 3
Let’s assume that in file vhc_extra/common.py you implement a driver do_something. If you cannot or do not want to use any of the factories described in Create and use a drivers, remember that driver must have the signature shown by libvhc.function_signature(); if this is not the case, the driver will fail. vhc will continue running but the failure will be notified in the v_results.txt, log.txt files and in the html recap.
To use the driver simply add the following line to the relevant v_driver.txt files:
vhc_extra.common:do_something
Once vhc will hit this line in the driver file, it will it will import the vhc_extra.common module get the do_something function and execute it.
If the module name is very long and/or you use various drivers from the package, you can shorten the driver name. If you have, let’s say, two drivers do_something, do_something_else in the module my_very_long_package.also_very_long.driver you can shorten the driver names in the v_driver.txt files to:
driver:do_something
driver:do_something_else
but you have to tell vhc where to look for them. You can do it adding the first part of the package name to the vhc_settings.cfg file:
[extension]
# list of comma separated python modules parents of the extra drivers
driver_parent_modules = my_very_long_package.also_very_long
At this point you ought to know how drivers are loaded and executed.
vhc looks for a v_driver.txt file in the input path: if it finds it, it reads all its content. Otherwise the files is created and filled with a default set of drivers for the given recipe (see libvhc.utils).
libvhc and all the modules provided by the configuration variable driver_parent_modules in the extension section are imported.
vhc cycles through the list of drivers. For each of them:
- split the module from the function name.
- it tries to import the module first as absolute, then as submodule of libvhc and of the driver_parent_modules; for any of the successful imports, it tries to get the function. If it can’t load the module at all, or can’t find the function in any of the loaded modules the driver is logged as a critical failure
- if it can find the function, vhc executes it: if this fails the driver is again logged as a critical failure.
What if your new driver needs to parse a reference file just like the one described in libvhc.reference_file_parser and you want to reuse all the machinery already implemented?
In your ref_file_parsers.py file subclass libvhc.reference_file_parser._BaseParser, or a derived one, and override the filename(). If you want it to be registered by vhc and accessible via picker() just do so. First mark the new parser for registration:
import libvhc.reference_file_parser as vhcrfp
@vhcrfp.register('parse_something')
class MyParser(vhcrfp._BaseParser):
def filename(self):
"""returns the file name from the configuration"""
return self.conf.get("extra_vhc", "some_reference")
def parse_value(self, str_value):
"""parse the extra columns at need"""
return str_value.split()
Then add the module name to the VHC configuration file.
[extension]
# list of comma separated python modules implementing extra reference file
# parsers. E.g.:
# mypackage.mymodule
reference_parser_extra = vhc_extra.ref_file_parsers
If you have done this you can get the initialised parser in your driver implementation calling vhcrfp.picker('parse_something') and obtain the relevant information with the method get_value().
There is a third, deeper way of customise VHC: the possibility to call hooks early in vhc.
Let’s suppose that you want to:
You can get it writing three functions in vhc_extra/hooks.py and tell vhc where to find them using the configuration file
[extension]
# list of comma separated modules and, optionally, callables, to be loaded and
# called after the configuration and logger objects have been setup. E.g.:
# mypackage.mymodule
# mypackage.mymodule:function, mypackage.mymodule2:func2
hooks = vhc_extra.hooks:load_conf, vhc_extra.hooks:new_recipe_pattern, vhc_extra.hooks:def_drivers
When loading and executing the hook functions, libvhc.loaders.execute_hooks(), vhc passes to them the configuration instance name and the logger instances.
Here is an example of the three functions:
import libvhc.config as vhcconf
import libvhc.utils as vhcutils
def load_conf(conf_name, logger):
"""load an extra configuration file
"""
extra_conf_file = "/path/to/conf.cfg"
conf = vhcconf.get_config(name=conf_name)
conf.read(extra_conf_file)
# put back the updated configuration instance
vhcconf._config_dic[conf_name] = conf
def new_recipe_pattern(*_):
"""insert a new recipe pattern match into vhc
We don't care about any of the input parameters
The new file name looks like::
20120301T000838_045LL_nrec.fits
"""
vhcutils.recipe_match['new_recipe'] = "*[0-9]*nrec.fits",
def def_drivers(conf_name, logger):
"""Create default drivers for our new recipe
"""
new_drivers = vhcutils.common_drivers
new_drivers += ['vhc_extra.common:do_something']
vhcutils.default_drivers['new_recipe'] = new_drivers
logger.info("new default drivers added")
Of course you can create an fourth function that calls them
def init(conf_name, logger):
load_conf(conf_name, logger)
new_recipe_pattern()
default_drivers(conf_name, logger)
and tell vhc about it
[extension]
hooks = vhc_extra.hooks:init
If your initialisation can all be done at the module level and doesn’t need any input parameter, you can also skip passing the function name. E.g.:
def new_recipe_pattern():
"""insert a new recipe pattern match into vhc
We don't care about any of the input parameters
The new file name looks like::
20120301T000838_045LL_nrec.fits
"""
vhcutils.recipe_match['new_recipe'] = "*[0-9]*nrec.fits",
def def_drivers():
"""Create default drivers for our new recipe
"""
new_drivers = vhcutils.common_drivers
new_drivers += ['vhc_extra.common:do_something']
vhcutils.default_drivers['new_recipe'] = new_drivers
new_recipe_pattern()
def_drivers()
[extension]
hooks = vhc_extra.hooks
This will load the module vhc_extra.hooks and, as side effect, execute the two functions.
If you want to do more you can, probably.
The vhc executable simply calls libvhc.vhc.main(). If you want to run some code before starting vhc, you can write a function that does what you need and call main(). E.g. in a entry_point.py module you can define:
import libvhc.vhc as vhc
def main(argv=None):
# do the setup that you need
[...]
# call vhc
vhc.main(argv=argv)
# do any cleanup if needed
[...]
Then add the entry point in your setup.py file to create a new vhc_extra executable when installing your package
entry_points={'console_scripts': ['vhc_extra = vhc_extra.entry_point:main']}
Enjoy your customisations!