libvdat – Stuff that isn’t GUI

Symlinking

Symlink raw files into a redux directory

vdat.libvdat.symlink.DATE_HEAD_KEY = u'{} {}'

Format string for the DATE-OBS and UT keys from the fits files

vdat.libvdat.symlink.FMT_HEAD_KEY = u'%Y-%m-%d %H:%M:%S.%f'

Format for converting DATE_HEAD_KEY into a datetime instance

vdat.libvdat.symlink.FMT_DATE_DIR = u'%Y%m%d_%H%M%S'

Format for converting a datetime instance into as string used as directory name

vdat.libvdat.symlink.symlink(log_message, redux_name=None)

Symlink shots in the directory ‘path’ to an output redux directory. Format of redux directory follows issue 820 <https://luna.mpe.mpg.de/redmine/issues/820>

redux_name is a unique name for this reduction. It is set to the current date and time if not specified. Each reduction puts its own folder called redux_name under.

Adds flt, sci, cmp, bias directories to config object after creation. In this way you can run the code without symlinking and just fill in the needed directories in the config file instead. After creating the required directories loops over all the fits files in that night and puts them in a sensible place.

Parameters:

log_message : string

Message to log as info before starting

redux_name : string

a name for this reduction, if not given generate it from the local time. This String has to be a valid name of a folder (XXX not currently tested)

vdat.libvdat.symlink._scan_dirs(redux_dir)

Scan the redux directories updating the ‘shot_name.txt’ file if the redux directory has changed and filling the database

Parameters:

redux_dir : string

name of the redux directory

vdat.libvdat.symlink._do_symlink(raw_dir, redux_dir)

Run the symlinking from the raw to the redux directory

Parameters:

raw_dir, redux_dir : string

name of the raw and the redux directory

vdat.libvdat.symlink._symlink_shot(shot_dir, redux_dir)

Create the redux directory for the shot and symlink all the files from the shot directory

Parameters:

shot_dir : string

name of the shot directory

redux_dir : string

name of the directory where the new directory and symlink must go

Raises:

:class:`~vdat.utilities.VDATFitsTypeError`

if the image type or object are not consistent or unknown

vdat.libvdat.symlink._symlink_sci(fits_files, vdat_dir)

Symlink the science shots

Parameters:

fits_files : list of strings

names of the fits files

vdat_dir : vdat.database.VDATDir instance

contains all the relevant information for the symlinking

Returns:

out_vdat_dir : vdat.database.VDATDir instance

database entry to save

vdat.libvdat.symlink._symlink_zro(fits_files, vdat_dir)

Symlink the bias shots.

Parameters:

fits_files : list of strings

names of the fits files

vdat_dir : vdat.database.VDATDir instance

contains all the relevant information for the symlinking

Returns:

out_vdat_dir : vdat.database.VDATDir instance

database entry to save

vdat.libvdat.symlink._symlink_cal(fits_files, vdat_dir)

Symlink the calibration, flat and arc, shots.

Flats and arcs taken together goes into the same directory.

1. If the shot is already symlinked, reuse the directory

2. If not look for directories with different original type or object,

   order them at increasing time distance and take the nearest one: if it’s within a maximum time distance, symlink into that directory

3. Otherwise create a new directory and symlink into it

Parameters:

fits_files : list of strings

names of the fits files

vdat_dir : vdat.database.VDATDir instance

contains all the relevant information for the symlinking

Returns:

out_vdat_dir : vdat.database.VDATDir instance

database entry to save

vdat.libvdat.symlink._mkdir(dirname, log, failsafe=True)

Create the directory.

If it exists, log it as error and, if failsafe is False, re-raise the exception

Parameters:

dirname : string

name of the directory to create

log : logging.Logging instance

log messages to this logger

safe : bool, optional

if true silently ignores OSError` due to existing directories

Raises:

:class:`~vdat.utilities.VDATDirError` if the creation fails with a

:class:`OSError` and ``failsafe`` is False

vdat.libvdat.symlink._symlink_file(file_list, target_dir, log, failsafe=True)

Symlink the files into the target directory.

If it exists, log it as error and, if failsafe is False, re-raise the exception

Parameters:

file_list : list of strings

names of the fits files to symlink

target_dir : string

name of the directory where to do the symlink

log : logging.Logging instance

log messages to this logger

safe : bool, optional

if true ignores OSError` due to existing files

Raises:

:class:`~vdat.utilities.VDATSymlinkError` if the symlink creation fails

with a :class:`OSError` and ``failsafe`` is False

vdat.libvdat.symlink._average_timestamps(dates, infmt=u'%Y-%m-%d %H:%M:%S.%f', outfmt=u'%Y%m%d_%H%M%S')

Average the list of timestamps.

Parameters:

dates : list of strings

strings containing timestamps

infmt : strings, optional

format of dates

outfmt : string, optional

format of the output time stamp

Returns:

avg_timestamp : datetime.datetime instance

average time

string

avg_timestamp formatted according to outfmt

Raises:

:class:`~vdat.utilities.VDATDateError`

if it fails to parse dates from the fits headers

vdat.libvdat.symlink._find_nearest(q, timestamp, n_nearest=1, nearest_then=None)

Go through the list of query results, order them according to the absolute distance from timestamp and return the n_nearest.

Parameters:

q : peewee.SelectQuery

query to use

timestamp : datetime instance

timestamp to use as reference

n_nearest : int, optional

maximum number of directories returned; set it to negative to return all

nearest_then : timedelta instance

if not None, don’t consider any directory whose delta time is larger than nearest_then; applied after n_nearest

Returns:

sorted_q : list of query results

ordered with respect to the timestamp

vdat.libvdat.symlink._db_create_references()

Logging

Deal with loggers

vdat.libvdat.loggers.setup_main_logger(conf, name=u'logger')

Setup the main vdat logger.

Add to the logger called name a standard output and a file logger

Parameters:

conf : ConfigParser instance

configuration options,

vdat.libvdat.loggers.make_logger(logger, conf=None, section=None)

Create a handler using the instructions in the configuration section and add it to the logger.

It looks for the following options:

• remove_old: if True remove all the previous handlers before adding the new one, default False
• remove_std: if True remove all the previous StreamHandler`, default False
• the ones used by make_handler()

Parameters:

logger : Logger instance

logger

conf : ConfigParser instance, optional

configuration options, if None use the default VDAT configuration object

section : string, default

name of the section containing the information to build the logger; defaults to logging.name, where name is the logger name

vdat.libvdat.loggers.astropy_handlers(conf)

Tweak the astropy logger removing the existing (stream) handlers and setting a new one if required

Parameters:

conf : ConfigParser instance

configuration options,

vdat.libvdat.loggers.ginga_handlers(conf)

Create/update the ginga logger removing existing handlers setting a new one if required

Parameters:

conf : ConfigParser instance

configuration options,

vdat.libvdat.loggers.cure_logger(name)

Get the logger with name. The first time this function is called with a name, the logger writing to a file in the log directory and called name.log is created

Parameters:

name : string

name of the logger

Returns:

logging.Logger instance

string

name of the file where the logs are saved

Todo

redo it when finishing the implementation of the command interpreter

vdat.libvdat.loggers.setup_command_loggers()

Set up the loggers for the commands connected or connectible to buttons

vdat.libvdat.loggers.conf_to_dict(conf, section, from_parent=[u'logdir', u'logger_level'])

Extract the section and convert it to a dictionary.

If from_parent is a non empty list, it will get the name of the parent section, e.g. if section=logging.main.stdout, the parent section is logging. Then if any element in from_parent doesn’t exist in section, it tries to get to from the parent section. If it doesn’t exist there, it’s skipped.

Parameters:

conf : ConfigParser instance

configuration options,

section : string

name of the section in the configuration where to find all the info

Returns:

dict

vdat.libvdat.loggers.make_handler(conf)

Create a handler following the instructions contained in section

It looks for the following options:

• enabled: if False, doesn’t create a logger and returns None; default False
• file_name: if not found, a StreamHandler is created, if exists, a FileHandler is created
• logdir: directory where the log file goes; default ‘.’
• format: formatting of the message, default %(levelname)s: %(message)s
• level: level of the handler, default INFO

Parameters:

conf : dict

configuration options for the handler; it can be created using conf_to_dict()

Returns:

None, if enabled is False

StreamHandler: if not file name is given

FileHandler: otherwise

vdat.libvdat.loggers._to_log_level(level, default=20)

Try to convert level to int or to extract the level level from the logging module. If both fail, return the default.

See here for more info about logging levels.

Parameters:

level : string

numerical or literal value of the level

Returns:

int

handler level

Background

Enqueue commands coming from users pushing buttons on the gui

class vdat.libvdat.callback.Worker(parent=None)

Generic worker that runs the given function

run(func)

Run the given function: it doesn’t have any argument

class vdat.libvdat.callback.Callback

Class to deal with the user pushing buttons in the GUI. Causes the buttons to add commands to a queue, specified by the set_queue command.

Attributes

queue

(vdat.gui.queue.Queue object) the queue the callback interacts with

queue = None

classmethod set_queue(queue)

Set the queue that the callbacks post jobs too.

Parameters:

queue : vdat.gui.queue Queue object

the queue the callback interacts with

static make_callback_function(name, alias)

Factory function, used to generate functions to attach to objects.

Parameters:

name : String

name of function to run on button click. Has to be in libvdat. Syntax is:

package_name:function_name

alias : String

a label for the job on the queue GUI.

immediate : bool (optional)

bypass the usual queue and run immediately