Classes and methods for interfacing with ltl::MArray.
LTL currently provides I/O interfaces for ltl::MArray to streams, columnar ASCII files and FITS files. Access to columnar ASCII files and to FITS files is done using the classes ltl::AscFile, ltl::FitsIn, and ltl::FitsOut. Access to streams is realized using ltl::operator>> and ltl::operator<<, in accordance with the standard library.
The class ltl::ExprBase<T,N> is interfaced to output streams and ltl::MArray<T,N> to input and output streams using the global operators
template<T,N>
ostream& operator<<( ostream& os, const ExprBase<T,N>& A );
template<T,N> istream& operator>>( istream& is, MArray<T,N>& A );
These are defined in the file <ltl/marray_io.h> which you will have to additionally include.
The output of operator<< is exactly what operator>> expects as input, so you can use the latter to read what you have written using the former. The rank has to be known a priori when reading from operator<<. The lengths and index ranges are read from the stream. The format generated is human readable and can also serve as pretty printing for MArrays. Each dimension is wrapped in brackets and indented.
For example:
#include <ltl/marray.h> #include <ltl/marray_io.h> MArray<float,2> A( Range(-3,3), Range(1,3) ); MArray<float,2> B; A = 1; ofstream os; os.open( "foo" ); os << A; ifstream is; is.open( "foo" ); is >> B; cout << B << endl;
And here's the programs output:
MArray<float,2> ( 7 x 3 ) : (-3,3) (1,3) [[ 1 1 1 1 1 1 1 ] [ 1 1 1 1 1 1 1 ] [ 1 1 1 1 1 1 1 ]]
You can also output the result of the evaluation of an MArray expression directly without using a temp[orary to store the result:
MArray<float,2> A(10), B(10); ... std::cout << A+sqrt(B)-1 << std::endl;
To map between type names and types, see the file <misc/type_name.h>, which provides type traits for this purpose. To add the string representation of a non-primitive type, use the macro LTL_TYPE_NAME:
struct mytype { ... }; LTL_TYPE_NAME(mytype, "my type name");
The class ltl::AscFile provides the interface to columnar ASCII files. It supports arbitrarily delimited columns of numbers, empty lines, comments, and reading a restricted range of lines and columns.
ltl::AscFile( string fname, char delim=0, string comment="#" ), ltl::AscFile( string fname, int firstLine, int lastLine=-1, char delim=0, string comment="#" ): Constructors of the class ltl::AscFile.
Columns in the file can be separated by arbitrary characters, to read whitespace delimited files, specify delim=0, the default, as delimiter. Otherwise set delim to the delimiter char.
Arbitrary strings may be used as comment delimiters. Using the second constructor, the firstLine is the first line to be read, counting from one, lastLine is the last line to be read, (-1 specifies reading until EOF, which is the default). When searching for the first line to be read, all nonempty and non-comment lines are counted, but they may contain arbitrary text, so they don't have to be columnar. When searching for the last line, only data rows are counted. The lines beyond the last line are not read at all.
int ltl::AscFile::rows(), int ltl::AscFile::cols()
Return the number of rows in the whole file, and the number of columns in the first data row, in the case that the complete file is read, or the number of data rows between firstLine and lastLine, i.e. lastLine - firstLine + 1, in case only a restricted range of lines is read. The columns are counted in the first line in the latter case.
MArray<int,1> ltl::AscFile::readIntColumn( col ), MArray<float,1> ltl::AscFile::readFloatColumn( col ), MArray<double,1> ltl::AscFile::readDoubleColumn( col )
Read a column out of the file, into an ltl::MArray, using either int, float, or double type.
AscFile File( "foo" );
MArray<float,1> A = File.readFloatColumn( 3 );
MArray<int,2> ltl::AscFile::readIntColumns( int first=1, int last=-1 ), MArray<double,2> ltl::AscFile::readFloatColumns( int first=1, int last=-1 ), MArray<double,2> ltl::AscFile::readDoubleColumns( int first=1, int last=-1 )
Read a range of columns into a 2-dimensional ltl::MArray. The default parameters read from the first to the last column, i.e.
MArray<float,2> A = a.readFloatColumns( 1, a.cols() );
and
MArray<float,2> A = a.readFloatColumns();
are identical.
For reading data into C-style arrays and STL containers, the following overloaded template methods are provided:
template<class T> int ltl::AscFile::readColumn( int col, T* &carray )
Read a single column into a C-style array. The memory is allocated using malloc.
template<class T> int ltl::AscFile::readColumn( int col, T& container )
Read a single column into an arbitrary container supporting a push_back function. In the STL these are vector, list, deque, queue, priority_queue, and stack.
To retrieve a commented header the ltl::AscFile::getHeader function is supplied.
int ltl::AscFile::getHeader( vector<string>& v, bool keepcs=false )
Retrieves the commented header from the file in the vector<string> v and return the number of lines read. The keepcs boolean controls whether the leading comment string is stripped from the returned strings.
| ltl::IOException | These methods throw an object of type ltl::IOException on error. This exception is derived from std::exception. |
The FITS files must conform to FITS definition NOST 100-2.0 with additional restrictions when syntax checking is on (see below).
EXTENSION keywords will be discarded. DATE will be reset with every writeHeader() call (i.e. when writing file). ORIGIN will be reset when writing, string can be set (see below). BLOCKED will discarded when writing EPOCH will be transformed to EQUINOX if not already present, and discarded when writing. BSCALE and BZERO will be applied when reading data, but ignored when writing; since FitsOut erases those cards in construction they have to be set and handled by the user if needed. BLANK is not applied when reading data, the user has to take care of it; it will be discarded when writing BITPIX < 0 (floating point) data . CTYPEn, CRPIXn, CRVAln, CDELTn, CROTAn are ignored when reading, n > naxis erased when writing.The header will be reformatted to NOST 100-2.0 recommendation: Mandatories first, Array keys, DATE, other reserved keys, other keys, COMMENT, HISTORY, (junk keys,) END. Junk keys are keywords with a questionable, but not completely wrong syntax (i.e. other than standard commentary keywords like HIERARCH, DATE keywords with wrong syntax.
To use LTL FITS I/O facilities include
#include <ltl/fitsio.h>
| ltl::FitsException | On errors a ltl::FitsException indicating the error is thrown. This exception is derived from std::exception. |
To read FITS files, use the class ltl::FitsIn. A short description follows:
ltl::FitsIn::FitsIn( string file, quiet=false, ignore_header=false );
Construct a ltl::FitsIn instance for reading from file named file. To turn off warnings on stderr one may set quiet = true.
ignore_header = true will cause all cards except the mandatory cards to be treated as junk cards, i.e. no syntax checking is performed. Caveat: BSCALE and BZERO will be ignored in this case!| ltl::FitsException | If the file is unreadable for whatever reason, a ltl::FitsException indicating the error is thrown. |
int ltl::FitsIn::getBitpix(); int ltl::FitsIn::getNaxis(); int ltl::FitsIn::getNaxis( const int axis );
Shortcuts to obtain the values of BITPIX, NAXIS, and NAXISn
double ltl::FitsIn::getBscale(); double ltl::FitsIn::getBzero();
Shortcuts to obtain the values of BSCALE and BZERO.
void ltl::FitsIn::describeSelf( ostream& os );
Print a short summary of the file to be read on given ostream: filename, BITPIX, NAXIS, and naxis parameters.
int ltl::FitsIn::getBytpix();
Gives the number of bytes per pixel.
off_t ltl::FitsIn::getDataOffset();
Gives the offset to the beginning of the data segment, i.e. the total length of the header in bytes.
off_t ltl::FitsIn::getDataLength();
Gives the number of bytes in the data segment.
Return a Region object displaying the NAXIS settings.
| ltl::FitsException | If a keyword cannot be found a ltl::FitsException is thrown. |
string ltl::FitsIn::adjustKeyword( string keyword );
Returns a keyword trimmed to 8 characters, which is required by all methods involving keywords.
string ltl::FitsIn::getString( string keyword ); bool ltl::FitsIn::getBool( string keyword ); long ltl::FitsIn::getInt( string keyword ); double ltl::FitsIn::getFloat( string keyword );
Get the value of a keyword.
bool isFixed( string keyword );
Returns true if the card is in fixed format.
string ltl::FitsIn::getComment( string keyword );
Retrieve the comment for the given keyword.
string ltl::FitsIn::getComment(); string ltl::FitsIn::getHistory();
Get all COMMENT or HISTORY lines as a string. The individual lines are separated by newline characters.
void ltl::FitsIn::setRegion( util::Region region );
Set the util::Region region of the file to be read.
void ltl::FitsIn::resetRegion();
Reset to full data.
ltl::FitsIn& operator>>( ltl::FitsIn& in, ltl::MArray<T,N>& A );
Read data from the active region in the FITS file in into the MArray A.
If the A has no memory allocated yet, the whole active region is read and new memory is allocated to hold the data. If A already points to a valid memory chunk, only as much data is read as will fit into A. The next call to operator>> will read the next chunk of data. Tiling is done along NAXIS1 first, then NAXIS2, etc., in a "rectangular" pattern. The user is responsible for choosing appropriate chunk sizes, such that the resulting tiling will cover the whole data.
Reading a whole FITS file into an MArray is, therefore, pretty easy:
MArray<float,2> A; // we will read a 2-dim image, BITPIX=-32 FitsIn FitsFile( "foo.fits" ) FitsFile >> A;
Internally, reading the data segment of a FITS file into MArrays is done by mapping the file into memory using mmap(). The mapping is done when the first read occurs. In order to free address space prior to destruction of the ltl::FitsIn object one can use
void ltl::FitsIn::freeData();
ltl::FitsOut::FitsOut( string filename, bool quiet=false ); ltl::FitsOut::FitsOut( string filename, FitsHeader& header, bool quiet=false );
Construct a ltl::FitsOut object, optionally copying the header information from header.
If no header is given, a default will be created, containing only mandatories. Those will be adjusted to reflect what is written when the actual writing is performed.
Note that ltl::FitsIn is derived from ltl::FitsHeader.
void ltl::FitsOut::setBitpixOut( int bitpix );
Set BITPIX explicitly. If you do this, type conversion to the type specified by BITPIX will be performed when writing. If you do not call this method, BITPIX will be set according to what go passed when actually performing the write operation, and no type conversion will be performed.
void ltl::FitsOut::setOrigin( string origin );
Set the ORIGIN keyword, defaults to "LTL FITS IO class".
void ltl::FitsOut::addComment( string comment );
void ltl::FitsOut::addHistory( string history );
Add comment or history text. The text will be automatically wrapped and added to the list of COMMENT or HISTORY keywords.
void ltl::FitsOut::addValueCard( string keyword, string value, string comment='''' ); void ltl::FitsOut::addValueCard( string keyword, bool value, string comment='''' ); void ltl::FitsOut::addValueCard( string keyword, int value, string comment='''' ); void ltl::FitsOut::addValueCard( string keyword, double value, string comment='''' );
Add a card with a value of given type to the header.
You may also use the methods for reading keywords described above in ltl::FitsIn.
void ltl::FitsOut::eraseCard( string keyword );
Remove card from header.
Actually writing data from an ltl::MArray in FITS format:
ltl::FitsOut& operator<<( ltl::FitsOut& out, ltl::MArray<T,N>& A );
Write data from A to FITS format file. If BITPIX has not been set explicitly using ltl::FitsOut::setBitpixOut(), it will be set according to the type T of the ltl::MArray, otherwise, data of A will be converted to the value specified by ltl::FitsOut::setBitpixOut().
The same options are possible as in reading using ltl::FitsIn, explained above.
Example writing an image to a FITS file:
MArray<float,2> Image( nx, ny );
Image = ... ;
FitsOut out( "foo.fits" );
out << Image;
For convenience, in case you want to copy (parts of) a FITS file and optionally only want to change header information, there is a version of operator<< using only ltl::FitsOut and ltl::FitsIn:
ltl::FitsOut& operator<<( ltl::FitsOut& out, ltl::FitsIn& in );
Copy data from in to out. Regions in in will be respected, so that portions of files can be copied.
For example:
FitsIn infile( "foo.fits" ); FitsOut outfile( "bar.fits", infile ); // infile argument copies header // immediately, so we can manipulate, // it is not necessary infile.setRegion( region ); // just optionally outfile.addHistory( "File copied by me ..." ); outfile << infile; // or infile >> outfile; // BSCALE and BZERO will be set according to the infile
You can use the FITS I/O facilities with C-style arrays, too, without using MArrays.
You may use any FITS I/O methods with ltl/fits.h instead of ltl/fitsio.h.
// include fitsio without LTL MArrays #include <fits.h> // get data arrays of type T and selected region T* infile.getDataArray( T dummy ); // read into container using an iterator T infile.readDataArray( T iterator ); // write from container outfile.setBscale( infile.getBscale() ); // optionally set outfile.setBzero( infile.getBzero() ); // optionally set outfile.openData( int newbitpix, class Region region ); // or int newnaxis_i [newnaxis] = { NAXIS1, ..., NAXIS_newnaxis }; outfile.openData( int newbitpix, int newnaxis, int * newnaxis_i ); // prepare for data write, // the NAXIS parameters maybe extracted from a Region or be set explicitly outfile.writeDataArray( T iterator ); outfile.closeData();
Additionally, you can perform I/O operations on a per pixel basis:
This is still under development. ltl::FitsIn::setRegion() is ignored for this set of methods. You can use setPosition() and getPosition() instead. No range checking is performed, you must prevent reading past EOF yourself.
// read pixel from stream: infile >> T value; // reset stream pointer to start of data infile.resetPosition(); // set stream pointer to position infile.setPosition( size_t offset ); // get offset from start of data ptrdiff_t offset = infile.getPosition(); // open output stream outfile.openData( int newbitpix, int newnaxis, int * newnaxis_i ); // write pixel to stream: outfile << T value; // reset stream pointer to start of data outfile.resetPosition(); // set stream pointer to position outfile.setPosition( size_t offset ); // get offset from start of data ptrdiff_t offset = outfile.getPosition(); // close output stream outfile.closeData();
1.6.1