Ntuple

class HepNtuple

Ntuple of blocks of (possibly heterogeneous) columns

Last update: 20 Sept. 2001

HepNtuple provides the flexibility of defining inhomogenous blocks with variables of different types, different packing strategies, and so forth. Its features are motivated by the HBook column-wise Ntuple.

An HepNtuple may be stored in row-wise or column-wise order. However, various managers may not support every storage strategy. For example, HBook does not support row-wise storage for inhomogeneous Ntuples.

Methods

HepObj Methods

Creating an Ntuple

Defining Properties of the Ntuple

Accumulating data

Retrieving Ntuple data

Accessing class information

Typical uses of this class

Example One

Methods

Creating an Ntuple

Defining Properties of the Ntuple

Defining blocks

Defining columns

Specifying storage strategies

Accumulating data

Retrieving Ntuple data

Accessing class information

Creating an Ntuple

The user does not directly instantiate a HepNtuple object. Instead, a reference (a HepNtuple&) is obtained by calling the ntuple() method in the manager.

These methods are not methods of HepNtuple but are documented here for convenience.

A fresh Ntuple:

HepFileManager:: ntuple(

const std::string& title,

int id = 0);

Both the title and the id must be unique within a directory. If the id is 0 (default) then the manager assigns a new unique id to the ntuple.

Merely creating an HepNtuple does not yet provide an object suitable for accumulating data.
Block and column definitions must be provided before data may be accumulated. |1|

For example, if m points to the manager, the user might do:


                tup = m->ntuple ( "Values" );

		// Now use various methods of HepNtuple class to define 
			properties 

                tup.capture (); 

                // and when finished -- 

                tup.release();

Create an HepNtuple refering to one existing in the manager (for retrieval access):

HepFileManager:: retrieveNtuple(const std::string& title); - - - - - |2|

Creating a new HepNtuple from a parent

Clone this HepNtuple, including all accumulated data. The clone is created in the current directory of the manager. The manager used in the first form is the same of the original ntuple.

HepNtuple & makeClone(const std::string& title, const int id = 0);

HepNtuple & makeClone(HepFileManager *mgr, const std::string& title, const int id = 0);

Create a fresh empty HepNtuple with the same characteristics as this HepNtuple. The clone is created in the current directory of the manager. The manager used in the first form is the same of the original ntuple.

HepNtuple & makeEmpty(const std::string& title, const int id = 0);

HepNtuple & makeEmpty(HepFileManager *mgr, const std::string& title, const int id = 0);

Top | Methods | Related Classes

Defining Properties of the Ntuple

These methods must be called to define the column and variable structure, before filling or retrieving Ntuple data. |1|

Defining Ntuple Blocks

Name a block, simultaneously defining its columns:

block (const std::string& blockName, const std::string& chform, vector void* data);

chform is a std::string providing information about the columns and data supplies the addresses of the variables in the block. |4|

Defining Properties of Ntuple Columns

These methods are alternatives to defining the entire block at one stroke via chform. Here, one defines and describes the properties of the blocks and columns on a column-by-column basis.

Define a column, by tag (name of column) and type:

column ( const std::string& tag,

const ColumnData_t value, ______________ // |5|

const ColumnData_t default_value);

In general, a tag is of the form BlockName::ColumnName. If a block name has already been defined, the ColumnName alone may be given and the BlockName will default to the "current" block. A ColumnName must be unique across the entire Ntuple except as noted in the following addendum.

As of 20 Sept. 2001, this uniqueness condition has been relaxed, but only for the Root manager. For the Root manager, duplicate column names may be used provided they appear in different blocks. In addition, there is an important consequence to the user associated with this added flexibility. Allowing duplicate column names introduces a potentially serious ambiguity at data capture time since column names alone no longer uniquely specify how to store the captured data. Therefore, when using this new feature, you must use the full tag (BlockName::ColumnName) in calls to any of the capture methods.

ColumnData_t specifies the data type, for example Float4_ct or Int2_ct. |5|
value is the designated value for data in this column and capture occurrs immediately. For deferred capture, see columnAt below. |6|

Define an array of columns: |8|

columnArray ( const std::string& tag, int ncols, ColumnData_t default_value );

This defines the number of dimensions of the array but does not specify their sizes. Data capture is deferred to allow the user to complete the definition of these columns.

Define a column by reference:

columnAt ( const std::string& tag, &variable, ColumnData_t default_value );

This defines the address of the variable to be accessed when capture is invoked. Note that variable may be a simple variable or the beginning of an array.

The remainder of these methods must refer to a previously defined column.

Define packing to compress storage for a column: |7|

columnPacking ( const std::string& tag, float rangeLo, float rangeHi, int nbits );

Associate an index variable with a column: |9|

index ( const std::string& tag, const std::string& indexTag );

An index variable allows the Ntuple to have a "variable-length" column, by treating the index variable as the number of entries in that column for this particular row.
The index variable applies to the slowest varying (first) array index. It must be another column in the same block, and must have a defined span (range of possible values).

Define the span of an index variable: |9|

span ( const std::string& tag, int min, int max );

Index variables are always packed in a lossless manner, using a number of bits determined from their span. Index variables may not have separately defined packing.

Define the extents of a column array: |9|

dimension ( const int dim1, dim2=0, ... dim10=0); ;

Only the first argument is required. All others may be omitted and default to zero. For a 4 by 7 array of columns, dim1 = 4, dim2 = 7 and all others are omitted. The number of arguments to dimension must match the value of ncols in the columnArray call.

Define a default value for unfilled data for a column: |10|

setColumnDefault ( const std::string& tag, ColumnData_t default );

The signature provides a measure of type safety; the float signature, for example, verifies that the defined column type is a Float4 or Float8 or Float16.
If this method is not called, the default value will be zero for integer and float types, and all blanks for character types.

Similarly, the current setting for the default value can be obtained via

columnDefault ( const std::string& tag, ColumnData_t* default );

Define a designated variable for a column which had been defined by chform: |3|

columnVariable ( const std::string& tag, void* variable );

This is used when it is conceivable that the compiler would provide empty space in a block structure to improve word alignment.

Specifying Storage Strategies for the Ntuple

Row-wise or Column-wise storage:

bool setRowWise( );

bool setColumnWise( );

bool setBlockWise( );

Returns true if the manager handles that choice.
Default is Column-Wise on any manager that is capable of handling Column-Wise Ntuples.

Block-wise storage is not supported by the HBook manager.

Disk or memory resident:

bool setMemResident( );

bool setDiskResident( );

bool setSharedMemory (const char* areaName);

Returns true if the manager handles that choice.
Default is DiskResident.

Buffer sizes:

void setNwbuff (const int& nwbuff);

void setBuffLimit(const int& nbytes);

nwbuff is expressed in number of words per column kept in the in-memory buffers.
buffLimit restricts the total buffer space used; this may force nwbuff to be below its stated value.
Defaults are 8K words per column up to 2 Mbytes for column-wise or 128K bytes for row-wise Ntuples.

Buffer allocation (for memory-resident and shared memory):

void setCircularBuff( );

void setDemandBuffers( );

Specifies whether to create a circular buffer in memory, dropping the oldest rows to make room for the newest, or to allocate another buffer of space any time you run out of room.
Default is DemandBuffers.

Top | Methods | Related Classes

Accumulating data - Filling the Ntuple

Data in the designated variables |11| may be captured a block at a time or the whole row at once.

Capture data for (all columns of) a single block of the Ntuple:

bool captureBlock ( const std::string& blockName );

This does not advance the Ntuple's row number. The package does not track a separate row number for each block.

Capture data for an entire a row of the Ntuple -- all blocks -- from the data in the designated variables. This does not advance the Ntuple's row number:

bool capture ( );

Capture individual items from the data in the designated variables. The item in question may be a single variable or a reference to an array according to what was declared for the corresponding column. Note that, for a column declared as an array, it is possible to use a standard library vector container. For a vector container named e. g., v in an ntuple object named tupl, one would write

tupl.capture( itemName, &(v[0]) ).

This does not advance the Ntuple's row number.

bool capture ( const std::string& nameTag, ColumnData_t* item );
or, for an indexed array

bool capture ( const std::string& nameTag, ColumnData_t* item, int indexVal );

Add captured data to the Ntuple, advancing the row number:

bool storeCapturedData ( );

Undo capture of data for this row (before it has been stored):

bool clearData ( );

bool clearDataBlock ( blockName );

void clearAllData ( );

When an Ntuple is part of a collection of Ntuples for each event, manager->clearAlliData() may be used to reject captured data for a whole event.

Append data coming from a different ntuple. The Block and columns must be of the same name and type in both ntuples.

HepNtuple & append ( const HepNtuple& from );

Top | Methods | Related Classes

Retrieving Ntuple data

Data may be retrieved by whole row, by block, by individual column, or by array of columns. In all cases, the retrieved data will be placed into the designated variables. |11|

Data retrieval is based on row number explicitly supplied by the user.

Retrieval of blocks and/or columns for Ntuples with Row-Wise storage is supported, but will not be much quicker than retrieving the entire row.

Retrieve the number of rows accumulated in the Ntuple:

int nrows( );

Retrieve one row of the Ntuple, into the designated variables:

int readRow(const int irow);

Retrieve one element (one column) of one row of the Ntuple, into its designated variable:

int readColumn(const int irow, const std::string& columnTag, ColumnData_t *value );

For an Ntuple stored row-wise, accessing a single element perforce reads in the entire row. For column-wise storage, reading the same colum of many sequential rows is very I/O-efficient.

Retrieve an array of columns from one row, into its designated variables: |8|

int readColumnArray ( const int irow, const std::string& columnTag, ColumnData_t value );

Retrieve one block for specified row of the Ntuple, into designated variables for that block:

int readBlock(const int irow, const std::string& blockName);

Bulk retrieval of column data

Data in an individual column may be retrieved multiple rows at once. The user is responsible for providing an area to place the retrieved data; the designated variable for that column is not applicable for these methods.

Retrieve data from a columns for a range of rows:

int bulkReadColumn ( const int irow, const int nrows, const std::string& columnTag, ColumnData_t *destination );

Top | Methods | Related Classes

Accessing class information

Title of the Ntuple

std::string title ( ) const;

Directory of the Ntuple (internal to file manager)

std::string dir ( ) const;

HepFileManager managing this Ntuple

HepFileManager * Manager ( ) const;

The id (assigned by the manager or user):

int id( ) const;

The type of the object:

char type() const;

Each class has a static data member named typeId containing its type indentification letter. Only full types (HepNtuple, HepHist1D, HepHist2D, HepHistProf and HepDir, for directories) have one.

For example HepNtuple::typeId

Storage Strategy:

bool isRowWise( ) const;

bool isColumnWise( ) const;

Disk or memory resident:

char resident( ) const;

Returns 'M' - memory resident --- 'D' - disk resident --- 'S' - shared memory resident

bool getSharedAreaName ( char* areaName, maxlen = 256 ) const;

(Not yet implemented)

Returns false if not in shared memory.

Buffer sizes and allocation:

int nwbuff( ) const;

int buffLimit( ) const;

bool isCircularBuff( ) const;

bool isDemandBuffers( ) const;

Block and Column Information

Some information may not be availabe from some managers, particularly for pre-existing HepNtuples.

Block information:

int nBlocks ( ) const;

std::string blockName ( const int blockNumber ) const;

blockNumber goes from 0 to nblocks-1.

Column information (See note on column tag and nametag:|12|):

int nColumns ( ) const;

Each column or array of columns counts as 1. This method returns the number of times the column() method would be called to define the contents of this ntuple.

int nColumns ( const std::string& blockName ) const;

Same as previous but restricted to one block.

std::string nametag ( const std::string& tag ) const;

std::string nametag ( const int columnNumber ) const;

columnNumber goes from 0 to nColumns()-1.

std::string nametag ( const int columnNumber, const std::string& blockName) ) const;

columnNumber goes from 0 to nColumns(blockName)-1.

std::string tag ( const int columnNumber ) const;

columnNumber goes from 0 to nColumns()-1.

std::string tag ( const int columnNumber, const std::string& blockName) ) const;

columnNumber goes from 0 to nColumns(blockName)-1.

Column properties:

columnDataType ( const std::string& columnTag )const;

std::string columnBlock ( const std::string& columnTag )const;

void* columnDesignatedVariable ( const std::string& columnTag )const;

Address of designated variable; 0 if no designated variable was set up for this column.

void* columnDestinationVariable ( const std::string& columnTag )const;

Address of destination variable; 0 if none was set up for this column.

void columnPacking ( const std::string& columnTag, float* low, float* high, int* bits )const;

int columnDimensions( const std::string& columnTag, int dims[])const;

std::string columnIndex (const std::string& nametag)const;

Returns std::string containing the columnTag of the index for the array. If no index was specified, or if the column is not an array, the empty std::string is returned.

bool columnSpan (const std::string& columnTag, int* min, int* max )const;

Return true if the tag refers to an index column, false (setting *min and *max to zero) otherwise.

bool columnPacking (const std::string& columnTag, float* low, float* high, int* inbits)const;

Return false if no packing has been set for this column.

Top | Methods | Related Classes