// HepTuple header/methods sketch #ifdef COMMENT Before discussing the HepTuple class, I want to say something about relevant methods in the HepTupleManager class, for makein instances of HepTuple*. I would have put this info in a cleaner form but have run out of time for the 1 week. Say the manager* is M. To create a HepTuple*, you do M->newHepTuple(title, id=0). There is a naming quiestion or two here; should we say newNtuple instead? Should we punt our "new" philosophy (which has mutated a bit anyway) in favor of M->ntuple? Since this is in the "creation" part we don't HAVE to preserve the name but... OPEN ISSUE At any rate, we do have an answer to how you attach to existing HepTuples, and this coordinates with the same issue in Histograms: Create a new HepTuple by M->newHepTuple(title, id=0). Attach to an existing one by M->oldHepTuple(title) or M->oldHepTuple(id) We must comment oldHepTuple (and oldHist1D etc) as follows: Make a new pointer to an **existing HepTuple (or Hist1D...) in the manager that has this supplied title. The user will be responsible for deleting this HepTuple* when through with it, or the memory assignment will persist till the end of the job. There are ways of finding out which Ntuples the manager has in this directory. So it is right to make a clash (new with same name or id, old based on one we can't find) either an error or return null pointer. We have no way to prevent constructs like HepTuple* h1 = M->newHepTuple(title, id=0). HepTuple* h2 = h1 but we don't encourage that. We DON'T allow attaching to an Ntuple where you already have attached to it. Our manager WILL track pointers it knows about and, if the user deletes the manager, they will be marked as Unmanaged so that attempts to use them will fail in a predictable way. Copying pinters defeats this safety feature. #endif /* COMMENT */ // THE REAL CONTENT: HepTuple class //---------------------------------- #ifdef COMMENT In this preliminary version of the header for HepTuple we omit the various const declarations the virtual keywords are omitted the code guards the "pragma interface" any needed includes Also, we don't know yet how we will want to handle colums of characters (in HBOOK they have 8 types, CHAR4 thru CHAR32). This decision won't seriously affect the rest of the interface. #endif /* COMMENT */ class HepTuple { // --- Original HepTuple form of column definition/filling // --- Other ways of defining columns // --- Refining column properties // --- Defining blocks // --- Storage Strategy // --- How to fill the histogram // --- Clearing captured data // --- Retrieving Ntuple data // --- Establishing destination variables // --- Accessing information about the HepTuple structure // --- Data types other than float #ifdef COMMENT Methods summary (Only the float forms of Tuple data are shown in this summary) NOTE THIS CHANGE: Methods are HepTuple* for syntax trick, not HepTuple. I have accidently put the latter. bool column (char nametag[], float value, float defval=0); /*H*/ bool newColumn (const char* lbl, float val, float defval=0); /*H*/ HepTuple columnAt (char nametag[], float* address, float defval=0); HepTuple columnVia (char nametag[], float(*accessMethod)(),float defval=0); HepTuple columnArray (char nametag[], int ncolumns, float defval=0); HepTuple columnDirect (char nametag[], float defval=0); HepTuple pack ( float min, float max, int nbits); HepTuple pack (char nametag[], float min, float max, int nbits); HepTuple span ( int min, int max); HepTuple span (char nametag[], int min, int max); HepTuple dimension (char nametag[], int dim); HepTuple dimension ( int dim); HepTuple dimension (char nametag[], int dim1, int dim2, int dim3=0, int dim4=0, int dim5=0, int dim6=0, int dim7=0, int dim8=0, int dim9=0, int dim10=0); HepTuple dimension ( int dim1, int dim2, int dim3=0, int dim4=0, int dim5=0, int dim6=0, int dim7=0, int dim8=0, int dim9=0, int dim10=0); HepTuple index (char nametag[], char indextag[]); HepTuple index ( char indextag[]); HepTuple characters (char nametag[], int length); HepTuple characters ( int length); bool block (char blockName[], char format[], HepClist addresses); bool setRowWise (); bool setColumnWise (); bool setBlockWise (); bool setDiskResident (); bool setMemResident (); bool setSharedMemory (char sharedMemoryArea[]); void setNwbuff (int nwbuff); void setBuffLimit (int nbytes); void setCircularBuffer (int nwbuff); void setDemandBuffers (int nbytes); bool captureThenStore ( ); bool capture (char nametag[], float value); int setExistingColumn (const char *lbl, float val); /*H*/ bool capture (char nametag[], float value[]); bool capture (char nametag[], float value[], int indexval); bool captureColumn (char nametag[]); bool captureBlock (char blockName[]); bool capture ( ); bool storeCapturedData ( ); bool clearData ( ); \*H*\ CHANGE 7/10 bool clearDataBlock (char blockName[]); bool clearDataCloumn (char nametag[]); int readColumn (int irow, char nametag[], float* value); int readColumnArray (int irow, char nametag[], float values[]); int readBlock (int irow, char blockName[]); int readRow (int irow); int bulkReadColumn (const int irow, const int nrows, const char nametag[], float destination[]); HepTuple destinationAt (char nametag[], float* address); HepTuple destinationVia (char nametag[], void (* setMethod) (float)); HepTuple destinationVia (char nametag[], void (* setMethod) (float[])); bool blockDestinations (char blockName[], char format[], HepClist addresses); HepString title ( ); HepString directory ( ); HepTupleManager manager ( ); int id ( ); bool isRowWise ( ); bool isColumnWise ( ); bool isBlockWise ( ); char resident ( ); HepString sharedAreaName ( ); int nwbuff ( ); int BuffLimit ( ); bool isCircularBuff ( ); bool isDemandBuffers ( ); int nBlocks ( ); HepString blockName (int blockNumber); HepString blockFormat (int blockNumber); HepString blockFormat (char blockName[]); int nColumns ( ); int nColumns (char blockname[]); HepString nametag (int columnNumber); HepString nametag (int columnNumber, char blockname[]); HepString tag (int columnNumber); HepString tag (int columnNumber, char blockname[]); HepString columnBlock (char tag[]); HepString columnNametag (char tag[]); type_info columnType_info (char nametag[]); ColumnData_t columnDataType (char nametag[]); int columnDimensions (const char nametag[], int dims[] ); HepString columnIndex (const char nametag[]); bool columnDefault (const char nametag[], float* default); bool columnSpan (const char nametag[], int* min, int* max); void* columnDesignatedVariable (char nametag[]); void* columnDestinationVariable (char nametag[]); bool columnPacking (char nametag[], float* low, float* high, int* inbits); #endif /* COMMENT */ public: // For routines which take a float or float* refering to a datum, // there are analogous routines for int, short, long, and double. // These require no separate explanation, and are grouped together // later in the class definition. // - - - - - // --- Original HepTuple form of column definition/filling --- // We indicate routines present in the original HepTUple with /*H*/ //************ bool column (char nametag[], float value, float defval=0); /*H*/ bool newColumn (const char* lbl, float val, float defval=0); /*H*/ //************ // This creates if not yet there and at the same time fills it. // (The newColumn form failes if the column already exists.) // Type checking at runtime happens, if you create via float and // later fill Via int or whatever. // We call this the Direct form; it is an amalgam of creation and filling. // You cannot mix the designated variable form with the column. // nametag is described in **[1]** // This form, which supplies a value, might routinely be used for filling // data. Therefore it ought not to return anything costly; it returns a // bool indicating sucess (for instance, new columns may have been locked // out, in which case it fails but we shouldn't destroy the job). // - - - - - // --- Other ways of defining columns: --- // All other column-definition methods return HepTuple* this, // so that a convenient syntax can be used. See **[2]** // They will return null pointer if for some reason you cannot do the // requested action. //***************** HepTuple columnAt (char nametag[], float* address, float defval=0); //***************** // Designated Variable form. See **[7]** // You cannot mix this with the defintion of a whole block including this tag, // or with the Direct form. //****************** HepTuple columnVia (char nametag[], float(*accessMethod)(),float defval=0); //****************** // Designated Accessor form. For objects which avoid public data members. // An example is needed here: See **[3]** //******************** HepTuple columnArray (char nametag[], int ncolumns, float defval=0); //******************** // No-Designated-Variable form. // Intended for a 1-D array which will be filled by Direct capture only. // To use this method to define column arrays of non-float data, see **[15]** //********************* HepTuple columnDirect (char nametag[], float defval=0); //********************* // No-Designated-Variable form. // Intended for a single column which will be filled by Direct capture only. // To use this method to define a column of non-float data, see **[15]** // - - - - - // --- Refining column properties --- // The following methods allow the user to refine the column definition. // For syntatic brevity, they also support a signature omitting the nametag // (assuming the previous one). See **[2]** //************* HepTuple pack ( float min, float max, int nbits); HepTuple pack (char nametag[], float min, float max, int nbits); //************* // Pack the floats in a lossy manner, dividing the range into 2**nbits // possible n-bit integers. Even if the column data type is not float, // the range is expressed as floats. //************* HepTuple span ( int min, int max); HepTuple span (char nametag[], int min, int max); //************* // This column becomes eligible to use as an index (see **[4]**) for // other columns. The maximum supplied delimits how many columns are // defined based on the index. An index column is packed in a lossless // manner, assuming all values lie in the defined span. //****************** HepTuple dimension (char nametag[], int dim); HepTuple dimension ( int dim); //****************** // A one-dimensional array of columns. These will end up with // tags derived from nametag. For example, if the tag is "XM" // and dim is 100, the tags made by HBook would be XM(0), XM(1), // ... XM(99). //****************** HepTuple dimension (char nametag[], int dim1, int dim2, int dim3=0, int dim4=0, int dim5=0, int dim6=0, int dim7=0, int dim8=0, int dim9=0, int dim10=0); HepTuple dimension ( int dim1, int dim2, int dim3=0, int dim4=0, int dim5=0, int dim6=0, int dim7=0, int dim8=0, int dim9=0, int dim10=0); //****************** // An N-dimensional array of columns -- up to 10 dimensions is supported. // Indices in each dimension are 0-based. The data is of course supplied // in a C "array" which implies a first-index-varies-slowest" storage. // The underlying manager may make column tags reflecting its own idea // of how an array is labeled. Our policy is to set up the dimensions // such that The C++ array ends up as an array of columns of that same // shape -- see **[5]** //************** HepTuple index (char nametag[], char indextag[]); HepTuple index ( char indextag[]); //************** // Associates an index variable to an array of columns; this is how // "variable length arrays" are supported. Of course, depending on // what storage mode is chosen, the underlying manager may or may not // yield space savings when an index does not have its maximum value. // indextag must refer to a column which is or will be set up as an index -- // it has (or will have) a defined span. The column being refined must // have defined dimensions. The index variable applies to the first // slowest varying) dimension and its span replaces the stated extent // of that dimension. //******************* HepTuple characters (char nametag[], int length); HepTuple characters ( int length); //******************* // Define a fixed-length character string column. // When the type of column is char, a column holding single-byte integer // data is formed. An array of these may be formed in the usual way by // the dimensions() method. // However, sometimes you may want instead a character string in the // HepTuple. This can be delared by the charcters() method, supplying // a fixed length for the string. The semantics for these are quite // different from the semantics for an array of floats or ints. A // particular restriction in the HBOOK implementtion is that it must be // 4*K where K is between 1 and 8. See **[13]**. // - - - - - // --- Defining blocks --- // Here is how to define a whole block at once. The HBOOK chform is used. //********** bool block (char blockName[], char format[], HepClist addresses); //********** // The designated variables for capturing data are specified by the // addresses. These need not be contiguous in memory, though typical // usage might have See **[7]** // An awkward point is that the addresses must match the variable // definitions in format. That is, if format starts with IMAX, XMAX // and the first two addresses supplied are those of a pair of floats, // there will be no notice taken of the fact that you are storing the // first float as an int. Better safety can be achieved by doing // columnAt() for each column. // There is, by the way, to be an extension to the format so that // if the column name is followed by empty parens, this specifies // the "address" supplied is that of an accessor function. Again // see **[3]**. // - - - - - // --- Storage Strategy // These calls must be made before one starts to fill the HepTuple. // The default is a Column-wise Disk-resident Ntuple with 8K-entry buffers. // However, if the manager does not support Column-Wise (or for that matter // disk-resident) storage, other supported defaults will apply. //****************** bool setRowWise (); bool setColumnWise (); bool setBlockWise (); //****************** // Choose either row-wise (an entire row is contiguous) column-wise // (each column is buffered in memory such that a large chunk of data // for a single column is contiguous) or -- if the underlying manager // supports it (HBOOK does not) bock-wise storage. // Returns false if the manager does not support this mode. //******************** bool setDiskResident (); bool setMemResident (); bool setSharedMemory (char sharedMemoryArea[]); //******************** // Return false if the manager does not support this mode. //******************** void setNwbuff (int nwbuff); void setBuffLimit (int nbytes); //******************** // Set the number of entries per column to be kept in the in-memory buffers. // setBuffLimit() sets a limit on totoal buffer space used, which may cause // the buffer sizes to be LESS that those implied by nwbuff. Default is // 8K entries per column for column-wise Ntuples, or the greater of 128K // bytes and the size of a full row for row-wise. //******************** void setCircularBuffer (int nwbuff); void setDemandBuffers (int nbytes); //******************** // For in-memory or shared-memory HepTuples, specifies whether to create a // circular buffer in memory, dropping the oldest rows to make room for the // newest, or to allowcate another buffer of space any time you run out of // room. Default is demand buffers. // - - - - - // --- How to fill the histogram --- // Capture of data, and storing captured data for a row and moving to next // row, is handled by the following methods. Some semantics are explained // in note **[6]**, and are summarized here. // The act of supplying data for one or more columns is called // capturing that data. The act of adding the captured data for // a row to the Ntuple is called storing the data. // The row number semantics for storing data is that of automatic // append: Store increments the row, and the user does not explicitly // specify the row. // Capture of data can be done on a per-column basis. Another // option is to capture data for an entire block or row by supplying // designated variables. These are explained in note **[7]**. // The definition of the columns composing the HepTuple may be extended // after some data has been captured, and even after rows of data have been // stored. In particular, the use of the column(name, val) method without // a specified block name can be done at any time. Rules for other cases // are described in **[6]**. // We do not permit data to be captured twice via multiple-column // methods or by single column. Capture methods that go over multiple // columns (block or row) fill only the ones that were not filled for // this row. Users can overwrite a filled column by clearing it first. //********************* bool captureThenStore ( ); //********************* // Capture data for all designated variables **[7]** in the entire row // THAT HAVE NOT ALREADY BEEN CAPTURED, // then store the data in the Ntuple, advancing to the next row. // This is equivalent to the sequence capture(); storeCapturedData(); // and is the convenient way of filling a row. //************ bool capture (char nametag[], float value); int setExistingColumn (const char *lbl, float val); /*H*/ //************ // Capture data for the one column specified. The column must have // previously been defined. Assuming it has been, this is equivalent // in effect to calling column() with the same arguments. We call this // "Direct" capture since the value is supplied rather than found in a // designated variable. CHANGE 7/10 // setExistingColumn(), per original HepTuple interface, will never ZMthrow, // returning 0 if the column is not known. capture(), on the other hand, will // ZMthrow the appropriate exception if a problem arises, returning false // if the capture could not happen. CHANGE 7/10 // See **[16]** //************ bool capture (char nametag[], float value[]); bool capture (char nametag[], float value[], int indexval); //************ // Capture data for an array of columns. The nametag must match that // of an array of columns which had been defined. This is a way to do // "Direct" capture, if you wish not to capture the entire block. // The value[] array should match in size the size defined for this array // of columns; if an indexval is supplied, only that number of values for // the slowest-varying dimension are captured. // See **[14]** //****************** bool captureColumn (char nametag[]); //****************** // Capture data for one column entire block, via its designated variable // (or access method). See **[7]** and **[3]**. CHANGE 7/7 //***************** bool captureBlock (char blockName[]); //***************** // Capture data for an entire block, via its designated variables **[7]**. CHANGE 6/25 // Not permitted if the block was already captured for this row; permitted if // some columns were already captured--these will be skipped now. //************ bool capture ( ); //************ // Capture data for all designated variables in the entire row. **[7]** //********************** bool storeCapturedData ( ); //********************** // Store the data previously captured for this row of the Ntuple, advancing // to the next row. Captured data may include data capture both by Direct // calls and by designated variables. This **replaces** the method in // HepTuple named dumpData(). // - - - - - // --- Clearing captured data --- // These are analogous to the capture routines, but used when rejecting // data that had tentatively been captured. //************** bool clearData ( ); \*H*\ //************** // Clear previously captured data for this row: all columns of all blocks. CHANGE 7/10/97 //******************* bool clearDataBlock (char blockName[]); //******************* // Clear previously captured data for this row: columns in specified block. CHANGE 7/10/97 //******************** bool clearDataColumn (char blockName[]); //******************** // Clear previously captured data for this row: columns in specified block. // - - - - - // --- Retrieving Ntuple data --- // The Ntuple must have been attached to using the manager method // oldNtuple (or must be an active one created via newNtuple). // Direct-style access to individual columns requires no other // preparations; access to entire blocks or rows using designated // destinations, will require preparation described below. // Retrieval methods have explicit row specification semantics; the user // must provide the row number desired (zero-based) and inclrement it // explicitly if he wants to loop over all the rows. // Read methods return the number of columns, blocks, or rows read. Since // zero is a valid return value, negative ints will be returned to signify // erroneous operation (e.g. supplying an invalid nametag[].) **[8]** //********* int nrows ( ) //********* // Returns the number of rows of data stored for the Ntuple. //************** int readColumn (int irow, char nametag[], float* value); //************** // Direct read of the value of a column, into *value. In the case of an // array of columns the nametag[] should include the specification of which // element was wanted, e.g. "mycols(3,5)". //******************** int readColumnArray (int irow, char nametag[], float values[]); //******************** // Direct read of the value of a column, into an array at values[]. // The nametag[] should be that of the array of columns. // The user must provide a large enough array at value[] to fit the array // of columns. // The method returns the number of data elements actually captured which, // in the case of indexed arrays, will be the value of its index variable // (and may be less than the maximum number of columns possible). //************** int readBlock (int irow, char blockName[]); //************** // Read contents of specified block for this row, into the destination // variables established. All destination variables pertaining to this block // will be filled; this need not be the whole block as written to the Ntuple // but must include at least one variable. // Will return the number of columns filled. //*********** int readRow (int irow); //*********** // Read contents of specified block for this row, into the destination // variables established. All destination variables established // will be filled; this need not be the whole row but must include at // least one variable. // Will return numbr of BLOCKS filled. //****************** int bulkReadColumn (const int irow, const int nrows, const char nametag[], float destination[]); //****************** // Retrieve data from one column (which may be one element of an array of // columns) for a whole range of rows. Data not captured for a row will // be reflected by the default value being placed in that slot of the // destination array. // Bulk reads of a column may be particularly efficient for column-wise // storage strategies, and gain no practical efficiency for row-wise. // The destination must be an array big enough to hold nrows floats. // The return value is the number of values placed; normally nrows, unless // the end of the Ntuple is reached before getting nrows rows. // - - - - - // --- Establishing destination variables --- // A destination variable is to reading in Ntuple rows what a designated // variable is to capturing rows: Except for Direct forms, data read in // is placed in the destination variables. Note that for an Ntuple which // is both being written and read back (not necessarily a recommended // practice but certainly permitted), the destination variables may be // the same or distinct from the designated variables for capture. // Destination definition methods return HepTuple this, so that the // convenient syntax for refining column definitions can be used. // In general, the destination definition has to match the column // definition used when the column was stored, but the match is tolerant // of differences such as not specifying packing for read back. See **[9]**. // These methods will return null pointer if for some reason you cannot do // the requested action. //********************** HepTuple destinationAt (char nametag[]; float* address); //********************** // Destination for one column OR array of columns. // For an array of columns, you need not refine the column definition using // the method dimension(), since this information is available from the // existing HepTuple. However, in this case address had better point to // an array of floats, big enough to hold the column array. Methods // are available to check the dimensions of a column array. // For a single column within an array, the nametag form of "nametag(3,4)" // is supported. //*********************** HepTuple destinationVia (char nametag[], void (* setMethod) (float)); HepTuple destinationVia (char nametag[], void (* setMethod) (float[])); //*********************** // Accessor form for destinations. For destination objects which avoid // public data members, requiring calling a method to set the desired data. // See **[10]** //********************** bool blockDestinations (char blockName[], char format[], HepClist addresses); //********************** // Set up destinations for a whole block at a time. See **[9]** for // information on which aspects of format must match the original definition. // - - - - - // --- Accessing information about the HepTuple structure --- // These routines provide information about how the columns and blocks were // defined, and so forth, as well as general information about this instance // of HepTuple. They do not provide information that would change when each // row of data is stored; the retrieval routines do that. //*********************** HepString title ( ); HepString directory ( ); HepTupleManager manager ( ); int id ( ); bool isRowWise ( ); bool isColumnWise ( ); bool isBlockWise ( ); char resident ( ); // **[11]** HepString sharedAreaName ( ); // **[11]** int nwbuff ( ); int BuffLimit ( ); bool isCircularBuff ( ); // **[11]** bool isDemandBuffers ( ); // **[11]** //*********************** // - - - // General information //******************* int nBlocks ( ); HepString blockName (int blockNumber); //******************* // Find the names of each of the blocks. The semantics is that the user // can learn how many blocks there are, and then find the name of each in // turn, forming whatever sort of list or array is prefered. //********************* HepString blockFormat (int blockNumber); HepString blockFormat (char blockName[]); //********************* // Return the format string for the columns in a block. This might not // look the same as the string used to define the block because it includes // both those columns, followed by any columns in that same block that were // Directly created by the column() method. // - - - // Information about columns //************ int nColumns ( ); int nColumns (char blockname[]); //************ // Number of columns in the whole HepTuple or in a block. A defined array // of columns counts one here, as the purpose is to support looping over // all the column defintions to get information. //***************** HepString nametag (int columnNumber); HepString nametag (int columnNumber, char blockname[]); HepString tag (int columnNumber); HepString tag (int columnNumber, char blockname[]); //***************** // Obtain the nametag (which can include blockname::tag), or just the column // tag, of the n-th column in the HepTUple, or the n-th column in a block. // The blockname can be blank (or empty) to find out about columns defined // by column(), with no block information supplied in their nametag. //*********************** HepString columnBlock (char tag[]); HepString columnNametag (char tag[]); //*********************** // Find out what block a column is in (meaningful because tags must // be unique). // - - - // The following methods provide information about a column specified by // its nametag. //*************************** type_info columnType_info (char nametag[]); ColumnData_t columnDataType (char nametag[]); //*************************** // Returns the type of variable in each column. On fully compliant compilers, // the standard type_info mechanism is supported used. For the convenience of // users with not-yet-standard compliers, the header provides a ColumnData_t // enum. **[12]** //******************** int columnDimensions (const char nametag[], int dims[] ); //******************** // Returns number of dimensions, zero if this was not defined as an array of // columns. The extents in each dimension are put into dims[]; if an // index variable is used, dims[0] will be the maximum value of that. // dims[] should be an array of length 10, in case a 10-dimensional // column array (the most we support) was established. //********************* HepString columnIndex (const char nametag[]); //********************* // Returns the nametag of the index column established for this array of // columns. Returns an empty HepString if the column is not an indexed array. //****************** bool columnDefault (const char nametag[], float* default); //****************** // Provides the default value for an un-captured column. There are other // signatures taking int*, long*, short*, double*, bool*, and HepString*. // They return false if the wrong type of pointer was provided for this // column type. //*************** bool columnSpan (const char nametag[], int* min, int* max); //*************** // Returns false if this column was not set up as an index column, with // a defined span. // - - - // The following are provided for completeness, though generally the user // either has established the information which he now seeks, or would not // normally care what the answer might be. //******************************* void* columnDesignatedVariable (char nametag[]); void* columnDestinationVariable (char nametag[]); //******************************* // Will return 0 if designated or destination variable was not established. //****************** bool columnPacking (char nametag[], float* low, float* high, int* inbits); //****************** // False if no packing was specified. // - - - - - // --- Data types other than float --- // For routines which take a float or float* refering to a datum, // there are analogous routines for int, short, long, and double. bool column (char nametag[], double value, double defval=0); bool column (char nametag[], int value, int defval=0); bool column (char nametag[], long value, long defval=0); bool column (char nametag[], short value, short defval=0); bool column (char nametag[], char value, char defval=0); bool column (char nametag[], bool value, bool defval=0); bool newColumn (char nametag[], double value, double defval=0); bool newColumn (char nametag[], int value, int defval=0); bool newColumn (char nametag[], long value, long defval=0); bool newColumn (char nametag[], short value, short defval=0); bool newColumn (char nametag[], char value, char defval=0); bool newColumn (char nametag[], bool value, bool defval=0); bool columnAt (char nametag[], double* address, double defval=0); bool columnAt (char nametag[], int* address, int defval=0); bool columnAt (char nametag[], long* address, long defval=0); bool columnAt (char nametag[], short* address, short defval=0); bool columnAt (char nametag[], char* address, char defval=0); bool columnAt (char nametag[], bool* address, bool defval=0); bool columnVia (char nametag[], double (*accessMethod(), double defval=0); bool columnVia (char nametag[], int (*accessMethod(), int defval=0); bool columnVia (char nametag[], long (*accessMethod(), long defval=0); bool columnVia (char nametag[], short (*accessMethod(), short defval=0); bool columnVia (char nametag[], char (*accessMethod(), char defval=0); bool columnVia (char nametag[], bool (*accessMethod(), bool defval=0); bool columnArray (char nametag[], int ncolumns, double defval); bool columnArray (char nametag[], int ncolumns, int defval); bool columnArray (char nametag[], int ncolumns, long defval); bool columnArray (char nametag[], int ncolumns, short defval); bool columnArray (char nametag[], int ncolumns, char defval); bool columnArray (char nametag[], int ncolumns, bool defval); bool columnDirect (char nametag[], double defval); bool columnDirect (char nametag[], int defval); bool columnDirect (char nametag[], long defval); bool columnDirect (char nametag[], short defval); bool columnDirect (char nametag[], char defval); bool columnDirect (char nametag[], bool defval); bool capture (char nametag[], double value); bool capture (char nametag[], int value); bool capture (char nametag[], long value); bool capture (char nametag[], short value); bool capture (char nametag[], char value); bool capture (char nametag[], bool value); bool capture (char nametag[], double value[]); bool capture (char nametag[], int value[]); bool capture (char nametag[], long value[]); bool capture (char nametag[], short value[]); bool capture (char nametag[], char value[]); bool capture (char nametag[], bool value[]); bool capture (char nametag[], double value[], int indexval); bool capture (char nametag[], int value[], int indexval); bool capture (char nametag[], long value[], int indexval); bool capture (char nametag[], short value[], int indexval); bool capture (char nametag[], char value[], int indexval); bool capture (char nametag[], bool value[], int indexval); bool readColumn (irow, char nametag[], double* value); bool readColumn (irow, char nametag[], int* value); bool readColumn (irow, char nametag[], long* value); bool readColumn (irow, char nametag[], short* value); bool readColumn (irow, char nametag[], char* value); bool readColumn (irow, char nametag[], bool* value); bool readColumnArray (irow, char nametag[], double values[]); bool readColumnArray (irow, char nametag[], int values[]); bool readColumnArray (irow, char nametag[], long values[]); bool readColumnArray (irow, char nametag[], short values[]); bool readColumnArray (irow, char nametag[], char values[]); bool readColumnArray (irow, char nametag[], bool values[]); int bulkReadColumn (const int irow, const int nrows, const char nametag[], double destination[]); int bulkReadColumn (const int irow, const int nrows, const char nametag[], int destination[]); int bulkReadColumn (const int irow, const int nrows, const char nametag[], long destination[]); int bulkReadColumn (const int irow, const int nrows, const char nametag[], short destination[]); int bulkReadColumn (const int irow, const int nrows, const char nametag[], char destination[]); int bulkReadColumn (const int irow, const int nrows, const char nametag[], bool destination[]); HepTuple destinationAt (irow, char nametag[], double* address); HepTuple destinationAt (irow, char nametag[], int* address); HepTuple destinationAt (irow, char nametag[], long* address); HepTuple destinationAt (irow, char nametag[], short* address); HepTuple destinationAt (irow, char nametag[], char* address); HepTuple destinationAt (irow, char nametag[], bool* address); HepTuple destinationVia (irow, char nametag[], void (* setMethod) (double)); HepTuple destinationVia (irow, char nametag[], void (* setMethod) (int)); HepTuple destinationVia (irow, char nametag[], void (* setMethod) (long)); HepTuple destinationVia (irow, char nametag[], void (* setMethod) (short)) HepTuple destinationVia (irow, char nametag[], void (* setMethod) (char)); HepTuple destinationVia (irow, char nametag[], void (* setMethod) (bool)); HepTuple destinationVia (irow, char nametag[], void (* setMethod) (double[])); HepTuple destinationVia (irow, char nametag[], void (* setMethod) (int[])); HepTuple destinationVia (irow, char nametag[], void (* setMethod) (long[])); HepTuple destinationVia (irow, char nametag[], void (* setMethod) (short[])) HepTuple destinationVia (irow, char nametag[], void (* setMethod) (char[])); HepTuple destinationVia (irow, char nametag[], void (* setMethod) (bool[])); bool columnDefault (const char nametag[], double* default); bool columnDefault (const char nametag[], int* default); bool columnDefault (const char nametag[], long* default); bool columnDefault (const char nametag[], short* default); bool columnDefault (const char nametag[], bool* default); bool columnDefault (const char nametag[], char* default); #ifdef COMMENT NOTES ----- **[1]** Block and column name and tag Column tags must be unique in their first 8 characters. (HBOOK only uses 8-character tags.) A column may be defined with no specified block (for example defined by the direct capture routine column(tag[], value). What this means is that the user does not care what block this is grouped with, and does not intent to retrieve that data "by whole block." Actually, all columns with no specified block which were defined between one row-store and another will end up in the same block. Block names again are treated as their first 8 characters. When a column is defined, you may use a longer form for its name than just its 8-byte column tag -- the format is bname::ctag and we call this a nametag. For example, in the nametag "BLOCK5::PxDev" the column tag is PxDev and it is in block BLOCK5. Unfortunately, one cannot use block names as a namespace mechanism to allow disjoint users to choose their column tags arbitrarily: The column tag must be unique across the entire HepTuple. (HBOOK imposes this restriction.) Some other form of column tag naming agreement must be reached for a given program. Other than the un-named ("don't care") block, which can be assigned anew when new columns are added after a store has been done, the block name assigned is the block name used. We have abandoned the thought of automagically changing the block name under various conditions. This has a manager-specific consequence for HBOOK: Character string columns may have to be placed into distinct blocks from ordinary variables. The manual implies this is necessary. If so, it will be an error to mix character and numeric values inm the same block if an HBOOK manager is used. **[2]** Convenient symtax for column definition You can combine definition of column nametag/designated variable, with refinements concerning packing, array dimensions, and so forth, in the following compact syntax: HepTuple *mytup; float whereItIs; mytup->columnAt ("mytag", &whereItIs)->dimension(4,2)->pack(0.1, 1.5, 12)->index("JNUM"); That is why these methods return a HepTuple* (this). The one method which does double-duty as a capture method -- and therefore may be called thousands of times after the single column definition -- can still afford to return HepTuple* (this). For now, it returns a bool but that may change. **[3]** Example of designating an accessor function Say you have a structure SpaceVector v, and you want to put r into the HepTuple, but r is accessed by the method v.r(). class SpaceVector { public: float r(); // and other stuff } SpaceVector v; HepTuple *mytup; mytup->columnVia ("ThisBLK::Radius", v.r()); // which is filled elsewhere in some loop, which then does ... mytup->capture("ThisBLK"); // captures data in ThisBLK, including // calling v.r() to capture r. Of course, now v had better stick around as long as ThisBLK is being captured, otherwise the call to v.r() will be invlaid. This is the same as always with designated variables: They must remain in scope if their block is to be captured. **[4]** Arrays of columns, and Index variables An array of columns is treated as a collectio of separate columns. They all share the same array name, but the HBOOK column names, for example, will be like NAME(0,0) NAME(1,0) ... NAME(3,5), ... etc. You in fact can retrieve a column by that specific array element name (but you cannot capture just one element at a time). So for example, with columnwise storage you can get just X(3,2) for all rows stored, and do it very efficiently. Note that indices are zero-based. An index variable is how varying-length arrays are done. It delimits the slowest-varying dimension, which in C++ is the first dimension (a in X[a][b][c]) but in Fortran is the last one (c in X(a,b,c)). That first dimension of an indexed array is REPLACED in extent by the maximum set in the span of the index variable. When the index variable is less than its maximum possible value the remaining columns may not have anything stored for them. HBOOK benefits from this optimization for column-wise Ntuples. Such un-captured columns, if read back, will give the default value. **[5]** Array ordering Because the most important implementation assigns column array element names using the Fortran convention X(0,0) X(1,0), ... X(M, 0), X(0,1), ... X(M,N) the HepTuple package will go along with that. This presents no problem at all on the side of filling the HepTuple. The confusion happens when a user knows he has stored an array shaped as x[40][20] in column array X, and wants to retrieve specific element -- he would have to say X(15,35) to get back the element which was in the C++ array as x[35][15]. That is what we get for mixing the two languages; further attempts to rectify the confusion will only make things worse. At least all our users KNOW to expect something cloudy whien this mixing goes on... **[6]** Semantics of filling an Ntuple The act of supplying data for one or more columns is called capturing that data. The act of adding the captured data for a row to the Ntuple is called storing the data. Data which is captured is not inevitably to be stored since the user can clearData for the row (for example when rejecting an event). Also, a subsequent capture for the same column will replace the earlier captured data. For managers which can benefit from the knowledge of which blocks were not filled, by not storing a block for a particular row, the information is kept. A block whcih has been captured is "marked"; clearing the block will "unmark" it, as well as setting all values to their defaults. The row number semantics for storing data is that of automatic append: The "active" row in an empty Ntuple would start at zero, and each time a row is stored, will increment. Thus you cannot overwrite a previously stored row, or leave a gap in row number. Capture of data can be done on a per-column basis using the "Direct" forms column(tag, value, defval) or capture(tag, value). Another way is to capture data for an entire block or row by supplying designated variables. These are explained in note **[7]**. The definition of the columns composing the HepTuple may be extended after some data has been captured, and even after rows of data have been stored. The rules are: * A column in an unnamed block may be defined at ANY time, as long as the column tag is unique. Adding colummns after data has been stored is even reasonably efficient for columns-wise storage, but can be quite time-consuming for row-wise HepTuples with lots of data already stored. * A column in a specific named block may be defined only if that block has not yet been stored. That is, if you capture a block (or data in a column in that block, then do storeCapturedData, then that block may no longer be added to. * A column in a specific named block may be defined even if that block has been captured (but not stored). Of course, the value for that column would be the default value until it is captured. * A column may not be defined with the same tag as some other column previously defined (even if they are in different blocks). * The definition of a defined column may not be refined (adding packing information and so forth) after data has been captured for that column. * For completeness: Storage strategy for the HepTuple may be refined up until the first storeCapturedData is done, and no later. The mode of operation where column(name, val) without a specified block name is done at arbitrary times for definition and Direct capture is a preferred of operation prefered at BaBar, where their framework allows different contributors to add their own information to a common Ntuple, but may not call every contributor for every event. The act of adding a new column to an **EXISTING** ntuple, supplying data for each row for that column, is treated as part of the concept of manipulating ntuples, specifically, as a mode of combining two ntuples. This will be implemented later. **[7]** Designated Variables and Destination Variables HBOOK has the convenience of allowing (forcing for heterogeneous column-wise Ntuples) the user to designate variables meaning that "Here is where the data will be when I want to fill a block or row." Those have to be in a COMMON block in HBOOK, to enforce a known pattern of word-alignment (namely, none). HepTuple, of cousrse, retains this convenience, but allows the designated variables to be in any struct or structs, not necessarily contiguous. C++ compilers are free to word-align structs by adding padding. So even if all the designated variables are in one struct, there would be a need to specify the addresses of each. In HepTuple this is done either by columnAt(), or for a whole block at a time by the HepClist of addresses in block(). We distinguish designated variables for filling the HepTuple, from destination variables for retrieval. To retrieve data other than one column or column array at a time, you must set up destinations for the data using destinationAt(). Don't forget to keep the designated and destination variables in scope! If you don't then either the HepTUple is being filled from meaningless data, or your retrival is putting data in a meaningless place! **[8]** Possible error returns for Read routines -1 Name not found -2 irow past end of Ntuple -3 No designated variables were established for desired block or row -4 Element of Column array read using tag without indices -5 Ordinary column read using readColumnArray Since accessing an existing Ntuple is inherently subject to these problems, we choose (for now) to return a code RATHER THAN to throw exceptions. In particular, placing all access inside individual try constructs will be badly inefficient in an area that might matter. **[9]** Refining destination definitions To specify a destination well enough for retrieval to proceed, you need to specify: * The nametag and address, as in destinationAt(). You do not need to specify, because the HepTuple knows about, the following: - The dimensions of a column array - Packing information - The span of an index column - The tag of the column used as an index It you do specify unneeded information, and it clashes in a meaningful way with the actual information, the specification routine will return 0 instead of a pointer to the HepTuple. This should not be ignored; it probably means you will fou up badly if you try to retrive data into space tailored to your wrong information! When an entire block is specified at a time, by blockDestinations(), the format supplied must match reality in tags, index variable, dimensions, and spans -- otherwise, there is too great a likelihood that reading into the supplied list of addresses will be blowing storage. For example, if you think index variable J has a span of 0-200 and it can really go to 210, you probably are allocating 200 words of space to hold a possible 210-word output. Similarly, the number of columns in the format string must match the number of addressses in the list. But these do not have to match the total number of columns in the block, or be in the same order -- you can skip columns on retrieval. And packing information in the destination format string is ignored. **[10]** SetMethods for destinations This is for purists who wish to fill their structures (from retireved Ntuple data) via "mutator methods" rather than direcly writing to their data. Say you have a structure SpaceVector v, and you have a method v.r(float) that would take one float value (IT MUST TAKE 1 VALUE FOR THIS FEATURE TO BE APPLICABLE) for r, and change the vector to keep the same theta and phi but use the new r. This is not just storing r in a place; the actual x, y, and z all change. class SpaceVector { public: void r(float); // and other stuff } SpaceVector v; HepTuple *mytup; mytup->distinationVia ("ThisBLK::Radius", v.r()); // which is filled elsewhere in some loop, which then does ... mytup->readBlock("ThisBLK"); // When up to Radius, it supplies that // value as the agrument to a call to // calling v.r(). Of course, now v had better stick around as long as ThisBLK is being read, otherwise the call to v.r() will be invlaid. (Frankly, v.r(x) can do anything it wants to with x. We just imagine it being used to modify some aspects of v.) You can use this mechanism to work with a single column or an array of columns. In the latter case, you have mutator method which takes as an argument an float[] instead of a float. **[11]** Memory resident information The resident() method returns: 'D' disk resident 'M' memory resident 'S' shared memory resident If shared memory resident, the method sharedAreaName returns the name of the shared section (otherwise, it returns a blank HepString). If the Ntuple is not disk resident, either a circular buffer is being used, or demand strategy (where additional buffers are added as the Ntuple grows). **[12]** Column data type identification On a compliant compiler one could do type_info x = mytup.columnType_info(ncol); if ( x == typeid(double) ) doWhatever; The alternative we supply is ColumnData_t x = mytup.columnDataType(ncol); if ( x == ColumnData_FLOAT8 ) doWhatever; Actually, that alternative may be more than a kludge for the compliance- differently-abled, because our types (FLOAT8 vs FLOAT4 etc.) more directly correspond to the types used underneath, R*8, R*4 and so forth. **[13]** Fixed-length character strings These correspond to the HBOOK C*4, C*8, ... C*32 formats. They are different than ordinary char arrays in at least two ways: - Even with columnwise storage, all 4 or 8 or ... or 32 bytes are stored together. - If there is any differnece in byte swap issues, the underlying manager will treat these as character strings and apply the proper swap for those. Arrays of columns of fixed-length character strings -- e.g., the concept expressed by the format string X:C*16(3,4) -- are not supported. By the way, what if you want a true array (of arbitrary length, perhaps with an index, maybe mulit-dimensional) of "integer*1" objects? Then you don't invoke characters(), you invoke dimension(). The HBOOK implementation MAY NOT ??? ALLOW YOU TO MIX CHARACTER STRINGS AND ORDINARY DATA IN THE SAME BLOCK. **[14]** Direct capture of arrays of columns Notice that although a 1-dimensional array can be defined without designated variables (the columnArray() method), multi-dimensional arrays will always have designated variables. Nonetheless, the capture (char nametag[], float values[]) method allows Direct capture; but if designated variables were assigned for that column array, one should be careful not to capture the block later lest the directly captured values be over-written. **[15]** columnArray method for non-float data In this method the default value is not required if float data is to be used -- the signature without defval assumes float and sets defval=0. But for coulmns of int, double, or other non-float data, the default value must be supplied. And if a constant is supplied, the user must be careful that the constant is of the desired data type, as that is the only way the method caln tell what type of columns the user wants. Similarly, columnDirect() requires a default value if non-float data is to be used, and the user must be careful that this is of the desired data type. CHANGE 7/10 **[16]** Capture of already-captured columns The semantics of capture are that we do not over-write data captured yet neither stored nor cleared. For capture of entire blocks or rows, this just implies "capture all previously uncaptured data." For the single- column or column-array routines, re-capturing like this is ZMthrown as HepTuple::ZMxRecapture. Similarly, if a block has been captured AS A WHOLE BLOCK, or an entire row has been captured, then re-capturing in that manner will ZM throw that. #endif /* COMMENT */