The ZOOM ErrorLogger Package:

Customization Hooks
and Issues Relating to Collective Logging

25) What an ELdestinationI Must Do

In order for a custom the ELdestinationI to work properly in the context of 
how the physicists and frameworker can use it and how the logger will 
interact with it, the destination class (and its associated ELdestControl) 
must support the following methods:

The destination class must have:

bool log ( ErrorObj )		This is allowed (and encouraged, based on limits
				and thresholds) to decide to ignore or act upon
				this message.  If the message is acted upon,
				log() should return true so that statistics
				can know somebody logged it.

bool msgStart(ErrorObj)		These methods can be null methods.  On the 
bool msgItem(ELstring)		other hand, if a destination chooses to output
bool msgEnd()			partial information before the termination of
				the error message, it can.  The msgStart()
				method provides an ErrorObj with empty item
				list, so that ELoutput can get at the message 
				"pre-amble" and	"post-amble."  
				These return true if the msg is being logged 
				rather than filtered.

startSummary(char* title=0);	These are present for the purpose of feeding 
summaryLine(char*);		formatted summary to a destination, a line at
endSummary();			a time.  A given destination is free to igmore

ELdestinationI * newCopy();	Pure virtual in ELdestinationI; must be 
				supplied.  This must new a copy of the 
				destination object and return its address.
				See note 5 "ELdestControl Details" for the
				ownership of destinations issue.

Also, the following public logicals are false by default; they should be set
appropriately by the derived class constructor:

bool skipImmediateItems		This destination will wait for the message to be
				complete and the full message to be sent.  Thus
				there is no need for individual items to be 
bool skipMessageObject		This destination is acting on individual items
				and will not need to be sent the full ErrorObj
				when completed.  Either skipMessageObject or 
				skipImmediateItems should be false (both could 
 				be false).
bool skipSummary		This destination will not output summary info.

						-- Customization Hooks --

Even if skipMessageObject is set true, the destination's operator()(ErrorObj)
ought to check the message's serial number and be prepared to act on it if it
does not match that of the message being output.  The reason is that a physicist
can construct a message object and do errlog(msg).  In that case, 
skipMessageObject is ignored, the msg is sent to each destination, and no 
immediate items are sent.  If operator()(ErrorObj) were  trivial, such 
messages would never be acted upon by the destination.

The ELdestControl object has, and the destination class must support:

setThreshold (ELseverityLevel sv)       The destination is free to ignore these,
setLimit     (ELstring s, int n)        as does ELstatistics which does not
setLimit     (ELseverityLevel sv,int n) apply limits or timespans.  If these
					are NOT ignored, then the string "*"
setTimespan  (ELstring s, int n)	must be treated as a wildcard, and 
setTimespan  (ELseverityLevel sv,int n) the semantics of precedence described 
					in this document must apply.

setTableLimit (int n)			Impose a limit on the number of 
					entries in any given table.  See note 3.

clearSummary();			Ignore clearSummary except for statistics
wipe();				destination; see "clearing Statistics and/or
zero();				Limits" behavior definitions.

setPreamble (ELstring preamble)		Establishes a substitute for the default
					%ERLOG start of message output.  
					Pre-ambles of up to 6 chars will work 
					properly with the standard formatter.

setNewline  (ELstring nl)		Establishes a substitute for the default
					\n as a newline character in the log.
					Provided in case NT needs this.

The control object for a destination intended as a statistics gatherer must 
also support:

summary(dest, char* title=0);	These methods should be ignored except in
summary(os,   char* title=0);	the case of the controller of a statistics
summary(s&,   char* title=0);	destination, which will output the summary
				as directed.

The ELdestControl class has two protected pointers,  
  ELdestinationI * d;
  ELdestIX * x;
It is possible to derive off this class, placing values do customized subclasses
of ELdestinationI and/or ELdestIX into these pointers in the constructor of the 
derived ELdestControl subclass.  This would minimize the amount of new code 
one needs to write, and also may eliminate in some cases the need to recompile 
framework compilation units when the header of the new customized class changes.

						-- Customization Hooks --

26) ELsaveBuffer Semantics and Layout

The semantics of the ELsaveBuffer are dominated by the idea that it ought to
be usable in the context of a collective system, in which a single "boss" may
need to extract the saved messages, a chunk at a time.

The buffer is logically a set of three limited heaps; the boss may designate
one heap as being read out and therefore "frozen" to additional inserts.  The
semantics of overflow are that it is possible to lose the last messages past
an overflow condition; and if that happens, the last message before the 
overflow will also be lost, replaced by a message warning of skipped messages.

The control variables relevant to this boss commumunication are as follows:

FrozenBuffer	- boss writable 	The buffer unavailable due to being 
					read out by the boss.  A, B, C or 0.

NoFlush		- boss writable		If this flag is set, then instead of 
					replacing any message with a skip 
					notice, BLOCK until the next buffer
					is unfrozen.  Ensures no message skips,
					at the cost of potential local blocking.

Dump		- boss writable		If a non-zero integer, instructs the 
					system to flush() the buffers and then
					to copy that ingeger into DumpResponse.

ActiveBuffer				The buffer being written at this instant
					(and thus unavailable to read even 
					though the boss may have just said it
					should be frozen).  If this is A, B or 
					C, it should quickly return to 0.

DumpResponse				Starts at 0.  Assuming the boss sends
					sequential integers to Dump, this lets
					the boss see if a given dump has been 
					done, to avoid accidentally missing or
					doing unwanted dumps.

When a buffer has been frozen, and now a new buffer is frozen (or FreezeBuffer 
is changed to 0) the ELsaveBuffer, upon noting this (which it will do whenever
any message come in) will immediately clear that previously frozen buffer.

So the general prcedure for the boss is:
At leisure, decide to read out some messages.  Having kept track, you know
the first unread messages are in A.  Write A to FrozenBuffer.  Read ActiveBuffer
and the state block (see above), which are contiguous.  Almost certainly, 
ActiveBuffer will not indicate A, but if it does, re-read until it does not.
Now read out A, and use the information in the state block to understand it.
Then either write 0 to FrozenBuffer, or if you want to read B immediately,
write B to FrozenBuffer, and repeat the process with buffer B.

The state block will look like:



There will also be a summary buffer.  Again, one will need to define procedures
for the boss to read this out without feaing that it will change while being
read.  We have not yet defined these additions to the state.  ****

To support this sort of (interprocess) communication we provide a way of
imposing a limit of the size of the ELsaveBuffer as well as a way to force
the buffer to reside at a given address:

	ELsaveBuffer ()		// Usual default constructor presenting 
				// unlimited room using std::list.
	ELsaveBuffer (int)	// Limit to that may bytes.  Divide into
				// buffers A B and C with 1/3 to each.
	ELsaveBuffer(ELsaveBuffer*, int)
				// Place at address, and limit to that many 
				// bytes.

						-- Customization Hooks --

27) Avoiding lists, maps, strings, streams, and templates

When first designing this with D0 L2 in mind, the following concerns came up:

Due to the lack of psot-startup memory allocation, and desired avoidance of C++
library classes, there several types of worries:

A Structures that may in principle grow indefinitely.
B Use of C++ strings.
C Use of stringstreams to build strings.
D Use of streams other than stringstreams.
E Use of maps.
F Use of templates penetrating to user interface

Ideally, we would like to feel free to use all of these in a basic 
implementation, yet have clean ways to avoid using any of them if necessary.

Here are the places where there can be concern:

A1 - The errorStats table could have unlimited entries
A2 - The errorLimits table could have unlimited entries
A3 - The save buffer ELbuffer might grow arbitrarily
A4 - The control structure for ELbuffer might grow arbitrarily
A5 - Capture buffers used to flush the ELbuffer need to be limited
A6 - The list of destinations could grow indefinitels

B1 - An item sent to log by << can be arbitrarily long
B2 - The formatted message string can be arbitrarily long
B3 - The date/time might be a C++ string
B4 - Part of the Context object might be a string
B5 - Objects contatined in a class (e.g. ELerrorStats) might be strings

C1 - Use of stringstream to implement operator<<

D1 - An ELdestination might inherently be associated with a stream

E1 - Use of maps in the limitsTable
E2 - Use of maps in ELstatistics

F1 - The ErrorLog and ErrorObj classes use templates for operator<<.

We have followed several design principles to allow a customizer to resolve 
any or all of these concerns.  These include keeping functionallity that we
do insist on implementing via full C++ in subclasses below interface 
definitions; the customizer can replace any of those subclasses which 
violate the conditions of the limited environment.

The following techniques can be followed to avoid (or avoid using in a 
post-init dynamic memory manner) the std:: concepts of list, string, and stream:

-- To avoid using lists and maps:

We use various ELlist_XXX as typedef-ed to std::list, for instance, 
	typedef std::list ELlist_destI
so one option is to replace those with your own container classes, but we do 
assume list semantics.  Similarly, we use various sorts of ELmap_XXX as
typedef-ed to std::map, and there we assume map semantics.  To try to 
avoid or put limits on those semantics:

  To avoid lists altogether you could supply a custom ELadministrator.  
  But if you can tolerate lists if they do not grow after init time, using the
  provided one should be fine.  

  The errorLogger will have an additional set of constructors taking
  an instance of ELstatsTableI & which will allow substituting a derived
  class that does not do any undesired operations.

  The ELdestinationI derived classes can utilize any limits table or mechanism

  A custom ELdestination may easily be put in place of ELsaveBuffer on the list
  of sinks (or howenver the ErrorLoggerI accepts sinks).

-- To avoid using strings:

The ErrorLogger package uses ELstring which is typedef-ed to std::string, so 
one can easily substitute any class with similar semantics.  In partticular,
a string class with some hacked allocator to avoid dynamic memory allocation 
after some startup epoch would be a plausible substitute.  The following 
classes would be places to check for whether your string semantics are adequate:

  This one is easy:  All string-like arguments are done as C-strings via char*,
  and all internal strings with the exception of the error summary are stored
  as ELstrings with specified maximum lengths.

  When a summary is sent to a destination, it is sent one fixed-length char* 
  representing a single line at a time.  So unless the framework explicitly
  does log.summary(string&), no strings or arbitrarily long char arrays
  which might have to be malloc-ed are used.

  Our provided class does use a string to hold the formatted error message 
  string, however, the customizer can subsitute a different class or use
  a different form of ELstring.

  Our provided classes for these do not use strings.

  Our provided class for ELdestination uses strings in an essential way.  One
  could derive off ELdestinationI to form sinks that do not use strings, or rely
  on a different form of ELstring.

  Our ELsaveBuffer does not use strings, since it strives for a clean and
  remote-accessible fixed format.
						-- Customization Hooks --
-- To avoid using streams:

  ErrorLogger does not actually use streams, though the syntax is patterned
  to some extent after the string << syntax.

  Our provided ELdestination uses streams extensively.  A customizer would have
  to create a custom derived class to avoid this.

-- To avoid using templates:

Templates are a trickier matter.  We have avoided using templates in all but two
essential circumstances:

a) The std::list and std::map make use of templates, so to avoid templates 
   altogether one would have to custom derive custom ELlist_dest and 
   similar classes defined in ELlist.h and ELmap.h.

b) ErrorLog accepts items of information supplied to it by operator<<.  It
   formats these into ELstrings using an ostringstream to format the message,
   and sends them to the list of ELdestinationI and the error message object.
   Both the accepting and the formatting via ostringstream use templated
   operator<<, and this HAS TO BE, because otherwise the user would not be 
   free to << any type of string, number or object to the logger!  
   To avoid the use of templates, one would have to substitute a custom 
   file in ErrorLog.icc, supporting the creation of ELstrings from some
   restricted set of object types.  By out rules, an ErrorLog must have << 
   from at least char*, so at least that trivial one must be present.  

   Similarly, one would have to substitute a custom file for ErrorObj.icc,
   supporting operator<< from presumedly the same restricted set of types.

28) Table Limitations

In implementations that allocate a fixed amount of space for tables, such as
a fixed number of ids in limit tables, the customizer needs to define the 
behavior when these limits are exceeded.  We suggest the following:  

  - When a statistics table (which is a pure map) cannot add another key of
    error, log that fact (the first time) as an ELwarning2 and ignore the

  - When the id-only part of a limits table gets full, additional setLimit
    commands log an ELwarning2 and become no-ops.  

  - When the count part of a limits table, which is keyed off extended id, 
    becomes full, log the first time as an ELerror2.  Additional types of 
    errors are simply not limited.

						-- Customization Hooks --

29) Direct logging to one destination

Although in general all logging is done through ErrorLog objects, which work 
through the ELadministrator, the frameworker may log a formed ErrorObj 
directly to an ELdestControl.  

	bool reacted = logfile.log ( ErrorObj & myMsg );  

As implied, the destination's thresholds and limits still apply; the log()
method returns false if the destination did not react to thes ErrorObj. 

If this backdoor mechanism is used, it has bypassed the ErrorLog functions.
In particular, ErrorLog normally provides module and subroutine strings.
To restore this ability, the ErrorObj class has two further methods:

	myMsg.setModule     ( "whatever" );
	myMsg.setSubroutine ( "whatever" );

Note that (unless the destination happens to be ELstatistics) error messages
logged in this manner will not be reflected in the statistics.  Generally,
it is a better idea to send things to all destinations using

	errorlog (myMsg); 

and let each destination filter out what it does not want.

30) Custom Severity Levels

The intent is that all assigned severity levels fall into one of the 14
levels provided; in fact, such fine granularity is normally met with derision.
We discourage but do not preclude inventing one's own levels.  To allow for
this, we establish the level_ variable for the provided ELseverityLevels at
intervals of 256 apart.

Thus one could do things like

const ELseverityLevel ELspecialWarning  ( ELseverityLevel::ELsev_Spacing *
 ELseverityLevel::ELsev_warning2 + 128, "-Q", "Special WARNING!" );

at global level, and use this as you would any of the provided severities.

Be warned, howver, that to avoid some nasty space or time overheads and
excessive use of the map container, ELadministrator, ELstatistics, and
ELlimitsTable all will treat the level_ as chopped to the nearest lower multiple
of 256.  Thus severityCount and the statistics for counting severities would
lump this new ELspecialWarning in with ELwarning2.  Similarly, limits set by 
severity level would lump these together -- but thresholds would distinguish 
between them.

You must not create levels lower than ELsev_zeroSeverity or higher than
ELsev_highestSeverity; this may cause problems and would not be caught at
compilation time.