and Issues Relating to Collective Logging
25) What an ELdestinationI Must Do ---------------------------------- In order for a custom the ELdestinationI to work properly in the context of how the physicists and frameworker can use it and how the logger will interact with it, the destination class (and its associated ELdestControl) must support the following methods: The destination class must have: bool log ( ErrorObj ) This is allowed (and encouraged, based on limits and thresholds) to decide to ignore or act upon this message. If the message is acted upon, log() should return true so that statistics can know somebody logged it. bool msgStart(ErrorObj) These methods can be null methods. On the bool msgItem(ELstring) other hand, if a destination chooses to output bool msgEnd() partial information before the termination of the error message, it can. The msgStart() method provides an ErrorObj with empty item list, so that ELoutput can get at the message "pre-amble" and "post-amble." These return true if the msg is being logged rather than filtered. startSummary(char* title=0); These are present for the purpose of feeding summaryLine(char*); formatted summary to a destination, a line at endSummary(); a time. A given destination is free to igmore these. ELdestinationI * newCopy(); Pure virtual in ELdestinationI; must be supplied. This must new a copy of the destination object and return its address. See note 5 "ELdestControl Details" for the ownership of destinations issue. Also, the following public logicals are false by default; they should be set appropriately by the derived class constructor: bool skipImmediateItems This destination will wait for the message to be complete and the full message to be sent. Thus there is no need for individual items to be sent. bool skipMessageObject This destination is acting on individual items and will not need to be sent the full ErrorObj when completed. Either skipMessageObject or skipImmediateItems should be false (both could be false). bool skipSummary This destination will not output summary info. -- Customization Hooks -- Even if skipMessageObject is set true, the destination's operator()(ErrorObj) ought to check the message's serial number and be prepared to act on it if it does not match that of the message being output. The reason is that a physicist can construct a message object and do errlog(msg). In that case, skipMessageObject is ignored, the msg is sent to each destination, and no immediate items are sent. If operator()(ErrorObj) were trivial, such messages would never be acted upon by the destination. The ELdestControl object has, and the destination class must support: setThreshold (ELseverityLevel sv) The destination is free to ignore these, setLimit (ELstring s, int n) as does ELstatistics which does not setLimit (ELseverityLevel sv,int n) apply limits or timespans. If these are NOT ignored, then the string "*" setTimespan (ELstring s, int n) must be treated as a wildcard, and setTimespan (ELseverityLevel sv,int n) the semantics of precedence described in this document must apply. setTableLimit (int n) Impose a limit on the number of entries in any given table. See note 3. clearSummary(); Ignore clearSummary except for statistics wipe(); destination; see "clearing Statistics and/or zero(); Limits" behavior definitions. setPreamble (ELstring preamble) Establishes a substitute for the default %ERLOG start of message output. Pre-ambles of up to 6 chars will work properly with the standard formatter. setNewline (ELstring nl) Establishes a substitute for the default \n as a newline character in the log. Provided in case NT needs this. The control object for a destination intended as a statistics gatherer must also support: summary(dest, char* title=0); These methods should be ignored except in summary(os, char* title=0); the case of the controller of a statistics summary(s&, char* title=0); destination, which will output the summary as directed. The ELdestControl class has two protected pointers, ELdestinationI * d; ELdestIX * x; It is possible to derive off this class, placing values do customized subclasses of ELdestinationI and/or ELdestIX into these pointers in the constructor of the derived ELdestControl subclass. This would minimize the amount of new code one needs to write, and also may eliminate in some cases the need to recompile framework compilation units when the header of the new customized class changes. -- Customization Hooks -- 26) ELsaveBuffer Semantics and Layout ------------------------------------- The semantics of the ELsaveBuffer are dominated by the idea that it ought to be usable in the context of a collective system, in which a single "boss" may need to extract the saved messages, a chunk at a time. The buffer is logically a set of three limited heaps; the boss may designate one heap as being read out and therefore "frozen" to additional inserts. The semantics of overflow are that it is possible to lose the last messages past an overflow condition; and if that happens, the last message before the overflow will also be lost, replaced by a message warning of skipped messages. The control variables relevant to this boss commumunication are as follows: FrozenBuffer - boss writable The buffer unavailable due to being read out by the boss. A, B, C or 0. NoFlush - boss writable If this flag is set, then instead of replacing any message with a skip notice, BLOCK until the next buffer is unfrozen. Ensures no message skips, at the cost of potential local blocking. Dump - boss writable If a non-zero integer, instructs the system to flush() the buffers and then to copy that ingeger into DumpResponse. ActiveBuffer The buffer being written at this instant (and thus unavailable to read even though the boss may have just said it should be frozen). If this is A, B or C, it should quickly return to 0. DumpResponse Starts at 0. Assuming the boss sends sequential integers to Dump, this lets the boss see if a given dump has been done, to avoid accidentally missing or doing unwanted dumps. When a buffer has been frozen, and now a new buffer is frozen (or FreezeBuffer is changed to 0) the ELsaveBuffer, upon noting this (which it will do whenever any message come in) will immediately clear that previously frozen buffer. So the general prcedure for the boss is: At leisure, decide to read out some messages. Having kept track, you know the first unread messages are in A. Write A to FrozenBuffer. Read ActiveBuffer and the state block (see above), which are contiguous. Almost certainly, ActiveBuffer will not indicate A, but if it does, re-read until it does not. Now read out A, and use the information in the state block to understand it. Then either write 0 to FrozenBuffer, or if you want to read B immediately, write B to FrozenBuffer, and repeat the process with buffer B. The state block will look like: ActiveBuffer DumpResponse startOfA startOfB startOfC nA nB nC capacityA capacityB capacityC NoFlush FrozenBuffer Dump There will also be a summary buffer. Again, one will need to define procedures for the boss to read this out without feaing that it will change while being read. We have not yet defined these additions to the state. **** To support this sort of (interprocess) communication we provide a way of imposing a limit of the size of the ELsaveBuffer as well as a way to force the buffer to reside at a given address: ELsaveBuffer () // Usual default constructor presenting // unlimited room using std::list. ELsaveBuffer (int) // Limit to that may bytes. Divide into // buffers A B and C with 1/3 to each. ELsaveBuffer(ELsaveBuffer*, int) // Place at address, and limit to that many // bytes. -- Customization Hooks -- 27) Avoiding lists, maps, strings, streams, and templates ---------------------------------------------------------- When first designing this with D0 L2 in mind, the following concerns came up: Due to the lack of psot-startup memory allocation, and desired avoidance of C++ library classes, there several types of worries: A Structures that may in principle grow indefinitely. B Use of C++ strings. C Use of stringstreams to build strings. D Use of streams other than stringstreams. E Use of maps. F Use of templates penetrating to user interface Ideally, we would like to feel free to use all of these in a basic implementation, yet have clean ways to avoid using any of them if necessary. Here are the places where there can be concern: A1 - The errorStats table could have unlimited entries A2 - The errorLimits table could have unlimited entries A3 - The save buffer ELbuffer might grow arbitrarily A4 - The control structure for ELbuffer might grow arbitrarily A5 - Capture buffers used to flush the ELbuffer need to be limited A6 - The list of destinations could grow indefinitels B1 - An item sent to log by << can be arbitrarily long B2 - The formatted message string can be arbitrarily long B3 - The date/time might be a C++ string B4 - Part of the Context object might be a string B5 - Objects contatined in a class (e.g. ELerrorStats) might be strings C1 - Use of stringstream to implement operator<< D1 - An ELdestination might inherently be associated with a stream E1 - Use of maps in the limitsTable E2 - Use of maps in ELstatistics F1 - The ErrorLog and ErrorObj classes use templates for operator<<. We have followed several design principles to allow a customizer to resolve any or all of these concerns. These include keeping functionallity that we do insist on implementing via full C++ in subclasses below interface definitions; the customizer can replace any of those subclasses which violate the conditions of the limited environment. The following techniques can be followed to avoid (or avoid using in a post-init dynamic memory manner) the std:: concepts of list, string, and stream: -- To avoid using lists and maps: We use various ELlist_XXX as typedef-ed to std::list, for instance, typedef std::list
ELlist_destI so one option is to replace those with your own container classes, but we do assume list semantics. Similarly, we use various sorts of ELmap_XXX as typedef-ed to std::map , and there we assume map semantics. To try to avoid or put limits on those semantics: ELadministrator To avoid lists altogether you could supply a custom ELadministrator. But if you can tolerate lists if they do not grow after init time, using the provided one should be fine. ELstatsTable The errorLogger will have an additional set of constructors taking an instance of ELstatsTableI & which will allow substituting a derived class that does not do any undesired operations. ELlimitsTable The ELdestinationI derived classes can utilize any limits table or mechanism desired. ELsavebuffer A custom ELdestination may easily be put in place of ELsaveBuffer on the list of sinks (or howenver the ErrorLoggerI accepts sinks). -- To avoid using strings: The ErrorLogger package uses ELstring which is typedef-ed to std::string, so one can easily substitute any class with similar semantics. In partticular, a string class with some hacked allocator to avoid dynamic memory allocation after some startup epoch would be a plausible substitute. The following classes would be places to check for whether your string semantics are adequate: ErrorLog This one is easy: All string-like arguments are done as C-strings via char*, and all internal strings with the exception of the error summary are stored as ELstrings with specified maximum lengths. ErrorSummary When a summary is sent to a destination, it is sent one fixed-length char* representing a single line at a time. So unless the framework explicitly does log.summary(string&), no strings or arbitrarily long char arrays which might have to be malloc-ed are used. ELerrorObj Our provided class does use a string to hold the formatted error message string, however, the customizer can subsitute a different class or use a different form of ELstring. ELlimitsTable ELstatsTable Our provided classes for these do not use strings. ELdestination Our provided class for ELdestination uses strings in an essential way. One could derive off ELdestinationI to form sinks that do not use strings, or rely on a different form of ELstring. ELsavebuffer Our ELsaveBuffer does not use strings, since it strives for a clean and remote-accessible fixed format. -- Customization Hooks -- -- To avoid using streams: ErrorLogger ErrorLogger does not actually use streams, though the syntax is patterned to some extent after the string << syntax. ELdestination Our provided ELdestination uses streams extensively. A customizer would have to create a custom derived class to avoid this. -- To avoid using templates: Templates are a trickier matter. We have avoided using templates in all but two essential circumstances: a) The std::list and std::map make use of templates, so to avoid templates altogether one would have to custom derive custom ELlist_dest and similar classes defined in ELlist.h and ELmap.h. b) ErrorLog accepts items of information supplied to it by operator<<. It formats these into ELstrings using an ostringstream to format the message, and sends them to the list of ELdestinationI and the error message object. Both the accepting and the formatting via ostringstream use templated operator<<, and this HAS TO BE, because otherwise the user would not be free to << any type of string, number or object to the logger! To avoid the use of templates, one would have to substitute a custom file in ErrorLog.icc, supporting the creation of ELstrings from some restricted set of object types. By out rules, an ErrorLog must have << from at least char*, so at least that trivial one must be present. Similarly, one would have to substitute a custom file for ErrorObj.icc, supporting operator<< from presumedly the same restricted set of types. 28) Table Limitations --------------------- In implementations that allocate a fixed amount of space for tables, such as a fixed number of ids in limit tables, the customizer needs to define the behavior when these limits are exceeded. We suggest the following: - When a statistics table (which is a pure map) cannot add another key of error, log that fact (the first time) as an ELwarning2 and ignore the overflow. - When the id-only part of a limits table gets full, additional setLimit commands log an ELwarning2 and become no-ops. - When the count part of a limits table, which is keyed off extended id, becomes full, log the first time as an ELerror2. Additional types of errors are simply not limited. -- Customization Hooks -- 29) Direct logging to one destination -------------------------------------- Although in general all logging is done through ErrorLog objects, which work through the ELadministrator, the frameworker may log a formed ErrorObj directly to an ELdestControl. bool reacted = logfile.log ( ErrorObj & myMsg ); As implied, the destination's thresholds and limits still apply; the log() method returns false if the destination did not react to thes ErrorObj. If this backdoor mechanism is used, it has bypassed the ErrorLog functions. In particular, ErrorLog normally provides module and subroutine strings. To restore this ability, the ErrorObj class has two further methods: myMsg.setModule ( "whatever" ); myMsg.setSubroutine ( "whatever" ); Note that (unless the destination happens to be ELstatistics) error messages logged in this manner will not be reflected in the statistics. Generally, it is a better idea to send things to all destinations using errorlog (myMsg); and let each destination filter out what it does not want. 30) Custom Severity Levels -------------------------- The intent is that all assigned severity levels fall into one of the 14 levels provided; in fact, such fine granularity is normally met with derision. We discourage but do not preclude inventing one's own levels. To allow for this, we establish the level_ variable for the provided ELseverityLevels at intervals of 256 apart. Thus one could do things like const ELseverityLevel ELspecialWarning ( ELseverityLevel::ELsev_Spacing * ELseverityLevel::ELsev_warning2 + 128, "-Q", "Special WARNING!" ); at global level, and use this as you would any of the provided severities. Be warned, howver, that to avoid some nasty space or time overheads and excessive use of the map container, ELadministrator, ELstatistics, and ELlimitsTable all will treat the level_ as chopped to the nearest lower multiple of 256. Thus severityCount and the statistics for counting severities would lump this new ELspecialWarning in with ELwarning2. Similarly, limits set by severity level would lump these together -- but thresholds would distinguish between them. You must not create levels lower than ELsev_zeroSeverity or higher than ELsev_highestSeverity; this may cause problems and would not be caught at compilation time.