Outdated Discussions of Enhancements in ZOOM Packages

This page presents discussion about proposed encancements to ZOOM packages.

More details can be found in the FeatureLog files for each package.

ErrorLogger 014: Time-zone information

Eric Wicklund notes an instance where the date/time on a user's report was hard to comprehend, because the job was done remotely and nobody knew the time zone. He has requested time-zone info be included with timestamps in ErrorLogger outputs.

It is suggested that this addtional info be made the default.

Mark Fischler points out that although this would change the behavior of the log output, it would not change it in any way that would invalidate automated verification (since the timestamp output, if enabled, inherently varies from job to job).

The time zone is hard to get in a portable way, but Greenwich Mean Time is not. The suggested output format is, then, 19-Nov-2002 16:38:17 [21:38 GMT]. In some cases, the date in GMT will differe from the local date; but given the local date and time and the GMT time, no ambiguity can exist.

Implemented in v2.2, R-03-04-04

ErrorLogger 013: debug-message syntax

Mark Fischler points out that much of the overhead in error message processing may be incurred even if no destination responds to a given message. This is suitable for the original intended purpose of the package -- output for rare and anomolous circumstances. But it is highly unsuitable for using these messages for debugging purposes, and later dynamically disabling the messages for production running.

The proposed enhancement to make this usage sensible is as follows:

Framework code:

User code: Such a message would, if acted upon, be equivalent to User code:

Implemented in v2.2, R-03-04-04

ErrorLogger 012: more severity levels

Yuyi Guo requests additional error severity levels. The reason is a desire to use ErrorLoggeras first tool for some debugging purpose. "I started to add debugging messages, but found out that we have used all the non error and non warning severity levels for other types of message."

Mark Fischler pointed out a couple of workarounds which would not require more severity levels. However...

Mark Fischler and Jim Kowalkowski note that using this package as a debugging outlet, depending on severity levels to control verbosity, is a bad idea for a couple of major reasons:

Instead, they propose enhancement 013 -- a special syntax for debug messages, which will allow for good granularity and quick discarding when less verbose output is desired.

This proposed enhancement was rejected in favor of ErrorLogger 013, which accomplishes the same benefits.

LinearAlgebra 003: Faster small-N inversion

Nick Macks points out that by adding a switch which allows use of the specialized inversion methods for inverting N by N matrices for N up to 6, we can improve the speed for inverting general matrices which happen to be of sizes 2x2 - 6x6, while not measurably affecting the speeds of other inversions. The typical speed improvement is a factor of about 3.

Leo Michelotti points out that the user can use the specialized matrix forms, and get this improved speed, in the existing package. However, user code that (for what ever reason) works with generalized ("generic") matricies would benefit from this change.

Mark Fischler points out that this improvement must be tested quite thoroughly, since it impcacts currently working code and only improves speed.

The code implementing this has in fact been written and tested. When we are convince that the testing covers all cases adequately, we intend to check the changes in.

Implemented in v2.2 R-20-10-28

LinearAlgebra 001: Avoid use of exit()

Jim Kowalkowski points out that a D0 user has been adversely afected by the fact that the LinearAlgebra error() routine ends by calling exit(0). The situation was that a singular matrix was being inverted. The request comes down to some way for the package to suggest (rather than call) an abort, so the framework can choose what to do.

M. Fischler agrees that calling exit() directly is a violation of an implied contract with the framework people -- the framework should have the only authority to do that. We have the mechanisms to avoid this problem, using the ErrorLogger or Exceptions package (which were not available when LinearAlgebra was written).

Three possible improvements would be:

  1. The error() routine can issue an errlog message, with id "LinearAlgebra Fault" and level ELsevere. Assumedly the framework will cause this not to abort the program.
  2. We can define a small collection of ZMexceptions and replace each call to error() with a ZMthrow. If this is done, we should probably do the work to supply a return value from the routine (in each case) which, if not sensible, at least will not force an immediate abort situation.
  3. We can as above replace each call to error() with a ZMthrow, but hook these (via the ErrLogEx package) to issue a message to the errlog.

This problem was fixed in release R-02-07-24.

ErrorLogger 010: Ability to re-establish handles to attached destinations

Jim Kowalkowski and others pointed out, at a ZOOM ErrorLogger post-design discussion, thatif the handle to the ELdestControl object returned by attach() is lost, there is no way to get it back. Everyone that uses ErrorLogger extensively has this happen. The solution that would make users happy would be to give a string name to the thing, and an interface that wold allow it to be searched for.

MF notes that two interface decisions are needed: How the name of the destination gets supplied, and what happens if a user looks up a name and it is not present.

The syntax to be implemented will look like:

  // in the setup code
  ELdestControl logfile;
  logfile = logger->attach(ELoutput ( "filename.log" ), "myLogFile" );

  // in later code, where that logfile has been lost
  ELdestControl logfile
  bool handleFound;
  handleFound = errlog.getELdestControl ("myLogFile", logfile);
  if (!handleFound) { ... oops, wrong name -- user chooses what to do here }

If somebody searchs for a name which is not found, the object returned will be a default ELdestControl, and controlling anything using that will result in nops. In that case, an error of severity SEVERE2 will be declared on the errlog.

This was done, and appears in release R-02-04-23.

PhysicsVectors 004: Removal of virtuals in operations involving Rotations

Evegenyi T. points out, after investigating a slowdown in Geant4 using CLHEP 1.7 noticed by Makoto Asia, that the operations R*v and R*R are musch slower (factor of 3.5 and 2.3) in CLHEP 1.7 than they were in 1.6. This is contributing to a factor of 1.2 slowdown in Geant4.

MF notes, and ET concurs, that the reason is that the function (actually operator()(v) is virtual and the calling overhead is large relative to the work, which was inlined in CLHEP 1.6.

If we can inline this method the unfavorable difference will go away.

Unfortunately, the function is virtual in Hep3RotationInterface precisely to allow polymorphic use of rotations of various kinds. Fortunately, neither CDF nor D0 ever use the base classes, or use Rotations in a polymorphic manner. Removing this ability will greatly simplify the code and may speed up Run II executables as well.

MP and WB point out that to try to have our cake and eat it by doing inline virtual would be inferior because

  1. compilers will still miss the optimization
  2. the simplification in removing the unwanted interface structure is considerable

Plans and Priority

I will strip out the inheritance structure and inline all methods where this is remotely sensible. No methods will be inherited in the rotation classes.

This is a goodly amount of work; fortunately most is straightforward and in fact simplifies the package. I will start on it right away, since CLHEP users would like the improvement sooner rather than later. A deadline to try for is to have the new code working by Feb 20, when Geant4 people will be here to investigate spped issues.

Documentation modifications, and perhaps expansion of testing coverage, will follow.

This was done, and appears in release R-02-04-23.

PhysicsVectors 003: operator>> matching << output

Scott snyder points out that absent good reason to the contrary, operator<< and operator>> should be inverses; that is, what you write out with operator<< you should be able to read back in with operator>>. However, that's not so for some of the Vector classes. For example, the HepEulerAngles class puts out is output like this
  (phi, theta, psi)
but expects to read input in the form
  phi theta psi
(no punctuation).

Leo Michelotti notes that Scott is "correct, abundantly so."

Mark Fischler notes that this fix can be done without affecting the behavior of existing correct programs, by making operator>> more flexible. This will take some work, along with some testing, but it is straightforward.

This was done and appears in release R-01-11-29.

PhysicsVectors 002: range of deltaPhi in deltaR

Petar Maksimovic, speaking for a CDF group, points out that w1.deltaR(w2), as it stands, is not the useful quantitiy it is meant to be. The problem is that if v1.phi() is PI-epsilon, and v2.phi() is -PI+epsilon, then the delta phi used to compute deltaR() should be small, not almost 2*PI.

Also, when told about aximAngle, PM stated that it was hard to see that this function was available; deltaPhi() would be a better name.

Liz Sexton-Kennedy notes that even if the code using deltaR() is repaired locally, we should not leave this trap in our library.

Mark Fischler notes that this problem, of course, is not restricted to LorentzVector::deltaPhi(). SpaceVector::deltaPhi() has the same behavior, and also the azimAngle() method without a reference direction has the same flaw.

The proposed course of action is to fix this:

  1. v1.azimAngle(v2) will be modified to return an angle in (-PI,PI].
  2. v1.azimangle(v2) will also be given an alias of v1.deltaPhi(v2).
  3. deltaR() for Hep3Vector will utilize deltaPhi().
  4. deltaR() for LorentzVector will return deltaR() of its 3-vector part.
  5. Appropriate documentation changes will reflect these range normalizations.

This was done and appears in release R-01-11-29.

PhysicsVectors 001: eta(zero vector)

Gustaaf Brooijmans has requested that when eta() is taken on a SpaceVector with components (0,0,0), the answer returned be 0 rather than the current 1.0E72.

Mark Fischler notes that

  1. If the vector is looked at as a vector with 0 z component and perhaps a tiny transverse component, then the answer would be zero.
  2. If the vector is looked at as a vector along the Z axis, then the answer is infinite but with the same sign as z. For z=0, zero makes more sense than plus or minus a large number.
  3. Thus the answer zero is a fairly sensible way to react to this in principle ambiguous case.
  4. However, this does represent a small change in CLHEP Vector behavior, and needs the OK of the CLHEP people.

The proposal, pending CLHEP acceptance, is to have eta() return 0 if the vector is zero.

This was done and appears in release R-01-11-29.

ErrorLogger 009: nobr feature eliminated

The documentation implies that: Mark Fischler notes that
  1. This feature will involve fooling with a template and adding a member to the ErrorLog class.
  2. Therefore, the change involves more than a trivial amount of work, some small risk of breaking things accidentally, and certainly force recompilations due to a header cahnge.
  3. Nobody, despite the glaring (not yet implemented) in the documentation has said they wished we would get around to implementing this.
  4. The lack of this feature is at worst a cosmetic issue.

Therefore, the proposal is to simply remove this feature from the documentation and do without it. This, of course, alters no code.

This documentation change has been made, as of v2.1.4 or earlier.

ErrorLogger 008: Thread Safety

Gordon Watts needs a way to use the ErrorLogger package in a multi-threaded application. The current implementation garbles messages if two threads are preparing messages at once.

Mark Fischler presents the following technical information:

Mark Fischler after some thought and discussions suggests (5/25) the following possible approaches: It is not yet clear which of the four solutions to choose:

The dilemma is how to allow the user to provide the mutex mechanism, and still employ the exception-safe "resource aquistion is object instantiation" idiom.

Walt Brown suggests (5/29) creating a new templated class, which takes the user-specified mutex class as a template argument. This turns out to be the right solution, with the templated class ThreadSafeErrorLog < Mutex > available as a replacement for ErrorLog.

and, most importantly, Marc Paterno concurs that this is a sensible approach.

This approach was done, and appears in release 2.1 (R-01-07-11).

ErrorLogger 007: Multiple module inclusion/exclusion

Liz Sexton writes that users want to be able to filter messages by a module list, not just one. In particular, they have interactive users who would benefit by being able to flexibly control filtering. This should be by methods of the ELdestControl, akin to filterModule().

Mark Fischler after discussion with Liz suggests the following interface:

At any given time, either the `respondToModule' list or the `ignoreModule' list may be active, but not both. At the start, the destination is in the same state as it would be in right after respondToModule("*"). The current routines (which of course would not be removed) can be re-written in terms of the new ones:

Exceptions 001: ZMthrow with specified line/file

Scott Snyder suggests we provide an alternate version of ZMthrow that allows specifying the file/line number.

If code goes through a wrapper module to formulate and ZMthrow a ZMexception, the desired line numner and file name would be those of the line calling the wrapper, not the line of the ZMthrow inside the wrapper. M. Fischler notes that we indeed would have to supply something new. The ZMthrow_ method does directly take a line and file, but the ZMthrow macro does more than just calling ZMthrow_() -- it also tests the return value and if indicated either does a C++ throw or calls abort().

I think the cleanest syntax that would help is a method

  ZMthrow_from ( ZMexception & ex, int line, std::string file );
Scott would then be able to do:
  throw_foo ( __LINE__, __FILE__ );
  void throw_foo (int line, std::string file) {
  ... prepare error message s ...
  ZMthrow_from (Exc_Foo (s), line, file);
The implementation of the ZMthrow_from method will be easy and safe.

Exceptions 002: Control of message format

Scott Snyder points out that for regression tests which capture the output of test programs and comparing against expected output, it is desirable to be able to supress time stamps the full pathnames of sources.

Scott has put together an alternate logging behavior that suppresses these. However, "it was not entirely straightforward to do this with the interfaces provided, and the result is kind of ugly." He suggests it would be much easier if the library provided a global switch to alter the formatting of the exceptions in this manner.

M. Fischler says the right way to do this is by a logging behavior. What I should do is take Scott's "kind of ugly" behavior, clean it up and make it into provide a new behavior ZMexLogFormatted which everybody can use. This has the same behavior as ZMexLogAlways, but omits any information which might routinely differ per run.

The choice of name ZMexLogFormatted is deliberate: Later, if somebody wants more detailed control (such as "suppress the file completely") this can be provided by methods to modify the default behavior of this new logger, and the change will be locallized to a single class.

If this behavior is provided, we should of course use Scott's code, and just clean up anything that appears "ugly."

M. Fischler after coding and testing the feature notes that the name is not very illuminating. Since the purpose of this log behavior is to make validation easier, the name ZMexValidationStyle seems better.

Exceptions 003: Override of assigned severity

M. Fischler noticed that while you can set up non-standard haldlers and logging behaviors for types of ZMexceptions coming from a library routine, there is no way to assign a severity other than the one it is constructed with. In one program it would have been useful to demote a ZMsevere down to ZMwarning.

I suggest adding to ZMexception static methods to modify each of the ZMexClassInfo fields set in the constructor:

  ZMexception::setSeverity(const ZMexSeverity & s);
  ZMexception::setName(const std::string & name);
  ZMexception::setFacility(const std::string & facility);
If we are anyway doing one or both of enhancements 001 and 002, this additional change will be very simple and safe.

ErrorLogger 003: ELstatistics::statisticsMap()

Wolfgang Wagner requested an access method to ELstatistics which returns a copy of the ELstatistics map.

Mark Fischler responds that while we shoiuld not expose the internal map (which in some future implementation may be some other sort of structure) there is no problem with a method supplying a method which returns a std::map . These are already public data types.

At March ZOOM meeting it was decided to go ahead with this.

ErrorLogger 004: ELdestControl::setHexTrigger()

Matthew Worcester writes that the experts would like to be able to output hex numbers in the error logger. This can't be done via manipulators.

Earlier comments from several people expressed the same request.

ZOOM developers point out that they previously tried to do this and that it was hard to do in a portable manner since the hex manipulator has different signatures on different platforms.

Mark Fischler sucggests the following limited substitute: A method of ELdestControl

with ELoutput doing the following when an int k is sent: If abs(k) is at least m, then output k in the form (for example) 65534 [0000FFFE]. An experiment could choose to set this trigger at, say, 101, and then all big ints would come out in this dual form.

ErrorLogger 005: ErrorLog::setDiscardThreshold()

Matthew Worcester writes that he has diagnostic ELinfo messages and when the thresholds are set high enough that these are ignored, the processing of them still severely impacts his code performance. He would like this effect to go away.

Mark Fischler finds that a typical ignored error message can eat about 15 usec. Though this is not very long, there is a nearly trivial change that could cut that to about .5 usec. The proposal is to add a method of ErrorLog

If you do this, any message sent to that logger with a lower severity will be discarded as quickly as possible; and further items streamed to that log (until the next message) will be discarded as quickly as possible.

Exceptions 005: code-shortening ctor for ZMexClassInfo

Scott Snyder points out that each time a user-defined ZMexception is defined, the ctor for its ZMexClassInfo generates more than 3K of code. Most of the code being generated for this file was the static initialization code that calls these ctors for each of the instances. He provides suggestiions that he finds cuts the code by a factor of 3.

Walter Brown notes three main ways to attack the code bloat:

  1. ctors that pass strings as const std::string instead of by reference should be modified to pass by reference.
  2. split the ctor for ZMexClassInfo such that there is one with just the first 3 args (no defaults for handler or logger) and others which take handler or all three. This way the most common case can use static handler and logger and so need not construct them.
  3. Most importantly, move the ctor to a .cc file rather than keeping it inline.

This was done and appears in release R-01-12-21.

Currently, the user is being asked to evaluate how far these steps go toward relieving the code bloat.

Fermilab Physics Class Library Task Force
Mark Fischler (mf@fnal.gov), Walter Brown (wb@fnal.gov), John Marraffino (marafino@fnal.gov), Philippe Canal (pcanal@fnal.gov)

Parent Pages:

ZOOM Home Page - Fermilab at Work - Fermilab Home