Adapting ROOT V2.00/11b to Build using Makefiles and Kai C++ v3.3 v1.1 Rob Kennedy 05-Oct-1998 ABSTRACT As a means of estimating the work necessary to adapt Root to work with CVS and build from Makefiles, I have adapted Root V2.00/11 to build entirely from a minimal set of files using enhanced makefiles on Red Hat 5.0 Linux with the Kai C++ v3.3c compiler. A similar build under SGI IRIX 6.2 with Kai C++ v3.3d fails during dictionary generation by rootcint, but succeeds when source files are substituted from the Linux build. I describe the process of reaching this state from the distributed source, binary, and cmz distributions, as well as what was learned during this process. I include in this process a cursory testing of each of the resulting binaries. The estimated future cost for this process, once it is somewhat automated and cleaned up, is about one FTE-day for the first supported Unix platform, and subsequently about one FTE-day total for several more supported Unix platforms. GOALS The primary goal of this project is to determine how long it would take to adapt ROOT as it is now supported (CMZ-based repository) and distributed (binary and source tarfiles) to be a Makefile-driven package which can be imported into a CVS repository. Another goal is to test Unix platform compatibility by reproducing with Makefiles all the steps to build ROOT from source that are now captured in CMZ, and testing the resulting binary executables. MODEL The model driving these goals is that at some point, ROOT will be maintained and distributed entirely in source form from a single CVS repository, to which active developers will have write access, and will use Makefiles to build itself from checked-out source. There are a number of possible transition plans to this model of open co-development, but I defer evaluating different plans for the moment. As vague as this model may seem, we can draw some conclusions. The ROOT package will probably be organized into distinct modules in the CVS repository to allow write permissions to be maintained separately for each module. For now, I treat Root as one single module, with source files all in one source directory and header files all in one include directory. STATUS ROOT V2.00/11 is supported with CMZ. Binary and source distributions are made available via http://root.cern.ch. Not all Run II platforms have binary distributions, but all Run II platforms are supported (at least those which are accessible to Rene Brun and Fons Rademakers) in the source distribution. The CMZ file containing the CMZ archive is publicly available at a different site, for those who need access to it. I have used the a development CMZ file from cdfsga by Rene Brun, dated 17 September 1998, which I refer to as v2.00/11b. PROCEDURE (skip to CONCLUSION to skip the details) The procedure outlined below replicates what I did in order to generate a Makefile-driven build on a ROOT source tree. This is the translation step from CMZ to CVS that would be performed with each new CMZ-based release, as long as CMZ is used as the repository mechanism. Many of these steps will disappear as the ROOT team reacts to some of the documented here, and the remaining stepe can be automated by scripts. 1) Setup the ROOT environment Setenv ROOTSYS, CERNLIB, LD_LIBRARY_PATH, and PATH. Setcompiler kai33. 2) Create source tree a) from CMZ The preferred method of generating the source tree is to do so directly from a CMZ file. This has the advantage that one depends only on the root CMZ file, and also that one can use pre-release CMZ files to test changes which the ROOT team have reposited. Here are the steps I used for Linux: 1) unsetup kai # avoid actually compilation of C++ w/Kai 2) cmz # startup cmz session 3) file root_v2.00.11.cmz 4) set C++ -lan 5) select GLIBC # LINUX ONLY, for glibc2 compatibility 6) select VENUS PYTHIA 7) select ROOTD PROOFD PROOFSERV 8) select GL GLX11 GLWIN32 9) exec setup LINUX g 10) exec install LINUX g 11) cp src/CINT_libstrm.cxx .. 12) select KCC 13) exec setup LINUX g 14) exec install LINUX g 15) mv ../CINT_libstrm.cxx src The steps (11)-(15) are required to get access to a compiler specific file CINT_libstrm.cxx which is not extracted from CMZ if KCC is select. This file is used for GNU C++-based compilers. b) From binary and source distributions i) Obtain the binary distribution for the appropriate platform and compiler, or at least for the appropriate platform and different compiler. The ROOT binary distribution contains the directory structure and any binary specializations required to build code on a particular platform. ii) Obtain the source code to build all ROOT libraries and executables The ROOT source distributions contain mostly just the source and header files, all lumped into one "src" sub-directory. The ROOT source distributions for some time have not in fact contained all the source code for ROOT. At present, the source and a header for "rootd" is missing, as well as the files PROOF_proofd.cxx (needed to build proofd) and PROOF_pmain.cxx (needed to build proofserv). This source must be obtained from the CMZ repository. 3) Move all headers from the source area to the include area CMZ maintains the header files in the source directory, then copies them into the include area during the "install". Our traditional approach, however, is to maintain the header files in the include area to avoid having duplicate files in one package tree. 4) Apply Known Source Patches One source file must be changed to get it to compile under Red Hat 5.0 Linux. Insert the following at line 54 of PROOF_proofd.cxx: #if defined(__sun) || defined(R__GLIBC) #include #endif There are also a number of weaknesses in the prototypes near the top of the same file, usually missing "C" linkage specifications, which do not prevent compilation. Compare this section to the equivalent in ROOTD_rootd.cxx. One source file must be changed to get it to compile under SGI IRIX 6.2. Comment out, as shown, the following at line 54 of ROOTD_rootd.cxx: #if defined(__sgi) && !defined(__GNUG__) //should be test for 6.2 //extern "C" { // int seteuid(int euid); // int setegid(int egid); //} #endif 5) Adjust the make procedures a) For header file location I defined VPATH (uppercase variant should be portable) to point to the $(ROOTSYS)/include directory so the dependence information lacking a full path would still be usefil in the Makefiles. b) For Kai v3.3 Kai v3.3 now has exception-handling turned on by default. Libraries compiled with exception-handling on are incompatible with those compiled with it turned off. CDF has exception-handling turned off in their builds, D0 has it turned on. I chose to turn it off at compilation and link time. I suppressed warning 830, which is related to a missing delete definition which can cause problems if an exception is thrown. With exceptions turned off, this should be harmless. I removed the link reference to libg++, which is not needed when compiling and linking ROOT with Kai C++. I added -O0 to insure the back-end does not optimize in addition to the +K0 to prevent the front-end from optimizing. c) For Fortran source support I chose (after consulting with Rene Brun) to use f2c under Linux to translate the Fortran source (used by g2root and h2root) into C and then compile the C code as other C code is compiled. Both make steps are in the same rule, so only the original Fortran source needs to be provided. Under IRIX6, I use f77 to compiler Fortran source. The Fortran used by g2root is re-entrant, so care must be taken when translating with f2c (must use automatic variables) or compiling with Fortran (compiler must support re-entrant routines... not all do). d) For g2root, h2root, proofd, proofserv, and rootd I added appropriate make rules for each of these programs, following as closely as possible the existing style. e) For general link issues I adjusted the makefiles to refer to my systems libXpm. To support the Fortran code which was being linked with a C++ compiler, I added a F77LIBS variable, similar to the existing SYSLIBS. To support rootd and proofd executable, I created a CRYPTLIBS variable. f) Separate AR,ARFLAGS from LD,LDFLAGS to allow variations between the two g) Install an "if" block to treat makedepend's assumed location h) Install a filter to remove libNew from targets and dependencies for IRIX6 and KCC 6) "gmake -f Makefile.linuxkcc depend" Note that I have sym-linked "gmake" to "make" on Linux to match FNAL's convention of naming GNU make "gmake", and leaving the native make "make". On Linux, of course, these are one and the same. This creates a file Make-depend which is used to define the dependencies of object files to source and header files. It must be recreated since header files moved, and more pieces are to be rebuilt. Running this will put in full pathnames, avoid system headers (-Y switch already in use), and include the new object files supported by the Makefiles. In general, the "makedepend" program does not process C++ files, or at least not files with a .cxx extension. An alternate makedepend is available in the ROOT binary distribution for Linux which does properly treat .cxx files, and this was used in my build. This means ROOT cannot be fully supported from a source-only distribution at this time. At FNAL, we use SoftRelTools which uses the compilers themselves to generate dependencies with the appropriate macro definitions set to insure that the dependencies reflect the actual compilation behavior. The SoftRelTools method is clearly superior in my opinion. After generating the Make-depend file on Linux2 using the Root-supplied makedepend program, I used a text editor to replace all absolute paths in the file to relative paths, so that the same file can be used on IRIX6. 7) gmake At this point, on Linux, all of the ROOT libraries (except libRGL.so) and all the distributed ROOT executables build under Linux. Under IRIX6, however, the META_G_Meta.cxx, META_G_Meta.h, X11_G__X11.cxx, and X11_G__X11.h files which are generated by rootcint do not compile. I used the versions of the files genearted by the Linux build, and was able to pick up the IRIX6 build from the point of these errors. A number of harmless errors are reported (by rootcint?) when building dictionary files for ROOT's own classes. In particular, errors are seen under both IRIX6 and Linux2 while generating META_G__Meta.cxxbut also for a few other Root classes. These appear to be non-fatal and do not affect the build process. 8) Testing the demo, g2root, and h2root I have tested the Root demo, g2root, and h2root programs in cursory fashion. The demo still suffers from the refresh/save-unders problem. My Trybos package's stand-alone Root "hello world" test still requires two File:[Quit ROOT] selections in order to exit the program. A CDF Run 2 Geant3 datafile supplied by Pasha Murat does appear to be translated into a valid (but itself untested) Cint macro source file. I have used h2root to convert a CDF Run 1 B-pad HBOOK n-tuple to Root datafile successfully. There were some differences betyween Linux and IRIX in the messages output during this process, but then very different versions of Cernlib were linked into h2root on these two OSs. 9) OPENGL If the environment variable OPENGL is defined, ROOT will attempt to build libRGL.so, an Open-GL interface library. ROOT uses the freeware Open-GL implementation "Mesa". When I attempted to build libRGL using my commercial XiG Accelerated-X (v4.1) Open-GL, derived from SGI's reference implementation, a compilation error occurred due to a type mismatch: "GL_GLKernel.cxx", line 183: error #167: argument of type "Int_t *" is incompatible with parameter of type "GLint *" {glGetIntegerv(GLCommand[mode],params);} ^ Here it appears that the GL's GLint and Root's Int_t types differ, and C++ refuses to implicitly convert a pointer to one built-in type to a pointer to any other distinct built-in type. Since I do not consider that an Open-GL interface is a requirement for a FNAL physics analysis package, I have not investigated this further. 10) Miscellaneous After this procedure, there are a few rootcint-generated header files to support object dictionaries in the source area. I have left them there for now. We should create a script to automate the process of generating a source tree from a CMZ file in preparation for entering it into a CVS repository. Some tailoring may be required to support alternative build conditions such as debug/optimize, exception support enable/disabled. We should create a script to drive a small canned test suite to be run at least before each release. KNOWN PROBLEMS It should be noted that most of these problems are translating the build procedures from CMZ to Makefiles. ROOT does build under CMZ for these systems, although I have not checked that it regenerates dictionaries under CMZ during compilation as I have attempted to do with Makefiles. 1) Linux dependency discovery requires an alternative makedepend, distributed in binary form. The Linux system's makedepend (from XFree86) does not treat .cxx files. 2) SGI dependency discovery appears to be incomplete when using the system's makedepend. Many objects involved in the build were not properly treated. I continued by using the Linux Make-depend file with relative paths. 3) Cint or rootcint emits errors while generating dictionaries for ROOT classes. In particular, for META_G__Meta.cxx (as well as X11_G__X11.cxx), under IRIX 6, the entire build fails with the following output: Generating dictionary META_G__Meta.cxx... Error: No symbol G__MAXFUNCPARA in current scope FILE:/spool/kennedy/root/include/common.h LINE:547 KCC --no_exceptions --signed_chars --display_error_number --diag_suppress 68 --diag_suppress 191 --diag_suppx "/spool/kennedy/root/include/TError.h", line 40: error #335: linkage specification is not allowed extern "C" void ErrorHandler(int level, const char *location, const char *fmt, ^ //... a number of warnings are emitted "/spool/kennedy/root/include/TError.h", line 34: warning #177-D: variable "kFatal" was declared but never referenced const int kFatal = 3000; ^ At end of source: error #67: expected a "}" This causes the Makefile-driven build under IRIX6 with Kai C++ v3.3 to fail outright. Cint and/or rootcint generate a poorly formed C++ source file which will not compile and lacks the object dictionary information. The IRIX6 build succeeded only when the dictionary files generated under Linux were used instead. Under Linux, one sees: Generating dictionary META_G__Meta.cxx... Error: No symbol G__MAXFUNCPARA in current scope FILE:/usr/scratch/kennedy/Root/Linux2/v2_00_11b/root/in> Error: No symbol G__MAXFUNCPARA in current scope FILE:/usr/scratch/kennedy/Root/Linux2/v2_00_11b/root/in> Error: No symbol G__MAXFUNCPARA in current scope FILE:/usr/scratch/kennedy/Root/Linux2/v2_00_11b/root/in> Error: No symbol G__MAXFUNCPARA in current scope FILE:/usr/scratch/kennedy/Root/Linux2/v2_00_11b/root/in> Error: No symbol G__MAXFUNCPARA in current scope FILE:/usr/scratch/kennedy/Root/Linux2/v2_00_11b/root/in> Error: No symbol G__MAXFUNCPARA in current scope FILE:/usr/scratch/kennedy/Root/Linux2/v2_00_11b/root/in> Error: No symbol G__MAXFUNCPARA in current scope FILE:/usr/scratch/kennedy/Root/Linux2/v2_00_11b/root/in> Error: No symbol G__MAXFUNCPARA in current scope FILE:/usr/scratch/kennedy/Root/Linux2/v2_00_11b/root/in> Error: No symbol G__MAXFUNCPARA in current scope FILE:/usr/scratch/kennedy/Root/Linux2/v2_00_11b/root/in> Error: No symbol G__MAXBASE in current scope FILE:/usr/scratch/kennedy/Root/Linux2/v2_00_11b/root/includ> Error: No symbol G__MAXBASE in current scope FILE:/usr/scratch/kennedy/Root/Linux2/v2_00_11b/root/includ> Error: No symbol G__UINT32security in current scope FILE:/usr/scratch/kennedy/Root/Linux2/v2_00_11b/root> Error: No symbol G__UINT32G__security in current scope FILE:/usr/scratch/kennedy/Root/Linux2/v2_00_11b/r> Limitation: macro handled as typedef char TWin32SendClass; FILE:/usr/scratch/kennedy/Root/Linux2/v2_00_11> While errors are emitted by cint and/or rootcint, under Linux the build continues and succeeds. 4) A few minor compilation errors are in the ROOTD and PROOFD source files under Linux and IRIX6. I patched these as documented above. 5) Canvases still require manual refresh. I understand this is caused by ROOT depending on the SaveUnder extension of the X-server being enabled. On most machines, this extension is disabled by default. A single application, in my opinion, should not determine how an X-server should be operated. Automatic refresh should be done if the X-server capability query indicates SaveUnder is not enabled, or it should always be done. 6) libNew malfunctions on IRIX6 when compiled with the Kai C++ compiler. The result is that while all libraries and executables build, the executables all crash immediately after start-up. This was rumored to be related to a problem with Kai C++ not permitting or respecting overloading of the global new and delete operators. I have shown in a small test program, however, that both the v3.2d and v3.3d IRIX6 Kai C++ compilers permit the override of global new and delete, without run-time crashes, at least in a small-scale example. Crashes can occur in the test example, for instance, if memory is allocated inside of the replacement global new operator by pushing strings to cout. CONCLUSION ROOT's current means of dependency discovery in Makefiles appears to be inadequate. The use of makedepend does not seem to be portable between operating systems, and requires an altered version for use under Linux. Makedepend, a part of the X11 distribution, was originally designed to handle the C language, not C++ or other languages. An approach similar to SoftRelTool's, in which the compiler itself is used to generate dependency information (using the -M switch) is far more robust and portable. My guess is that upgrading the dependency discovery phase in ROOT's Makefiles should take one to four FTE-weeks, and then require only a small amount of maintenance and support. Ignoring dependency generation issues, it took only a few FTE-days of my time to adapt ROOT V2.00/11 from its current state to one in which it can be built with Makefiles under Linux2-KCC_3_3 in a source tree that is ready for check-in to CVS. Considering my familiarity with ROOT (adjust estimate up for a random, reasonable person), the expected cleaning up of the Makefile issues by the ROOT team (adjust estimate down), and the ability to script most of the work after that clean-up (adjust estimate down), I would say that a reasonable person would take one FTE day to repeat this procedure, test the end-product with a small canned test suite on the first platform, and follow-up on any small problems. After that, it would take another one FTE-day to build (in parallel) and repeat tests across several more supported platforms. This does not include porting ROOT to new operating systems or compilers, only the translation from CMZ to CVS on a supported platforms. In the "worst" case, if we apply this one FTE-day/release adaptation to a CMZ-to-CVS transition plan in which, as an intermediate step, we create a new version branch in a FNAL CVS repository for each ROOT release, then we will spend about two FTE-days total per ROOT release for several supported Unix platforms. This cost be be higher if the various build variations (exceptions on or off) trigger different build or run-time behavior in ROOT. Nevertheless, this particular cost seems relatively small compared to others relating to physics analysis packages. These estimates do no include the effort required to research, report, and track any build and execution failures that may occur.