Back to documentaton overview

Integrating buildmanager, cvs,  template_product, and upd

Abstract

This document will describe using several tools for building ups packages for multiple platforms.  While these tools can be used in isolation, they were designed to be used together to provide the following benefits:
  • Keeping sources in sync between build areas for multiple platforms
  • Doing builds for several platforms in parallel
  • Maintaining revision history and revision control of sources.
  • Extending the base of products that can be built and tested in a standardized fashion.
  • Overview

    We will be describing these tools and how to use them together by way of building an example product,
    demo, and in the process give an introduction to several of the tools involved.  In particular, we will examine the following in detail:
  • creating a prototype of your product file tree.
  • placing the file tree in a CVS repository
  • establishing sessions on multiple build machines
  • creating a working copy for each session
  • building and testing your software in parallel
  • making changes to your file trees
  • distributing your software via upd
  • If you are logged into one of the nodes on the OSS cluster build machine, and have the appropriate permissions already, you should be able to go through the example in this document step by step and build and release the demo product in order to familiarize yourself with the process.

    Creating a prototype of your product file tree

    This is really the main role of the template_product package for most users; getting a
    skeleton of a product together quickly.  While the web page for the template_product has
    examples of  using it to make products out of scripts or precompiled binaries, here we will discuss using it to make products out of  packages that are compiled from source code.  This generally involves somewhat more customization than the case where no compilation is required, however it starts off very much the same way.  We begin by: Once this is completed, we will have a prototype product tree that we can put in a Fermilab cvs repository.

    This looks like the following, with what a user types in bold face (and [tab] is a Tab on your keyboard):



    <build_host> # Setup the product
    <build_host> setup template_product
    <build_host> # make the directory
    <build_host> mkdir demo
    <build_host> cd demo
    <build_host> # run the script and answer the questions
    <build_host> CloneTemplate
    Product name? demo
    Product version? v0_0
    Platform specific product [yN]? y
    Dependent products [list as fred:joe:harry]?
    installing template product files in .../demo
    .../demo/ups
    .../demo/ups/Version
    .../demo/ups/INSTALL_NOTE.template
    .../demo/ups/template_product.table
    .../demo/ups/.manifest.template_product
    .../demo/Makefile
    .../demo/test
    .../demo/test/TestScript
    <build_host> # make a source directory
    <build_host> mkdir src
    <build_host> cd src
    <build_host> # add a source file and makefile
    <build_host> echo '#include <stdio.h>' > hello.c
    <build_host> echo 'main(){ printf("hello world\n"); }' >> hello.c
    <build_host> echo 'all: hello' > Makefile
    <build_host> cd ..
    <build_host> # edit the toplevel makefile to build and install "hello"
    <build_host> ed Makefile
    17373
    /^all:/
    all: proddir_is_set build_prefix
    a
    [tab]cd src; make ; cp hello ../bin
    .
    w
    q
    <build_host>

    Of course, you will probably want to use  your favorite editor rather than ed, but it is notoriously
    hard to show other editors in use in a document.  We now have a reasonable first approximation of the file tree we want to have under version control.

    Putting the file tree into a cvs repository

    Since we will be using cvs to maintain and track our sources, before doing much else, let us
    go ahead and put this file tree into a repository using cvs import.  If you are not already familiar with cvs you may want to look at the online documentation , in particular the sections  What is CVS?, A sample session, and Starting a new project.  In our case we will be using the second approach from the
    latter manual section, and importing a whole file tree that we have just created.
    Before we can do this, however,  we must have a CVS repository into which we can place this source code module for our product.   On most Fermilab development systems, we accomplish this by setting up a ups
    product like uascvs or cdcvs: setup uascvs Actually adding the module is accomplished with a simple command line:
    >cvs import -m "demo product" demo Fermilab username_version which will take the current directory tree and install it into our cvs repository as version username_version.
    I reccomend you actually put your username in there, so that several people following these instructions will check in different versions of the demo product sources.

    If the above command gives you an error like:
    Permission denied.
    cvs [import aborted]: end of file from server (consult above messages if any)
    you will need to contact the administrative contact for the repository, which should be
    the first part of your CVSROOT environment variable -- for example if your CVSROOT
    is cvsuser@oss-cvs.fnal.gov:/usr/products/olsshare/cvs you would send mail to
    cvsuser@oss-cvs.fnal.gov to get permission to access that repository.
     

    Establishing sessions on build machines

    We can now start working on several machines in parallel, in order to speed up the process of getting the software built on several machines.  The tool I prefer for his work is a package called buildmanager, which  is an expectk script to run login sessions on multiple machines.   To run it:


    <build_host> # Setup the product
    <build_host> setup buildmanager
    <build_host> # start buildmanager
    <build_host> buildmanager demo username_version


    This will pop up  a login window:
    partial screen image
    and an interaction window:
    partial screen image
    You can see the list of hosts you will be connecting to, and click the checkboxes to
    the left of each host to include them or not.   Once the host list is to your liking, enter your
    login name and password in the password window,  and hit enter in the password field,
    or press the Dismiss button to start logging into the various systems.  More details
    on using buildmanager can be found by clicking on the [Help...] button, or on the
     buildmanager web page .

    Checking out a copy on each platform

    To get a working copy for each platform build, you can simply pick cvs checkout from the Commands menu, and buildmanager will change directories to the designated scratch area on each system and issue the appropriate commands.  You probably  then want to select cvs update -A from the same menu in
    order to clear up the "sticky" tags that checking out a specific revision creates (see the bottom of  this page  in the cvs manual).
     

    Doing a test build

    For early stages of development you probably want to do this in several stages, first setting up
    the product for build by choosing setup -q build, then doing a compilation by doing a make all,
    and run any provided tests with make test, all of which can be chosen from the Commands menu.

    Propagating changes with cvs

    If you make a change in one working copy of the sources, and want to get the copies all the same, first do a cvs commit -m "some reason" in the window for the host where the change was made, and then do
    a cvs update in all the windows (either from the entry box at the top of the buildmanager window, or via the Commands menu.)
     

    Distributing your product

    Once your product works to your satisfaction, you can distribute it by doing a make kits from the
    Commands menu, which will take turns running that make command in each window to add the
    product to a distribution system as defined in the Makefile.