This document outlines the steps required to successfully upgrade ups from v3.x (or earlier) to v4.0 (a.k.a. erupt, a.k.a. upsII, a.k.a. Reconstructed UPS for Run II Computing), or to install ups v4.0 from scratch. It is assumed that the persons performing the upgrade have some familiarity with the syntax of ups commands (ups declare, ups configure, etc.), and that they know the current ups configuration (i.e., how is $PRODUCTS defined, where are products installed, how are product areas mounted cluster-wide, what special configurations exist (i.e., per-node databases), etc.). Additionally, the person performing the upgrade will need to know the desired new configuration for ups.
Note: this document describes the ups upgrade process in very general terms. Details such as specific directory names, general naming conventions, product version numbers, etc., are not to be taken as anything other than guidance, written at a particular point in time. In other words, while this document talks about ups v4.0, you may actually be installing ups v4.3, etc.
A much briefer installation checklist summary can be used once you are familiar with the concepts presented herein.
If you want to leave a back-door for people so that they can use the old-style ups after the conversion (probably a good idea), now would be a good time to prepare for it. For example,
$ cd /usr/local/etc
$ cp fermi.shrc fermi.ups1.shrc
$ cp fermi.cshrc fermi.ups1.cshrc
$ cp setups.sh setups.ups1.sh
$ cp setups.csh setups.ups1.csh
Then edit the new fermi.ups1.[c]shrc files so that they call setups.ups1.[c]sh instead of setups.[c]sh.
Users who want to take advantage of this feature can modify their .[c]shrc files. For example,
#
# use old-style ups if the file $HOME/ups1 exists:
if [ -r $HOME/ups1 ]
then
. /usr/local/etc/fermi.ups1.shrc
else
#
# Setup UPS
#
if [ -r /usr/local/etc/fermi.shrc ]
then
. /usr/local/etc/fermi.shrc
fi
fi
Eventually we hope that these will be collected into a product bundle named coreFUE with its own installation script that automates much of what is listed below; however, this hasn't been implemented yet. For the present, use the old upd to obtain and unwind the tarfiles.
Unwind the tarfiles into the location where the product files will be installed in the new ups (i.e., into its correct $UPS_DIR location).
Warning! Most existing systems on site are already using the /usr/products/flavor/product/version convention, with their ups database at /usr/products/ups_database/declared. (i.e., PROD_DIR_PREFIX is /usr/products). Changing this convention may involve lots of things not directly related to ups: NFS mount points, automount configurations, backup schemes, disk partitioning, etc. We plan to change both the AFS and KITS configurations to be PROD_DIR_PREFIX/product/version/flavor, and we suspect that on certain multi-flavor development clusters this would be a beneficial change. The new upd contains features that will make maintenance of such a configuration much easier. But do not minimize the amount of effort that would go into reconfiguring an existing system to use a different naming convention!
If you do not have the luxury of reconfiguring your system, we suggest using the conventions:
However, the new recommended naming convention, if you have the luxury to change your system configuration (or if this is a new installation of ups), is:
In most cases, within a cluster, different flavors of products/nodes will share the same ups database. If you are using more than one ups database, the keywords SETUPS_DIR and UPD_USERCODE_DIR will be used only from the one database in which ups/upd are declared.
SETUPS_DIR considerations:
The ups initialization files (setups.[c]sh, setups.cache.flavor, etc.)
will be copied to this location when ups is declared current. The users will source the initialization
scripts from here. The upsdb_list file (list of ups_databases on your system)
is also located in this directory.
For nodes using cmd adduser and other Fermi systools,
the natural location would be /usr/local/etc, but this is discouraged.
If SETUPS_DIR is set to /usr/local/etc
it is inherently not cluster-common, and you will need to maintain
multiple copies of the upsdb_list file (as well as keeping all of your
setups.[c]sh files in synch across the cluster).
We recommend that you create a cluster-common SETUPS_DIR which is NOT in /usr/local/etc. The suggested naming convention is /fnal/ups/etc. We strongly recommend that all on-site nodes create setups.[c]sh links in /usr/local/etc for backwards compatibility. (If you do not create these links, you will need to clean out existing files and modify any existing Fermi login files to point to the new setups.[c]sh files; you will also need to tell your users to modify any of their scripts to point to the new setups.[c]sh files).
UPD_USERCODE_DIR considerations:
The upd configuration file(s) live here, as well as the user-callout updusr.pm module. We expect that the
default updusr.pm module will handle nearly all of the various upd configurations in use; but if you
need to, you can add your own by editing this file.
We suggest that UPD_USERCODE_DIR live in ${UPS_THIS_DB}/.updfiles, but this is not a requirement.
Steps in Creating and Configuring the New UPS Database:
The statistics directory must be world-writeable; if you aren't keeping any product statistics, you don't need the directory.
$ mkdir /path-to-upsdb
$ mkdir /path-to-upsdb/.upsfiles
$ mkdir /path-to-upsdb/.upsfiles/startup
$ mkdir /path-to-upsdb/.upsfiles/shutdown
$ mkdir /path-to-upsdb/.upsfiles/statistics
$ chmod 777 /path-to-upsdb/.upsfiles/statistics
$ cp /usr/products/ups_database/startup/* \
/path-to-upsdb/.upsfiles/startup
$ cp /usr/products/ups_database/shutdown/* \
/path-to-upsdb/.upsfiles/shutdown
As a template, you can use the ups dbconfig.template file from the ups directory of the new ups product:
$ cp /usr/products/ups/v4_0/IRIX+5/ups/dbconfig.template \
/path-to-upsdb/.upsfiles/dbconfig
$ ${EDITOR} /path-to-upsdb/.upsfiles/dbconfig
Example dbconfig file:
FILE = DBCONFIG
# all nodes can read/use the products declared in this db
AUTHORIZED_NODES = *
# all product roots are under /usr/products
PROD_DIR_PREFIX = /usr/products
# keep statistics about the following products:
STATISTICS = ups:upd:perl
# manpages, info files, etc. go here:
MAN_TARGET_DIR = /usr/products/man
CATMAN_TARGET_DIR = /usr/products/catman
INFO_TARGET_DIR = /usr/products/Info
# html and news not implemented in this release.
HTML_TARGET_DIR = /dev/null
NEWS_TARGET_DIR = /dev/null
# this db contains the ups product; copy the setups.[c]sh
# and cached files to this directory
SETUPS_DIR = /fnal/ups/etc
# this db contains the upd product; look for the upd
# configuration files here:
UPD_USERCODE_DIR = /path-to-upsdb/.updfiles
This file contains a list of all potential ups_databases on this system. It may contain comments, but only if they begin with a "#" in the first column. It may contain ups_databases that are not mounted on this node (e.g., so that there is one common upsdb_list file for an entire cluster, even if they have different databases) -- any ups_databases that do not exist when setups.[c]sh is called will not be appended to $PRODUCTS.
Example upsdb_list file:
#
# Sample upsdb_list file.
#
${Node}/products/ups_database
/fnal/ups/db
/afs/fnal/ups/db
#
# Example of a database relative to a specific user's $HOME
#
# ~username/ups_database
#
# Example of a database referenced via an environment variable
#
# ${ENV_VARIABLE}/ups_database
Users may also set the ${UPS_EXTRA_DIR} environmental variable to prepend their own ups database(s) to the front of the system-wide lists (e.g., developers may prepend their development database).
As a template, you can use the updconfig.template and updusr.pm.template files from the ups subdirectory of the upd product. You should not need to modify the example updusr.pm file for most configurations; you will need to edit the updconfig file. The most important keyword is UPS_PROD_DIR, which determines the product root naming convention on your system.
$ cp /path-to-upd-v4/ups/updusr.pm.template \
/path-to-upsdb/.updfiles/updusr.pm
$ cp /path-to-upd-v4/ups/updconfig.template \
/path-to-upsdb/.updfiles/updconfig
$ ${EDITOR} /path-to-upsdb/.updfiles/updconfig
Example updconfig file:
File = updconfig
GROUP:
product = ANY
flavor = ANY
qualifiers = ANY
options = ANY
COMMON:
UPS_THIS_DB = /fnal/ups/db
UPS_PROD_DIR = "${PROD_DIR_PREFIX}/${UPS_PROD_NAME}/${UPS_PROD_VERSION}/${UPS_PROD_FLAVOR}${UPS_PROD_QUALIFIERS}"
UNWIND_PROD_DIR = "${UPS_PROD_DIR}"
UPS_UPS_DIR = "ups"
UNWIND_UPS_DIR = "${UNWIND_PROD_DIR}/${UPS_UPS_DIR}"
UPS_TABLE_FILE = "${UPS_PROD_VERSION}.table"
UNWIND_TABLE_DIR = "${UPS_THIS_DB}/${UPS_PROD_NAME}"
UPS_TABLE_DIR = "${UNWIND_TABLE_DIR}"
END:
The new ${UPS_DIR}/bin/ups_convert script will convert your old ups database to the new format. All of your existing products can continue to be used as-is under the new ups, with no modification to the products themselves. The ups_convert modifies only the ups database, generating "default" table files which call the old-style setup scripts (and other ups files). Note, you cannot merge databases with the ups_convert. You must convert each old database into one brand new database.
Each database should be converted by the appropriate owner of the corresponding new database. Each database must be converted from a node/account which can read all of the ups directories of the products declared in that database.
Make sure that you are using the version of perl that you're trying to install, rather than some other perl that is on your system (e.g., something provided by the vendor). Do this by "faking" a setup of the new perl.
Set the OLD_PRODUCTS environmental variable to point to the old ups_database you want to convert. Set PRODUCTS to point to the new ups_database. (This directory must exist). Set UPS_DIR to point to the new soon-to-be UPS product root directory. Then run ${UPS_DIR}/bin/ups_convert.
Example ups database conversion:
PERL_DIR=/usr/products/perl/v5_004/IRIX+5; export PERL_DIR
. $PERL_DIR/ups/setup.sh
UPS_DIR=/usr/products/ups/v4_0/IRIX+5; export UPS_DIR
OLD_PRODUCTS=/usr/products/ups_database/declared; export OLD_PRODUCTS
PRODUCTS=/path-to-upsdb; export PRODUCTS
${UPS_DIR}/bin/ups_convert
# you should move the declarations of the
# OLD-style ups and upd out of the NEW-style
# database to prevent confusion:
cd $PRODUCTS
mv ups ups.old
mv upd upd.old
#
# after the conversion, you may wish to hide
# the obsolete pieces from the old ups database
# (and delete them later when you're sure they're
# no longer needed):
#
cd /path-to-upsdb
mv declared .declared-old
mv startup .startup-old
mv shutdown .shutdown-old
At this point, you may beta test the new
ups prior to cutting over cluster-wide.
This complicates the
upgrade procedure enormously -- kind of like stopping in the middle of an OS upgrade.
Nevertheless, it is possible to beta test the new ups and upd before going "live"
if you wish.
cd $HOME
mv .shrc .shrc-save # prevent confusion with old ups
UPS_SHELL=sh; export UPS_SHELL
UPS_DIR=/usr/products/ups/v4_0/IRIX+5; export UPS_DIR
PRODUCTS=/path-to-upsdb; export PRODUCTS
. ${UPS_DIR}/ups/setup.sh # get the additional ups functions/aliases
${UPS_DIR}/bin/ups declare \
-r ${UPS_DIR} \
-2 \
-m ups.table \
ups v4_0
${UPS_DIR}/bin/ups declare \
-r /usr/products/perl/v5_004/IRIX+5 \
-2 \
-m perl.table \
perl v5_004
${UPS_DIR}/bin/ups declare \
-r /usr/products/upd/v4_0/NULL \
-0 \
-m upd.table \
upd v4_0
cd $HOME
mv .shrc-save .shrc # restore shrc file
UPS_SHELL=sh; export UPS_SHELL
PRODUCTS=/path-to-upsdb; export PRODUCTS
UPS_DIR=/usr/products/ups/v4_0/IRIX+5; export UPS_DIR
. `${UPS_DIR}/bin/ups setup ups v4_0`
setup upd v4_0
If you have an extended beta test period during which changes might have been made to your old ups database, we recommend that you reconvert the old database at the end of the testing period. You will need to redeclare ups, upd, and perl into the reconverted new database!
Do not declare it "current" until you are done with any beta testing!
The database into which you declare ups should be the database which specifies SETUPS_DIR. You must be in an account which can write into the SETUPS_DIR directory.
Flavor is no longer optional on the ups declare command. You must specify the flavor when declaring the product. You can either use the old syntax:
-f flavor
or the new syntax (-0, -1, -2, or -3), where:
-0 = -f NULL
-1 = -f MajorOS (e.g., -f IRIX or -f OSF1)
-2 = -f MajorOS+MajorRev (e.g., -f IRIX+5 or -f OSF1+V3)
-3 = -f MajorOS+MajorRev+minorRev (e.g., -f IRIX+5.3 or -f OSF1+V3.2)
New required command line element: -m table_file:
the name of the table file for this product.
This must be specified if the product has a table file (the default is NO table file).
New optional command line element: -M table_dir: the directory in which the table file for this product lives. Does not need to be specified if the table_file resides in one of the following default locations:
In the case of the tarfile that you've obtained from kits, the table file for ups is named ups.table and it resides in the ups directory.
Example first-time declaration of ups v4_0:
cd $HOME
mv .shrc .shrc-save # prevent confusion with old ups
UPS_SHELL=sh; export UPS_SHELL
UPS_DIR=/usr/products/ups/v4_0/IRIX+5; export UPS_DIR
PRODUCTS=/path-to-upsdb; export PRODUCTS
. ${UPS_DIR}/ups/setup.sh # get the additional ups functions/aliases
${UPS_DIR}/bin/ups declare \
-r ${UPS_DIR} \
-2 \ # recommended, and equivalent to -f OS.major-release
-m ups.table \ # new requirement!
-c
ups v4_0
${UPS_DIR}/bin/ups declare \
-r /usr/products/perl/v5_004/IRIX+5 \
-2 \ # recommended, and equivalent to -f OS.major-release
-m perl.table \ # new requirement!
-c \ # THIS WILL TAKE A *LONG* TIME!!!
perl v5_004
${UPS_DIR}/bin/ups declare \
-r /usr/products/upd/v4_0/NULL \
-0 \ # recommended, and equivalent to -f NULL
-m upd.table \ # new requirement!
-c \
upd v4_0
cd $HOME
mv .shrc-save .shrc # restore shrc file
If SETUPS_DIR is set to something other than /usr/local/etc,
you should now create the backwards compatible links to the setups.[c]sh files:
cd /usr/local/etc rm setups.sh rm setups.csh ln -s /path/to/setups_dir/setups.csh ./setups.csh ln -s /path/to/setups_dir/setups.sh ./setups.shIf you do not create these links, you will need to edit the fermi.[c]shrc files to change /usr/local/etc/setups.[c]sh to /path-to-SETUPS_DIR/setups.[c]sh. (And you should remove any existing setups.[c]sh scripts from /usr/local/etc).
You should also create backwards compatible links for funame and dropit in /usr/local/bin. These products are maintained in the new ups product, but previous products may be looking for files in /usr/local/bin:
cd /usr/local/bin
rm funame
rm dropit
ln -s ${UPS_DIR}/bin/funame ./funame
ln -s ${UPS_DIR}/bin/dropit ./dropit
The perl declaration will take a rather long time. Be patient. It comes with 512 man pages which need to be copied, slowly and carefully, to the MAN_TARGET_DIR and CATMAN_TARGET_DIR directories.
When you declare ups current, the setups.[c]sh (and related) scripts are copied into SETUPS_DIR. If SETUPS_DIR is /usr/local/etc, anybody who logs in after this time will get the new ups, not the old. If users are already logged in, and their PRODUCTS is set to the old value, but they source the new setups.[c]sh files, they will be in a very confused state.
For this reason, we recommend that you reboot after declaring the ups v4_0 "current".
This is configured on a system-by-system basis; your system may or may not
be using these locations for the ups startup files.
AIX: Look in /etc/rc.local for files that call ups start directly, or
the ups start script in the old ups database area. Modify this file so that it calls
the ups startup script from the new location (e.g., /path-to-upsdb/.upsfiles/startup/ups_startup.csh).
IRIX, OSF1, SunOS, Linux: Look in /etc/init.d for a script with the name "ups" in it, which is
linked from something like /rc2.d/S99ups. Modify this file so that it calls
the ups startup script from the new location (e.g., /path-to-upsdb/.upsfiles/startup/ups_startup.csh).
???
This is not yet documented, but it should be fairly easy. Something on the order of
setup upd
for product in `ls $PRODUCTS`
do
upd update $product
done
(but there may be a little more complication due to the differences between flavors on your cluster
and flavors in fnkits).