Fermilab
Fermilab
Matthew J. Wicks January 19, 1993
Marc W. Mengel February 15, 2001
The Core Services and Infrastructure Department (CSI) provides and supports several products that are part of the core Fermilab UNIX environment. Some of these utilities have been written in-house, while others are public-domain software that have been tested on the four major UNIX platforms (i.e. IRIX 6.5, SunOS 5.6, OSF1 4.0, and Linux 2.2). [Note that we refer to these operating systems by the names they report via uname.] Requests for porting to other platforms as needed at Fermilab will be considered.
The purpose of this Guide is to provide the necessary information for the end user to configure and use the utilities. Some of these utilities are system administration utilities and thus the end user is the local system manager, while other utilities are geared for the general user. In general, any configuration of the utilities must be performed by the local system manager, although some tools can be configured individually by the user.
The utilities provided by CSI that are documented in this guide currently are divided into two product bundles, futil and systools which are intended to be installed and declared to the ups database on the system. CSI also supports a general purpose backup facilty, fmb (Fermi Modular Backup), which we are in the process of phasing out, in favor of amanda (University of Maryland Network Dump Archiver), which are documented in The fmb Users Guide and Amanda at Fermilab respectively.
All of these products are available using upd from the central distribution machine fnkits.fnal.gov a.k.a. ftp.fnal.gov. The following indicates the latest versions of the products, and their subsidiary products:
systools v6_1
|__cmdbin v1_1
| |__cmdbincmd v1_1
| |__cmdbingen v1_1
| |__cmdbinutl v1_1
|__cmdscripts v1_1
| |__cmdscriptsbase v1_1
| |__cmdscriptsctrl v1_1
| |__cmdscriptsguid v1_1
| |__cmdscriptslgrp v1_1
| |__cmdscriptshell v1_1
| |__cmdscriptsuser v1_1
| |__shells v3_1
| | |__bash v2_03
| | |__tcsh v6_09
| |__template_home v1_3
| |__login v1_6
| | |__setpath v1_10
| | |__setterm v1_4
| | |__setxwin v1_12
| |__shrc v1_10
|__systoolsutl v1_3
futil v6_0
|__Info v1_5a
|__flpr v1_12
|__psutils vp_17_1
|__telephone v5_2
|__stock v1_1
|__gtools v2_2
It is reccomended that systools and futil be installed using upd, as it will automatically install the whole dependency tree depicted above; manual installation by downloading tarfiles, table files, and ups directories by hand would be tedious, to say the least.
Many of the files for the systools product eventually must be copied to the cmdscripts product directory ($CMDSCRIPTS_DIR) as root. The ups InstallAsRoot systools action has been provided with the product that will perform the file copies as well as insure that the files have correct ownership and permissions. In addition, the script will save any critical configuration files that were used in a previous version of the product. A reminder to run this action is printed when you make the product current.
You must be logged in as root when the InstallAsRoot action is executed since it installs some setuid root programs and changes the ownership of various files. Several of the products (the login shells tcsh and bash, and the perl product) will also copy files into /usr/local/bin, if permissions allow.
As distributed, the products should not need to re-compiled. However, source code has been provided in the event that you need to re-build a product. After re-building a product, you need to execute the appropriate ups InstallAsRoot action in order to install the appropriate files.
The following is an example of how to re-build the cmdbin product. Other products are re-built in a similar manner.
The futil product is a set of user level utility programs. Some of these programs were written in-house such as flpr, Info and stock while others are public domain programs that have been tested on the various architectures.
The flpr program is a Fermi implementation of the standard UNIX lpr utility. It is used to submit jobs to Fermilab sitewide print queues. Its usage, along with the associated programs flpq and flpk, are documented fully in the flpr manual page and will not be repeated here.
The following precedence (from highest to lowest) is used to determine items such as printer, host, and nicknames:
Command line arguments allow the specification of the host with the -h option and the printer queue with the -q option. In addition, using the -P option you can specify a nickname that is a combined host and printer queue. Nicknames are specified in either the $HOME/.flprc or /usr/local/etc/flp.defaults files.
Environment variables allow the specification of the host via the FLPHOST variable and printer queues via the FLPQUE variable.
The $HOME/.flprc and /usr/local/etc/flp.defaults files have the same syntax. To specify a host, include a line with host <hostname>. To specify a printer queue, include a line with queue <queuename>. To specify a nickname (combined host and printer queue, include a line with printer <nickname> <hostname> <queuename>.
More detailed information can be found in the flpr manual page.
The less program is a sophisticated public domain paging program, that many people feel is superior to the more paging program. Its use is fully documented in the less manual page.
The Info program allows posting of information to the general user community. The permissions of the /usr/Info directory determine which users can submit, delete, and purge articles.
Info provides a number of different command line arguments in order to add, display, delete, etc. articles. The following text is the help message from Info which documents all the different options
Info may take any one of the following arguments as requests:
-help requests this help text tag(s) requests presentation
of selected articles by tag
-display tag(s) requests presentation of selected articles by tag
-anynew requests message about number of new articles
-any requests message about number of articles
-anynewcode requests exit status about number of new articles
-anycode requests exit status about number of articles
-new requests list of new articles
-list requests list of articles
-add requests prompting to add new article from text file
-replace requests prompting to replace existing article from
text file
-insert requests prompting to insert article from text file
-Add tag headline expiredate textfile
-Replace tag headline expiredate textfile
-Insert tag headline expiredate textfile
requests corresponding batch operation
-del requests the removal of a specific article by tag
-delete requests the removal of a specific article by tag
-del tag requests the removal of a specific article by tag
-delete tag requests the removal of a specific article by tag
-purge requests the removal of expired articles
If none of the above are supplied, the default is -list. A tag is a short (1 to 20 character) article identifier. A headline is a short (1 to 32 character) description of the contents. A expire date is a date (see getdate), 'standard' (or 'std'), or 'never'. The expiration date can not be earlier than today. A text file is a existing readable readable file from which the text for an article is to be copied.
A new article has a time stamp newer than your $HOME/.Info file. If you do not have a $HOME/.Info file then all articles are new.
The article modification operations (-add, -replace, -insert, -Add, -Replace, -Replace, -Insert -delete and -purge) require that you have permission to change the /usr/Info directory.
When adding an article (through Info -add) the user is prompted for the additional information required. This information includes article name, article headline, and expiration date. The expiration date can be added in a number of formats. For a complete description of valid formats, see the documentation for getdate.
In order to purge articles, it is suggested that a cron job be setup for an account that has write permission in the Info directory. The following cron job would purge articles once a week early Sunday morning
05 01 * * 0 /usr/local/bin/Info -purge
The third DCD product, systools is a series of system administration utilities. To use the utilities both the funkern and futil products must be installed. If you are going to use the addproduct, delproduct, and modproduct utilities from this package, you must create an account with a login name of kits, with a UID of 1628, and a group of upd, with a GID of 3531 You must create this account prior to installing systools.
There are several configuration files that control the operation of the systools package. When you install systools default versions of these files are installed. Before using the utilities you must configure these files to reflect the specifics of your local system. In addition, you may need to change these files from time-to-time as part of the ongoing system management.
The following sections describe the format and purpose of each configuration file.
The machine specific definitions primarily are variables that indicate the location of specific system programs. For example, the location of the wc program is different on the various UNIX platforms and so the WC variable is used to indicate the location of that program. The definitions of these programs should be correct upon installation.
The YPDIR variable is contained at the beginning of the machine specific definitions. This variable is used for two purposes. The first purpose is to indicate if the system is running NIS (Network Information Service, the new name for yellow pages). Uncommenting this variable indicates that NIS is being used on the system. The second purpose of the variable is to indicate the base directory for NIS since the location is machine dependent. It is not necessary to specify the directory location if NIS is not being used on the system.
The PANDORA variable is used to indicate that utilities that install or modify passwords should not age the password. The variable should be defined if for some reason your system can't handle password aging. One instance of a system that can't handle password aging is an SGI system running pandora (hence the name of the variable). Pandora is the program on the SGI console that displays login icons instead of the normal login: prompt.
In general, this file will be edited once upon initial configuration of the system administration utilities and then left unchanged. However, if you install NIS on your systems, you would need to edit this file and re-define a few variables. Once again, the comments in the file should indicate the correct value of each variable.
Two important variables are PASSWD and GROUP. These variables are used to indicate the names of the file that should be edited when accounts are added or modified. Normally, PASSWD is set to /etc/passwd and GROUP is set to /etc/group. However, if for some reason a different file is used for the password and group files then you should change these variables. One of the more common reason for a change occurs when running NIS. It is very common for the NIS master server to use a different file (i.e. /var/etc/passwd or /etc/passwd.yp) as the password file. In addition, if either of these variables is set to the value of NULL then the systools utilities will not allow the system to modify the password or group files. This configuration would be desirable if your system is running NIS, but is not the master server.
The end of the sys.env file contains definitions used by the addproduct, delproduct, and modproduct utilities. Any change of the variables must be coordinated with all machines using these utilities to provide products to the central UPD distribution machine.
utilities.
The getlock program is a simple locking program that allows two or more programs to cooperate with each other and lock-out the other program from performing a certain function. For example, the various utilities in systools that modify the password and groups file use this program in order to insure that only one program that modifies these files is run at a time.
The usage of the program is: getlock pid directory filename
The getlock program now attempts to create a file of the specified name in the specified directory. The contents of the file are the value of pid. When used from a shell script, the special variable $$ is used when calling the program since that is the value of the current process id.
If the file already exists, getlock reads its contents to determine the pid value from an earlier successful invocation. If this process still exists, then getlock fails. If the process no longer exists, then getlock succeeds and replaces the contents of the file with the new pid.
Using standard shell conventions, when getlock is successful, a value of 0 is returned. A non-negative value is returned for failure.
This locking mechanism doesn't require programs to remove the lock file since when the process no longer exists, the lock file is no longer considered valid.
The obtain and namefix programs are used to retrieve a copy of the FNAL global UID and GID databases, and to get the default lpr and flpr printer queue configuration files. The obtain program is used to retrieve the files via HTTP from www-giduid.fnal.gov, or fnprt.fnal.gov; while smushuid and smushgid change the output of the files into a format more easily parsed by various shell programs. smushuid and smushgid are called by the obtain program and thus are not normally used directly by a user.
The obtain program allows one of four arguments corresponding to two different formats of both the UID and GID database. The files are placed in the current working directory. If the current copy of the local file is less than a day old, the obtain program asks if you want to retrieve a new copy of the file.
The arguments and corresponding local file names created are:
There are a series of programs beginning with log that print out a specific field in the password file for the given user. In addition, there are a series of commands beginning with grp that print out information in the group file. The log commands take either the account name or uid as an argument. The grp commands take either the group name or gid as an argument. The full list of these commands are:
The directory $TEMPLATE_HOME_DIR/dotfiles.localized contains templates for system wide defaults for the following user files.
These files use files from $SETUPS_DIR, and products to setup the users standard environment
These files and products are a critical part of the Fermi UNIX environment. They provide a consistent login environment across the various UNIX platforms and it is hoped that the local system manager has no need to change these files. The setups.* files are used to setup a ups environment. This can be done directly by scripts or the root account to obtain a ups environment without executing other aspects of the fermi files. Once this is setup, the remaining standardized setup is done by setting up ups products, with dependencies.
The files placed in the template_home product are placed there so the local system manager can inspect their contents before declaring the product current. If the system manager has made any local changes, the product should be declared with a localized version name, and then declared current. Utilities such as adduser and clearaccounts copy files from the current template_home product directory in setting up the account. In addition, the login files files source their files from $SETUPS_DIR.
The file $CMDSCRIPTS_DIR/library/greeting is a greeting message mailed to users by the adduser program when a new account is added. Change this file as appropriate.
The file $CMDSCRIPTS_DIR/library/functions determines which accounts will be allowed to run the various commands in $CMDSCRIPTS_DIR/library. Furthermore, it allows you to indicate which UID and GID will execute the command. Thus, you can allow people other than the super-user (root) to run certain commands as root (or some other UID). The format of the lines are:
commandname:ruid,euid,rgid,egid,loginlist
The first field (commandname) is the name of the program in $CMDSCRIPTS_DIR/library. The second field is four (4) comma separated numbers of the real and effective user id (UID) and group id (GID) that will be used to run the program. In almost all cases, these numbers will be zero (0). The third field is a comma separated list of accounts that can execute the given command. The following is an example functions file along with an explanation.
addmember:0,0,0,0:root,*gm adduser:0,0,0,0:root,xxx
chguser:0,0,0,0:root,xxx deldata:0,0,0,0:root,xxx
delmember:0,0,0,0:root,*gm disuser:0,0,0,0:root,xxx
moduser:0,0,0,0:root,xxx resetuser:0,0,0,0:root,xxx
vigr:0,0,0,0:root vipw:0,0,0,0:root finduser:0,0,0,0:+*
clearaccounts:0,0,0,0:root,xxx,yyy addlocal:0,0,0,0:-local
dellocal:0,0,0,0:-local
rm:0,0,0,0:root chmod:0,0,0,0:root kill:0,0,0,0:root
renice:0,0,0,0:root
In all cases the commands will be executed with real and effective UID and GID of 0 (i.e. ruid,euid,rgid,egid is equal to zero). In the case of addmember, both root and group managers can execute this command (denoted by *gm). A group manager account is defined as an account whose login name is the same as the group name. In the case of adduser, both root and the xxx account can execute this command. The clearaccounts command can be executed by root, xxx, and yyy and the finduser command can be executed by anyone (denoted by *).
Any item in the loginlist (including * and *gm) can be preceded with a + or - sign. The following describes the semantics:
The functions file should be mode 640 (read and write permission to owner, and read permission to group) and should have the same owner and group as /usr/local/bin/cmd.
In addition, you may create a file called functions.hostname (i.e. functions.fnsg01) in the $CMDSCRIPTS_DIR/library directory. This file is identical to format, but takes precedence over the functions file. If no entry for a function is found in the functions.hostname file or it doesn't exist, the functions file is then searched.
The file $CMDSCRIPTS_DIR/library/adduser.work/disks is used by the adduser program to determine where accounts should be placed. (Note: You may want to copy the version in /systools/library/adduser.work from a previous version of systools.) Each line in this file contains a directory name followed by a list of GIDS and UIDS. The following is a simple example of a disks file:
/usr/people,g=1234,u=456, /usr1,g=1256, /usr2,misc,g=1489,
This file indicates that if the GID is 1234 or the UID is 456 the account should be placed in /usr/people. If the GID is 1256 the account should be placed in /usr1. If the GID is 1489 the account is placed in /usr
The first example shows how different disk partitions can be allocated to different experiments. A simpler example of the disks file would be to place all accounts in the same area. The following line would handle that case:
/usr/people,misc,
The clearaccounts program uses a data file to determine which logins may be cleared. These files are of the form $CMDSCRIPTS_DIR/library/loginlist.logname. Thus if the xxx account had permission to use clearaccounts, a file should be created in $CMDSCRIPTS_DIR/library with the name loginlist.xxx which contained a list of logins that xxx had permission to clear. This list should contain one login per line. These files should be mode 640 with group and ownership the same as /usr/local/bin/cmd.
The addmember and delmember programs look for a data file to determine which groups a given user may modify. These files are not required as documented in the section for addmember and delmember.
These files are similar in format to the loginlist files except they contain a list of group names instead of a list of logins.
These files contain as their extension an account name and use the cwi syntax to specify lists of users/groups. Thus the file rm.work/user.wicks would contain a list of users the wicks account would have the ability to delete (rm) or change permissions (chmod) their files. The files in kill.work are similar except they specify permissions that allow the designated user to kill processes (kill) or change the process priority (renice). The group.* files specify lists of groups instead of users.
The library.dirs file contains a list of legal library directories other than $CMDSCRIPTS_DIR/library. This file must be owned and writable only by root and is required to maintain security, but allow flexibility in providing alternate library directory locations.
An alternate library location is used by the cmd program when the environment variable $SYSTOOLS LIB is defined to a value contained in the library.dirs file. It should be noted that when an alternate library directory is being used, all of the configuration files described in the above sections are now relative to the new library directory location.
cmd adduser wicks
Of course you would change wicks to the account name of the account you wish to add.
The cmd program may evaluate the above rules in any directory specified by $CMDSCRIPTS_DIR/library as long as suitable file permissions have been provided.
The deldata program is used to delete the UID and GID databases from the $CMDSCRIPTS_DIR/library/adduser.work directory. You should use this command when you want one of the other system administration utilities which uses these databases to obtain a fresh copy from the network. This is needed from time to time as ACCESS updates the database.
The finduser program searchs the UID and GID databases for the string specified as search criteria. A common use of this program is to search for someone's name prior to adding an account in order to have the correct UID. You may specify multiple strings as the search criteria.
The adduser program adds an account for the specified UID or loginname. It consults the UID and GID databases to determine the correct loginname and GID. It consults the disks files to determine the correct location for the account. The program is interactive and will ask you to confirm these choices. It also installs a .profile, .login, .cshrc, etc from the template_home product. It also creates a bin and rje directory in the users $HOME directory.
If the $HOME directory already exists, adduser will ask if the account should still be created. Adduser also prompts you for the login shell for the account. The default value is specified in the sys.env file.
If the GID doesn't exist in the group file, an entry is added. In addition, an entry is added in the passwd file. The location of the group and passwd files are set by the GROUP and PASSWD variables that are set in $CMDSCRIPTS_DIR/library/sys.env. If either one of these variables has been set to the word NULL, then adduser will exit immediately, not allowing changes to these files. This configuration would be desirable if the machine is a NIS client. If the YPDIR directory variable is defined in sys.env file, then it is assumed that the system is the NIS master, and a yp make will be executed.
A password is automatically generated and displayed on the screen. This password is mailed to the user along with a greeting message specified in $CMDSCRIPTS_DIR/library/greeting. The adduser program asks for an e-mail address (in typical UNIX form: accountname@machine) in order to send this information. The default machine for the e-mail address is specified in the sys.env file. The program allows an option to not send mail by entering three dashes for the e-mail address. When this option is chosen, the system manager must find another way to provide the initial password to the user.
Additional user specified tasks can be accomplished by creating files called adduser.local (al21 ways executed) and adduser.hostname (only executed for the specific host) in the library directory. These file must be Bourne shell scripts and must have the same permissions as the adduser script. The following shell variables are defined at the time of execution of the scripts:
$login account name $uid numeric uid $gid numeric gid $name full user name (value in comment field of passwd file) $home home directory of user
The chguser and moduser programs are identical and are used to change the UID, GID, or $HOME directory of the user specified by the uid or loginname. It still uses the information in the UID and GID databases, and disks files so the use of this program implies that one of these files has been changed.
The chguser and moduser programs use the same rules regarding NIS and the location of the group and passwd files as the adduser program.
Usage: cmd disuser loginname
The disuser program disables the users account by inserting a dash (-) in the front of the encrypted password field of the password file. Re-enabling of the account with the same password can then be accomplished by removing the dash (by using the vipw program).
The disuser program use the same rules regarding NIS and the location of the passwd file as the adduser program.
The resetuser program assigns new automatically generated password to the specified account and mails this information in the same manner as the adduser program. If an optional second argument is given, the password is set to that argument.
The resetuser program use the same rules regarding NIS and the location of the passwd file as the adduser program.
The addmember and delmember programs allow the addition and deletion of login names from a specific group in the group file. If a groups.loginname file exists in the library directory for the user and the user is listed in the functions file, the user is prompted for the group to be modified, which must be listed in the groups.loginname file.
If there is no groups.loginname file for the user, are not a group manager account, and are listed in the functions file, any group can be modified.
Group manager accounts (assuming the default functions file) may only modify the group manager group.
The addmember and delmember programs use the same rules regarding NIS and the location of the group and passwd files as the adduser program.
The vigr and vipw programs invoke the vi editor to allow modification of the group and passwd files. As with all other system administration utilities it creates a lock file this preventing access from any other system administration utility.
The vigr and vipw programs use the same rules regarding NIS and the location of the group and passwd files as the adduser program.
Usage: cmd clearaccounts
The clearaccounts program is used to clear-out an account. It removes all files in the user's $HOME directory as well as their mail file. It then re-installs the initial files in the same manner as the adduser program. Finally, it eithers disables the password, or allows you to choose a new password for the account.
The clearaccounts program uses file of the form loginlist.loginname in the directory $CMDSCRIPTS_DIR/library. This file should contain a list of accounts that can be cleared by the login name given in the loginname extension to the file. Each line in the file should contain a single login name.
You have the option of clearing all accounts, typing in specific accounts, are having clearaccounts prompt you for each account listed in the loginlist file. After you have specified which accounts will be cleared, you are asked, if you want to disable the password, or set a new one. Because of the destructive nature of this procedure, you are asked to confirm your desire to continue with the procedure before any file deletions occur.
The clearaccounts program use the same rules regarding NIS and the location of the passwd file as the adduser program.
Usage: cmd addlocal relative path full
path
The addlocal program is used to add files to or update files in the /usr/local area. The argument relative path refers to directory in which the file should be installed. This argument is given relative to the /usr/local directory. The argument full path is the full pathname of the file to be installed. For example to install the file /usr/people/newutil in the /usr/local/bin directory you would execute:
cmd addlocal bin /usr/people/newutil
If the file you are trying to add already exists, you can only replace it if your effective uid (before executing cmd) is the same as the owner of the currently installed file. If you are the owner addlocal will copy the current version to the same directory with a .O extension.
Actions taken by addlocal are logged in $CMDSCRIPTS_DIR/logfile.
Usage: cmd dellocal relative path
The dellocal program is used to delete files from the /usr/local area. The argument relative path is the path name of the file to be deleted from the /usr/local directory. You are only allowed to delete the file if your effective uid (before executing cmd) is the same as the owner of the currently installed file.
Actions taken by dellocal are logged in $CMDSCRIPTS_DIR/logfile.
The shell program allows a user to change his/her login shell to a value specified in the $CMDSCRIPTS_DIR/library/shells file. If the optional second argument is specified, the process is performed non-interactively. Otherwise, the user is prompted for the new login shell.
The rm and chmod commands allow designated users to remove and change permissions on the files specified on the command line. A designated user must have either a user.account name or a group.account name file in the $CMDSCRIPTS_DIR/library/rm.work directory. These files contain list of users/groups using the syntax of cwi. In order to perform the action, the files must either be owned by an account specified in the user list or grouped to an group in the group list.
The kill and renice commands allow designated users to kill and change the priority of the process ids specified on the command line. A designated user must have either a user.account name or a group.account name file in the $CMDSCRIPTS_DIR/library/kill.work directory. These files contain list of users/groups using the syntax of cwi. In order to perform the action, the process must either belong to an account specified in the user list or belong to an account with a primary group of a group specified in the group list.
UNIX is a registered trademark of UNIX Systems Laboratories
IRIX is a registered trademark of Silicon Graphics, Inc.
SunOS is a registered trademark of Sun MicroSystems
ULTRIX is a registered trademark of Digital Equipment Corporation
AIX is a registered trademark of International Business Machines