Enstore and dCache User Documentation

Chapter 2: Getting Ready to Use Enstore and DCache

Chapter 2: Getting Ready to Use Enstore and DCache

---

2.1 Setting up Access to Enstore

2.1.1 Initial Steps for All Users

1. Find out what your volume quota is from your experiment's Enstore administrator, and make sure you reserve what you need, according to your experiment's procedures. The experiment's Enstore liaison should use the online form STKEN Mass Storage Request Form at http://computing.fnal.gov/cd/forms/storagerequest.html to request quota for the experiment.
2. Find out what area in /pnfs namespace your experiment uses.
3. Read about file families (see section 1.4.1 File Family), and find out from the people in your experiment responsible for implementing Enstore how file families have been configured for your experiment. Determine what file family(ies), and hence which subdirectories in /pnfs namespace, you want to write to and/or read from.
4. Encp and Enstore commands use whatever routing the client system or network administrator sets between the client system and the Enstore system for data transfers. If you (as the sysadmin or network admin of the large client machine) want to restrict the set of interfaces that encp/Enstore uses, you need to create the file enstore.conf. This file controls the interface-router mapping for the network connections used by encp/Enstore. For information and instructions, see Appendix A: Network Control.
5. Navigate to the Enstore monitoring system web page, titled Fermilab Mass Storage System, at http://hppc.fnal.gov/enstore/. Select the Enstore system that your experiment uses, and browse the system information for it. You might want to bookmark this page.
6. Subscribe to the stk-users@fnal.gov listserv mailing list for announcments about Enstore and the STKEN Enstore system. D0 users, subscribe to d0en-announce@fnal.gov and sam-admin@fnal.gov. CDF users, subscribe to cdfdh_oper@fnal.gov.

Notes:

• Data are moved with the default TCP window size on the machine. There is a potential for an extreme performance degradation if the default window is set too large. A value of about 32K works well at most locations at Fermilab.
• t is recommended to keep the number of files in any given directory under 2000.

2.1.2 Further Steps for non-dCache Users Only

1. Make sure your node and network can provide adequate throughput. To determine the optimal data transfer rate, consult the Enstore administrators.
2. See if your experiment's /pnfs area is mounted on your machine, by using standard UNIX utilities like cd and ls. If it's already mounted, skip to step (6). If not, continue.
3. Check to see if authorization has been granted to mount the /pnfs area on the machine you plan to use. To do so:
   1. Go to the PNFS Exports Page1, at http://www-<xyz>en.fnal.gov:/enstore/pnfsExports.html, where <xyz> is one of stk, d0 or cdf, depending on the Enstore system used by your experiment.
   2. Scroll down to the PNFS ExportList Fetch Begin: <date/time> area, and look for your node and /pnfs/storage-group area. If they're listed, authorization has been granted; skip to (5). If not, continue.
4. Notify your experiment's Enstore liaison that you need authorization to mount the /pnfs area on the machine you plan to use. He or she will need to send your request on to enstore-admin@fnal.gov.
5. Once authorization has been granted, mount the /pnfs area on your machine if you have root permission, or send a request to the machine's system administrator to mount it. To mount the area yourself, edit the /etc/fstab file and add a line with the following strings (they should appear all on the same line in the file; we separate them into six lines here for clarity):

   remote_enstore_server_node:enstore_server_directory
    
   /pnfs/local_mount_point
    
   mount type
    
   comma_separated_attributes
    
   0
    
   0
    
   

where the 0 in the 2nd-to-last line means no dump of filesystem, and the 0 in the last line means no fsck checks at boot time. For example:

stkensrv1:/E872 /pnfs/E872 nfs user,intr,bg,hard,rw,noac 0 0 
 

Usually, local_mount_point is the same as enstore_server_directory. Make sure that local_mount_point exists! (A typical error message is "backgrounding".)

6. Install UPS/UPD on your system. See Part III of the UPS/UPD manual at http://www.fnal.gov/docs/products/ups/ReferenceManual/parts.html#partIII.
7. Install the encp product on your machine (see below).

Installing Encp

To install the encp product from KITS using UPD, run:

$ setup upd
 
$ upd install -G "-c -q <xyz>" encp
 

where <xyz> stands for one of the Enstore systems. Currently, these include:

stken

for general Fermilab users

d0en

for D0 users

cdfen

for CDF users

For example, a CDF experimenter would type:

$ upd install -G "-c -q cdfen" encp
 

2.2 Important Environment Variables

There are two important environment variables that are generally set in the Enstore setup script. Users who work on more than one Enstore system (e.g., stken and cdfen) at a time in different windows may need to know about these in case they use the wrong window for a particular Enstore system!

The variables are:

$ENSTORE_CONFIG_HOST

points to the Enstore server that is running the configuration server (see section 7.6 Configuration Server).

All production systems currently use srv2 (i.e., cdfensrv2, d0ensrv2 or stkensrv2) as the $ENSTORE_CONFIG_HOST computer. This is different from the computer from which the pnfs filesystem is mounted (which is srv1, i.e., cdfensrv1, d0ensrv1, stkensrv1).

$ENSTORE_CONFIG_PORT

sets the port number; the value is (by convention) 7500 for all installations at Fermilab.

1Note that you can also get to the PNFS Exports Page from the Fermilab Mass Storage System web page at http://hppc.fnal.gov/enstore/ via the following path: Under Installed Enstore Systems, click the system your experiment is using. On the Enstore System Status page, click "Log Files". Under User Specified Log Files, click "PNFS Export List" to arrive at the PNFS Exports Page.

View/print PDF file | Back to Enstore Doc Home Page | Fermilab Mass Storage System | Computing Division | Fermilab at Work | Fermilab Home