System administration of CKM Debian Linux Cluster

SSH razreshit X11 forwarding - po defaultu on vikliuchen. Mojno dlia vseh hostov
  1. System Allocations
  2. BIOS settings
  3. Making system secure
  4. Adding user to the system
  5. Compiling kernel
  6. Setting up X11
  7. Setting up PAM
  8. Setting up email (exim)
  9. Setting up OpenAFS
  10. Setting up kerberized CVS
  11. Mandatory seurity notice
  12. Setting up DVD player
  13. Adding USB support
  14. Adding support for lm-sensors
  15. Changing sizes of partitions
  16. Setting up applications for File Manager
  17. Setting up UPS/UPD from scratch
  18. Recovering backup files

System Allocations / Locations / Addresses / Accounts

BIOS settgins

There are several BIOS setting you want to adjust


Making system secure.

Adding user to the system.

Compiling kernel.

Kernel should be compiled on ckm00 computer in //usr/src/kernel-source-2.4.16 directory. Command make-kpkg is very helpful

  1. Make a cleanup
    # make-kpkg clean
  2. Prepare kernel configuration. You may want to start from already existing configurations:
    # cp config.ckmxx .config
    # make menuconfig
  3. Compile the kernel and prepare Debian package with the kernel image
    # make-kpkg --revision=ckmxx.1.1 kernel_image
  4. Compile the kernel and prepare Debian package with the modules images (openafs, lm-sensors and so on)
    # make-kpkg modules_image
  5. Goto upper directory and copy relevant *.deb files to /usr/ckm/debian directory
  6. Install kernel-image on all computers
    # slogin ckm01
    # dpkg -i /usr/ckm/debian/kernel-image-2.4.16_ckmxx.1.1_i386.deb

Hint: If during compilation some header files are missing you may need to install some lib*-dev.deb pacakges. You can use Debain Packages contents search page to find appropriate Debian package.

Setting up X11

  1. Standard /etc/X11/Xsession from Debian distribution is fine
  2. To run wincenter we have to allow Xserver to listen to TCP Modify /etc/X11/xdm/Xservers (we start X only through xdm) and take out -nolisten tcp .
  3. Syncronize default X application. Those are programs that X will launch at the beginnig. These applications are not more than symbolic links. To see and modify them run:
    # update-alternatives --display x-window-manager
    # update-alternatives --config x-window-manager

    application choice
    x-session-manager /usr/bin/gnome-session
    x-window-manager /usr/bin/enlightenment
    x-terminal-emulator /usr/X11R6/bin/xterm

Setting up PAM.

The idea of this setup is to allow user to use both Kerberos and local passoword to login into the system. Getting AFS token is also important if user uses kerberos passowrd.

Setting up email (exim).

Fermilab Run eximconfig and choose option 1 (Internet site). Hit return for all questions except for Replace /etc/aliases file (keep original file).

Adjust file /etc/email-addresses so that outgoing mail has format

IHEP Run eximconfig and choose option 2 (Internet site using smarthost). Hit return for most questions except for smarthost (choose and Replace /etc/aliases file (keep original file).

Adjust file /etc/email-addresses so that outgoing mail has format

Setting up OpenAFS.

You need to compile openafs module. This is done when you compile a kernel on ckm00. Source code is in package openafs-modules-source . As a result of compilation you will get Debian package openafs-modules-2.4.16

Following packages also have to be installed:
libpam-openafs-session To generate AFS token during login
openafs-client To access AFS file system
openafs-krb5 Kerberos 5 support in OpenAFS
openafs-modules-2.4.16 Kernel OpenAFS modules
It is usefull to limit number of Cells that you can access from a single computer. Reasonable set is in /etc/openafs/CellServDB

Setting up kerberized CVS.

There is no kerberized CVS package in Debian. But not to worry - it is extremely simple to prepare your own kerberized Debian package. Check official CVS manual for reference if needed
  1. Get debian package cvs-sourceand go to some working directory # apt-get source cvs
  2. There will be a directory tree with applied patches and all ready to be compiled. Go there
  3. There is a directory debian inside. Edit file debian/rules . Somewhere inside it there is a configuration line. Add flag --with-gssapi
            ./configure --prefix=/usr  --sysconfdir=/etc \
                    --mandir=\$${prefix}/share/man \
                    --infodir=\$${prefix}/share/info --with-gssapi
  4. Compile the tree
    # debian/rules configure
    # debian/rules build
    # debian/rules binary

    You may need some extra packages ( dbs and development libraries mostly, for example libkrb5-dev) for successfull compilation. Install them. Basically configure will complain and you would see what is absent. Run debian/rules clean before each try.

  5. As a result you will get debian package in the upper directory. Install it

    cd ..
    dpkg -i cvs_1.11.1p1-3_i386.deb

    In configuration stage don't forget to enable pserver . This port is used for kerberos authentication.

  6. Restart inetd service
    /etc/init.d/inetd restart
  7. Check that cvs-pserver is allowed in /etc/hosts.allow
  8. Define correct setting for CVSROOT in /etc/environment

Mandatory Security notice.

You uhave to modify file /etc/motd So that it reflects the mandatory Fermilab security notice. This file also coontains computer name so it is unique to each system. This note is gracefully modified each time you install new kernel. It also could be regenerated after update of base-files Debian package.

Setting up DVD player

  1. Creadte symbolic link that points to DVD ROM IDE device
    # ln -s /dev/hdd /dev/dvd
  2. Install packages ogle and ogle-gui While instaling it you would get message that you need to run /usr/share/doc/ogle/examples/ . Do it after packages are installed.
    # apt-get install ogle ogle-gui
    # /usr/share/doc/ogle/examples/

Adding USB support

If in trouble check excellent Linux USB web page.
  1. Find out which USB adapter you are using OHCI or UHCI. You can do it carefully reading output of lspci -v | grep -i usb
    00:07.4 USB Controller: Advanced Micro Devices [AMD] AMD-765 [Viper] USB (rev 07) (prog-if 10 [OHCI]) So in this example it is clearly OHCI. In make menuconfig stage of kernel configuration go USB support --> OHCI . Put it in as a module if you think that it is going to be rarely used. Compile it in otherwise. As most of our mice are USB it is worse compiling in.
  2. Add USB mouse support. It carries encyptic name Human Interface Device (HID). This option is not seen under USB support, unless you turn on Input core support in the main menu. Compile in Mouse support in that submenu.
    Now go back to USB support and you will see that Human Interface device options showed up. Choose only USB Human Interface Device (full HID) support
  3. Compile in Preliminary USB device filesystem
  4. You may also choose USB Scanner support as a module.
  5. Go ahead and compile the kernel.
  6. Install packages usbmgr usbutils usbview
  7. Create USB devices and INPUT devices (USB mouse)
    # cd /dev
    # ./MAKEDEV usb
    # ./MAKEDEV input

    Check that mouse works: do cat /dev/input/mice and move the mouse.

  8. Make X11 aware of the USB Mouse adding to /etc/X11/XF86Config-4
    Section "InputDevice"
         Identifier  "USB Mice"
         Driver      "mouse"
         Option      "Protocol"   "IMPS/2"
         Option      "Device"     "/dev/input/mice"
         Option      "ZAxisMapping"   "4 5"
         Option      "Buttons"        "5"
     ... snap ...
    Section "ServerLayout"
        Identifier  "Simple Layout"
        Screen 	"Screen 1"
        InputDevice "USB Mice"    	"CorePointer"
        InputDevice "Keyboard1"   	"CoreKeyboard"

Adding support for lm-sensors

lm-sensors can provide information about CPU temperature and FAN rotation speeds. Can be quite useful. Check official lm-sensors page for details.

Note pc781[b-e] do not have sensors installed on motherboard.

  1. Compile lm-sensors and i2c drivers. You would do it on ckm00 computer. You need packages lm-sensors-source and i2c-source. Usually you would do it each time you compile kernel.
    # cd /usr/src/kernel-source-2.4.17
    # make-kpkg clean
    # make-kpkg --revision=ckmxx.1.2 kernel_image
    # make-kpkg modules_image
  2. Move newly created debina packages to /usr/ckm/debian . Slogin to ckmxx computer and install compiled modules
    # dpkg -i /usr/ckm/debian/i2c-2.4.17_2.6.2-1+ckmxx.1.2_i386.deb 
    # dpkg -i /usr/ckm/debian/lm-sensors-2.4.17_2.6.2-2+ckmxx.1.2_i386.deb 
  3. Install packages to support lm-sensors: lm-sensors sensord
  4. Insert necessary modules. You can try sensors-detect but it sucks. For ckmxx it gave me completely wrong answer. You are better off carefully reading lspci -v and reading Motherboard manual. Or search the Web :)
    computer ckm00 ckmxx pc781x oka08
    motherboard Tyan Thunder K7 S2462N Gigabyte GA-7DXR FIC VL-601
    monitoring chip W83627HF VT82C686A PII4X
    modules i2c-amd756 i2c-viapro
    i2c-isa i2c-isa
    w83781d via686a

Changing sizes of partitions

Useful tool is called GNU parted (package parted). It is very easy to use. Check official web page details. You are usually going to resize partitions. There is a catch : you can move the "tail" of partition not its "beginning". I used this tool several times to create additional Linux partitions, creating dual boot partitions and so on. No problems so far.

  1. Backup data! You don't want to be screwed.
  2. Unmount disk (all partitions). If it is shared with NFS use:
    # /etc/init.d/etc/init.d/nfs-kernel-server stop
  3. Run parted . Type help and read.
  4. Restart NFS if necessary
    # /etc/init.d/etc/init.d/nfs-kernel-server start
  5. Mount all partitions that belong to this disk.

Setting up applications for File Manager

If you open File manager and click on a file icon certain applications should start to process those files (acroread, gv, mozilla...). Sometimes choices in official distribution are odd - xpdf for example. Tweak the file /usr/share/mime-info/mc.keys . Changes will be used by all users (unless they overwrote them for their account).

Alexander Kushnirenko
Last modified: Mon Oct 28 18:07:02 MSK 2002