SLinstall

Instructions for installing SL5 on DAQ machines

Notes:

• these instructions are periodically updated to include items needed for older/newer versions of Linux. They are marked like this: (SL4.2+) means Scientific Linux 4.2 and newer; (SL4 is equivalent to FC3). (FC5 only) means Fedora Core 5; etc.
• obsolete items are marked by the "#" sign at the beginning of the line and sometimes have a comment about the reason for removal.
• typically, we do not "upgrade" machines using the Red Hat "upgrade" function. Instead, we save critical files from the old installation and do a "fresh install" from scratch

Preparation

save /etc, /var, /root, /usr/local, /opt and /tftpboot (tar and scp to another machine or use either of: /triumfcs/trshare/midas/Disks/rsync_all_NIS.csh or /triumfcs/trshare/midas/Disks/rsync_all_noNIS.csh)

NIS only: ascertain NIS domain name (use authconfig) e.g. "DAQ-NIS","MUSR-NIS" etc.

check existing partition sizes on machine: df -hl, fdisk -l

note which are the /home1 and /data partitions

if /home1 is inside the "/"partition you must save it also

shutdown

Running SL installer

Start installation of the new system:

IMPORTANT: if you have WDC "advanced partitioning disks" (4kB sectors), disks have to be repartitioned before use, see special instructions (TBW) (note: use fdisk -H 224 -S 56 /dev/sdx)

boot from latest "SL5 kickstart" CD from Kelvin Raywood or PXE boot the latest SL installation image

after the system enters graphical mode, one can remove the CD- the installation is running over the network

two questions will be asked: how to partition the disks and the root password. The rest of the installation is automatic.

select "Custom partioniong". You can create either simple normal partitions if you have only one disk or RAID 1(mirror) for / and /home1 if you have two disks.

   <strong>For the case of one disk only</strong>: - no "/boot" filesystem,
   - allocate at least 20 GBytes for "/" filesystem (primary partition, hda1)
   - allocate at least 8 GBytes for swap (primary partition, hda2)
   - allocate the rest as /home1 and /data (primary partitions hda3 and hda4)

<strong>For the case of two or more disks:</strong>
- If there are already some partitions on the disks, consider DELETING them all 
- Click New, select software RAID for /dev/sda, 20000MB (20 GB) and Force primary partition
- Same as above for /dev/sdb
- Click RAID, select create a RAID device /dev/md0, mount point /, RAID Level1 (mirror)
- Repeat for the swap partition (/dev/md1, make it at least 8 Gbytes)
- Leave the rest of the disks free. The other RAID partitions will be created AFTER installation.

• enter the root password for the new system
• continue with installation, at the very end, the system will ask you to reboot
  
• boot newly installed system and answer the few questions for SL5
  
• Firewall: Leave it disabled; SELinux: choose Disabled; KDump: Leave it disabled; Date and Time: Leave kickstart defaults (should be NTP using TRIUMF time servers)
  
• Create user: not necessary if you are using NIS; Sound card: ignore possibly.
• The system will reboot again.

Post installation

• Login from the console, restore the SSH keys from backup (/etc/ssh/*key*), "service sshd restart"
  
• ssh into the new machine as root
• If you created only one mirror partitions, it is time to create the others: Create one 60 GB primary partition on each disks for /home1 and another one using the remainder of space.
  

<strong>fdisk /dev/sda</strong>
...
Command (m for help): <strong>n</strong>
Command action ...   <strong>p</strong>
Partition number ... <strong>2, 3 or 4</strong> according to what has been defined before
First cylinder ... default
Last cylinder ...  <strong>+60000M</strong>  or default
Command action ...   t
Partition number ... : <strong>2, 3 or 4</strong> according to what has been defined before
Hex code ... : fd if you intend to include this into a RAID array
Command action ...   <strong>p to check all is correct</strong>
Command (m for help): <strong>w
.... </strong>The new table will be used at the next reboot.<strong>
</strong><strong><strong>fdisk /dev/sdb  </strong></strong>and repeat as above
<strong><em>-  Reboot the machine  ---
</em></strong>Check the newly created partitions: <strong>fdisk -l /dev/sda; fdisk -l /dev/sdb
mdadm --create /dev/md2 -a yes  -l 1 -n /dev/sda3 /dev/sdb2</strong>
Check the progress of building the RAID with <strong>more /proc/mdstat
</strong>When finished:<strong> mkfs -j -L /home1 /dev/md2; tune2fs -i 0 -c 0 /dev/md2; mkdir /home1</strong>
Add a line to /etc/fstab: 
<strong>/dev/md2                /home1                  ext3    defaults        1 2</strong>
Finally mount this new partition: <strong>mount -a</strong>
Repeat from <strong>mkfs</strong>  for each of the data partitions.

• At this point you should have these disk partitions (single-disk in parenthesis)
  
• /dev/md0 (/dev/hda1) is the system partition, 20 GBytes or more
• /dev/md1 (/dev/hda2) is the swap partition, 8 GBytes or more
• /dev/md2 (/dev/hda3) is the /home1 partition, 60 GBytes or more
• /dev/md3 or /dev/sda4 (/dev/hda4), etc are the data partitions
  
• (not needed for SL5.5 kickstart) check that /etc/ssh/ssh_config contains:

          ForwardX11 yes
          ForwardX11Trusted yes

• (on midm15/midm9b/midm20 only) install correct ethernet driver eepro100 not e100
• restore /home (non-NIS) or /home1 (NIS) and other required user directories from backup.
  (Can use /triumfcs/trshare/midas/Disks/rsync_back.csh ).
• if needed, for non-NIS only, make a softlink for /home1 (NIS automounts /home).
  ln -s /home /home1
• Restore users accounts (non-NIS and NIS master only):
  - edit /etc/passwd and /etc/shadow
  Append users' login info to the end of these files from the backup versions.
• edit ~root/.forward
  (create file if not present, or restore from backup) In file add email address of the person(s) to receive root's email - always include Konstantin - olchansk@triumf.ca)
• cleanup network setup (or postfix would not work):
•  check that /etc/sysconfig/network has the entry "HOSTNAME=xxx.triumf.ca" (fully qualified domain name)
• check that /etc/hosts has only the entry for "localhost".
  
• ###run "system-config-authentication", check "select NIS support", click on "configure NIS", set NIS domain, leave NIS server field blank.
• run "authconfig --enablenis --nisdomain LADD-NIS --update"
• run "sed 's/^hosts:.*/hosts: files dns/' -i /etc/nsswitch.conf" (to undo a mistake from authconfig)
• On the master NIS node (ladd00), add this new node to /etc/netgroup, and update NIS maps (cd /var/yp; make)
  
• (if needed) To setup NIS slave server:
•  check if /usr/lib64/yp exists. If not, run "yum install ypserv"

• /usr/lib64/yp/ypinit -s ladd00 (/usr/lib/yp/ypinit on 32-bit machines)

• chkconfig ypserv on

• service ypserv start

• service ypbind restart

• add the new machine to /var/yp/ypservers on the NIS master, run "make -C /var/yp" and also "cd /var/yp; yppush -h newmachine ypservers"

• If using securenets on the NIS master, remember to copy this file to the new slave server in /var/yp

• (if NIS master or standalone) check /etc/auto.* against backups, particularly auto.master if NIS master
• (if needed) add "+auto.master" at the end of /etc/auto.master
• restart autofs to use the newly configured NIS maps: "service autofs restart"
• Verify time and date configuration. Run "ntpstat", it should say "synchronised to NTP server (142.90.x.y). If not, run "system-config-time", check "use network time", enter NTP servers: time1, time2, time3
• (If needed) enable automatic system updates: run "/triumfcs/trshare/olchansk/linux/triumf-update/yum-setup.perl -o"
• (If needed) enable automatic kernel updates: run "/triumfcs/trshare/olchansk/linux/triumf-update/yum-setup.perl -k -o"
• disable unwanted services, use "chkconfig --list | grep on | sort" to see what is enabled. SL4- disable unnecessary services:

  (only if amanda is not used) -> chkconfig --level 12345 xinetd off
  chkconfig --level 12345 canna off
  chkconfig --level 12345 FreeWnn off
  chkconfig --level 12345 hpoj off
  chkconfig --level 12345 ip6tables off
  chkconfig --level 12345 iptables off
  chkconfig --level 12345 isdn off
  chkconfig --level 12345 pcmcia off
  chkconfig --level 12345 rhnsd off
  chkconfig --level 12345 spamassassin off
  chkconfig --level 12345 bluetooth off
  chkconfig --level 12345 apmd off
  chkconfig --level 12345 iiim off
  chkconfig --level 12345 fenced off
  chkconfig --level 12345 ccsd off
  chkconfig --level 12345 cpuspeed off
  chkconfig --level 12345 pcp off
  chkconfig --level 12345 pmie off
  chkconfig --level 12345 yum-updatesd off
  chkconfig --level 12345 clvmd off
  chkconfig --level 12345 cman off
  chkconfig --level 12345 lvm2-monitor off
  chkconfig --level 12345 modclusterd off
  chkconfig --level 12345 yum-updateonboot off
  chkconfig --level 12345 cmirror off
  chkconfig --level 12345 lock_gulmd off
  chkconfig --level 12345 firstboot off
  chkconfig --level 12345 ricci off
  chkconfig --level 12345 gfs off
  chkconfig --level 12345 scsi_reserve off
  chkconfig --level 12345 openibd off
  chkconfig --level 12345 arptables_jf off
  chkconfig --level 12345 auditd off
  chkconfig --level 12345 avahi-daemon off
  chkconfig --level 12345 hplip off
  chkconfig --level 12345 iscsi off
  chkconfig --level 12345 iscsid off
  chkconfig --level 12345 mcstrans off
  chkconfig --level 12345 pcscd off
  chkconfig --level 12345 restorecond off
  chkconfig --level 12345 setroubleshoot off
  chkconfig --level 12345 xend off
  chkconfig --level 12345 xendomains off
  chkconfig --level 12345 kudzu off
  #chkconfig --level 12345 yum-cron off

section bbb

• configure hardware sensors:
• run "yum install lm_sensors"
  
• run "sensors-detect", accept default answer to all questions (press ENTER)
  

• run "service lm_sensors restart" to load the kernel modules

• run "sensors", observe the sensors data

• if sensors reports "General parse error", do "cp /triumfcs/trshare/olchansk/linux/lm_sensors/lm_sensors-3.0.1/prog/sensors/sensors /usr/bin/sensors" and try again.

section ccc

• ssh root@localhost, ctrl-C
• scp root@ladd00:/root/authorized_keys ~root/.ssh/
• Use "system-config-users" to add local user accounts
• NIS: check user accounts: run "ypcat -k passwd"

Enable NFS

• to enable NFS:

  edit /etc/hosts.allow, add or uncomment "mountd: 142.90.0.0/255.255.0.0"
  create /etc/exports, e.g. "/home1  @daqmachines(rw,no_root_squash,async)"
  chkconfig nfs on
  chkconfig nfslock on
  service nfs restart

Enable AMANDA backups

AMANDA backups are already enabled by TRIUMF kickstart installs. For non-kickstart installation, follow instructions at [http://amanda/~amanda], or look at "/triumfcs/trshare/olchansk/linux/amanda/amanda-enable.perl". As final step, use [https://helpdesk.triumf.ca] to contact TRIUMF CS to add this new machine to the amanda backup list.

Enable DCACHE

• mkdir -p /pnfs/triumf.ca
• edit /etc/rc.local, add to the end of file: "mount -o intr,rw,noac,hard,nfsvers=2 trdata00:/pnfsdoor /pnfs/triumf.ca"
• . /etc/rc.local

section ddd

• (not needed for SL5.5 kickstart) erase unwanted packages: yum erase logwatch mailman mrtg inn inn-devel cyrus-imapd cyrus-imapd-devel cyrus-imapd-murder cyrus-imapd-nntp webalizer squirrelmail rhn-applet yumex-applet apt-autoupdate SL_enable_serialconsole tog-pegasus kernel-largesmp kernel-hugemem kernel-largesmp-devel spamassassin slrn-pull openafs kernel-module-openafs openafs-debug openafs-devel openafs-kernel-source kernel-largesmp kernel-hugemem kernel-hugemem-devel xen kernel-xen bash-completion
• install xemacs (from EPEL): yum install "xemacs*"
  
• install ganglia:

  scp ladd00:/etc/gmond.conf /etc/gmond.conf
  yum install ganglia ganglia-gmond
  chkconfig gmond on
  service gmond restart

• install misc stuff

  cd /triumfcs/trshare/olchansk/linux/misc-rpms
  rpm -vh --install triumf-ko-ganglia-*
  rpm -vh --install emailonreboot-*
  rpm -vh --install monitor_nfs-*
  rpm -vh --install /triumfcs/trshare/olchansk/public_html/diskscrub/download/diskscrub*

• install memtest86: "yum install memtest86+", "ls -1 /boot/* | grep memtest" to find out the name of the memtest boot file to use on the "kernel" line of the incantation to be added at the end of /boot/grub/grub.conf:
  

cat >> /boot/grub/grub.conf

title memtest
      root (hd0,0)
      kernel /boot/memtest86+-1.65

• yum install tkcvs
• install node monitoring

  cd /triumfcs/trshare/olchansk/linux/misc-rpms
  rpm -vh --install triumf_nodeinfo.noarch.rpm
  /usr/sbin/sendnodeinfo.perl --config ladd00:8600
  edit /etc/nodeinfo
  /usr/sbin/sendnodeinfo.perl ladd00:8600

• (before FC5) remove the fancy screen savers

  rm -f /usr/X11R6/lib/xscreensaver/*
  rm -f /usr/bin/*.kss

• chmod a+r /var/log/messages
• (before FC5) edit /etc/updatedb.conf, set "DAILY_UPDATE=yes"
• cd ~; yum update

Install packages needed for ROOT and MIDAS DAQ

yum install giflib.i386 giflib.x86_64 compat-libf2c-34.i386 mysql-devel sysstat "libusb-devel*" unixODBC-devel postgresql-devel libxml2-devel libXpm-devel libgfortran libstdc++-devel.i386 git

Section eee

• install xpdf (from EPEL, on SL5): yum install xpdf
  
• (if desired) install all missing packages: /triumfcs/trshare/olchansk/linux/triumf-update/yum-everything.perl
• (SL5) If yum complains about unsigned java packages: cd ~; rpm -vh --upgrade /triumfcs/mirror/scientificlinux.org/5x/i386/updates/security/jdk*rpm /triumfcs/mirror/scientificlinux.org/5x/i386/updates/security/java*rpm

Section fff

• (SL5.5 or newer) enable raid1 bitmap files, for each /dev/mdX device, run "mdadm --grow --bitmap=internal /dev/mdX". In /proc/mdstat, verify that there is a "bitmap" entry. This will fail if the md device reconstruction is in progress.
  

Configure GRUB boot loader

• edit /boot/grub/grub.conf, remove the "quiet" and "rhgb" options
• edit /boot/grub/grub.conf, comment out (with "#") the "splashimage=" line
• check that GRUB boot loader is installed on all system disks: dd if=/dev/sda bs=1 count=1024 2>&1 | strings | grep GRUB
• if GRUB is not installed, (i.e. on the 2nd disk of machines with mirrored system disk), (but check that /dev/sdb is the right disk):

# grub
grub> device (hd0) /dev/sdb
grub> root (hd0,0)
grub> setup (hd0)

Configure X11 graphics

Special settings for DAQ

• add the following at the end of /etc/X11/xorg.conf. The enables Ctrl-Alt-KP-/ and Ctrl-Alt-KP-* to unlock the keyboard after Altera Quartus crash:

Section "ServerFlags"
        Option "AllowDeactivateGrabs" "true"
        Option "AllowClosedownGrabs" "true"
EndSection

Install NVIDIA drivers

• yum install nvidia-x11-drv kmod-nvidia (if it fails due to conflict with module-init-tools, run "yum --disablerepo \* --enablerepo elrepo update module-init-tools")
• mv /etc/X11/xorg.conf /etc/X11/xorg.conf-xxx
• nvidia-xconfig
• /dev/MAKEDEV nvidia
• restart the X11 server (Ctrl-Alt-Backspace or killall gdm-binary)
• observe that X11 server restarts using the NVIDIA driver (big NVIDIA logo on startup)
• if needed, login as root and run "nvidia-settings" to setup dual-screen configuration, etc

Manual selection of monitor, video mode and resolution

Automatic selection of monitor and video mode usually works. When it does not, configure it manualls:

• physically go to the computer
• login as root
• run "system-config-display".
• In the "hardware" tab, select monitor type: "generic LCD 1280x1024" or "generic LCD 1600x1200".
• In the "settings" tab, select "1280x1024" or "1600x1200" and "Thousands of colors".
• Press "ok", the display settings application should close.
• Logout, the new login window should use the new settings.

Finish installation

• logout and reboot the computer to have all the changes to take effect