DEAP

Links

• https://deap06.triumf.ca/ MIDAS status page
• https://deap06.triumf.ca/elog/ ELOG
• https://deap06.triumf.ca/ganglia/ GANGLIA system monitoring
• https://deap06.triumf.ca/nodeinfo/config.html computer configuration and status
• https://deap06.triumf.ca/vme01/ VME crate 1
• https://deap06.triumf.ca/vme02/ VME crate 2
• https://deap06.triumf.ca/vme03/ VME crate 3
• https://deap06.triumf.ca/ups/ UPS
• https://deap06.triumf.ca/cdu/ Power Distribution Unit
• https://deapkvm1 (192.168.1.16) (deapkvm1) Lantronix Spider KVM (only works from deap06 gateway machine) (sysadmin, deap)

DAQ machines

• deap00: main daq machine (storage, home directories, central services, etc)
• deap01..05: A3818 daq machines
• deap06.triumf.ca: temporary network gateway
• deap07: spare A3818 daq machine (old deap00) (used for PCIe ADC DAQ)
• deap08: spare deap00 machine
• lxdeap01: VME daq machine
• deapvme01..03: VME crate power supplies
• deapups: DAQ UPS unit
• deapcdu: DAQ power distribution unit

Network configuration (TRIUMF)

DEAP DAQ machines are on the private network 192.168.1.x (netmask 255.255.255.0)

Gateway to TRIUMF network is 1U machine deap06.triumf.ca connected to the LADD-NIS cluster (deap account on ladd00).

Gateway services running on the gateway:

• DHCP server for the 192.168.1.x network (/etc/hosts, /etc/dhcp/dhcpd.conf)
• apache SSL/https proxy for MIDAS status page, ELOG, ganglia and nodeinfo (/etc/httpd/conf.d/ssl.conf, /etc/httpd/htpasswd)
• NAT proxy from private network to the TRIUMF network (/etc/rc.local). Makes the internet accessible from deapNN machines.

Network configuration (DEAP)

The DEAP DAQ cluster is configured for standalone running with or without an internet connection.

(NB: Some internet functions are required: access to NTP for time synchronization and access to Linux package repositories to install packages, etc)

Usernames, passwords and hostnames are distributed using NIS:

• domain name: DEAP-NIS
• deap00 is the master server
• there are no secondary servers
• hostnames are distributed using NIS (from deap00:/etc/hosts, MUST MATCH deap06:/etc/hosts!)
• to solve chicken-and-egg problem deap00 IP address has to be listed in each machine /etc/hosts (MUST MATCH deap06 and deap00 /etc/hosts!) (SL6.2+ NIS broadcast does not work so deap00 has to be listed in each machine /etc/yp.conf, also NFS filesystems are mounted before NIS is started).
• also NIS has to be listed in front of DNS in the "hosts:" entry of /etc/nsswitch.conf

DNS kludge:

• normally DNS would be used to distribute IP addresses and hostnames to the DHCP server, to deap00 and to other deap machines. But we do not have a private DNS server and the TRIUMF DNS server has the wrong IP addresses for deap machines (142.90.x.x).
• deap06 DHCP is telling all machines to use the TRIUMF DNS server (to resolve internet addresses - google, etc). To avoid confusion between local deap00, etc hostnames and deap00, etc hostnames from TRIUMF, /etc/nsswitch.conf "hosts:" entry has to list "nis" before "dns".
• hopefully the deap00, etc hostnames will be resolved correctly by the SNOlab DNS servers and all this kludging can go away.

System monitoring tools:

• ganglia
• triumf_nodeinfo
• konstantin's ganglia packages (monitor_nfs, ganglia sensors, top, etc) - To install/update: yum --disablerepo="*" --enablerepo=konstantin update
• diskscrub

Backups

• backups of Linux images:
  • backups of linux images are done to deap00:/data/root/backups using cron job on deap00:/etc/cron.d/backup.lxdaq.cron and deap00:~root/backup.lxdaq
• backups of home directories: NONE
• backups of data disks: NONE

Creating boot disks for deap01..deap05

64GB SSD boot disks

• attach SSD disk to any of the deap01..deap05 machines (SATA+power)
• login as root to that machine
• "fdisk -l" to identify which /dev/sdX disk it is
• cd /data/root/backups
• ./clone.perl ./deap01 /dev/sdX
• observe script completes sucessfully and prints "Done. You can remove /dev/sdX and try to boot from it."
• disconnect the disk
• connect to new machine, try to boot from it