UCN: Difference between revisions
| (10 intermediate revisions by the same user not shown) | |||
| Line 172: | Line 172: | ||
As per the instructions [https://daq00.triumf.ca/DaqWiki/index.php/Ubuntu here] | As per the instructions [https://daq00.triumf.ca/DaqWiki/index.php/Ubuntu here] | ||
=== Create online and packages directory === | |||
<pre> | |||
mkdir /home/ucn/online_tucan01 | |||
mkdir /home/ucn/packages_tucan01 | |||
</pre> | |||
=== Install MySQL === | === Install MySQL === | ||
| Line 201: | Line 208: | ||
<pre> | <pre> | ||
sudo apt install mysql-server | sudo apt install mysql-server | ||
sudo | sudo systemctl status mysql-server # should show enabled and active | ||
sudo stop mysql | </pre> | ||
sudo | |||
Now we find the default database directory, should likely be <code>/var/lib/mysql</code>. If not make the substitute in the commands that follow. | |||
<pre> | |||
sudo mysql -u root | |||
SELECT @@datadir; | |||
exit; | |||
</pre> | |||
With this confirmed, we change the data directory: | |||
<pre> | |||
sudo systemctl stop mysql | |||
sudo systemctl status mysql # confirmed inactive | |||
sudo rsync -av /var/lib/mysql /sqlstorage # copy database to new location | sudo rsync -av /var/lib/mysql /sqlstorage # copy database to new location | ||
sudo mv /var/lib/mysql /var/lib/mysql.bak # rename old database in case of errors | sudo mv /var/lib/mysql /var/lib/mysql.bak # rename old database in case of errors | ||
</pre> | </pre> | ||
| Line 220: | Line 238: | ||
port = 3306 | port = 3306 | ||
datadir = /sqlstorage/mysql | datadir = /sqlstorage/mysql | ||
</pre> | |||
Even if apparmor is disabled, you still need to change the alias file and re-enable apparmor, otherwise you will get the error <code>Job for mysql.service failed because the control process exited with error code.</code>. To resolve this, edit the apparmor alias file (<code>sudo vim /etc/apparmor.d/tunables/alias</code>) and add the following line to the end: | |||
<pre> | |||
alias /var/lib/mysql/ -> /sqlstorage/mysql/, | |||
</pre> | |||
Apply changes by restarting apparmor, then re-disable: | |||
<pre> | |||
sudo systemctl restart apparmor | |||
sudo systemctl stop apparmor | |||
sudo systemctl disable apparmor | |||
sudo systemctl status apparmor | |||
</pre> | </pre> | ||
| Line 225: | Line 258: | ||
<pre> | <pre> | ||
sudo systemctl restart mysql | sudo systemctl restart mysql | ||
</pre> | </pre> | ||
Confirm that it worked by re-checking the data directory inside mysql | |||
<pre> | |||
sudo mysql -u root | |||
SELECT @@datadir; | |||
exit; | |||
</pre> | |||
==== Uninstall MySQL ==== | When finished, delete the original directory | ||
<pre> | |||
sudo rm -rf /var/lib/mysql.bak | |||
</pre> | |||
==== Uninstall MySQL (only if needed!) ==== | |||
In the case of serious issue, here are steps to completely purge mysql from your system: | In the case of serious issue, here are steps to completely purge mysql from your system: | ||
| Line 245: | Line 289: | ||
==== Setup MySQL Users and Database ==== | ==== Setup MySQL Users and Database ==== | ||
By default there should be a root user which can access the database without any password. Use that and the following commands to set up the database: | By default there should be a root user which can access the database without any password. Use that and the following commands to set up the database, following the steps in the [https://daq00.triumf.ca/MidasWiki/index.php/Setup_MySQL_database_for_MIDAS midas wiki]: | ||
<pre> | |||
sudo mysql -u root | |||
CREATE DATABASE IF NOT EXISTS ucn_history; | |||
CREATE USER 'ucn_reader'@'localhost' IDENTIFIED BY 'password'; # save this password somewhere | |||
CREATE USER 'ucn_writer'@'localhost' IDENTIFIED BY 'password'; # save this password, different from the reader | |||
GRANT SELECT ON ucn_history.* TO ucn_reader@'localhost'; # permissions | |||
GRANT SELECT,INSERT,CREATE,ALTER,INDEX ON ucn_history.* TO 'ucn_writer'@'localhost'; | |||
flush privileges; | |||
SHOW GRANTS FOR 'ucn_reader'@'localhost'; # verify that commands worked | |||
SHOW GRANTS FOR 'ucn_writer'@'localhost'; | |||
exit; | |||
</pre> | |||
==== Setup ODB Connection ==== | |||
First, we check the socket location: | |||
<pre> | |||
grep sock /etc/mysql/mysql.conf.d/mysqld.cnf | |||
</pre> | |||
which should result in something like | |||
<pre> | |||
# socket = /var/run/mysqld/mysqld.sock | |||
</pre> | |||
denoting the default location. You can uncomment this line, change it, or leave it as is. We assume that this location is ok and use it in the following. | |||
Now we create the files needed. We do this in <code>/home/ucn/online_tucan01</code> since we are setting this up in parallel with the already-running instance in <code>/home/ucn/online</code>. | |||
Create ucn_reader file: copy/paste all the following lines | |||
<pre> | |||
cat <<EOF > /home/ucn/online_tucan01/ucn_reader.txt | |||
server=localhost | |||
port=3306 | |||
socket=NULL | |||
database=ucn_history | |||
user=ucn_reader | |||
socket=/var/run/mysqld/mysqld.sock | |||
password=THE PASSWORD FROM EARLIER | |||
EOF | |||
</pre> | |||
Create ucn_writer file: copy/paste all the following lines | |||
<pre> | |||
cat <<EOF > /home/ucn/online_tucan01/ucn_writer.txt | |||
server=localhost | |||
port=3306 | |||
socket=NULL | |||
database=ucn_history | |||
user=ucn_writer | |||
socket=/datassd/mysql/mysql.sock | |||
password=THE OTHER PASSWORD FROM EARLIER | |||
EOF | |||
</pre> | |||
=== Install APACHE === | |||
<pre> | |||
sudo apt install apache2 | |||
</pre> | |||
=== Install ROOT === | |||
We point the installation to the pre-compiled root on daqstore. Edited <code>.bashrc </code> to the following: | |||
<pre> | |||
# setup root version for different distros | |||
os_id=$(. /etc/os-release && echo "$ID") | |||
case "$os_id" in | |||
ubuntu) source /daq/daqshare/olchansk/root/root_v6.36.06.Linux-ubuntu24.04-x86_64-gcc13.3/bin/thisroot.sh ;; | |||
centos) source /daq/daqshare/olchansk/root/root_v6.24.08.Linux-centos7-x86_64-gcc4.8/bin/thisroot.sh ;; | |||
*) echo "Need to set a root binary for $os_id" ;; | |||
esac | |||
</pre> | |||
=== Install EPICS === | |||
=== Install MIDAS === | |||
==== Set environment variables ==== | |||
Because we want two midas installs side-by-side for a seamless transition, edit the .bashrc file to ensure the following: | |||
<pre> | |||
# this later needs changing for the final switch to tucan01 | |||
if [ "$HOSTNAME" == daq01.ucn.triumf.ca ] || [ "$HOSTNAME" == tucan01.ucn.triumf.ca ] ; then | |||
unset MIDAS_SERVER_HOST | |||
else | |||
export MIDAS_SERVER_HOST=daq01.ucn.triumf.ca | |||
fi | |||
if [ "$HOSTNAME" == tucan01.ucn.triumf.ca ] ; then | |||
export MIDASSYS=/home/ucn/packages_tucan01/midas | |||
export MIDAS_EXPTAB=/home/ucn/online_tucan01/exptab | |||
export ROOTANASYS=/home/ucn/packages_tucan01/rootana | |||
else | |||
export MIDASSYS=/home/ucn/packages/midas | |||
export MIDAS_EXPTAB=/home/ucn/online/exptab | |||
export ROOTANASYS=/home/ucn/packages/rootana | |||
fi | |||
</pre> | |||
==== Clone and Make ==== | |||
Clone the repository as user ucn then make: | |||
<pre> | |||
mkdir /home/ucn/packages_tucan01 | |||
cd /home/ucn/packages_tucan01 | |||
git clone https://dfujimoto1@bitbucket.org/tmidas/midas.git | |||
cd midas | |||
git submodule update --init --recursive | |||
git pull --recurse-submodules | |||
make cmake | |||
</pre> | |||
==== Setup online directory ==== | |||
=== Install ROOTANA === | === Install ROOTANA === | ||
Many of the UCN analysis codes still rely on ROOTANA. | |||
= Contacts = | = Contacts = | ||
Latest revision as of 22:30, 24 July 2026
The Ultracold Neutron group, now the TRIUMF Ultracold Advanced Neutron (TUCAN) collaboration was created to measure the neutron electric dipole moment (nEDM). This fundamental property, if non-zero, has serious ramifications for resolving the Baryogenesis problem.
Overview
The project has two main components: (a) the UCN source and (b) the nEDM experiment.
The source was largely completed and shown to be functional in 2026. It is operated primarily though EPICS, with monitoring though MIDAS histories with a MySQL database. It operates on beamline 1U (BL1U), splitting off of 1V (which later becomes BL1A). With a fast kicker magnet, the TUCAN collaboration can kick individual beam buckets to BL1U to take a fraction of beam, while enabling beam to BL1A. Design operating current is 40 uA. The UCN source has two end stations for experiments: one for the nEDM experiment, and the other is planned for PENeLOPE.
The nEDM experiment is the primary experiment for the UCN source and is composed of many subsystems, each with their own controls and DAQ setup. Experiments are generally conducted as follows:
- produce UCNs in the source,
- open a valve to let the UCNs diffuse to the experimental chamber
- store the UCNs in the chamber for some time, while applying high voltage, and RF pulses
- open the chamber valve to let the UCNs diffuse to the detectors
The subsystems with DAQ interfaces are as follows:
UCN Source
Produces UCN
Controlled through EPICS, monitored with MIDAS. Some MIDAS controls are implemented, such as the AutoStat page and the UCN sequencer.
Comagnetometer
An atomic gas (199Hg) inhabits the cell the UCNs are stored in. The gas is polarized with a free laser, and that polarization state is also measured with a laser. Measures magnetic field.
DAQ not yet implemented
Cs Magnetometers
An array of glass cells containing Cs gas are distributed inside the magnetically shielded room. The polarization state is also pumped and probed with lasers, this time through fibre optics. Measures magnetic field gradients.
DAQ exists as a set of student python scripts, yet to be integrated into MIDAS
B0 Coil
A coil which produces a highly stable and uniform static magnetic field
DAQ not yet implemented
B1 Coil
A coil which produces the oscillating RF fields for spin flips
DAQ not yet implemented
Shim Coils
An array of coils inside the MSR to homogenize the field
DAQ exists as a python script which controls an arduino, yet to be integrated into MIDAS
The main repository is here, with the arduino code here.
Guiding coils
An array of coils to maintain UCN polarization in their journey to and from the magnetically shielded room
DAQ not yet implemented
Ambient Magnetic Coil (AMC)
A large Helmholtz coil to counteract the cyclotron magnetic field
DAQ not yet implemented
High voltage
A large electric field to tease out the electric dipole moment
DAQ not yet implemented
Ambient monitoring
Monitoring of ambient temperature and magnetic field
Temperature
Measured through "canaries": battery-powered arduinos with a temperature sensor on the TRIUMF IOT network. Integrated into MIDAS. Frontend repo
Magnetic Fields
Measured through fluxgates and a custom DAQ produced by the TRIUMF electronics group. Frontend repo
Machines
The below list is incomplete, needs expanding
| Hostname (.triumf.ca) | Purpose | Location | OS | Status |
|---|---|---|---|---|
| daq01.ucn | Legacy primary DAQ machine and MIDAS host | UCN counting room | CentOS7 | Active |
| daq02.ucn | Runs digitizer frontend | Meson hall B2 | CentOS7 | Active |
| daq04.ucn | Multi-monitor EPICS control station | UCN counting room | Ubuntu 24 | Active |
| daq06.ucn | Test environment | Virtual machine | Ubuntu 24 | Active |
| daq07.ucn | Laser enclosure frontend | Laser enclosure, Meson hall B2 | Ubuntu 24 | Active |
| tucan01.ucn | New primary DAQ machine and MIDAS host | UCN counting room | Ubuntu 24 | In setup |
| ktm-readout.ucn | Kicker timing monitor frontend and web host | Cyclotron roof | Ubuntu 24 | Active, untested |
| ktmserver | Legacy KTM web host | Virtual machine | CentOS7 | Active |
DAQ Software and Setup
Directory Organization
Here we describe the organization and setup of the DAQ computer. While there are many other directories not listed here, they are outside of the core functionality of the main DAQ.
/home/ucn/(shared home)online/(daq01 midas main directory)autostat(python script to automate cryostate functions)epics2midas(read EPICS variables to midas ODB)fe_canary_env_temp(canary temperature monitor)fefluxgate(read custom orange DAQ box for fluxgates)ktm(Legacy KTM frontend)odb_dumps(Main repository of regular ODB dumps)scmfe(SCM frontend)sequence-control(UCN sequencer frontend)shift_schedule(Scheduler frontend)ucn-alarms-notification(Alarm actions: phone/email users)ucn_detector_analyzer(Main UCN counting: processing/online analysis - produce root files)ucn-v1725(CAEN digitizer frontend, runs on daq02.ucn)ucn-web-control(Custom pages)
packages/(source and build directories: MIDAS, EPICS, CAEN, ROOTANA, etc)
/data3/(large storage drive)ucn/midas_files(.mid.gz files and similar)ucn/root_files(.root, for analysis, after preprocessing)
Setup Instructions
Here's the general path to setting up the main DAQ machine (steps taken on tucan01.ucn):
Install ubuntu 24
As per the instructions here
Create online and packages directory
mkdir /home/ucn/online_tucan01 mkdir /home/ucn/packages_tucan01
Install MySQL
Create partition on drive for MySQL histories
In ZFS you don't really create fixed-size "partitions" the way you would with fdisk/gparted. Instead a pool (created from disks) is carved into datasets (filesystems) or volumes/zvols (block devices). Space is shared from the pool by default, and you constrain size with quota/reservation (for datasets) or a fixed volume size (for zvols).
See existing pools
ucn@tucan01:~$ zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT bpool 1.88G 210M 1.67G - - 7% 10% 1.00x ONLINE - rpool 936G 6.09G 930G - - 0% 0% 1.00x ONLINE -
Clearly rpool is the main filesystem. We want to allocate about 700 GB from this for the mysql database:
sudo zfs create rpool/sqlstorage # create sudo zfs set reservation=700G rpool/sqlstorage # guarantee the size that other datasets can't eat into zfs list # check success
Install MySQL
We will install mysql and set the data base location to /sqlstorage (following this tutorial). Note that the mysql error log defaults to /var/log/mysql/error.log in the case of issues.
sudo apt install mysql-server sudo systemctl status mysql-server # should show enabled and active
Now we find the default database directory, should likely be /var/lib/mysql. If not make the substitute in the commands that follow.
sudo mysql -u root SELECT @@datadir; exit;
With this confirmed, we change the data directory:
sudo systemctl stop mysql sudo systemctl status mysql # confirmed inactive sudo rsync -av /var/lib/mysql /sqlstorage # copy database to new location sudo mv /var/lib/mysql /var/lib/mysql.bak # rename old database in case of errors
Now edit the basic settings section of the config file (sudo vim /etc/mysql/mysql.conf.d/mysqld.cnf) to match the following:
# # * Basic Settings # user = mysql # pid-file = /var/run/mysqld/mysqld.pid # socket = /var/run/mysqld/mysqld.sock port = 3306 datadir = /sqlstorage/mysql
Even if apparmor is disabled, you still need to change the alias file and re-enable apparmor, otherwise you will get the error Job for mysql.service failed because the control process exited with error code.. To resolve this, edit the apparmor alias file (sudo vim /etc/apparmor.d/tunables/alias) and add the following line to the end:
alias /var/lib/mysql/ -> /sqlstorage/mysql/,
Apply changes by restarting apparmor, then re-disable:
sudo systemctl restart apparmor sudo systemctl stop apparmor sudo systemctl disable apparmor sudo systemctl status apparmor
Then restart the server
sudo systemctl restart mysql
Confirm that it worked by re-checking the data directory inside mysql
sudo mysql -u root SELECT @@datadir; exit;
When finished, delete the original directory
sudo rm -rf /var/lib/mysql.bak
Uninstall MySQL (only if needed!)
In the case of serious issue, here are steps to completely purge mysql from your system:
sudo systemctl stop mysql sudo apt purge mysql-server mysql-client mysql-common mysql-server-core-* mysql-client-core-* sudo rm -rf /etc/mysql /var/lib/mysql /var/log/mysql sudo apt autoremove sudo apt autoclean
Setup MySQL Users and Database
By default there should be a root user which can access the database without any password. Use that and the following commands to set up the database, following the steps in the midas wiki:
sudo mysql -u root CREATE DATABASE IF NOT EXISTS ucn_history; CREATE USER 'ucn_reader'@'localhost' IDENTIFIED BY 'password'; # save this password somewhere CREATE USER 'ucn_writer'@'localhost' IDENTIFIED BY 'password'; # save this password, different from the reader GRANT SELECT ON ucn_history.* TO ucn_reader@'localhost'; # permissions GRANT SELECT,INSERT,CREATE,ALTER,INDEX ON ucn_history.* TO 'ucn_writer'@'localhost'; flush privileges; SHOW GRANTS FOR 'ucn_reader'@'localhost'; # verify that commands worked SHOW GRANTS FOR 'ucn_writer'@'localhost'; exit;
Setup ODB Connection
First, we check the socket location:
grep sock /etc/mysql/mysql.conf.d/mysqld.cnf
which should result in something like
# socket = /var/run/mysqld/mysqld.sock
denoting the default location. You can uncomment this line, change it, or leave it as is. We assume that this location is ok and use it in the following.
Now we create the files needed. We do this in /home/ucn/online_tucan01 since we are setting this up in parallel with the already-running instance in /home/ucn/online.
Create ucn_reader file: copy/paste all the following lines
cat <<EOF > /home/ucn/online_tucan01/ucn_reader.txt server=localhost port=3306 socket=NULL database=ucn_history user=ucn_reader socket=/var/run/mysqld/mysqld.sock password=THE PASSWORD FROM EARLIER EOF
Create ucn_writer file: copy/paste all the following lines
cat <<EOF > /home/ucn/online_tucan01/ucn_writer.txt server=localhost port=3306 socket=NULL database=ucn_history user=ucn_writer socket=/datassd/mysql/mysql.sock password=THE OTHER PASSWORD FROM EARLIER EOF
Install APACHE
sudo apt install apache2
Install ROOT
We point the installation to the pre-compiled root on daqstore. Edited .bashrc to the following:
# setup root version for different distros
os_id=$(. /etc/os-release && echo "$ID")
case "$os_id" in
ubuntu) source /daq/daqshare/olchansk/root/root_v6.36.06.Linux-ubuntu24.04-x86_64-gcc13.3/bin/thisroot.sh ;;
centos) source /daq/daqshare/olchansk/root/root_v6.24.08.Linux-centos7-x86_64-gcc4.8/bin/thisroot.sh ;;
*) echo "Need to set a root binary for $os_id" ;;
esac
Install EPICS
Install MIDAS
Set environment variables
Because we want two midas installs side-by-side for a seamless transition, edit the .bashrc file to ensure the following:
# this later needs changing for the final switch to tucan01 if [ "$HOSTNAME" == daq01.ucn.triumf.ca ] || [ "$HOSTNAME" == tucan01.ucn.triumf.ca ] ; then unset MIDAS_SERVER_HOST else export MIDAS_SERVER_HOST=daq01.ucn.triumf.ca fi if [ "$HOSTNAME" == tucan01.ucn.triumf.ca ] ; then export MIDASSYS=/home/ucn/packages_tucan01/midas export MIDAS_EXPTAB=/home/ucn/online_tucan01/exptab export ROOTANASYS=/home/ucn/packages_tucan01/rootana else export MIDASSYS=/home/ucn/packages/midas export MIDAS_EXPTAB=/home/ucn/online/exptab export ROOTANASYS=/home/ucn/packages/rootana fi
Clone and Make
Clone the repository as user ucn then make:
mkdir /home/ucn/packages_tucan01 cd /home/ucn/packages_tucan01 git clone https://dfujimoto1@bitbucket.org/tmidas/midas.git cd midas git submodule update --init --recursive git pull --recurse-submodules make cmake
Setup online directory
Install ROOTANA
Many of the UCN analysis codes still rely on ROOTANA.
Contacts
Some useful contact persons (TRIUMF PIs):
- R. Picker
- N. Yazdandoost