POL: Hardware and software
Links
Hardware
The VME hardware for the POL DAQ system is located in a VME crate in a blue rack on the ISAC-1 experimental area floor underneath the BNQR platform.
The VME crate contains 3 modules:
| Name | Purpose | VME Base Address |
|---|---|---|
| VMIC | CPU running linux | |
| SIS3820 | MultiChannel Scaler (MCS) | 0x38000000 |
| VME-PPG32 | TRIUMF Pulse Programmer (PPG) | 0x00100000 |
Other DAQ hardware required is the Galil RIO DAC/ADC. The VMIC communicates with this module via a
private Ethernet connection.
| Hardware Module | Purpose |
|---|---|
| Galil Rio-47120 16bit | DAC/ADC |
A diagram of the DAQ Hardware setup for the Pol Experiment is shown in Figure 1 below.
- Figure 1
- Diagram of the DAQ Hardware setup for the Pol experiment
| Module | Name | I/O | Channel Number(s) | Purpose |
|---|---|---|---|---|
| SIS3820 | MCS | input | 21-24 | data channels (lemo)) referred to in this document as Scaler Inputs 0,1,2,3 |
| input | 1 | control channel (LNE**) connected to PPG Output 1 (MCS Next) | ||
| VME PPG-32 | PPG | output | 1 | MCS Next connected to MCS control input 1 |
| output | 2 | Cntr Gate debug only | ||
| output | 3 | Start TOF to experiment to start the TOF | ||
| output | 4 | PPG Running debug only | ||
| output | 5-8 | Duplicate of outputs 1-4 respectively for debugging | ||
| input | Ext. Trig | External trigger starts PPG |
- ** Load Next Event
VME Crate
Four MCS (Multi-Channel Scaler) channels are enabled, referred to in this document as Scaler Inputs 0,1,2,3. Input 1 is currently connected to a pulser to give a constant count.
Two PPG Outputs (1 and 3) are used in the experiment (more are programmed but are used for debugging purposes).
- PPG output 1 (MCS Next) is connected to SIS3820 Input 1 (LNE). The PPG provides the "Load Next Event" pulse for the SIS3820, i.e. it advances the time bins.
- PPG output 3 (DAQ SVC) produces a pulse at the end of the PPG cycle to send a signal to the RFQ and start the ToF.
Except for the first PPG cycle at the beginning of each SuperCycle, the PPG is started externally by an input signal from the experiment sent at the end of the ToF to the PPG External Trigger input.
Note: The minimum width of the external input "PPG start" signal is 10ns.
Galil Rio 47120 DAC/ADC (16 bits)
Galil Rio Documentation can be found on the galil website. Local copies can be obtained at the links below:
The Galil Rio used by Pol is 16 bit, meaning the LSB is .0003V when running in ±10V range. The user can choose to run with a ±5V, 0-10V, or 0-5V range instead to reduce the LSB.
The IP address of the Galil rio device is currently defined as 192.168.1.100 (note it uses a private link to lxpol).
The table below shows the channels in the Galil Rio that are currently used in the Pol Experiment and the typical ranges.
| Module | Name | I/O | Channel | Usual range | Purpose | Connected |
|---|---|---|---|---|---|---|
| Galil Rio-47120 16bit | DAC | Output | 0 | +/- 10V | DAC Set Value | DAC Readback Value |
| Output | 1 | +/-5V | DVM | |||
| ADC | Input | 0 | +/- 10V | DAC Readback Value | DAC Set Value | |
| 1 | +/- 5V | User DAC Readback | HV Monitor | |||
| 2 | +/- 10V | User Readback | ||||
| 3 | +/- 10V | User Readback |
The POL Experiment is presently set up so that the DAC set value is output on Channel 0 of the DAC. This is connected directly to Channel 0 of the ADC to provide a direct readback of the voltage output by the DAC (for sanity-checking during analysis). It is also connected to the HV power supply to drive the output voltage.
Channel 1 of the ADC is reserved for a user-supplied readback. This is connected to the HV power supply output monitor.
The users can connect Channels 2 and 3 of the ADC as required for monitoring purposes. All the channels shown in the table above are read out into the data banks.
The ADC Average values are also sent out in one of the data banks.
DAC and ADC channel and range settings
The DAC and ADC channel numbers and range settings can now be changed by the users, rather than being hard-coded in to the frontend.
ADC Averaging program
A program has been downloaded into the Galil Rio to perform
averaging of the first 4 ADC channels (0-3). A copy of this
program can be found in Appendix 1.
It takes parameters Filter Window (W) in volts and Filter Factor (F)
(both arrays). Default values are F[*]=100 W[*]=0.1 volts.
The Filter Window and Factor values for each channel have been
stored in the ODB so they can be easily changed. They are located in the
ODB at /Scanning/GalilRIOScan/RangeSettings/Filter window and
/Scanning/GalilRIOScan/RangeSettings/Filter factor.
The program continually averages the ADC input values. If the
difference between the average value and the latest readback is
greater than W[i] for that channel i, the average is cleared and a
new average calculated. This can be used to detect the DAC having
stepped. The Filter Factor F is used to start a new average after a
certain number of readings.
Software
How the Frontend Program Works
The Frontend program fepol_32bit.exe controls the DAQ hardware in the VME crate and sends out the data in the form of MIDAS-format data banks. (See POL frontend code for location of code, and how to build the frontend).
At the beginning of run, the PPG program is compiled based on the user's settings (number of time bins etc), then loaded onto the PPG. An example of the PPG pulses that get output is shown below.
Each time a signal is sent to CH1, the scaler moves to the next time bin. At the end a signal is sent to the RFQ. A signal from the RFQ is used to start the PPG program again.
In the new DAQ a cycle is defined as the scalers having seen N time bins. That means the PPG program is run multiple times within a cycle (e.g. if there are 100 time bins per program, and the cycle requires 1000 bins, the program will run 10 times). The first iteration is meaningless, as it is started by software rather than the RFQ (so the timing is not synced with any ToF of interest). The first time bin of each iteration is also meaningless, as the scaler has been counting for the entire time between when we sent a signal to the RFQ and when the RFQ sent a signal to us.
Once the scaler has seen all the bins we told it to expect, the data for this cycle can be completely read out and sent to midas banks.
The frontend also keeps track of the buffer level in the MCS, and will read data during the cycle if the buffer is getting full. This means both that the buffer will never get full (causing data loss) and that the final end-of-cycle readout is faster (reducing deadtime).
Once the cycle is ended and data has been read from the scaler, the frontend program increments the DAC voltage.
The next cycle is then started by re-arming the scaler and starting the PPG by a software signal.
Stepping the DAC
The Galil Rio DAC used by the POL experiment is 16-bit, meaning the LSB is 0.0003 V for a DAC range of +/- 10Volts.
The old DAQ had some complicated logic for ensuring that the DAC voltage had actually changed, but in practice these checks were disabled via a flag in the ODB. Similarly, the new DAQ does not do any sanity-checks of the DAC output - it just assumes that the voltage has changed. Note that the readback voltages are stored in the output data file.
Running the experiment (Real Data)
First - ensure that the PPG cables are connected to the "Real RFQ" inputs on the yellow plate above the VME crate.
The following settings should be set:
- Scan settings
- Enable scanning - true
- Scan Galil-RIO - whatever voltage range should be scanned
- Scan Epics device - N/A
- Timing settings
- Use pulsed beam - true
- Histogram settings
- Ignore first bin - true
- Ignore first iteration - true
Testing the DAQ
There are two ways to run the DAQ without using the real RFQ.
- Set "Use pulsed beam" to false in the "Timing settings" section of the main webpage. This will both disable the external trigger and will cause us to configure the PPG to run its program N times automatically. This means we will run N times from a just a single start command issued by software.
- Connect the PPG to a "fake RFQ" in the NIM crate. This is just a simple dual gate generator that will see the signal we normally send to the RFQ and immediately generate a pulse to restart the PPG.
TODO - add a wiring diagram.
Environment variables
The name of the MIDAS experiment is "pol" and it is currently run from midpol. Relevant Environment variables are set up as follows:
MIDASSYS=/home/pol/packages/midas MIDAS_EXPTAB=/home/pol/online/exptab MIDAS_EXPT_NAME=pol_new HOSTNAME=midpol.triumf.ca DAQ_HOST=midpol VMIC_HOST=lxpol
The experimental clients are run on the host machine midpol except for the frontend code fepol_32bit.exe which is run on the VMIC lxpol in POL's VME crate. lxpol has access to the pol user's home directory via NFS.
Building the software
Updating POL software
The main repository uses submodules for some dependencies, so checking out updates requires:
git pull git submodule update
To compile the real software, run the following on lxpol from the repository root:
# For the first install mkdir build cd build cmake .. make install # For updates cd build make install
This will:
- Compile the frontend to run on the 32-bit VMIC machine
- Compile various debug executables
- Compile various automated tests
To compile a dummy version of the software that does not talk to the real hardware or EPICS (and which can be run on a laptop):
mkdir build cd build cmake .. -DDUMMY_MODE=1 make install
Updating midas
On midpol:
# Get updates cd $MIDASSYS git pull git submodule update # Compile 64-bit version cd build make install # Change status page from midas default to a symlink if ! [ -L $MIDASSYS/resources/status.html ]; then mv $MIDASSYS/resources/status.html $MIDASSYS/resources/status.html.orig ln -s ~/packages/pol/pol_settings.html $MIDASSYS/resources/status.html fi
On lxpol:
# Compile 32-bit version cd $MIDASSYS make linux32
Then re-compile the POL software.
Note that midas and the POL software do support cross-compilation, but Ubuntu 22.04 (on midpol) and Debian 11 (on lxpol) don't play nicely together (lxpol glibc is too old). Therefore we must compile the 32bit software on lxpol itself. This is slow!
Scripts
start-all
start-all is an alias to the script "/home/pol/online/pol/bin/start-daq-tasks". This script will start the various clients for the experiment, including the Midas web browser client mhttpd on port 8088. The experiment is controlled through the Midas web browser (see Main Status page).
start-all will also start the frontend program running using the screen manager utility screen, which multiplexes a physical terminal between several processes. This has the advantage that deleting the frontend window (by mistake) will not kill the frontend program. Furthermore, the frontend window can be shown on any terminal, not just the terminal where start-all was originally run.
- NOTE
- If this is not successful, or the frontend window immediately
disappears, there may be hardware issues. See POL#Troubleshooting">troubleshooting.
start-all can be run at any time during the experiment, and will restart any of the clients if they have died. Clients can also be restarted using the
mhttpd programs page.
show-windows
A script show-windows run on any terminal (logged in as pol to midpol) will display the frontend window. An alias show-fe has been set up to run show-windows.
Appendix 1: Averaging Program
This program by Donald Arseneau is downloaded in the Galil Rio module and performs averaging. It can also be found at
~pol/online/rio/POL.dmc.
REM Galil RIO used for ADC and DAC and digital IO by POL REM Version 1.01 20-Jun-2014 REM REM variables: REM AV[] filtered voltage readings REM F[] Filter number or factor (Set like F[0]=30) REM W[] Filtering window in volts. REM REM Query a filtered reading with "MG AV[i]" REM Query all four with "MG AV[0],AV[1],AV[2],AV[3]" REM ------------------------------ REM Automatic initialization. REM ------------------------------ #AUTO REM ------------------------------ REM Device configuration: REM Digital outputs logic-normal REM Analog inputs 0-3 range +-10V REM Analog outputs 0-3 range +-10V REM ------------------------------ IQ65535 AQ0,2;AQ1,2;AQ2,2;AQ3,2 DQ0,4;DQ1,4;DQ2,4;DQ3,4 REM ------------------------------ REM Set up filtering for readbacks. REM DFloop runs in thread 3 REM Default filter factor 50, REM Default filtering window 0.1V REM ------------------------------ DM F[8];DM W[8];DM AV[8] F[0]=50;F[1]=50 F[2]=50;F[3]=50 W[0]=0.1;W[1]=0.1 W[2]=0.1;W[3]=0.1 I=0 #IDF AV[I]=@AN[I];W[I]=1.0;F[I]=100;I=I+1 JP#IDF,I<8 XQ#DFloop,3 EN REM ------------------------------ REM The digital filter for analog inputs 0-4 REM AV[] The averaged voltages REM F[] The filter number REM W[] The window (volts) REM Query filtered readings with "MG AV[i]" REM ------------------------------ #DFloop I=0 #DFch IF(F[I]<1);F[I]=1;ENDIF VI=@AN[I];FI=F[I] IF(@ABS[(VI-AV[I])]>W[I]);FI=1;ENDIF AV[I]=AV[I]*(FI-1)/FI+(VI/FI) I=I+1 JP#DFch,I<4 JP#DFloop EN REM 56789 123456789 123456789 123456789