View source for CPET DAQ

== Links ==

* [[TITAN DAQ General Documentation]] that explains the core principles of the DAQ system
* [http://titan02.triumf.ca:8081/?cmd=Status DAQ status]
* [http://titan02.triumf.ca:8081/?cmd=custom&page=Scan%20settings Main settings]
* [http://titan02.triumf.ca:8081/?cmd=custom&page=Web%20plots Online plots]
* [https://bitbucket.org/ttriumfdaq/titan_data/src/master/ titan_data repository] for user-friendly python tools for reading the data
* The CPET DAQ system is based on the [https://midas.triumf.ca/MidasWiki/index.php/Midas_documentation MIDAS DAQ package]

== Hardware ==
The DAQ hardware for CPET consists of a VME crate containing the VMIC CPU '''''lxcpet''''' and a VMEIO-based PPG, the [[#PPG32 (Pulse Programmer)]] as shown in Figure 1.

[[Image:Cpet_vme.jpg|center|title|frame|Figure 1: CPET VME Crate and contents]]

It can also use the MCA4 (when not being used by EBIT), an SR430 (when on loan from the laser group), and a LeCroy Wavesurfer 3504 scope. Voltages can be controlled via a LabView integration and a DAC.

=== PPG32 (Pulse Programmer) ===

The PPG used by CPET (and EBIT) is a PPG32, built at TRIUMF (see [http://www.triumf.info/wiki/DAQwiki/index.php/VME-NIMIO32 manual]). The PPG32 is designed to be clocked at 10MHz,
and an internal "divide down" is used to produce frequencies of up to 100MHz. Due to the requirement that the PPG32 be clocked by the special TITAN 100MHz "since wave" pulse generator, the firmware was modified to allow this with a particular value of "divide downs". This also requires that the module be set to TTL Input Clock and Trigger signals. The CPET PPG32 is therefore clocked at 100MHz  either by an internal or external clock. 

The PPG32 supports up to 32 Output channels, and the minimal delay is 3 clock cycles.  The PPG program can be started internally (i.e. by software) or with an external start (trigger) pulse. The External Clock and Trigger signals are either both  NIM or both TTL (selected by jumpers only available on newer boards).

In summary,
* For CPET currently both the External Clock and Trigger signals are TTL , because it uses the TITAN 100MHz pulse generator as the external clock (also used by MPET's  PulseBlaster PPG ). It runs custom firmware  to enable the use of an external 100MHz clock.
* For EBIT, currently both the External Clock and Trigger signals are NIM, because it is using the internal 10MHz clock, stepped up internally to 100MHz. It has an older board (TTL/NIM jumpers absent).
* MPET experiment still uses the PulseBlaster PPG (i.e. the "old PPG"). The PulseBlaster PPG supports up to 24 output channels with a minimal delay of 5 clock cycles. The PulseBlaster PPG is set up with TITAN external clock of 100MHz.  Unlike the PPG32, the PulseBlaster PPG cannot be switched between external and internal clock by software.

=== MCA4 / MCS4 (time-of-flight) ===

The MCA4/MCS4 is a USB-based device that produces time-of-flight histograms. There is one BNC input that starts a clock, and two inputs that can "stop" the clock. The two "stop" channels work independently (i.e. you can measure the "stop" times of both channels even though the "start" signal was only sent once). The bin width of the histograms (aka the dwell time) and the number of bins is configurable. The minimum dwell time is 30ns if only one "stop" channel is being used, or 50ns if both "stop" channels are enabled. The maximum number of bins is 16 million, but reading out that much data would significantly degrade DAQ performance.

=== SR430 (time-of-flight) ===

The SR430 is an old device that produces time-of-flight histograms. There is one input for a trigger signal, and one input for the "stop" (discriminator) signal. The device is currently connected via a GPIB-to-LAN converter. Communication is via the LXI (VXI-11) protocol. We discovered that the C++ vxi11 library (from Nottingham University) was unable to read the binary histogram data reliably. The python implementation works much better.

=== LeCroy Wavesurfer 3504 scope ===

The 3504 is used to measure the electron count. It is connected via USB to the titan-pi04 machine, as the ethernet interface only supports the VICP protocol, which is not available on Linux (most LeCroy scopes also support LXI/VXI-11 over ethernet, but not this one). The usbtmc python package seems to work very reliably.

=== Voltage control ===

CPET runs a LabView program on titan14 (142.90.119.149) that controls various trap voltages. This program includes 2 ZMQ (ZeroMQ/0MQ) reply sockets. Port 5554 accepts the "Send pnames" command, and responds with a multi-part message listing all the parameter names it knows about. Port 5555 accepts a 2-part command of the parameter name and voltage to apply, and will respond with "Success" or "Failure" (the failure message may take up to 45s to appear).

A python program (fe_labview_voltages) running on titan02 handles the midas/labview communication. The main fecpet program uses midas RPC commands to talk to fe_labview_voltages throughout the scan.

=== DAC control ===

CPET has a raspberry pi (titan-pi01) that controls 16 DACs. The pi includes a PHP server that operators use to configure most of the settings, plus a handful of shell scripts. The system uses JSON files to define "instructions", which are then parsed by a "response loop" executable.

A python program (fe_dac) running on titan02 handles the integration between midas and the tools on titan-pi01. The principle is that the operator will configure most of the settings using the PHP interface, which will write a JSON file to disk. fe_dac reads that file, tweaks voltages as needed, writes the updated file back to disk, then instructs the response loop program to apply the updated instructions.

The DAC control can work in 3 modes:

* DAQ controls DACs and we're scanning - update the voltages each step
* DAQ controls DACs and we're not scanning - update the voltages at the start of the run
* DAQ does not control DACs - try to read the current voltages at the start of the run (but not a problem if we can't)

== Feature configuration ==

The DAQ system is configured on the [http://titan02.triumf.ca:8081/?cmd=custom&page=Scan%20settings Scan settings] webpage.

=== Scanning ===

It is possible to automatically scan different EPICS variables, PPG variables (pulse widths, time offsets, formula variables), and trap voltages. See the [[TITAN_DAQ_General_Documentation#Core_principles]] for more information about 1D and 2D scans.

=== MCA4 settings ===

You can specify whether ToF data is read out from the MCA4, and if so, whether to read out data just from STOP1 or also from STOP2. The histogram bin width (dwell time) is the same for both channels, but you may specify a different number of bins for each channel if desired.

=== SR430 settings ===

You can specify whether ToF data is read out from the SR430, and if so the bin width and number of bins.

=== WaveSurfer 3504 settings ===

You can specify whether electron count data is read out from the scope. You can also specify whether a constant background value should be subtracted when calculating the electron count.

The background subtraction can work in 3 ways:
* Constant subtraction - manually measure the background (in uV*s) before the run starts, set the constant background value on the Scan webpage. Each reading will be automatically subtracted from that value.
* Automatic subtraction - if “constant subtraction” is disabled AND you have automatic muting/unmuting of PPG CH15 enabled (the ES Wehnelt pulse), then the analyzer will automatically subtract the unmuted reading from the muted reading.
* No subtraction -  if “constant subtraction” is disabled and you’re not toggling CH15 muting, it will just convert the area to # electrons.

=== PPG muted/unmuted toggling ===

You can specify that data should be taken with a PPG block, then that block unmuted. For CPET, this is particularly useful for subtracting backgrounds when calculating the electron count. The muting/unmuting is done at the inner-most scope of the scan (e.g. we'd do unmuted and muted with an EPICS value of 3, then unmuted and muted with an EPICS value of 4, ...)

=== Voltage scanning ===

You can specify which trap voltages are scanned, and their voltage ranges. TRAP FLOAT is not connected and can be used for testing.

If you add a new voltage option to the LabView program, restart fe_labview_voltages so that it can discover it (it only checks the list of valid voltages at program start).

=== DAC scanning ===

You can specify which DAC input to control, and up to 4 potential wells that get summed to create the voltage distribution.

Rectangular wells of length N increase the voltage of (N-2) channels; the first and last ones remain at the base voltage. Harmonic wells take a quadratic form.

When scanning, you can change any of the well parameters, the offset voltage, or the number of ramp steps.

== Data format ==

In the final data file, each event contains data from a single scan configuration (i.e. X step, Y step, muted/unmuted). The data from all cycles within a supercycle are combined into a single event. The histograms contain only the data from that scan configuration. Each event also contains data explaining the scan configuration that applied to that event.

If the run continued for many loops, there may be several events with the same scan configuration. E.g.

  Event #1: Loop 0, X step 0, muted
  Event #2: Loop 0, X step 0, unmuted
  Event #3: Loop 0, X step 1, muted
  Event #4: Loop 0, X step 1, unmuted
  Event #5: Loop 1, X step 0, muted
  Event #6: Loop 1, X step 0, unmuted
  Event #7: Loop 1, X step 1, muted
  Event #8: Loop 1, X step 1, unmuted

Offline analysis would need to sum the histograms in events #1 and #5 to get all the data for the "X step 0, muted" configuration.

Python tools for reading the data in a user-friendly manner can be found in the [https://bitbucket.org/ttriumfdaq/titan_data/src/master/ titan_data] repository.

If you are logged-on to titan02 as the cpey user, you can run the '''cpetdump''' command to dump data to screen as it is being taken (it's only written at the end of each supercycle).

See [[TITAN_DAQ_General_Documentation#CPET.2FMPET.2FEBIT_data_format]] for the gory details of the file format.

== Software ==

The DAQ system is based on the [https://midas.triumf.ca/MidasWiki/index.php/Midas_documentation MIDAS DAQ package]. Software written for CPET is linked with the MIDAS libraries and uses common framework for MPET/CPET/EBIT. 

The CPET software can be found at ''/home/cpet/packages/cpet'' on '''titan02''' (note that historical versions of the software were located in ''/home/cpet/online'').  

Run control is done using the MIDAS webserver [https://midas.triumf.ca/MidasWiki/index.php/Mhttpd mhttpd] with [https://midas.triumf.ca/MidasWiki/index.php/Custom_Page Custom Pages] written for ''cpet'' users so that they can easily enter the parameters required to program the PPG and to run the DAQ.

=== Computers ===

* titan02 is a "normal" linux PC that runs the main midas programs (mhttpd web server etc). titan02 also runs the CPET DAQ.
* lxcpet is a PC that lives in the CPET VME crate. It has direct access to the PPG (which runs a programmable sequence of pulses to other hardware) and any other hardware in the VME crate. lxcpet uses an NFS mount and NIS accounts, so sees the same /home/cpet as titan02. This machine runs the "fecpet" program.
* titan-pi04 is a raspberry pi that talks to the Fastcom MCA4 and LeCroy WaveSurfer 3504. It can only be accessed from within TRIUMF (either on-site or via the TRIUMF VPN). It does not use NFS/NIS, and is currently configured to only have an "ebit" user, not a "cpet" user (though cpet@titan02 can log in as ebit@titan-pi04 without needing a password). It has its own /home/ebit. Sourcing the use_cpet.sh script will set up an environment that connects to CPET rather than EBIT. 
* titan14 is a Windows PC that runs the LabView program used to control trap voltages.

=== Starting and stopping DAQ programs ===
Programs can be started/stopped from the [http://titan02.triumf.ca:8081/?cmd=Programs Midas programs page]. All programs that appear on that page should be running (green status).

To start/stop everything, you can use the following aliases after logging in as the "cpet" user on '''titan02''':

  kill-all (an alias of ~/package/cpet/bin/kill-all.sh)
  start-all (an alias of ~/package/cpet/bin/start-all.sh)

=== Program listing ===

In the order that programs appear on the [http://titan02.triumf.ca:8089/?cmd=Programs Programs page]:
* mhttpd - midas web server
* PPGCompilerFrontend - converts a user-supplied PPG program into bytecode that can be loaded onto the PPG
* saveload - helps save and load scan settings
* mserver - midas RPC server that allows the programs on lxebit and titan-pi04 to talk to the main experiment on titan02
* fecpet - main CPET frontend that talks to the PPG, scans EPICS values etc. Runs on lxcpet.
* Logger - midas data logger
* fe_mca4 - frontend that interacts with the MCA4 and sends the histogram data to fecpet at the end of each supercycle
* cpetana - python analyzer that creates plots based on live data (plots are viewed via the [http://titan02.triumf.ca:8081/?cmd=custom&page=Web%20plots Web plots] page)
* fe_sr430 - frontend that interacts with the SR430 and sends the histogram data to fecpet at the end of each cycle
* fe_ws3504 - frontend that interacts with the WaveSurfer3504 and sends the histogram data to fecpet at the end of each cycle
* fe_labview_voltages - frontend that interacts with the LabView script running on titan14 that controls trap voltages.
* fe_dac - frontend that interacts with the DAC tools running on titan-pi01.

=== Pulling the latest version of the CPET DAQ software ===
  cd ~/packages/cpet
  git pull
  git submodule update

=== Building the CPET DAQ software ===
To build the C++ code {{File|name=fecpet}}, log onto '''titan02''' as user "cpet".
 cd ~/packages/cpet/build
 cmake3 ..
 make
 make install

This will build the appropriate 32-bit frontend program that can be used by lxcpet.