AgWiki - User contributions [en]

Equipment

2022-07-19T08:40:00Z

Acapra:

Back to [[Main Page]]

= UPS - Tripp-Lite - Model SUINT1500LCD2U =

SmartOnline 208/230V 1.5kVA 1.35kW Double-Conversion UPS, 2U, Extended Run, no(SNMP Card Option), LCD, USB, DB9, ENERGY STAR
* [[Image:UPS-TRIPP LITE-SUINT1500LCD2U.pdf|thumb|UPS-Tripp-Lite]]

= PL512 - WIENER - Low Voltage Power unit =

Cable plug STAK3N with the locking retainer STASI3

Hirschmann connector: https://sc2.premierfarnell.com/sc/product.aspx?productid=1176412
* [[Image:Manual_PL512_PL506_00679_A4.pdf|thumb|LV-PL512]]

= R1471 - CAEN - High Voltage Power Supply =
4 channels 0..8KV 3mA

* [[Image:R14xx_rev7.pdf|thumb|HV-R1470]]

= MV2 - Metrolab - 3-axis Hall device =

* [[Image:MagVector-MV2-Datasheet-v1.1.pdf|thumb|MV2]]

= ETH002 - 2 x 16A ethernet relay =

[http://www.robot-electronics.co.uk/eth002-16amp-2-channel-ethernet-relay.html Online Documentation]
* [[File:Eth002.pdf|thumb]]

= Quantel Laser power supply ICE450 and Laser system Ultra =
* [[File:Brio_ICE450_Power_Supply_Manual.pdf|thumb]]
* [[File:Full Laser System Short Manual.pdf|thumb|Full Laser System Short Manual]]
* [[File:LaserSpecs.pdf|thumb|Laser Specifications]]

= ThermoFisher ThermoChill III chiller =
* [[File:Thermochill 3 Manual.pdf|thumb|Thermochill 3 Manual]]

= Alpha Omega Instruments Trace Oxygen Analyzer SERIES 3000 =
* [[File:Series_3000_Manual.pdf|thumb|Oxygen Analyzer Manual]]

File:Series 3000 Manual.pdf

2022-07-19T08:39:18Z

Acapra: Alpha Omega Instruments Corp. SERIES 3000 TRACE OXYGEN ANALYZER USER MANUAL Rev 3.0517, May 2017

== Summary ==
Alpha Omega Instruments Corp. SERIES 3000 TRACE OXYGEN ANALYZER USER MANUAL
Rev 3.0517, May 2017

Equipment

2022-07-19T08:37:12Z

Acapra:

Magnetometry

2022-06-29T17:56:11Z

Acapra:

> > > > NMRxy: NMR probe combined with MV2 Hall probe
> > > > Mxy: MV2 Hall probe only

{| class="wikitable sortable"
|-
! NMR+Hall !! PWB !! Column !! Row !! Approx. Location
|-
| NMR1 || PWB08
|-
| NMR2 || PWB06
|-
| NMR3 || PWB02
|-
| NMR4 || PWB78
|-
| NMR5 || PWB33
|-
| NMR6 || PWB65
|-
| NMR7 || PWB18
|-
| NMR8 || PWB45
|-
| NMR9 || PWB10
|-
| NMR10 || PWB64
|-
| M11 || PWB13
|-
| M12 || PWB49
|-
| M13 || PWB11
|-
| M14 || PWB03
|-
| M15 || PWB35
|-
| M16 || PWB67
|}

[[File:TPC magnetometer locations.pdf|thumb|Location of NMR+Hall in the TPC]]

Magnetometry

2022-06-29T17:54:54Z

Acapra:

{| class="wikitable sortable"
|-
! NMR+Hall !! PWB !! Column !! Row !! Approx. Location
|-
| NMR1 || PWB08
|-
| NMR2 || PWB06
|-
| NMR3 || PWB02
|-
| NMR4 || PWB78
|-
| NMR5 || PWB33
|-
| NMR6 || PWB65
|-
| NMR7 || PWB18
|-
| NMR8 || PWB45
|-
| NMR9 || PWB10
|-
| NMR10 || PWB64
|-
| M11 || PWB13
|-
| M12 || PWB49
|-
| M13 || PWB11
|-
| M14 || PWB03
|-
| M15 || PWB35
|-
| M16 || PWB67
|}

[[File:TPC magnetometer locations.pdf|thumb|Location of NMR+Hall in the TPC]]

PWB

2021-08-03T05:10:35Z

Acapra: /* Hardware */

= Links =

* https://bitbucket.org/expalpha/pwb_rev1_firmware
* https://edev-group.triumf.ca/fw/exp/alphag/feam/rev1
* https://edev-group.triumf.ca/hw/alphag/feam/rev1
* https://ladd00.triumf.ca/daqinv/frontend/list/170 - PWB inventory database

= Schematics =

* [[Image:pwb_rev0.pdf]]
* [[Image:Pwb-rev1-after.pdf]]
* [[Image:AlphaG Rev1.pdf]]

= Manuals =

* [[Image:AFTER_MANUAL_V1.pdf]] AFTER SCA manual v1
* [[Image:AFTER_DataSheet1_1.pdf]] AFTER SCA manual v1.1
* [[Image:AFTER_SCA_table2.pdf]] AFTER SCA channel map

= Hardware =

== Power distribution ==

External power supply is:
* +5V
* +2V

Central board:

* Ext_5V - F1 - RS1 (0.1R) - Fuse_5V
* Ext_2V - F2/F3 - RS2 (0.1R) - Idig2-
* LDO3 - Fuse_5V - DIG_3.3V - selfenable - ADM7154
* LDO5 - Fuse_5V - DIG_2.5V - selfenable - ADM7154
* LDO_CLN - Fuse5V - CLN_3.3V - selfenable -
* VREG - DIG_3.3V - -2.5V SCA BIAS - selfenable - LTC1983ES6-3#TRMPBF
* LDO7 - DIG_2.5V - DDR3_VRef - enable DDR3_1P5V, DDR3_PG, DDR3_VttEN
* LDO2 - Idig2V- - DIG_1.1V - enable DIG_3.3V - LT3071
* LDO6 - Idig2V- - DDR3_1P5V - enable DIG_3.3V - LT3083

SCA Wing:

* Ext_5V - F1 - RS1 (0.3R) - Isca- - SCA_Fuse5V
* LDO1 - SCA_Fuse5V - SCA3.3V - enable ADC_PWR_EN - ADM7154
* LDO2 - SCA3.3V - ADC1.8V - selfenable - ADM7154
* LDO3 - SCA3.3V - TRN2.5V - selfenable - LT1761
* REF - SCA3.3V - 1.45V Vicm - selfenable - LM4121

== Trigger and Clock distribution ==

Clock (62.5MHz) and trigger are distributed to the PWB columns via the [https://edev-group.triumf.ca/hw/alphag/trigger-card-sata-adapter/rev1 trigger SATA adapter].

The two miniSAS cables from cdm03 port 5 and 6 are connected to a miniSAS-SATA splitter 1-to-4 (CS Electronics "iSAS SATA adapter" ADP-887P-1X). Since there are only 8 columns and the clock and trigger signals are daisy-chained across the 8 rows, this scheme is sufficient to provide the necessary inputs to all 64 PWBs.

[https://www.techcable.com/sas-mini-sas-cables/sata-cables/ SATA 6Gb/s Cable with Latching Clip. Straight Pinout 1-1 247726-L 40in]

== Optical Link ==

Main data uplink.

SFP (Small Form Pluggable) Avago Technologies AFBR-57M5APZ [[File:AVAGO SFP specs.pdf|SFP (Small Form Pluggable) Avago Technologies AFBR-57M5APZ]]

Fiber optic Tripp Lite N820 - 10 m [[File:FiberOptic TrippLite specs.pdf|Fiber optic Tripp Lite N820 - 10 m]]

== SATA Link ==

Secondary data uplink, capable also to deliver clock and trigger.

The SATA link connects PWB from adjacent columuns on the same row.

The SATA cable is a "crossed cable".

= Firmware =

== Firmware update ==

PWB firmware update is done using "esper-tool".

<pre>
esper-tool -v upload -f file.rpd http://pwbNN update factory_rpd
esper-tool -v upload -f file.rpd http://pwbNN update file_rpd
</pre>

As of PWB firmware pwb_rev1_20200706_ko:

Permission to write into the flash memory is controlled by esper variables:

* allow_write set to "false": "esper-tool upload" does a "verify": successful upload means epcq flash content is same as rpd file, error means content is not the same.
* allow_write set to "true": "esper-tool upload" updates the epcq flash from the rpd file: for each data block, read the epcq flash, compare to rpd data, if different, erase the flash block, write the flash, read the flash, if there is a mismatch (bad write), return an error.
* allow_factory_write set to "true" to enable writing to the "factory_rpd". DO NOT SET IT TO "true" UNLESS YOU MEAN TO UPDATE THE PWB BOOTLOADER. IF YOU FLASH BUM FIRMWARE or IF THERE IS A POWER OUTAGE WHILE UPDATING, THE PWB WILL BE BRICKED and has to be recovered by connecting a USB blaster.

To update firmware of multiple boards to correct version, recommended is the script "update_pwb.perl":
<pre>
cd ~/online/src
./update_pwb.perl pwb12 pwb06 pwb78
</pre>

== Factory page and user page firmware boot ==

Each PWB board has 2 firmware images: boot loader (factory page)
and data acquisition (user page). On power up, the FPGA loads and runs
the boot loader firmware from the factory page, later the control
software reboots the FPGA into the data acquisition firmware from
the user page.

Because loading defective firmware can brick https://en.wikipedia.org/wiki/Brick_(electronics)
the PWB, new firmware images are loaded into the user page, if anything is wrong,
the PWB can still boot from the factory page and one can use the firmware update
function to load a good firmware image into the user page.

The boot loader firmware in the factory page is not intended for data acquisition, it is only meant to provide three functions:
* boot enough hardware and software to communicate via the ethernet and the sata link
* firmware update (both factory page and user page)
* reboot into the user page

The boot loader firmware in the factory page is usually never updated unless absolutely
needed to fix a problem with these three functions (i.e. compatibility with sata link communications).

If factory image is corrupted or the contains defective firmware and the PWB does not boot (no dhcp, no ping, no esper),
the only way to unbrick it is by loading good firmware using a jtag flash programmer (usb-blaster). Connecting it requires
physical access to the jtag connector on the PWB board. It is impossible when the detector is fully assembled in the experiment.

The PWB has a 32 Mbyte EPCQ flash memory chip, it is divided into 2 pages 16 Mbytes of factory page and 16 Mbytes of user page.

A complete firmware image generally contains 3 pieces:
* FPGA firmware and NIOS "feam_bootloader" software RAM image (sof file)
* NIOS "feam" software RAM image (feam.elf.flash.hex) - all the C code for esper communications and PWB data acquisition
* NIOS filesystem image (feam.webpkg.hex) - esper web pages

This is the boot sequence:
* on power-up the FPGA automatically loads the sof file from address 0 of the EPCQ flash (this is the factory image sof file)
* the FPGA is "started"
* the NIOS CPU starts executing the "feam_bootloader" C code embedded in the sof file (NIOS project hdl/software/feam_bootloader)
* feam_bootloader initializes the hardware (clocks, DDR memory, etc)
* feam_bootloader write-protects the EPCQ flash memory
* feam_bootloader copies the "feam" software from EPCQ flash to the DDR memory (after checking for correct checkum)
* feam_bootloader restarts the NIOS CPU
* the NIOS CPU starts running from DDR memory, executes the "feam" software (NIOS project hdl/software/feam)
* call main() in hdl/software/feam/src/task_init.c
* call task_init() in the same file
* infinite loop waiting for IP address via DHCP
* after have IP address, call task_esper() from hdl/software/feam/src/task_esper.c
* task_esper() creates all the esper modules, esper variables, etc,
* mod_http.c starts the mongoose web server
* PWB is open for business

This is the protection against booting corrupted firmware:
* FPGA hardware loads the sof file from epcq flash and checks for correct checksum before "starting" it
* NIOS CPU is part of the sof file, protected by sof file checksum
* NIOS feam_bootloader C code is part of the sof file, protected by the sof checksum
* feam_bootloader checks for correct signatures and checksums of the DDR memory image
* "feam" C code is protected by the checksum of the DDR memory image

== NIOS terminal ==

<pre>
$ ssh agmini@daq16
$ /opt/intelFPGA/16.1/quartus/bin/jtagconfig
1) USB-Blaster [2-1.2]
02B030DD 5CGTFD7(B5|C5|D5)/5CGXBC7B6/..
$ /opt/intelFPGA/17.0/quartus/bin/nios2-terminal
nios2-terminal: connected to hardware target using JTAG UART on cable
nios2-terminal: "USB-Blaster [2-1.2]", device 1, instance 0
nios2-terminal: (Use the IDE stop button or Ctrl-C to terminate)
PWB Revision 1 Boot Loader
Ver 2.0 Build 357 - Wed Jun 6 15:05:35 PDT 2018
...
</pre>

== Flash boot loader firmware via jtag ==

<pre>
$ ssh agmini@daq16
$ /opt/intelFPGA/16.1/quartus/bin/jtagconfig
1) USB-Blaster [2-1.2]
02B030DD 5CGTFD7(B5|C5|D5)/5CGXBC7B6/..
$ cd ~/online/firmware/pwb_rev1
$ ls -l
$ /opt/intelFPGA/17.1/quartus/bin/quartus_pgmw
... auto detect
... load the jic file
... in menu tools->programmer, enable "unprotect device"
... start program/configure operation
</pre>

== Flash user page firmware via esper-tool ==

<pre>
$ ssh agmini@daq16
$ cd online/src
$ more update_pwb.perl ### check that $fw is set to the desired firmware file
$ ./update_pwb.perl pwb06 ### or give more PWB names or give "all"
</pre>

== Build firmware ==

=== quartus version to use ===

* quartus 16.1 should be used for jtag (jtagconfig and jtagd)
* quartus 17.0 should be used to build the PWB firmware (17.1 is not compatible)

=== start license server ===

<pre>
ssh agmini@daq16
/opt/intelFPGA/17.0/quartus/linux64/lmgrd -c ~agmini/online/license-daq16.dat
</pre>

=== prepare the build environment ===

<pre>
$ ssh agmini@daq16
$ /opt/intelFPGA/17.0/nios2eds/nios2_command_shell.sh
</pre>

=== get latest version of the code and build it ===

<pre>
$ cd online/firmware/git/pwb_rev1_firmware
$ git pull
$ git checkout alphag
$ git pull
$ ./scripts/compile_project.sh
... about 30-40 minutes ...
$ ls -l bin/*.sof bin/*.jic bin/*.rpd
-rw-r--r-- 1 agmini alpha 12727389 Jan 24 2018 bin/feam_rev1_auto.rpd
-rw-r--r-- 1 agmini alpha 33554661 Jan 24 2018 bin/feam_rev1.jic
-rw-r--r-- 1 agmini alpha 6974754 Jan 24 2018 bin/rev1.sof
$ ### feam.jic is loaded via jtag
$ ### feam_auto.rpd is loaded via esper
$ ### feam.sof is used to attach the quartus signal tap logic analyzer
</pre>

=== firmware build order and sequence ===

* erase previous build
* regenerate qsys
* regenerate NIOS BSP "feam_bootloader_bsp" and "feam_bsp"
* build NIOS projects "feam_bootloader" and "feam"
* build quartus sof file (with the "feam_bootloader" nios project inside)
* build jic and rpd files (with the quartus sof file and the "feam" nios project inside)
* load sof file into the FPGA using jtag (be careful about compatible "feam" nios project already in the epcq flash)
* load jic file into the epcq factory page using jtag (cycle power for them to take effect)
* load rpd file into the epcq factory or user page using esper

=== scripts for building the firmware ===

* scripts/compile_project.sh - build everything from scratch:
* scripts/compile_qsys.sh - regenerate the main qsys component
* scripts/compile_nios.sh - regenerate the BSPs, then same as compile_elf.sh
* scripts/compile_elf.sh - build the BSPs, "feam_bootloader" and "feam" NIOS projects
* scripts/update_elf.sh - regenerate the rpd and jic files to include the new "feam" NIOS build (keeps the old "feam_bootloader" in the sof file)
* scripts/update_bootloader.sh - also regenerate the sof file to include the new "feam_bootloader" build
* scripts/clean_quartus.sh - prepare quartus fpga sof file to build from scratch
* scripts/compile_quartus.sh - build the quartus fpga sof file incrementally (run clean_quartus.sh to build from scratch).
* scripts/load_jtag_sof.pl bin/rev1.sof - load sof file into the FPGA. this will boot the new fpga firmware, the new feam_bootloader (if update_bootloader.sh was run), but with the old "feam" already in epcq flash. (beware of incompatible sof, "feam_bootloader" and "feam" builds!)
* scripts/load_jtag_jic.pl bin/feam_rev1.jic - load jic file into the EPCQ flash, same as using the graphical flash programmer tool. (this will leave the FPGA running the quartus jtag flash loader firmware, not the PWB firmware. Cycle power for new jic file to take effect).

== ESPER Variables ==

* Board
** invert_ext_trig - invert trigger signal from CDM before it drives any logic (to undo incorrect signal polarity)
** reset_nios - goggle up, them down to reset NIOS subsystem

* Signalproc
** test_mode - ADC data is replaced with a test pattern, see test mode bits in sca_x_ch_ctrl.
** sca_a_ch_ctrl, sca_b_ch_ctrl, sca_c_ch_ctrl, sca_d_ch_ctrl - channel control bits:
<pre>
11..0 - threshold - channel suppression threshold
14..12 - ctrl_test_mode - test mode:
0=fixed patter 0xa5a,
1=time bin counter,
2=time bin counter with channel number,
3=sequential adc sample counter,
4={ch_crossed_out,trig_pos,trig_neg,adc[8:0]},
5={trig,adc[10:0]},
6={ch_crossed_min,adc[10:0]}
15 - ctrl_supp_mode - channel suppression mode: 0=adc<=(baseline-threshold), 1=adc<=threshold
</pre>

* Link
** link_ctrl - sata link control. The bits are:
<pre>
0 - sata_link_udp_stream_in_enable - permit data flow from sata link to OFFLOAD_SATA
1 - udp_stream_out_enable - permit sca data flow to sata link
2 - sata_to_eth_enable - permit ethernet data flow from sata link to TSE_MAC
3 - eth_to_sata_enable - permit ethernet data flow from TSE MAC to sata link
4 - enable_stop_our_tx - enable flow control: allow stop_tx
#5 - stop_our_tx - manually activate the flow control signal into link_tx
6 - enable_stop_remote_tx - enable flow control: allow send "stop_tx"
#7 - stop_remote_tx - manually activate the flow control signal into link_tx
8 - tx_test_pattern_udp - udp data is replaced by test pattern 0x11111111, 0x22222222, etc.
9 - tx_test_pattern_eth - nios-to-sata data is replaced by test pattern 0x11111111, 0x22222222, etc.
10 - udp_delay_enable - delay between udp packets, see udp_delay_value below
11 - reboot_tx - send K_REBOOT command to the sata link mate, where it has same effect as board/reset_nios.
12 - sata_to_nios_disable
13 - nios_to_sata_disable
14 -
15 -
16 -
24..31 - udp_delay_value - delay between udp packets (top 8 bits of a 20-bit counter)
</pre>

== Firmware data path ==

Main data path:
<pre>
feam_top
|
sca_sigproc
|
sca_event_control
4 * sca_control (sca_channel.sv)
4 * channel_fifo (1024 samples)
</pre>
<pre>
sca_control (sca_channel.sv)
|
state machine to control SCA read and write enables
|
sca_write_control (what does it do?!?)
sca_read_control
</pre>
<pre>
sca_read_control
|
state machine to store ADC samples into per-channel FIFOs (channel_fifo)
selector to replace ADC samples with a test pattern
79*sca_trig_one -> ch_crossed_out - channel hit detector
</pre>
<pre>
sca_event_control
|
event_fifo (hdl/mf/event_descriptor_fifo)
state machine to take ADC data from 4 per-channel FIFOs (channel_fifo) and store it into DDR memory (write addr increment is 510*8 (number of samples)*8 = 4080).
state machine to read transposed ADC data from DDR memory (read addr increment is 8+8=16 - 2 samples * 2 bytes * 4 sca = 16) and create 4 per-sca data streams
</pre>
<pre>
data stream:
4 per-sca data streams from sca_event_control -> sca_sigproc(event_dat)
|
packet_chunker (hdl/lib/packet_chunker.sv) (4*event_dat -> event_segment_dat) - multiplex 4 data streams into 1 stream of UDP-sized packets
|
packet_length_prepender (hdl/lib/packet_length_prepender.sv) (event_segment_dat -> udp_event_val_out)
|
udp_event_val -> info qsys (udp_stream_sca) and into sata link (mux channel 2).
</pre>

== Firmware build instructions from Bryerton (obsolete) ==

This is the old README.md file from the firmware git repository.

These instructions are not used to build the firmware (use scrips/compile_project.sh).

They are still useful for figuring out which buttons to push in the altera gui programs.

<pre>
# ALPHAg FrontEnd Acquisition Module (FEAM) - REV0

Welcome to the ALPHAg FEAM Project

# Table of Contents

* [Overview](#overview)
* [Inital Setup](#initial-setup)
* [Quartus Setup](#quartus-setup)
* [Create the .cdf](#create-the-cdf)
* [NIOS Setup](#nios-Setup)
* [Create the project files](#create-the-project-files)
* [Build Instructions](#build-instructions)
* [Building from Quartus GUI](#building-from-quartus-gui)
* [Generate the QSys File](#generate-the-qsys-file)
* [Compile the NIOSII Project](#compile-the-niosii-project)
* [Compile the Quartus Project](#compile-the-quartus-project)
* [Building from Scripts](#building-from-scripts)
* [Installing from JTAG](#installing-from-jtag)
* [Installing from ESPER](#installing-from-esper)

# Overview

The FEAM is a digitizer located on the ALPHAg detector. Each FEAM consists of 4 AFTER SCAs for a total of 288 channels.

# Initial Setup

## Quartus Setup

### Create the .cdf
1. Open Quartus Prime Standard Edition
2. Click on **Tools -> Programmer**. The **Programmer** window will appear
3. In the **Programmer** window, click **Auto Detect**. A **Select Device** dialog box will appear
1. (Optional) If **Auto Detect** is greyed out, click on **Hardware Setup**. The **Hardware Setup** dialog box will appear
2. Under **Hardware Settings**, double-click on the USB JTAG hardware you wish to use
3. Click **Close**. The **Hardware Setup** dialog box will close
5. Click on **File -> Save As**. The **Save As** dialog box will appear
6. In the **Save As** dialog box, browse to the **/\<project_dir\>/bin/** directory
7. Make sure the **Add file to current project** checkbox is checked
7. In the **File name** input box, type "alphag_feam.cdf" and click **Save**
8. Close the **Programmer** window
9. Done!

## NIOS Setup

### Create the project files
1. Open Quartus Prime Standard Edition
2. Open the NIOS II Software Build Tools by clicking **Tools -> NIOS II Software Build Tools for Eclipse**
3. The **Workspace Launcher** window will appear, click *OK* to accept the default workspace.
4. Click **File -> Import...**. The **Import** dialog box will appear
5. Click on **General -> Existing Projects into Workspace**
6. Click **Next \>**
7. Next to **Select root directory** click **Browse...**. A **Browse for Folder** dialog box will appear
8. Browse to **/\<project_dir\>/hdl/software** and click **OK**
8. Four Projects should appear in the **Projects** panel:
* alphag_feam
* alphag\_feam\_bsp
* alphag\_feam\_bootloader
* alphag\_feam\_bootloader\_bsp
9. Click **Select All**
10. Click **Finish**
11. Done!

# Build Instructions

## Building from Quartus GUI

**Warning: Please use Quartus Prime Standard Edition 16.1**
### Generate the QSys File
1. Open Quartus Standard Edition 16.1
1. Open QSys by clicking **Tools -> QSys**
2. Click on **File -> Open** (CTRL+O). An **Open File** dialog box should appear.
3. Select the **alphag_feam.qsys** file from the **/\<project_dir\>/hdl** directory and click **Open**
4. Modify as necessary. NOTE: Any change to the QSys, even cosmetic such as expanding or collapsing components will cause the QSys to desire a re-generate.
5. Click **File -> Save** (CTRL+S)
6. Click **Generate HDL**. A **Generation** dialog box will appear
7. Select the settings you desire to be changed, if any
8. Click **Generate**
9. Wait for the generation to complete.
10. Done!

### Compile the NIOSII Project
1. Open the NIOS II Software Build Tools by clicking **Tools -> NIOS II Software Build Tools for Eclipse**
2. (Optional) If the QSys file has been re-generated performing the following
1. Right-click on **alphag\_feam\_bsp** and select **NIOS II -> Generate BSP**. The BSP will regenerate
2. Right-click on **alphag\_feam\_bootloader\_bsp** and select **NIOS II -> Generate BSP**. The BSP will regenerate
3. Modify files as needed. WARNING: Do not modify any BSP files by hand! All changes will be lost on next generate.
4. Click **Project -> Build All** (CTRL+B). If your workspace as multiple projects, it is recommend to close all but the ones related to the FEAM.
5. Right-click on **alphag\_feam\_bootloader** and select **Make Targets -> Build...**. The **Make Targets** dialog box will appear
6. In the **Make Targets** dialog box, select **mem\_init\_generate** and click **Build**
7. Done!

### Compile the Quartus Project
1. Open Quartus Standard Edition 16.1
1. Under **Processing** click **Start Compilation** (CTRL+L)
2. Wait about 16-30 minutes.
3. Done!

**Note:** If Quartus crashed while compiling, delete the **\<project_dir\>/hdl/db** and **\<project_dir\>/hdl/incremental_db** directories and try again

## Installing from JTAG
1. Generate the .jic file using the provided /bin/alphag\_feam\_jic.cof
1. In the main Quartus window, click **File -> Convert Programming Files**, A **Convert Programming Files** window will appear
2. In the **Convert Programming Files** window, click **Open Conversion Setup Data**. An **Open** dialog box will appear
3. In the **Open** dialog box, select the **alphag_feam.cof** file located in **/bin/**
4. Click **Open**. The **Open dialog box will close
* Note the **File name** field, it should display /\<project\_dir\>/bin/alphag\_feam.jic
5. In the **Convert Programming Files** window, click **Generate**
6. When the .jic is done being generated, a small dialog box will appear, click **OK**
7. In the **Convert Programming Files** window, click **Close**
2. Load the .JIC file using the Programmer
1. In the main Quartus window in the **Project Navigator** drop-down input select **Files**.
2. Scroll down the window and find the **alphag_feam.cdf**, and double-click it. The **Programmer** window will open
3. Click on the icon labelled **5CGXFC4C6**
4. Click on the **Change File** button. The **Select New Programming File** dialog box will appear
5. Browse to **/\<project_dir\>/bin/** and select **alphag\_feam.jic**
6. Click Open. The **Select New Programming File** dialog box will close
7. In the main panel, check the **Program/Configure** box on the line that starts **./bin/alphag\_feam.jic**
8. Click the **Start** button
9. Wait for the **Progress** bar to go to 100% and stop
10. Done!
</pre>

= Internal test pulser =

Documentation from Bryerton:

<pre>
---------- Forwarded message ---------
From: Bryerton Shaw <bryerton@triumf.ca>
Date: Tue, 24 Apr 2018 at 16:18
Subject: RE: PWB/FEAM internal test pulse
To: Lars Martin <lmartin@triumf.ca>
Cc: Andrea Capra <acapra@triumf.ca>

Hi Lars (and Andrea),

In the new(er) firmware the test pulse code has been modified slightly to
fit in better with the upgraded trigger system. I can add an option to
allow it to be triggered by the external trigger (currently it has an
interval option, or you can hit the manual trigger to fire it, if it’s
enabled)

The *options for the test pulser* are under the *signalproc module*:

*test_pulse_ena*

An array of four booleans, that control turning on the test pulse for each
SCA. If this is False, the other options below will do nothings for that SCA

*test_pulse_interval_ena*

An array of four booleans, that control turning on the interval trigger for
each SCA

*test_pulse_interval*

An unsigned 32-bit value that marks how many 16ns clock cycles occur
between interval triggers. Only one value, as there is only one interval
timer.

*test_pulse_wdt*

An array of four 16-bit unsigned integers that control the width of the
pulse for each SCA when the trigger occurs.

Now, there is a second set of options to control the delay of the *test
pulse interval trigger*. These options are located under the *trigger
module*

*intp_trig_ena*

Boolean value enabling/disabling the interval test pulse trigger.

*intp_trig_delay*

Unsigned 32-bit integer controlling the trigger delay, which is the time
between when the trigger is requested, and when the trigger actually fires.

You can setup the interval trigger and use that to create event packets
with the test pulse on set intervals. If the interval trigger is
insufficient, I can create another option, which is to fire the test pulse
for enabled SCAs when the external trigger goes. If so, I’ll add this
option in the *signalproc *module for you:

*test_pulse_on_ext*

An array of four Booleans, that control if the test pulse for that SCA
fires on external trigger.

Remember, in order to use the SCA test pulse at all, you must also set the
SCA to test mode. You can do this by going into the *sca0-3 modules* and
setting *test* to *functionality *(the value is *3* if you want to set it
via esper-tool)

Btw currently the SCAs are hard-coded to have all 72 channels receive the
test pulse on firing, and the option to de-select/select which channels to
fire is not offered. Would you like this ability? And if so do you want it
to set all four SCAs the same, or have an option for all 288 channels?

The waveform viewer on the webpage was removed for technical reasons, and
so all viewing is done via the MIDAS DAQ. There are some preliminary tools
I’ve created to verify the event packets and their waveforms, but I
wouldn’t call them easy to setup or use at the moment. (Or very useful for
fine comparisons). If KO doesn’t have a way of saving the events to disk, I
could deliver something that captured and saved waveforms quite quickly.

Regards,
Bryerton

*From:* Lars Martin
*Sent:* April 23, 2018 5:10 PM
*To:* Bryerton Shaw <bryerton@triumf.ca>
*Cc:* Andrea Capra <acapra@triumf.ca>
*Subject:* PWB/FEAM internal test pulse

Hi Bryerton,

Andrea and I would like to check the response function of the PWBs. For
that Daryl suggested to first look at the internal test pulse.

Can you point us to what settings are required to make that go? And to use
the external trigger if we want to acquire the data?

Ideally we would acquire Midas data from that, but it's unclear if KO's
frontend allows for that right now. Can we access the data with Esper
instead?

Finally, we haven't had the waveform display on the web page for a while,
but Daryl said you guys use it. Are we using a different page somehow?

Hope all is well with the baby, cheers,

Lars
</pre>

= TODO =

* IMPOSSIBLE (ext clock only connected to clock cleaner) add frequency counter for the external clock (62.5 MHz)
* DONE add random delay before memtest
* DONE add 4 more bits to udp_delay
* DONE fix crash on boot if run is active and triggers arrive (check that trigger is off on boot) (do not set trigger/enable_all, ext_trig_ena, inp_trig_ena, signal_proc/force_run to 1 after boot)
* DONE (kludge jtag uart write()) increase size of jtag uart output buffer, make it unlimited if possible.
* DONE (kludge jtag uart write()) on dhcp timeout, reboot the fpga - reboot of nios does not clear the jtag uart, which becomes full and nios stops.
* DONE add NIOS watchdog timeout for fectrl
* DONE do not use external clock until instructed by fectrl (clock cleaner default is pin select mode, default pin mode is input with pull down, so clock 0 is selected, this is the ext clock. but we can drive clock select pins from the fpga).
* DONE change clock cleaner power up: use the 2 lines status_clkin0 and status_clkin1 to tell clock cleaner to use the internal clock (clkin2) - from the FPGA drive first line low, second line high. Also change all NIOS code that programs the clock cleaner to keep these two lines in the "input" mode. Current code switches them to "output" mode.
* DONE clock cleaner power up state is wrong for both choices of clock: 62.5 MHz ext clock and 125 MHz internal clock, see explanation in feam_top.sv. It looks like I need to drive status_clkin0 and status_clkin1 to "holdover" mode, both logic level 1. Currently status_clkin0 is used as SPI read line, this has to be moved to a different line first.
* add watchdog timeout that works in factory mode ("remote update" watchdog only works in user mode)
* enable HTTP pipelining - copy changes on mongoose from ADC project.
* reset everything if it looks like NIOS TCP/IP code has crashed, but enough code is still running to prevent tripping of the fpga-remote-update watchdog timer. such crash tends to happens when the sata link mate is rebooted and incomplete data arrives from the disrupted ethernet data path.
* when loading AFTER ASIC do not go into infinite loop if read-back is broken (always mismatch).
* read and report error status information bits for 2983fc.pdf

= ZZZ =

ZZZ

PWB

2021-08-03T05:06:42Z

Acapra: /* Hardware */

= Links =

* https://bitbucket.org/expalpha/pwb_rev1_firmware
* https://edev-group.triumf.ca/fw/exp/alphag/feam/rev1
* https://edev-group.triumf.ca/hw/alphag/feam/rev1
* https://ladd00.triumf.ca/daqinv/frontend/list/170 - PWB inventory database

= Schematics =

* [[Image:pwb_rev0.pdf]]
* [[Image:Pwb-rev1-after.pdf]]
* [[Image:AlphaG Rev1.pdf]]

= Manuals =

* [[Image:AFTER_MANUAL_V1.pdf]] AFTER SCA manual v1
* [[Image:AFTER_DataSheet1_1.pdf]] AFTER SCA manual v1.1
* [[Image:AFTER_SCA_table2.pdf]] AFTER SCA channel map

= Hardware =

== Power distribution ==

External power supply is:
* +5V
* +2V

Central board:

* Ext_5V - F1 - RS1 (0.1R) - Fuse_5V
* Ext_2V - F2/F3 - RS2 (0.1R) - Idig2-
* LDO3 - Fuse_5V - DIG_3.3V - selfenable - ADM7154
* LDO5 - Fuse_5V - DIG_2.5V - selfenable - ADM7154
* LDO_CLN - Fuse5V - CLN_3.3V - selfenable -
* VREG - DIG_3.3V - -2.5V SCA BIAS - selfenable - LTC1983ES6-3#TRMPBF
* LDO7 - DIG_2.5V - DDR3_VRef - enable DDR3_1P5V, DDR3_PG, DDR3_VttEN
* LDO2 - Idig2V- - DIG_1.1V - enable DIG_3.3V - LT3071
* LDO6 - Idig2V- - DDR3_1P5V - enable DIG_3.3V - LT3083

SCA Wing:

* Ext_5V - F1 - RS1 (0.3R) - Isca- - SCA_Fuse5V
* LDO1 - SCA_Fuse5V - SCA3.3V - enable ADC_PWR_EN - ADM7154
* LDO2 - SCA3.3V - ADC1.8V - selfenable - ADM7154
* LDO3 - SCA3.3V - TRN2.5V - selfenable - LT1761
* REF - SCA3.3V - 1.45V Vicm - selfenable - LM4121

== Trigger and Clock distribution ==

Clock (62.5MHz) and trigger are distributed to the PWB columns via the [https://edev-group.triumf.ca/hw/alphag/trigger-card-sata-adapter/rev1 trigger SATA adapter].

The two miniSAS cables from cdm03 port 5 and 6 are connected to a miniSAS-SATA splitter 1-to-4 (CS Electronics "iSAS SATA adapter" ADP-887P-1X). Since there are only 8 columns and the clock and trigger signals are daisy-chained across the 8 rows, this scheme is sufficient to provide the necessary inputs to all 64 PWBs.

[https://www.techcable.com/sas-mini-sas-cables/sata-cables/ SATA 6Gb/s Cable with Latching Clip. Straight Pinout 1-1 247726-L 40in]

== Optical Link ==

Main data uplink.

[[File:AVAGO SFP specs.pdf|SFP (Small Form Pluggable) Avago Technologies AFBR-57M5APZ]]

[[File:FiberOptic TrippLite specs.pdf|Fiber optic Tripp Lite N820 - 10 m]]

== SATA Link ==

Secondary data uplink, capable also to deliver clock and trigger.
The SATA link connects PWB from adjacent columuns on the same row.
The SATA cable is a "crossed cable".

= Firmware =

== Firmware update ==

PWB firmware update is done using "esper-tool".

<pre>
esper-tool -v upload -f file.rpd http://pwbNN update factory_rpd
esper-tool -v upload -f file.rpd http://pwbNN update file_rpd
</pre>

As of PWB firmware pwb_rev1_20200706_ko:

Permission to write into the flash memory is controlled by esper variables:

* allow_write set to "false": "esper-tool upload" does a "verify": successful upload means epcq flash content is same as rpd file, error means content is not the same.
* allow_write set to "true": "esper-tool upload" updates the epcq flash from the rpd file: for each data block, read the epcq flash, compare to rpd data, if different, erase the flash block, write the flash, read the flash, if there is a mismatch (bad write), return an error.
* allow_factory_write set to "true" to enable writing to the "factory_rpd". DO NOT SET IT TO "true" UNLESS YOU MEAN TO UPDATE THE PWB BOOTLOADER. IF YOU FLASH BUM FIRMWARE or IF THERE IS A POWER OUTAGE WHILE UPDATING, THE PWB WILL BE BRICKED and has to be recovered by connecting a USB blaster.

To update firmware of multiple boards to correct version, recommended is the script "update_pwb.perl":
<pre>
cd ~/online/src
./update_pwb.perl pwb12 pwb06 pwb78
</pre>

== Factory page and user page firmware boot ==

Each PWB board has 2 firmware images: boot loader (factory page)
and data acquisition (user page). On power up, the FPGA loads and runs
the boot loader firmware from the factory page, later the control
software reboots the FPGA into the data acquisition firmware from
the user page.

Because loading defective firmware can brick https://en.wikipedia.org/wiki/Brick_(electronics)
the PWB, new firmware images are loaded into the user page, if anything is wrong,
the PWB can still boot from the factory page and one can use the firmware update
function to load a good firmware image into the user page.

The boot loader firmware in the factory page is not intended for data acquisition, it is only meant to provide three functions:
* boot enough hardware and software to communicate via the ethernet and the sata link
* firmware update (both factory page and user page)
* reboot into the user page

The boot loader firmware in the factory page is usually never updated unless absolutely
needed to fix a problem with these three functions (i.e. compatibility with sata link communications).

If factory image is corrupted or the contains defective firmware and the PWB does not boot (no dhcp, no ping, no esper),
the only way to unbrick it is by loading good firmware using a jtag flash programmer (usb-blaster). Connecting it requires
physical access to the jtag connector on the PWB board. It is impossible when the detector is fully assembled in the experiment.

The PWB has a 32 Mbyte EPCQ flash memory chip, it is divided into 2 pages 16 Mbytes of factory page and 16 Mbytes of user page.

A complete firmware image generally contains 3 pieces:
* FPGA firmware and NIOS "feam_bootloader" software RAM image (sof file)
* NIOS "feam" software RAM image (feam.elf.flash.hex) - all the C code for esper communications and PWB data acquisition
* NIOS filesystem image (feam.webpkg.hex) - esper web pages

This is the boot sequence:
* on power-up the FPGA automatically loads the sof file from address 0 of the EPCQ flash (this is the factory image sof file)
* the FPGA is "started"
* the NIOS CPU starts executing the "feam_bootloader" C code embedded in the sof file (NIOS project hdl/software/feam_bootloader)
* feam_bootloader initializes the hardware (clocks, DDR memory, etc)
* feam_bootloader write-protects the EPCQ flash memory
* feam_bootloader copies the "feam" software from EPCQ flash to the DDR memory (after checking for correct checkum)
* feam_bootloader restarts the NIOS CPU
* the NIOS CPU starts running from DDR memory, executes the "feam" software (NIOS project hdl/software/feam)
* call main() in hdl/software/feam/src/task_init.c
* call task_init() in the same file
* infinite loop waiting for IP address via DHCP
* after have IP address, call task_esper() from hdl/software/feam/src/task_esper.c
* task_esper() creates all the esper modules, esper variables, etc,
* mod_http.c starts the mongoose web server
* PWB is open for business

This is the protection against booting corrupted firmware:
* FPGA hardware loads the sof file from epcq flash and checks for correct checksum before "starting" it
* NIOS CPU is part of the sof file, protected by sof file checksum
* NIOS feam_bootloader C code is part of the sof file, protected by the sof checksum
* feam_bootloader checks for correct signatures and checksums of the DDR memory image
* "feam" C code is protected by the checksum of the DDR memory image

== NIOS terminal ==

<pre>
$ ssh agmini@daq16
$ /opt/intelFPGA/16.1/quartus/bin/jtagconfig
1) USB-Blaster [2-1.2]
02B030DD 5CGTFD7(B5|C5|D5)/5CGXBC7B6/..
$ /opt/intelFPGA/17.0/quartus/bin/nios2-terminal
nios2-terminal: connected to hardware target using JTAG UART on cable
nios2-terminal: "USB-Blaster [2-1.2]", device 1, instance 0
nios2-terminal: (Use the IDE stop button or Ctrl-C to terminate)
PWB Revision 1 Boot Loader
Ver 2.0 Build 357 - Wed Jun 6 15:05:35 PDT 2018
...
</pre>

== Flash boot loader firmware via jtag ==

<pre>
$ ssh agmini@daq16
$ /opt/intelFPGA/16.1/quartus/bin/jtagconfig
1) USB-Blaster [2-1.2]
02B030DD 5CGTFD7(B5|C5|D5)/5CGXBC7B6/..
$ cd ~/online/firmware/pwb_rev1
$ ls -l
$ /opt/intelFPGA/17.1/quartus/bin/quartus_pgmw
... auto detect
... load the jic file
... in menu tools->programmer, enable "unprotect device"
... start program/configure operation
</pre>

== Flash user page firmware via esper-tool ==

<pre>
$ ssh agmini@daq16
$ cd online/src
$ more update_pwb.perl ### check that $fw is set to the desired firmware file
$ ./update_pwb.perl pwb06 ### or give more PWB names or give "all"
</pre>

== Build firmware ==

=== quartus version to use ===

* quartus 16.1 should be used for jtag (jtagconfig and jtagd)
* quartus 17.0 should be used to build the PWB firmware (17.1 is not compatible)

=== start license server ===

<pre>
ssh agmini@daq16
/opt/intelFPGA/17.0/quartus/linux64/lmgrd -c ~agmini/online/license-daq16.dat
</pre>

=== prepare the build environment ===

<pre>
$ ssh agmini@daq16
$ /opt/intelFPGA/17.0/nios2eds/nios2_command_shell.sh
</pre>

=== get latest version of the code and build it ===

<pre>
$ cd online/firmware/git/pwb_rev1_firmware
$ git pull
$ git checkout alphag
$ git pull
$ ./scripts/compile_project.sh
... about 30-40 minutes ...
$ ls -l bin/*.sof bin/*.jic bin/*.rpd
-rw-r--r-- 1 agmini alpha 12727389 Jan 24 2018 bin/feam_rev1_auto.rpd
-rw-r--r-- 1 agmini alpha 33554661 Jan 24 2018 bin/feam_rev1.jic
-rw-r--r-- 1 agmini alpha 6974754 Jan 24 2018 bin/rev1.sof
$ ### feam.jic is loaded via jtag
$ ### feam_auto.rpd is loaded via esper
$ ### feam.sof is used to attach the quartus signal tap logic analyzer
</pre>

=== firmware build order and sequence ===

* erase previous build
* regenerate qsys
* regenerate NIOS BSP "feam_bootloader_bsp" and "feam_bsp"
* build NIOS projects "feam_bootloader" and "feam"
* build quartus sof file (with the "feam_bootloader" nios project inside)
* build jic and rpd files (with the quartus sof file and the "feam" nios project inside)
* load sof file into the FPGA using jtag (be careful about compatible "feam" nios project already in the epcq flash)
* load jic file into the epcq factory page using jtag (cycle power for them to take effect)
* load rpd file into the epcq factory or user page using esper

=== scripts for building the firmware ===

* scripts/compile_project.sh - build everything from scratch:
* scripts/compile_qsys.sh - regenerate the main qsys component
* scripts/compile_nios.sh - regenerate the BSPs, then same as compile_elf.sh
* scripts/compile_elf.sh - build the BSPs, "feam_bootloader" and "feam" NIOS projects
* scripts/update_elf.sh - regenerate the rpd and jic files to include the new "feam" NIOS build (keeps the old "feam_bootloader" in the sof file)
* scripts/update_bootloader.sh - also regenerate the sof file to include the new "feam_bootloader" build
* scripts/clean_quartus.sh - prepare quartus fpga sof file to build from scratch
* scripts/compile_quartus.sh - build the quartus fpga sof file incrementally (run clean_quartus.sh to build from scratch).
* scripts/load_jtag_sof.pl bin/rev1.sof - load sof file into the FPGA. this will boot the new fpga firmware, the new feam_bootloader (if update_bootloader.sh was run), but with the old "feam" already in epcq flash. (beware of incompatible sof, "feam_bootloader" and "feam" builds!)
* scripts/load_jtag_jic.pl bin/feam_rev1.jic - load jic file into the EPCQ flash, same as using the graphical flash programmer tool. (this will leave the FPGA running the quartus jtag flash loader firmware, not the PWB firmware. Cycle power for new jic file to take effect).

== ESPER Variables ==

* Board
** invert_ext_trig - invert trigger signal from CDM before it drives any logic (to undo incorrect signal polarity)
** reset_nios - goggle up, them down to reset NIOS subsystem

* Signalproc
** test_mode - ADC data is replaced with a test pattern, see test mode bits in sca_x_ch_ctrl.
** sca_a_ch_ctrl, sca_b_ch_ctrl, sca_c_ch_ctrl, sca_d_ch_ctrl - channel control bits:
<pre>
11..0 - threshold - channel suppression threshold
14..12 - ctrl_test_mode - test mode:
0=fixed patter 0xa5a,
1=time bin counter,
2=time bin counter with channel number,
3=sequential adc sample counter,
4={ch_crossed_out,trig_pos,trig_neg,adc[8:0]},
5={trig,adc[10:0]},
6={ch_crossed_min,adc[10:0]}
15 - ctrl_supp_mode - channel suppression mode: 0=adc<=(baseline-threshold), 1=adc<=threshold
</pre>

* Link
** link_ctrl - sata link control. The bits are:
<pre>
0 - sata_link_udp_stream_in_enable - permit data flow from sata link to OFFLOAD_SATA
1 - udp_stream_out_enable - permit sca data flow to sata link
2 - sata_to_eth_enable - permit ethernet data flow from sata link to TSE_MAC
3 - eth_to_sata_enable - permit ethernet data flow from TSE MAC to sata link
4 - enable_stop_our_tx - enable flow control: allow stop_tx
#5 - stop_our_tx - manually activate the flow control signal into link_tx
6 - enable_stop_remote_tx - enable flow control: allow send "stop_tx"
#7 - stop_remote_tx - manually activate the flow control signal into link_tx
8 - tx_test_pattern_udp - udp data is replaced by test pattern 0x11111111, 0x22222222, etc.
9 - tx_test_pattern_eth - nios-to-sata data is replaced by test pattern 0x11111111, 0x22222222, etc.
10 - udp_delay_enable - delay between udp packets, see udp_delay_value below
11 - reboot_tx - send K_REBOOT command to the sata link mate, where it has same effect as board/reset_nios.
12 - sata_to_nios_disable
13 - nios_to_sata_disable
14 -
15 -
16 -
24..31 - udp_delay_value - delay between udp packets (top 8 bits of a 20-bit counter)
</pre>

== Firmware data path ==

Main data path:
<pre>
feam_top
|
sca_sigproc
|
sca_event_control
4 * sca_control (sca_channel.sv)
4 * channel_fifo (1024 samples)
</pre>
<pre>
sca_control (sca_channel.sv)
|
state machine to control SCA read and write enables
|
sca_write_control (what does it do?!?)
sca_read_control
</pre>
<pre>
sca_read_control
|
state machine to store ADC samples into per-channel FIFOs (channel_fifo)
selector to replace ADC samples with a test pattern
79*sca_trig_one -> ch_crossed_out - channel hit detector
</pre>
<pre>
sca_event_control
|
event_fifo (hdl/mf/event_descriptor_fifo)
state machine to take ADC data from 4 per-channel FIFOs (channel_fifo) and store it into DDR memory (write addr increment is 510*8 (number of samples)*8 = 4080).
state machine to read transposed ADC data from DDR memory (read addr increment is 8+8=16 - 2 samples * 2 bytes * 4 sca = 16) and create 4 per-sca data streams
</pre>
<pre>
data stream:
4 per-sca data streams from sca_event_control -> sca_sigproc(event_dat)
|
packet_chunker (hdl/lib/packet_chunker.sv) (4*event_dat -> event_segment_dat) - multiplex 4 data streams into 1 stream of UDP-sized packets
|
packet_length_prepender (hdl/lib/packet_length_prepender.sv) (event_segment_dat -> udp_event_val_out)
|
udp_event_val -> info qsys (udp_stream_sca) and into sata link (mux channel 2).
</pre>

== Firmware build instructions from Bryerton (obsolete) ==

This is the old README.md file from the firmware git repository.

These instructions are not used to build the firmware (use scrips/compile_project.sh).

They are still useful for figuring out which buttons to push in the altera gui programs.

<pre>
# ALPHAg FrontEnd Acquisition Module (FEAM) - REV0

Welcome to the ALPHAg FEAM Project

# Table of Contents

* [Overview](#overview)
* [Inital Setup](#initial-setup)
* [Quartus Setup](#quartus-setup)
* [Create the .cdf](#create-the-cdf)
* [NIOS Setup](#nios-Setup)
* [Create the project files](#create-the-project-files)
* [Build Instructions](#build-instructions)
* [Building from Quartus GUI](#building-from-quartus-gui)
* [Generate the QSys File](#generate-the-qsys-file)
* [Compile the NIOSII Project](#compile-the-niosii-project)
* [Compile the Quartus Project](#compile-the-quartus-project)
* [Building from Scripts](#building-from-scripts)
* [Installing from JTAG](#installing-from-jtag)
* [Installing from ESPER](#installing-from-esper)

# Overview

The FEAM is a digitizer located on the ALPHAg detector. Each FEAM consists of 4 AFTER SCAs for a total of 288 channels.

# Initial Setup

## Quartus Setup

### Create the .cdf
1. Open Quartus Prime Standard Edition
2. Click on **Tools -> Programmer**. The **Programmer** window will appear
3. In the **Programmer** window, click **Auto Detect**. A **Select Device** dialog box will appear
1. (Optional) If **Auto Detect** is greyed out, click on **Hardware Setup**. The **Hardware Setup** dialog box will appear
2. Under **Hardware Settings**, double-click on the USB JTAG hardware you wish to use
3. Click **Close**. The **Hardware Setup** dialog box will close
5. Click on **File -> Save As**. The **Save As** dialog box will appear
6. In the **Save As** dialog box, browse to the **/\<project_dir\>/bin/** directory
7. Make sure the **Add file to current project** checkbox is checked
7. In the **File name** input box, type "alphag_feam.cdf" and click **Save**
8. Close the **Programmer** window
9. Done!

## NIOS Setup

### Create the project files
1. Open Quartus Prime Standard Edition
2. Open the NIOS II Software Build Tools by clicking **Tools -> NIOS II Software Build Tools for Eclipse**
3. The **Workspace Launcher** window will appear, click *OK* to accept the default workspace.
4. Click **File -> Import...**. The **Import** dialog box will appear
5. Click on **General -> Existing Projects into Workspace**
6. Click **Next \>**
7. Next to **Select root directory** click **Browse...**. A **Browse for Folder** dialog box will appear
8. Browse to **/\<project_dir\>/hdl/software** and click **OK**
8. Four Projects should appear in the **Projects** panel:
* alphag_feam
* alphag\_feam\_bsp
* alphag\_feam\_bootloader
* alphag\_feam\_bootloader\_bsp
9. Click **Select All**
10. Click **Finish**
11. Done!

# Build Instructions

## Building from Quartus GUI

**Warning: Please use Quartus Prime Standard Edition 16.1**
### Generate the QSys File
1. Open Quartus Standard Edition 16.1
1. Open QSys by clicking **Tools -> QSys**
2. Click on **File -> Open** (CTRL+O). An **Open File** dialog box should appear.
3. Select the **alphag_feam.qsys** file from the **/\<project_dir\>/hdl** directory and click **Open**
4. Modify as necessary. NOTE: Any change to the QSys, even cosmetic such as expanding or collapsing components will cause the QSys to desire a re-generate.
5. Click **File -> Save** (CTRL+S)
6. Click **Generate HDL**. A **Generation** dialog box will appear
7. Select the settings you desire to be changed, if any
8. Click **Generate**
9. Wait for the generation to complete.
10. Done!

### Compile the NIOSII Project
1. Open the NIOS II Software Build Tools by clicking **Tools -> NIOS II Software Build Tools for Eclipse**
2. (Optional) If the QSys file has been re-generated performing the following
1. Right-click on **alphag\_feam\_bsp** and select **NIOS II -> Generate BSP**. The BSP will regenerate
2. Right-click on **alphag\_feam\_bootloader\_bsp** and select **NIOS II -> Generate BSP**. The BSP will regenerate
3. Modify files as needed. WARNING: Do not modify any BSP files by hand! All changes will be lost on next generate.
4. Click **Project -> Build All** (CTRL+B). If your workspace as multiple projects, it is recommend to close all but the ones related to the FEAM.
5. Right-click on **alphag\_feam\_bootloader** and select **Make Targets -> Build...**. The **Make Targets** dialog box will appear
6. In the **Make Targets** dialog box, select **mem\_init\_generate** and click **Build**
7. Done!

### Compile the Quartus Project
1. Open Quartus Standard Edition 16.1
1. Under **Processing** click **Start Compilation** (CTRL+L)
2. Wait about 16-30 minutes.
3. Done!

**Note:** If Quartus crashed while compiling, delete the **\<project_dir\>/hdl/db** and **\<project_dir\>/hdl/incremental_db** directories and try again

## Installing from JTAG
1. Generate the .jic file using the provided /bin/alphag\_feam\_jic.cof
1. In the main Quartus window, click **File -> Convert Programming Files**, A **Convert Programming Files** window will appear
2. In the **Convert Programming Files** window, click **Open Conversion Setup Data**. An **Open** dialog box will appear
3. In the **Open** dialog box, select the **alphag_feam.cof** file located in **/bin/**
4. Click **Open**. The **Open dialog box will close
* Note the **File name** field, it should display /\<project\_dir\>/bin/alphag\_feam.jic
5. In the **Convert Programming Files** window, click **Generate**
6. When the .jic is done being generated, a small dialog box will appear, click **OK**
7. In the **Convert Programming Files** window, click **Close**
2. Load the .JIC file using the Programmer
1. In the main Quartus window in the **Project Navigator** drop-down input select **Files**.
2. Scroll down the window and find the **alphag_feam.cdf**, and double-click it. The **Programmer** window will open
3. Click on the icon labelled **5CGXFC4C6**
4. Click on the **Change File** button. The **Select New Programming File** dialog box will appear
5. Browse to **/\<project_dir\>/bin/** and select **alphag\_feam.jic**
6. Click Open. The **Select New Programming File** dialog box will close
7. In the main panel, check the **Program/Configure** box on the line that starts **./bin/alphag\_feam.jic**
8. Click the **Start** button
9. Wait for the **Progress** bar to go to 100% and stop
10. Done!
</pre>

= Internal test pulser =

Documentation from Bryerton:

<pre>
---------- Forwarded message ---------
From: Bryerton Shaw <bryerton@triumf.ca>
Date: Tue, 24 Apr 2018 at 16:18
Subject: RE: PWB/FEAM internal test pulse
To: Lars Martin <lmartin@triumf.ca>
Cc: Andrea Capra <acapra@triumf.ca>

Hi Lars (and Andrea),

In the new(er) firmware the test pulse code has been modified slightly to
fit in better with the upgraded trigger system. I can add an option to
allow it to be triggered by the external trigger (currently it has an
interval option, or you can hit the manual trigger to fire it, if it’s
enabled)

The *options for the test pulser* are under the *signalproc module*:

*test_pulse_ena*

An array of four booleans, that control turning on the test pulse for each
SCA. If this is False, the other options below will do nothings for that SCA

*test_pulse_interval_ena*

An array of four booleans, that control turning on the interval trigger for
each SCA

*test_pulse_interval*

An unsigned 32-bit value that marks how many 16ns clock cycles occur
between interval triggers. Only one value, as there is only one interval
timer.

*test_pulse_wdt*

An array of four 16-bit unsigned integers that control the width of the
pulse for each SCA when the trigger occurs.

Now, there is a second set of options to control the delay of the *test
pulse interval trigger*. These options are located under the *trigger
module*

*intp_trig_ena*

Boolean value enabling/disabling the interval test pulse trigger.

*intp_trig_delay*

Unsigned 32-bit integer controlling the trigger delay, which is the time
between when the trigger is requested, and when the trigger actually fires.

You can setup the interval trigger and use that to create event packets
with the test pulse on set intervals. If the interval trigger is
insufficient, I can create another option, which is to fire the test pulse
for enabled SCAs when the external trigger goes. If so, I’ll add this
option in the *signalproc *module for you:

*test_pulse_on_ext*

An array of four Booleans, that control if the test pulse for that SCA
fires on external trigger.

Remember, in order to use the SCA test pulse at all, you must also set the
SCA to test mode. You can do this by going into the *sca0-3 modules* and
setting *test* to *functionality *(the value is *3* if you want to set it
via esper-tool)

Btw currently the SCAs are hard-coded to have all 72 channels receive the
test pulse on firing, and the option to de-select/select which channels to
fire is not offered. Would you like this ability? And if so do you want it
to set all four SCAs the same, or have an option for all 288 channels?

The waveform viewer on the webpage was removed for technical reasons, and
so all viewing is done via the MIDAS DAQ. There are some preliminary tools
I’ve created to verify the event packets and their waveforms, but I
wouldn’t call them easy to setup or use at the moment. (Or very useful for
fine comparisons). If KO doesn’t have a way of saving the events to disk, I
could deliver something that captured and saved waveforms quite quickly.

Regards,
Bryerton

*From:* Lars Martin
*Sent:* April 23, 2018 5:10 PM
*To:* Bryerton Shaw <bryerton@triumf.ca>
*Cc:* Andrea Capra <acapra@triumf.ca>
*Subject:* PWB/FEAM internal test pulse

Hi Bryerton,

Andrea and I would like to check the response function of the PWBs. For
that Daryl suggested to first look at the internal test pulse.

Can you point us to what settings are required to make that go? And to use
the external trigger if we want to acquire the data?

Ideally we would acquire Midas data from that, but it's unclear if KO's
frontend allows for that right now. Can we access the data with Esper
instead?

Finally, we haven't had the waveform display on the web page for a while,
but Daryl said you guys use it. Are we using a different page somehow?

Hope all is well with the baby, cheers,

Lars
</pre>

= TODO =

* IMPOSSIBLE (ext clock only connected to clock cleaner) add frequency counter for the external clock (62.5 MHz)
* DONE add random delay before memtest
* DONE add 4 more bits to udp_delay
* DONE fix crash on boot if run is active and triggers arrive (check that trigger is off on boot) (do not set trigger/enable_all, ext_trig_ena, inp_trig_ena, signal_proc/force_run to 1 after boot)
* DONE (kludge jtag uart write()) increase size of jtag uart output buffer, make it unlimited if possible.
* DONE (kludge jtag uart write()) on dhcp timeout, reboot the fpga - reboot of nios does not clear the jtag uart, which becomes full and nios stops.
* DONE add NIOS watchdog timeout for fectrl
* DONE do not use external clock until instructed by fectrl (clock cleaner default is pin select mode, default pin mode is input with pull down, so clock 0 is selected, this is the ext clock. but we can drive clock select pins from the fpga).
* DONE change clock cleaner power up: use the 2 lines status_clkin0 and status_clkin1 to tell clock cleaner to use the internal clock (clkin2) - from the FPGA drive first line low, second line high. Also change all NIOS code that programs the clock cleaner to keep these two lines in the "input" mode. Current code switches them to "output" mode.
* DONE clock cleaner power up state is wrong for both choices of clock: 62.5 MHz ext clock and 125 MHz internal clock, see explanation in feam_top.sv. It looks like I need to drive status_clkin0 and status_clkin1 to "holdover" mode, both logic level 1. Currently status_clkin0 is used as SPI read line, this has to be moved to a different line first.
* add watchdog timeout that works in factory mode ("remote update" watchdog only works in user mode)
* enable HTTP pipelining - copy changes on mongoose from ADC project.
* reset everything if it looks like NIOS TCP/IP code has crashed, but enough code is still running to prevent tripping of the fpga-remote-update watchdog timer. such crash tends to happens when the sata link mate is rebooted and incomplete data arrives from the disrupted ethernet data path.
* when loading AFTER ASIC do not go into infinite loop if read-back is broken (always mismatch).
* read and report error status information bits for 2983fc.pdf

= ZZZ =

ZZZ

File:FiberOptic TrippLite specs.pdf

2021-08-03T04:58:16Z

Acapra:

File:AVAGO SFP specs.pdf

2021-08-03T04:57:41Z

Acapra:

PWB

2021-08-03T04:56:39Z

Acapra: /* Optical Link */

= Links =

* https://bitbucket.org/expalpha/pwb_rev1_firmware
* https://edev-group.triumf.ca/fw/exp/alphag/feam/rev1
* https://edev-group.triumf.ca/hw/alphag/feam/rev1
* https://ladd00.triumf.ca/daqinv/frontend/list/170 - PWB inventory database

= Schematics =

* [[Image:pwb_rev0.pdf]]
* [[Image:Pwb-rev1-after.pdf]]
* [[Image:AlphaG Rev1.pdf]]

= Manuals =

* [[Image:AFTER_MANUAL_V1.pdf]] AFTER SCA manual v1
* [[Image:AFTER_DataSheet1_1.pdf]] AFTER SCA manual v1.1
* [[Image:AFTER_SCA_table2.pdf]] AFTER SCA channel map

= Hardware =

== Power distribution ==

External power supply is:
* +5V
* +2V

Central board:

* Ext_5V - F1 - RS1 (0.1R) - Fuse_5V
* Ext_2V - F2/F3 - RS2 (0.1R) - Idig2-
* LDO3 - Fuse_5V - DIG_3.3V - selfenable - ADM7154
* LDO5 - Fuse_5V - DIG_2.5V - selfenable - ADM7154
* LDO_CLN - Fuse5V - CLN_3.3V - selfenable -
* VREG - DIG_3.3V - -2.5V SCA BIAS - selfenable - LTC1983ES6-3#TRMPBF
* LDO7 - DIG_2.5V - DDR3_VRef - enable DDR3_1P5V, DDR3_PG, DDR3_VttEN
* LDO2 - Idig2V- - DIG_1.1V - enable DIG_3.3V - LT3071
* LDO6 - Idig2V- - DDR3_1P5V - enable DIG_3.3V - LT3083

SCA Wing:

* Ext_5V - F1 - RS1 (0.3R) - Isca- - SCA_Fuse5V
* LDO1 - SCA_Fuse5V - SCA3.3V - enable ADC_PWR_EN - ADM7154
* LDO2 - SCA3.3V - ADC1.8V - selfenable - ADM7154
* LDO3 - SCA3.3V - TRN2.5V - selfenable - LT1761
* REF - SCA3.3V - 1.45V Vicm - selfenable - LM4121

== Trigger and Clock distribution ==

Clock (62.5MHz) and trigger are distributed to the PWB columns via the [https://edev-group.triumf.ca/hw/alphag/trigger-card-sata-adapter/rev1 trigger SATA adapter].

The two miniSAS cables from cdm03 port 5 and 6 are connected to a miniSAS-SATA splitter 1-to-4 (CS Electronics "iSAS SATA adapter" ADP-887P-1X). Since there are only 8 columns and the clock and trigger signals are daisy-chained across the 8 rows, this scheme is sufficient to provide the necessary inputs to all 64 PWBs.

== Optical Link ==

Main data uplink.

SFP (Small Form Pluggable) Avago Technologies AFBR-57M5APZ

Fiber optic Tripp Lite N820 - 10 m

== SATA Link ==

Secondary data uplink, capable also to deliver clock and trigger.

= Firmware =

== Firmware update ==

PWB firmware update is done using "esper-tool".

<pre>
esper-tool -v upload -f file.rpd http://pwbNN update factory_rpd
esper-tool -v upload -f file.rpd http://pwbNN update file_rpd
</pre>

As of PWB firmware pwb_rev1_20200706_ko:

Permission to write into the flash memory is controlled by esper variables:

* allow_write set to "false": "esper-tool upload" does a "verify": successful upload means epcq flash content is same as rpd file, error means content is not the same.
* allow_write set to "true": "esper-tool upload" updates the epcq flash from the rpd file: for each data block, read the epcq flash, compare to rpd data, if different, erase the flash block, write the flash, read the flash, if there is a mismatch (bad write), return an error.
* allow_factory_write set to "true" to enable writing to the "factory_rpd". DO NOT SET IT TO "true" UNLESS YOU MEAN TO UPDATE THE PWB BOOTLOADER. IF YOU FLASH BUM FIRMWARE or IF THERE IS A POWER OUTAGE WHILE UPDATING, THE PWB WILL BE BRICKED and has to be recovered by connecting a USB blaster.

To update firmware of multiple boards to correct version, recommended is the script "update_pwb.perl":
<pre>
cd ~/online/src
./update_pwb.perl pwb12 pwb06 pwb78
</pre>

== Factory page and user page firmware boot ==

Each PWB board has 2 firmware images: boot loader (factory page)
and data acquisition (user page). On power up, the FPGA loads and runs
the boot loader firmware from the factory page, later the control
software reboots the FPGA into the data acquisition firmware from
the user page.

Because loading defective firmware can brick https://en.wikipedia.org/wiki/Brick_(electronics)
the PWB, new firmware images are loaded into the user page, if anything is wrong,
the PWB can still boot from the factory page and one can use the firmware update
function to load a good firmware image into the user page.

The boot loader firmware in the factory page is not intended for data acquisition, it is only meant to provide three functions:
* boot enough hardware and software to communicate via the ethernet and the sata link
* firmware update (both factory page and user page)
* reboot into the user page

The boot loader firmware in the factory page is usually never updated unless absolutely
needed to fix a problem with these three functions (i.e. compatibility with sata link communications).

If factory image is corrupted or the contains defective firmware and the PWB does not boot (no dhcp, no ping, no esper),
the only way to unbrick it is by loading good firmware using a jtag flash programmer (usb-blaster). Connecting it requires
physical access to the jtag connector on the PWB board. It is impossible when the detector is fully assembled in the experiment.

The PWB has a 32 Mbyte EPCQ flash memory chip, it is divided into 2 pages 16 Mbytes of factory page and 16 Mbytes of user page.

A complete firmware image generally contains 3 pieces:
* FPGA firmware and NIOS "feam_bootloader" software RAM image (sof file)
* NIOS "feam" software RAM image (feam.elf.flash.hex) - all the C code for esper communications and PWB data acquisition
* NIOS filesystem image (feam.webpkg.hex) - esper web pages

This is the boot sequence:
* on power-up the FPGA automatically loads the sof file from address 0 of the EPCQ flash (this is the factory image sof file)
* the FPGA is "started"
* the NIOS CPU starts executing the "feam_bootloader" C code embedded in the sof file (NIOS project hdl/software/feam_bootloader)
* feam_bootloader initializes the hardware (clocks, DDR memory, etc)
* feam_bootloader write-protects the EPCQ flash memory
* feam_bootloader copies the "feam" software from EPCQ flash to the DDR memory (after checking for correct checkum)
* feam_bootloader restarts the NIOS CPU
* the NIOS CPU starts running from DDR memory, executes the "feam" software (NIOS project hdl/software/feam)
* call main() in hdl/software/feam/src/task_init.c
* call task_init() in the same file
* infinite loop waiting for IP address via DHCP
* after have IP address, call task_esper() from hdl/software/feam/src/task_esper.c
* task_esper() creates all the esper modules, esper variables, etc,
* mod_http.c starts the mongoose web server
* PWB is open for business

This is the protection against booting corrupted firmware:
* FPGA hardware loads the sof file from epcq flash and checks for correct checksum before "starting" it
* NIOS CPU is part of the sof file, protected by sof file checksum
* NIOS feam_bootloader C code is part of the sof file, protected by the sof checksum
* feam_bootloader checks for correct signatures and checksums of the DDR memory image
* "feam" C code is protected by the checksum of the DDR memory image

== NIOS terminal ==

<pre>
$ ssh agmini@daq16
$ /opt/intelFPGA/16.1/quartus/bin/jtagconfig
1) USB-Blaster [2-1.2]
02B030DD 5CGTFD7(B5|C5|D5)/5CGXBC7B6/..
$ /opt/intelFPGA/17.0/quartus/bin/nios2-terminal
nios2-terminal: connected to hardware target using JTAG UART on cable
nios2-terminal: "USB-Blaster [2-1.2]", device 1, instance 0
nios2-terminal: (Use the IDE stop button or Ctrl-C to terminate)
PWB Revision 1 Boot Loader
Ver 2.0 Build 357 - Wed Jun 6 15:05:35 PDT 2018
...
</pre>

== Flash boot loader firmware via jtag ==

<pre>
$ ssh agmini@daq16
$ /opt/intelFPGA/16.1/quartus/bin/jtagconfig
1) USB-Blaster [2-1.2]
02B030DD 5CGTFD7(B5|C5|D5)/5CGXBC7B6/..
$ cd ~/online/firmware/pwb_rev1
$ ls -l
$ /opt/intelFPGA/17.1/quartus/bin/quartus_pgmw
... auto detect
... load the jic file
... in menu tools->programmer, enable "unprotect device"
... start program/configure operation
</pre>

== Flash user page firmware via esper-tool ==

<pre>
$ ssh agmini@daq16
$ cd online/src
$ more update_pwb.perl ### check that $fw is set to the desired firmware file
$ ./update_pwb.perl pwb06 ### or give more PWB names or give "all"
</pre>

== Build firmware ==

=== quartus version to use ===

* quartus 16.1 should be used for jtag (jtagconfig and jtagd)
* quartus 17.0 should be used to build the PWB firmware (17.1 is not compatible)

=== start license server ===

<pre>
ssh agmini@daq16
/opt/intelFPGA/17.0/quartus/linux64/lmgrd -c ~agmini/online/license-daq16.dat
</pre>

=== prepare the build environment ===

<pre>
$ ssh agmini@daq16
$ /opt/intelFPGA/17.0/nios2eds/nios2_command_shell.sh
</pre>

=== get latest version of the code and build it ===

<pre>
$ cd online/firmware/git/pwb_rev1_firmware
$ git pull
$ git checkout alphag
$ git pull
$ ./scripts/compile_project.sh
... about 30-40 minutes ...
$ ls -l bin/*.sof bin/*.jic bin/*.rpd
-rw-r--r-- 1 agmini alpha 12727389 Jan 24 2018 bin/feam_rev1_auto.rpd
-rw-r--r-- 1 agmini alpha 33554661 Jan 24 2018 bin/feam_rev1.jic
-rw-r--r-- 1 agmini alpha 6974754 Jan 24 2018 bin/rev1.sof
$ ### feam.jic is loaded via jtag
$ ### feam_auto.rpd is loaded via esper
$ ### feam.sof is used to attach the quartus signal tap logic analyzer
</pre>

=== firmware build order and sequence ===

* erase previous build
* regenerate qsys
* regenerate NIOS BSP "feam_bootloader_bsp" and "feam_bsp"
* build NIOS projects "feam_bootloader" and "feam"
* build quartus sof file (with the "feam_bootloader" nios project inside)
* build jic and rpd files (with the quartus sof file and the "feam" nios project inside)
* load sof file into the FPGA using jtag (be careful about compatible "feam" nios project already in the epcq flash)
* load jic file into the epcq factory page using jtag (cycle power for them to take effect)
* load rpd file into the epcq factory or user page using esper

=== scripts for building the firmware ===

* scripts/compile_project.sh - build everything from scratch:
* scripts/compile_qsys.sh - regenerate the main qsys component
* scripts/compile_nios.sh - regenerate the BSPs, then same as compile_elf.sh
* scripts/compile_elf.sh - build the BSPs, "feam_bootloader" and "feam" NIOS projects
* scripts/update_elf.sh - regenerate the rpd and jic files to include the new "feam" NIOS build (keeps the old "feam_bootloader" in the sof file)
* scripts/update_bootloader.sh - also regenerate the sof file to include the new "feam_bootloader" build
* scripts/clean_quartus.sh - prepare quartus fpga sof file to build from scratch
* scripts/compile_quartus.sh - build the quartus fpga sof file incrementally (run clean_quartus.sh to build from scratch).
* scripts/load_jtag_sof.pl bin/rev1.sof - load sof file into the FPGA. this will boot the new fpga firmware, the new feam_bootloader (if update_bootloader.sh was run), but with the old "feam" already in epcq flash. (beware of incompatible sof, "feam_bootloader" and "feam" builds!)
* scripts/load_jtag_jic.pl bin/feam_rev1.jic - load jic file into the EPCQ flash, same as using the graphical flash programmer tool. (this will leave the FPGA running the quartus jtag flash loader firmware, not the PWB firmware. Cycle power for new jic file to take effect).

== ESPER Variables ==

* Board
** invert_ext_trig - invert trigger signal from CDM before it drives any logic (to undo incorrect signal polarity)
** reset_nios - goggle up, them down to reset NIOS subsystem

* Signalproc
** test_mode - ADC data is replaced with a test pattern, see test mode bits in sca_x_ch_ctrl.
** sca_a_ch_ctrl, sca_b_ch_ctrl, sca_c_ch_ctrl, sca_d_ch_ctrl - channel control bits:
<pre>
11..0 - threshold - channel suppression threshold
14..12 - ctrl_test_mode - test mode:
0=fixed patter 0xa5a,
1=time bin counter,
2=time bin counter with channel number,
3=sequential adc sample counter,
4={ch_crossed_out,trig_pos,trig_neg,adc[8:0]},
5={trig,adc[10:0]},
6={ch_crossed_min,adc[10:0]}
15 - ctrl_supp_mode - channel suppression mode: 0=adc<=(baseline-threshold), 1=adc<=threshold
</pre>

* Link
** link_ctrl - sata link control. The bits are:
<pre>
0 - sata_link_udp_stream_in_enable - permit data flow from sata link to OFFLOAD_SATA
1 - udp_stream_out_enable - permit sca data flow to sata link
2 - sata_to_eth_enable - permit ethernet data flow from sata link to TSE_MAC
3 - eth_to_sata_enable - permit ethernet data flow from TSE MAC to sata link
4 - enable_stop_our_tx - enable flow control: allow stop_tx
#5 - stop_our_tx - manually activate the flow control signal into link_tx
6 - enable_stop_remote_tx - enable flow control: allow send "stop_tx"
#7 - stop_remote_tx - manually activate the flow control signal into link_tx
8 - tx_test_pattern_udp - udp data is replaced by test pattern 0x11111111, 0x22222222, etc.
9 - tx_test_pattern_eth - nios-to-sata data is replaced by test pattern 0x11111111, 0x22222222, etc.
10 - udp_delay_enable - delay between udp packets, see udp_delay_value below
11 - reboot_tx - send K_REBOOT command to the sata link mate, where it has same effect as board/reset_nios.
12 - sata_to_nios_disable
13 - nios_to_sata_disable
14 -
15 -
16 -
24..31 - udp_delay_value - delay between udp packets (top 8 bits of a 20-bit counter)
</pre>

== Firmware data path ==

Main data path:
<pre>
feam_top
|
sca_sigproc
|
sca_event_control
4 * sca_control (sca_channel.sv)
4 * channel_fifo (1024 samples)
</pre>
<pre>
sca_control (sca_channel.sv)
|
state machine to control SCA read and write enables
|
sca_write_control (what does it do?!?)
sca_read_control
</pre>
<pre>
sca_read_control
|
state machine to store ADC samples into per-channel FIFOs (channel_fifo)
selector to replace ADC samples with a test pattern
79*sca_trig_one -> ch_crossed_out - channel hit detector
</pre>
<pre>
sca_event_control
|
event_fifo (hdl/mf/event_descriptor_fifo)
state machine to take ADC data from 4 per-channel FIFOs (channel_fifo) and store it into DDR memory (write addr increment is 510*8 (number of samples)*8 = 4080).
state machine to read transposed ADC data from DDR memory (read addr increment is 8+8=16 - 2 samples * 2 bytes * 4 sca = 16) and create 4 per-sca data streams
</pre>
<pre>
data stream:
4 per-sca data streams from sca_event_control -> sca_sigproc(event_dat)
|
packet_chunker (hdl/lib/packet_chunker.sv) (4*event_dat -> event_segment_dat) - multiplex 4 data streams into 1 stream of UDP-sized packets
|
packet_length_prepender (hdl/lib/packet_length_prepender.sv) (event_segment_dat -> udp_event_val_out)
|
udp_event_val -> info qsys (udp_stream_sca) and into sata link (mux channel 2).
</pre>

== Firmware build instructions from Bryerton (obsolete) ==

This is the old README.md file from the firmware git repository.

These instructions are not used to build the firmware (use scrips/compile_project.sh).

They are still useful for figuring out which buttons to push in the altera gui programs.

<pre>
# ALPHAg FrontEnd Acquisition Module (FEAM) - REV0

Welcome to the ALPHAg FEAM Project

# Table of Contents

* [Overview](#overview)
* [Inital Setup](#initial-setup)
* [Quartus Setup](#quartus-setup)
* [Create the .cdf](#create-the-cdf)
* [NIOS Setup](#nios-Setup)
* [Create the project files](#create-the-project-files)
* [Build Instructions](#build-instructions)
* [Building from Quartus GUI](#building-from-quartus-gui)
* [Generate the QSys File](#generate-the-qsys-file)
* [Compile the NIOSII Project](#compile-the-niosii-project)
* [Compile the Quartus Project](#compile-the-quartus-project)
* [Building from Scripts](#building-from-scripts)
* [Installing from JTAG](#installing-from-jtag)
* [Installing from ESPER](#installing-from-esper)

# Overview

The FEAM is a digitizer located on the ALPHAg detector. Each FEAM consists of 4 AFTER SCAs for a total of 288 channels.

# Initial Setup

## Quartus Setup

### Create the .cdf
1. Open Quartus Prime Standard Edition
2. Click on **Tools -> Programmer**. The **Programmer** window will appear
3. In the **Programmer** window, click **Auto Detect**. A **Select Device** dialog box will appear
1. (Optional) If **Auto Detect** is greyed out, click on **Hardware Setup**. The **Hardware Setup** dialog box will appear
2. Under **Hardware Settings**, double-click on the USB JTAG hardware you wish to use
3. Click **Close**. The **Hardware Setup** dialog box will close
5. Click on **File -> Save As**. The **Save As** dialog box will appear
6. In the **Save As** dialog box, browse to the **/\<project_dir\>/bin/** directory
7. Make sure the **Add file to current project** checkbox is checked
7. In the **File name** input box, type "alphag_feam.cdf" and click **Save**
8. Close the **Programmer** window
9. Done!

## NIOS Setup

### Create the project files
1. Open Quartus Prime Standard Edition
2. Open the NIOS II Software Build Tools by clicking **Tools -> NIOS II Software Build Tools for Eclipse**
3. The **Workspace Launcher** window will appear, click *OK* to accept the default workspace.
4. Click **File -> Import...**. The **Import** dialog box will appear
5. Click on **General -> Existing Projects into Workspace**
6. Click **Next \>**
7. Next to **Select root directory** click **Browse...**. A **Browse for Folder** dialog box will appear
8. Browse to **/\<project_dir\>/hdl/software** and click **OK**
8. Four Projects should appear in the **Projects** panel:
* alphag_feam
* alphag\_feam\_bsp
* alphag\_feam\_bootloader
* alphag\_feam\_bootloader\_bsp
9. Click **Select All**
10. Click **Finish**
11. Done!

# Build Instructions

## Building from Quartus GUI

**Warning: Please use Quartus Prime Standard Edition 16.1**
### Generate the QSys File
1. Open Quartus Standard Edition 16.1
1. Open QSys by clicking **Tools -> QSys**
2. Click on **File -> Open** (CTRL+O). An **Open File** dialog box should appear.
3. Select the **alphag_feam.qsys** file from the **/\<project_dir\>/hdl** directory and click **Open**
4. Modify as necessary. NOTE: Any change to the QSys, even cosmetic such as expanding or collapsing components will cause the QSys to desire a re-generate.
5. Click **File -> Save** (CTRL+S)
6. Click **Generate HDL**. A **Generation** dialog box will appear
7. Select the settings you desire to be changed, if any
8. Click **Generate**
9. Wait for the generation to complete.
10. Done!

### Compile the NIOSII Project
1. Open the NIOS II Software Build Tools by clicking **Tools -> NIOS II Software Build Tools for Eclipse**
2. (Optional) If the QSys file has been re-generated performing the following
1. Right-click on **alphag\_feam\_bsp** and select **NIOS II -> Generate BSP**. The BSP will regenerate
2. Right-click on **alphag\_feam\_bootloader\_bsp** and select **NIOS II -> Generate BSP**. The BSP will regenerate
3. Modify files as needed. WARNING: Do not modify any BSP files by hand! All changes will be lost on next generate.
4. Click **Project -> Build All** (CTRL+B). If your workspace as multiple projects, it is recommend to close all but the ones related to the FEAM.
5. Right-click on **alphag\_feam\_bootloader** and select **Make Targets -> Build...**. The **Make Targets** dialog box will appear
6. In the **Make Targets** dialog box, select **mem\_init\_generate** and click **Build**
7. Done!

### Compile the Quartus Project
1. Open Quartus Standard Edition 16.1
1. Under **Processing** click **Start Compilation** (CTRL+L)
2. Wait about 16-30 minutes.
3. Done!

**Note:** If Quartus crashed while compiling, delete the **\<project_dir\>/hdl/db** and **\<project_dir\>/hdl/incremental_db** directories and try again

## Installing from JTAG
1. Generate the .jic file using the provided /bin/alphag\_feam\_jic.cof
1. In the main Quartus window, click **File -> Convert Programming Files**, A **Convert Programming Files** window will appear
2. In the **Convert Programming Files** window, click **Open Conversion Setup Data**. An **Open** dialog box will appear
3. In the **Open** dialog box, select the **alphag_feam.cof** file located in **/bin/**
4. Click **Open**. The **Open dialog box will close
* Note the **File name** field, it should display /\<project\_dir\>/bin/alphag\_feam.jic
5. In the **Convert Programming Files** window, click **Generate**
6. When the .jic is done being generated, a small dialog box will appear, click **OK**
7. In the **Convert Programming Files** window, click **Close**
2. Load the .JIC file using the Programmer
1. In the main Quartus window in the **Project Navigator** drop-down input select **Files**.
2. Scroll down the window and find the **alphag_feam.cdf**, and double-click it. The **Programmer** window will open
3. Click on the icon labelled **5CGXFC4C6**
4. Click on the **Change File** button. The **Select New Programming File** dialog box will appear
5. Browse to **/\<project_dir\>/bin/** and select **alphag\_feam.jic**
6. Click Open. The **Select New Programming File** dialog box will close
7. In the main panel, check the **Program/Configure** box on the line that starts **./bin/alphag\_feam.jic**
8. Click the **Start** button
9. Wait for the **Progress** bar to go to 100% and stop
10. Done!
</pre>

= Internal test pulser =

Documentation from Bryerton:

<pre>
---------- Forwarded message ---------
From: Bryerton Shaw <bryerton@triumf.ca>
Date: Tue, 24 Apr 2018 at 16:18
Subject: RE: PWB/FEAM internal test pulse
To: Lars Martin <lmartin@triumf.ca>
Cc: Andrea Capra <acapra@triumf.ca>

Hi Lars (and Andrea),

In the new(er) firmware the test pulse code has been modified slightly to
fit in better with the upgraded trigger system. I can add an option to
allow it to be triggered by the external trigger (currently it has an
interval option, or you can hit the manual trigger to fire it, if it’s
enabled)

The *options for the test pulser* are under the *signalproc module*:

*test_pulse_ena*

An array of four booleans, that control turning on the test pulse for each
SCA. If this is False, the other options below will do nothings for that SCA

*test_pulse_interval_ena*

An array of four booleans, that control turning on the interval trigger for
each SCA

*test_pulse_interval*

An unsigned 32-bit value that marks how many 16ns clock cycles occur
between interval triggers. Only one value, as there is only one interval
timer.

*test_pulse_wdt*

An array of four 16-bit unsigned integers that control the width of the
pulse for each SCA when the trigger occurs.

Now, there is a second set of options to control the delay of the *test
pulse interval trigger*. These options are located under the *trigger
module*

*intp_trig_ena*

Boolean value enabling/disabling the interval test pulse trigger.

*intp_trig_delay*

Unsigned 32-bit integer controlling the trigger delay, which is the time
between when the trigger is requested, and when the trigger actually fires.

You can setup the interval trigger and use that to create event packets
with the test pulse on set intervals. If the interval trigger is
insufficient, I can create another option, which is to fire the test pulse
for enabled SCAs when the external trigger goes. If so, I’ll add this
option in the *signalproc *module for you:

*test_pulse_on_ext*

An array of four Booleans, that control if the test pulse for that SCA
fires on external trigger.

Remember, in order to use the SCA test pulse at all, you must also set the
SCA to test mode. You can do this by going into the *sca0-3 modules* and
setting *test* to *functionality *(the value is *3* if you want to set it
via esper-tool)

Btw currently the SCAs are hard-coded to have all 72 channels receive the
test pulse on firing, and the option to de-select/select which channels to
fire is not offered. Would you like this ability? And if so do you want it
to set all four SCAs the same, or have an option for all 288 channels?

The waveform viewer on the webpage was removed for technical reasons, and
so all viewing is done via the MIDAS DAQ. There are some preliminary tools
I’ve created to verify the event packets and their waveforms, but I
wouldn’t call them easy to setup or use at the moment. (Or very useful for
fine comparisons). If KO doesn’t have a way of saving the events to disk, I
could deliver something that captured and saved waveforms quite quickly.

Regards,
Bryerton

*From:* Lars Martin
*Sent:* April 23, 2018 5:10 PM
*To:* Bryerton Shaw <bryerton@triumf.ca>
*Cc:* Andrea Capra <acapra@triumf.ca>
*Subject:* PWB/FEAM internal test pulse

Hi Bryerton,

Andrea and I would like to check the response function of the PWBs. For
that Daryl suggested to first look at the internal test pulse.

Can you point us to what settings are required to make that go? And to use
the external trigger if we want to acquire the data?

Ideally we would acquire Midas data from that, but it's unclear if KO's
frontend allows for that right now. Can we access the data with Esper
instead?

Finally, we haven't had the waveform display on the web page for a while,
but Daryl said you guys use it. Are we using a different page somehow?

Hope all is well with the baby, cheers,

Lars
</pre>

= TODO =

* IMPOSSIBLE (ext clock only connected to clock cleaner) add frequency counter for the external clock (62.5 MHz)
* DONE add random delay before memtest
* DONE add 4 more bits to udp_delay
* DONE fix crash on boot if run is active and triggers arrive (check that trigger is off on boot) (do not set trigger/enable_all, ext_trig_ena, inp_trig_ena, signal_proc/force_run to 1 after boot)
* DONE (kludge jtag uart write()) increase size of jtag uart output buffer, make it unlimited if possible.
* DONE (kludge jtag uart write()) on dhcp timeout, reboot the fpga - reboot of nios does not clear the jtag uart, which becomes full and nios stops.
* DONE add NIOS watchdog timeout for fectrl
* DONE do not use external clock until instructed by fectrl (clock cleaner default is pin select mode, default pin mode is input with pull down, so clock 0 is selected, this is the ext clock. but we can drive clock select pins from the fpga).
* DONE change clock cleaner power up: use the 2 lines status_clkin0 and status_clkin1 to tell clock cleaner to use the internal clock (clkin2) - from the FPGA drive first line low, second line high. Also change all NIOS code that programs the clock cleaner to keep these two lines in the "input" mode. Current code switches them to "output" mode.
* DONE clock cleaner power up state is wrong for both choices of clock: 62.5 MHz ext clock and 125 MHz internal clock, see explanation in feam_top.sv. It looks like I need to drive status_clkin0 and status_clkin1 to "holdover" mode, both logic level 1. Currently status_clkin0 is used as SPI read line, this has to be moved to a different line first.
* add watchdog timeout that works in factory mode ("remote update" watchdog only works in user mode)
* enable HTTP pipelining - copy changes on mongoose from ADC project.
* reset everything if it looks like NIOS TCP/IP code has crashed, but enough code is still running to prevent tripping of the fpga-remote-update watchdog timer. such crash tends to happens when the sata link mate is rebooted and incomplete data arrives from the disrupted ethernet data path.
* when loading AFTER ASIC do not go into infinite loop if read-back is broken (always mismatch).
* read and report error status information bits for 2983fc.pdf

= ZZZ =

ZZZ

PWB

2021-07-31T19:56:42Z

Acapra: added all the hardware on the PWSs

= Links =

* https://bitbucket.org/expalpha/pwb_rev1_firmware
* https://edev-group.triumf.ca/fw/exp/alphag/feam/rev1
* https://edev-group.triumf.ca/hw/alphag/feam/rev1
* https://ladd00.triumf.ca/daqinv/frontend/list/170 - PWB inventory database

= Schematics =

* [[Image:pwb_rev0.pdf]]
* [[Image:Pwb-rev1-after.pdf]]
* [[Image:AlphaG Rev1.pdf]]

= Manuals =

* [[Image:AFTER_MANUAL_V1.pdf]] AFTER SCA manual v1
* [[Image:AFTER_DataSheet1_1.pdf]] AFTER SCA manual v1.1
* [[Image:AFTER_SCA_table2.pdf]] AFTER SCA channel map

= Hardware =

== Power distribution ==

External power supply is:
* +5V
* +2V

Central board:

* Ext_5V - F1 - RS1 (0.1R) - Fuse_5V
* Ext_2V - F2/F3 - RS2 (0.1R) - Idig2-
* LDO3 - Fuse_5V - DIG_3.3V - selfenable - ADM7154
* LDO5 - Fuse_5V - DIG_2.5V - selfenable - ADM7154
* LDO_CLN - Fuse5V - CLN_3.3V - selfenable -
* VREG - DIG_3.3V - -2.5V SCA BIAS - selfenable - LTC1983ES6-3#TRMPBF
* LDO7 - DIG_2.5V - DDR3_VRef - enable DDR3_1P5V, DDR3_PG, DDR3_VttEN
* LDO2 - Idig2V- - DIG_1.1V - enable DIG_3.3V - LT3071
* LDO6 - Idig2V- - DDR3_1P5V - enable DIG_3.3V - LT3083

SCA Wing:

* Ext_5V - F1 - RS1 (0.3R) - Isca- - SCA_Fuse5V
* LDO1 - SCA_Fuse5V - SCA3.3V - enable ADC_PWR_EN - ADM7154
* LDO2 - SCA3.3V - ADC1.8V - selfenable - ADM7154
* LDO3 - SCA3.3V - TRN2.5V - selfenable - LT1761
* REF - SCA3.3V - 1.45V Vicm - selfenable - LM4121

== Trigger and Clock distribution ==

Clock (62.5MHz) and trigger are distributed to the PWB columns via the [https://edev-group.triumf.ca/hw/alphag/trigger-card-sata-adapter/rev1 trigger SATA adapter].

The two miniSAS cables from cdm03 port 5 and 6 are connected to a miniSAS-SATA splitter 1-to-4 (CS Electronics "iSAS SATA adapter" ADP-887P-1X). Since there are only 8 columns and the clock and trigger signals are daisy-chained across the 8 rows, this scheme is sufficient to provide the necessary inputs to all 64 PWBs.

== Optical Link ==

Main data uplink.

SFP Avago

Optical fiber 10m

== SATA Link ==

Secondary data uplink, capable also to deliver clock and trigger.

= Firmware =

== Firmware update ==

PWB firmware update is done using "esper-tool".

<pre>
esper-tool -v upload -f file.rpd http://pwbNN update factory_rpd
esper-tool -v upload -f file.rpd http://pwbNN update file_rpd
</pre>

As of PWB firmware pwb_rev1_20200706_ko:

Permission to write into the flash memory is controlled by esper variables:

* allow_write set to "false": "esper-tool upload" does a "verify": successful upload means epcq flash content is same as rpd file, error means content is not the same.
* allow_write set to "true": "esper-tool upload" updates the epcq flash from the rpd file: for each data block, read the epcq flash, compare to rpd data, if different, erase the flash block, write the flash, read the flash, if there is a mismatch (bad write), return an error.
* allow_factory_write set to "true" to enable writing to the "factory_rpd". DO NOT SET IT TO "true" UNLESS YOU MEAN TO UPDATE THE PWB BOOTLOADER. IF YOU FLASH BUM FIRMWARE or IF THERE IS A POWER OUTAGE WHILE UPDATING, THE PWB WILL BE BRICKED and has to be recovered by connecting a USB blaster.

To update firmware of multiple boards to correct version, recommended is the script "update_pwb.perl":
<pre>
cd ~/online/src
./update_pwb.perl pwb12 pwb06 pwb78
</pre>

== Factory page and user page firmware boot ==

Each PWB board has 2 firmware images: boot loader (factory page)
and data acquisition (user page). On power up, the FPGA loads and runs
the boot loader firmware from the factory page, later the control
software reboots the FPGA into the data acquisition firmware from
the user page.

Because loading defective firmware can brick https://en.wikipedia.org/wiki/Brick_(electronics)
the PWB, new firmware images are loaded into the user page, if anything is wrong,
the PWB can still boot from the factory page and one can use the firmware update
function to load a good firmware image into the user page.

The boot loader firmware in the factory page is not intended for data acquisition, it is only meant to provide three functions:
* boot enough hardware and software to communicate via the ethernet and the sata link
* firmware update (both factory page and user page)
* reboot into the user page

The boot loader firmware in the factory page is usually never updated unless absolutely
needed to fix a problem with these three functions (i.e. compatibility with sata link communications).

If factory image is corrupted or the contains defective firmware and the PWB does not boot (no dhcp, no ping, no esper),
the only way to unbrick it is by loading good firmware using a jtag flash programmer (usb-blaster). Connecting it requires
physical access to the jtag connector on the PWB board. It is impossible when the detector is fully assembled in the experiment.

The PWB has a 32 Mbyte EPCQ flash memory chip, it is divided into 2 pages 16 Mbytes of factory page and 16 Mbytes of user page.

A complete firmware image generally contains 3 pieces:
* FPGA firmware and NIOS "feam_bootloader" software RAM image (sof file)
* NIOS "feam" software RAM image (feam.elf.flash.hex) - all the C code for esper communications and PWB data acquisition
* NIOS filesystem image (feam.webpkg.hex) - esper web pages

This is the boot sequence:
* on power-up the FPGA automatically loads the sof file from address 0 of the EPCQ flash (this is the factory image sof file)
* the FPGA is "started"
* the NIOS CPU starts executing the "feam_bootloader" C code embedded in the sof file (NIOS project hdl/software/feam_bootloader)
* feam_bootloader initializes the hardware (clocks, DDR memory, etc)
* feam_bootloader write-protects the EPCQ flash memory
* feam_bootloader copies the "feam" software from EPCQ flash to the DDR memory (after checking for correct checkum)
* feam_bootloader restarts the NIOS CPU
* the NIOS CPU starts running from DDR memory, executes the "feam" software (NIOS project hdl/software/feam)
* call main() in hdl/software/feam/src/task_init.c
* call task_init() in the same file
* infinite loop waiting for IP address via DHCP
* after have IP address, call task_esper() from hdl/software/feam/src/task_esper.c
* task_esper() creates all the esper modules, esper variables, etc,
* mod_http.c starts the mongoose web server
* PWB is open for business

This is the protection against booting corrupted firmware:
* FPGA hardware loads the sof file from epcq flash and checks for correct checksum before "starting" it
* NIOS CPU is part of the sof file, protected by sof file checksum
* NIOS feam_bootloader C code is part of the sof file, protected by the sof checksum
* feam_bootloader checks for correct signatures and checksums of the DDR memory image
* "feam" C code is protected by the checksum of the DDR memory image

== NIOS terminal ==

<pre>
$ ssh agmini@daq16
$ /opt/intelFPGA/16.1/quartus/bin/jtagconfig
1) USB-Blaster [2-1.2]
02B030DD 5CGTFD7(B5|C5|D5)/5CGXBC7B6/..
$ /opt/intelFPGA/17.0/quartus/bin/nios2-terminal
nios2-terminal: connected to hardware target using JTAG UART on cable
nios2-terminal: "USB-Blaster [2-1.2]", device 1, instance 0
nios2-terminal: (Use the IDE stop button or Ctrl-C to terminate)
PWB Revision 1 Boot Loader
Ver 2.0 Build 357 - Wed Jun 6 15:05:35 PDT 2018
...
</pre>

== Flash boot loader firmware via jtag ==

<pre>
$ ssh agmini@daq16
$ /opt/intelFPGA/16.1/quartus/bin/jtagconfig
1) USB-Blaster [2-1.2]
02B030DD 5CGTFD7(B5|C5|D5)/5CGXBC7B6/..
$ cd ~/online/firmware/pwb_rev1
$ ls -l
$ /opt/intelFPGA/17.1/quartus/bin/quartus_pgmw
... auto detect
... load the jic file
... in menu tools->programmer, enable "unprotect device"
... start program/configure operation
</pre>

== Flash user page firmware via esper-tool ==

<pre>
$ ssh agmini@daq16
$ cd online/src
$ more update_pwb.perl ### check that $fw is set to the desired firmware file
$ ./update_pwb.perl pwb06 ### or give more PWB names or give "all"
</pre>

== Build firmware ==

=== quartus version to use ===

* quartus 16.1 should be used for jtag (jtagconfig and jtagd)
* quartus 17.0 should be used to build the PWB firmware (17.1 is not compatible)

=== start license server ===

<pre>
ssh agmini@daq16
/opt/intelFPGA/17.0/quartus/linux64/lmgrd -c ~agmini/online/license-daq16.dat
</pre>

=== prepare the build environment ===

<pre>
$ ssh agmini@daq16
$ /opt/intelFPGA/17.0/nios2eds/nios2_command_shell.sh
</pre>

=== get latest version of the code and build it ===

<pre>
$ cd online/firmware/git/pwb_rev1_firmware
$ git pull
$ git checkout alphag
$ git pull
$ ./scripts/compile_project.sh
... about 30-40 minutes ...
$ ls -l bin/*.sof bin/*.jic bin/*.rpd
-rw-r--r-- 1 agmini alpha 12727389 Jan 24 2018 bin/feam_rev1_auto.rpd
-rw-r--r-- 1 agmini alpha 33554661 Jan 24 2018 bin/feam_rev1.jic
-rw-r--r-- 1 agmini alpha 6974754 Jan 24 2018 bin/rev1.sof
$ ### feam.jic is loaded via jtag
$ ### feam_auto.rpd is loaded via esper
$ ### feam.sof is used to attach the quartus signal tap logic analyzer
</pre>

=== firmware build order and sequence ===

* erase previous build
* regenerate qsys
* regenerate NIOS BSP "feam_bootloader_bsp" and "feam_bsp"
* build NIOS projects "feam_bootloader" and "feam"
* build quartus sof file (with the "feam_bootloader" nios project inside)
* build jic and rpd files (with the quartus sof file and the "feam" nios project inside)
* load sof file into the FPGA using jtag (be careful about compatible "feam" nios project already in the epcq flash)
* load jic file into the epcq factory page using jtag (cycle power for them to take effect)
* load rpd file into the epcq factory or user page using esper

=== scripts for building the firmware ===

* scripts/compile_project.sh - build everything from scratch:
* scripts/compile_qsys.sh - regenerate the main qsys component
* scripts/compile_nios.sh - regenerate the BSPs, then same as compile_elf.sh
* scripts/compile_elf.sh - build the BSPs, "feam_bootloader" and "feam" NIOS projects
* scripts/update_elf.sh - regenerate the rpd and jic files to include the new "feam" NIOS build (keeps the old "feam_bootloader" in the sof file)
* scripts/update_bootloader.sh - also regenerate the sof file to include the new "feam_bootloader" build
* scripts/clean_quartus.sh - prepare quartus fpga sof file to build from scratch
* scripts/compile_quartus.sh - build the quartus fpga sof file incrementally (run clean_quartus.sh to build from scratch).
* scripts/load_jtag_sof.pl bin/rev1.sof - load sof file into the FPGA. this will boot the new fpga firmware, the new feam_bootloader (if update_bootloader.sh was run), but with the old "feam" already in epcq flash. (beware of incompatible sof, "feam_bootloader" and "feam" builds!)
* scripts/load_jtag_jic.pl bin/feam_rev1.jic - load jic file into the EPCQ flash, same as using the graphical flash programmer tool. (this will leave the FPGA running the quartus jtag flash loader firmware, not the PWB firmware. Cycle power for new jic file to take effect).

== ESPER Variables ==

* Board
** invert_ext_trig - invert trigger signal from CDM before it drives any logic (to undo incorrect signal polarity)
** reset_nios - goggle up, them down to reset NIOS subsystem

* Signalproc
** test_mode - ADC data is replaced with a test pattern, see test mode bits in sca_x_ch_ctrl.
** sca_a_ch_ctrl, sca_b_ch_ctrl, sca_c_ch_ctrl, sca_d_ch_ctrl - channel control bits:
<pre>
11..0 - threshold - channel suppression threshold
14..12 - ctrl_test_mode - test mode:
0=fixed patter 0xa5a,
1=time bin counter,
2=time bin counter with channel number,
3=sequential adc sample counter,
4={ch_crossed_out,trig_pos,trig_neg,adc[8:0]},
5={trig,adc[10:0]},
6={ch_crossed_min,adc[10:0]}
15 - ctrl_supp_mode - channel suppression mode: 0=adc<=(baseline-threshold), 1=adc<=threshold
</pre>

* Link
** link_ctrl - sata link control. The bits are:
<pre>
0 - sata_link_udp_stream_in_enable - permit data flow from sata link to OFFLOAD_SATA
1 - udp_stream_out_enable - permit sca data flow to sata link
2 - sata_to_eth_enable - permit ethernet data flow from sata link to TSE_MAC
3 - eth_to_sata_enable - permit ethernet data flow from TSE MAC to sata link
4 - enable_stop_our_tx - enable flow control: allow stop_tx
#5 - stop_our_tx - manually activate the flow control signal into link_tx
6 - enable_stop_remote_tx - enable flow control: allow send "stop_tx"
#7 - stop_remote_tx - manually activate the flow control signal into link_tx
8 - tx_test_pattern_udp - udp data is replaced by test pattern 0x11111111, 0x22222222, etc.
9 - tx_test_pattern_eth - nios-to-sata data is replaced by test pattern 0x11111111, 0x22222222, etc.
10 - udp_delay_enable - delay between udp packets, see udp_delay_value below
11 - reboot_tx - send K_REBOOT command to the sata link mate, where it has same effect as board/reset_nios.
12 - sata_to_nios_disable
13 - nios_to_sata_disable
14 -
15 -
16 -
24..31 - udp_delay_value - delay between udp packets (top 8 bits of a 20-bit counter)
</pre>

== Firmware data path ==

Main data path:
<pre>
feam_top
|
sca_sigproc
|
sca_event_control
4 * sca_control (sca_channel.sv)
4 * channel_fifo (1024 samples)
</pre>
<pre>
sca_control (sca_channel.sv)
|
state machine to control SCA read and write enables
|
sca_write_control (what does it do?!?)
sca_read_control
</pre>
<pre>
sca_read_control
|
state machine to store ADC samples into per-channel FIFOs (channel_fifo)
selector to replace ADC samples with a test pattern
79*sca_trig_one -> ch_crossed_out - channel hit detector
</pre>
<pre>
sca_event_control
|
event_fifo (hdl/mf/event_descriptor_fifo)
state machine to take ADC data from 4 per-channel FIFOs (channel_fifo) and store it into DDR memory (write addr increment is 510*8 (number of samples)*8 = 4080).
state machine to read transposed ADC data from DDR memory (read addr increment is 8+8=16 - 2 samples * 2 bytes * 4 sca = 16) and create 4 per-sca data streams
</pre>
<pre>
data stream:
4 per-sca data streams from sca_event_control -> sca_sigproc(event_dat)
|
packet_chunker (hdl/lib/packet_chunker.sv) (4*event_dat -> event_segment_dat) - multiplex 4 data streams into 1 stream of UDP-sized packets
|
packet_length_prepender (hdl/lib/packet_length_prepender.sv) (event_segment_dat -> udp_event_val_out)
|
udp_event_val -> info qsys (udp_stream_sca) and into sata link (mux channel 2).
</pre>

== Firmware build instructions from Bryerton (obsolete) ==

This is the old README.md file from the firmware git repository.

These instructions are not used to build the firmware (use scrips/compile_project.sh).

They are still useful for figuring out which buttons to push in the altera gui programs.

<pre>
# ALPHAg FrontEnd Acquisition Module (FEAM) - REV0

Welcome to the ALPHAg FEAM Project

# Table of Contents

* [Overview](#overview)
* [Inital Setup](#initial-setup)
* [Quartus Setup](#quartus-setup)
* [Create the .cdf](#create-the-cdf)
* [NIOS Setup](#nios-Setup)
* [Create the project files](#create-the-project-files)
* [Build Instructions](#build-instructions)
* [Building from Quartus GUI](#building-from-quartus-gui)
* [Generate the QSys File](#generate-the-qsys-file)
* [Compile the NIOSII Project](#compile-the-niosii-project)
* [Compile the Quartus Project](#compile-the-quartus-project)
* [Building from Scripts](#building-from-scripts)
* [Installing from JTAG](#installing-from-jtag)
* [Installing from ESPER](#installing-from-esper)

# Overview

The FEAM is a digitizer located on the ALPHAg detector. Each FEAM consists of 4 AFTER SCAs for a total of 288 channels.

# Initial Setup

## Quartus Setup

### Create the .cdf
1. Open Quartus Prime Standard Edition
2. Click on **Tools -> Programmer**. The **Programmer** window will appear
3. In the **Programmer** window, click **Auto Detect**. A **Select Device** dialog box will appear
1. (Optional) If **Auto Detect** is greyed out, click on **Hardware Setup**. The **Hardware Setup** dialog box will appear
2. Under **Hardware Settings**, double-click on the USB JTAG hardware you wish to use
3. Click **Close**. The **Hardware Setup** dialog box will close
5. Click on **File -> Save As**. The **Save As** dialog box will appear
6. In the **Save As** dialog box, browse to the **/\<project_dir\>/bin/** directory
7. Make sure the **Add file to current project** checkbox is checked
7. In the **File name** input box, type "alphag_feam.cdf" and click **Save**
8. Close the **Programmer** window
9. Done!

## NIOS Setup

### Create the project files
1. Open Quartus Prime Standard Edition
2. Open the NIOS II Software Build Tools by clicking **Tools -> NIOS II Software Build Tools for Eclipse**
3. The **Workspace Launcher** window will appear, click *OK* to accept the default workspace.
4. Click **File -> Import...**. The **Import** dialog box will appear
5. Click on **General -> Existing Projects into Workspace**
6. Click **Next \>**
7. Next to **Select root directory** click **Browse...**. A **Browse for Folder** dialog box will appear
8. Browse to **/\<project_dir\>/hdl/software** and click **OK**
8. Four Projects should appear in the **Projects** panel:
* alphag_feam
* alphag\_feam\_bsp
* alphag\_feam\_bootloader
* alphag\_feam\_bootloader\_bsp
9. Click **Select All**
10. Click **Finish**
11. Done!

# Build Instructions

## Building from Quartus GUI

**Warning: Please use Quartus Prime Standard Edition 16.1**
### Generate the QSys File
1. Open Quartus Standard Edition 16.1
1. Open QSys by clicking **Tools -> QSys**
2. Click on **File -> Open** (CTRL+O). An **Open File** dialog box should appear.
3. Select the **alphag_feam.qsys** file from the **/\<project_dir\>/hdl** directory and click **Open**
4. Modify as necessary. NOTE: Any change to the QSys, even cosmetic such as expanding or collapsing components will cause the QSys to desire a re-generate.
5. Click **File -> Save** (CTRL+S)
6. Click **Generate HDL**. A **Generation** dialog box will appear
7. Select the settings you desire to be changed, if any
8. Click **Generate**
9. Wait for the generation to complete.
10. Done!

### Compile the NIOSII Project
1. Open the NIOS II Software Build Tools by clicking **Tools -> NIOS II Software Build Tools for Eclipse**
2. (Optional) If the QSys file has been re-generated performing the following
1. Right-click on **alphag\_feam\_bsp** and select **NIOS II -> Generate BSP**. The BSP will regenerate
2. Right-click on **alphag\_feam\_bootloader\_bsp** and select **NIOS II -> Generate BSP**. The BSP will regenerate
3. Modify files as needed. WARNING: Do not modify any BSP files by hand! All changes will be lost on next generate.
4. Click **Project -> Build All** (CTRL+B). If your workspace as multiple projects, it is recommend to close all but the ones related to the FEAM.
5. Right-click on **alphag\_feam\_bootloader** and select **Make Targets -> Build...**. The **Make Targets** dialog box will appear
6. In the **Make Targets** dialog box, select **mem\_init\_generate** and click **Build**
7. Done!

### Compile the Quartus Project
1. Open Quartus Standard Edition 16.1
1. Under **Processing** click **Start Compilation** (CTRL+L)
2. Wait about 16-30 minutes.
3. Done!

**Note:** If Quartus crashed while compiling, delete the **\<project_dir\>/hdl/db** and **\<project_dir\>/hdl/incremental_db** directories and try again

## Installing from JTAG
1. Generate the .jic file using the provided /bin/alphag\_feam\_jic.cof
1. In the main Quartus window, click **File -> Convert Programming Files**, A **Convert Programming Files** window will appear
2. In the **Convert Programming Files** window, click **Open Conversion Setup Data**. An **Open** dialog box will appear
3. In the **Open** dialog box, select the **alphag_feam.cof** file located in **/bin/**
4. Click **Open**. The **Open dialog box will close
* Note the **File name** field, it should display /\<project\_dir\>/bin/alphag\_feam.jic
5. In the **Convert Programming Files** window, click **Generate**
6. When the .jic is done being generated, a small dialog box will appear, click **OK**
7. In the **Convert Programming Files** window, click **Close**
2. Load the .JIC file using the Programmer
1. In the main Quartus window in the **Project Navigator** drop-down input select **Files**.
2. Scroll down the window and find the **alphag_feam.cdf**, and double-click it. The **Programmer** window will open
3. Click on the icon labelled **5CGXFC4C6**
4. Click on the **Change File** button. The **Select New Programming File** dialog box will appear
5. Browse to **/\<project_dir\>/bin/** and select **alphag\_feam.jic**
6. Click Open. The **Select New Programming File** dialog box will close
7. In the main panel, check the **Program/Configure** box on the line that starts **./bin/alphag\_feam.jic**
8. Click the **Start** button
9. Wait for the **Progress** bar to go to 100% and stop
10. Done!
</pre>

= Internal test pulser =

Documentation from Bryerton:

<pre>
---------- Forwarded message ---------
From: Bryerton Shaw <bryerton@triumf.ca>
Date: Tue, 24 Apr 2018 at 16:18
Subject: RE: PWB/FEAM internal test pulse
To: Lars Martin <lmartin@triumf.ca>
Cc: Andrea Capra <acapra@triumf.ca>

Hi Lars (and Andrea),

In the new(er) firmware the test pulse code has been modified slightly to
fit in better with the upgraded trigger system. I can add an option to
allow it to be triggered by the external trigger (currently it has an
interval option, or you can hit the manual trigger to fire it, if it’s
enabled)

The *options for the test pulser* are under the *signalproc module*:

*test_pulse_ena*

An array of four booleans, that control turning on the test pulse for each
SCA. If this is False, the other options below will do nothings for that SCA

*test_pulse_interval_ena*

An array of four booleans, that control turning on the interval trigger for
each SCA

*test_pulse_interval*

An unsigned 32-bit value that marks how many 16ns clock cycles occur
between interval triggers. Only one value, as there is only one interval
timer.

*test_pulse_wdt*

An array of four 16-bit unsigned integers that control the width of the
pulse for each SCA when the trigger occurs.

Now, there is a second set of options to control the delay of the *test
pulse interval trigger*. These options are located under the *trigger
module*

*intp_trig_ena*

Boolean value enabling/disabling the interval test pulse trigger.

*intp_trig_delay*

Unsigned 32-bit integer controlling the trigger delay, which is the time
between when the trigger is requested, and when the trigger actually fires.

You can setup the interval trigger and use that to create event packets
with the test pulse on set intervals. If the interval trigger is
insufficient, I can create another option, which is to fire the test pulse
for enabled SCAs when the external trigger goes. If so, I’ll add this
option in the *signalproc *module for you:

*test_pulse_on_ext*

An array of four Booleans, that control if the test pulse for that SCA
fires on external trigger.

Remember, in order to use the SCA test pulse at all, you must also set the
SCA to test mode. You can do this by going into the *sca0-3 modules* and
setting *test* to *functionality *(the value is *3* if you want to set it
via esper-tool)

Btw currently the SCAs are hard-coded to have all 72 channels receive the
test pulse on firing, and the option to de-select/select which channels to
fire is not offered. Would you like this ability? And if so do you want it
to set all four SCAs the same, or have an option for all 288 channels?

The waveform viewer on the webpage was removed for technical reasons, and
so all viewing is done via the MIDAS DAQ. There are some preliminary tools
I’ve created to verify the event packets and their waveforms, but I
wouldn’t call them easy to setup or use at the moment. (Or very useful for
fine comparisons). If KO doesn’t have a way of saving the events to disk, I
could deliver something that captured and saved waveforms quite quickly.

Regards,
Bryerton

*From:* Lars Martin
*Sent:* April 23, 2018 5:10 PM
*To:* Bryerton Shaw <bryerton@triumf.ca>
*Cc:* Andrea Capra <acapra@triumf.ca>
*Subject:* PWB/FEAM internal test pulse

Hi Bryerton,

Andrea and I would like to check the response function of the PWBs. For
that Daryl suggested to first look at the internal test pulse.

Can you point us to what settings are required to make that go? And to use
the external trigger if we want to acquire the data?

Ideally we would acquire Midas data from that, but it's unclear if KO's
frontend allows for that right now. Can we access the data with Esper
instead?

Finally, we haven't had the waveform display on the web page for a while,
but Daryl said you guys use it. Are we using a different page somehow?

Hope all is well with the baby, cheers,

Lars
</pre>

= TODO =

* IMPOSSIBLE (ext clock only connected to clock cleaner) add frequency counter for the external clock (62.5 MHz)
* DONE add random delay before memtest
* DONE add 4 more bits to udp_delay
* DONE fix crash on boot if run is active and triggers arrive (check that trigger is off on boot) (do not set trigger/enable_all, ext_trig_ena, inp_trig_ena, signal_proc/force_run to 1 after boot)
* DONE (kludge jtag uart write()) increase size of jtag uart output buffer, make it unlimited if possible.
* DONE (kludge jtag uart write()) on dhcp timeout, reboot the fpga - reboot of nios does not clear the jtag uart, which becomes full and nios stops.
* DONE add NIOS watchdog timeout for fectrl
* DONE do not use external clock until instructed by fectrl (clock cleaner default is pin select mode, default pin mode is input with pull down, so clock 0 is selected, this is the ext clock. but we can drive clock select pins from the fpga).
* DONE change clock cleaner power up: use the 2 lines status_clkin0 and status_clkin1 to tell clock cleaner to use the internal clock (clkin2) - from the FPGA drive first line low, second line high. Also change all NIOS code that programs the clock cleaner to keep these two lines in the "input" mode. Current code switches them to "output" mode.
* DONE clock cleaner power up state is wrong for both choices of clock: 62.5 MHz ext clock and 125 MHz internal clock, see explanation in feam_top.sv. It looks like I need to drive status_clkin0 and status_clkin1 to "holdover" mode, both logic level 1. Currently status_clkin0 is used as SPI read line, this has to be moved to a different line first.
* add watchdog timeout that works in factory mode ("remote update" watchdog only works in user mode)
* enable HTTP pipelining - copy changes on mongoose from ADC project.
* reset everything if it looks like NIOS TCP/IP code has crashed, but enough code is still running to prevent tripping of the fpga-remote-update watchdog timer. such crash tends to happens when the sata link mate is rebooted and incomplete data arrives from the disrupted ethernet data path.
* when loading AFTER ASIC do not go into infinite loop if read-back is broken (always mismatch).
* read and report error status information bits for 2983fc.pdf

= ZZZ =

ZZZ

File:TPC magnetometer locations.pdf

2021-06-03T23:16:29Z

Acapra: Acapra uploaded a new version of File:TPC magnetometer locations.pdf

Sketch of NMR+Hall in the TPC

Detector Services

2021-04-10T04:12:24Z

Acapra:

Back to [[Main Page]]

= Table of Figures =

[[#_Toc493684884|Figure 1 - Gas system schematic 6]]

[[#_Toc493684885|Figure 2 - A half-rack space is used for the gas control implementation. 7]]

[[#_Toc493684886|Figure 3 - Gas system half rack ready for installation 8]]

[[#_Toc493684887|Figure 4 - Cathode pad half-cylinder. Pairs of cooling pipes are visible 9]]

[[#_Toc493684888|Figure 5 - Water cooling system during test. 9]]

[[#_Toc493684889|Figure 6 - Model and 3D-printed realization of the dry-air manifold, very similar to the manifolds for the water cooling system. 12]]

[[#_Toc493684890|Figure 7 - Overall water cooling system 13]]

[[#_Toc493684891|Figure 8 - Temperature Monitor Board 14]]

[[#_Toc493684892|Figure 9 - End plate view of the layout of the different electronics boards & HV connections 15]]

[[#_Toc493684893|Figure 10 – Unpotted Anode Wire Card and Board with HV distribution 17]]

[[#_Toc493684894|Figure 11 - Anode wire pins and Field wire pins under yellow cover 17]]

[[#_Toc493684895|Figure 12 – New Anode Wire card with 16 potted HV capacitors, gold crimp pin Mill-max socket to the anode wire 18]]

[[#_Toc493684896|Figure 13 - Anode wire pre-amp power distribution & 16 channel temperature boards 19]]

[[#_Toc493684897|Figure 14 - Overall Low voltage distribution for anode wires and cathode pads 21]]

[[#_Toc493684898|Figure 15 - High Voltage & Grounding scheme for the rTPC 22]]

[[#_Toc493684899|Figure 16- Alpha-g rTPC rack organization 23]]

= Introduction =



This document is to describe the different services for the rTPC detector. 3 main sections cover the ''Gas system'', the ''Water cooling system'' and the ''Power distribution''.

= References and related Documents =

A dedicated TRIUMF Wiki site gathers all documentations for the rTPC detector system and equipment. Gas system, cooling water system and power distribution can be found there.

= Gas system =

The gas system as described below includes the entire gas infrastructure required to deliver gas to the detector, monitor and control the gas flow into the detector. It also includes the necessary equipment to notify any change from the nominal gas flow or gas mixture ratio delivered to the detector.

This type of mixed gas system is well understood by the Triumf detector group staff and no issue in its realization is expected.

This system is a stand-alone infrastructure assembled and delivered to CERN. It will be placed in the Alpha-g experimental area floor. The delivery of the different gases to the experimental area is under the responsibility of the Gas handling group from CERN.

The gas delivery system for the radial TPC (rTPC) is to provide Ar/CO2 gas mixture between 90/10 to 50/50 depending on the user requirements. This system does not recycle the gas as the environmental impact of either gas is minimal and the operation cost versus a recycling system would not be economic.

In the event of an additional quenching gas requirement, the overall gas system will have to be reconsidered and possible recycling system envisaged.

A simple gas mixer using digital flow meters will provide the precise mixture dialed by hand in the control system.

During the prototype phase, an ad-hoc gas system will be put in place. A portable gas system is available for such a test.

== Specifications ==

{|
! '''''Parameter'''''
! '''''Value'''''
|-
| Gas mixture
| Ar/CO2 (90/10..50/50)
|-
| Gas flow requirements
| 0.3l/min (50%), 0.3l/min (50%)
|-
| Detector gas Volume
| 0.18m3
|-
| Flushing time
| 5h @ 600cc/min
|-
| Flow rate
| 100 to 300 cc/minute
|}

== Operation ==

This gas system is quite simple as it takes 2 or 3 (spare) gas supplies and mixes them in a manifold before delivery to the rTPC detector. Necessary valves for purging lines and by-passing the detector are foreseen. The individual in-flows are controlled by gas dedicated mass flow controllers (MFC-x). The return flow is measured for leak assessment. An additional manometer monitors the internal detector pressure for evaluation of the electron drift characteristics.

[[File:imageservice2.png|575x315px|PA_doc-1.PNG]]

Figure 1 - Gas system schematic

The main flow controller and monitoring is performed by industrial devices from MKS. We added analog flowmeters on each gas line to provide a quick visual inspection of the gas mixing system.

This system will be hooked to the CERN AD-Gas facility. The CERN gas delivers gas through 2 switchable packs of 12 bottles for each gas.

The gas exhaust is evacuated to the exterior of the experimental area with an elevation of about 8m.

[[File:imageservice3.png|366x263px|PA_doc-2.PNG]]

Figure 2 - A half-rack space is used for the gas control implementation.



[[File:imageservice4.png|324x364px]]

Figure 3 - Gas system half rack ready for installation

== Quick Start ==
[[File:20181205 103217.jpg|thumb|Normal configuration of the manual valves of the GHS.]]
From-scratch (Neither gas supply lines to the GHS nor connections to and from the TPC connected) connection of the GHS should follow the following procedure:
# '''Flush supply lines:''' Gas lines from outside bottles going to the GHS should be briefly (5 min or so) flushed with a high gas flow ''before'' connecting to the GHS to remove possible dust, and to minimize the amount of air in the lines.
# '''Flush GHS:'''
## Close BV1 and BV3, open BV2 and BV4 to flush input lines, only briefly.
## Open BV1 and BV3, close BV2 and BV4 (normal operation configuration)
## With Midas frontend running, use the "Gas" custom page to set a desired flow, for flushing probably between 1l/min and 1.5l/min (Without Midas, use the telnet interface), then press the "Bypass Flow" button, and let flow for a while (5-15min?)
# '''Flush TPC input line:'''
## Connect the line that will go from the GHS to the TPC input-tee on the GHS side, but leave disconnected from the tee
## do the same for the pressure gauge line
## On the "Gas" custom page, press the "Flow TPC" button and let flow for 5 min (Without Midas, use the telnet interface).
# Make all remaining connections

== Troubleshooting ==
''' No flow '''

Make sure that the black valves on the gas rack at the entrance of the zone are open (right on the pictures).

[[File:20190109 165410.jpg|thumb|The rightmost valve (black) must be open during operation.]]

== Telnet Interface ==
If the Midas frontend is not available the GHS can be controlled via telnet. Connect with
telnet algas
then use the following command to control solenoid valves:
cowr do i 1
to energize SVi, i.e. to switch it from its normal closed or open state to the other state,
cowr do i 0
to de-energize, i.e. switch to default state.

Gas flow is set with:
mfcwr ao i <desired flow>
where i is 1 for Ar and 2 for CO2, and the desired flow is in the internal units of the system.

= Water cooling system =

The outer surface of the detector has an arrangement of 8x8 electronics boards handling the readout of the cathode pads. These boards require operating at a controlled temperature to prevent heat build-up at the surface of the detector and improving the reliability of the system. In order to do so, 8 independent water pipe loops running from the bottom-up and down in the longitudinal axis of the detector are in a thermal contact to 8 successive readout boards. From the bottom, a pair of water hoses connect to 2 custom 3D-printed manifolds supplying the water feed and return to the 8 water loops.

These 8 loops are run in parallel to a negative pressure water cooling system. This negative pressure system is to ensure that no water leak will damage the electronics even during possible pipe or delivery hose breakage.

[[File:imageservice5.png|516x273px]]

Figure 4 - Cathode pad half-cylinder. Pairs of cooling pipes are visible

[[File:imageservice6.jpg|249x429px]]

Figure 5 - Water cooling system during test.

== Specifications ==

{|
! '''''Parameters'''''
! '''''Values'''''
|-
| Number of parallel water loops
| 8
|-
| Copper pipe OD/ID
| ¼” / 4.7mm
|-
| Length of one loop (on the rTPC)
| 4.6m
|-
| Water speed per loop
| …
|-
| Pressure drop per loop
| …
|-
| Power extraction per water loop
| 8x15W= 120W
|-
| Total cooling power
| 1KW
|-
| Inlet water temperature
| 16..20 degC (to be confirmed)
|-
| Outlet water temperature
| 20..24 degC
|-
| Total water flow
| 3..6 l/minute
|}

== Operation ==

''This description corresponds to how we think the system should work, but doesn't seem fully comatible with the behaviour of the system. E.g. the air pump only seems to turn on when PID1 falls below a certain value.''

The Pressure PID (proportional/integral/differential controller), PID2, monitors the air pressure in the vacuum vessel. When the pressure rises above a trigger value (perhaps 350mbar) it activates the aspirator vacuum pump to bring the pressure down to maintain a long time averaged pressure (say 300mbar).

This vacuum is then used to draw water through the detector into the vessel, from the Open Tank. The vacuum vessel's water level is monitored by PID1, using a differential pressure transducer measuring the weight of the water column by the difference between the air pressure in the top of the vessel and the water pressure at the bottom of the vessel. PID1 provides a variable signal to the Proportional Solenoid Valve, which varies the flow through the return loop around the (fixed flow) water pump, and thus changing the overall flow of water from the vacuum vessel to keep the water level in the vessel constant despite water being constantly drawn in by the vacuum. The water pulled from the vacuum vessel is pushed through the filter and into the Open Tank, completing the circulation.

Cooling is accomplished in the water pump leg of the system, as coolant pulled from the vacuum vessel passes through a heat exchanger coupled to CERN’s cooling water supply on its way to the pump. The rate that water flows through the system can be determined by many obstacles in the loop, but ideally it should be dominated by the position of the main Throttle Valve. Other valves should generally be set either fully open of fully closed, except for the Proportional Valve, whose position responds to the setting of the Throttle Valve.

Data outputs: there are five data outputs planned. The two process variables, the vacuum vessel absolute pressure, and differential water column pressure, will be sent to the Detector DAQ system. As well, the total flow rate is monitored by the flow meter, whose output is sent to the DAQ system. This data is not used by the circulation system itself. Likewise, the back pressure on the water filter is only sent to the DAQ system to indicate if the filter may need changing, and the temperature of the open tank is monitored and sent to the DAQ system, but currently there is no plan to use it in a temperature control feedback. The temperature of CERN's cooling water supply is not yet known, but we expect we can set the heat exchanger to run at a rate to hold the system temperature near 20°C without feedback control.

For the negative pressure to work effectively to prevent leaks, the water pressure in the detector ought to be down at least to half an atmosphere. With the detector cooling tubes 8x 4.6m of parallel 4.7mm ID copper tubes (2.3m down then 2.3 back-ups), and the supply using (perhaps 6m, 3 to the detector then 3 back) 9.7mm ID (=1/2" OD) poly tubing, the flow will need to be somewhat greater than the required 3 l/min in order to maintain that degree of vacuum in the detector, unless the Throttle Valve is placed upstream of the detector near the Open Tank, or an additional constriction is added in that region.

The water pump leg of the circuit residing in the water cooling/gas rack is placed far enough from the fringe field of the superconducting magnet to prevent disruptive effects on the solenoid valves and electric motors. Therefore the main 1/2" poly line to and from the water pump and its flow loop will add a pressure drop which is estimated at 0.4atm for a 10m line at 6 l/min (9.7mmID), which is within our requirements. As mentioned earlier, the splitting (and reassembly) of the line to the 8 cooling lines close to detector will be accomplished with the use of 2 separate manifolds equipped with temperature sensors for constant in/out global water temperature monitoring.

[[File:imageservice7.png|286x206px]][[File:imageservice8.jpg|286x206px]]

Figure 6 - Model and 3D-printed realization of the dry-air manifold, very similar to the manifolds for the water cooling system.

Currently, PVC valves with "true union" couplings are planned to be provided around the water pump to permit it to be swapped out from the system for servicing and a valved bypass to the water filter is provided for the same purpose to allow easily changing filter elements with the system running. True union valves are generally only available to 1/2NPT and larger, so an adapter fitting is required. The vacuum pump for the system is a small (2 x 5 x 7.5cm) cascaded dual Venturi aspirator, which is operated with site air, switched with two simultaneously operated normally closed solenoid valves, one for the site air, and one for the vessel vacuum input. If the system is leak free, the vacuum in the vessel should only be degraded by the dissolved air absorbed in the Open Tank and extracted in the vacuum vessel, which from past experience might mean running for under a minute every six hours or so.

The wetted surfaces of this system will be mainly PVC and Polyethylene for the valves, tubing and vessels, plus brass fittings, and the copper cooling tubes, and the surfaces of the heat exchanger, which will most likely be stainless steel. It may be possible to devise an equivalent of a bellows communicating with the Open Tank, so that it is only virtually open, thus sealing the cooling water from contaminants, which may allow us to run with just distilled water plus a strip of silver as a biocide (although I'm not sure if the mix of metals, copper plus stainless, may not interfere with the function of the silver, which is mostly used in computer cooling systems which are copper and plastic only). Otherwise, we will need a biocide and corrosion inhibitor. We have the latter, tolyltriazole, but the biocide is problematic. They tend to be short-lived; for systems which are open to atmosphere, typical industrial practice is chlorination, regularly refreshed, which builds up NaCl as it degrades, making the coolant corrosive. For T2K in Japan, we used the (human-nontoxic) "Germall", which has a two-year lifetime, after which when its effectiveness decays it turns into food, which is the case for any solute that is not actively biocidal. Eventually, any such additives build up and require the coolant to be flushed and replaced. Silver is a lovely method, but only works for extended periods with a sealed system which remains deoxygenated. My research so far has not turned up a good candidate which does not entail recurring changes of the coolant.

Power: the system needs to be able to run on European 220VAC power, but also operate here for testing purposes. The PIDs (Omega CN76160s) will accept both; for the rest of the system, I will have an a universal 90-240VAC input to 24VDC (5A) supply which will drive all the other components, including the pump, which will be a brushless 24VDC centrifugal pump, 1 - 1.5A range. The whole system will run about 125W.

Information on the temperature and flow rate of the CERN cooling water system will be provided later but we don’t anticipate any major issues about the sizing of the heat exchanger.

[[File:imageservice9.png|603x344px]]

Figure 7 - Overall water cooling system



== Temperature Measurement ==

[[File:imageservice10.jpeg|225x356px]] [[File:imageservice11.png|194x358px]]

Figure 8 Temperature Monitor Board

The water temperature is read by the Temperature Monitor Board (TEMPB). Thermistors located inside the manifold connect to a 16 bit ADC on the TEMPB which is read out by a Raspberry Pi mounted on board. In a similar manner the air system temperature is read with sensors in the air manifold. The TEMPB can provide temperature readout of 32 thermistors which connect to terminal blocks. The channels can also be readout differentially providing up to 16 channels for thermocouples. An on board thermistor allows for cold junction compensation. The board is powered via USB on the Raspberry Pi.

== Quick Start ==
This assumes all water lines and compressed air are connected, and both the blue water tank and the chiller are filled.

=== Begin running ===
# Close vent valve and main valve (3-way valve in T position)
# Switch on control box with power switch in the back
# confirm water pump is off, i.e. blue button is not illuminated
# Vacuum pump should now be running
# Open main valve, i.e. vertical connection open, connection to vent valve closed [[File:PWBmainOpen.jpg|thumb|Cooling system main valve open]]
# water should start flowing through the PWB cooling system and coming back through the clear pipe connected to the vacuum vessel
# Default settings are PID1 = 200, PID2 = 300, but may have been adjusted
# once the water level in the vacuum vessel has risen to ~20cm, turn on the water pump

=== Drain system ===
# from normal operation, i.e. vacuum pump and water pump running
# switch main valve to upside down T position
[[File:PWBmainVent.jpg|thumb|Cooling system main valve to vent valve]]
# open vent valve
# air should now enter the system, draining all water from the TPC pipes into the vacuum vessel
# water pump will drain the vacuum vessel into supply tank, turn it off before the vessel is completely empty to avoid air in the water pump
# switch off main power

== Troubleshooting ==
The water system is temperamental, and not intuitive at all. Common problems and solution attempts:

; water pump very loud, doesn't appear to pump water
: most likely air in the water pump. Open the vent of the vacuum vessel to allow water to enter pump by gravity alone, pump should stop screaming and start pumping. Close vent.
; vacuum pump comes on often (more then every 10-20 min)
: probably an air leak in the system. Check for bubbles and try to find leak.
; water level/flow/vacuum oscillate between extreme values
: unclear, try playing with PID settings, throttle valve, possibly orange ball valves by the water pump.

= Air Cooling =

This section describes the cooling system for the BSC.

== Overview ==

The dry air flow in the BSC sleeve aims at reducing the risk of condensation due to the temperature gradient caused by the hotter PWB.
The constant flow has also affords a minimal control over the overall temperature of the BSC.

The dry cold air in the SiPM enclosure is intended to set a fixed temperature for the SiPM operation and to cool 'Analog Sum Discriminator' modules or 'ASD'.

[[File:Aircooling.svg|Schematic view of the distribution of cold air to the SiPM and the BSC sleeve.|292x523px]]

== Implementation ==

Air cooling scheme mediated by heat exchangers, fueled by the water chiller is shown below.

[[File:Aircooling2.svg|Schematic view of the cooling circuit at the service rack]]

Note that the circuit to cool the SiPM is going to be replaced in 2021 by 'Vortex cooler'.

= Power distribution =

This section covers the different voltages and power requirements for the operation of the rTPC. This includes:

# High Voltage for the rTPC electron drift and amplification operation
# Low Voltage to the Anode wire frontend electronics
# Low Voltage to the Barrel Scintillator frontend electronics
# Low Voltage to the Cathode Pads electronics
# Grounding scheme of the overall rTPC

The provision of power to the different components of the detector is distributed from the top end plate of the rTPC.

[[File:imageservice12.png|507x354px]]

Figure 9 - End plate view of the layout of the different electronics boards & HV connections

== High Voltage for the rTPC ==

Three main components of the rTPC require High Voltage

* Main cathode wall (Negative ~5KV)
* Field wires (Negative ~ few hundred volts)
* Anode wires (Positive ~3.2KV )

These voltages are provided by a Quad High Voltage unit from CAEN. Three independent cables will be fed to the top of the rTPC.

High Voltage provision to the bottom endplate HV ring is taken from the central cathode near the bottom endplate.

The Field wire voltage is connected to all the field wire crimp-pins through a dedicated socket chain inserted on each of the crimp-pin on the top and bottom endplates. The field wires themselves interconnect the high voltage between the top and bottom pins.

The High Voltage for the anode wires is delivered to the Anode Wire Card (AWC) which will distribute it to its 16 wires, Figure 11. There are 16 AWC cards covering each end of the anode wires. All the top and bottom 32 AWC are connected serially. The anode wires themselves interconnect the high voltage between the top and bottom pins.

The anode wires pre-amplifiers boards (AWB) are sitting on the top of the Anode Wire Card (AWC) Figure 10. As the wire is powered by the HV, each anode wire has a high Voltage decoupling capacitor to isolate the pre-amp stage from the HV. This circuit resides on the AWC. The AWB will see the AC coupled anode signal only. With a new design of the AWC Figure 12, the decoupling capacitors are potted in epoxy to ensure proper isolation of the HV toward the surrounding circuit as well to prevent corona discharge due to relative humidity of the surrounding.

[[File:imageservice13.jpg|402x309px]]

Figure 10 – Unpotted Anode Wire Card and Board with HV distribution

[[File:imageservice14.jpg|402x309px]]

Figure 11 - Anode wire pins and Field wire pins under yellow cover

[[File:imageservice15.jpg|402x309px]]

Figure 12 – New Anode Wire card with 16 potted HV capacitors, gold crimp pin Mill-max socket to the anode wire

== Power distribution to the Anode wire frontend electronics ==

The AWB pre-amp board requires 2 voltages (+5V, -5V) which are fed through dedicated cable connections to each of the AWB. There are 16 AWB per end of the rTPC, therefore, 16 individual cable bundle will be reaching the top and another 16 the bottom of the rTPC. The cable bundle is to also provide the power return lines to the Power Supply.

The PL508 Low Voltage power Supply from Wiener will provide the main power source for all the 32 pre-amplifiers. A dedicated connection breakout board with current monitoring capability splits the source power to the 2 x 16 necessary connections to the Anode wire boards. Based on a RaspeberryPi card, a 16 power distribution board is already implemented for the prototype. This board has extra 4 channels of temperature monitoring which can be used later on. In the same process we produce a similar board for 16 channel of NTC, RTC temperature monitoring. They will be used for overall detector and cooling system monitoring.

{|
! '''''Component'''''
! '''''+5V Supply'''''
! '''''-5V Supply'''''
|-
| Anode Pre-amp
| 1.5W (0.3A)
| 1.5W (0.3A)
|}

[[File:imageservice16.jpg|239x363px]] [[File:imageservice17.png|239x363px]]

Figure 13 - Anode wire pre-amp power distribution & 16 channel temperature boards

== Power distribution to the Barrel Scintillator frontend electronics ==

Similar to the Anode wire pre-amplifier voltage distribution scheme, the Barrel Scintillators will require several low voltages for power and control. The Barrel Scintillator Boards (BSB) are mounted on both ends of the Scintillator bar with the same segmentation as the Anode wires i.e.: 16 BSB per end. The design of the electronic chain for the barrel scintillator is not fully designed yet.

The SiPM sensors require a bias voltage of the order of 28V. This voltage will be provided through the same mean as the low voltage power. Expected threshold voltage for the local signal level discriminator is also foreseen.

== Low Voltage to the Cathode Pads electronics ==

The cathode pads electronics mounted on the external cylinder surface is composed of the analog frontend and the digital conversion electronics. Two voltages are required (+5.3V, +2.3V). The delivery is done through dedicated “power bar” running the full length of the rTPC. A set of 3 bars covers the 8 axial cathode pad electronics boards. 8 of them cover the circumference of the rTPC.

{|
! '''''Component'''''
! '''''+5.3V Supply'''''
! '''''+2.3V Supply'''''
|-
| Padwing board
| 10W (1.8A)
| 2W (0.8A)
|}

Overall the low power supplies are provided by a PL508-3U with 8 modules of either single or dual channels from Wiener.

[[File:imageservice18.png|469x360px]]

Figure 14 - Overall Low voltage distribution for anode wires and cathode pads



== Grounding scheme of the overall rTPC ==

The grounding scheme challenge in this setup is the far distance between the 2 end of the rTPC (2.3m) and the lack of accessibility to the bottom of the detector. The proposal is to use a Copper sheet mounted on the inner surface of the inner cylinder. This ground path will connect the 2 endplates to the High Voltage return. All the pre-amps will merge their ground to this central ground.

Additional ground lugs have been included in the Anode wire pre-amplifier boards to strengthen the ground as each anode wire is readout out from both end. This may be used once the full detector is assembled to address possible ground loop issues.

[[File:imageservice19.png|439x409px]]

= Safety and Hazard considerations =

== Gas ==

The Ar and CO2 do not require special health safety system. The detector will be operated in the ventilated hall of the AD building at CERN.

Gas flow, detector gas pressure, and gas inlet temperature are constantly monitored and can trigger adequate response for shutdown or closure of the gas system for safety purpose.

== Cooling water ==

The water cooling system operates in negative pressure mode preventing any water leak to flood the detector outer surface. In addition water flow and inlet/outlet temperature are constantly monitored to allow fast response to off range parameter in order to shut down the electronics. The cooling water system requires a primary cooling tap water loop from the AD building. Water quality, water flow and water temperature are not under our responsibility. This aspect will be dealt during the initial phase of the rTPC commissioning at CERN.

== Power distribution ==

All electronics voltages are below 48V. No special safety circuits are envisaged.

The high-voltage circuit is fully protected outside the detector. The access to the high-voltage on the detector is restricted to the two end-plates. Interlock scheme will be put in place to ensure the shutdown of the HV in case of manual access to the detector itself.

The LV (Low Voltage) power units are also interlocked to prevent false powering sequence which can damage the downstream electronics modules.

[[File:imageservice20.png|571x411px]]

Figure 16- Alpha-g rTPC rack organization

File:Aircooling2.svg

2021-04-10T04:07:06Z

Acapra:

Cooling schematics

File:Aircooling.svg

2021-04-10T03:32:43Z

Acapra:

Schematic view of the air distribution.

Daq

2021-02-09T23:23:00Z

Acapra: /* Schematic */

= Links =

* https://alphacpc05.cern.ch/agdaq/ -- midas daq page for the agdaq system at CERN
* https://daq16.triumf.ca -- midas daq page for the agmini system at TRIUMF
* https://daq.triumf.ca/elog-alphag/alphag/ -- elog
* https://daqstore.triumf.ca/AgWiki/index.php/Main_Page -- this wiki
* https://bitbucket.org/ttriumfdaq/agdaq - git repository for agdaq
* https://bitbucket.org/expalpha -- git repository on bitbucket
* https://edev-group.triumf.ca/fw/exp/alphag -- git repository on edev gitlab
* https://ladd00.triumf.ca/daqinv/frontend/list/170 - PWB inventory database
* https://ladd00.triumf.ca/daqinv/frontend/list/168 - ADC (GRIF-16) inventory, FMC-ADC32rev0,rev1 inventory
* https://ladd00.triumf.ca/daqinv/frontend/list/176 - FMC-ADC32rev1.1 inventory

= Hardware manuals =

* [[TRG]] - trigger board (GRIF-C) manual
* CDM - https://daq.triumf.ca/DaqWiki/index.php/VME-CDM -- clock VME-CDM manual
* ADC - https://daq.triumf.ca/DaqWiki/index.php/VME-GRIF-ADC16-Rev1 -- GRIF-16 ADC manual
* [[PWB]] - TPC Pad Wing Board manual
* TDC - https://www.triumf.info/wiki/DAQwiki/index.php/GSI_TRB3
* [[chronobox]] - chronobox manual

= Performance =

Full system: (ADC fw rel-20201112-ko, PWB fw 20201005_ko)
* 1 Hz stable, ADC adc32 9 supp off rate 0.3 M/s, adc16 off, PWB udp_delay 25 supp off active 64 rate 6.6+4+6+4.6=21.2 M/s, TDC off, total rate 21 M/s
* 500 Hz stable, ADC adc32 9 supp 1500 rate 6 M/s, adc16 off, PWB udp_delay 10 supp 100 active 63 rate 3+4+3+3=13 M/s, TDC off, total rate 19 M/s

Single ADC: (fw rel-20201112-ko)

Fully suppressed data, sthreshold 5000
* 1 Hz: stable
* 500 Hz: stable (nominal max trigger rate)
* 6700 Hz: stable, 8 Mbytes/sec
* 6800 Hz: evb eventually confused
* 6850 Hz: evb confused, no lost triggers in the ADC, no crash in the ADC

All data, 510 samples/event, sthreshold 10 (fw rel-20201122-ko)
* 1 Hz: stable
* 100 Hz: stable, 3.4 Mbytes/sec
* 500 Hz: stable, 17 Mbytes/sec (nominal max trigger rate)
* 1000 Hz: stable, 32 Mbytes/sec
* 3100 Hz: stable, 106 M/s (approaching saturation of 1gige ethernet)
* 3200 Hz: stable, 110 M/s - TRG communication is in trouble
* 3300 Hz: stable, 116 M/s
* 3400 Hz: incomplete events, no ADC crash ("dropped due to full" counting, "dropped due to busy" are zero, this is as expected).

no suppression (different fpga data path)
* 1 Hz: stable
* 100 Hz: stable
* 500 Hz: stable, 17 M/s (nominal max trigger rate)
* 1000 Hz: stable, 32 M/s
* 3000 Hz: stable, 100 M/s (approaching saturation of 1gige ethernet)
* 3300 Hz: incomplete events, no ADC crash ("dropped due to full" counting, "dropped due to busy" are zero, this is as expected).

= Network connections =

The experiment private network number is: 192.168.1.x, gateway 192.168.1.1, netmask 255.255.255.0

network map:
<pre>
---- TRIUMF/CERN network --1gige-- alphagdaq --10gige-- juniper switch --1gige-- centrecom switch
</pre>

== alphagdaq network connections ==

on the rear of the machine:
<pre>
-----------------------------------------
| | rj45 | rj45 | | sfp | sfp |
-----------------------------------------

- left rj45 - copper 1gige - eno1 - spare (inactive)
- right rj45 - copper 1gige - eno2 - dhcp - cern/triumf network
- left sfp - 10gige - enp1s0f1 - spare (inactive)
- right sfp - 10gige - enp1s0f0 - static 192.168.1.1
</pre>

== connections between switches ==

* alphagdaq to juniper switch: direct attach (DAC) 10gige cable into any port of juniper switch
* juniper switch to centrecom switch: rj45 sfp in any port of juniper switch - cat5e/cat6 cable - to any port of centrecom switch
* cat5e/cat6 cable from any port of centrecom switch to the XXX port on the back of the juniper switch

== juniper switch ==

connected to juniper switch front is:

* everything that sends event data to alphagdaq, specifically:
** TRG (rj45 sfp)
** 16x ADC (fiber sfp)
** 64x PWB (fiber sfp)
* 10gige uplink to alphagdaq (10gige DAC cable)
* 1gige link to centrecom switch (rj45 sfp)
* 40gige ports are not used

connected to the juniper switch rear:

* CON is the serial console via RJ45-to-DB9 to USB-serial adapter to alphagdaq USB port (9600bps, 8N1, no flow control)
* C0 is the management ethernet interface port, connect to any port on the centrecom switch (juniper-private will ping)

== centrecom switch ==

* everything with copper rj45 connections, specifically:
** CDM boards
** HV, LV and VME power supplies
** RaspberryPi3 boards
** gas handling MFCs (algas)
** cooling system controller (moxa01)

= USB connections =

Note:
* alphagdaq has USB2 and USB3 connections. They are not the same, they are not generally interchangeable.
* USB3 connections and cables are generally blue coloured.

List of USB connections from alphagdaq:

* front USB3 port (blue): USB-A-to-B cable to USB blaster connected to the trigger board (front of rack)
* rear USB3 port (blue): USB3 short jumper to USB3 hub
* rear USB2 port (black): USB2 unpowered hub (Logiix) to USB-A-to-B cable to UPS USB connector
* rear USB2 port (black): USB-to-Serial adaper to DB9 to RJ45 adapter to Juniper switch console port

Note: UPS USB connection does not work with alphagdaq USB ports (both usb2 and usb3), but combination
using the unpowered USB2 hub seems to work ok.

List of USB connections from the USB hub:

* this is a powered USB3 hub
* USB-A-to-MicroUSB - to lvdb boards
* USB-A-to-Wiener interlock cable

= Clock and trigger distribution =

== Explanation ==

Trigger is generated by the trigger board (aka TRG, aka GRIF-C),
from the TRG eSATA output through the eSATA splitter it is fed into
the master CDM. The master CDM outputs the trigger on all MiniSAS output ports.
One of the master CDM MiniSAS outputs is connected to the CDM slave eSATA input.
The same as the master CDM, the slave CDM outputs the trigger on all MiniSAS output ports.

The master 62.5MHz clock is generated by the master CDM using the internal 10MHz oscillator
or the external 10MHz AD clock. Also on MiniSAS port 6 the master CDM generates
the 200MHz clock for the TDC. This clock is frequency-locked to the master 62.5MHz clock.

Master CDM clock selection (external CD 10MHz clock or internal oscillator 10MHz clock)
is done by a magic esper command (see below).

From the master CDM, the clock is fed to: the slave CDM, the TRG (via the eSATA splitter)
and to the TDC (via the RJ45 splitter).

The slave CDM sends the clock and the trigger to the ADCs and PWBs.

== Schematic ==

<pre>
TRG (GRIF-C)
------------
eSATA <------> eSATA splitter

eSATA splitter
--------------
trigger ---> eSATA ---> CDM-Master eSATA (trigger signal)
clock <--- (eSATA --- MiniSAS) <--- CDM MiniSAS (62.5MHz clock)
</pre>

<pre>
CDM-Master
----------
eSATA <--- eSATA splitter <--- eSATA from TRG (trigger signal)
MiniSAS 1 --> (MiniSAS --- eSATA) --> CDM-Slave eSATA (62.5 MHz)
MiniSAS 6 --> (MiniSAS --- eSATA) --> eSATA splitter --> RJ45 --> TDC (200 MHz)
</pre>

<pre>
CDM-Slave
---------
eSATA <--- trigger and 62.5MHz clock from CDM-Master
MiniSAS 1 --> ADC trigger and 62.5MHz clock
MiniSAS 2 --> same
MiniSAS 3 --> same
MiniSAS 4 --> same
MiniSAS 5 --> PWB trigger and 62.5MHz clock
MiniSAS 6 --> same
</pre>

[[File:VME_Crate_CERN.jpeg|800x450px|Layout of the VME crate at CERN.]]

Layout of the VME crate at CERN. The ALPHA-T module is in the rightmost slot. The Master CDM is next to it and it's labelled as #02. The Slave CDM is on the lefmost position and labelled by #03.

== Setup of CDM boards ==

ALPHA-g uses the Rev3 CDM boards. The 4 CDM boards that belong to ALPHA-g
have serial numbers 0, 1, 2 and 3 and are labeled cdm00, cdm01, cdm02, cdm03.

All the boards can be used as master and slave positions, but have to be
configured appropriately.

=== Slave setup ===

* install the board
* connect ethernet
* connect esata external clock
* esper-tool cdmNN ### i.e. esper-tool cdm01
* cd template
* write current_setup 4
* cd /cdm
* read
* observe clock frequency counters report correct values. "ext_clk" will only count if the lemo jumper is installed (see below)
<pre>
11 esata_clk uint32 R [62500246]
12 ext_clk uint32 R [62500246]
</pre>
* cd /lmk
* read
* observe both PLL1 and PLL2 are locked: both values should be "1"
<pre>
144 pll1_ld uint8 R [1]
147 pll2_ld uint8 R [1]
156 ld1_counter uint32 RW [3]
157 ld2_counter uint32 RW [3]
</pre>

If needed, check correct operation of the pll lock monitoring:

* unplug the esata cable from the master CDM
* "red" light should go off
* read of lmk should report "pll1_ld" value 0 and "ld1_counter" and "ld2_counter" should increment
* reconnect the esata cable
* "red" light should return
* read of lmk should report both pll1_ld and pll2_ld locked (values "1").

=== Master setup ===

* install the board
* connect ethernet
* esper-tool cdmNN ### i.e. esper-tool cdm01
* cd template
* write current_setup 3
* cd /lmk
* write clkin_sel_mode 0
* cd /cdm
* read
* observe clock frequency counters report correct values. "ext_clk" will only count if the "clock loopback" lemo jumper is installed (see below)
<pre>
12 ext_clk uint32 R [62500189]
</pre>
* cd /lmk
* read
* observe both PLL1 and PLL2 are locked: both values should be "1"
<pre>
144 pll1_ld uint8 R [1]
147 pll2_ld uint8 R [1]
156 ld1_counter uint32 RW [3]
157 ld2_counter uint32 RW [3]
</pre>

Check the 200MHz clock:
* connect MiniSAS-eSATA splitter cable to MiniSAS output 6, connect eSATA end to CDM eSATA input
* in esper-tool, read cdm: esata_clk should report 200MHz clock
<pre>
11 esata_clk uint32 R [200000603]
12 ext_clk uint32 R [62500188]
</pre>

=== Master setup with external 10 MHz clock ===

* setup master CDM as above (using internal clock)
* connect 10MHz external clock to right LEMO #1
* esper-tool cdmNN
* cd /cdm
* write sel_nim 1 # if clock is NIM signal
* write sel_nim 0 # if clock is TTL signal (i.e. chronobox LEMO output)
* read ext_clk
<pre>
[http://cdm00:/cdm]> read ext_clk
9999556
</pre>
* if ext_clk reads 0 or does not read something close to 10MHz, stop here. debug the signal, try changing sel_nim, etc.
* cd /lmk
* write clkin_sel_mode 2 # select external clock
* read
* observe both PLL1 and PLL2 are locked: both values should be "1"
<pre>
150 clkin2_sel bool R [True]
144 pll1_ld uint8 R [1]
147 pll2_ld uint8 R [1]
</pre>
* on the MIDAS TRG page, observe that TRG measured external clock frequency is close to 62.5 MHz

== clkin_sel_mode positions ==

* 0 = 10MHz internal oscillator (use on master CDM in standalone mode)
* 1 = eSATA clock (use on slave CDM)
* 2 = external clock, should be 10MHz (use on master CDM to run from AD clock)

== LEMO connections ==

* clock loopback jumper: to read the frequency of the generated 62.5MHz clock, connect LEMO cable from left LEMO #2 to the right LEMO #1
* 10MHz internal clock jumper: to feed 10MHz internal oscillator to the external clock, connect LEMO cable from right LEMO #3 to the right LEMO #1.
* 10MHz external clock: to feed 10MHz external clock, connect LEMO cable to right LEMO #1

= MIDAS frontends =

== UDP ==

feudp receives udp packets from the ADC and PWB boards. For each udp packet 1 midas event is created,
with 1 data bank containing the udp data. Events are sent into the MIDAS event buffer BUFUDP. The names
of the data banks are assigned in ODB /eq/feudp/settings.

{ADC,PWB} --> 1gige --> switch --> 10gige --> alphagdaq --> feudp -> BUFUDP

== CTRL ==

fectrl controls all the ADC, PWB and trigger boards. It provides run start and stop sequencing,
runs the event synchronization sequence at the begin of the run, and records slow controls data (temperatures,
voltages, etc).

fectrl receives UDP data packets from the trigger board and sends them to the MIDAS event buffer BUFUDP.

fectrl receives commands from the control web pages (for ADC, PWB and trigger) to reboot, initialize, etc the boards.

fectrl configures the event builder via odb /eq/fectrl/evbconfig.

ADC <-> http esper <-> fectrl -> slow control data into midas history

PWB <-> http esper <-> fectrl -> slow control data into midas history

TRG <-> udp comm <-> fectrl -> BUFUDP, slow control and counters into midas history

fectrl <-> midas rpc <-> mhttpd <-> json rpc <-> control web pages for ADC, PWB and trigger

== EVB ==

feevb receives udp packet data from the MIDAS event buffer BUFUDP, inspects the packet timestamps
and collects the data with matching timestamps into physics events. feevb has provisions to do
data suppression, reduction and compression in addition to the data reduction done
in the ADC and PWB firmware.

feevb event synchronization relies on the event synchronization sequence done by fectrl when a run is started.

{ADC, PWB, TRG} -> BUFUDP -> feevb -> SYSTEM -> mlogger -> compression -> disk storage

=== internal structure ===

* event buffer readers, instances for BUFTRG, BUFTDC, BUFADC, BUFUDP
<pre>
BufferReader class - reads one midas event buffer:
- BufferReaderThread()
-- in batches of 1000 events
--- lock
--- TickLocked()
----- ReadEvent() -> bm_receive_event_alloc()
----- fHandler->HandleEvent()
-- fHandler->XMaybeFlushBank()
-- sleep 100 usec
</pre>
* event header decoder and intermediate buffer
<pre>
Handler class - accumulates events, sends them to the EVB in batches

- HandleEvent() <- BufferReaderThread() - receive incoming events
-- loop over all banks, examine bank name
--- call AddAdcBank(), AddPwbBank(), AddTrgBank(), AddTdcBank() according to bank name
---- AddXxxBank() -> XAddBank()
--- unknown banks go directly to evb->SendQueue
-- check if synchronization has completed

- XAddBank() - save BankBuf objects in a buffer

- XMaybeFlushBank(), XFlushBank() - send all accumulated data to the Evb class
-- evb->lock()
-- for each buffered BankBuf objects,
--- call evb->AddBankLocked()
-- buffer is now empty
</pre>
* event builder
<pre>
Evb class - assemble event fragments according to timestamps

- fEvents fifo (deque) - time sorted list of partially assembled events
-- new entries are added by FindEvent()
-- entries are modified by MergeSlot()
-- old entries are removed by GetLocked()

- AddBankLocked() <- XFlushBank() <- BufferReaderThread()
-- call fSync->Add()
-- transform BankBuf to EvbEventBuf
-- push it into per-module buffer fBuf[imodule]

- EvbThread()
-- EvbTickLocked()
--- BuildLocked()
---- in a loop, for no more than 1 sec
----- loop over all per-module buffers fBuf
------ pop first entry, pass it to BuildSlot()
------- call FindEvent()
------- call MergeSlot()
------- call CheckEvent()
--- in a loop
---- get all completed events via GetLocked() pass them to SendQueue->PushEvent()
-- if done nothing, sleep 100 usec, otherwise nominal sleep to free the evb lock

- GetNext() <- GetLocked() <- EvbThread() - get completed events
-- examine oldest EvbEvent entry in fEvents:
-- pop it if it is complete
-- pop it if it is too old (to prevent old incomplete events from stalling the evb) ("pop age" on the evb page)
-- pop it if a following event is complete ("pop following" on the evb page)
</pre>
* send queue
<pre>
SendQueue class - prepare events for writing to midas SYSTEM event buffer

- PushEvent() <- EvbThread()
-- push FragmentBuf into the queue

- SendQueueThread()
-- in a batch of 1000 events, call Tick()
--- pop FragmentBuf from the queue, call SendEvent()
--- compose midas event (memcpy()!)
--- TMFE fEq->SendEvent()
--- bm_send_event()
-- sleep 100 usec
</pre>
* equipment object
<pre>
EvbEq class - implements the midas equipment
- HandleBeginRun()
- HandleEndRun()
- HandlePeriodic()
</pre>

=== multithreading ===

* BufferReaderThread() one per midas event buffer
** high CPU use: bm_receive_event(), AddXxxBank()
** lock contention: midas event buffer lock, evb lock into XFlushBank->AddBankLocked()
* EvbThread()
** O(size of fEvents): FindEvent(), GetNext()
** lock contention: evb lock against XFlushBank(), SendQueue lock (1 per output event)
* SendQueueThread()
** high CPU use: memcpy() in SendEvent(), memcpy() in bm_send_event
** lock contention: event queue lock against EvbThread, midas event buffer lock
* main thread
** lock contention: evb lock in ReportEvb() and elsewhere

= ODB entries =

== /Equipment/CTRL/Settings ==
* TBW

== /Equipment/CTRL/Settings/TrigSrc - trigger source selection ==
* TrigPulser - trigger on the pulser
* TrigEsataNimGrandOr - trigger on external NIM inputs of GRIF-16 ADC modules
* TrigAdc16GrandOr - trigger on any adc16 signals (see adc16_masks)
* TrigAdc32GrandOr - trigger on any adc32 signals (see adc32_masks)
* Trig1ormore - trigger on adc16 multiplicity 1 or more
* Trig2ormore - ditto, 2 or more
* Trig3ormore - ditto, 3 or more
* Trig4ormore - ditto, 4 or more
* TrigAdcGrandOr - trigger on any ADC signal
* TrigAwCoincA - trigger on TPC AW coincidence (see [[TRG]] manual)
* TrigAwCoincB - ditto
* TrigAwCoincC - ditto
* TrigAwCoincD - ditto
* TrigAwCoinc - ditto
* TrigAw1ormore - trigger on TPC AW per-preamp multiplicity 1 or more (see [[TRG]] manual)
* TrigAw2ormore - ditto, 2 or more
* TrigAw3ormore - ditto, 3 or more
* TrigAw4ormore - ditto, 4 or more
* TrigAwMLU - trigger on TPC AW MLU signal (see [[TRG]] manual)
* TrigBscGrandOr - trigger on any BSC signal
* TrigBscMult - trigger on predefined BSC multiplicity (see [[TRG]] manual)
* TrigCoinc - trigger on predefined coincidence of TPC AW, BSC and external signal (see [[TRG]] manual)

== /Equipment/CTRL/Settings/TRG - trigger settings ==
* Enable - if set to "n", fectrl will not do anything with the trigger board. Normal value "y"
* Modules - hostnames of the trigger board. Normal value "alphat01"
* NimMask - 32-bit mask for the 32x NIM inputs (see TRG manual). Normal value 0, unless external trigger is in use.
* EsataMask - 32-bit mask for the 32x eSATA signals (see TRG manual). Normal value 0.
* adc16_mask[16] - 16x 16-bit mask for the 100MHz GRIF-16 onboard ADCs (see TRG manual). Normal value 0xFFFF for all 16 data links.
* adc32_mask[16] - 16x 32-bit mask for the 62.5MHz FMC-ADC32 ADCs (see TRG manual). Normal value 0xFFFFFFFF for all 16 data links.
* PassThrough - if set to "y", trigger is passed through the trigger module without generating any events and without causing deadtime.
* AwCoincA, B, C and D - 32-bit masks for the aw16 coincidence trigger (see TRG manual)
* Aw16FromAdc16 - route AW trigger (aw16) from adc16 (100MHz ADCs 0..15)
* Aw16FromAdc32a - route AW trigger (aw16) from adc32 group 15..0 (62.5MHz FMC-ADC32 in ADCs 0..7)
* Aw16FromAdc32b - route AW trigger (aw16) from adc32 group 31..16 (62.5MHz FMC-ADC32 in ADCs 8..15)

== /Equipment/CTRL/Settings/PWB - PWB settings ==
* Enable - if set to "n", fectrl will not do anything with PWBs. Normal value "y".
* enable_boot_user_page - if set to "n", fectrl will not try to reboot PWBs into the user page firmware. Normal value "y".
* enable_trigger - if set to "n", all PWBs will be set to ignore the trigger. Normal value "y".
* enable_trigger_group_a - field wire pulser defeats the data suppression (every channel has a pulse) resulting in too much data overwhelming the DAQ (lost UDP packets). To permit correct data taking of field wire pulser data, PWBs are split into two groups (a and b), only one of the two groups should be enabled for field wire pulser runs. Both groups should be enabled for normal data taking. Normal value is "y"
* enable_trigger_group_b - see "enable_trigger_group_a". Normal value is "y".
* modules[64] - hostnames of PWB boards
* boot_user_page[64] - if set to "y" and enable_boot_user_page is "y", fectrl will reboot the corresponding PWB to the user page firmware.
* trigger[64] - if set to "y" and enable_trigget is "y", fectrl will set the PWB to accept the trigger.
* ch_enable - if set to "n" disables all PWB channels. Normal value is "y".
* ch_force - is set to "y" disables channel suppression (all PWB channels are read)
* suppress_reset - if set to "y" enables channel suppression for reset channels (threshold has to be set correctly)
* suppress_fpn - if set to "y" enables channel suppression for fpn channels (threshold has to be set correctly)
* suppress_pads - set to "y" enables channel suppression for TPC pad channels (threshold has to be set correctly)
* disable_reset1 - data suppression algorithm does not work for the channel "reset1" and it has to be suppressed explicitly by setting this value to "y". Normal value is "y".
* baseline_{reset,fpn,pads}[64] - waveform baseline for each PWB position
* threshold_{reset,fpn,pads} - waveform suppression threshold, see below:
* ch_threshold - PWB sca_N_ch_threshold, if set to 0, threshold is computed as "baseline_pads[seqpwb]-threshold_pads". Normal value is 0.
* test_mode - enable PWB test mode, the ADC data is replaced with a test pattern, see PWB esper variable signalproc/test_mode

== /Equipment/CTRL/Settings/TDC - TDC settings ==
* Enable - if set to "n", fectrl will tell the event builder that the TDC is not running. Normal value is "y".

== /Equipment/CTRL/EvbConfig - EVB configuration ==
All arrays have the same size corresponding to the number of EVB slots.
* name[] - slot name for the web page (trg01, adc01, adc01/adc32, pwb31/0, tdc01, etc)
* type[] - slot type: 1=TRG, 2=ADC16 and ADC32, 3=obsolete PWB, 4=obsolete PWB, 5=PWB, 6=TDC
* module[] - module number (adcNN, adcNN+100, pwbNN, etc)
* nbanks[] - number of expected banks (16 for ADC16, 32 for ADC32, not used, set to 1 for all others)
* tsfreq[] - timestamp frequency in Hz.

= Trigger configuration =

These trigger modes are implemented:
* (manual trigger from web page)
* software pulser trigger (from fectrl)
* hardware pulser trigger (from the trigger board)
* NIM and eSATA trigger from ADC front panel inputs
* adc16 and adc32 discriminator triggers:
** adc16 grand-or trigger
** adc32 grand-or trigger
** adc (adc16+adc32) grand-or trigger
** adc16 multiplicity trigger (1ormore, 2ormore, 3ormore, 4ormore) for use with barrel scintillator
** anode wire multiplicity trigger (1ormore, 2ormore, 3ormore, 4ormore) (anode wire signals can be fed from adc16 or from adc32 signals)
** anode wire coincidence trigger
** anode wire MLU (memory lookup) trigger

== NIM trigger from ADC front panel ==

Trigger path:
<pre>
NIM signal --> LEMO input on ADC front panel --> data encoder --> data link to trigger board --> data decoder --> mask --> nim-esata-grand-or --> trigger
</pre>

To enable NIM trigger, do this:
* connect NIM signal to ADC front panel input
** Note1: NIM signal must be inverted
** Note2: LEMO connectors should be set to "NIM input" mode by ADC on-board jumpers (see ADC manual)
* observe correct bit goes to zero in the TRG web page: data link high work should change:
<pre>
0xNF00'0000 (N is the module id) to
0xN700'0000 (left lemo) or
0xNB00'0000 (right lemo)
</pre>
* Note3: if data link bits do not change, most likely the LEMO connectors are set to "DAC output" mode. Try to use LEMO inputs of a different ADC.
* compute the NIM mask: there are 32 bits, 2 bits per ADC data link. Bit 0x1 is right LEMO, bit 0x2 is left LEMO
* for example, to use left LEMO of ADC data link 12, set nim mask to (0x2)<<(2*12) = 0x02000000
* development branch of the git repository contains ''scripts/nim_mask.py'', which can compute forwards and backwards
* initialize the trigger board (on the TRG web page press button "initialize"), or start a new run
* on the history plot with "NIM grand or" counter, the rate should change from zero
* if a run is active, if one of the ADC LEMO outputs is set to output the trigger signal as a NIM pulse, one should see it fire use a scope. The delay or jitter between NIM trigger in and NIM trigger out should be around 100-200 ns (FIXME - measure this!)
* to set the DAQ to trigger on the NIM signal, on the run start page, check the box "TrigEsataNimGrandOr".
** Note: remember to set the scaledown to 0 if so desired.

= TPC field wire pulser =

The TPC field wire pulser input should be connected directly to the DAC output of adc01 (right lemo connector).

DAC settings in ODB /Equipment/CTRL/Settings/ADC/DAC/

* for pulsing PWBs (maximum possible pulse height in PWBs. Overdrives the AW ADC waveforms)
** dac_baseline: -8000
** dac_amplitude: 8000
** ramp_up_rate: 128
** ramp_down_rate: 70
** ramp_top_len: 128

* for pulsing AW: (AW ADC waveforms are in range. PWB waveforms are too small for goor time resolution)
** dac_baseline: -2000
** dac_amplitude: 4000
** ramp_up_rate: 20
** ramp_down_rate: 20
** ramp_top_len: 128

= TDC connections =

* SFP8 - fiber SFP to Juniper switch
* right RJ45 - 200MHz clock
* left RJ45 - external trigger

= MIDAS Data banks =

* ATAT - fectrl - UDP packets from GRIF-C/TRG trigger board
* TRBA - fetdc - TDC data
* TRTM - fetdc - TDC FPGA temperatures, TID_FLOAT[5].
* AAnn - feudp - UDP packets from GRIF-16 ADC (before the event builder), "nn" is the ADC module number: adc01 is AA01 through adc20 is AA20.
* BBxx - feudp - UDP packets from PWB, obsolete
* Bnnm, Cnnm - feevb - EVB output banks: ADC data, "nn" is the ADC module number (1..20), "m" is the channel number (0..9,A..F), format is same as AAnn banks.
* PAnn - feudp - UDP packets from PWB, "nn" is the PWB module number (00..99), obsolete
* PBnn - feudp - UDP packets from PWB, "nn" is the PWB module number (00..99), PB01 is pwb01, PB78 is pwb78
* PCnn - feevb - EVB output banks: PWB data, "nn" is the PWB module number (00..99), same as above, format is same as PBnn banks

= ZZZ =

ZZZ

Daq

2021-02-09T23:22:44Z

Acapra: /* Schematic */

= Links =

* https://alphacpc05.cern.ch/agdaq/ -- midas daq page for the agdaq system at CERN
* https://daq16.triumf.ca -- midas daq page for the agmini system at TRIUMF
* https://daq.triumf.ca/elog-alphag/alphag/ -- elog
* https://daqstore.triumf.ca/AgWiki/index.php/Main_Page -- this wiki
* https://bitbucket.org/ttriumfdaq/agdaq - git repository for agdaq
* https://bitbucket.org/expalpha -- git repository on bitbucket
* https://edev-group.triumf.ca/fw/exp/alphag -- git repository on edev gitlab
* https://ladd00.triumf.ca/daqinv/frontend/list/170 - PWB inventory database
* https://ladd00.triumf.ca/daqinv/frontend/list/168 - ADC (GRIF-16) inventory, FMC-ADC32rev0,rev1 inventory
* https://ladd00.triumf.ca/daqinv/frontend/list/176 - FMC-ADC32rev1.1 inventory

= Hardware manuals =

* [[TRG]] - trigger board (GRIF-C) manual
* CDM - https://daq.triumf.ca/DaqWiki/index.php/VME-CDM -- clock VME-CDM manual
* ADC - https://daq.triumf.ca/DaqWiki/index.php/VME-GRIF-ADC16-Rev1 -- GRIF-16 ADC manual
* [[PWB]] - TPC Pad Wing Board manual
* TDC - https://www.triumf.info/wiki/DAQwiki/index.php/GSI_TRB3
* [[chronobox]] - chronobox manual

= Performance =

Full system: (ADC fw rel-20201112-ko, PWB fw 20201005_ko)
* 1 Hz stable, ADC adc32 9 supp off rate 0.3 M/s, adc16 off, PWB udp_delay 25 supp off active 64 rate 6.6+4+6+4.6=21.2 M/s, TDC off, total rate 21 M/s
* 500 Hz stable, ADC adc32 9 supp 1500 rate 6 M/s, adc16 off, PWB udp_delay 10 supp 100 active 63 rate 3+4+3+3=13 M/s, TDC off, total rate 19 M/s

Single ADC: (fw rel-20201112-ko)

Fully suppressed data, sthreshold 5000
* 1 Hz: stable
* 500 Hz: stable (nominal max trigger rate)
* 6700 Hz: stable, 8 Mbytes/sec
* 6800 Hz: evb eventually confused
* 6850 Hz: evb confused, no lost triggers in the ADC, no crash in the ADC

All data, 510 samples/event, sthreshold 10 (fw rel-20201122-ko)
* 1 Hz: stable
* 100 Hz: stable, 3.4 Mbytes/sec
* 500 Hz: stable, 17 Mbytes/sec (nominal max trigger rate)
* 1000 Hz: stable, 32 Mbytes/sec
* 3100 Hz: stable, 106 M/s (approaching saturation of 1gige ethernet)
* 3200 Hz: stable, 110 M/s - TRG communication is in trouble
* 3300 Hz: stable, 116 M/s
* 3400 Hz: incomplete events, no ADC crash ("dropped due to full" counting, "dropped due to busy" are zero, this is as expected).

no suppression (different fpga data path)
* 1 Hz: stable
* 100 Hz: stable
* 500 Hz: stable, 17 M/s (nominal max trigger rate)
* 1000 Hz: stable, 32 M/s
* 3000 Hz: stable, 100 M/s (approaching saturation of 1gige ethernet)
* 3300 Hz: incomplete events, no ADC crash ("dropped due to full" counting, "dropped due to busy" are zero, this is as expected).

= Network connections =

The experiment private network number is: 192.168.1.x, gateway 192.168.1.1, netmask 255.255.255.0

network map:
<pre>
---- TRIUMF/CERN network --1gige-- alphagdaq --10gige-- juniper switch --1gige-- centrecom switch
</pre>

== alphagdaq network connections ==

on the rear of the machine:
<pre>
-----------------------------------------
| | rj45 | rj45 | | sfp | sfp |
-----------------------------------------

- left rj45 - copper 1gige - eno1 - spare (inactive)
- right rj45 - copper 1gige - eno2 - dhcp - cern/triumf network
- left sfp - 10gige - enp1s0f1 - spare (inactive)
- right sfp - 10gige - enp1s0f0 - static 192.168.1.1
</pre>

== connections between switches ==

* alphagdaq to juniper switch: direct attach (DAC) 10gige cable into any port of juniper switch
* juniper switch to centrecom switch: rj45 sfp in any port of juniper switch - cat5e/cat6 cable - to any port of centrecom switch
* cat5e/cat6 cable from any port of centrecom switch to the XXX port on the back of the juniper switch

== juniper switch ==

connected to juniper switch front is:

* everything that sends event data to alphagdaq, specifically:
** TRG (rj45 sfp)
** 16x ADC (fiber sfp)
** 64x PWB (fiber sfp)
* 10gige uplink to alphagdaq (10gige DAC cable)
* 1gige link to centrecom switch (rj45 sfp)
* 40gige ports are not used

connected to the juniper switch rear:

* CON is the serial console via RJ45-to-DB9 to USB-serial adapter to alphagdaq USB port (9600bps, 8N1, no flow control)
* C0 is the management ethernet interface port, connect to any port on the centrecom switch (juniper-private will ping)

== centrecom switch ==

* everything with copper rj45 connections, specifically:
** CDM boards
** HV, LV and VME power supplies
** RaspberryPi3 boards
** gas handling MFCs (algas)
** cooling system controller (moxa01)

= USB connections =

Note:
* alphagdaq has USB2 and USB3 connections. They are not the same, they are not generally interchangeable.
* USB3 connections and cables are generally blue coloured.

List of USB connections from alphagdaq:

* front USB3 port (blue): USB-A-to-B cable to USB blaster connected to the trigger board (front of rack)
* rear USB3 port (blue): USB3 short jumper to USB3 hub
* rear USB2 port (black): USB2 unpowered hub (Logiix) to USB-A-to-B cable to UPS USB connector
* rear USB2 port (black): USB-to-Serial adaper to DB9 to RJ45 adapter to Juniper switch console port

Note: UPS USB connection does not work with alphagdaq USB ports (both usb2 and usb3), but combination
using the unpowered USB2 hub seems to work ok.

List of USB connections from the USB hub:

* this is a powered USB3 hub
* USB-A-to-MicroUSB - to lvdb boards
* USB-A-to-Wiener interlock cable

= Clock and trigger distribution =

== Explanation ==

Trigger is generated by the trigger board (aka TRG, aka GRIF-C),
from the TRG eSATA output through the eSATA splitter it is fed into
the master CDM. The master CDM outputs the trigger on all MiniSAS output ports.
One of the master CDM MiniSAS outputs is connected to the CDM slave eSATA input.
The same as the master CDM, the slave CDM outputs the trigger on all MiniSAS output ports.

The master 62.5MHz clock is generated by the master CDM using the internal 10MHz oscillator
or the external 10MHz AD clock. Also on MiniSAS port 6 the master CDM generates
the 200MHz clock for the TDC. This clock is frequency-locked to the master 62.5MHz clock.

Master CDM clock selection (external CD 10MHz clock or internal oscillator 10MHz clock)
is done by a magic esper command (see below).

From the master CDM, the clock is fed to: the slave CDM, the TRG (via the eSATA splitter)
and to the TDC (via the RJ45 splitter).

The slave CDM sends the clock and the trigger to the ADCs and PWBs.

== Schematic ==

<pre>
TRG (GRIF-C)
------------
eSATA <------> eSATA splitter

eSATA splitter
--------------
trigger ---> eSATA ---> CDM-Master eSATA (trigger signal)
clock <--- (eSATA --- MiniSAS) <--- CDM MiniSAS (62.5MHz clock)
</pre>

<pre>
CDM-Master
----------
eSATA <--- eSATA splitter <--- eSATA from TRG (trigger signal)
MiniSAS 1 --> (MiniSAS --- eSATA) --> CDM-Slave eSATA (62.5 MHz)
MiniSAS 6 --> (MiniSAS --- eSATA) --> eSATA splitter --> RJ45 --> TDC (200 MHz)
</pre>

<pre>
CDM-Slave
---------
eSATA <--- trigger and 62.5MHz clock from CDM-Master
MiniSAS 1 --> ADC trigger and 62.5MHz clock
MiniSAS 2 --> same
MiniSAS 3 --> same
MiniSAS 4 --> same
MiniSAS 5 --> PWB trigger and 62.5MHz clock
MiniSAS 6 --> same
</pre>

[[File:VME_Crate_CERN.jpeg|800x450px|Layout of the VME crate at CERN.]]
Layout of the VME crate at CERN. The ALPHA-T module is in the rightmost slot. The Master CDM is next to it and it's labelled as #02. The Slave CDM is on the lefmost position and labelled by #03.

== Setup of CDM boards ==

ALPHA-g uses the Rev3 CDM boards. The 4 CDM boards that belong to ALPHA-g
have serial numbers 0, 1, 2 and 3 and are labeled cdm00, cdm01, cdm02, cdm03.

All the boards can be used as master and slave positions, but have to be
configured appropriately.

=== Slave setup ===

* install the board
* connect ethernet
* connect esata external clock
* esper-tool cdmNN ### i.e. esper-tool cdm01
* cd template
* write current_setup 4
* cd /cdm
* read
* observe clock frequency counters report correct values. "ext_clk" will only count if the lemo jumper is installed (see below)
<pre>
11 esata_clk uint32 R [62500246]
12 ext_clk uint32 R [62500246]
</pre>
* cd /lmk
* read
* observe both PLL1 and PLL2 are locked: both values should be "1"
<pre>
144 pll1_ld uint8 R [1]
147 pll2_ld uint8 R [1]
156 ld1_counter uint32 RW [3]
157 ld2_counter uint32 RW [3]
</pre>

If needed, check correct operation of the pll lock monitoring:

* unplug the esata cable from the master CDM
* "red" light should go off
* read of lmk should report "pll1_ld" value 0 and "ld1_counter" and "ld2_counter" should increment
* reconnect the esata cable
* "red" light should return
* read of lmk should report both pll1_ld and pll2_ld locked (values "1").

=== Master setup ===

* install the board
* connect ethernet
* esper-tool cdmNN ### i.e. esper-tool cdm01
* cd template
* write current_setup 3
* cd /lmk
* write clkin_sel_mode 0
* cd /cdm
* read
* observe clock frequency counters report correct values. "ext_clk" will only count if the "clock loopback" lemo jumper is installed (see below)
<pre>
12 ext_clk uint32 R [62500189]
</pre>
* cd /lmk
* read
* observe both PLL1 and PLL2 are locked: both values should be "1"
<pre>
144 pll1_ld uint8 R [1]
147 pll2_ld uint8 R [1]
156 ld1_counter uint32 RW [3]
157 ld2_counter uint32 RW [3]
</pre>

Check the 200MHz clock:
* connect MiniSAS-eSATA splitter cable to MiniSAS output 6, connect eSATA end to CDM eSATA input
* in esper-tool, read cdm: esata_clk should report 200MHz clock
<pre>
11 esata_clk uint32 R [200000603]
12 ext_clk uint32 R [62500188]
</pre>

=== Master setup with external 10 MHz clock ===

* setup master CDM as above (using internal clock)
* connect 10MHz external clock to right LEMO #1
* esper-tool cdmNN
* cd /cdm
* write sel_nim 1 # if clock is NIM signal
* write sel_nim 0 # if clock is TTL signal (i.e. chronobox LEMO output)
* read ext_clk
<pre>
[http://cdm00:/cdm]> read ext_clk
9999556
</pre>
* if ext_clk reads 0 or does not read something close to 10MHz, stop here. debug the signal, try changing sel_nim, etc.
* cd /lmk
* write clkin_sel_mode 2 # select external clock
* read
* observe both PLL1 and PLL2 are locked: both values should be "1"
<pre>
150 clkin2_sel bool R [True]
144 pll1_ld uint8 R [1]
147 pll2_ld uint8 R [1]
</pre>
* on the MIDAS TRG page, observe that TRG measured external clock frequency is close to 62.5 MHz

== clkin_sel_mode positions ==

* 0 = 10MHz internal oscillator (use on master CDM in standalone mode)
* 1 = eSATA clock (use on slave CDM)
* 2 = external clock, should be 10MHz (use on master CDM to run from AD clock)

== LEMO connections ==

* clock loopback jumper: to read the frequency of the generated 62.5MHz clock, connect LEMO cable from left LEMO #2 to the right LEMO #1
* 10MHz internal clock jumper: to feed 10MHz internal oscillator to the external clock, connect LEMO cable from right LEMO #3 to the right LEMO #1.
* 10MHz external clock: to feed 10MHz external clock, connect LEMO cable to right LEMO #1

= MIDAS frontends =

== UDP ==

feudp receives udp packets from the ADC and PWB boards. For each udp packet 1 midas event is created,
with 1 data bank containing the udp data. Events are sent into the MIDAS event buffer BUFUDP. The names
of the data banks are assigned in ODB /eq/feudp/settings.

{ADC,PWB} --> 1gige --> switch --> 10gige --> alphagdaq --> feudp -> BUFUDP

== CTRL ==

fectrl controls all the ADC, PWB and trigger boards. It provides run start and stop sequencing,
runs the event synchronization sequence at the begin of the run, and records slow controls data (temperatures,
voltages, etc).

fectrl receives UDP data packets from the trigger board and sends them to the MIDAS event buffer BUFUDP.

fectrl receives commands from the control web pages (for ADC, PWB and trigger) to reboot, initialize, etc the boards.

fectrl configures the event builder via odb /eq/fectrl/evbconfig.

ADC <-> http esper <-> fectrl -> slow control data into midas history

PWB <-> http esper <-> fectrl -> slow control data into midas history

TRG <-> udp comm <-> fectrl -> BUFUDP, slow control and counters into midas history

fectrl <-> midas rpc <-> mhttpd <-> json rpc <-> control web pages for ADC, PWB and trigger

== EVB ==

feevb receives udp packet data from the MIDAS event buffer BUFUDP, inspects the packet timestamps
and collects the data with matching timestamps into physics events. feevb has provisions to do
data suppression, reduction and compression in addition to the data reduction done
in the ADC and PWB firmware.

feevb event synchronization relies on the event synchronization sequence done by fectrl when a run is started.

{ADC, PWB, TRG} -> BUFUDP -> feevb -> SYSTEM -> mlogger -> compression -> disk storage

=== internal structure ===

* event buffer readers, instances for BUFTRG, BUFTDC, BUFADC, BUFUDP
<pre>
BufferReader class - reads one midas event buffer:
- BufferReaderThread()
-- in batches of 1000 events
--- lock
--- TickLocked()
----- ReadEvent() -> bm_receive_event_alloc()
----- fHandler->HandleEvent()
-- fHandler->XMaybeFlushBank()
-- sleep 100 usec
</pre>
* event header decoder and intermediate buffer
<pre>
Handler class - accumulates events, sends them to the EVB in batches

- HandleEvent() <- BufferReaderThread() - receive incoming events
-- loop over all banks, examine bank name
--- call AddAdcBank(), AddPwbBank(), AddTrgBank(), AddTdcBank() according to bank name
---- AddXxxBank() -> XAddBank()
--- unknown banks go directly to evb->SendQueue
-- check if synchronization has completed

- XAddBank() - save BankBuf objects in a buffer

- XMaybeFlushBank(), XFlushBank() - send all accumulated data to the Evb class
-- evb->lock()
-- for each buffered BankBuf objects,
--- call evb->AddBankLocked()
-- buffer is now empty
</pre>
* event builder
<pre>
Evb class - assemble event fragments according to timestamps

- fEvents fifo (deque) - time sorted list of partially assembled events
-- new entries are added by FindEvent()
-- entries are modified by MergeSlot()
-- old entries are removed by GetLocked()

- AddBankLocked() <- XFlushBank() <- BufferReaderThread()
-- call fSync->Add()
-- transform BankBuf to EvbEventBuf
-- push it into per-module buffer fBuf[imodule]

- EvbThread()
-- EvbTickLocked()
--- BuildLocked()
---- in a loop, for no more than 1 sec
----- loop over all per-module buffers fBuf
------ pop first entry, pass it to BuildSlot()
------- call FindEvent()
------- call MergeSlot()
------- call CheckEvent()
--- in a loop
---- get all completed events via GetLocked() pass them to SendQueue->PushEvent()
-- if done nothing, sleep 100 usec, otherwise nominal sleep to free the evb lock

- GetNext() <- GetLocked() <- EvbThread() - get completed events
-- examine oldest EvbEvent entry in fEvents:
-- pop it if it is complete
-- pop it if it is too old (to prevent old incomplete events from stalling the evb) ("pop age" on the evb page)
-- pop it if a following event is complete ("pop following" on the evb page)
</pre>
* send queue
<pre>
SendQueue class - prepare events for writing to midas SYSTEM event buffer

- PushEvent() <- EvbThread()
-- push FragmentBuf into the queue

- SendQueueThread()
-- in a batch of 1000 events, call Tick()
--- pop FragmentBuf from the queue, call SendEvent()
--- compose midas event (memcpy()!)
--- TMFE fEq->SendEvent()
--- bm_send_event()
-- sleep 100 usec
</pre>
* equipment object
<pre>
EvbEq class - implements the midas equipment
- HandleBeginRun()
- HandleEndRun()
- HandlePeriodic()
</pre>

=== multithreading ===

* BufferReaderThread() one per midas event buffer
** high CPU use: bm_receive_event(), AddXxxBank()
** lock contention: midas event buffer lock, evb lock into XFlushBank->AddBankLocked()
* EvbThread()
** O(size of fEvents): FindEvent(), GetNext()
** lock contention: evb lock against XFlushBank(), SendQueue lock (1 per output event)
* SendQueueThread()
** high CPU use: memcpy() in SendEvent(), memcpy() in bm_send_event
** lock contention: event queue lock against EvbThread, midas event buffer lock
* main thread
** lock contention: evb lock in ReportEvb() and elsewhere

= ODB entries =

== /Equipment/CTRL/Settings ==
* TBW

== /Equipment/CTRL/Settings/TrigSrc - trigger source selection ==
* TrigPulser - trigger on the pulser
* TrigEsataNimGrandOr - trigger on external NIM inputs of GRIF-16 ADC modules
* TrigAdc16GrandOr - trigger on any adc16 signals (see adc16_masks)
* TrigAdc32GrandOr - trigger on any adc32 signals (see adc32_masks)
* Trig1ormore - trigger on adc16 multiplicity 1 or more
* Trig2ormore - ditto, 2 or more
* Trig3ormore - ditto, 3 or more
* Trig4ormore - ditto, 4 or more
* TrigAdcGrandOr - trigger on any ADC signal
* TrigAwCoincA - trigger on TPC AW coincidence (see [[TRG]] manual)
* TrigAwCoincB - ditto
* TrigAwCoincC - ditto
* TrigAwCoincD - ditto
* TrigAwCoinc - ditto
* TrigAw1ormore - trigger on TPC AW per-preamp multiplicity 1 or more (see [[TRG]] manual)
* TrigAw2ormore - ditto, 2 or more
* TrigAw3ormore - ditto, 3 or more
* TrigAw4ormore - ditto, 4 or more
* TrigAwMLU - trigger on TPC AW MLU signal (see [[TRG]] manual)
* TrigBscGrandOr - trigger on any BSC signal
* TrigBscMult - trigger on predefined BSC multiplicity (see [[TRG]] manual)
* TrigCoinc - trigger on predefined coincidence of TPC AW, BSC and external signal (see [[TRG]] manual)

== /Equipment/CTRL/Settings/TRG - trigger settings ==
* Enable - if set to "n", fectrl will not do anything with the trigger board. Normal value "y"
* Modules - hostnames of the trigger board. Normal value "alphat01"
* NimMask - 32-bit mask for the 32x NIM inputs (see TRG manual). Normal value 0, unless external trigger is in use.
* EsataMask - 32-bit mask for the 32x eSATA signals (see TRG manual). Normal value 0.
* adc16_mask[16] - 16x 16-bit mask for the 100MHz GRIF-16 onboard ADCs (see TRG manual). Normal value 0xFFFF for all 16 data links.
* adc32_mask[16] - 16x 32-bit mask for the 62.5MHz FMC-ADC32 ADCs (see TRG manual). Normal value 0xFFFFFFFF for all 16 data links.
* PassThrough - if set to "y", trigger is passed through the trigger module without generating any events and without causing deadtime.
* AwCoincA, B, C and D - 32-bit masks for the aw16 coincidence trigger (see TRG manual)
* Aw16FromAdc16 - route AW trigger (aw16) from adc16 (100MHz ADCs 0..15)
* Aw16FromAdc32a - route AW trigger (aw16) from adc32 group 15..0 (62.5MHz FMC-ADC32 in ADCs 0..7)
* Aw16FromAdc32b - route AW trigger (aw16) from adc32 group 31..16 (62.5MHz FMC-ADC32 in ADCs 8..15)

== /Equipment/CTRL/Settings/PWB - PWB settings ==
* Enable - if set to "n", fectrl will not do anything with PWBs. Normal value "y".
* enable_boot_user_page - if set to "n", fectrl will not try to reboot PWBs into the user page firmware. Normal value "y".
* enable_trigger - if set to "n", all PWBs will be set to ignore the trigger. Normal value "y".
* enable_trigger_group_a - field wire pulser defeats the data suppression (every channel has a pulse) resulting in too much data overwhelming the DAQ (lost UDP packets). To permit correct data taking of field wire pulser data, PWBs are split into two groups (a and b), only one of the two groups should be enabled for field wire pulser runs. Both groups should be enabled for normal data taking. Normal value is "y"
* enable_trigger_group_b - see "enable_trigger_group_a". Normal value is "y".
* modules[64] - hostnames of PWB boards
* boot_user_page[64] - if set to "y" and enable_boot_user_page is "y", fectrl will reboot the corresponding PWB to the user page firmware.
* trigger[64] - if set to "y" and enable_trigget is "y", fectrl will set the PWB to accept the trigger.
* ch_enable - if set to "n" disables all PWB channels. Normal value is "y".
* ch_force - is set to "y" disables channel suppression (all PWB channels are read)
* suppress_reset - if set to "y" enables channel suppression for reset channels (threshold has to be set correctly)
* suppress_fpn - if set to "y" enables channel suppression for fpn channels (threshold has to be set correctly)
* suppress_pads - set to "y" enables channel suppression for TPC pad channels (threshold has to be set correctly)
* disable_reset1 - data suppression algorithm does not work for the channel "reset1" and it has to be suppressed explicitly by setting this value to "y". Normal value is "y".
* baseline_{reset,fpn,pads}[64] - waveform baseline for each PWB position
* threshold_{reset,fpn,pads} - waveform suppression threshold, see below:
* ch_threshold - PWB sca_N_ch_threshold, if set to 0, threshold is computed as "baseline_pads[seqpwb]-threshold_pads". Normal value is 0.
* test_mode - enable PWB test mode, the ADC data is replaced with a test pattern, see PWB esper variable signalproc/test_mode

== /Equipment/CTRL/Settings/TDC - TDC settings ==
* Enable - if set to "n", fectrl will tell the event builder that the TDC is not running. Normal value is "y".

== /Equipment/CTRL/EvbConfig - EVB configuration ==
All arrays have the same size corresponding to the number of EVB slots.
* name[] - slot name for the web page (trg01, adc01, adc01/adc32, pwb31/0, tdc01, etc)
* type[] - slot type: 1=TRG, 2=ADC16 and ADC32, 3=obsolete PWB, 4=obsolete PWB, 5=PWB, 6=TDC
* module[] - module number (adcNN, adcNN+100, pwbNN, etc)
* nbanks[] - number of expected banks (16 for ADC16, 32 for ADC32, not used, set to 1 for all others)
* tsfreq[] - timestamp frequency in Hz.

= Trigger configuration =

These trigger modes are implemented:
* (manual trigger from web page)
* software pulser trigger (from fectrl)
* hardware pulser trigger (from the trigger board)
* NIM and eSATA trigger from ADC front panel inputs
* adc16 and adc32 discriminator triggers:
** adc16 grand-or trigger
** adc32 grand-or trigger
** adc (adc16+adc32) grand-or trigger
** adc16 multiplicity trigger (1ormore, 2ormore, 3ormore, 4ormore) for use with barrel scintillator
** anode wire multiplicity trigger (1ormore, 2ormore, 3ormore, 4ormore) (anode wire signals can be fed from adc16 or from adc32 signals)
** anode wire coincidence trigger
** anode wire MLU (memory lookup) trigger

== NIM trigger from ADC front panel ==

Trigger path:
<pre>
NIM signal --> LEMO input on ADC front panel --> data encoder --> data link to trigger board --> data decoder --> mask --> nim-esata-grand-or --> trigger
</pre>

To enable NIM trigger, do this:
* connect NIM signal to ADC front panel input
** Note1: NIM signal must be inverted
** Note2: LEMO connectors should be set to "NIM input" mode by ADC on-board jumpers (see ADC manual)
* observe correct bit goes to zero in the TRG web page: data link high work should change:
<pre>
0xNF00'0000 (N is the module id) to
0xN700'0000 (left lemo) or
0xNB00'0000 (right lemo)
</pre>
* Note3: if data link bits do not change, most likely the LEMO connectors are set to "DAC output" mode. Try to use LEMO inputs of a different ADC.
* compute the NIM mask: there are 32 bits, 2 bits per ADC data link. Bit 0x1 is right LEMO, bit 0x2 is left LEMO
* for example, to use left LEMO of ADC data link 12, set nim mask to (0x2)<<(2*12) = 0x02000000
* development branch of the git repository contains ''scripts/nim_mask.py'', which can compute forwards and backwards
* initialize the trigger board (on the TRG web page press button "initialize"), or start a new run
* on the history plot with "NIM grand or" counter, the rate should change from zero
* if a run is active, if one of the ADC LEMO outputs is set to output the trigger signal as a NIM pulse, one should see it fire use a scope. The delay or jitter between NIM trigger in and NIM trigger out should be around 100-200 ns (FIXME - measure this!)
* to set the DAQ to trigger on the NIM signal, on the run start page, check the box "TrigEsataNimGrandOr".
** Note: remember to set the scaledown to 0 if so desired.

= TPC field wire pulser =

The TPC field wire pulser input should be connected directly to the DAC output of adc01 (right lemo connector).

DAC settings in ODB /Equipment/CTRL/Settings/ADC/DAC/

* for pulsing PWBs (maximum possible pulse height in PWBs. Overdrives the AW ADC waveforms)
** dac_baseline: -8000
** dac_amplitude: 8000
** ramp_up_rate: 128
** ramp_down_rate: 70
** ramp_top_len: 128

* for pulsing AW: (AW ADC waveforms are in range. PWB waveforms are too small for goor time resolution)
** dac_baseline: -2000
** dac_amplitude: 4000
** ramp_up_rate: 20
** ramp_down_rate: 20
** ramp_top_len: 128

= TDC connections =

* SFP8 - fiber SFP to Juniper switch
* right RJ45 - 200MHz clock
* left RJ45 - external trigger

= MIDAS Data banks =

* ATAT - fectrl - UDP packets from GRIF-C/TRG trigger board
* TRBA - fetdc - TDC data
* TRTM - fetdc - TDC FPGA temperatures, TID_FLOAT[5].
* AAnn - feudp - UDP packets from GRIF-16 ADC (before the event builder), "nn" is the ADC module number: adc01 is AA01 through adc20 is AA20.
* BBxx - feudp - UDP packets from PWB, obsolete
* Bnnm, Cnnm - feevb - EVB output banks: ADC data, "nn" is the ADC module number (1..20), "m" is the channel number (0..9,A..F), format is same as AAnn banks.
* PAnn - feudp - UDP packets from PWB, "nn" is the PWB module number (00..99), obsolete
* PBnn - feudp - UDP packets from PWB, "nn" is the PWB module number (00..99), PB01 is pwb01, PB78 is pwb78
* PCnn - feevb - EVB output banks: PWB data, "nn" is the PWB module number (00..99), same as above, format is same as PBnn banks

= ZZZ =

ZZZ

Daq

2021-02-09T23:21:43Z

Acapra: /* Schematic */

= Links =

* https://alphacpc05.cern.ch/agdaq/ -- midas daq page for the agdaq system at CERN
* https://daq16.triumf.ca -- midas daq page for the agmini system at TRIUMF
* https://daq.triumf.ca/elog-alphag/alphag/ -- elog
* https://daqstore.triumf.ca/AgWiki/index.php/Main_Page -- this wiki
* https://bitbucket.org/ttriumfdaq/agdaq - git repository for agdaq
* https://bitbucket.org/expalpha -- git repository on bitbucket
* https://edev-group.triumf.ca/fw/exp/alphag -- git repository on edev gitlab
* https://ladd00.triumf.ca/daqinv/frontend/list/170 - PWB inventory database
* https://ladd00.triumf.ca/daqinv/frontend/list/168 - ADC (GRIF-16) inventory, FMC-ADC32rev0,rev1 inventory
* https://ladd00.triumf.ca/daqinv/frontend/list/176 - FMC-ADC32rev1.1 inventory

= Hardware manuals =

* [[TRG]] - trigger board (GRIF-C) manual
* CDM - https://daq.triumf.ca/DaqWiki/index.php/VME-CDM -- clock VME-CDM manual
* ADC - https://daq.triumf.ca/DaqWiki/index.php/VME-GRIF-ADC16-Rev1 -- GRIF-16 ADC manual
* [[PWB]] - TPC Pad Wing Board manual
* TDC - https://www.triumf.info/wiki/DAQwiki/index.php/GSI_TRB3
* [[chronobox]] - chronobox manual

= Performance =

Full system: (ADC fw rel-20201112-ko, PWB fw 20201005_ko)
* 1 Hz stable, ADC adc32 9 supp off rate 0.3 M/s, adc16 off, PWB udp_delay 25 supp off active 64 rate 6.6+4+6+4.6=21.2 M/s, TDC off, total rate 21 M/s
* 500 Hz stable, ADC adc32 9 supp 1500 rate 6 M/s, adc16 off, PWB udp_delay 10 supp 100 active 63 rate 3+4+3+3=13 M/s, TDC off, total rate 19 M/s

Single ADC: (fw rel-20201112-ko)

Fully suppressed data, sthreshold 5000
* 1 Hz: stable
* 500 Hz: stable (nominal max trigger rate)
* 6700 Hz: stable, 8 Mbytes/sec
* 6800 Hz: evb eventually confused
* 6850 Hz: evb confused, no lost triggers in the ADC, no crash in the ADC

All data, 510 samples/event, sthreshold 10 (fw rel-20201122-ko)
* 1 Hz: stable
* 100 Hz: stable, 3.4 Mbytes/sec
* 500 Hz: stable, 17 Mbytes/sec (nominal max trigger rate)
* 1000 Hz: stable, 32 Mbytes/sec
* 3100 Hz: stable, 106 M/s (approaching saturation of 1gige ethernet)
* 3200 Hz: stable, 110 M/s - TRG communication is in trouble
* 3300 Hz: stable, 116 M/s
* 3400 Hz: incomplete events, no ADC crash ("dropped due to full" counting, "dropped due to busy" are zero, this is as expected).

no suppression (different fpga data path)
* 1 Hz: stable
* 100 Hz: stable
* 500 Hz: stable, 17 M/s (nominal max trigger rate)
* 1000 Hz: stable, 32 M/s
* 3000 Hz: stable, 100 M/s (approaching saturation of 1gige ethernet)
* 3300 Hz: incomplete events, no ADC crash ("dropped due to full" counting, "dropped due to busy" are zero, this is as expected).

= Network connections =

The experiment private network number is: 192.168.1.x, gateway 192.168.1.1, netmask 255.255.255.0

network map:
<pre>
---- TRIUMF/CERN network --1gige-- alphagdaq --10gige-- juniper switch --1gige-- centrecom switch
</pre>

== alphagdaq network connections ==

on the rear of the machine:
<pre>
-----------------------------------------
| | rj45 | rj45 | | sfp | sfp |
-----------------------------------------

- left rj45 - copper 1gige - eno1 - spare (inactive)
- right rj45 - copper 1gige - eno2 - dhcp - cern/triumf network
- left sfp - 10gige - enp1s0f1 - spare (inactive)
- right sfp - 10gige - enp1s0f0 - static 192.168.1.1
</pre>

== connections between switches ==

* alphagdaq to juniper switch: direct attach (DAC) 10gige cable into any port of juniper switch
* juniper switch to centrecom switch: rj45 sfp in any port of juniper switch - cat5e/cat6 cable - to any port of centrecom switch
* cat5e/cat6 cable from any port of centrecom switch to the XXX port on the back of the juniper switch

== juniper switch ==

connected to juniper switch front is:

* everything that sends event data to alphagdaq, specifically:
** TRG (rj45 sfp)
** 16x ADC (fiber sfp)
** 64x PWB (fiber sfp)
* 10gige uplink to alphagdaq (10gige DAC cable)
* 1gige link to centrecom switch (rj45 sfp)
* 40gige ports are not used

connected to the juniper switch rear:

* CON is the serial console via RJ45-to-DB9 to USB-serial adapter to alphagdaq USB port (9600bps, 8N1, no flow control)
* C0 is the management ethernet interface port, connect to any port on the centrecom switch (juniper-private will ping)

== centrecom switch ==

* everything with copper rj45 connections, specifically:
** CDM boards
** HV, LV and VME power supplies
** RaspberryPi3 boards
** gas handling MFCs (algas)
** cooling system controller (moxa01)

= USB connections =

Note:
* alphagdaq has USB2 and USB3 connections. They are not the same, they are not generally interchangeable.
* USB3 connections and cables are generally blue coloured.

List of USB connections from alphagdaq:

* front USB3 port (blue): USB-A-to-B cable to USB blaster connected to the trigger board (front of rack)
* rear USB3 port (blue): USB3 short jumper to USB3 hub
* rear USB2 port (black): USB2 unpowered hub (Logiix) to USB-A-to-B cable to UPS USB connector
* rear USB2 port (black): USB-to-Serial adaper to DB9 to RJ45 adapter to Juniper switch console port

Note: UPS USB connection does not work with alphagdaq USB ports (both usb2 and usb3), but combination
using the unpowered USB2 hub seems to work ok.

List of USB connections from the USB hub:

* this is a powered USB3 hub
* USB-A-to-MicroUSB - to lvdb boards
* USB-A-to-Wiener interlock cable

= Clock and trigger distribution =

== Explanation ==

Trigger is generated by the trigger board (aka TRG, aka GRIF-C),
from the TRG eSATA output through the eSATA splitter it is fed into
the master CDM. The master CDM outputs the trigger on all MiniSAS output ports.
One of the master CDM MiniSAS outputs is connected to the CDM slave eSATA input.
The same as the master CDM, the slave CDM outputs the trigger on all MiniSAS output ports.

The master 62.5MHz clock is generated by the master CDM using the internal 10MHz oscillator
or the external 10MHz AD clock. Also on MiniSAS port 6 the master CDM generates
the 200MHz clock for the TDC. This clock is frequency-locked to the master 62.5MHz clock.

Master CDM clock selection (external CD 10MHz clock or internal oscillator 10MHz clock)
is done by a magic esper command (see below).

From the master CDM, the clock is fed to: the slave CDM, the TRG (via the eSATA splitter)
and to the TDC (via the RJ45 splitter).

The slave CDM sends the clock and the trigger to the ADCs and PWBs.

== Schematic ==

<pre>
TRG (GRIF-C)
------------
eSATA <------> eSATA splitter

eSATA splitter
--------------
trigger ---> eSATA ---> CDM-Master eSATA (trigger signal)
clock <--- (eSATA --- MiniSAS) <--- CDM MiniSAS (62.5MHz clock)
</pre>

<pre>
CDM-Master
----------
eSATA <--- eSATA splitter <--- eSATA from TRG (trigger signal)
MiniSAS 1 --> (MiniSAS --- eSATA) --> CDM-Slave eSATA (62.5 MHz)
MiniSAS 6 --> (MiniSAS --- eSATA) --> eSATA splitter --> RJ45 --> TDC (200 MHz)
</pre>

<pre>
CDM-Slave
---------
eSATA <--- trigger and 62.5MHz clock from CDM-Master
MiniSAS 1 --> ADC trigger and 62.5MHz clock
MiniSAS 2 --> same
MiniSAS 3 --> same
MiniSAS 4 --> same
MiniSAS 5 --> PWB trigger and 62.5MHz clock
MiniSAS 6 --> same
</pre>

[[File:VME_Crate_CERN.jpeg|800x450px|Layout of the VME crate at CERN. The ALPHA-T module is in the rightmost slot. The Master CDM is next to it and it's labelled as #02. The Slave CDM is on the lefmost position and labelled by #03.]]

== Setup of CDM boards ==

ALPHA-g uses the Rev3 CDM boards. The 4 CDM boards that belong to ALPHA-g
have serial numbers 0, 1, 2 and 3 and are labeled cdm00, cdm01, cdm02, cdm03.

All the boards can be used as master and slave positions, but have to be
configured appropriately.

=== Slave setup ===

* install the board
* connect ethernet
* connect esata external clock
* esper-tool cdmNN ### i.e. esper-tool cdm01
* cd template
* write current_setup 4
* cd /cdm
* read
* observe clock frequency counters report correct values. "ext_clk" will only count if the lemo jumper is installed (see below)
<pre>
11 esata_clk uint32 R [62500246]
12 ext_clk uint32 R [62500246]
</pre>
* cd /lmk
* read
* observe both PLL1 and PLL2 are locked: both values should be "1"
<pre>
144 pll1_ld uint8 R [1]
147 pll2_ld uint8 R [1]
156 ld1_counter uint32 RW [3]
157 ld2_counter uint32 RW [3]
</pre>

If needed, check correct operation of the pll lock monitoring:

* unplug the esata cable from the master CDM
* "red" light should go off
* read of lmk should report "pll1_ld" value 0 and "ld1_counter" and "ld2_counter" should increment
* reconnect the esata cable
* "red" light should return
* read of lmk should report both pll1_ld and pll2_ld locked (values "1").

=== Master setup ===

* install the board
* connect ethernet
* esper-tool cdmNN ### i.e. esper-tool cdm01
* cd template
* write current_setup 3
* cd /lmk
* write clkin_sel_mode 0
* cd /cdm
* read
* observe clock frequency counters report correct values. "ext_clk" will only count if the "clock loopback" lemo jumper is installed (see below)
<pre>
12 ext_clk uint32 R [62500189]
</pre>
* cd /lmk
* read
* observe both PLL1 and PLL2 are locked: both values should be "1"
<pre>
144 pll1_ld uint8 R [1]
147 pll2_ld uint8 R [1]
156 ld1_counter uint32 RW [3]
157 ld2_counter uint32 RW [3]
</pre>

Check the 200MHz clock:
* connect MiniSAS-eSATA splitter cable to MiniSAS output 6, connect eSATA end to CDM eSATA input
* in esper-tool, read cdm: esata_clk should report 200MHz clock
<pre>
11 esata_clk uint32 R [200000603]
12 ext_clk uint32 R [62500188]
</pre>

=== Master setup with external 10 MHz clock ===

* setup master CDM as above (using internal clock)
* connect 10MHz external clock to right LEMO #1
* esper-tool cdmNN
* cd /cdm
* write sel_nim 1 # if clock is NIM signal
* write sel_nim 0 # if clock is TTL signal (i.e. chronobox LEMO output)
* read ext_clk
<pre>
[http://cdm00:/cdm]> read ext_clk
9999556
</pre>
* if ext_clk reads 0 or does not read something close to 10MHz, stop here. debug the signal, try changing sel_nim, etc.
* cd /lmk
* write clkin_sel_mode 2 # select external clock
* read
* observe both PLL1 and PLL2 are locked: both values should be "1"
<pre>
150 clkin2_sel bool R [True]
144 pll1_ld uint8 R [1]
147 pll2_ld uint8 R [1]
</pre>
* on the MIDAS TRG page, observe that TRG measured external clock frequency is close to 62.5 MHz

== clkin_sel_mode positions ==

* 0 = 10MHz internal oscillator (use on master CDM in standalone mode)
* 1 = eSATA clock (use on slave CDM)
* 2 = external clock, should be 10MHz (use on master CDM to run from AD clock)

== LEMO connections ==

* clock loopback jumper: to read the frequency of the generated 62.5MHz clock, connect LEMO cable from left LEMO #2 to the right LEMO #1
* 10MHz internal clock jumper: to feed 10MHz internal oscillator to the external clock, connect LEMO cable from right LEMO #3 to the right LEMO #1.
* 10MHz external clock: to feed 10MHz external clock, connect LEMO cable to right LEMO #1

= MIDAS frontends =

== UDP ==

feudp receives udp packets from the ADC and PWB boards. For each udp packet 1 midas event is created,
with 1 data bank containing the udp data. Events are sent into the MIDAS event buffer BUFUDP. The names
of the data banks are assigned in ODB /eq/feudp/settings.

{ADC,PWB} --> 1gige --> switch --> 10gige --> alphagdaq --> feudp -> BUFUDP

== CTRL ==

fectrl controls all the ADC, PWB and trigger boards. It provides run start and stop sequencing,
runs the event synchronization sequence at the begin of the run, and records slow controls data (temperatures,
voltages, etc).

fectrl receives UDP data packets from the trigger board and sends them to the MIDAS event buffer BUFUDP.

fectrl receives commands from the control web pages (for ADC, PWB and trigger) to reboot, initialize, etc the boards.

fectrl configures the event builder via odb /eq/fectrl/evbconfig.

ADC <-> http esper <-> fectrl -> slow control data into midas history

PWB <-> http esper <-> fectrl -> slow control data into midas history

TRG <-> udp comm <-> fectrl -> BUFUDP, slow control and counters into midas history

fectrl <-> midas rpc <-> mhttpd <-> json rpc <-> control web pages for ADC, PWB and trigger

== EVB ==

feevb receives udp packet data from the MIDAS event buffer BUFUDP, inspects the packet timestamps
and collects the data with matching timestamps into physics events. feevb has provisions to do
data suppression, reduction and compression in addition to the data reduction done
in the ADC and PWB firmware.

feevb event synchronization relies on the event synchronization sequence done by fectrl when a run is started.

{ADC, PWB, TRG} -> BUFUDP -> feevb -> SYSTEM -> mlogger -> compression -> disk storage

=== internal structure ===

* event buffer readers, instances for BUFTRG, BUFTDC, BUFADC, BUFUDP
<pre>
BufferReader class - reads one midas event buffer:
- BufferReaderThread()
-- in batches of 1000 events
--- lock
--- TickLocked()
----- ReadEvent() -> bm_receive_event_alloc()
----- fHandler->HandleEvent()
-- fHandler->XMaybeFlushBank()
-- sleep 100 usec
</pre>
* event header decoder and intermediate buffer
<pre>
Handler class - accumulates events, sends them to the EVB in batches

- HandleEvent() <- BufferReaderThread() - receive incoming events
-- loop over all banks, examine bank name
--- call AddAdcBank(), AddPwbBank(), AddTrgBank(), AddTdcBank() according to bank name
---- AddXxxBank() -> XAddBank()
--- unknown banks go directly to evb->SendQueue
-- check if synchronization has completed

- XAddBank() - save BankBuf objects in a buffer

- XMaybeFlushBank(), XFlushBank() - send all accumulated data to the Evb class
-- evb->lock()
-- for each buffered BankBuf objects,
--- call evb->AddBankLocked()
-- buffer is now empty
</pre>
* event builder
<pre>
Evb class - assemble event fragments according to timestamps

- fEvents fifo (deque) - time sorted list of partially assembled events
-- new entries are added by FindEvent()
-- entries are modified by MergeSlot()
-- old entries are removed by GetLocked()

- AddBankLocked() <- XFlushBank() <- BufferReaderThread()
-- call fSync->Add()
-- transform BankBuf to EvbEventBuf
-- push it into per-module buffer fBuf[imodule]

- EvbThread()
-- EvbTickLocked()
--- BuildLocked()
---- in a loop, for no more than 1 sec
----- loop over all per-module buffers fBuf
------ pop first entry, pass it to BuildSlot()
------- call FindEvent()
------- call MergeSlot()
------- call CheckEvent()
--- in a loop
---- get all completed events via GetLocked() pass them to SendQueue->PushEvent()
-- if done nothing, sleep 100 usec, otherwise nominal sleep to free the evb lock

- GetNext() <- GetLocked() <- EvbThread() - get completed events
-- examine oldest EvbEvent entry in fEvents:
-- pop it if it is complete
-- pop it if it is too old (to prevent old incomplete events from stalling the evb) ("pop age" on the evb page)
-- pop it if a following event is complete ("pop following" on the evb page)
</pre>
* send queue
<pre>
SendQueue class - prepare events for writing to midas SYSTEM event buffer

- PushEvent() <- EvbThread()
-- push FragmentBuf into the queue

- SendQueueThread()
-- in a batch of 1000 events, call Tick()
--- pop FragmentBuf from the queue, call SendEvent()
--- compose midas event (memcpy()!)
--- TMFE fEq->SendEvent()
--- bm_send_event()
-- sleep 100 usec
</pre>
* equipment object
<pre>
EvbEq class - implements the midas equipment
- HandleBeginRun()
- HandleEndRun()
- HandlePeriodic()
</pre>

=== multithreading ===

* BufferReaderThread() one per midas event buffer
** high CPU use: bm_receive_event(), AddXxxBank()
** lock contention: midas event buffer lock, evb lock into XFlushBank->AddBankLocked()
* EvbThread()
** O(size of fEvents): FindEvent(), GetNext()
** lock contention: evb lock against XFlushBank(), SendQueue lock (1 per output event)
* SendQueueThread()
** high CPU use: memcpy() in SendEvent(), memcpy() in bm_send_event
** lock contention: event queue lock against EvbThread, midas event buffer lock
* main thread
** lock contention: evb lock in ReportEvb() and elsewhere

= ODB entries =

== /Equipment/CTRL/Settings ==
* TBW

== /Equipment/CTRL/Settings/TrigSrc - trigger source selection ==
* TrigPulser - trigger on the pulser
* TrigEsataNimGrandOr - trigger on external NIM inputs of GRIF-16 ADC modules
* TrigAdc16GrandOr - trigger on any adc16 signals (see adc16_masks)
* TrigAdc32GrandOr - trigger on any adc32 signals (see adc32_masks)
* Trig1ormore - trigger on adc16 multiplicity 1 or more
* Trig2ormore - ditto, 2 or more
* Trig3ormore - ditto, 3 or more
* Trig4ormore - ditto, 4 or more
* TrigAdcGrandOr - trigger on any ADC signal
* TrigAwCoincA - trigger on TPC AW coincidence (see [[TRG]] manual)
* TrigAwCoincB - ditto
* TrigAwCoincC - ditto
* TrigAwCoincD - ditto
* TrigAwCoinc - ditto
* TrigAw1ormore - trigger on TPC AW per-preamp multiplicity 1 or more (see [[TRG]] manual)
* TrigAw2ormore - ditto, 2 or more
* TrigAw3ormore - ditto, 3 or more
* TrigAw4ormore - ditto, 4 or more
* TrigAwMLU - trigger on TPC AW MLU signal (see [[TRG]] manual)
* TrigBscGrandOr - trigger on any BSC signal
* TrigBscMult - trigger on predefined BSC multiplicity (see [[TRG]] manual)
* TrigCoinc - trigger on predefined coincidence of TPC AW, BSC and external signal (see [[TRG]] manual)

== /Equipment/CTRL/Settings/TRG - trigger settings ==
* Enable - if set to "n", fectrl will not do anything with the trigger board. Normal value "y"
* Modules - hostnames of the trigger board. Normal value "alphat01"
* NimMask - 32-bit mask for the 32x NIM inputs (see TRG manual). Normal value 0, unless external trigger is in use.
* EsataMask - 32-bit mask for the 32x eSATA signals (see TRG manual). Normal value 0.
* adc16_mask[16] - 16x 16-bit mask for the 100MHz GRIF-16 onboard ADCs (see TRG manual). Normal value 0xFFFF for all 16 data links.
* adc32_mask[16] - 16x 32-bit mask for the 62.5MHz FMC-ADC32 ADCs (see TRG manual). Normal value 0xFFFFFFFF for all 16 data links.
* PassThrough - if set to "y", trigger is passed through the trigger module without generating any events and without causing deadtime.
* AwCoincA, B, C and D - 32-bit masks for the aw16 coincidence trigger (see TRG manual)
* Aw16FromAdc16 - route AW trigger (aw16) from adc16 (100MHz ADCs 0..15)
* Aw16FromAdc32a - route AW trigger (aw16) from adc32 group 15..0 (62.5MHz FMC-ADC32 in ADCs 0..7)
* Aw16FromAdc32b - route AW trigger (aw16) from adc32 group 31..16 (62.5MHz FMC-ADC32 in ADCs 8..15)

== /Equipment/CTRL/Settings/PWB - PWB settings ==
* Enable - if set to "n", fectrl will not do anything with PWBs. Normal value "y".
* enable_boot_user_page - if set to "n", fectrl will not try to reboot PWBs into the user page firmware. Normal value "y".
* enable_trigger - if set to "n", all PWBs will be set to ignore the trigger. Normal value "y".
* enable_trigger_group_a - field wire pulser defeats the data suppression (every channel has a pulse) resulting in too much data overwhelming the DAQ (lost UDP packets). To permit correct data taking of field wire pulser data, PWBs are split into two groups (a and b), only one of the two groups should be enabled for field wire pulser runs. Both groups should be enabled for normal data taking. Normal value is "y"
* enable_trigger_group_b - see "enable_trigger_group_a". Normal value is "y".
* modules[64] - hostnames of PWB boards
* boot_user_page[64] - if set to "y" and enable_boot_user_page is "y", fectrl will reboot the corresponding PWB to the user page firmware.
* trigger[64] - if set to "y" and enable_trigget is "y", fectrl will set the PWB to accept the trigger.
* ch_enable - if set to "n" disables all PWB channels. Normal value is "y".
* ch_force - is set to "y" disables channel suppression (all PWB channels are read)
* suppress_reset - if set to "y" enables channel suppression for reset channels (threshold has to be set correctly)
* suppress_fpn - if set to "y" enables channel suppression for fpn channels (threshold has to be set correctly)
* suppress_pads - set to "y" enables channel suppression for TPC pad channels (threshold has to be set correctly)
* disable_reset1 - data suppression algorithm does not work for the channel "reset1" and it has to be suppressed explicitly by setting this value to "y". Normal value is "y".
* baseline_{reset,fpn,pads}[64] - waveform baseline for each PWB position
* threshold_{reset,fpn,pads} - waveform suppression threshold, see below:
* ch_threshold - PWB sca_N_ch_threshold, if set to 0, threshold is computed as "baseline_pads[seqpwb]-threshold_pads". Normal value is 0.
* test_mode - enable PWB test mode, the ADC data is replaced with a test pattern, see PWB esper variable signalproc/test_mode

== /Equipment/CTRL/Settings/TDC - TDC settings ==
* Enable - if set to "n", fectrl will tell the event builder that the TDC is not running. Normal value is "y".

== /Equipment/CTRL/EvbConfig - EVB configuration ==
All arrays have the same size corresponding to the number of EVB slots.
* name[] - slot name for the web page (trg01, adc01, adc01/adc32, pwb31/0, tdc01, etc)
* type[] - slot type: 1=TRG, 2=ADC16 and ADC32, 3=obsolete PWB, 4=obsolete PWB, 5=PWB, 6=TDC
* module[] - module number (adcNN, adcNN+100, pwbNN, etc)
* nbanks[] - number of expected banks (16 for ADC16, 32 for ADC32, not used, set to 1 for all others)
* tsfreq[] - timestamp frequency in Hz.

= Trigger configuration =

These trigger modes are implemented:
* (manual trigger from web page)
* software pulser trigger (from fectrl)
* hardware pulser trigger (from the trigger board)
* NIM and eSATA trigger from ADC front panel inputs
* adc16 and adc32 discriminator triggers:
** adc16 grand-or trigger
** adc32 grand-or trigger
** adc (adc16+adc32) grand-or trigger
** adc16 multiplicity trigger (1ormore, 2ormore, 3ormore, 4ormore) for use with barrel scintillator
** anode wire multiplicity trigger (1ormore, 2ormore, 3ormore, 4ormore) (anode wire signals can be fed from adc16 or from adc32 signals)
** anode wire coincidence trigger
** anode wire MLU (memory lookup) trigger

== NIM trigger from ADC front panel ==

Trigger path:
<pre>
NIM signal --> LEMO input on ADC front panel --> data encoder --> data link to trigger board --> data decoder --> mask --> nim-esata-grand-or --> trigger
</pre>

To enable NIM trigger, do this:
* connect NIM signal to ADC front panel input
** Note1: NIM signal must be inverted
** Note2: LEMO connectors should be set to "NIM input" mode by ADC on-board jumpers (see ADC manual)
* observe correct bit goes to zero in the TRG web page: data link high work should change:
<pre>
0xNF00'0000 (N is the module id) to
0xN700'0000 (left lemo) or
0xNB00'0000 (right lemo)
</pre>
* Note3: if data link bits do not change, most likely the LEMO connectors are set to "DAC output" mode. Try to use LEMO inputs of a different ADC.
* compute the NIM mask: there are 32 bits, 2 bits per ADC data link. Bit 0x1 is right LEMO, bit 0x2 is left LEMO
* for example, to use left LEMO of ADC data link 12, set nim mask to (0x2)<<(2*12) = 0x02000000
* development branch of the git repository contains ''scripts/nim_mask.py'', which can compute forwards and backwards
* initialize the trigger board (on the TRG web page press button "initialize"), or start a new run
* on the history plot with "NIM grand or" counter, the rate should change from zero
* if a run is active, if one of the ADC LEMO outputs is set to output the trigger signal as a NIM pulse, one should see it fire use a scope. The delay or jitter between NIM trigger in and NIM trigger out should be around 100-200 ns (FIXME - measure this!)
* to set the DAQ to trigger on the NIM signal, on the run start page, check the box "TrigEsataNimGrandOr".
** Note: remember to set the scaledown to 0 if so desired.

= TPC field wire pulser =

The TPC field wire pulser input should be connected directly to the DAC output of adc01 (right lemo connector).

DAC settings in ODB /Equipment/CTRL/Settings/ADC/DAC/

* for pulsing PWBs (maximum possible pulse height in PWBs. Overdrives the AW ADC waveforms)
** dac_baseline: -8000
** dac_amplitude: 8000
** ramp_up_rate: 128
** ramp_down_rate: 70
** ramp_top_len: 128

* for pulsing AW: (AW ADC waveforms are in range. PWB waveforms are too small for goor time resolution)
** dac_baseline: -2000
** dac_amplitude: 4000
** ramp_up_rate: 20
** ramp_down_rate: 20
** ramp_top_len: 128

= TDC connections =

* SFP8 - fiber SFP to Juniper switch
* right RJ45 - 200MHz clock
* left RJ45 - external trigger

= MIDAS Data banks =

* ATAT - fectrl - UDP packets from GRIF-C/TRG trigger board
* TRBA - fetdc - TDC data
* TRTM - fetdc - TDC FPGA temperatures, TID_FLOAT[5].
* AAnn - feudp - UDP packets from GRIF-16 ADC (before the event builder), "nn" is the ADC module number: adc01 is AA01 through adc20 is AA20.
* BBxx - feudp - UDP packets from PWB, obsolete
* Bnnm, Cnnm - feevb - EVB output banks: ADC data, "nn" is the ADC module number (1..20), "m" is the channel number (0..9,A..F), format is same as AAnn banks.
* PAnn - feudp - UDP packets from PWB, "nn" is the PWB module number (00..99), obsolete
* PBnn - feudp - UDP packets from PWB, "nn" is the PWB module number (00..99), PB01 is pwb01, PB78 is pwb78
* PCnn - feevb - EVB output banks: PWB data, "nn" is the PWB module number (00..99), same as above, format is same as PBnn banks

= ZZZ =

ZZZ

Daq

2021-02-09T23:18:55Z

Acapra: /* Schematic */

= Links =

* https://alphacpc05.cern.ch/agdaq/ -- midas daq page for the agdaq system at CERN
* https://daq16.triumf.ca -- midas daq page for the agmini system at TRIUMF
* https://daq.triumf.ca/elog-alphag/alphag/ -- elog
* https://daqstore.triumf.ca/AgWiki/index.php/Main_Page -- this wiki
* https://bitbucket.org/ttriumfdaq/agdaq - git repository for agdaq
* https://bitbucket.org/expalpha -- git repository on bitbucket
* https://edev-group.triumf.ca/fw/exp/alphag -- git repository on edev gitlab
* https://ladd00.triumf.ca/daqinv/frontend/list/170 - PWB inventory database
* https://ladd00.triumf.ca/daqinv/frontend/list/168 - ADC (GRIF-16) inventory, FMC-ADC32rev0,rev1 inventory
* https://ladd00.triumf.ca/daqinv/frontend/list/176 - FMC-ADC32rev1.1 inventory

= Hardware manuals =

* [[TRG]] - trigger board (GRIF-C) manual
* CDM - https://daq.triumf.ca/DaqWiki/index.php/VME-CDM -- clock VME-CDM manual
* ADC - https://daq.triumf.ca/DaqWiki/index.php/VME-GRIF-ADC16-Rev1 -- GRIF-16 ADC manual
* [[PWB]] - TPC Pad Wing Board manual
* TDC - https://www.triumf.info/wiki/DAQwiki/index.php/GSI_TRB3
* [[chronobox]] - chronobox manual

= Performance =

Full system: (ADC fw rel-20201112-ko, PWB fw 20201005_ko)
* 1 Hz stable, ADC adc32 9 supp off rate 0.3 M/s, adc16 off, PWB udp_delay 25 supp off active 64 rate 6.6+4+6+4.6=21.2 M/s, TDC off, total rate 21 M/s
* 500 Hz stable, ADC adc32 9 supp 1500 rate 6 M/s, adc16 off, PWB udp_delay 10 supp 100 active 63 rate 3+4+3+3=13 M/s, TDC off, total rate 19 M/s

Single ADC: (fw rel-20201112-ko)

Fully suppressed data, sthreshold 5000
* 1 Hz: stable
* 500 Hz: stable (nominal max trigger rate)
* 6700 Hz: stable, 8 Mbytes/sec
* 6800 Hz: evb eventually confused
* 6850 Hz: evb confused, no lost triggers in the ADC, no crash in the ADC

All data, 510 samples/event, sthreshold 10 (fw rel-20201122-ko)
* 1 Hz: stable
* 100 Hz: stable, 3.4 Mbytes/sec
* 500 Hz: stable, 17 Mbytes/sec (nominal max trigger rate)
* 1000 Hz: stable, 32 Mbytes/sec
* 3100 Hz: stable, 106 M/s (approaching saturation of 1gige ethernet)
* 3200 Hz: stable, 110 M/s - TRG communication is in trouble
* 3300 Hz: stable, 116 M/s
* 3400 Hz: incomplete events, no ADC crash ("dropped due to full" counting, "dropped due to busy" are zero, this is as expected).

no suppression (different fpga data path)
* 1 Hz: stable
* 100 Hz: stable
* 500 Hz: stable, 17 M/s (nominal max trigger rate)
* 1000 Hz: stable, 32 M/s
* 3000 Hz: stable, 100 M/s (approaching saturation of 1gige ethernet)
* 3300 Hz: incomplete events, no ADC crash ("dropped due to full" counting, "dropped due to busy" are zero, this is as expected).

= Network connections =

The experiment private network number is: 192.168.1.x, gateway 192.168.1.1, netmask 255.255.255.0

network map:
<pre>
---- TRIUMF/CERN network --1gige-- alphagdaq --10gige-- juniper switch --1gige-- centrecom switch
</pre>

== alphagdaq network connections ==

on the rear of the machine:
<pre>
-----------------------------------------
| | rj45 | rj45 | | sfp | sfp |
-----------------------------------------

- left rj45 - copper 1gige - eno1 - spare (inactive)
- right rj45 - copper 1gige - eno2 - dhcp - cern/triumf network
- left sfp - 10gige - enp1s0f1 - spare (inactive)
- right sfp - 10gige - enp1s0f0 - static 192.168.1.1
</pre>

== connections between switches ==

* alphagdaq to juniper switch: direct attach (DAC) 10gige cable into any port of juniper switch
* juniper switch to centrecom switch: rj45 sfp in any port of juniper switch - cat5e/cat6 cable - to any port of centrecom switch
* cat5e/cat6 cable from any port of centrecom switch to the XXX port on the back of the juniper switch

== juniper switch ==

connected to juniper switch front is:

* everything that sends event data to alphagdaq, specifically:
** TRG (rj45 sfp)
** 16x ADC (fiber sfp)
** 64x PWB (fiber sfp)
* 10gige uplink to alphagdaq (10gige DAC cable)
* 1gige link to centrecom switch (rj45 sfp)
* 40gige ports are not used

connected to the juniper switch rear:

* CON is the serial console via RJ45-to-DB9 to USB-serial adapter to alphagdaq USB port (9600bps, 8N1, no flow control)
* C0 is the management ethernet interface port, connect to any port on the centrecom switch (juniper-private will ping)

== centrecom switch ==

* everything with copper rj45 connections, specifically:
** CDM boards
** HV, LV and VME power supplies
** RaspberryPi3 boards
** gas handling MFCs (algas)
** cooling system controller (moxa01)

= USB connections =

Note:
* alphagdaq has USB2 and USB3 connections. They are not the same, they are not generally interchangeable.
* USB3 connections and cables are generally blue coloured.

List of USB connections from alphagdaq:

* front USB3 port (blue): USB-A-to-B cable to USB blaster connected to the trigger board (front of rack)
* rear USB3 port (blue): USB3 short jumper to USB3 hub
* rear USB2 port (black): USB2 unpowered hub (Logiix) to USB-A-to-B cable to UPS USB connector
* rear USB2 port (black): USB-to-Serial adaper to DB9 to RJ45 adapter to Juniper switch console port

Note: UPS USB connection does not work with alphagdaq USB ports (both usb2 and usb3), but combination
using the unpowered USB2 hub seems to work ok.

List of USB connections from the USB hub:

* this is a powered USB3 hub
* USB-A-to-MicroUSB - to lvdb boards
* USB-A-to-Wiener interlock cable

= Clock and trigger distribution =

== Explanation ==

Trigger is generated by the trigger board (aka TRG, aka GRIF-C),
from the TRG eSATA output through the eSATA splitter it is fed into
the master CDM. The master CDM outputs the trigger on all MiniSAS output ports.
One of the master CDM MiniSAS outputs is connected to the CDM slave eSATA input.
The same as the master CDM, the slave CDM outputs the trigger on all MiniSAS output ports.

The master 62.5MHz clock is generated by the master CDM using the internal 10MHz oscillator
or the external 10MHz AD clock. Also on MiniSAS port 6 the master CDM generates
the 200MHz clock for the TDC. This clock is frequency-locked to the master 62.5MHz clock.

Master CDM clock selection (external CD 10MHz clock or internal oscillator 10MHz clock)
is done by a magic esper command (see below).

From the master CDM, the clock is fed to: the slave CDM, the TRG (via the eSATA splitter)
and to the TDC (via the RJ45 splitter).

The slave CDM sends the clock and the trigger to the ADCs and PWBs.

== Schematic ==

<pre>
TRG (GRIF-C)
------------
eSATA <------> eSATA splitter

eSATA splitter
--------------
trigger ---> eSATA ---> CDM-Master eSATA (trigger signal)
clock <--- (eSATA --- MiniSAS) <--- CDM MiniSAS (62.5MHz clock)
</pre>

<pre>
CDM-Master
----------
eSATA <--- eSATA splitter <--- eSATA from TRG (trigger signal)
MiniSAS 1 --> (MiniSAS --- eSATA) --> CDM-Slave eSATA (62.5 MHz)
MiniSAS 6 --> (MiniSAS --- eSATA) --> eSATA splitter --> RJ45 --> TDC (200 MHz)
</pre>

<pre>
CDM-Slave
---------
eSATA <--- trigger and 62.5MHz clock from CDM-Master
MiniSAS 1 --> ADC trigger and 62.5MHz clock
MiniSAS 2 --> same
MiniSAS 3 --> same
MiniSAS 4 --> same
MiniSAS 5 --> PWB trigger and 62.5MHz clock
MiniSAS 6 --> same
</pre>

[[File:VME_Crate_CERN.jpeg|frameless|left|Layout of the VME crate at CERN. The ALPHA-T module is in the rightmost slot. The Master CDM is next to it and it's labelled as #02. The Slave CDM is on the lefmost position and labelled by #03]]

== Setup of CDM boards ==

ALPHA-g uses the Rev3 CDM boards. The 4 CDM boards that belong to ALPHA-g
have serial numbers 0, 1, 2 and 3 and are labeled cdm00, cdm01, cdm02, cdm03.

All the boards can be used as master and slave positions, but have to be
configured appropriately.

=== Slave setup ===

* install the board
* connect ethernet
* connect esata external clock
* esper-tool cdmNN ### i.e. esper-tool cdm01
* cd template
* write current_setup 4
* cd /cdm
* read
* observe clock frequency counters report correct values. "ext_clk" will only count if the lemo jumper is installed (see below)
<pre>
11 esata_clk uint32 R [62500246]
12 ext_clk uint32 R [62500246]
</pre>
* cd /lmk
* read
* observe both PLL1 and PLL2 are locked: both values should be "1"
<pre>
144 pll1_ld uint8 R [1]
147 pll2_ld uint8 R [1]
156 ld1_counter uint32 RW [3]
157 ld2_counter uint32 RW [3]
</pre>

If needed, check correct operation of the pll lock monitoring:

* unplug the esata cable from the master CDM
* "red" light should go off
* read of lmk should report "pll1_ld" value 0 and "ld1_counter" and "ld2_counter" should increment
* reconnect the esata cable
* "red" light should return
* read of lmk should report both pll1_ld and pll2_ld locked (values "1").

=== Master setup ===

* install the board
* connect ethernet
* esper-tool cdmNN ### i.e. esper-tool cdm01
* cd template
* write current_setup 3
* cd /lmk
* write clkin_sel_mode 0
* cd /cdm
* read
* observe clock frequency counters report correct values. "ext_clk" will only count if the "clock loopback" lemo jumper is installed (see below)
<pre>
12 ext_clk uint32 R [62500189]
</pre>
* cd /lmk
* read
* observe both PLL1 and PLL2 are locked: both values should be "1"
<pre>
144 pll1_ld uint8 R [1]
147 pll2_ld uint8 R [1]
156 ld1_counter uint32 RW [3]
157 ld2_counter uint32 RW [3]
</pre>

Check the 200MHz clock:
* connect MiniSAS-eSATA splitter cable to MiniSAS output 6, connect eSATA end to CDM eSATA input
* in esper-tool, read cdm: esata_clk should report 200MHz clock
<pre>
11 esata_clk uint32 R [200000603]
12 ext_clk uint32 R [62500188]
</pre>

=== Master setup with external 10 MHz clock ===

* setup master CDM as above (using internal clock)
* connect 10MHz external clock to right LEMO #1
* esper-tool cdmNN
* cd /cdm
* write sel_nim 1 # if clock is NIM signal
* write sel_nim 0 # if clock is TTL signal (i.e. chronobox LEMO output)
* read ext_clk
<pre>
[http://cdm00:/cdm]> read ext_clk
9999556
</pre>
* if ext_clk reads 0 or does not read something close to 10MHz, stop here. debug the signal, try changing sel_nim, etc.
* cd /lmk
* write clkin_sel_mode 2 # select external clock
* read
* observe both PLL1 and PLL2 are locked: both values should be "1"
<pre>
150 clkin2_sel bool R [True]
144 pll1_ld uint8 R [1]
147 pll2_ld uint8 R [1]
</pre>
* on the MIDAS TRG page, observe that TRG measured external clock frequency is close to 62.5 MHz

== clkin_sel_mode positions ==

* 0 = 10MHz internal oscillator (use on master CDM in standalone mode)
* 1 = eSATA clock (use on slave CDM)
* 2 = external clock, should be 10MHz (use on master CDM to run from AD clock)

== LEMO connections ==

* clock loopback jumper: to read the frequency of the generated 62.5MHz clock, connect LEMO cable from left LEMO #2 to the right LEMO #1
* 10MHz internal clock jumper: to feed 10MHz internal oscillator to the external clock, connect LEMO cable from right LEMO #3 to the right LEMO #1.
* 10MHz external clock: to feed 10MHz external clock, connect LEMO cable to right LEMO #1

= MIDAS frontends =

== UDP ==

feudp receives udp packets from the ADC and PWB boards. For each udp packet 1 midas event is created,
with 1 data bank containing the udp data. Events are sent into the MIDAS event buffer BUFUDP. The names
of the data banks are assigned in ODB /eq/feudp/settings.

{ADC,PWB} --> 1gige --> switch --> 10gige --> alphagdaq --> feudp -> BUFUDP

== CTRL ==

fectrl controls all the ADC, PWB and trigger boards. It provides run start and stop sequencing,
runs the event synchronization sequence at the begin of the run, and records slow controls data (temperatures,
voltages, etc).

fectrl receives UDP data packets from the trigger board and sends them to the MIDAS event buffer BUFUDP.

fectrl receives commands from the control web pages (for ADC, PWB and trigger) to reboot, initialize, etc the boards.

fectrl configures the event builder via odb /eq/fectrl/evbconfig.

ADC <-> http esper <-> fectrl -> slow control data into midas history

PWB <-> http esper <-> fectrl -> slow control data into midas history

TRG <-> udp comm <-> fectrl -> BUFUDP, slow control and counters into midas history

fectrl <-> midas rpc <-> mhttpd <-> json rpc <-> control web pages for ADC, PWB and trigger

== EVB ==

feevb receives udp packet data from the MIDAS event buffer BUFUDP, inspects the packet timestamps
and collects the data with matching timestamps into physics events. feevb has provisions to do
data suppression, reduction and compression in addition to the data reduction done
in the ADC and PWB firmware.

feevb event synchronization relies on the event synchronization sequence done by fectrl when a run is started.

{ADC, PWB, TRG} -> BUFUDP -> feevb -> SYSTEM -> mlogger -> compression -> disk storage

=== internal structure ===

* event buffer readers, instances for BUFTRG, BUFTDC, BUFADC, BUFUDP
<pre>
BufferReader class - reads one midas event buffer:
- BufferReaderThread()
-- in batches of 1000 events
--- lock
--- TickLocked()
----- ReadEvent() -> bm_receive_event_alloc()
----- fHandler->HandleEvent()
-- fHandler->XMaybeFlushBank()
-- sleep 100 usec
</pre>
* event header decoder and intermediate buffer
<pre>
Handler class - accumulates events, sends them to the EVB in batches

- HandleEvent() <- BufferReaderThread() - receive incoming events
-- loop over all banks, examine bank name
--- call AddAdcBank(), AddPwbBank(), AddTrgBank(), AddTdcBank() according to bank name
---- AddXxxBank() -> XAddBank()
--- unknown banks go directly to evb->SendQueue
-- check if synchronization has completed

- XAddBank() - save BankBuf objects in a buffer

- XMaybeFlushBank(), XFlushBank() - send all accumulated data to the Evb class
-- evb->lock()
-- for each buffered BankBuf objects,
--- call evb->AddBankLocked()
-- buffer is now empty
</pre>
* event builder
<pre>
Evb class - assemble event fragments according to timestamps

- fEvents fifo (deque) - time sorted list of partially assembled events
-- new entries are added by FindEvent()
-- entries are modified by MergeSlot()
-- old entries are removed by GetLocked()

- AddBankLocked() <- XFlushBank() <- BufferReaderThread()
-- call fSync->Add()
-- transform BankBuf to EvbEventBuf
-- push it into per-module buffer fBuf[imodule]

- EvbThread()
-- EvbTickLocked()
--- BuildLocked()
---- in a loop, for no more than 1 sec
----- loop over all per-module buffers fBuf
------ pop first entry, pass it to BuildSlot()
------- call FindEvent()
------- call MergeSlot()
------- call CheckEvent()
--- in a loop
---- get all completed events via GetLocked() pass them to SendQueue->PushEvent()
-- if done nothing, sleep 100 usec, otherwise nominal sleep to free the evb lock

- GetNext() <- GetLocked() <- EvbThread() - get completed events
-- examine oldest EvbEvent entry in fEvents:
-- pop it if it is complete
-- pop it if it is too old (to prevent old incomplete events from stalling the evb) ("pop age" on the evb page)
-- pop it if a following event is complete ("pop following" on the evb page)
</pre>
* send queue
<pre>
SendQueue class - prepare events for writing to midas SYSTEM event buffer

- PushEvent() <- EvbThread()
-- push FragmentBuf into the queue

- SendQueueThread()
-- in a batch of 1000 events, call Tick()
--- pop FragmentBuf from the queue, call SendEvent()
--- compose midas event (memcpy()!)
--- TMFE fEq->SendEvent()
--- bm_send_event()
-- sleep 100 usec
</pre>
* equipment object
<pre>
EvbEq class - implements the midas equipment
- HandleBeginRun()
- HandleEndRun()
- HandlePeriodic()
</pre>

=== multithreading ===

* BufferReaderThread() one per midas event buffer
** high CPU use: bm_receive_event(), AddXxxBank()
** lock contention: midas event buffer lock, evb lock into XFlushBank->AddBankLocked()
* EvbThread()
** O(size of fEvents): FindEvent(), GetNext()
** lock contention: evb lock against XFlushBank(), SendQueue lock (1 per output event)
* SendQueueThread()
** high CPU use: memcpy() in SendEvent(), memcpy() in bm_send_event
** lock contention: event queue lock against EvbThread, midas event buffer lock
* main thread
** lock contention: evb lock in ReportEvb() and elsewhere

= ODB entries =

== /Equipment/CTRL/Settings ==
* TBW

== /Equipment/CTRL/Settings/TrigSrc - trigger source selection ==
* TrigPulser - trigger on the pulser
* TrigEsataNimGrandOr - trigger on external NIM inputs of GRIF-16 ADC modules
* TrigAdc16GrandOr - trigger on any adc16 signals (see adc16_masks)
* TrigAdc32GrandOr - trigger on any adc32 signals (see adc32_masks)
* Trig1ormore - trigger on adc16 multiplicity 1 or more
* Trig2ormore - ditto, 2 or more
* Trig3ormore - ditto, 3 or more
* Trig4ormore - ditto, 4 or more
* TrigAdcGrandOr - trigger on any ADC signal
* TrigAwCoincA - trigger on TPC AW coincidence (see [[TRG]] manual)
* TrigAwCoincB - ditto
* TrigAwCoincC - ditto
* TrigAwCoincD - ditto
* TrigAwCoinc - ditto
* TrigAw1ormore - trigger on TPC AW per-preamp multiplicity 1 or more (see [[TRG]] manual)
* TrigAw2ormore - ditto, 2 or more
* TrigAw3ormore - ditto, 3 or more
* TrigAw4ormore - ditto, 4 or more
* TrigAwMLU - trigger on TPC AW MLU signal (see [[TRG]] manual)
* TrigBscGrandOr - trigger on any BSC signal
* TrigBscMult - trigger on predefined BSC multiplicity (see [[TRG]] manual)
* TrigCoinc - trigger on predefined coincidence of TPC AW, BSC and external signal (see [[TRG]] manual)

== /Equipment/CTRL/Settings/TRG - trigger settings ==
* Enable - if set to "n", fectrl will not do anything with the trigger board. Normal value "y"
* Modules - hostnames of the trigger board. Normal value "alphat01"
* NimMask - 32-bit mask for the 32x NIM inputs (see TRG manual). Normal value 0, unless external trigger is in use.
* EsataMask - 32-bit mask for the 32x eSATA signals (see TRG manual). Normal value 0.
* adc16_mask[16] - 16x 16-bit mask for the 100MHz GRIF-16 onboard ADCs (see TRG manual). Normal value 0xFFFF for all 16 data links.
* adc32_mask[16] - 16x 32-bit mask for the 62.5MHz FMC-ADC32 ADCs (see TRG manual). Normal value 0xFFFFFFFF for all 16 data links.
* PassThrough - if set to "y", trigger is passed through the trigger module without generating any events and without causing deadtime.
* AwCoincA, B, C and D - 32-bit masks for the aw16 coincidence trigger (see TRG manual)
* Aw16FromAdc16 - route AW trigger (aw16) from adc16 (100MHz ADCs 0..15)
* Aw16FromAdc32a - route AW trigger (aw16) from adc32 group 15..0 (62.5MHz FMC-ADC32 in ADCs 0..7)
* Aw16FromAdc32b - route AW trigger (aw16) from adc32 group 31..16 (62.5MHz FMC-ADC32 in ADCs 8..15)

== /Equipment/CTRL/Settings/PWB - PWB settings ==
* Enable - if set to "n", fectrl will not do anything with PWBs. Normal value "y".
* enable_boot_user_page - if set to "n", fectrl will not try to reboot PWBs into the user page firmware. Normal value "y".
* enable_trigger - if set to "n", all PWBs will be set to ignore the trigger. Normal value "y".
* enable_trigger_group_a - field wire pulser defeats the data suppression (every channel has a pulse) resulting in too much data overwhelming the DAQ (lost UDP packets). To permit correct data taking of field wire pulser data, PWBs are split into two groups (a and b), only one of the two groups should be enabled for field wire pulser runs. Both groups should be enabled for normal data taking. Normal value is "y"
* enable_trigger_group_b - see "enable_trigger_group_a". Normal value is "y".
* modules[64] - hostnames of PWB boards
* boot_user_page[64] - if set to "y" and enable_boot_user_page is "y", fectrl will reboot the corresponding PWB to the user page firmware.
* trigger[64] - if set to "y" and enable_trigget is "y", fectrl will set the PWB to accept the trigger.
* ch_enable - if set to "n" disables all PWB channels. Normal value is "y".
* ch_force - is set to "y" disables channel suppression (all PWB channels are read)
* suppress_reset - if set to "y" enables channel suppression for reset channels (threshold has to be set correctly)
* suppress_fpn - if set to "y" enables channel suppression for fpn channels (threshold has to be set correctly)
* suppress_pads - set to "y" enables channel suppression for TPC pad channels (threshold has to be set correctly)
* disable_reset1 - data suppression algorithm does not work for the channel "reset1" and it has to be suppressed explicitly by setting this value to "y". Normal value is "y".
* baseline_{reset,fpn,pads}[64] - waveform baseline for each PWB position
* threshold_{reset,fpn,pads} - waveform suppression threshold, see below:
* ch_threshold - PWB sca_N_ch_threshold, if set to 0, threshold is computed as "baseline_pads[seqpwb]-threshold_pads". Normal value is 0.
* test_mode - enable PWB test mode, the ADC data is replaced with a test pattern, see PWB esper variable signalproc/test_mode

== /Equipment/CTRL/Settings/TDC - TDC settings ==
* Enable - if set to "n", fectrl will tell the event builder that the TDC is not running. Normal value is "y".

== /Equipment/CTRL/EvbConfig - EVB configuration ==
All arrays have the same size corresponding to the number of EVB slots.
* name[] - slot name for the web page (trg01, adc01, adc01/adc32, pwb31/0, tdc01, etc)
* type[] - slot type: 1=TRG, 2=ADC16 and ADC32, 3=obsolete PWB, 4=obsolete PWB, 5=PWB, 6=TDC
* module[] - module number (adcNN, adcNN+100, pwbNN, etc)
* nbanks[] - number of expected banks (16 for ADC16, 32 for ADC32, not used, set to 1 for all others)
* tsfreq[] - timestamp frequency in Hz.

= Trigger configuration =

These trigger modes are implemented:
* (manual trigger from web page)
* software pulser trigger (from fectrl)
* hardware pulser trigger (from the trigger board)
* NIM and eSATA trigger from ADC front panel inputs
* adc16 and adc32 discriminator triggers:
** adc16 grand-or trigger
** adc32 grand-or trigger
** adc (adc16+adc32) grand-or trigger
** adc16 multiplicity trigger (1ormore, 2ormore, 3ormore, 4ormore) for use with barrel scintillator
** anode wire multiplicity trigger (1ormore, 2ormore, 3ormore, 4ormore) (anode wire signals can be fed from adc16 or from adc32 signals)
** anode wire coincidence trigger
** anode wire MLU (memory lookup) trigger

== NIM trigger from ADC front panel ==

Trigger path:
<pre>
NIM signal --> LEMO input on ADC front panel --> data encoder --> data link to trigger board --> data decoder --> mask --> nim-esata-grand-or --> trigger
</pre>

To enable NIM trigger, do this:
* connect NIM signal to ADC front panel input
** Note1: NIM signal must be inverted
** Note2: LEMO connectors should be set to "NIM input" mode by ADC on-board jumpers (see ADC manual)
* observe correct bit goes to zero in the TRG web page: data link high work should change:
<pre>
0xNF00'0000 (N is the module id) to
0xN700'0000 (left lemo) or
0xNB00'0000 (right lemo)
</pre>
* Note3: if data link bits do not change, most likely the LEMO connectors are set to "DAC output" mode. Try to use LEMO inputs of a different ADC.
* compute the NIM mask: there are 32 bits, 2 bits per ADC data link. Bit 0x1 is right LEMO, bit 0x2 is left LEMO
* for example, to use left LEMO of ADC data link 12, set nim mask to (0x2)<<(2*12) = 0x02000000
* development branch of the git repository contains ''scripts/nim_mask.py'', which can compute forwards and backwards
* initialize the trigger board (on the TRG web page press button "initialize"), or start a new run
* on the history plot with "NIM grand or" counter, the rate should change from zero
* if a run is active, if one of the ADC LEMO outputs is set to output the trigger signal as a NIM pulse, one should see it fire use a scope. The delay or jitter between NIM trigger in and NIM trigger out should be around 100-200 ns (FIXME - measure this!)
* to set the DAQ to trigger on the NIM signal, on the run start page, check the box "TrigEsataNimGrandOr".
** Note: remember to set the scaledown to 0 if so desired.

= TPC field wire pulser =

The TPC field wire pulser input should be connected directly to the DAC output of adc01 (right lemo connector).

DAC settings in ODB /Equipment/CTRL/Settings/ADC/DAC/

* for pulsing PWBs (maximum possible pulse height in PWBs. Overdrives the AW ADC waveforms)
** dac_baseline: -8000
** dac_amplitude: 8000
** ramp_up_rate: 128
** ramp_down_rate: 70
** ramp_top_len: 128

* for pulsing AW: (AW ADC waveforms are in range. PWB waveforms are too small for goor time resolution)
** dac_baseline: -2000
** dac_amplitude: 4000
** ramp_up_rate: 20
** ramp_down_rate: 20
** ramp_top_len: 128

= TDC connections =

* SFP8 - fiber SFP to Juniper switch
* right RJ45 - 200MHz clock
* left RJ45 - external trigger

= MIDAS Data banks =

* ATAT - fectrl - UDP packets from GRIF-C/TRG trigger board
* TRBA - fetdc - TDC data
* TRTM - fetdc - TDC FPGA temperatures, TID_FLOAT[5].
* AAnn - feudp - UDP packets from GRIF-16 ADC (before the event builder), "nn" is the ADC module number: adc01 is AA01 through adc20 is AA20.
* BBxx - feudp - UDP packets from PWB, obsolete
* Bnnm, Cnnm - feevb - EVB output banks: ADC data, "nn" is the ADC module number (1..20), "m" is the channel number (0..9,A..F), format is same as AAnn banks.
* PAnn - feudp - UDP packets from PWB, "nn" is the PWB module number (00..99), obsolete
* PBnn - feudp - UDP packets from PWB, "nn" is the PWB module number (00..99), PB01 is pwb01, PB78 is pwb78
* PCnn - feevb - EVB output banks: PWB data, "nn" is the PWB module number (00..99), same as above, format is same as PBnn banks

= ZZZ =

ZZZ

File:VME Crate CERN.jpeg

2021-02-09T23:18:19Z

Acapra: Layout of VME crate at CERN with 8 ALPHA-16

Layout of VME crate at CERN with 8 ALPHA-16

TRG

2021-02-09T22:52:13Z

Acapra:

= TRG, the trigger control board for the ALPHA-g detector =

= Links =

* xxx
* https://bitbucket.org/expalpha/trg_firmware - firmware sources on bitbucket

= General characteristics =

== Photos ==

== Board schematics ==

== Available hardware ==

= ALPHA-T front panel connections =

From top to bottom:

<pre>
|
| minisas-C - links 8-11
| minisas-D - links 12-15
|
| ethernet sfp - use copper sfp only
| minisas - not used, do not connect
| esata - clock in, trigger out
|
| minisas-A - links 0-3
| minisas-B - links 4-7
|
| LEDs
|
</pre>

MiniSAS and SATA flail cable connections:

<pre>
P1 <--> link 3
P2 <--> link 2
P3 <--> link 1
P4 <--> link 0
</pre>

LEDs:

<pre>
--PCB-- (left)
LED[0] - trigger out - ag_trig_out
LED[1] - network packet received - sfp_rx_data_valid
LED[2] - Xilinx FPGA boot ok - FMC1_CFG_DONE
LED[3] - not used
(right)
</pre>

= ALPHA-T onboard jumpers and switches =

* rotary switches: select IP address: 00=192.168.1.50, 11=.51, 22=.52, 33=.53, 44=.54, otherwire=.251

= Firmware update procedure =

* use quartus to generate the grifc.pof file (use grifc.cof)
* open quartus programmer
* attach grifc.pof to the 1st CFI_512 flash
* mark program/configure for the same flash (3 marks: pof, page_0 and OPTION_BITS
* check that "initiate configuration after programming" is set in tools->options
* say "start"
* should take about 5 minutes
* grifc should reboot into the new firmware

If CFI_512 flash is not visible to the quartus programmer, load correct
code into the MaxV CPLD:

* open quartus programmer,
* say "auto detect"
* change file for the 5M2210 device
* use .../boot/maxv_3xfmc_normal.pof from the firmware git repository
* say "program", it should finish in 10-15 seconds
* say "auto detect"
* CFI_512 flash should be visible now.

= Firmware revisions =

= Firmware build =

* use quartus 17.1

= ALPHA-g trigger functions =

Trigger modes:
* software trigger
* pulser trigger
* trigger on GRIF16 ADC NIM and ESATA inputs ("external trigger")
* adc16 grand-or
* adc32 grand-or
* adc16 wire multiplicity
* TPC anode wire per-preamp trigger (aw16 section)
** trigger on coincidences of preamps
** trigger on multiplicity of preamps
** trigger on specific topology of preamps (multiplicity of preamp clusters, etc) via the aw16_mlu trigger
* BSC trigger
** BSC grand_or
** BSC bar multiplicity
* coincidence trigger - coincidence of external, TPC AW and BSC triggers

== software trigger ==

to programmatically fire the trigger, one should:
* set the conf_enable_sw_trigger bit in conf_trig_enable, this will permit the software trigger to fire the trigger
* write value 0x4 to register 0x2B, this will cause the trigger to fire

== pulser trigger ==

To use the pulser trigger, one should program:
* pulser period (in clocks)
* pulser width (in clocks, use value 5, the pulser no longer goes outside of the trigger board, so changing the width does not do anything)
* set the run_pulser bit in conf_trig_enable, this will cause the pulser to run
* set the enable_pulser bit in conf_trig_enable, this will cause the pulser to fire the trigger
* set the output_pulser bit in conf_trig_enable, this will cause the pulser to be output from the alphag firmware block (the pulser no longer goes outside of the trigger board, so changing the bit does not do anything)

== timeout trigger ==

The event builder uses event timestamps to assemble physics events. The timestamps
are usually 32 bit wide and run at 125 MHz (8 ns) or 62.5 MHz (16 ns). If
triggers are generated often (1 per second or more), timestamp wrap around
is easily detected by the event builder. If triggers come infrequently,
timestamp wrap around may occur while there are no triggers, not be seen
by the event builder causing loss of event synchronization.

To avoid this problem in the event builder, a timeout trigger is generated
if no triggers were generated inside a 10 second interval.

The guaranties that even if all physics triggers temporarily stop for any reason,
the event builder will stay synchronized. (0.1 Hz timeout trigger frequency).

== adc16 multiplicity trigger ==

This trigger is intended for use with the barrel scintillator signals connected to adc16 digitizers. It does not work well with TPC AW signals.

The 16 adc16 trigger bits (16x16=256) are added together to form the hit multiplicity value (adc16_mult, 16 bits, actual range 0 to 256).

Four threshold signals are computed: adc16_mult_1ormore = adc16_mult>=1, 2ormore, 3ormore, 4ormore.

There is a counter for each of these four signals.

Each of these four signals can fire the trigger, see conf_trig_enable bits.

== TPC anode wire triggers ==

TBW: mapping of adc16 and adc32 into aw16[15:0] signals

=== aw16_grand_or ===

TPC AW "grand or" trigger: use aw16_mult_1ormore

=== aw16_mult ===

TPC AW preamp multiplicity trigger: counts how many preamps
have hits, fires:
* aw16_mult_1ormore (same as aw16_grand_or)
* aw16_mult_2ormore
* aw16_mult_3ormore
* aw16_mult_4ormore

(256 anode wires, 16 wires per preamp, 16 preamps, 16 aw16 signals)

=== aw16_coinc ===

TBW: TPC AW preamp coincidence trigger

=== aw16_mlu ===

TPC AW preamp MLU trigger: the MLU (memory lookup
table) implements an arbitrary logical function
to map the 16 aw16 preamp signals into 1 aw16_mlu_trig trigger output.

Description of the aw16 mlu trigger.

In words:

* any aw16 signal starts the aw16_prompt_run gate
* aw16 signals are collected for the duration of conf_mlu_prompt (125MHz/8ns clock)
* this corresponds to time spread of TPC AW signals due to the drift time in the AW drift cell and to the variation in signal transmission time from the ADC to the TRG (digital trigger data link).
* when this prompt time expires, the MLU looks at the accumulated aw16_prompt bit pattern and makes the "yes/no" trigger decision
* after the trigger decision is made, we wait for the TPC to become quiet, meaning there is no aw16 signals for the duration of the aw16_prompt_wait gate (conf_mlu_wait), if any signals arrive during the gate, they restart the gate.
* after the TPC has been quiet for conf_mlu_wait time (125MHz/8ns clock), the MLU trigger is ready for the next event.

In logic: (see alphag.v)

* aw16_or <= grand-or of 16 aw16 signals
* "aw16_or" starts the "aw16_prompt_cnt" counter, opens the aw16_prompt_run gate
* during the aw16_prompt_run gate, aw16_prompt collects (latches) the aw16 bits (aw16_prompt <= aw16_prompt | aw16)
* this is the time when we wait for all "prompt" signals from the TPC to arrive into the trigger
* at the end of the aw16_prompt_run gate, the signal aw16_prompt_out is pulsed, latching the output of the MLU into aw16_mlu_trig
* this is the MLU trigger "yes/no" trigger decision.
* after the MLU trigger decision is made, we wait for the arrival and passing of all TPC drift signals
* the aw16_prompt_wait_cnt and aw_prompt_wait gate opens
* if any new aw16 signals arrive (they are TPC drift signals at this time), the aw16_prompt_wait gate is restarted
* if no aw16 signals arrive during the full run of the aw16_prompt_wait gate ("TPC quiet time")
* the aw16_prompt_wait and aw16_prompt gates are cleared and the MLU trigger is ready for the next event.

== BSC trigger ==

TBW

== Coincidence trigger ==

Coincidence trigger inputs:

<pre>
assign coinc_input[0] = aw16_mult_1ormore;
assign coinc_input[1] = aw16_mlu_trig;
assign coinc_input[2] = bsc_grand_or_trig;
assign coinc_input[3] = bsc_mult_trig;
assign coinc_input[4] = esata_nim_grand_or;
assign coinc_input[5] = 0;
assign coinc_input[6] = 0;
assign coinc_input[7] = 0;
</pre>

Coincidence trigger "start" mask selects which of the inputs can start the coincidence window.

Coincidence trigger "require" mask selects which inputs must have a hits for the coincidence ot be satisfied.

Input hits are collected for the duration of the coincidence window, if all required inputs have hits, coinc_trig is fired.

Example settings:

* coincidence of external scintillator, BSC and TPC AW "grand or" signals: start 0x10, required 0x15
* coincidence of BSC and TPC AW "grand or" signals: start 0x4, required 0x5
* coincidence of BSC and TPC AW MLU triggers: start 0x4, required 0x6
* coincidence of BSC multiplicity and TPC AW MLU triggers: start 0x8, required 0xA

= Control registers =

== Register addresses ==

Note: all registers are 32 bits unless otherwise noted.

Note: RW means read-write register, RO means read-only, ROL means read-only, contents is latched by the "latch_out" bit

{| cellpadding="10" cellspacing="0" border="1"
! Address || Quartus Name || Access || FwRev || Description
|-
| 0x08 || num_sample || RW || GRIF-C || magic register to control UDP packet size
|-
| 0x1F || compilation_time || RO || || Firmware revision
|-
| 0x20 || conf_trig_width <= par_value;
|-
| 0x21 || conf_busy_width <= par_value;
|-
| 0x22 || conf_pulser_width <= par_value;
|-
| 0x23 || conf_pulser_period <= par_value;
|-
| 0x24 || sw_trigger_counter <= 10;
|-
| 0x24 || - || || 0x5b120c9f (01Jun18_20:18) || not used
|-
| 0x25 || conf_trig_enable <= par_value;
|-
| 0x26 || conf_sas_trig_mask <= par_value; || obsolete
|-
| 0x27 || conf_sas_trig_mask_a <= par_value; || obsolete
|-
| 0x28 || conf_sas_trig_mask_b <= par_value; || obsolete
|-
| 0x29 || conf_nim_mask <= par_value;
|-
| 0x2A || conf_esata_mask <= par_value;
|-
| 0x2B || reset_out, latch_out, trigger_out || pulsed || 0x5b120c9f (01Jun18_20:18) || see reg 0x2B bits
|-
| 0x2C || conf_aw16_coinc_a || RW || ??? || configure AW coincidence trigger
|-
| 0x2D || conf_aw16_coinc_b || RW || ??? ||
|-
| 0x2E || conf_aw16_coinc_c || RW || ??? ||
|-
| 0x2F || conf_aw16_coinc_d || RW || ??? ||
|-
| 0x30 || sas_sd[15:0] || ROL || 0x5b120c9f (01Jun18_20:18) || sas signal detect for each link
|-
| 0x31 || pll_625_status || ROL || 0x5b2057f5 (12Jun18_16:32) || 62.5MHz PLL status, bits: 31=locked, 30=active clock, 29=esata_clk_bad, 28=clk625_bad, 27:0 not used, to be lock counter
|-
| 0x32 || clk_counter || ROL || 0x5b2057f5 (12Jun18_16:32) || frequency counter of the system 125 MHz clock
|-
| 0x33 || clk_625_counter || ROL || 0x5b2057f5 (12Jun18_16:32) || frequency counter of the currently selected 62.5 MHz clock (my_clk_625)
|-
| 0x34 || conf_control || RW || 0x5b398983 (01Jul18_19:10) || see conf_control bits below
|-
| 0x35 || conf_drift_width || RW || 0x5b398983 (01Jul18_19:10) || width of drift time blank-out in 62.5MHz clocks
|-
| 0x36 || conf_scaledown || RW || 0x5b398983 (01Jul18_19:10) || trigger scaledown, 0=no scaledown, 1=divide-by-2, 2=divide-by-3, etc
|-
| 0x37 || conf_mlu || RW || 0x5b398983 (01Jul18_19:10) || aw trigger mlu control
|-
| 0x38 || conf_trig_delay || RW || 0x5baedfca (28Sep18_19:13) || delay the trigger pulse
|-
| 0x39 || esata_clk_counter || ROL || 0x5baedfca (28Sep18_19:13) || frequency counter of the system 125 MHz clock
|-
| 0x3A || esata_clk_esata_counter || ROL || 0x5baedfca (28Sep18_19:13) || requency counter of the external clock (eSATA clock)
|-
| 0x3B || conf_counter_adc_select || RW || 0x5bb7f6ae (05Oct18_16:41) || select which 16 ADCs are connected to the selected adc counters, 0..15 - connect the adc16 signals, 16..47 connect the adc32 signals
|-
| 0x3C || sysreset_ts || RO || 0x5bb7f6ae (05Oct18_16:41) || 125MHz counter of VME-SYSRESET active state
|-
| 0x3D || GRIF-C csr || RW || GRIF-C || <= csr & ~par_value; // Selective clear
|-
| 0x3E || GRIF-C csr || RW || GRIF-C || <= csr | par_value; // Selective set
|-
| 0x3F || GRIF-C csr || RW || GRIF-C || <= par_value; // write
|-
| 0x40 || conf_bsc_control || RW || 0x5bc13541 (12Oct18_16:58) || Barrel Scintillator trigger control
|-
| 0x41 || conf_coinc_control || RW || 0x5bc13541 (12Oct18_16:58) || Coincidence trigger control
|-
| 0x42 || reconfig_out || WO || 0x5bc13541 (12Oct18_16:58) || reserved for rebooting the FPGA (does not work)
|-
| 0x43 || conf_trigger_timeout || RW || 0x5bc8f90f (18Oct18_14:20) || Trigger timeout in 125MHz clocks (8ns)
|-
| 0x100 || ts_625 || ROL || || latched timestamp
|-
| 0x101 || counter_trig_out || ROL || || counter of issued triggers
|-
| 0x102 || counter_trig_in || ROL || || counter of triggers
|-
| 0x103 || counter_pulser || ROL || || counter of pulser
|-
| 0x104 || counter_adc16_grand_or || ROL ||
|-
| 0x105 || counter_adc32_grand_or || ROL ||
|-
| 0x106 || counter_adc_grand_or || ROL ||
|-
| 0x107 || counter_esata_nim_grand_or || ROL ||
|-
| 0x108 || counter_adc16_mult_1ormore || ROL ||
|-
| 0x109 || counter_adc16_mult_2ormore || ROL ||
|-
| 0x10A || counter_adc16_mult_3ormore || ROL ||
|-
| 0x10B || counter_adc16_mult_4ormore || ROL ||
|-
| 0x10C || counter_aw16_coinc_a || ROL || ??? || counter of AW coincidence triggers
|-
| 0x10D || counter_aw16_coinc_b || ROL || ??? ||
|-
| 0x10E || counter_aw16_coinc_c || ROL || ??? ||
|-
| 0x10F || counter_aw16_coinc_d || ROL || ??? ||
|-
| 0x110 || counter_drift || ROL || 0x5b398983 (01Jul18_19:10) || counter of triggers that passed the drift time blank-out
|-
| 0x111 || counter_scaledown || ROL || 0x5b398983 (01Jul18_19:10) || counter of triggers that passed the scaledown
|-
| 0x112 || counter_aw16_coinc || ROL || 0x5b398983 (01Jul18_19:10) || counter of AW coincidence triggers
|-
| 0x113 || counter_aw16_mlu || ROL || 0x5b398983 (01Jul18_19:10) || counter of AW MLU triggers
|-
| 0x114 || counter_aw16_mult_1ormore || ROL || 0x5bb7f6ae (05Oct18_16:41) || counter of AW 1ormore triggers
|-
| 0x115 || counter_aw16_mult_2ormore || ROL || 0x5bb7f6ae (05Oct18_16:41) || counter of AW 2ormore triggers
|-
| 0x116 || counter_aw16_mult_3ormore || ROL || 0x5bb7f6ae (05Oct18_16:41) || counter of AW 3ormore triggers
|-
| 0x117 || counter_aw16_mult_4ormore || ROL || 0x5bb7f6ae (05Oct18_16:41) || counter of AW 4ormore triggers
|-
| 0x118 || counter_bsc_grand_or || ROL || 0x5bc13541 (12Oct18_16:58) || counter of BSC grand-or triggers
|-
| 0x119 || counter_bsc_mult || ROL || 0x5bc13541 (12Oct18_16:58) || counter of BSC multiplicity triggers
|-
| 0x11A || counter_coinc_mult || ROL || 0x5bc13541 (12Oct18_16:58) || counter of coinidence triggers
|-
| 0x200..0x207 || conf_adc16_masks || RW || all || adc16 enable bits
|-
| 0x300..0x30F || conf_adc32_masks || RW || all || adc32 enable bits
|-
| 0x400..0x41F || sas_bits[15:0][63:0] || ROL || all || sas data bits for each link
|-
| 0x420..0x42F || sas_sd_counters[15:0] || ROL || 0x5b120c9f (01Jun18_20:18) || sas signal detect counter for each link
|-
| 0x430..0x43F || counters_adc16_or16[15:0] || ROL || 0x5b120c9f (01Jun18_20:18) || 16 counters of adc16 ORes
|-
| 0x440..0x45F || counters_adc32_or16[31:0] || ROL || 0x5b120c9f (01Jun18_20:18) || 32 counters of adc32 ORes
|-
| 0x460..0x46F || counters_adc_selected[15:0] || ROL || 0x5bb7f6ae (05Oct18_16:41) || 16 counters for selected group of adcs, see conf_counter_adc_select
|-
| 0x470..0x4AF || counters_bsc64[63:0] || ROL || 0x5bc13541 (12Oct18_16:58) || 64 counters for the BSC bars
|-
|}

== conf_control bits ==

{| cellpadding="10" cellspacing="0" border="1"
! Bit || Quartus Name || FwRev || Description
|-
| 0 || conf_clock_select || 0x5b3aa19f (02Jul18_15:05) || select 0=internal 62.5 MHz clock or 1=esata 62.5 MHz clock
|-
| 1 || conf_enable_aw_adc16 || 0x5baedfca (28Sep18_19:13) || route AW trigger (aw16) from adc16
|-
| 2 || conf_enable_aw_adc32a || 0x5baedfca (28Sep18_19:13) || route AW trigger (aw16) from adc32 15..0
|-
| 3 || conf_enable_aw_adc32b || 0x5baedfca (28Sep18_19:13) || route AW trigger (aw16) from adc32 31..16
|-
| 15..4 || || ||
|-
| 23..16 || conf_mlu_prompt || 0x5b3aa19f (02Jul18_15:05) || MLU trigger prompt window
|-
| 31..24 || conf_mlu_wait || 0x5b3aa19f (02Jul18_15:05) || MLU trigger wait window
|-
|}

== conf_mlu bits ==

{| cellpadding="10" cellspacing="0" border="1"
! Bit || Quartus Name || FwRev || Description
|-
| 31 || mlu_reset || 0x5b3aa19f (02Jul18_15:05) || not used
|-
| 30 || mlu_write || 0x5b3aa19f (02Jul18_15:05) || 1=write data bit into given address
|-
| 16 || mlu_data || 0x5b3aa19f (02Jul18_15:05) || data bit for writing to MLU RAM
|-
| 15..0 || mlu_write_addr || 0x5b3aa19f (02Jul18_15:05) || MLU RAM write address for writing the data bit
|-
|}

== conf_bsc_control bits ==

{| cellpadding="10" cellspacing="0" border="1"
! Bit || Quartus Name || FwRev || Description
|-
| 0 || conf_bsc_adc_a || 0x5bc13541 (12Oct18_16:58) || enable mapping of BSC ADCs to ADC links 0..7
|-
| 1 || conf_bsc_adc_b || 0x5bc13541 (12Oct18_16:58) || enable mapping of BSC ADCs to ADC links 8..15
|-
| 2 || conf_bsc64_bot_only || 0x5bc13541 (12Oct18_16:58) || bsc64 uses bottom channels only
|-
| 3 || conf_bsc64_top_only || 0x5bc13541 (12Oct18_16:58) || bsc64 uses top channels only
|-
| 4 || conf_bsc64_bot_top_or || 0x5bc13541 (12Oct18_16:58) || bsc64 uses "or" of top and bottom
|-
| 5 || conf_bsc64_bot_top_and || 0x5bc13541 (12Oct18_16:58) || bsc64 uses "and" of top and bottom
|-
| 6 ||
|-
| 7 ||
|-
| 15..8 || conf_bsc64_mult || 0x5bc13541 (12Oct18_16:58) || bsc trigger fires if bsc64 multiplicity is equal or more than this value
|-
| 31..16 ||
|-
|}

== conf_coinc_control bits ==

{| cellpadding="10" cellspacing="0" border="1"
! Bit || Quartus Name || FwRev || Description
|-
| 0 || conf_coinc_required[0] || 0x5bc13541 (12Oct18_16:58) || enable aw16_mult_1ormore
|-
| 1 || conf_coinc_required[1] || 0x5bc13541 (12Oct18_16:58) || enable aw16_mlu_trig
|-
| 2 || conf_coinc_required[2] || 0x5bc13541 (12Oct18_16:58) || enable bsc_grand_or_trig
|-
| 3 || conf_coinc_required[3] || 0x5bc13541 (12Oct18_16:58) || enable bsc_mult_trig
|-
| 4 || conf_coinc_required[4] || 0x5bc13541 (12Oct18_16:58) || enable esata_nim_grand_or
|-
| 5 ||
|-
| 6 ||
|-
| 7 ||
|-
| 15..8 || conf_coinc_window[7:0] || 0x5bc13541 (12Oct18_16:58) || coincidence window length in units of 125MHz clock (8 ns)
|-
| 23..16 || conf_coinc_start[7:0] || 0x5bc13541 (12Oct18_16:58) || start the coincidence window, same bits as "required"
|-
| 31..24 ||
|-
|}

== conf_trig_enable bits ==

{| cellpadding="10" cellspacing="0" border="1"
! Bit || Quartus Name || FwRev || Description
|-
| 0 || conf_enable_sw_trigger || || trigger on software trigger (how generated???)
|-
| 1 || conf_enable_pulser || || trigger on pulser
|-
| 2 || conf_enable_sas_or || || obsolete
|-
| 2 || - || 0x5b120c9f (01Jun18_20:18) || not used
|-
| 3 || conf_run_pulser || || let the pulser run (frequency has to be set beforehand)
|-
| 4 || conf_output_pulser || || enable external output of pulser signal
|-
| 5 || conf_enable_esata_nim || || trigger on ESATA and NIM grand-or (esata and nim masks should be set beforehand)
|-
| 6 || conf_enable_adc16 || || trigger on adc16 grand-or
|-
| 7 || conf_enable_adc32 || || trigger on adc32 grand-or
|-
| 8 || conf_enable_adc16_1ormore || || trigger on adc16 grand multiplicity "1 or more"
|-
| 9 || conf_enable_adc16_2ormore || || same, "2 or more"
|-
| 10 || conf_enable_adc16_3ormore || || same, "3 or more"
|-
| 11 || conf_enable_adc16_4ormore || || same, "4 or more"
|-
| 12 || conf_enable_adc16_coinc || || trigger on special coincidence of adc16 links
|-
| 12 || - || 0x5b08938c (25May18_15:51) || not used
|-
| 13 || conf_enable_udp || 0x5a7a3fbd (06Feb18_15:52) || enable sending UDP packets
|-
| 14 || conf_enable_busy || 0x5a7a3fbd (06Feb18_15:52) || enable activation of busy counter
|-
| 15 || conf_enable_adc || 0x5b120c9f (01Jun18_20:18) || trigger on adc grand-or
|-
| 16 || conf_enable_aw16_coinc_a || ??? || enable AW coincidence trigger
|-
| 17 || conf_enable_aw16_coinc_b || ??? ||
|-
| 18 || conf_enable_aw16_coinc_c || ??? ||
|-
| 19 || conf_enable_aw16_coinc_d || ??? ||
|-
| 20 || conf_clock_select || 0x5b2057f5 (12Jun18_16:32) || select 0=internal 62.5 MHz clock or 1=esata 62.5 MHz clock
|-
| 20 || not used || 0x5b3aa19f (02Jul18_15:05) || not used, clock select moved to the conf_control register
|-
| 21 || conf_enable_aw16_coinc || 0x5b3aa19f (02Jul18_15:05) || enable AW coincidence trigger
|-
| 22 || conf_enable_aw16_mlu || 0x5b3aa19f (02Jul18_15:05) || enable AW MLU trigger
|-
| 23 || conf_enable_timeout_trig || 0x5bc8f90f (18Oct18_14:20) || enable trigger timeout trigger
|-
| 24 || conf_enable_aw16_1ormore || 0x5b3aa19f (02Jul18_15:05) || trigger on AW preamp multiplicity "1 or more"
|-
| 25 || conf_enable_aw16_2ormore || 0x5b3aa19f (02Jul18_15:05) || same, "2 or more"
|-
| 26 || conf_enable_aw16_3ormore || 0x5b3aa19f (02Jul18_15:05) || same, "3 or more"
|-
| 27 || conf_enable_aw16_4ormore || 0x5b3aa19f (02Jul18_15:05) || same, "4 or more"
|-
| 28 || conf_enable_bsc_grand_or || 0x5bc13541 (12Oct18_16:58) || trigger on BSC grand-or
|-
| 29 || conf_enable_bsc_mult || 0x5bc13541 (12Oct18_16:58) || trigger on BSC multiplicity
|-
| 30 || conf_enable_coinc || 0x5bc13541 (12Oct18_16:58) || trigger on coincidence output
|-
| 31 ||
|-
|}

== reg 0x2B bits ==

{| cellpadding="10" cellspacing="0" border="1"
! Bit || Quartus Name || FwRev || Description
|-
| 0 || reset_out || 0x5b120c9f (01Jun18_20:18) || reset counters, etc
|-
| 1 || latch_out || 0x5b120c9f (01Jun18_20:18) || latch counters, sas bits, etc
|-
| 2 || trigger_out || 0x5b120c9f (01Jun18_20:18) || generate a trigger
|-
|}

= UDP data format =

{| cellpadding="10" cellspacing="0" border="1"
! Offset || Name || Quartus name || FwRev || Description
|-
| 0 || packet counter || none || all || UDP packet counter, counting from 0, reset by reboot. Automatically added by UDP transmitter code in the GRIFC base firmware
|-
| 1 || 0x8 packet header || || all || 28 bits of counter_trig_out
|-
| 2 || trigger timestamp || trig_ts || all || 62.5MHz trigger timestamp
|-
| 3 || accepted triggers || counter_trig_out || all || counter of accepted triggers
|-
| 4 || input triggers || counter_trig_in || all || counter of trigger input
|-
| 5 || pulser triggers || counter_pulser || all || counter of pulser triggers
|-
| 6 || trigger bitmap || udp_trig_bits || 0x5a48448d (30Dec17_17:59) || bitmap of trigger information, see below
|-
| 7 || nim bits || udp_nim_bits_masked || ??? || 32 bits of ADC NIM inputs (2 bits per ADC)
|-
| 8 || esata bits || udp_esata_bits_masked || ??? || 32 bits of ADC ESATA inputs (2 bits per ADC)
|-
| 9 || mlu bits || aw16_prompt || 0x5b398983 (01Jul18_19:10) || 16 bits of MLU input, 1 bit per preamp
|-
| 10 || drift-blank-off counter || udp_drift_counter || 0x5b3aa19f (02Jul18_15:05) || counter of triggers that passed the drift blank-off
|-
| 11 || scaledown counter || udp_scaledown_counter || 0x5b3aa19f (02Jul18_15:05) || counter of triggers that passed the scaledown
|-
| X || || || ||
|-
| 12 || 0x9 packet footer || || all || 28 bits of counter_trig_out
|-
|}

== trigger bitmap ==
<pre>
udp_trig_bits[0] <= adc16_grand_or;
udp_trig_bits[1] <= adc32_grand_or;
udp_trig_bits[2] <= adc_grand_or;
udp_trig_bits[3] <= esata_nim_grand_or;
udp_trig_bits[4] <= adc16_coinc_top;
udp_trig_bits[5] <= adc16_coinc_bot;
udp_trig_bits[6] <= adc16_coinc;
udp_trig_bits[7] <= 0;
udp_trig_bits[23:8] <= adc16_coinc_dff[15:0];
udp_trig_bits[31:24] <= mult16[7:0];

</pre>

Firmware 0x5b08938c (25May18_15:51)

<pre>
udp_trig_bits[0] <= adc16_grand_or;
udp_trig_bits[1] <= adc32_grand_or;
udp_trig_bits[2] <= adc_grand_or;
udp_trig_bits[3] <= esata_nim_grand_or;
udp_trig_bits[4] <= 0; // not used
udp_trig_bits[5] <= 0; // not used
udp_trig_bits[6] <= 0; // not used
udp_trig_bits[7] <= 0;
udp_trig_bits[23:8] <= 0; // not used
udp_trig_bits[31:24] <= mult16[7:0];
</pre>

Firmware 0x5b217dc4 (13Jun18_13:25)

<pre>
udp_trig_bits[0] <= adc16_grand_or;
udp_trig_bits[1] <= adc32_grand_or;
udp_trig_bits[2] <= adc_grand_or;
udp_trig_bits[3] <= esata_nim_grand_or;
udp_trig_bits[4] <= 0; // not used
udp_trig_bits[5] <= 0; // not used
udp_trig_bits[6] <= 0; // not used
udp_trig_bits[7] <= 0;
udp_trig_bits[23:8] <= xadc16_or16[15:0]; // 16x AW preamps hits
udp_trig_bits[31:24] <= xadc16_or16_mult[7:0]; // 16x AW preamp multiplicity (sum of xadc16_or16 bits)
</pre>

Firmware 0x5b3aa19f (02Jul18_15:05)

<pre>
udp_trig_bits[0] <= adc16_grand_or;
udp_trig_bits[1] <= adc32_grand_or;
udp_trig_bits[2] <= adc_grand_or;
udp_trig_bits[3] <= esata_nim_grand_or;
udp_trig_bits[4] <= 0; // not used
udp_trig_bits[5] <= 0; // not used
udp_trig_bits[6] <= 0; // not used
udp_trig_bits[7] <= udp_mlu_out;
udp_trig_bits[23:8] <= xadc16_or16[15:0]; // 16x AW preamps hits
udp_trig_bits[31:24] <= xadc16_or16_mult[7:0]; // 16x AW preamp multiplicity (sum of xadc16_or16 bits)
</pre>

Firmware 0x5bc8f90f (18Oct18_14:20)

<pre>
assign trig32_in[0] = trigger; // software trigger
assign trig32_in[1] = 0; // not used pulser trigger
assign trig32_in[2] = 0; // not used conf_enable_sas_or
assign trig32_in[3] = 0; // not used conf_run_pulser
assign trig32_in[4] = 0; // not used conf_output_pulser
assign trig32_in[5] = esata_nim_grand_or;
assign trig32_in[6] = adc16_grand_or;
assign trig32_in[7] = adc32_grand_or;
assign trig32_in[8] = adc16_mult_1ormore;
assign trig32_in[9] = adc16_mult_2ormore;
assign trig32_in[10] = adc16_mult_3ormore;
assign trig32_in[11] = adc16_mult_4ormore;
assign trig32_in[12] = 0; // not used conf_enable_adc16_coinc
assign trig32_in[13] = 0; // not used conf_enable_udp
assign trig32_in[14] = 0; // not used conf_enable_busy
assign trig32_in[15] = adc_grand_or;
assign trig32_in[16] = aw16_coinc_a;
assign trig32_in[17] = aw16_coinc_b;
assign trig32_in[18] = aw16_coinc_c;
assign trig32_in[19] = aw16_coinc_d;
assign trig32_in[20] = 0; // not used conf_clock_select
assign trig32_in[21] = aw16_coinc_trig;
assign trig32_in[22] = aw16_mlu_trig;
assign trig32_in[23] = timeout_trig;
assign trig32_in[24] = aw16_mult_1ormore;
assign trig32_in[25] = aw16_mult_2ormore;
assign trig32_in[26] = aw16_mult_3ormore;
assign trig32_in[27] = aw16_mult_4ormore;
assign trig32_in[28] = bsc_grand_or_trig;
assign trig32_in[29] = bsc_mult_trig;
assign trig32_in[30] = coinc_trig;
assign trig32_in[31] = 0; // not used
</pre>

= TODO =

* add FwRev to the UDP data output
* fix "write did not happen" (send udp write command, there is no reply, read returns old data as if write never happened). suspect udp "write" packet is lost?
* fix reply to wrong socket - udp packets should reply to same socket as they came from, udp data packets should be sent to same socket as requested them, not same socket as last command.
* add fixes from Chris P. for the auto-negotiation and dhcp (see email from him).
* speed up programming the MLU by writing more than 1 bit at a time

= End =

//KO

TRG

2021-02-09T22:50:54Z

Acapra:

= TRG, the trigger control board for the ALPHA-g detector =

= Links =

* xxx
* https://bitbucket.org/expalpha/alphat_firmware - firmware sources on bitbucket

= General characteristics =

== Photos ==

== Board schematics ==

== Available hardware ==

= ALPHA-T front panel connections =

From top to bottom:

<pre>
|
| minisas-C - links 8-11
| minisas-D - links 12-15
|
| ethernet sfp - use copper sfp only
| minisas - not used, do not connect
| esata - clock in, trigger out
|
| minisas-A - links 0-3
| minisas-B - links 4-7
|
| LEDs
|
</pre>

MiniSAS and SATA flail cable connections:

<pre>
P1 <--> link 3
P2 <--> link 2
P3 <--> link 1
P4 <--> link 0
</pre>

LEDs:

<pre>
--PCB-- (left)
LED[0] - trigger out - ag_trig_out
LED[1] - network packet received - sfp_rx_data_valid
LED[2] - Xilinx FPGA boot ok - FMC1_CFG_DONE
LED[3] - not used
(right)
</pre>

= ALPHA-T onboard jumpers and switches =

* rotary switches: select IP address: 00=192.168.1.50, 11=.51, 22=.52, 33=.53, 44=.54, otherwire=.251

= Firmware update procedure =

* use quartus to generate the grifc.pof file (use grifc.cof)
* open quartus programmer
* attach grifc.pof to the 1st CFI_512 flash
* mark program/configure for the same flash (3 marks: pof, page_0 and OPTION_BITS
* check that "initiate configuration after programming" is set in tools->options
* say "start"
* should take about 5 minutes
* grifc should reboot into the new firmware

If CFI_512 flash is not visible to the quartus programmer, load correct
code into the MaxV CPLD:

* open quartus programmer,
* say "auto detect"
* change file for the 5M2210 device
* use .../boot/maxv_3xfmc_normal.pof from the firmware git repository
* say "program", it should finish in 10-15 seconds
* say "auto detect"
* CFI_512 flash should be visible now.

= Firmware revisions =

= Firmware build =

* use quartus 17.1

= ALPHA-g trigger functions =

Trigger modes:
* software trigger
* pulser trigger
* trigger on GRIF16 ADC NIM and ESATA inputs ("external trigger")
* adc16 grand-or
* adc32 grand-or
* adc16 wire multiplicity
* TPC anode wire per-preamp trigger (aw16 section)
** trigger on coincidences of preamps
** trigger on multiplicity of preamps
** trigger on specific topology of preamps (multiplicity of preamp clusters, etc) via the aw16_mlu trigger
* BSC trigger
** BSC grand_or
** BSC bar multiplicity
* coincidence trigger - coincidence of external, TPC AW and BSC triggers

== software trigger ==

to programmatically fire the trigger, one should:
* set the conf_enable_sw_trigger bit in conf_trig_enable, this will permit the software trigger to fire the trigger
* write value 0x4 to register 0x2B, this will cause the trigger to fire

== pulser trigger ==

To use the pulser trigger, one should program:
* pulser period (in clocks)
* pulser width (in clocks, use value 5, the pulser no longer goes outside of the trigger board, so changing the width does not do anything)
* set the run_pulser bit in conf_trig_enable, this will cause the pulser to run
* set the enable_pulser bit in conf_trig_enable, this will cause the pulser to fire the trigger
* set the output_pulser bit in conf_trig_enable, this will cause the pulser to be output from the alphag firmware block (the pulser no longer goes outside of the trigger board, so changing the bit does not do anything)

== timeout trigger ==

The event builder uses event timestamps to assemble physics events. The timestamps
are usually 32 bit wide and run at 125 MHz (8 ns) or 62.5 MHz (16 ns). If
triggers are generated often (1 per second or more), timestamp wrap around
is easily detected by the event builder. If triggers come infrequently,
timestamp wrap around may occur while there are no triggers, not be seen
by the event builder causing loss of event synchronization.

To avoid this problem in the event builder, a timeout trigger is generated
if no triggers were generated inside a 10 second interval.

The guaranties that even if all physics triggers temporarily stop for any reason,
the event builder will stay synchronized. (0.1 Hz timeout trigger frequency).

== adc16 multiplicity trigger ==

This trigger is intended for use with the barrel scintillator signals connected to adc16 digitizers. It does not work well with TPC AW signals.

The 16 adc16 trigger bits (16x16=256) are added together to form the hit multiplicity value (adc16_mult, 16 bits, actual range 0 to 256).

Four threshold signals are computed: adc16_mult_1ormore = adc16_mult>=1, 2ormore, 3ormore, 4ormore.

There is a counter for each of these four signals.

Each of these four signals can fire the trigger, see conf_trig_enable bits.

== TPC anode wire triggers ==

TBW: mapping of adc16 and adc32 into aw16[15:0] signals

=== aw16_grand_or ===

TPC AW "grand or" trigger: use aw16_mult_1ormore

=== aw16_mult ===

TPC AW preamp multiplicity trigger: counts how many preamps
have hits, fires:
* aw16_mult_1ormore (same as aw16_grand_or)
* aw16_mult_2ormore
* aw16_mult_3ormore
* aw16_mult_4ormore

(256 anode wires, 16 wires per preamp, 16 preamps, 16 aw16 signals)

=== aw16_coinc ===

TBW: TPC AW preamp coincidence trigger

=== aw16_mlu ===

TPC AW preamp MLU trigger: the MLU (memory lookup
table) implements an arbitrary logical function
to map the 16 aw16 preamp signals into 1 aw16_mlu_trig trigger output.

Description of the aw16 mlu trigger.

In words:

* any aw16 signal starts the aw16_prompt_run gate
* aw16 signals are collected for the duration of conf_mlu_prompt (125MHz/8ns clock)
* this corresponds to time spread of TPC AW signals due to the drift time in the AW drift cell and to the variation in signal transmission time from the ADC to the TRG (digital trigger data link).
* when this prompt time expires, the MLU looks at the accumulated aw16_prompt bit pattern and makes the "yes/no" trigger decision
* after the trigger decision is made, we wait for the TPC to become quiet, meaning there is no aw16 signals for the duration of the aw16_prompt_wait gate (conf_mlu_wait), if any signals arrive during the gate, they restart the gate.
* after the TPC has been quiet for conf_mlu_wait time (125MHz/8ns clock), the MLU trigger is ready for the next event.

In logic: (see alphag.v)

* aw16_or <= grand-or of 16 aw16 signals
* "aw16_or" starts the "aw16_prompt_cnt" counter, opens the aw16_prompt_run gate
* during the aw16_prompt_run gate, aw16_prompt collects (latches) the aw16 bits (aw16_prompt <= aw16_prompt | aw16)
* this is the time when we wait for all "prompt" signals from the TPC to arrive into the trigger
* at the end of the aw16_prompt_run gate, the signal aw16_prompt_out is pulsed, latching the output of the MLU into aw16_mlu_trig
* this is the MLU trigger "yes/no" trigger decision.
* after the MLU trigger decision is made, we wait for the arrival and passing of all TPC drift signals
* the aw16_prompt_wait_cnt and aw_prompt_wait gate opens
* if any new aw16 signals arrive (they are TPC drift signals at this time), the aw16_prompt_wait gate is restarted
* if no aw16 signals arrive during the full run of the aw16_prompt_wait gate ("TPC quiet time")
* the aw16_prompt_wait and aw16_prompt gates are cleared and the MLU trigger is ready for the next event.

== BSC trigger ==

TBW

== Coincidence trigger ==

Coincidence trigger inputs:

<pre>
assign coinc_input[0] = aw16_mult_1ormore;
assign coinc_input[1] = aw16_mlu_trig;
assign coinc_input[2] = bsc_grand_or_trig;
assign coinc_input[3] = bsc_mult_trig;
assign coinc_input[4] = esata_nim_grand_or;
assign coinc_input[5] = 0;
assign coinc_input[6] = 0;
assign coinc_input[7] = 0;
</pre>

Coincidence trigger "start" mask selects which of the inputs can start the coincidence window.

Coincidence trigger "require" mask selects which inputs must have a hits for the coincidence ot be satisfied.

Input hits are collected for the duration of the coincidence window, if all required inputs have hits, coinc_trig is fired.

Example settings:

* coincidence of external scintillator, BSC and TPC AW "grand or" signals: start 0x10, required 0x15
* coincidence of BSC and TPC AW "grand or" signals: start 0x4, required 0x5
* coincidence of BSC and TPC AW MLU triggers: start 0x4, required 0x6
* coincidence of BSC multiplicity and TPC AW MLU triggers: start 0x8, required 0xA

= Control registers =

== Register addresses ==

Note: all registers are 32 bits unless otherwise noted.

Note: RW means read-write register, RO means read-only, ROL means read-only, contents is latched by the "latch_out" bit

{| cellpadding="10" cellspacing="0" border="1"
! Address || Quartus Name || Access || FwRev || Description
|-
| 0x08 || num_sample || RW || GRIF-C || magic register to control UDP packet size
|-
| 0x1F || compilation_time || RO || || Firmware revision
|-
| 0x20 || conf_trig_width <= par_value;
|-
| 0x21 || conf_busy_width <= par_value;
|-
| 0x22 || conf_pulser_width <= par_value;
|-
| 0x23 || conf_pulser_period <= par_value;
|-
| 0x24 || sw_trigger_counter <= 10;
|-
| 0x24 || - || || 0x5b120c9f (01Jun18_20:18) || not used
|-
| 0x25 || conf_trig_enable <= par_value;
|-
| 0x26 || conf_sas_trig_mask <= par_value; || obsolete
|-
| 0x27 || conf_sas_trig_mask_a <= par_value; || obsolete
|-
| 0x28 || conf_sas_trig_mask_b <= par_value; || obsolete
|-
| 0x29 || conf_nim_mask <= par_value;
|-
| 0x2A || conf_esata_mask <= par_value;
|-
| 0x2B || reset_out, latch_out, trigger_out || pulsed || 0x5b120c9f (01Jun18_20:18) || see reg 0x2B bits
|-
| 0x2C || conf_aw16_coinc_a || RW || ??? || configure AW coincidence trigger
|-
| 0x2D || conf_aw16_coinc_b || RW || ??? ||
|-
| 0x2E || conf_aw16_coinc_c || RW || ??? ||
|-
| 0x2F || conf_aw16_coinc_d || RW || ??? ||
|-
| 0x30 || sas_sd[15:0] || ROL || 0x5b120c9f (01Jun18_20:18) || sas signal detect for each link
|-
| 0x31 || pll_625_status || ROL || 0x5b2057f5 (12Jun18_16:32) || 62.5MHz PLL status, bits: 31=locked, 30=active clock, 29=esata_clk_bad, 28=clk625_bad, 27:0 not used, to be lock counter
|-
| 0x32 || clk_counter || ROL || 0x5b2057f5 (12Jun18_16:32) || frequency counter of the system 125 MHz clock
|-
| 0x33 || clk_625_counter || ROL || 0x5b2057f5 (12Jun18_16:32) || frequency counter of the currently selected 62.5 MHz clock (my_clk_625)
|-
| 0x34 || conf_control || RW || 0x5b398983 (01Jul18_19:10) || see conf_control bits below
|-
| 0x35 || conf_drift_width || RW || 0x5b398983 (01Jul18_19:10) || width of drift time blank-out in 62.5MHz clocks
|-
| 0x36 || conf_scaledown || RW || 0x5b398983 (01Jul18_19:10) || trigger scaledown, 0=no scaledown, 1=divide-by-2, 2=divide-by-3, etc
|-
| 0x37 || conf_mlu || RW || 0x5b398983 (01Jul18_19:10) || aw trigger mlu control
|-
| 0x38 || conf_trig_delay || RW || 0x5baedfca (28Sep18_19:13) || delay the trigger pulse
|-
| 0x39 || esata_clk_counter || ROL || 0x5baedfca (28Sep18_19:13) || frequency counter of the system 125 MHz clock
|-
| 0x3A || esata_clk_esata_counter || ROL || 0x5baedfca (28Sep18_19:13) || requency counter of the external clock (eSATA clock)
|-
| 0x3B || conf_counter_adc_select || RW || 0x5bb7f6ae (05Oct18_16:41) || select which 16 ADCs are connected to the selected adc counters, 0..15 - connect the adc16 signals, 16..47 connect the adc32 signals
|-
| 0x3C || sysreset_ts || RO || 0x5bb7f6ae (05Oct18_16:41) || 125MHz counter of VME-SYSRESET active state
|-
| 0x3D || GRIF-C csr || RW || GRIF-C || <= csr & ~par_value; // Selective clear
|-
| 0x3E || GRIF-C csr || RW || GRIF-C || <= csr | par_value; // Selective set
|-
| 0x3F || GRIF-C csr || RW || GRIF-C || <= par_value; // write
|-
| 0x40 || conf_bsc_control || RW || 0x5bc13541 (12Oct18_16:58) || Barrel Scintillator trigger control
|-
| 0x41 || conf_coinc_control || RW || 0x5bc13541 (12Oct18_16:58) || Coincidence trigger control
|-
| 0x42 || reconfig_out || WO || 0x5bc13541 (12Oct18_16:58) || reserved for rebooting the FPGA (does not work)
|-
| 0x43 || conf_trigger_timeout || RW || 0x5bc8f90f (18Oct18_14:20) || Trigger timeout in 125MHz clocks (8ns)
|-
| 0x100 || ts_625 || ROL || || latched timestamp
|-
| 0x101 || counter_trig_out || ROL || || counter of issued triggers
|-
| 0x102 || counter_trig_in || ROL || || counter of triggers
|-
| 0x103 || counter_pulser || ROL || || counter of pulser
|-
| 0x104 || counter_adc16_grand_or || ROL ||
|-
| 0x105 || counter_adc32_grand_or || ROL ||
|-
| 0x106 || counter_adc_grand_or || ROL ||
|-
| 0x107 || counter_esata_nim_grand_or || ROL ||
|-
| 0x108 || counter_adc16_mult_1ormore || ROL ||
|-
| 0x109 || counter_adc16_mult_2ormore || ROL ||
|-
| 0x10A || counter_adc16_mult_3ormore || ROL ||
|-
| 0x10B || counter_adc16_mult_4ormore || ROL ||
|-
| 0x10C || counter_aw16_coinc_a || ROL || ??? || counter of AW coincidence triggers
|-
| 0x10D || counter_aw16_coinc_b || ROL || ??? ||
|-
| 0x10E || counter_aw16_coinc_c || ROL || ??? ||
|-
| 0x10F || counter_aw16_coinc_d || ROL || ??? ||
|-
| 0x110 || counter_drift || ROL || 0x5b398983 (01Jul18_19:10) || counter of triggers that passed the drift time blank-out
|-
| 0x111 || counter_scaledown || ROL || 0x5b398983 (01Jul18_19:10) || counter of triggers that passed the scaledown
|-
| 0x112 || counter_aw16_coinc || ROL || 0x5b398983 (01Jul18_19:10) || counter of AW coincidence triggers
|-
| 0x113 || counter_aw16_mlu || ROL || 0x5b398983 (01Jul18_19:10) || counter of AW MLU triggers
|-
| 0x114 || counter_aw16_mult_1ormore || ROL || 0x5bb7f6ae (05Oct18_16:41) || counter of AW 1ormore triggers
|-
| 0x115 || counter_aw16_mult_2ormore || ROL || 0x5bb7f6ae (05Oct18_16:41) || counter of AW 2ormore triggers
|-
| 0x116 || counter_aw16_mult_3ormore || ROL || 0x5bb7f6ae (05Oct18_16:41) || counter of AW 3ormore triggers
|-
| 0x117 || counter_aw16_mult_4ormore || ROL || 0x5bb7f6ae (05Oct18_16:41) || counter of AW 4ormore triggers
|-
| 0x118 || counter_bsc_grand_or || ROL || 0x5bc13541 (12Oct18_16:58) || counter of BSC grand-or triggers
|-
| 0x119 || counter_bsc_mult || ROL || 0x5bc13541 (12Oct18_16:58) || counter of BSC multiplicity triggers
|-
| 0x11A || counter_coinc_mult || ROL || 0x5bc13541 (12Oct18_16:58) || counter of coinidence triggers
|-
| 0x200..0x207 || conf_adc16_masks || RW || all || adc16 enable bits
|-
| 0x300..0x30F || conf_adc32_masks || RW || all || adc32 enable bits
|-
| 0x400..0x41F || sas_bits[15:0][63:0] || ROL || all || sas data bits for each link
|-
| 0x420..0x42F || sas_sd_counters[15:0] || ROL || 0x5b120c9f (01Jun18_20:18) || sas signal detect counter for each link
|-
| 0x430..0x43F || counters_adc16_or16[15:0] || ROL || 0x5b120c9f (01Jun18_20:18) || 16 counters of adc16 ORes
|-
| 0x440..0x45F || counters_adc32_or16[31:0] || ROL || 0x5b120c9f (01Jun18_20:18) || 32 counters of adc32 ORes
|-
| 0x460..0x46F || counters_adc_selected[15:0] || ROL || 0x5bb7f6ae (05Oct18_16:41) || 16 counters for selected group of adcs, see conf_counter_adc_select
|-
| 0x470..0x4AF || counters_bsc64[63:0] || ROL || 0x5bc13541 (12Oct18_16:58) || 64 counters for the BSC bars
|-
|}

== conf_control bits ==

{| cellpadding="10" cellspacing="0" border="1"
! Bit || Quartus Name || FwRev || Description
|-
| 0 || conf_clock_select || 0x5b3aa19f (02Jul18_15:05) || select 0=internal 62.5 MHz clock or 1=esata 62.5 MHz clock
|-
| 1 || conf_enable_aw_adc16 || 0x5baedfca (28Sep18_19:13) || route AW trigger (aw16) from adc16
|-
| 2 || conf_enable_aw_adc32a || 0x5baedfca (28Sep18_19:13) || route AW trigger (aw16) from adc32 15..0
|-
| 3 || conf_enable_aw_adc32b || 0x5baedfca (28Sep18_19:13) || route AW trigger (aw16) from adc32 31..16
|-
| 15..4 || || ||
|-
| 23..16 || conf_mlu_prompt || 0x5b3aa19f (02Jul18_15:05) || MLU trigger prompt window
|-
| 31..24 || conf_mlu_wait || 0x5b3aa19f (02Jul18_15:05) || MLU trigger wait window
|-
|}

== conf_mlu bits ==

{| cellpadding="10" cellspacing="0" border="1"
! Bit || Quartus Name || FwRev || Description
|-
| 31 || mlu_reset || 0x5b3aa19f (02Jul18_15:05) || not used
|-
| 30 || mlu_write || 0x5b3aa19f (02Jul18_15:05) || 1=write data bit into given address
|-
| 16 || mlu_data || 0x5b3aa19f (02Jul18_15:05) || data bit for writing to MLU RAM
|-
| 15..0 || mlu_write_addr || 0x5b3aa19f (02Jul18_15:05) || MLU RAM write address for writing the data bit
|-
|}

== conf_bsc_control bits ==

{| cellpadding="10" cellspacing="0" border="1"
! Bit || Quartus Name || FwRev || Description
|-
| 0 || conf_bsc_adc_a || 0x5bc13541 (12Oct18_16:58) || enable mapping of BSC ADCs to ADC links 0..7
|-
| 1 || conf_bsc_adc_b || 0x5bc13541 (12Oct18_16:58) || enable mapping of BSC ADCs to ADC links 8..15
|-
| 2 || conf_bsc64_bot_only || 0x5bc13541 (12Oct18_16:58) || bsc64 uses bottom channels only
|-
| 3 || conf_bsc64_top_only || 0x5bc13541 (12Oct18_16:58) || bsc64 uses top channels only
|-
| 4 || conf_bsc64_bot_top_or || 0x5bc13541 (12Oct18_16:58) || bsc64 uses "or" of top and bottom
|-
| 5 || conf_bsc64_bot_top_and || 0x5bc13541 (12Oct18_16:58) || bsc64 uses "and" of top and bottom
|-
| 6 ||
|-
| 7 ||
|-
| 15..8 || conf_bsc64_mult || 0x5bc13541 (12Oct18_16:58) || bsc trigger fires if bsc64 multiplicity is equal or more than this value
|-
| 31..16 ||
|-
|}

== conf_coinc_control bits ==

{| cellpadding="10" cellspacing="0" border="1"
! Bit || Quartus Name || FwRev || Description
|-
| 0 || conf_coinc_required[0] || 0x5bc13541 (12Oct18_16:58) || enable aw16_mult_1ormore
|-
| 1 || conf_coinc_required[1] || 0x5bc13541 (12Oct18_16:58) || enable aw16_mlu_trig
|-
| 2 || conf_coinc_required[2] || 0x5bc13541 (12Oct18_16:58) || enable bsc_grand_or_trig
|-
| 3 || conf_coinc_required[3] || 0x5bc13541 (12Oct18_16:58) || enable bsc_mult_trig
|-
| 4 || conf_coinc_required[4] || 0x5bc13541 (12Oct18_16:58) || enable esata_nim_grand_or
|-
| 5 ||
|-
| 6 ||
|-
| 7 ||
|-
| 15..8 || conf_coinc_window[7:0] || 0x5bc13541 (12Oct18_16:58) || coincidence window length in units of 125MHz clock (8 ns)
|-
| 23..16 || conf_coinc_start[7:0] || 0x5bc13541 (12Oct18_16:58) || start the coincidence window, same bits as "required"
|-
| 31..24 ||
|-
|}

== conf_trig_enable bits ==

{| cellpadding="10" cellspacing="0" border="1"
! Bit || Quartus Name || FwRev || Description
|-
| 0 || conf_enable_sw_trigger || || trigger on software trigger (how generated???)
|-
| 1 || conf_enable_pulser || || trigger on pulser
|-
| 2 || conf_enable_sas_or || || obsolete
|-
| 2 || - || 0x5b120c9f (01Jun18_20:18) || not used
|-
| 3 || conf_run_pulser || || let the pulser run (frequency has to be set beforehand)
|-
| 4 || conf_output_pulser || || enable external output of pulser signal
|-
| 5 || conf_enable_esata_nim || || trigger on ESATA and NIM grand-or (esata and nim masks should be set beforehand)
|-
| 6 || conf_enable_adc16 || || trigger on adc16 grand-or
|-
| 7 || conf_enable_adc32 || || trigger on adc32 grand-or
|-
| 8 || conf_enable_adc16_1ormore || || trigger on adc16 grand multiplicity "1 or more"
|-
| 9 || conf_enable_adc16_2ormore || || same, "2 or more"
|-
| 10 || conf_enable_adc16_3ormore || || same, "3 or more"
|-
| 11 || conf_enable_adc16_4ormore || || same, "4 or more"
|-
| 12 || conf_enable_adc16_coinc || || trigger on special coincidence of adc16 links
|-
| 12 || - || 0x5b08938c (25May18_15:51) || not used
|-
| 13 || conf_enable_udp || 0x5a7a3fbd (06Feb18_15:52) || enable sending UDP packets
|-
| 14 || conf_enable_busy || 0x5a7a3fbd (06Feb18_15:52) || enable activation of busy counter
|-
| 15 || conf_enable_adc || 0x5b120c9f (01Jun18_20:18) || trigger on adc grand-or
|-
| 16 || conf_enable_aw16_coinc_a || ??? || enable AW coincidence trigger
|-
| 17 || conf_enable_aw16_coinc_b || ??? ||
|-
| 18 || conf_enable_aw16_coinc_c || ??? ||
|-
| 19 || conf_enable_aw16_coinc_d || ??? ||
|-
| 20 || conf_clock_select || 0x5b2057f5 (12Jun18_16:32) || select 0=internal 62.5 MHz clock or 1=esata 62.5 MHz clock
|-
| 20 || not used || 0x5b3aa19f (02Jul18_15:05) || not used, clock select moved to the conf_control register
|-
| 21 || conf_enable_aw16_coinc || 0x5b3aa19f (02Jul18_15:05) || enable AW coincidence trigger
|-
| 22 || conf_enable_aw16_mlu || 0x5b3aa19f (02Jul18_15:05) || enable AW MLU trigger
|-
| 23 || conf_enable_timeout_trig || 0x5bc8f90f (18Oct18_14:20) || enable trigger timeout trigger
|-
| 24 || conf_enable_aw16_1ormore || 0x5b3aa19f (02Jul18_15:05) || trigger on AW preamp multiplicity "1 or more"
|-
| 25 || conf_enable_aw16_2ormore || 0x5b3aa19f (02Jul18_15:05) || same, "2 or more"
|-
| 26 || conf_enable_aw16_3ormore || 0x5b3aa19f (02Jul18_15:05) || same, "3 or more"
|-
| 27 || conf_enable_aw16_4ormore || 0x5b3aa19f (02Jul18_15:05) || same, "4 or more"
|-
| 28 || conf_enable_bsc_grand_or || 0x5bc13541 (12Oct18_16:58) || trigger on BSC grand-or
|-
| 29 || conf_enable_bsc_mult || 0x5bc13541 (12Oct18_16:58) || trigger on BSC multiplicity
|-
| 30 || conf_enable_coinc || 0x5bc13541 (12Oct18_16:58) || trigger on coincidence output
|-
| 31 ||
|-
|}

== reg 0x2B bits ==

{| cellpadding="10" cellspacing="0" border="1"
! Bit || Quartus Name || FwRev || Description
|-
| 0 || reset_out || 0x5b120c9f (01Jun18_20:18) || reset counters, etc
|-
| 1 || latch_out || 0x5b120c9f (01Jun18_20:18) || latch counters, sas bits, etc
|-
| 2 || trigger_out || 0x5b120c9f (01Jun18_20:18) || generate a trigger
|-
|}

= UDP data format =

{| cellpadding="10" cellspacing="0" border="1"
! Offset || Name || Quartus name || FwRev || Description
|-
| 0 || packet counter || none || all || UDP packet counter, counting from 0, reset by reboot. Automatically added by UDP transmitter code in the GRIFC base firmware
|-
| 1 || 0x8 packet header || || all || 28 bits of counter_trig_out
|-
| 2 || trigger timestamp || trig_ts || all || 62.5MHz trigger timestamp
|-
| 3 || accepted triggers || counter_trig_out || all || counter of accepted triggers
|-
| 4 || input triggers || counter_trig_in || all || counter of trigger input
|-
| 5 || pulser triggers || counter_pulser || all || counter of pulser triggers
|-
| 6 || trigger bitmap || udp_trig_bits || 0x5a48448d (30Dec17_17:59) || bitmap of trigger information, see below
|-
| 7 || nim bits || udp_nim_bits_masked || ??? || 32 bits of ADC NIM inputs (2 bits per ADC)
|-
| 8 || esata bits || udp_esata_bits_masked || ??? || 32 bits of ADC ESATA inputs (2 bits per ADC)
|-
| 9 || mlu bits || aw16_prompt || 0x5b398983 (01Jul18_19:10) || 16 bits of MLU input, 1 bit per preamp
|-
| 10 || drift-blank-off counter || udp_drift_counter || 0x5b3aa19f (02Jul18_15:05) || counter of triggers that passed the drift blank-off
|-
| 11 || scaledown counter || udp_scaledown_counter || 0x5b3aa19f (02Jul18_15:05) || counter of triggers that passed the scaledown
|-
| X || || || ||
|-
| 12 || 0x9 packet footer || || all || 28 bits of counter_trig_out
|-
|}

== trigger bitmap ==
<pre>
udp_trig_bits[0] <= adc16_grand_or;
udp_trig_bits[1] <= adc32_grand_or;
udp_trig_bits[2] <= adc_grand_or;
udp_trig_bits[3] <= esata_nim_grand_or;
udp_trig_bits[4] <= adc16_coinc_top;
udp_trig_bits[5] <= adc16_coinc_bot;
udp_trig_bits[6] <= adc16_coinc;
udp_trig_bits[7] <= 0;
udp_trig_bits[23:8] <= adc16_coinc_dff[15:0];
udp_trig_bits[31:24] <= mult16[7:0];

</pre>

Firmware 0x5b08938c (25May18_15:51)

<pre>
udp_trig_bits[0] <= adc16_grand_or;
udp_trig_bits[1] <= adc32_grand_or;
udp_trig_bits[2] <= adc_grand_or;
udp_trig_bits[3] <= esata_nim_grand_or;
udp_trig_bits[4] <= 0; // not used
udp_trig_bits[5] <= 0; // not used
udp_trig_bits[6] <= 0; // not used
udp_trig_bits[7] <= 0;
udp_trig_bits[23:8] <= 0; // not used
udp_trig_bits[31:24] <= mult16[7:0];
</pre>

Firmware 0x5b217dc4 (13Jun18_13:25)

<pre>
udp_trig_bits[0] <= adc16_grand_or;
udp_trig_bits[1] <= adc32_grand_or;
udp_trig_bits[2] <= adc_grand_or;
udp_trig_bits[3] <= esata_nim_grand_or;
udp_trig_bits[4] <= 0; // not used
udp_trig_bits[5] <= 0; // not used
udp_trig_bits[6] <= 0; // not used
udp_trig_bits[7] <= 0;
udp_trig_bits[23:8] <= xadc16_or16[15:0]; // 16x AW preamps hits
udp_trig_bits[31:24] <= xadc16_or16_mult[7:0]; // 16x AW preamp multiplicity (sum of xadc16_or16 bits)
</pre>

Firmware 0x5b3aa19f (02Jul18_15:05)

<pre>
udp_trig_bits[0] <= adc16_grand_or;
udp_trig_bits[1] <= adc32_grand_or;
udp_trig_bits[2] <= adc_grand_or;
udp_trig_bits[3] <= esata_nim_grand_or;
udp_trig_bits[4] <= 0; // not used
udp_trig_bits[5] <= 0; // not used
udp_trig_bits[6] <= 0; // not used
udp_trig_bits[7] <= udp_mlu_out;
udp_trig_bits[23:8] <= xadc16_or16[15:0]; // 16x AW preamps hits
udp_trig_bits[31:24] <= xadc16_or16_mult[7:0]; // 16x AW preamp multiplicity (sum of xadc16_or16 bits)
</pre>

Firmware 0x5bc8f90f (18Oct18_14:20)

<pre>
assign trig32_in[0] = trigger; // software trigger
assign trig32_in[1] = 0; // not used pulser trigger
assign trig32_in[2] = 0; // not used conf_enable_sas_or
assign trig32_in[3] = 0; // not used conf_run_pulser
assign trig32_in[4] = 0; // not used conf_output_pulser
assign trig32_in[5] = esata_nim_grand_or;
assign trig32_in[6] = adc16_grand_or;
assign trig32_in[7] = adc32_grand_or;
assign trig32_in[8] = adc16_mult_1ormore;
assign trig32_in[9] = adc16_mult_2ormore;
assign trig32_in[10] = adc16_mult_3ormore;
assign trig32_in[11] = adc16_mult_4ormore;
assign trig32_in[12] = 0; // not used conf_enable_adc16_coinc
assign trig32_in[13] = 0; // not used conf_enable_udp
assign trig32_in[14] = 0; // not used conf_enable_busy
assign trig32_in[15] = adc_grand_or;
assign trig32_in[16] = aw16_coinc_a;
assign trig32_in[17] = aw16_coinc_b;
assign trig32_in[18] = aw16_coinc_c;
assign trig32_in[19] = aw16_coinc_d;
assign trig32_in[20] = 0; // not used conf_clock_select
assign trig32_in[21] = aw16_coinc_trig;
assign trig32_in[22] = aw16_mlu_trig;
assign trig32_in[23] = timeout_trig;
assign trig32_in[24] = aw16_mult_1ormore;
assign trig32_in[25] = aw16_mult_2ormore;
assign trig32_in[26] = aw16_mult_3ormore;
assign trig32_in[27] = aw16_mult_4ormore;
assign trig32_in[28] = bsc_grand_or_trig;
assign trig32_in[29] = bsc_mult_trig;
assign trig32_in[30] = coinc_trig;
assign trig32_in[31] = 0; // not used
</pre>

= TODO =

* add FwRev to the UDP data output
* fix "write did not happen" (send udp write command, there is no reply, read returns old data as if write never happened). suspect udp "write" packet is lost?
* fix reply to wrong socket - udp packets should reply to same socket as they came from, udp data packets should be sent to same socket as requested them, not same socket as last command.
* add fixes from Chris P. for the auto-negotiation and dhcp (see email from him).
* speed up programming the MLU by writing more than 1 bit at a time

= End =

//KO

Chronobox

2021-01-06T23:21:31Z

Acapra: /* Links */

= Chronobox =

= Links =

* https://daq.triumf.ca/DaqWiki/index.php/DE10-Nano
* https://bitbucket.org/expalpha/chronobox_firmware
* https://bitbucket.org/expalpha/chronobox_software

= Chronobox connectors =

<pre>
|
| LEMO - CLK_IN - jumpers: SINE<->CLK<->NIM/TTL and NIM<->CLK<->TTL
|
| ECL P1
| 32
| |
| 1
|
| ECL P2
| 32
| |
| 1
|
| PLED - power-on LED
|
| jumper: bank A direction IN<->X<->OUT
| LEMO 0-1
| LED 0-1
| LEMO 2-3
| LED 2-3
|
| jumper: bank B direction IN<->X<->OUT
| LEMO 4-5
| LED 4-5
| LEMO 6-7
| LED 6-7
|
</pre>

= Input channel mapping =

* 0+16 : first ECL connector
* 16+16 : second ECL connector
* 32+8 : LEMO inputs (TTL)
* 40+18 : GPIO inputs (FPGA pins)
* 58 : external clock (10 MHz nominal)
* 59 : internal clock (100 MHz)

= Install chronobox software =

<pre>
git clone https://bitbucket.org/teamalphag/chronobox_software.git
cd chronobox_software
make clean
make
ls -l *.exe
-rwxr-xr-x 1 olchansk users 18808 Aug 16 15:26 reboot_cb.exe
-rwxr-xr-x 1 olchansk users 47256 Aug 16 15:26 srunner_cb.exe
-rwxr-xr-x 1 olchansk users 20732 Aug 16 15:26 test_cb.exe
</pre>

= Install chronobox quartus firmware project =

<pre>
cd /home/agdaq/online/firmware/git
git clone https://bitbucket.org/teamalphag/chronobox_firmware.git
cd chronobox_firmware
cat timestamp.v
ls -l output_files/*.jic
</pre>

= Firmware revisions =

* 0x5aceaed2 - April 2018 - all inputs connected to counters (except ext clock), 58 inputs, 50 MHz clock
* 0x5b6de806 - August 2018 - added fpga boot flash programmer and fpga reboot
* 0x5b7c827d - August 2018 - bshaw added debounce on GPIO inputs (for flow meters), 100 MHz clock
* 0x5b873169 - August 2018 - rebuilt, no changes
* 0x5b89e4b4 - August 2018 - added external clock signal, 59 inputs, 100 MHz clock
* 0x5b8de2b0 - September 2018 - added data fifo and timestamp counters (TSCs) for the first 4 inputs
* 0x5b906c56 - September 2018 - improved overflow markers, added scalers readout into the data fifo
* 0x5da8e4c2 - October 2019 - timestamps for all inputs, inversion of inputs
* 0x5db764a7 - October 2019 - FPGA reboot works now.

= Firmware update =

If FPGA is not running compatible firmware srunner_cb will not work. To proceed,
load the correct SOF file via JTAG (https://daq.triumf.ca/DaqWiki/index.php/DE10-Nano#JTAG_load_sof_file),
then srunner_cb should work and will be able to load the jic or rpd file into the FPGA boot flash memory.

Different revisions of the DE10-Nano board have different FPGA boot flash chips, some have EPCQ128 (use the "-128" option),
some have the EPCQ64 chip (use the "-64" option). Use "srunner_cb -id" per example below to identify which
flash chip is present on each specific chronobox. Note that the DE10-Nano documentation and the firmware quartus project
generally refer to the EPCS64/EPCQ64 chip. The only practical difference is the use of "-128" or "-64" srunner_cb options.

<pre>
./srunner_cb.exe -id -64 /dev/null # identify EPCS64 flash
./srunner_cb.exe -id -128 /dev/null # identify EPCQ128 flash
./srunner_cb.exe -read -128 test.rpd # read flash contents into a file
#./srunner_cb.exe -program -128 /home/olchansk/git/chronobox_firmware/output_files/DE10_NANO_SoC_GHRD_auto.rpd # write firmware rpd file into flash
./srunner_cb.exe -program -128 ~agmini/online/firmware/git/chronobox_firmware/output_files/DE10_NANO_SoC_GHRD_auto.rpd
./reboot_cb.exe # reboot the fpga into the new firmware
</pre>

= Firmware build =

* use quartus 17.1

= Chronobox firmware registers =

<pre>
reg | rw/ro | quartus name | firmware | description
0 | ro | sof_revision_in | all | firmware revision timestamp code
0 | wo | latch_scalers_out, zero_scalers_out | all | see [[#reg_0x00_write_bits]]
1 | rw | reg1_led_out | all | DE10-Nano LED output
2 | ro | switches_in | all | read DE10-Nano switches
3 | ro | buttons_in | all | read DE10-Nano buttons
4 | rw | reg4_test | all | 32-bit read-write test register
5 | rw | flash_programmer_in, reg5_flash_programmer_out | 0x5b6de806 | 0xABCD srunner flash programmer
6 | ro | ecl_in | all | read state of ECL inputs
7 | ro | reg7_test_in | all | ???
8 | rw | scaler_addr_out, reg8_scaler_data_in | all | top 16 bits of address becomes scaler bus address, 32 bit read is the corresponding scaler data
9 | ro | lemo_in | all | read state of LEMO inputs
A | ro | gpio_in | all | read state of GPIO inputs
B | rw | regB_lemo_out | all | LEMO output data
C | rw | regC_gpio_out | all | GPIO output data
D | rw | regD_out_enable_out | all | enable output tristates: [31:24] - LEMO_OUT, [17:0] - GPIO_OUT
E | rw | regE, reconfig_out | 0x5b6de806 | FPGA reboot: write inverted firmware revision (reg0) to reboot the FPGA
F | ro | regF_input_num_in | 0x5b89e4b4 | number of chronobox inputs (to read scalers, add 1 for the clock counter)
10 | ro | reg10_fifo_status | 0x5b8de2b0 | data fifo status, see below
11 | ro | reg11_fifo_data | 0x5b8de2b0 | data fifo data, see below
12 | rw | cb_invert_a | 0x5bf7557e | invert inputs 31..0
13 | rw | cb_invert_b | 0x5bf7557e | invert inputs 63..32
14 | rw | cb_enable_le_a | 0x5bf7557e | enable TSC leading edge
15 | rw | cb_enable_le_b | 0x5bf7557e | enable TSC leading edge
16 | rw | cb_enable_te_a | 0x5bf7557e | enable TSC trailing edge
17 | rw | cb_enable_te_b | 0x5bf7557e | enable TSC trailing edge
18 | ro | fc_ext_clk_100_counter | 0x5bf7557e | external clock frequency counter 100MHz reference
19 | ro | fc_ext_clk_ext_counter | 0x5bf7557e | external clock frequency counter
1A | ro | fc_ts_clk_100_counter | 0x5bf7557e | timestamp clock frequency counter 100MHz reference
1B | ro | fc_ts_clk_ts_counter | 0x5bf7557e | timestamp clock frequency counter
1C | ro | ts_clk_pll_status | 0x5bf7557e | timestamp clock PLL status
1D | rw | cb_lemo_out_mux_ctrl | 0x5bfdc798 | 8*4 bits to control 8 LEMO output multiplexers (4 bits/16 options each mux)
1E | rw | cb_sync_mask[31:0] | NEXT | source of chronobox sync signal, low bits
1F | rw | cb_sync_mask[63:32] | NEXT | source of chronobox sync signal, high bits
</pre>

== reg 0x00 write bits ==

<pre>
bit | fw revision | quartus signal | description
0 | all | | latch scalers
1 | all | | zero scalers
2 | ... | fifo_rdreq_out | fifo_rdreq_out
3 | ... | ts_clk_pll_extswitch_out | clear ts_clk_pll_extswitch_out
4 | ... | ts_clk_pll_extswitch_out | set ts_clk_pll_extswitch_out
5 | NEXT | sync_arm | arm the synchronization sequence
6 | NEXT | cb_sync | activate the synchronization
</pre>

== reg 0x10 ==

Data FIFO status bits:
<pre>
31: fifo_full
30: fifo_empty
29: 0
28: 0
24+4: 0
0+24: fifo_usedw
</pre>

== reg 0x1C ==

Timestamp clock PLL status bits:
<pre>
31 : PLL locked
30 : PLL active clock (0=internal, 1=external
29 : external clock bad
28 : internal clock bad
27 : ts_clk_pll_extswitch
0..26 : not used
</pre>

== Timestamp clock ==

Timestamp clock is 10 MHz selectable from internal oscillator or external reference on the CLK_IN input.

CLK_IN input can be selected using 2 two-position jumpers:
* NIM input: CLK<->NIM/TTL and CLK<->NIM

To select the clock from command line, use:
* test_cb.exe intclk # select internal clock
* test_cb.exe extclk # select external clock

From software:
<pre>
Chronobox* cb = ...;
cb->cb_int_clock(); # select internal clock
cb->cb_ext_clock(); # select external clock
</pre>

To see current status, run "test_cb.exe clocks":
<pre>
# ./test_cb.exe clocks
...
Chronobox firmware revision: 0x5bf7557e
...
clock status: ext_clk: counter 0x00cccf2a, freq 10000450.9 Hz, ts_clk: counter 0x00cccf2a, freq 10000450.9 Hz, PLL status 0xc0000000
</pre>

Reported is external clock frequency, currently selected timestamp clock frequency and PLL status (register 0x1C).

Normal values for the PLL status:
* internal clock: 0x80000000
* external clock: 0xC0000000
* external clock selected, but invalid: 0x60000000
* internal clock selected, external clock invalid: 0xa0000000

Disconnected/absent/broken external clock will report ext_clk frequency zero, bit 0x20000000 in the PLL status register 0x1C.

== LEMO outputs ==

LEMO connectors can be used as TTL level outputs:

* set the "bank a" and/or "bank b" jumpers for "output"
* set the lemo output multiplexor bits in register 0x1D
* set the "lemo output enable" bit in register 0x0D, use bits 24..31 for LEMO outputs 0..7.
* observe the corresponding LED is on or off according to the LEMO output TTL logic level

The function of each LEMO output is controlled by the lemo output multiplexor. Up to 16 different
signals can be routed into each output. This is controlled by register 0x1D.

Register 0x1D multiplexor control is 32 bits organized into 8 groups of 4 bits, per each of the 8 LEMO outputs: 0x76543210,
i.e. value 0x00000001 routes signal function 1 into output 0, and signal function 0 into outputs 1..7.

For each output, there are 16 possible signal functions (4 bits):

<pre>
bit | firmware signal | firmware revision | signal description
0 | cb_lemo_out_reg[n] | 0x5bfdc798 | output is controlled by register 0xB bits 0..7 for outputs 0..7
1 | clk_ts | 0x5bfdc798 | output is the 10MHz timestamp clock
2 | cb_sync_out | NEXT | output the cb_sync signal
3..30 | gnd | 0x5bfdc798 | not used
31 | vcc | 0x5bfdc798 | logic level 1
</pre>

== front panel LEDs ==

The front panel LEDs are numbered 0..7 per [[#Chronobox_connectors]]

Each LED can be individually lit by setting a bit in register 0x1 bits 8..15.

If LEMO outputs are active, corresponding LEDs will show LEMO output status (logic level 1 -> LED is on, logic level 0 -> LED is off).

If LEMO inputs are active ...

== Multiple chronobox connection ==

To operate several chronoboxes as one unit, they have to have two common signals, a clock and a sync signal.

Described is a master-slave configuration of 2 chronoboxes. Up to 2 slaves can be connected, number of LEMO outputs permitting. Alternately, slave chronoboxes can be daisy-chained for an unlimited number of slaves.

In a master-slave configuration, the master unit provides the timestamp clock and drives the sync signal. (For daisy-chained configuration, the slave can repeat the clock and sync signals into the next slave).

Electrical connection, lemo direction jumpers:
<pre>
master: bank A set to "out"
slave: bank B set to "in"
slave daisy-chain: bank A set to "out"
</pre>

Electrical connection, 1st slave:
<pre>
master lemo out #0 -> slave clk_in (TTL mode)
master lemo out #1 -> slave lemo in #4
</pre>

Electrical connection, 2nd slave:
<pre>
master lemo out #2 -> slave clk_in (TTL mode)
master lemo out #3 -> slave lemo in #4
</pre>

Electrical connection, daisy-chained:
<pre>
slave lemo out #0 -> next slave clk_in (TTL mode)
slave lemo out #1 -> next slave lemo in #4
</pre>

Clock configuration:
* master: set to internal clock or external clock
* slave: set to external clock

Sync configuration:
* master: "sync_arm" activated by software, "sync" activated by software (cb_sync_mask set to zero)
** register 0x1D output mux to 0xXXXX'XX21 (2=output sync signal, 1=output clock)
** register 0x0D output enable, set bits 0x3000'0000 (output enable lemo #0 clock and #1 sync)
** register 0x1E and 0x1F, set to 0 (cb_sync external source)
* slave: "sync_arm" activated by software, "sync" activated by lemo in #4 (cb_sync_mask set bit 32+4).
** register 0x1E set to zero
** register 0x1F set to 0x0000'0004
** for daisy-chain operation, set registers 0x1D and 0x0D same as master

Sync operation:
* issue a "sync_arm" command. Scalers and timestamps will be stopped. Time counter will be reset to zero, scalers will be reset to zero.
* issue a "sync" command. Scalers and timestamps will be started at the same time in all units.

== FIFO data format ==

* 0x8ntttttt: TSC data, 24 bits "tttttt" of timestamp, 7 bits "nn" of channel number, top bit set to 1. Low bit of "t" indicates 0=leading edge, 1=trailing edge.
* 0xffTTmmmm: timestamp wrap around marker: "TT" is the top 8 bits of the timestamp, "mmmm" increments for each marker
* 0xfe00nnnn: scaler data, following "nnnn" words are the latched scalers

== timestamp wrap around marker ==

The timestamp data is only 24 bits, to allow timestamping
with longer time range, wrap around markers are added to the
data stream.

For input signals that arrive close to the time of timestamp wrap around,
there is ambiguity in the ordering of the data fifo: does the wrap around
marker or the signal timestamp show up first? For example for rare
signals, one can see this:
<pre>
wrap 1
wrap 2
timestamp 0x00000003
wrap 3
wrap 4
</pre>
does the hit belong with wrap marker 2 (written to the fifo just after wrap marker 2)
or with marker 3 (written to the fifo just before wrap marker 3)?

To remove this ambiguity, additional markers are written to the data stream
half way between the wrap arounds, making it obvious that the signal
arrived right after wrap marker 3 (but was written to the FIFO before the marker):
<pre>
wrap 1 0x00
wrap 1 0x80
wrap 2 0x00
wrap 2 0x80
timestamp 0x00000003
wrap 3 0x00
wrap 3 0x80
</pre>

= test_cb.exe =

test_cb.exe is the general test program for the chronobox.

* test_cb.exe 0 # read chronobox register 0
* test_cb.exe 4 0x1234 # write to chronobox register 4
* test_cb.exe reboot # reboot the FPGA (the ARM CPU keeps running)
* test_cb.exe scalers # read all scalers in a loop
* test_cb.exe fifo # read the data fifo in a loop
* test_cb.exe intclk # select internal timestamp clock
* test_cb.exe extclk # select external timestamp clock (10MHz)
* test_cb.exe clocks # report current status of timstamp clock

= ZZZ =

Daq

2020-12-23T22:56:16Z

Acapra: redo spaces

= Links =

* https://alphacpc05.cern.ch/agdaq/ -- midas daq page for the agdaq system at CERN
* https://daq16.triumf.ca -- midas daq page for the agmini system at TRIUMF
* https://daq.triumf.ca/elog-alphag/alphag/ -- elog
* https://daqstore.triumf.ca/AgWiki/index.php/Main_Page -- this wiki
* https://bitbucket.org/ttriumfdaq/agdaq - git repository for agdaq
* https://bitbucket.org/expalpha -- git repository on bitbucket
* https://edev-group.triumf.ca/fw/exp/alphag -- git repository on edev gitlab
* https://ladd00.triumf.ca/daqinv/frontend/list/170 - PWB inventory database
* https://ladd00.triumf.ca/daqinv/frontend/list/168 - ADC (GRIF-16) inventory, FMC-ADC32rev0,rev1 inventory
* https://ladd00.triumf.ca/daqinv/frontend/list/176 - FMC-ADC32rev1.1 inventory

= Hardware manuals =

* [[TRG]] - trigger board (GRIF-C) manual
* CDM - https://daq.triumf.ca/DaqWiki/index.php/VME-CDM -- clock VME-CDM manual
* ADC - https://daq.triumf.ca/DaqWiki/index.php/VME-GRIF-ADC16-Rev1 -- GRIF-16 ADC manual
* [[PWB]] - TPC Pad Wing Board manual
* TDC - https://www.triumf.info/wiki/DAQwiki/index.php/GSI_TRB3
* [[chronobox]] - chronobox manual

= Performance =

Full system: (ADC fw rel-20201112-ko, PWB fw 20201005_ko)
* 1 Hz stable, ADC adc32 9 supp off rate 0.3 M/s, adc16 off, PWB udp_delay 25 supp off active 64 rate 6.6+4+6+4.6=21.2 M/s, TDC off, total rate 21 M/s
* 500 Hz stable, ADC adc32 9 supp 1500 rate 6 M/s, adc16 off, PWB udp_delay 10 supp 100 active 63 rate 3+4+3+3=13 M/s, TDC off, total rate 19 M/s

Single ADC: (fw rel-20201112-ko)

Fully suppressed data, sthreshold 5000
* 1 Hz: stable
* 500 Hz: stable (nominal max trigger rate)
* 6700 Hz: stable, 8 Mbytes/sec
* 6800 Hz: evb eventually confused
* 6850 Hz: evb confused, no lost triggers in the ADC, no crash in the ADC

All data, 510 samples/event, sthreshold 10 (fw rel-20201122-ko)
* 1 Hz: stable
* 100 Hz: stable, 3.4 Mbytes/sec
* 500 Hz: stable, 17 Mbytes/sec (nominal max trigger rate)
* 1000 Hz: stable, 32 Mbytes/sec
* 3100 Hz: stable, 106 M/s (approaching saturation of 1gige ethernet)
* 3200 Hz: stable, 110 M/s - TRG communication is in trouble
* 3300 Hz: stable, 116 M/s
* 3400 Hz: incomplete events, no ADC crash ("dropped due to full" counting, "dropped due to busy" are zero, this is as expected).

no suppression (different fpga data path)
* 1 Hz: stable
* 100 Hz: stable
* 500 Hz: stable, 17 M/s (nominal max trigger rate)
* 1000 Hz: stable, 32 M/s
* 3000 Hz: stable, 100 M/s (approaching saturation of 1gige ethernet)
* 3300 Hz: incomplete events, no ADC crash ("dropped due to full" counting, "dropped due to busy" are zero, this is as expected).

= Network connections =

The experiment private network number is: 192.168.1.x, gateway 192.168.1.1, netmask 255.255.255.0

network map:
<pre>
---- TRIUMF/CERN network --1gige-- alphagdaq --10gige-- juniper switch --1gige-- centrecom switch
</pre>

== alphagdaq network connections ==

on the rear of the machine:
<pre>
-----------------------------------------
| | rj45 | rj45 | | sfp | sfp |
-----------------------------------------

- left rj45 - copper 1gige - eno1 - spare (inactive)
- right rj45 - copper 1gige - eno2 - dhcp - cern/triumf network
- left sfp - 10gige - enp1s0f1 - spare (inactive)
- right sfp - 10gige - enp1s0f0 - static 192.168.1.1
</pre>

== connections between switches ==

* alphagdaq to juniper switch: direct attach (DAC) 10gige cable into any port of juniper switch
* juniper switch to centrecom switch: rj45 sfp in any port of juniper switch - cat5e/cat6 cable - to any port of centrecom switch
* cat5e/cat6 cable from any port of centrecom switch to the XXX port on the back of the juniper switch

== juniper switch ==

connected to juniper switch front is:

* everything that sends event data to alphagdaq, specifically:
** TRG (rj45 sfp)
** 16x ADC (fiber sfp)
** 64x PWB (fiber sfp)
* 10gige uplink to alphagdaq (10gige DAC cable)
* 1gige link to centrecom switch (rj45 sfp)
* 40gige ports are not used

connected to the juniper switch rear:

* CON is the serial console via RJ45-to-DB9 to USB-serial adapter to alphagdaq USB port (9600bps, 8N1, no flow control)
* C0 is the management ethernet interface port, connect to any port on the centrecom switch (juniper-private will ping)

== centrecom switch ==

* everything with copper rj45 connections, specifically:
** CDM boards
** HV, LV and VME power supplies
** RaspberryPi3 boards
** gas handling MFCs (algas)
** cooling system controller (moxa01)

= USB connections =

Note:
* alphagdaq has USB2 and USB3 connections. They are not the same, they are not generally interchangeable.
* USB3 connections and cables are generally blue coloured.

List of USB connections from alphagdaq:

* front USB3 port (blue): USB-A-to-B cable to USB blaster connected to the trigger board (front of rack)
* rear USB3 port (blue): USB3 short jumper to USB3 hub
* rear USB2 port (black): USB2 unpowered hub (Logiix) to USB-A-to-B cable to UPS USB connector
* rear USB2 port (black): USB-to-Serial adaper to DB9 to RJ45 adapter to Juniper switch console port

Note: UPS USB connection does not work with alphagdaq USB ports (both usb2 and usb3), but combination
using the unpowered USB2 hub seems to work ok.

List of USB connections from the USB hub:

* this is a powered USB3 hub
* USB-A-to-MicroUSB - to lvdb boards
* USB-A-to-Wiener interlock cable

= Clock and trigger distribution =

== Explanation ==

Trigger is generated by the trigger board (aka TRG, aka GRIF-C),
from the TRG eSATA output through the eSATA splitter it is fed into
the master CDM. The master CDM outputs the trigger on all MiniSAS output ports.
One of the master CDM MiniSAS outputs is connected to the CDM slave eSATA input.
The same as the master CDM, the slave CDM outputs the trigger on all MiniSAS output ports.

The master 62.5MHz clock is generated by the master CDM using the internal 10MHz oscillator
or the external 10MHz AD clock. Also on MiniSAS port 6 the master CDM generates
the 200MHz clock for the TDC. This clock is frequency-locked to the master 62.5MHz clock.

Master CDM clock selection (external CD 10MHz clock or internal oscillator 10MHz clock)
is done by a magic esper command (see below).

From the master CDM, the clock is fed to: the slave CDM, the TRG (via the eSATA splitter)
and to the TDC (via the RJ45 splitter).

The slave CDM sends the clock and the trigger to the ADCs and PWBs.

== Schematic ==

<pre>
TRG (GRIF-C)
------------
eSATA <------> eSATA splitter

eSATA splitter
--------------
trigger ---> eSATA ---> CDM-Master eSATA (trigger signal)
clock <--- (eSATA --- MiniSAS) <--- CDM MiniSAS (62.5MHz clock)
</pre>

<pre>
CDM-Master
----------
eSATA <--- eSATA splitter <--- eSATA from TRG (trigger signal)
MiniSAS 1 --> (MiniSAS --- eSATA) --> CDM-Slave eSATA (62.5 MHz)
MiniSAS 6 --> (MiniSAS --- eSATA) --> eSATA splitter --> RJ45 --> TDC (200 MHz)
</pre>

<pre>
CDM-Slave
---------
eSATA <--- trigger and 62.5MHz clock from CDM-Master
MiniSAS 1 --> ADC trigger and 62.5MHz clock
MiniSAS 2 --> same
MiniSAS 3 --> same
MiniSAS 4 --> same
MiniSAS 5 --> PWB trigger and 62.5MHz clock
MiniSAS 6 --> same
</pre>

== Setup of CDM boards ==

ALPHA-g uses the Rev3 CDM boards. The 4 CDM boards that belong to ALPHA-g
have serial numbers 0, 1, 2 and 3 and are labeled cdm00, cdm01, cdm02, cdm03.

All the boards can be used as master and slave positions, but have to be
configured appropriately.

=== Slave setup ===

* install the board
* connect ethernet
* connect esata external clock
* esper-tool cdmNN ### i.e. esper-tool cdm01
* cd template
* write current_setup 4
* cd /cdm
* read
* observe clock frequency counters report correct values. "ext_clk" will only count if the lemo jumper is installed (see below)
<pre>
11 esata_clk uint32 R [62500246]
12 ext_clk uint32 R [62500246]
</pre>
* cd /lmk
* read
* observe both PLL1 and PLL2 are locked: both values should be "1"
<pre>
144 pll1_ld uint8 R [1]
147 pll2_ld uint8 R [1]
156 ld1_counter uint32 RW [3]
157 ld2_counter uint32 RW [3]
</pre>

If needed, check correct operation of the pll lock monitoring:

* unplug the esata cable from the master CDM
* "red" light should go off
* read of lmk should report "pll1_ld" value 0 and "ld1_counter" and "ld2_counter" should increment
* reconnect the esata cable
* "red" light should return
* read of lmk should report both pll1_ld and pll2_ld locked (values "1").

=== Master setup ===

* install the board
* connect ethernet
* esper-tool cdmNN ### i.e. esper-tool cdm01
* cd template
* write current_setup 3
* cd /lmk
* write clkin_sel_mode 0
* cd /cdm
* read
* observe clock frequency counters report correct values. "ext_clk" will only count if the "clock loopback" lemo jumper is installed (see below)
<pre>
12 ext_clk uint32 R [62500189]
</pre>
* cd /lmk
* read
* observe both PLL1 and PLL2 are locked: both values should be "1"
<pre>
144 pll1_ld uint8 R [1]
147 pll2_ld uint8 R [1]
156 ld1_counter uint32 RW [3]
157 ld2_counter uint32 RW [3]
</pre>

Check the 200MHz clock:
* connect MiniSAS-eSATA splitter cable to MiniSAS output 6, connect eSATA end to CDM eSATA input
* in esper-tool, read cdm: esata_clk should report 200MHz clock
<pre>
11 esata_clk uint32 R [200000603]
12 ext_clk uint32 R [62500188]
</pre>

=== Master setup with external 10 MHz clock ===

* setup master CDM as above (using internal clock)
* connect 10MHz external clock to right LEMO #1
* esper-tool cdmNN
* cd /cdm
* write sel_nim 1 # if clock is NIM signal
* write sel_nim 0 # if clock is TTL signal (i.e. chronobox LEMO output)
* read ext_clk
<pre>
[http://cdm00:/cdm]> read ext_clk
9999556
</pre>
* if ext_clk reads 0 or does not read something close to 10MHz, stop here. debug the signal, try changing sel_nim, etc.
* cd /lmk
* write clkin_sel_mode 2 # select external clock
* read
* observe both PLL1 and PLL2 are locked: both values should be "1"
<pre>
150 clkin2_sel bool R [True]
144 pll1_ld uint8 R [1]
147 pll2_ld uint8 R [1]
</pre>
* on the MIDAS TRG page, observe that TRG measured external clock frequency is close to 62.5 MHz

== clkin_sel_mode positions ==

* 0 = 10MHz internal oscillator (use on master CDM in standalone mode)
* 1 = eSATA clock (use on slave CDM)
* 2 = external clock, should be 10MHz (use on master CDM to run from AD clock)

== LEMO connections ==

* clock loopback jumper: to read the frequency of the generated 62.5MHz clock, connect LEMO cable from left LEMO #2 to the right LEMO #1
* 10MHz internal clock jumper: to feed 10MHz internal oscillator to the external clock, connect LEMO cable from right LEMO #3 to the right LEMO #1.
* 10MHz external clock: to feed 10MHz external clock, connect LEMO cable to right LEMO #1

= MIDAS frontends =

== UDP ==

feudp receives udp packets from the ADC and PWB boards. For each udp packet 1 midas event is created,
with 1 data bank containing the udp data. Events are sent into the MIDAS event buffer BUFUDP. The names
of the data banks are assigned in ODB /eq/feudp/settings.

{ADC,PWB} --> 1gige --> switch --> 10gige --> alphagdaq --> feudp -> BUFUDP

== CTRL ==

fectrl controls all the ADC, PWB and trigger boards. It provides run start and stop sequencing,
runs the event synchronization sequence at the begin of the run, and records slow controls data (temperatures,
voltages, etc).

fectrl receives UDP data packets from the trigger board and sends them to the MIDAS event buffer BUFUDP.

fectrl receives commands from the control web pages (for ADC, PWB and trigger) to reboot, initialize, etc the boards.

fectrl configures the event builder via odb /eq/fectrl/evbconfig.

ADC <-> http esper <-> fectrl -> slow control data into midas history

PWB <-> http esper <-> fectrl -> slow control data into midas history

TRG <-> udp comm <-> fectrl -> BUFUDP, slow control and counters into midas history

fectrl <-> midas rpc <-> mhttpd <-> json rpc <-> control web pages for ADC, PWB and trigger

== EVB ==

feevb receives udp packet data from the MIDAS event buffer BUFUDP, inspects the packet timestamps
and collects the data with matching timestamps into physics events. feevb has provisions to do
data suppression, reduction and compression in addition to the data reduction done
in the ADC and PWB firmware.

feevb event synchronization relies on the event synchronization sequence done by fectrl when a run is started.

{ADC, PWB, TRG} -> BUFUDP -> feevb -> SYSTEM -> mlogger -> compression -> disk storage

=== internal structure ===

* event buffer readers, instances for BUFTRG, BUFTDC, BUFADC, BUFUDP
<pre>
BufferReader class - reads one midas event buffer:
- BufferReaderThread()
-- in batches of 1000 events
--- lock
--- TickLocked()
----- ReadEvent() -> bm_receive_event_alloc()
----- fHandler->HandleEvent()
-- fHandler->XMaybeFlushBank()
-- sleep 100 usec
</pre>
* event header decoder and intermediate buffer
<pre>
Handler class - accumulates events, sends them to the EVB in batches

- HandleEvent() <- BufferReaderThread() - receive incoming events
-- loop over all banks, examine bank name
--- call AddAdcBank(), AddPwbBank(), AddTrgBank(), AddTdcBank() according to bank name
---- AddXxxBank() -> XAddBank()
--- unknown banks go directly to evb->SendQueue
-- check if synchronization has completed

- XAddBank() - save BankBuf objects in a buffer

- XMaybeFlushBank(), XFlushBank() - send all accumulated data to the Evb class
-- evb->lock()
-- for each buffered BankBuf objects,
--- call evb->AddBankLocked()
-- buffer is now empty
</pre>
* event builder
<pre>
Evb class - assemble event fragments according to timestamps

- fEvents fifo (deque) - time sorted list of partially assembled events
-- new entries are added by FindEvent()
-- entries are modified by MergeSlot()
-- old entries are removed by GetLocked()

- AddBankLocked() <- XFlushBank() <- BufferReaderThread()
-- call fSync->Add()
-- transform BankBuf to EvbEventBuf
-- push it into per-module buffer fBuf[imodule]

- EvbThread()
-- EvbTickLocked()
--- BuildLocked()
---- in a loop, for no more than 1 sec
----- loop over all per-module buffers fBuf
------ pop first entry, pass it to BuildSlot()
------- call FindEvent()
------- call MergeSlot()
------- call CheckEvent()
--- in a loop
---- get all completed events via GetLocked() pass them to SendQueue->PushEvent()
-- if done nothing, sleep 100 usec, otherwise nominal sleep to free the evb lock

- GetNext() <- GetLocked() <- EvbThread() - get completed events
-- examine oldest EvbEvent entry in fEvents:
-- pop it if it is complete
-- pop it if it is too old (to prevent old incomplete events from stalling the evb) ("pop age" on the evb page)
-- pop it if a following event is complete ("pop following" on the evb page)
</pre>
* send queue
<pre>
SendQueue class - prepare events for writing to midas SYSTEM event buffer

- PushEvent() <- EvbThread()
-- push FragmentBuf into the queue

- SendQueueThread()
-- in a batch of 1000 events, call Tick()
--- pop FragmentBuf from the queue, call SendEvent()
--- compose midas event (memcpy()!)
--- TMFE fEq->SendEvent()
--- bm_send_event()
-- sleep 100 usec
</pre>
* equipment object
<pre>
EvbEq class - implements the midas equipment
- HandleBeginRun()
- HandleEndRun()
- HandlePeriodic()
</pre>

=== multithreading ===

* BufferReaderThread() one per midas event buffer
** high CPU use: bm_receive_event(), AddXxxBank()
** lock contention: midas event buffer lock, evb lock into XFlushBank->AddBankLocked()
* EvbThread()
** O(size of fEvents): FindEvent(), GetNext()
** lock contention: evb lock against XFlushBank(), SendQueue lock (1 per output event)
* SendQueueThread()
** high CPU use: memcpy() in SendEvent(), memcpy() in bm_send_event
** lock contention: event queue lock against EvbThread, midas event buffer lock
* main thread
** lock contention: evb lock in ReportEvb() and elsewhere

= ODB entries =

== /Equipment/CTRL/Settings ==
* TBW

== /Equipment/CTRL/Settings/TrigSrc - trigger source selection ==
* TrigPulser - trigger on the pulser
* TrigEsataNimGrandOr - trigger on external NIM inputs of GRIF-16 ADC modules
* TrigAdc16GrandOr - trigger on any adc16 signals (see adc16_masks)
* TrigAdc32GrandOr - trigger on any adc32 signals (see adc32_masks)
* Trig1ormore - trigger on adc16 multiplicity 1 or more
* Trig2ormore - ditto, 2 or more
* Trig3ormore - ditto, 3 or more
* Trig4ormore - ditto, 4 or more
* TrigAdcGrandOr - trigger on any ADC signal
* TrigAwCoincA - trigger on TPC AW coincidence (see [[TRG]] manual)
* TrigAwCoincB - ditto
* TrigAwCoincC - ditto
* TrigAwCoincD - ditto
* TrigAwCoinc - ditto
* TrigAw1ormore - trigger on TPC AW per-preamp multiplicity 1 or more (see [[TRG]] manual)
* TrigAw2ormore - ditto, 2 or more
* TrigAw3ormore - ditto, 3 or more
* TrigAw4ormore - ditto, 4 or more
* TrigAwMLU - trigger on TPC AW MLU signal (see [[TRG]] manual)
* TrigBscGrandOr - trigger on any BSC signal
* TrigBscMult - trigger on predefined BSC multiplicity (see [[TRG]] manual)
* TrigCoinc - trigger on predefined coincidence of TPC AW, BSC and external signal (see [[TRG]] manual)

== /Equipment/CTRL/Settings/TRG - trigger settings ==
* Enable - if set to "n", fectrl will not do anything with the trigger board. Normal value "y"
* Modules - hostnames of the trigger board. Normal value "alphat01"
* NimMask - 32-bit mask for the 32x NIM inputs (see TRG manual). Normal value 0, unless external trigger is in use.
* EsataMask - 32-bit mask for the 32x eSATA signals (see TRG manual). Normal value 0.
* adc16_mask[16] - 16x 16-bit mask for the 100MHz GRIF-16 onboard ADCs (see TRG manual). Normal value 0xFFFF for all 16 data links.
* adc32_mask[16] - 16x 32-bit mask for the 62.5MHz FMC-ADC32 ADCs (see TRG manual). Normal value 0xFFFFFFFF for all 16 data links.
* PassThrough - if set to "y", trigger is passed through the trigger module without generating any events and without causing deadtime.
* AwCoincA, B, C and D - 32-bit masks for the aw16 coincidence trigger (see TRG manual)
* Aw16FromAdc16 - route AW trigger (aw16) from adc16 (100MHz ADCs 0..15)
* Aw16FromAdc32a - route AW trigger (aw16) from adc32 group 15..0 (62.5MHz FMC-ADC32 in ADCs 0..7)
* Aw16FromAdc32b - route AW trigger (aw16) from adc32 group 31..16 (62.5MHz FMC-ADC32 in ADCs 8..15)

== /Equipment/CTRL/Settings/PWB - PWB settings ==
* Enable - if set to "n", fectrl will not do anything with PWBs. Normal value "y".
* enable_boot_user_page - if set to "n", fectrl will not try to reboot PWBs into the user page firmware. Normal value "y".
* enable_trigger - if set to "n", all PWBs will be set to ignore the trigger. Normal value "y".
* enable_trigger_group_a - field wire pulser defeats the data suppression (every channel has a pulse) resulting in too much data overwhelming the DAQ (lost UDP packets). To permit correct data taking of field wire pulser data, PWBs are split into two groups (a and b), only one of the two groups should be enabled for field wire pulser runs. Both groups should be enabled for normal data taking. Normal value is "y"
* enable_trigger_group_b - see "enable_trigger_group_a". Normal value is "y".
* modules[64] - hostnames of PWB boards
* boot_user_page[64] - if set to "y" and enable_boot_user_page is "y", fectrl will reboot the corresponding PWB to the user page firmware.
* trigger[64] - if set to "y" and enable_trigget is "y", fectrl will set the PWB to accept the trigger.
* ch_enable - if set to "n" disables all PWB channels. Normal value is "y".
* ch_force - is set to "y" disables channel suppression (all PWB channels are read)
* suppress_reset - if set to "y" enables channel suppression for reset channels (threshold has to be set correctly)
* suppress_fpn - if set to "y" enables channel suppression for fpn channels (threshold has to be set correctly)
* suppress_pads - set to "y" enables channel suppression for TPC pad channels (threshold has to be set correctly)
* disable_reset1 - data suppression algorithm does not work for the channel "reset1" and it has to be suppressed explicitly by setting this value to "y". Normal value is "y".
* baseline_{reset,fpn,pads}[64] - waveform baseline for each PWB position
* threshold_{reset,fpn,pads} - waveform suppression threshold, see below:
* ch_threshold - PWB sca_N_ch_threshold, if set to 0, threshold is computed as "baseline_pads[seqpwb]-threshold_pads". Normal value is 0.
* test_mode - enable PWB test mode, the ADC data is replaced with a test pattern, see PWB esper variable signalproc/test_mode

== /Equipment/CTRL/Settings/TDC - TDC settings ==
* Enable - if set to "n", fectrl will tell the event builder that the TDC is not running. Normal value is "y".

== /Equipment/CTRL/EvbConfig - EVB configuration ==
All arrays have the same size corresponding to the number of EVB slots.
* name[] - slot name for the web page (trg01, adc01, adc01/adc32, pwb31/0, tdc01, etc)
* type[] - slot type: 1=TRG, 2=ADC16 and ADC32, 3=obsolete PWB, 4=obsolete PWB, 5=PWB, 6=TDC
* module[] - module number (adcNN, adcNN+100, pwbNN, etc)
* nbanks[] - number of expected banks (16 for ADC16, 32 for ADC32, not used, set to 1 for all others)
* tsfreq[] - timestamp frequency in Hz.

= Trigger configuration =

These trigger modes are implemented:
* (manual trigger from web page)
* software pulser trigger (from fectrl)
* hardware pulser trigger (from the trigger board)
* NIM and eSATA trigger from ADC front panel inputs
* adc16 and adc32 discriminator triggers:
** adc16 grand-or trigger
** adc32 grand-or trigger
** adc (adc16+adc32) grand-or trigger
** adc16 multiplicity trigger (1ormore, 2ormore, 3ormore, 4ormore) for use with barrel scintillator
** anode wire multiplicity trigger (1ormore, 2ormore, 3ormore, 4ormore) (anode wire signals can be fed from adc16 or from adc32 signals)
** anode wire coincidence trigger
** anode wire MLU (memory lookup) trigger

== NIM trigger from ADC front panel ==

Trigger path:
<pre>
NIM signal --> LEMO input on ADC front panel --> data encoder --> data link to trigger board --> data decoder --> mask --> nim-esata-grand-or --> trigger
</pre>

To enable NIM trigger, do this:
* connect NIM signal to ADC front panel input
** Note1: NIM signal must be inverted
** Note2: LEMO connectors should be set to "NIM input" mode by ADC on-board jumpers (see ADC manual)
* observe correct bit goes to zero in the TRG web page: data link high work should change:
<pre>
0xNF00'0000 (N is the module id) to
0xN700'0000 (left lemo) or
0xNB00'0000 (right lemo)
</pre>
* Note3: if data link bits do not change, most likely the LEMO connectors are set to "DAC output" mode. Try to use LEMO inputs of a different ADC.
* compute the NIM mask: there are 32 bits, 2 bits per ADC data link. Bit 0x1 is right LEMO, bit 0x2 is left LEMO
* for example, to use left LEMO of ADC data link 12, set nim mask to (0x2)<<(2*12) = 0x02000000
* development branch of the git repository contains ''scripts/nim_mask.py'', which can compute forwards and backwards
* initialize the trigger board (on the TRG web page press button "initialize"), or start a new run
* on the history plot with "NIM grand or" counter, the rate should change from zero
* if a run is active, if one of the ADC LEMO outputs is set to output the trigger signal as a NIM pulse, one should see it fire use a scope. The delay or jitter between NIM trigger in and NIM trigger out should be around 100-200 ns (FIXME - measure this!)
* to set the DAQ to trigger on the NIM signal, on the run start page, check the box "TrigEsataNimGrandOr".
** Note: remember to set the scaledown to 0 if so desired.

= TPC field wire pulser =

The TPC field wire pulser input should be connected directly to the DAC output of adc01 (right lemo connector).

DAC settings in ODB /Equipment/CTRL/Settings/ADC/DAC/

* for pulsing PWBs:
** dac_baseline: -8000
** dac_amplitude: 4000
** ramp_up_rate: 128
** ramp_down_rate: 70
** ramp_top_len: 128

* for pulsing AW: (PWB settings drive the ADCs out of range!)
** dac_baseline: -2000
** dac_amplitude: 4000
** ramp_up_rate: 20
** ramp_down_rate: 20
** ramp_top_len: 128

= TDC connections =

* SFP8 - fiber SFP to Juniper switch
* right RJ45 - 200MHz clock
* left RJ45 - external trigger

= MIDAS Data banks =

* ATAT - fectrl - UDP packets from GRIF-C/TRG trigger board
* TRBA - fetdc - TDC data
* TRTM - fetdc - TDC FPGA temperatures, TID_FLOAT[5].
* AAnn - feudp - UDP packets from GRIF-16 ADC (before the event builder), "nn" is the ADC module number: adc01 is AA01 through adc20 is AA20.
* BBxx - feudp - UDP packets from PWB, obsolete
* Bnnm, Cnnm - feevb - EVB output banks: ADC data, "nn" is the ADC module number (1..20), "m" is the channel number (0..9,A..F), format is same as AAnn banks.
* PAnn - feudp - UDP packets from PWB, "nn" is the PWB module number (00..99), obsolete
* PBnn - feudp - UDP packets from PWB, "nn" is the PWB module number (00..99), PB01 is pwb01, PB78 is pwb78
* PCnn - feevb - EVB output banks: PWB data, "nn" is the PWB module number (00..99), same as above, format is same as PBnn banks

= ZZZ =

ZZZ

Daq

2020-12-23T22:55:46Z

Acapra: correct git repo link

= Links =

* https://alphacpc05.cern.ch/agdaq/ -- midas daq page for the agdaq system at CERN
* https://daq16.triumf.ca -- midas daq page for the agmini system at TRIUMF
* https://daq.triumf.ca/elog-alphag/alphag/ -- elog
* https://daqstore.triumf.ca/AgWiki/index.php/Main_Page -- this wiki
* https://bitbucket.org/ttriumfdaq/agdaq - git repository for agdaq
* https://bitbucket.org/expalpha-- git repository on bitbucket
* https://edev-group.triumf.ca/fw/exp/alphag -- git repository on edev gitlab
* https://ladd00.triumf.ca/daqinv/frontend/list/170 - PWB inventory database
* https://ladd00.triumf.ca/daqinv/frontend/list/168 - ADC (GRIF-16) inventory, FMC-ADC32rev0,rev1 inventory
* https://ladd00.triumf.ca/daqinv/frontend/list/176 - FMC-ADC32rev1.1 inventory

= Hardware manuals =

* [[TRG]] - trigger board (GRIF-C) manual
* CDM - https://daq.triumf.ca/DaqWiki/index.php/VME-CDM -- clock VME-CDM manual
* ADC - https://daq.triumf.ca/DaqWiki/index.php/VME-GRIF-ADC16-Rev1 -- GRIF-16 ADC manual
* [[PWB]] - TPC Pad Wing Board manual
* TDC - https://www.triumf.info/wiki/DAQwiki/index.php/GSI_TRB3
* [[chronobox]] - chronobox manual

= Performance =

Full system: (ADC fw rel-20201112-ko, PWB fw 20201005_ko)
* 1 Hz stable, ADC adc32 9 supp off rate 0.3 M/s, adc16 off, PWB udp_delay 25 supp off active 64 rate 6.6+4+6+4.6=21.2 M/s, TDC off, total rate 21 M/s
* 500 Hz stable, ADC adc32 9 supp 1500 rate 6 M/s, adc16 off, PWB udp_delay 10 supp 100 active 63 rate 3+4+3+3=13 M/s, TDC off, total rate 19 M/s

Single ADC: (fw rel-20201112-ko)

Fully suppressed data, sthreshold 5000
* 1 Hz: stable
* 500 Hz: stable (nominal max trigger rate)
* 6700 Hz: stable, 8 Mbytes/sec
* 6800 Hz: evb eventually confused
* 6850 Hz: evb confused, no lost triggers in the ADC, no crash in the ADC

All data, 510 samples/event, sthreshold 10 (fw rel-20201122-ko)
* 1 Hz: stable
* 100 Hz: stable, 3.4 Mbytes/sec
* 500 Hz: stable, 17 Mbytes/sec (nominal max trigger rate)
* 1000 Hz: stable, 32 Mbytes/sec
* 3100 Hz: stable, 106 M/s (approaching saturation of 1gige ethernet)
* 3200 Hz: stable, 110 M/s - TRG communication is in trouble
* 3300 Hz: stable, 116 M/s
* 3400 Hz: incomplete events, no ADC crash ("dropped due to full" counting, "dropped due to busy" are zero, this is as expected).

no suppression (different fpga data path)
* 1 Hz: stable
* 100 Hz: stable
* 500 Hz: stable, 17 M/s (nominal max trigger rate)
* 1000 Hz: stable, 32 M/s
* 3000 Hz: stable, 100 M/s (approaching saturation of 1gige ethernet)
* 3300 Hz: incomplete events, no ADC crash ("dropped due to full" counting, "dropped due to busy" are zero, this is as expected).

= Network connections =

The experiment private network number is: 192.168.1.x, gateway 192.168.1.1, netmask 255.255.255.0

network map:
<pre>
---- TRIUMF/CERN network --1gige-- alphagdaq --10gige-- juniper switch --1gige-- centrecom switch
</pre>

== alphagdaq network connections ==

on the rear of the machine:
<pre>
-----------------------------------------
| | rj45 | rj45 | | sfp | sfp |
-----------------------------------------

- left rj45 - copper 1gige - eno1 - spare (inactive)
- right rj45 - copper 1gige - eno2 - dhcp - cern/triumf network
- left sfp - 10gige - enp1s0f1 - spare (inactive)
- right sfp - 10gige - enp1s0f0 - static 192.168.1.1
</pre>

== connections between switches ==

* alphagdaq to juniper switch: direct attach (DAC) 10gige cable into any port of juniper switch
* juniper switch to centrecom switch: rj45 sfp in any port of juniper switch - cat5e/cat6 cable - to any port of centrecom switch
* cat5e/cat6 cable from any port of centrecom switch to the XXX port on the back of the juniper switch

== juniper switch ==

connected to juniper switch front is:

* everything that sends event data to alphagdaq, specifically:
** TRG (rj45 sfp)
** 16x ADC (fiber sfp)
** 64x PWB (fiber sfp)
* 10gige uplink to alphagdaq (10gige DAC cable)
* 1gige link to centrecom switch (rj45 sfp)
* 40gige ports are not used

connected to the juniper switch rear:

* CON is the serial console via RJ45-to-DB9 to USB-serial adapter to alphagdaq USB port (9600bps, 8N1, no flow control)
* C0 is the management ethernet interface port, connect to any port on the centrecom switch (juniper-private will ping)

== centrecom switch ==

* everything with copper rj45 connections, specifically:
** CDM boards
** HV, LV and VME power supplies
** RaspberryPi3 boards
** gas handling MFCs (algas)
** cooling system controller (moxa01)

= USB connections =

Note:
* alphagdaq has USB2 and USB3 connections. They are not the same, they are not generally interchangeable.
* USB3 connections and cables are generally blue coloured.

List of USB connections from alphagdaq:

* front USB3 port (blue): USB-A-to-B cable to USB blaster connected to the trigger board (front of rack)
* rear USB3 port (blue): USB3 short jumper to USB3 hub
* rear USB2 port (black): USB2 unpowered hub (Logiix) to USB-A-to-B cable to UPS USB connector
* rear USB2 port (black): USB-to-Serial adaper to DB9 to RJ45 adapter to Juniper switch console port

Note: UPS USB connection does not work with alphagdaq USB ports (both usb2 and usb3), but combination
using the unpowered USB2 hub seems to work ok.

List of USB connections from the USB hub:

* this is a powered USB3 hub
* USB-A-to-MicroUSB - to lvdb boards
* USB-A-to-Wiener interlock cable

= Clock and trigger distribution =

== Explanation ==

Trigger is generated by the trigger board (aka TRG, aka GRIF-C),
from the TRG eSATA output through the eSATA splitter it is fed into
the master CDM. The master CDM outputs the trigger on all MiniSAS output ports.
One of the master CDM MiniSAS outputs is connected to the CDM slave eSATA input.
The same as the master CDM, the slave CDM outputs the trigger on all MiniSAS output ports.

The master 62.5MHz clock is generated by the master CDM using the internal 10MHz oscillator
or the external 10MHz AD clock. Also on MiniSAS port 6 the master CDM generates
the 200MHz clock for the TDC. This clock is frequency-locked to the master 62.5MHz clock.

Master CDM clock selection (external CD 10MHz clock or internal oscillator 10MHz clock)
is done by a magic esper command (see below).

From the master CDM, the clock is fed to: the slave CDM, the TRG (via the eSATA splitter)
and to the TDC (via the RJ45 splitter).

The slave CDM sends the clock and the trigger to the ADCs and PWBs.

== Schematic ==

<pre>
TRG (GRIF-C)
------------
eSATA <------> eSATA splitter

eSATA splitter
--------------
trigger ---> eSATA ---> CDM-Master eSATA (trigger signal)
clock <--- (eSATA --- MiniSAS) <--- CDM MiniSAS (62.5MHz clock)
</pre>

<pre>
CDM-Master
----------
eSATA <--- eSATA splitter <--- eSATA from TRG (trigger signal)
MiniSAS 1 --> (MiniSAS --- eSATA) --> CDM-Slave eSATA (62.5 MHz)
MiniSAS 6 --> (MiniSAS --- eSATA) --> eSATA splitter --> RJ45 --> TDC (200 MHz)
</pre>

<pre>
CDM-Slave
---------
eSATA <--- trigger and 62.5MHz clock from CDM-Master
MiniSAS 1 --> ADC trigger and 62.5MHz clock
MiniSAS 2 --> same
MiniSAS 3 --> same
MiniSAS 4 --> same
MiniSAS 5 --> PWB trigger and 62.5MHz clock
MiniSAS 6 --> same
</pre>

== Setup of CDM boards ==

ALPHA-g uses the Rev3 CDM boards. The 4 CDM boards that belong to ALPHA-g
have serial numbers 0, 1, 2 and 3 and are labeled cdm00, cdm01, cdm02, cdm03.

All the boards can be used as master and slave positions, but have to be
configured appropriately.

=== Slave setup ===

* install the board
* connect ethernet
* connect esata external clock
* esper-tool cdmNN ### i.e. esper-tool cdm01
* cd template
* write current_setup 4
* cd /cdm
* read
* observe clock frequency counters report correct values. "ext_clk" will only count if the lemo jumper is installed (see below)
<pre>
11 esata_clk uint32 R [62500246]
12 ext_clk uint32 R [62500246]
</pre>
* cd /lmk
* read
* observe both PLL1 and PLL2 are locked: both values should be "1"
<pre>
144 pll1_ld uint8 R [1]
147 pll2_ld uint8 R [1]
156 ld1_counter uint32 RW [3]
157 ld2_counter uint32 RW [3]
</pre>

If needed, check correct operation of the pll lock monitoring:

* unplug the esata cable from the master CDM
* "red" light should go off
* read of lmk should report "pll1_ld" value 0 and "ld1_counter" and "ld2_counter" should increment
* reconnect the esata cable
* "red" light should return
* read of lmk should report both pll1_ld and pll2_ld locked (values "1").

=== Master setup ===

* install the board
* connect ethernet
* esper-tool cdmNN ### i.e. esper-tool cdm01
* cd template
* write current_setup 3
* cd /lmk
* write clkin_sel_mode 0
* cd /cdm
* read
* observe clock frequency counters report correct values. "ext_clk" will only count if the "clock loopback" lemo jumper is installed (see below)
<pre>
12 ext_clk uint32 R [62500189]
</pre>
* cd /lmk
* read
* observe both PLL1 and PLL2 are locked: both values should be "1"
<pre>
144 pll1_ld uint8 R [1]
147 pll2_ld uint8 R [1]
156 ld1_counter uint32 RW [3]
157 ld2_counter uint32 RW [3]
</pre>

Check the 200MHz clock:
* connect MiniSAS-eSATA splitter cable to MiniSAS output 6, connect eSATA end to CDM eSATA input
* in esper-tool, read cdm: esata_clk should report 200MHz clock
<pre>
11 esata_clk uint32 R [200000603]
12 ext_clk uint32 R [62500188]
</pre>

=== Master setup with external 10 MHz clock ===

* setup master CDM as above (using internal clock)
* connect 10MHz external clock to right LEMO #1
* esper-tool cdmNN
* cd /cdm
* write sel_nim 1 # if clock is NIM signal
* write sel_nim 0 # if clock is TTL signal (i.e. chronobox LEMO output)
* read ext_clk
<pre>
[http://cdm00:/cdm]> read ext_clk
9999556
</pre>
* if ext_clk reads 0 or does not read something close to 10MHz, stop here. debug the signal, try changing sel_nim, etc.
* cd /lmk
* write clkin_sel_mode 2 # select external clock
* read
* observe both PLL1 and PLL2 are locked: both values should be "1"
<pre>
150 clkin2_sel bool R [True]
144 pll1_ld uint8 R [1]
147 pll2_ld uint8 R [1]
</pre>
* on the MIDAS TRG page, observe that TRG measured external clock frequency is close to 62.5 MHz

== clkin_sel_mode positions ==

* 0 = 10MHz internal oscillator (use on master CDM in standalone mode)
* 1 = eSATA clock (use on slave CDM)
* 2 = external clock, should be 10MHz (use on master CDM to run from AD clock)

== LEMO connections ==

* clock loopback jumper: to read the frequency of the generated 62.5MHz clock, connect LEMO cable from left LEMO #2 to the right LEMO #1
* 10MHz internal clock jumper: to feed 10MHz internal oscillator to the external clock, connect LEMO cable from right LEMO #3 to the right LEMO #1.
* 10MHz external clock: to feed 10MHz external clock, connect LEMO cable to right LEMO #1

= MIDAS frontends =

== UDP ==

feudp receives udp packets from the ADC and PWB boards. For each udp packet 1 midas event is created,
with 1 data bank containing the udp data. Events are sent into the MIDAS event buffer BUFUDP. The names
of the data banks are assigned in ODB /eq/feudp/settings.

{ADC,PWB} --> 1gige --> switch --> 10gige --> alphagdaq --> feudp -> BUFUDP

== CTRL ==

fectrl controls all the ADC, PWB and trigger boards. It provides run start and stop sequencing,
runs the event synchronization sequence at the begin of the run, and records slow controls data (temperatures,
voltages, etc).

fectrl receives UDP data packets from the trigger board and sends them to the MIDAS event buffer BUFUDP.

fectrl receives commands from the control web pages (for ADC, PWB and trigger) to reboot, initialize, etc the boards.

fectrl configures the event builder via odb /eq/fectrl/evbconfig.

ADC <-> http esper <-> fectrl -> slow control data into midas history

PWB <-> http esper <-> fectrl -> slow control data into midas history

TRG <-> udp comm <-> fectrl -> BUFUDP, slow control and counters into midas history

fectrl <-> midas rpc <-> mhttpd <-> json rpc <-> control web pages for ADC, PWB and trigger

== EVB ==

feevb receives udp packet data from the MIDAS event buffer BUFUDP, inspects the packet timestamps
and collects the data with matching timestamps into physics events. feevb has provisions to do
data suppression, reduction and compression in addition to the data reduction done
in the ADC and PWB firmware.

feevb event synchronization relies on the event synchronization sequence done by fectrl when a run is started.

{ADC, PWB, TRG} -> BUFUDP -> feevb -> SYSTEM -> mlogger -> compression -> disk storage

=== internal structure ===

* event buffer readers, instances for BUFTRG, BUFTDC, BUFADC, BUFUDP
<pre>
BufferReader class - reads one midas event buffer:
- BufferReaderThread()
-- in batches of 1000 events
--- lock
--- TickLocked()
----- ReadEvent() -> bm_receive_event_alloc()
----- fHandler->HandleEvent()
-- fHandler->XMaybeFlushBank()
-- sleep 100 usec
</pre>
* event header decoder and intermediate buffer
<pre>
Handler class - accumulates events, sends them to the EVB in batches

- HandleEvent() <- BufferReaderThread() - receive incoming events
-- loop over all banks, examine bank name
--- call AddAdcBank(), AddPwbBank(), AddTrgBank(), AddTdcBank() according to bank name
---- AddXxxBank() -> XAddBank()
--- unknown banks go directly to evb->SendQueue
-- check if synchronization has completed

- XAddBank() - save BankBuf objects in a buffer

- XMaybeFlushBank(), XFlushBank() - send all accumulated data to the Evb class
-- evb->lock()
-- for each buffered BankBuf objects,
--- call evb->AddBankLocked()
-- buffer is now empty
</pre>
* event builder
<pre>
Evb class - assemble event fragments according to timestamps

- fEvents fifo (deque) - time sorted list of partially assembled events
-- new entries are added by FindEvent()
-- entries are modified by MergeSlot()
-- old entries are removed by GetLocked()

- AddBankLocked() <- XFlushBank() <- BufferReaderThread()
-- call fSync->Add()
-- transform BankBuf to EvbEventBuf
-- push it into per-module buffer fBuf[imodule]

- EvbThread()
-- EvbTickLocked()
--- BuildLocked()
---- in a loop, for no more than 1 sec
----- loop over all per-module buffers fBuf
------ pop first entry, pass it to BuildSlot()
------- call FindEvent()
------- call MergeSlot()
------- call CheckEvent()
--- in a loop
---- get all completed events via GetLocked() pass them to SendQueue->PushEvent()
-- if done nothing, sleep 100 usec, otherwise nominal sleep to free the evb lock

- GetNext() <- GetLocked() <- EvbThread() - get completed events
-- examine oldest EvbEvent entry in fEvents:
-- pop it if it is complete
-- pop it if it is too old (to prevent old incomplete events from stalling the evb) ("pop age" on the evb page)
-- pop it if a following event is complete ("pop following" on the evb page)
</pre>
* send queue
<pre>
SendQueue class - prepare events for writing to midas SYSTEM event buffer

- PushEvent() <- EvbThread()
-- push FragmentBuf into the queue

- SendQueueThread()
-- in a batch of 1000 events, call Tick()
--- pop FragmentBuf from the queue, call SendEvent()
--- compose midas event (memcpy()!)
--- TMFE fEq->SendEvent()
--- bm_send_event()
-- sleep 100 usec
</pre>
* equipment object
<pre>
EvbEq class - implements the midas equipment
- HandleBeginRun()
- HandleEndRun()
- HandlePeriodic()
</pre>

=== multithreading ===

* BufferReaderThread() one per midas event buffer
** high CPU use: bm_receive_event(), AddXxxBank()
** lock contention: midas event buffer lock, evb lock into XFlushBank->AddBankLocked()
* EvbThread()
** O(size of fEvents): FindEvent(), GetNext()
** lock contention: evb lock against XFlushBank(), SendQueue lock (1 per output event)
* SendQueueThread()
** high CPU use: memcpy() in SendEvent(), memcpy() in bm_send_event
** lock contention: event queue lock against EvbThread, midas event buffer lock
* main thread
** lock contention: evb lock in ReportEvb() and elsewhere

= ODB entries =

== /Equipment/CTRL/Settings ==
* TBW

== /Equipment/CTRL/Settings/TrigSrc - trigger source selection ==
* TrigPulser - trigger on the pulser
* TrigEsataNimGrandOr - trigger on external NIM inputs of GRIF-16 ADC modules
* TrigAdc16GrandOr - trigger on any adc16 signals (see adc16_masks)
* TrigAdc32GrandOr - trigger on any adc32 signals (see adc32_masks)
* Trig1ormore - trigger on adc16 multiplicity 1 or more
* Trig2ormore - ditto, 2 or more
* Trig3ormore - ditto, 3 or more
* Trig4ormore - ditto, 4 or more
* TrigAdcGrandOr - trigger on any ADC signal
* TrigAwCoincA - trigger on TPC AW coincidence (see [[TRG]] manual)
* TrigAwCoincB - ditto
* TrigAwCoincC - ditto
* TrigAwCoincD - ditto
* TrigAwCoinc - ditto
* TrigAw1ormore - trigger on TPC AW per-preamp multiplicity 1 or more (see [[TRG]] manual)
* TrigAw2ormore - ditto, 2 or more
* TrigAw3ormore - ditto, 3 or more
* TrigAw4ormore - ditto, 4 or more
* TrigAwMLU - trigger on TPC AW MLU signal (see [[TRG]] manual)
* TrigBscGrandOr - trigger on any BSC signal
* TrigBscMult - trigger on predefined BSC multiplicity (see [[TRG]] manual)
* TrigCoinc - trigger on predefined coincidence of TPC AW, BSC and external signal (see [[TRG]] manual)

== /Equipment/CTRL/Settings/TRG - trigger settings ==
* Enable - if set to "n", fectrl will not do anything with the trigger board. Normal value "y"
* Modules - hostnames of the trigger board. Normal value "alphat01"
* NimMask - 32-bit mask for the 32x NIM inputs (see TRG manual). Normal value 0, unless external trigger is in use.
* EsataMask - 32-bit mask for the 32x eSATA signals (see TRG manual). Normal value 0.
* adc16_mask[16] - 16x 16-bit mask for the 100MHz GRIF-16 onboard ADCs (see TRG manual). Normal value 0xFFFF for all 16 data links.
* adc32_mask[16] - 16x 32-bit mask for the 62.5MHz FMC-ADC32 ADCs (see TRG manual). Normal value 0xFFFFFFFF for all 16 data links.
* PassThrough - if set to "y", trigger is passed through the trigger module without generating any events and without causing deadtime.
* AwCoincA, B, C and D - 32-bit masks for the aw16 coincidence trigger (see TRG manual)
* Aw16FromAdc16 - route AW trigger (aw16) from adc16 (100MHz ADCs 0..15)
* Aw16FromAdc32a - route AW trigger (aw16) from adc32 group 15..0 (62.5MHz FMC-ADC32 in ADCs 0..7)
* Aw16FromAdc32b - route AW trigger (aw16) from adc32 group 31..16 (62.5MHz FMC-ADC32 in ADCs 8..15)

== /Equipment/CTRL/Settings/PWB - PWB settings ==
* Enable - if set to "n", fectrl will not do anything with PWBs. Normal value "y".
* enable_boot_user_page - if set to "n", fectrl will not try to reboot PWBs into the user page firmware. Normal value "y".
* enable_trigger - if set to "n", all PWBs will be set to ignore the trigger. Normal value "y".
* enable_trigger_group_a - field wire pulser defeats the data suppression (every channel has a pulse) resulting in too much data overwhelming the DAQ (lost UDP packets). To permit correct data taking of field wire pulser data, PWBs are split into two groups (a and b), only one of the two groups should be enabled for field wire pulser runs. Both groups should be enabled for normal data taking. Normal value is "y"
* enable_trigger_group_b - see "enable_trigger_group_a". Normal value is "y".
* modules[64] - hostnames of PWB boards
* boot_user_page[64] - if set to "y" and enable_boot_user_page is "y", fectrl will reboot the corresponding PWB to the user page firmware.
* trigger[64] - if set to "y" and enable_trigget is "y", fectrl will set the PWB to accept the trigger.
* ch_enable - if set to "n" disables all PWB channels. Normal value is "y".
* ch_force - is set to "y" disables channel suppression (all PWB channels are read)
* suppress_reset - if set to "y" enables channel suppression for reset channels (threshold has to be set correctly)
* suppress_fpn - if set to "y" enables channel suppression for fpn channels (threshold has to be set correctly)
* suppress_pads - set to "y" enables channel suppression for TPC pad channels (threshold has to be set correctly)
* disable_reset1 - data suppression algorithm does not work for the channel "reset1" and it has to be suppressed explicitly by setting this value to "y". Normal value is "y".
* baseline_{reset,fpn,pads}[64] - waveform baseline for each PWB position
* threshold_{reset,fpn,pads} - waveform suppression threshold, see below:
* ch_threshold - PWB sca_N_ch_threshold, if set to 0, threshold is computed as "baseline_pads[seqpwb]-threshold_pads". Normal value is 0.
* test_mode - enable PWB test mode, the ADC data is replaced with a test pattern, see PWB esper variable signalproc/test_mode

== /Equipment/CTRL/Settings/TDC - TDC settings ==
* Enable - if set to "n", fectrl will tell the event builder that the TDC is not running. Normal value is "y".

== /Equipment/CTRL/EvbConfig - EVB configuration ==
All arrays have the same size corresponding to the number of EVB slots.
* name[] - slot name for the web page (trg01, adc01, adc01/adc32, pwb31/0, tdc01, etc)
* type[] - slot type: 1=TRG, 2=ADC16 and ADC32, 3=obsolete PWB, 4=obsolete PWB, 5=PWB, 6=TDC
* module[] - module number (adcNN, adcNN+100, pwbNN, etc)
* nbanks[] - number of expected banks (16 for ADC16, 32 for ADC32, not used, set to 1 for all others)
* tsfreq[] - timestamp frequency in Hz.

= Trigger configuration =

These trigger modes are implemented:
* (manual trigger from web page)
* software pulser trigger (from fectrl)
* hardware pulser trigger (from the trigger board)
* NIM and eSATA trigger from ADC front panel inputs
* adc16 and adc32 discriminator triggers:
** adc16 grand-or trigger
** adc32 grand-or trigger
** adc (adc16+adc32) grand-or trigger
** adc16 multiplicity trigger (1ormore, 2ormore, 3ormore, 4ormore) for use with barrel scintillator
** anode wire multiplicity trigger (1ormore, 2ormore, 3ormore, 4ormore) (anode wire signals can be fed from adc16 or from adc32 signals)
** anode wire coincidence trigger
** anode wire MLU (memory lookup) trigger

== NIM trigger from ADC front panel ==

Trigger path:
<pre>
NIM signal --> LEMO input on ADC front panel --> data encoder --> data link to trigger board --> data decoder --> mask --> nim-esata-grand-or --> trigger
</pre>

To enable NIM trigger, do this:
* connect NIM signal to ADC front panel input
** Note1: NIM signal must be inverted
** Note2: LEMO connectors should be set to "NIM input" mode by ADC on-board jumpers (see ADC manual)
* observe correct bit goes to zero in the TRG web page: data link high work should change:
<pre>
0xNF00'0000 (N is the module id) to
0xN700'0000 (left lemo) or
0xNB00'0000 (right lemo)
</pre>
* Note3: if data link bits do not change, most likely the LEMO connectors are set to "DAC output" mode. Try to use LEMO inputs of a different ADC.
* compute the NIM mask: there are 32 bits, 2 bits per ADC data link. Bit 0x1 is right LEMO, bit 0x2 is left LEMO
* for example, to use left LEMO of ADC data link 12, set nim mask to (0x2)<<(2*12) = 0x02000000
* development branch of the git repository contains ''scripts/nim_mask.py'', which can compute forwards and backwards
* initialize the trigger board (on the TRG web page press button "initialize"), or start a new run
* on the history plot with "NIM grand or" counter, the rate should change from zero
* if a run is active, if one of the ADC LEMO outputs is set to output the trigger signal as a NIM pulse, one should see it fire use a scope. The delay or jitter between NIM trigger in and NIM trigger out should be around 100-200 ns (FIXME - measure this!)
* to set the DAQ to trigger on the NIM signal, on the run start page, check the box "TrigEsataNimGrandOr".
** Note: remember to set the scaledown to 0 if so desired.

= TPC field wire pulser =

The TPC field wire pulser input should be connected directly to the DAC output of adc01 (right lemo connector).

DAC settings in ODB /Equipment/CTRL/Settings/ADC/DAC/

* for pulsing PWBs:
** dac_baseline: -8000
** dac_amplitude: 4000
** ramp_up_rate: 128
** ramp_down_rate: 70
** ramp_top_len: 128

* for pulsing AW: (PWB settings drive the ADCs out of range!)
** dac_baseline: -2000
** dac_amplitude: 4000
** ramp_up_rate: 20
** ramp_down_rate: 20
** ramp_top_len: 128

= TDC connections =

* SFP8 - fiber SFP to Juniper switch
* right RJ45 - 200MHz clock
* left RJ45 - external trigger

= MIDAS Data banks =

* ATAT - fectrl - UDP packets from GRIF-C/TRG trigger board
* TRBA - fetdc - TDC data
* TRTM - fetdc - TDC FPGA temperatures, TID_FLOAT[5].
* AAnn - feudp - UDP packets from GRIF-16 ADC (before the event builder), "nn" is the ADC module number: adc01 is AA01 through adc20 is AA20.
* BBxx - feudp - UDP packets from PWB, obsolete
* Bnnm, Cnnm - feevb - EVB output banks: ADC data, "nn" is the ADC module number (1..20), "m" is the channel number (0..9,A..F), format is same as AAnn banks.
* PAnn - feudp - UDP packets from PWB, "nn" is the PWB module number (00..99), obsolete
* PBnn - feudp - UDP packets from PWB, "nn" is the PWB module number (00..99), PB01 is pwb01, PB78 is pwb78
* PCnn - feevb - EVB output banks: PWB data, "nn" is the PWB module number (00..99), same as above, format is same as PBnn banks

= ZZZ =

ZZZ

Detector Services

2019-01-09T17:35:39Z

Acapra:

Back to [[Main Page]]

= Table of Figures =

[[#_Toc493684884|Figure 1 - Gas system schematic 6]]

[[#_Toc493684885|Figure 2 - A half-rack space is used for the gas control implementation. 7]]

[[#_Toc493684886|Figure 3 - Gas system half rack ready for installation 8]]

[[#_Toc493684887|Figure 4 - Cathode pad half-cylinder. Pairs of cooling pipes are visible 9]]

[[#_Toc493684888|Figure 5 - Water cooling system during test. 9]]

[[#_Toc493684889|Figure 6 - Model and 3D-printed realization of the dry-air manifold, very similar to the manifolds for the water cooling system. 12]]

[[#_Toc493684890|Figure 7 - Overall water cooling system 13]]

[[#_Toc493684891|Figure 8 - Temperature Monitor Board 14]]

[[#_Toc493684892|Figure 9 - End plate view of the layout of the different electronics boards & HV connections 15]]

[[#_Toc493684893|Figure 10 – Unpotted Anode Wire Card and Board with HV distribution 17]]

[[#_Toc493684894|Figure 11 - Anode wire pins and Field wire pins under yellow cover 17]]

[[#_Toc493684895|Figure 12 – New Anode Wire card with 16 potted HV capacitors, gold crimp pin Mill-max socket to the anode wire 18]]

[[#_Toc493684896|Figure 13 - Anode wire pre-amp power distribution & 16 channel temperature boards 19]]

[[#_Toc493684897|Figure 14 - Overall Low voltage distribution for anode wires and cathode pads 21]]

[[#_Toc493684898|Figure 15 - High Voltage & Grounding scheme for the rTPC 22]]

[[#_Toc493684899|Figure 16- Alpha-g rTPC rack organization 23]]

= Introduction =



This document is to describe the different services for the rTPC detector. 3 main sections cover the ''Gas system'', the ''Water cooling system'' and the ''Power distribution''.

= References and related Documents =

A dedicated TRIUMF Wiki site gathers all documentations for the rTPC detector system and equipment. Gas system, cooling water system and power distribution can be found there.

= Gas system =

The gas system as described below includes the entire gas infrastructure required to deliver gas to the detector, monitor and control the gas flow into the detector. It also includes the necessary equipment to notify any change from the nominal gas flow or gas mixture ratio delivered to the detector.

This type of mixed gas system is well understood by the Triumf detector group staff and no issue in its realization is expected.

This system is a stand-alone infrastructure assembled and delivered to CERN. It will be placed in the Alpha-g experimental area floor. The delivery of the different gases to the experimental area is under the responsibility of the Gas handling group from CERN.

The gas delivery system for the radial TPC (rTPC) is to provide Ar/CO2 gas mixture between 90/10 to 50/50 depending on the user requirements. This system does not recycle the gas as the environmental impact of either gas is minimal and the operation cost versus a recycling system would not be economic.

In the event of an additional quenching gas requirement, the overall gas system will have to be reconsidered and possible recycling system envisaged.

A simple gas mixer using digital flow meters will provide the precise mixture dialed by hand in the control system.

During the prototype phase, an ad-hoc gas system will be put in place. A portable gas system is available for such a test.

== Specifications ==

{|
! '''''Parameter'''''
! '''''Value'''''
|-
| Gas mixture
| Ar/CO2 (90/10..50/50)
|-
| Gas flow requirements
| 0.3l/min (50%), 0.3l/min (50%)
|-
| Detector gas Volume
| 0.18m3
|-
| Flushing time
| 5h @ 600cc/min
|-
| Flow rate
| 100 to 300 cc/minute
|}

== Operation ==

This gas system is quite simple as it takes 2 or 3 (spare) gas supplies and mixes them in a manifold before delivery to the rTPC detector. Necessary valves for purging lines and by-passing the detector are foreseen. The individual in-flows are controlled by gas dedicated mass flow controllers (MFC-x). The return flow is measured for leak assessment. An additional manometer monitors the internal detector pressure for evaluation of the electron drift characteristics.

[[File:imageservice2.png|575x315px|PA_doc-1.PNG]]

Figure 1 - Gas system schematic

The main flow controller and monitoring is performed by industrial devices from MKS. We added analog flowmeters on each gas line to provide a quick visual inspection of the gas mixing system.

This system will be hooked to the CERN AD-Gas facility. The CERN gas delivers gas through 2 switchable packs of 12 bottles for each gas.

The gas exhaust is evacuated to the exterior of the experimental area with an elevation of about 8m.

[[File:imageservice3.png|366x263px|PA_doc-2.PNG]]

Figure 2 - A half-rack space is used for the gas control implementation.



[[File:imageservice4.png|324x364px]]

Figure 3 - Gas system half rack ready for installation

== Quick Start ==
[[File:20181205 103217.jpg|thumb|Normal configuration of the manual valves of the GHS.]]
From-scratch (Neither gas supply lines to the GHS nor connections to and from the TPC connected) connection of the GHS should follow the following procedure:
# '''Flush supply lines:''' Gas lines from outside bottles going to the GHS should be briefly (5 min or so) flushed with a high gas flow ''before'' connecting to the GHS to remove possible dust, and to minimize the amount of air in the lines.
# '''Flush GHS:'''
## Close BV1 and BV3, open BV2 and BV4 to flush input lines, only briefly.
## Open BV1 and BV3, close BV2 and BV4 (normal operation configuration)
## With Midas frontend running, use the "Gas" custom page to set a desired flow, for flushing probably between 1l/min and 1.5l/min (Without Midas, use the telnet interface), then press the "Bypass Flow" button, and let flow for a while (5-15min?)
# '''Flush TPC input line:'''
## Connect the line that will go from the GHS to the TPC input-tee on the GHS side, but leave disconnected from the tee
## do the same for the pressure gauge line
## On the "Gas" custom page, press the "Flow TPC" button and let flow for 5 min (Without Midas, use the telnet interface).
# Make all remaining connections

== Troubleshooting ==
''' No flow '''

Make sure that the black valves on the gas rack at the entrance of the zone are open (right on the pictures).

[[File:20190109 165410.jpg|thumb|The rightmost valve (black) must be open during operation.]]

== Telnet Interface ==
If the Midas frontend is not available the GHS can be controlled via telnet. Connect with
telnet algas
then use the following command to control solenoid valves:
cowr do i 1
to energize SVi, i.e. to switch it from its normal closed or open state to the other state,
cowr do i 0
to de-energize, i.e. switch to default state.

Gas flow is set with:
mfcwr ao i <desired flow>
where i is 1 for Ar and 2 for CO2, and the desired flow is in the internal units of the system.

= Water cooling system =

The outer surface of the detector has an arrangement of 8x8 electronics boards handling the readout of the cathode pads. These boards require operating at a controlled temperature to prevent heat build-up at the surface of the detector and improving the reliability of the system. In order to do so, 8 independent water pipe loops running from the bottom-up and down in the longitudinal axis of the detector are in a thermal contact to 8 successive readout boards. From the bottom, a pair of water hoses connect to 2 custom 3D-printed manifolds supplying the water feed and return to the 8 water loops.

These 8 loops are run in parallel to a negative pressure water cooling system. This negative pressure system is to ensure that no water leak will damage the electronics even during possible pipe or delivery hose breakage.

[[File:imageservice5.png|516x273px]]

Figure 4 - Cathode pad half-cylinder. Pairs of cooling pipes are visible

[[File:imageservice6.jpg|249x429px]]

Figure 5 - Water cooling system during test.

== Specifications ==

{|
! '''''Parameters'''''
! '''''Values'''''
|-
| Number of parallel water loops
| 8
|-
| Copper pipe OD/ID
| ¼” / 4.7mm
|-
| Length of one loop (on the rTPC)
| 4.6m
|-
| Water speed per loop
| …
|-
| Pressure drop per loop
| …
|-
| Power extraction per water loop
| 8x15W= 120W
|-
| Total cooling power
| 1KW
|-
| Inlet water temperature
| 16..20 degC (to be confirmed)
|-
| Outlet water temperature
| 20..24 degC
|-
| Total water flow
| 3..6 l/minute
|}

== Operation ==

''This description corresponds to how we think the system should work, but doesn't seem fully comatible with the behaviour of the system. E.g. the air pump only seems to turn on when PID1 falls below a certain value.''

The Pressure PID (proportional/integral/differential controller), PID2, monitors the air pressure in the vacuum vessel. When the pressure rises above a trigger value (perhaps 350mbar) it activates the aspirator vacuum pump to bring the pressure down to maintain a long time averaged pressure (say 300mbar).

This vacuum is then used to draw water through the detector into the vessel, from the Open Tank. The vacuum vessel's water level is monitored by PID1, using a differential pressure transducer measuring the weight of the water column by the difference between the air pressure in the top of the vessel and the water pressure at the bottom of the vessel. PID1 provides a variable signal to the Proportional Solenoid Valve, which varies the flow through the return loop around the (fixed flow) water pump, and thus changing the overall flow of water from the vacuum vessel to keep the water level in the vessel constant despite water being constantly drawn in by the vacuum. The water pulled from the vacuum vessel is pushed through the filter and into the Open Tank, completing the circulation.

Cooling is accomplished in the water pump leg of the system, as coolant pulled from the vacuum vessel passes through a heat exchanger coupled to CERN’s cooling water supply on its way to the pump. The rate that water flows through the system can be determined by many obstacles in the loop, but ideally it should be dominated by the position of the main Throttle Valve. Other valves should generally be set either fully open of fully closed, except for the Proportional Valve, whose position responds to the setting of the Throttle Valve.

Data outputs: there are five data outputs planned. The two process variables, the vacuum vessel absolute pressure, and differential water column pressure, will be sent to the Detector DAQ system. As well, the total flow rate is monitored by the flow meter, whose output is sent to the DAQ system. This data is not used by the circulation system itself. Likewise, the back pressure on the water filter is only sent to the DAQ system to indicate if the filter may need changing, and the temperature of the open tank is monitored and sent to the DAQ system, but currently there is no plan to use it in a temperature control feedback. The temperature of CERN's cooling water supply is not yet known, but we expect we can set the heat exchanger to run at a rate to hold the system temperature near 20°C without feedback control.

For the negative pressure to work effectively to prevent leaks, the water pressure in the detector ought to be down at least to half an atmosphere. With the detector cooling tubes 8x 4.6m of parallel 4.7mm ID copper tubes (2.3m down then 2.3 back-ups), and the supply using (perhaps 6m, 3 to the detector then 3 back) 9.7mm ID (=1/2" OD) poly tubing, the flow will need to be somewhat greater than the required 3 l/min in order to maintain that degree of vacuum in the detector, unless the Throttle Valve is placed upstream of the detector near the Open Tank, or an additional constriction is added in that region.

The water pump leg of the circuit residing in the water cooling/gas rack is placed far enough from the fringe field of the superconducting magnet to prevent disruptive effects on the solenoid valves and electric motors. Therefore the main 1/2" poly line to and from the water pump and its flow loop will add a pressure drop which is estimated at 0.4atm for a 10m line at 6 l/min (9.7mmID), which is within our requirements. As mentioned earlier, the splitting (and reassembly) of the line to the 8 cooling lines close to detector will be accomplished with the use of 2 separate manifolds equipped with temperature sensors for constant in/out global water temperature monitoring.

[[File:imageservice7.png|286x206px]][[File:imageservice8.jpg|286x206px]]

Figure 6 - Model and 3D-printed realization of the dry-air manifold, very similar to the manifolds for the water cooling system.

Currently, PVC valves with "true union" couplings are planned to be provided around the water pump to permit it to be swapped out from the system for servicing and a valved bypass to the water filter is provided for the same purpose to allow easily changing filter elements with the system running. True union valves are generally only available to 1/2NPT and larger, so an adapter fitting is required. The vacuum pump for the system is a small (2 x 5 x 7.5cm) cascaded dual Venturi aspirator, which is operated with site air, switched with two simultaneously operated normally closed solenoid valves, one for the site air, and one for the vessel vacuum input. If the system is leak free, the vacuum in the vessel should only be degraded by the dissolved air absorbed in the Open Tank and extracted in the vacuum vessel, which from past experience might mean running for under a minute every six hours or so.

The wetted surfaces of this system will be mainly PVC and Polyethylene for the valves, tubing and vessels, plus brass fittings, and the copper cooling tubes, and the surfaces of the heat exchanger, which will most likely be stainless steel. It may be possible to devise an equivalent of a bellows communicating with the Open Tank, so that it is only virtually open, thus sealing the cooling water from contaminants, which may allow us to run with just distilled water plus a strip of silver as a biocide (although I'm not sure if the mix of metals, copper plus stainless, may not interfere with the function of the silver, which is mostly used in computer cooling systems which are copper and plastic only). Otherwise, we will need a biocide and corrosion inhibitor. We have the latter, tolyltriazole, but the biocide is problematic. They tend to be short-lived; for systems which are open to atmosphere, typical industrial practice is chlorination, regularly refreshed, which builds up NaCl as it degrades, making the coolant corrosive. For T2K in Japan, we used the (human-nontoxic) "Germall", which has a two-year lifetime, after which when its effectiveness decays it turns into food, which is the case for any solute that is not actively biocidal. Eventually, any such additives build up and require the coolant to be flushed and replaced. Silver is a lovely method, but only works for extended periods with a sealed system which remains deoxygenated. My research so far has not turned up a good candidate which does not entail recurring changes of the coolant.

Power: the system needs to be able to run on European 220VAC power, but also operate here for testing purposes. The PIDs (Omega CN76160s) will accept both; for the rest of the system, I will have an a universal 90-240VAC input to 24VDC (5A) supply which will drive all the other components, including the pump, which will be a brushless 24VDC centrifugal pump, 1 - 1.5A range. The whole system will run about 125W.

Information on the temperature and flow rate of the CERN cooling water system will be provided later but we don’t anticipate any major issues about the sizing of the heat exchanger.

[[File:imageservice9.png|603x344px]]

Figure 7 - Overall water cooling system



== Temperature Measurement ==

[[File:imageservice10.jpeg|225x356px]] [[File:imageservice11.png|194x358px]]

Figure 8 Temperature Monitor Board

The water temperature is read by the Temperature Monitor Board (TEMPB). Thermistors located inside the manifold connect to a 16 bit ADC on the TEMPB which is read out by a Raspberry Pi mounted on board. In a similar manner the air system temperature is read with sensors in the air manifold. The TEMPB can provide temperature readout of 32 thermistors which connect to terminal blocks. The channels can also be readout differentially providing up to 16 channels for thermocouples. An on board thermistor allows for cold junction compensation. The board is powered via USB on the Raspberry Pi.

== Quick Start ==
This assumes all water lines and compressed air are connected, and both the blue water tank and the chiller are filled.

=== Begin running ===
# Close vent valve and main valve (3-way valve in T position)
# Switch on control box with power switch in the back
# confirm water pump is off, i.e. blue button is not illuminated
# Vacuum pump should now be running
# Open main valve, i.e. vertical connection open, connection to vent valve closed [[File:PWBmainOpen.jpg|thumb|Cooling system main valve open]]
# water should start flowing through the PWB cooling system and coming back through the clear pipe connected to the vacuum vessel
# Default settings are PID1 = 200, PID2 = 300, but may have been adjusted
# once the water level in the vacuum vessel has risen to ~20cm, turn on the water pump

=== Drain system ===
# from normal operation, i.e. vacuum pump and water pump running
# switch main valve to upside down T position
[[File:PWBmainVent.jpg|thumb|Cooling system main valve to vent valve]]
# open vent valve
# air should now enter the system, draining all water from the TPC pipes into the vacuum vessel
# water pump will drain the vacuum vessel into supply tank, turn it off before the vessel is completely empty to avoid air in the water pump
# switch off main power

== Troubleshooting ==
The water system is temperamental, and not intuitive at all. Common problems and solution attempts:

; water pump very loud, doesn't appear to pump water
: most likely air in the water pump. Open the vent of the vacuum vessel to allow water to enter pump by gravity alone, pump should stop screaming and start pumping. Close vent.
; vacuum pump comes on often (more then every 10-20 min)
: probably an air leak in the system. Check for bubbles and try to find leak.
; water level/flow/vacuum oscillate between extreme values
: unclear, try playing with PID settings, throttle valve, possibly orange ball valves by the water pump.

= Power distribution =

This section covers the different voltages and power requirements for the operation of the rTPC. This includes:

# High Voltage for the rTPC electron drift and amplification operation
# Low Voltage to the Anode wire frontend electronics
# Low Voltage to the Barrel Scintillator frontend electronics
# Low Voltage to the Cathode Pads electronics
# Grounding scheme of the overall rTPC

The provision of power to the different components of the detector is distributed from the top end plate of the rTPC.

[[File:imageservice12.png|507x354px]]

Figure 9 - End plate view of the layout of the different electronics boards & HV connections

== High Voltage for the rTPC ==

Three main components of the rTPC require High Voltage

* Main cathode wall (Negative ~5KV)
* Field wires (Negative ~ few hundred volts)
* Anode wires (Positive ~3.2KV )

These voltages are provided by a Quad High Voltage unit from CAEN. Three independent cables will be fed to the top of the rTPC.

High Voltage provision to the bottom endplate HV ring is taken from the central cathode near the bottom endplate.

The Field wire voltage is connected to all the field wire crimp-pins through a dedicated socket chain inserted on each of the crimp-pin on the top and bottom endplates. The field wires themselves interconnect the high voltage between the top and bottom pins.

The High Voltage for the anode wires is delivered to the Anode Wire Card (AWC) which will distribute it to its 16 wires, Figure 11. There are 16 AWC cards covering each end of the anode wires. All the top and bottom 32 AWC are connected serially. The anode wires themselves interconnect the high voltage between the top and bottom pins.

The anode wires pre-amplifiers boards (AWB) are sitting on the top of the Anode Wire Card (AWC) Figure 10. As the wire is powered by the HV, each anode wire has a high Voltage decoupling capacitor to isolate the pre-amp stage from the HV. This circuit resides on the AWC. The AWB will see the AC coupled anode signal only. With a new design of the AWC Figure 12, the decoupling capacitors are potted in epoxy to ensure proper isolation of the HV toward the surrounding circuit as well to prevent corona discharge due to relative humidity of the surrounding.

[[File:imageservice13.jpg|402x309px]]

Figure 10 – Unpotted Anode Wire Card and Board with HV distribution

[[File:imageservice14.jpg|402x309px]]

Figure 11 - Anode wire pins and Field wire pins under yellow cover

[[File:imageservice15.jpg|402x309px]]

Figure 12 – New Anode Wire card with 16 potted HV capacitors, gold crimp pin Mill-max socket to the anode wire

== Power distribution to the Anode wire frontend electronics ==

The AWB pre-amp board requires 2 voltages (+5V, -5V) which are fed through dedicated cable connections to each of the AWB. There are 16 AWB per end of the rTPC, therefore, 16 individual cable bundle will be reaching the top and another 16 the bottom of the rTPC. The cable bundle is to also provide the power return lines to the Power Supply.

The PL508 Low Voltage power Supply from Wiener will provide the main power source for all the 32 pre-amplifiers. A dedicated connection breakout board with current monitoring capability splits the source power to the 2 x 16 necessary connections to the Anode wire boards. Based on a RaspeberryPi card, a 16 power distribution board is already implemented for the prototype. This board has extra 4 channels of temperature monitoring which can be used later on. In the same process we produce a similar board for 16 channel of NTC, RTC temperature monitoring. They will be used for overall detector and cooling system monitoring.

{|
! '''''Component'''''
! '''''+5V Supply'''''
! '''''-5V Supply'''''
|-
| Anode Pre-amp
| 1.5W (0.3A)
| 1.5W (0.3A)
|}

[[File:imageservice16.jpg|239x363px]] [[File:imageservice17.png|239x363px]]

Figure 13 - Anode wire pre-amp power distribution & 16 channel temperature boards

== Power distribution to the Barrel Scintillator frontend electronics ==

Similar to the Anode wire pre-amplifier voltage distribution scheme, the Barrel Scintillators will require several low voltages for power and control. The Barrel Scintillator Boards (BSB) are mounted on both ends of the Scintillator bar with the same segmentation as the Anode wires i.e.: 16 BSB per end. The design of the electronic chain for the barrel scintillator is not fully designed yet.

The SiPM sensors require a bias voltage of the order of 28V. This voltage will be provided through the same mean as the low voltage power. Expected threshold voltage for the local signal level discriminator is also foreseen.

== Low Voltage to the Cathode Pads electronics ==

The cathode pads electronics mounted on the external cylinder surface is composed of the analog frontend and the digital conversion electronics. Two voltages are required (+5.3V, +2.3V). The delivery is done through dedicated “power bar” running the full length of the rTPC. A set of 3 bars covers the 8 axial cathode pad electronics boards. 8 of them cover the circumference of the rTPC.

{|
! '''''Component'''''
! '''''+5.3V Supply'''''
! '''''+2.3V Supply'''''
|-
| Padwing board
| 10W (1.8A)
| 2W (0.8A)
|}

Overall the low power supplies are provided by a PL508-3U with 8 modules of either single or dual channels from Wiener.

[[File:imageservice18.png|469x360px]]

Figure 14 - Overall Low voltage distribution for anode wires and cathode pads



== Grounding scheme of the overall rTPC ==

The grounding scheme challenge in this setup is the far distance between the 2 end of the rTPC (2.3m) and the lack of accessibility to the bottom of the detector. The proposal is to use a Copper sheet mounted on the inner surface of the inner cylinder. This ground path will connect the 2 endplates to the High Voltage return. All the pre-amps will merge their ground to this central ground.

Additional ground lugs have been included in the Anode wire pre-amplifier boards to strengthen the ground as each anode wire is readout out from both end. This may be used once the full detector is assembled to address possible ground loop issues.

[[File:imageservice19.png|439x409px]]

= Safety and Hazard considerations =

== Gas ==

The Ar and CO2 do not require special health safety system. The detector will be operated in the ventilated hall of the AD building at CERN.

Gas flow, detector gas pressure, and gas inlet temperature are constantly monitored and can trigger adequate response for shutdown or closure of the gas system for safety purpose.

== Cooling water ==

The water cooling system operates in negative pressure mode preventing any water leak to flood the detector outer surface. In addition water flow and inlet/outlet temperature are constantly monitored to allow fast response to off range parameter in order to shut down the electronics. The cooling water system requires a primary cooling tap water loop from the AD building. Water quality, water flow and water temperature are not under our responsibility. This aspect will be dealt during the initial phase of the rTPC commissioning at CERN.

== Power distribution ==

All electronics voltages are below 48V. No special safety circuits are envisaged.

The high-voltage circuit is fully protected outside the detector. The access to the high-voltage on the detector is restricted to the two end-plates. Interlock scheme will be put in place to ensure the shutdown of the HV in case of manual access to the detector itself.

The LV (Low Voltage) power units are also interlocked to prevent false powering sequence which can damage the downstream electronics modules.

[[File:imageservice20.png|571x411px]]

Figure 16- Alpha-g rTPC rack organization

Detector Services

2019-01-09T17:30:05Z

Acapra:

Back to [[Main Page]]

= Table of Figures =

[[#_Toc493684884|Figure 1 - Gas system schematic 6]]

[[#_Toc493684885|Figure 2 - A half-rack space is used for the gas control implementation. 7]]

[[#_Toc493684886|Figure 3 - Gas system half rack ready for installation 8]]

[[#_Toc493684887|Figure 4 - Cathode pad half-cylinder. Pairs of cooling pipes are visible 9]]

[[#_Toc493684888|Figure 5 - Water cooling system during test. 9]]

[[#_Toc493684889|Figure 6 - Model and 3D-printed realization of the dry-air manifold, very similar to the manifolds for the water cooling system. 12]]

[[#_Toc493684890|Figure 7 - Overall water cooling system 13]]

[[#_Toc493684891|Figure 8 - Temperature Monitor Board 14]]

[[#_Toc493684892|Figure 9 - End plate view of the layout of the different electronics boards & HV connections 15]]

[[#_Toc493684893|Figure 10 – Unpotted Anode Wire Card and Board with HV distribution 17]]

[[#_Toc493684894|Figure 11 - Anode wire pins and Field wire pins under yellow cover 17]]

[[#_Toc493684895|Figure 12 – New Anode Wire card with 16 potted HV capacitors, gold crimp pin Mill-max socket to the anode wire 18]]

[[#_Toc493684896|Figure 13 - Anode wire pre-amp power distribution & 16 channel temperature boards 19]]

[[#_Toc493684897|Figure 14 - Overall Low voltage distribution for anode wires and cathode pads 21]]

[[#_Toc493684898|Figure 15 - High Voltage & Grounding scheme for the rTPC 22]]

[[#_Toc493684899|Figure 16- Alpha-g rTPC rack organization 23]]

= Introduction =



This document is to describe the different services for the rTPC detector. 3 main sections cover the ''Gas system'', the ''Water cooling system'' and the ''Power distribution''.

= References and related Documents =

A dedicated TRIUMF Wiki site gathers all documentations for the rTPC detector system and equipment. Gas system, cooling water system and power distribution can be found there.

= Gas system =

The gas system as described below includes the entire gas infrastructure required to deliver gas to the detector, monitor and control the gas flow into the detector. It also includes the necessary equipment to notify any change from the nominal gas flow or gas mixture ratio delivered to the detector.

This type of mixed gas system is well understood by the Triumf detector group staff and no issue in its realization is expected.

This system is a stand-alone infrastructure assembled and delivered to CERN. It will be placed in the Alpha-g experimental area floor. The delivery of the different gases to the experimental area is under the responsibility of the Gas handling group from CERN.

The gas delivery system for the radial TPC (rTPC) is to provide Ar/CO2 gas mixture between 90/10 to 50/50 depending on the user requirements. This system does not recycle the gas as the environmental impact of either gas is minimal and the operation cost versus a recycling system would not be economic.

In the event of an additional quenching gas requirement, the overall gas system will have to be reconsidered and possible recycling system envisaged.

A simple gas mixer using digital flow meters will provide the precise mixture dialed by hand in the control system.

During the prototype phase, an ad-hoc gas system will be put in place. A portable gas system is available for such a test.

== Specifications ==

{|
! '''''Parameter'''''
! '''''Value'''''
|-
| Gas mixture
| Ar/CO2 (90/10..50/50)
|-
| Gas flow requirements
| 0.3l/min (50%), 0.3l/min (50%)
|-
| Detector gas Volume
| 0.18m3
|-
| Flushing time
| 5h @ 600cc/min
|-
| Flow rate
| 100 to 300 cc/minute
|}

== Operation ==

This gas system is quite simple as it takes 2 or 3 (spare) gas supplies and mixes them in a manifold before delivery to the rTPC detector. Necessary valves for purging lines and by-passing the detector are foreseen. The individual in-flows are controlled by gas dedicated mass flow controllers (MFC-x). The return flow is measured for leak assessment. An additional manometer monitors the internal detector pressure for evaluation of the electron drift characteristics.

[[File:imageservice2.png|575x315px|PA_doc-1.PNG]]

Figure 1 - Gas system schematic

The main flow controller and monitoring is performed by industrial devices from MKS. We added analog flowmeters on each gas line to provide a quick visual inspection of the gas mixing system.

This system will be hooked to the CERN AD-Gas facility. The CERN gas delivers gas through 2 switchable packs of 12 bottles for each gas.

The gas exhaust is evacuated to the exterior of the experimental area with an elevation of about 8m.

[[File:imageservice3.png|366x263px|PA_doc-2.PNG]]

Figure 2 - A half-rack space is used for the gas control implementation.



[[File:imageservice4.png|324x364px]]

Figure 3 - Gas system half rack ready for installation

== Quick Start ==
[[File:20181205 103217.jpg|thumb|Normal configuration of the manual valves of the GHS.]]
From-scratch (Neither gas supply lines to the GHS nor connections to and from the TPC connected) connection of the GHS should follow the following procedure:
# '''Flush supply lines:''' Gas lines from outside bottles going to the GHS should be briefly (5 min or so) flushed with a high gas flow ''before'' connecting to the GHS to remove possible dust, and to minimize the amount of air in the lines.
# '''Flush GHS:'''
## Close BV1 and BV3, open BV2 and BV4 to flush input lines, only briefly.
## Open BV1 and BV3, close BV2 and BV4 (normal operation configuration)
## With Midas frontend running, use the "Gas" custom page to set a desired flow, for flushing probably between 1l/min and 1.5l/min (Without Midas, use the telnet interface), then press the "Bypass Flow" button, and let flow for a while (5-15min?)
# '''Flush TPC input line:'''
## Connect the line that will go from the GHS to the TPC input-tee on the GHS side, but leave disconnected from the tee
## do the same for the pressure gauge line
## On the "Gas" custom page, press the "Flow TPC" button and let flow for 5 min (Without Midas, use the telnet interface).
# Make all remaining connections

== Troubleshooting ==
Make sure that the black valves on the gas rack at the entrance of the zone are open (right on the pictures).
[[File:20190109 165410.jpg|thumb|The rightmost valve (black) must be open during operation.]]

== Telnet Interface ==
If the Midas frontend is not available the GHS can be controlled via telnet. Connect with
telnet algas
then use the following command to control solenoid valves:
cowr do i 1
to energize SVi, i.e. to switch it from its normal closed or open state to the other state,
cowr do i 0
to de-energize, i.e. switch to default state.

Gas flow is set with:
mfcwr ao i <desired flow>
where i is 1 for Ar and 2 for CO2, and the desired flow is in the internal units of the system.

= Water cooling system =

The outer surface of the detector has an arrangement of 8x8 electronics boards handling the readout of the cathode pads. These boards require operating at a controlled temperature to prevent heat build-up at the surface of the detector and improving the reliability of the system. In order to do so, 8 independent water pipe loops running from the bottom-up and down in the longitudinal axis of the detector are in a thermal contact to 8 successive readout boards. From the bottom, a pair of water hoses connect to 2 custom 3D-printed manifolds supplying the water feed and return to the 8 water loops.

These 8 loops are run in parallel to a negative pressure water cooling system. This negative pressure system is to ensure that no water leak will damage the electronics even during possible pipe or delivery hose breakage.

[[File:imageservice5.png|516x273px]]

Figure 4 - Cathode pad half-cylinder. Pairs of cooling pipes are visible

[[File:imageservice6.jpg|249x429px]]

Figure 5 - Water cooling system during test.

== Specifications ==

{|
! '''''Parameters'''''
! '''''Values'''''
|-
| Number of parallel water loops
| 8
|-
| Copper pipe OD/ID
| ¼” / 4.7mm
|-
| Length of one loop (on the rTPC)
| 4.6m
|-
| Water speed per loop
| …
|-
| Pressure drop per loop
| …
|-
| Power extraction per water loop
| 8x15W= 120W
|-
| Total cooling power
| 1KW
|-
| Inlet water temperature
| 16..20 degC (to be confirmed)
|-
| Outlet water temperature
| 20..24 degC
|-
| Total water flow
| 3..6 l/minute
|}

== Operation ==

''This description corresponds to how we think the system should work, but doesn't seem fully comatible with the behaviour of the system. E.g. the air pump only seems to turn on when PID1 falls below a certain value.''

The Pressure PID (proportional/integral/differential controller), PID2, monitors the air pressure in the vacuum vessel. When the pressure rises above a trigger value (perhaps 350mbar) it activates the aspirator vacuum pump to bring the pressure down to maintain a long time averaged pressure (say 300mbar).

This vacuum is then used to draw water through the detector into the vessel, from the Open Tank. The vacuum vessel's water level is monitored by PID1, using a differential pressure transducer measuring the weight of the water column by the difference between the air pressure in the top of the vessel and the water pressure at the bottom of the vessel. PID1 provides a variable signal to the Proportional Solenoid Valve, which varies the flow through the return loop around the (fixed flow) water pump, and thus changing the overall flow of water from the vacuum vessel to keep the water level in the vessel constant despite water being constantly drawn in by the vacuum. The water pulled from the vacuum vessel is pushed through the filter and into the Open Tank, completing the circulation.

Cooling is accomplished in the water pump leg of the system, as coolant pulled from the vacuum vessel passes through a heat exchanger coupled to CERN’s cooling water supply on its way to the pump. The rate that water flows through the system can be determined by many obstacles in the loop, but ideally it should be dominated by the position of the main Throttle Valve. Other valves should generally be set either fully open of fully closed, except for the Proportional Valve, whose position responds to the setting of the Throttle Valve.

Data outputs: there are five data outputs planned. The two process variables, the vacuum vessel absolute pressure, and differential water column pressure, will be sent to the Detector DAQ system. As well, the total flow rate is monitored by the flow meter, whose output is sent to the DAQ system. This data is not used by the circulation system itself. Likewise, the back pressure on the water filter is only sent to the DAQ system to indicate if the filter may need changing, and the temperature of the open tank is monitored and sent to the DAQ system, but currently there is no plan to use it in a temperature control feedback. The temperature of CERN's cooling water supply is not yet known, but we expect we can set the heat exchanger to run at a rate to hold the system temperature near 20°C without feedback control.

For the negative pressure to work effectively to prevent leaks, the water pressure in the detector ought to be down at least to half an atmosphere. With the detector cooling tubes 8x 4.6m of parallel 4.7mm ID copper tubes (2.3m down then 2.3 back-ups), and the supply using (perhaps 6m, 3 to the detector then 3 back) 9.7mm ID (=1/2" OD) poly tubing, the flow will need to be somewhat greater than the required 3 l/min in order to maintain that degree of vacuum in the detector, unless the Throttle Valve is placed upstream of the detector near the Open Tank, or an additional constriction is added in that region.

The water pump leg of the circuit residing in the water cooling/gas rack is placed far enough from the fringe field of the superconducting magnet to prevent disruptive effects on the solenoid valves and electric motors. Therefore the main 1/2" poly line to and from the water pump and its flow loop will add a pressure drop which is estimated at 0.4atm for a 10m line at 6 l/min (9.7mmID), which is within our requirements. As mentioned earlier, the splitting (and reassembly) of the line to the 8 cooling lines close to detector will be accomplished with the use of 2 separate manifolds equipped with temperature sensors for constant in/out global water temperature monitoring.

[[File:imageservice7.png|286x206px]][[File:imageservice8.jpg|286x206px]]

Figure 6 - Model and 3D-printed realization of the dry-air manifold, very similar to the manifolds for the water cooling system.

Currently, PVC valves with "true union" couplings are planned to be provided around the water pump to permit it to be swapped out from the system for servicing and a valved bypass to the water filter is provided for the same purpose to allow easily changing filter elements with the system running. True union valves are generally only available to 1/2NPT and larger, so an adapter fitting is required. The vacuum pump for the system is a small (2 x 5 x 7.5cm) cascaded dual Venturi aspirator, which is operated with site air, switched with two simultaneously operated normally closed solenoid valves, one for the site air, and one for the vessel vacuum input. If the system is leak free, the vacuum in the vessel should only be degraded by the dissolved air absorbed in the Open Tank and extracted in the vacuum vessel, which from past experience might mean running for under a minute every six hours or so.

The wetted surfaces of this system will be mainly PVC and Polyethylene for the valves, tubing and vessels, plus brass fittings, and the copper cooling tubes, and the surfaces of the heat exchanger, which will most likely be stainless steel. It may be possible to devise an equivalent of a bellows communicating with the Open Tank, so that it is only virtually open, thus sealing the cooling water from contaminants, which may allow us to run with just distilled water plus a strip of silver as a biocide (although I'm not sure if the mix of metals, copper plus stainless, may not interfere with the function of the silver, which is mostly used in computer cooling systems which are copper and plastic only). Otherwise, we will need a biocide and corrosion inhibitor. We have the latter, tolyltriazole, but the biocide is problematic. They tend to be short-lived; for systems which are open to atmosphere, typical industrial practice is chlorination, regularly refreshed, which builds up NaCl as it degrades, making the coolant corrosive. For T2K in Japan, we used the (human-nontoxic) "Germall", which has a two-year lifetime, after which when its effectiveness decays it turns into food, which is the case for any solute that is not actively biocidal. Eventually, any such additives build up and require the coolant to be flushed and replaced. Silver is a lovely method, but only works for extended periods with a sealed system which remains deoxygenated. My research so far has not turned up a good candidate which does not entail recurring changes of the coolant.

Power: the system needs to be able to run on European 220VAC power, but also operate here for testing purposes. The PIDs (Omega CN76160s) will accept both; for the rest of the system, I will have an a universal 90-240VAC input to 24VDC (5A) supply which will drive all the other components, including the pump, which will be a brushless 24VDC centrifugal pump, 1 - 1.5A range. The whole system will run about 125W.

Information on the temperature and flow rate of the CERN cooling water system will be provided later but we don’t anticipate any major issues about the sizing of the heat exchanger.

[[File:imageservice9.png|603x344px]]

Figure 7 - Overall water cooling system



== Temperature Measurement ==

[[File:imageservice10.jpeg|225x356px]] [[File:imageservice11.png|194x358px]]

Figure 8 Temperature Monitor Board

The water temperature is read by the Temperature Monitor Board (TEMPB). Thermistors located inside the manifold connect to a 16 bit ADC on the TEMPB which is read out by a Raspberry Pi mounted on board. In a similar manner the air system temperature is read with sensors in the air manifold. The TEMPB can provide temperature readout of 32 thermistors which connect to terminal blocks. The channels can also be readout differentially providing up to 16 channels for thermocouples. An on board thermistor allows for cold junction compensation. The board is powered via USB on the Raspberry Pi.

== Quick Start ==
This assumes all water lines and compressed air are connected, and both the blue water tank and the chiller are filled.

=== Begin running ===
# Close vent valve and main valve (3-way valve in T position)
# Switch on control box with power switch in the back
# confirm water pump is off, i.e. blue button is not illuminated
# Vacuum pump should now be running
# Open main valve, i.e. vertical connection open, connection to vent valve closed [[File:PWBmainOpen.jpg|thumb|Cooling system main valve open]]
# water should start flowing through the PWB cooling system and coming back through the clear pipe connected to the vacuum vessel
# Default settings are PID1 = 200, PID2 = 300, but may have been adjusted
# once the water level in the vacuum vessel has risen to ~20cm, turn on the water pump

=== Drain system ===
# from normal operation, i.e. vacuum pump and water pump running
# switch main valve to upside down T position
[[File:PWBmainVent.jpg|thumb|Cooling system main valve to vent valve]]
# open vent valve
# air should now enter the system, draining all water from the TPC pipes into the vacuum vessel
# water pump will drain the vacuum vessel into supply tank, turn it off before the vessel is completely empty to avoid air in the water pump
# switch off main power

== Troubleshooting ==
The water system is temperamental, and not intuitive at all. Common problems and solution attempts:

; water pump very loud, doesn't appear to pump water
: most likely air in the water pump. Open the vent of the vacuum vessel to allow water to enter pump by gravity alone, pump should stop screaming and start pumping. Close vent.
; vacuum pump comes on often (more then every 10-20 min)
: probably an air leak in the system. Check for bubbles and try to find leak.
; water level/flow/vacuum oscillate between extreme values
: unclear, try playing with PID settings, throttle valve, possibly orange ball valves by the water pump.

= Power distribution =

This section covers the different voltages and power requirements for the operation of the rTPC. This includes:

# High Voltage for the rTPC electron drift and amplification operation
# Low Voltage to the Anode wire frontend electronics
# Low Voltage to the Barrel Scintillator frontend electronics
# Low Voltage to the Cathode Pads electronics
# Grounding scheme of the overall rTPC

The provision of power to the different components of the detector is distributed from the top end plate of the rTPC.

[[File:imageservice12.png|507x354px]]

Figure 9 - End plate view of the layout of the different electronics boards & HV connections

== High Voltage for the rTPC ==

Three main components of the rTPC require High Voltage

* Main cathode wall (Negative ~5KV)
* Field wires (Negative ~ few hundred volts)
* Anode wires (Positive ~3.2KV )

These voltages are provided by a Quad High Voltage unit from CAEN. Three independent cables will be fed to the top of the rTPC.

High Voltage provision to the bottom endplate HV ring is taken from the central cathode near the bottom endplate.

The Field wire voltage is connected to all the field wire crimp-pins through a dedicated socket chain inserted on each of the crimp-pin on the top and bottom endplates. The field wires themselves interconnect the high voltage between the top and bottom pins.

The High Voltage for the anode wires is delivered to the Anode Wire Card (AWC) which will distribute it to its 16 wires, Figure 11. There are 16 AWC cards covering each end of the anode wires. All the top and bottom 32 AWC are connected serially. The anode wires themselves interconnect the high voltage between the top and bottom pins.

The anode wires pre-amplifiers boards (AWB) are sitting on the top of the Anode Wire Card (AWC) Figure 10. As the wire is powered by the HV, each anode wire has a high Voltage decoupling capacitor to isolate the pre-amp stage from the HV. This circuit resides on the AWC. The AWB will see the AC coupled anode signal only. With a new design of the AWC Figure 12, the decoupling capacitors are potted in epoxy to ensure proper isolation of the HV toward the surrounding circuit as well to prevent corona discharge due to relative humidity of the surrounding.

[[File:imageservice13.jpg|402x309px]]

Figure 10 – Unpotted Anode Wire Card and Board with HV distribution

[[File:imageservice14.jpg|402x309px]]

Figure 11 - Anode wire pins and Field wire pins under yellow cover

[[File:imageservice15.jpg|402x309px]]

Figure 12 – New Anode Wire card with 16 potted HV capacitors, gold crimp pin Mill-max socket to the anode wire

== Power distribution to the Anode wire frontend electronics ==

The AWB pre-amp board requires 2 voltages (+5V, -5V) which are fed through dedicated cable connections to each of the AWB. There are 16 AWB per end of the rTPC, therefore, 16 individual cable bundle will be reaching the top and another 16 the bottom of the rTPC. The cable bundle is to also provide the power return lines to the Power Supply.

The PL508 Low Voltage power Supply from Wiener will provide the main power source for all the 32 pre-amplifiers. A dedicated connection breakout board with current monitoring capability splits the source power to the 2 x 16 necessary connections to the Anode wire boards. Based on a RaspeberryPi card, a 16 power distribution board is already implemented for the prototype. This board has extra 4 channels of temperature monitoring which can be used later on. In the same process we produce a similar board for 16 channel of NTC, RTC temperature monitoring. They will be used for overall detector and cooling system monitoring.

{|
! '''''Component'''''
! '''''+5V Supply'''''
! '''''-5V Supply'''''
|-
| Anode Pre-amp
| 1.5W (0.3A)
| 1.5W (0.3A)
|}

[[File:imageservice16.jpg|239x363px]] [[File:imageservice17.png|239x363px]]

Figure 13 - Anode wire pre-amp power distribution & 16 channel temperature boards

== Power distribution to the Barrel Scintillator frontend electronics ==

Similar to the Anode wire pre-amplifier voltage distribution scheme, the Barrel Scintillators will require several low voltages for power and control. The Barrel Scintillator Boards (BSB) are mounted on both ends of the Scintillator bar with the same segmentation as the Anode wires i.e.: 16 BSB per end. The design of the electronic chain for the barrel scintillator is not fully designed yet.

The SiPM sensors require a bias voltage of the order of 28V. This voltage will be provided through the same mean as the low voltage power. Expected threshold voltage for the local signal level discriminator is also foreseen.

== Low Voltage to the Cathode Pads electronics ==

The cathode pads electronics mounted on the external cylinder surface is composed of the analog frontend and the digital conversion electronics. Two voltages are required (+5.3V, +2.3V). The delivery is done through dedicated “power bar” running the full length of the rTPC. A set of 3 bars covers the 8 axial cathode pad electronics boards. 8 of them cover the circumference of the rTPC.

{|
! '''''Component'''''
! '''''+5.3V Supply'''''
! '''''+2.3V Supply'''''
|-
| Padwing board
| 10W (1.8A)
| 2W (0.8A)
|}

Overall the low power supplies are provided by a PL508-3U with 8 modules of either single or dual channels from Wiener.

[[File:imageservice18.png|469x360px]]

Figure 14 - Overall Low voltage distribution for anode wires and cathode pads



== Grounding scheme of the overall rTPC ==

The grounding scheme challenge in this setup is the far distance between the 2 end of the rTPC (2.3m) and the lack of accessibility to the bottom of the detector. The proposal is to use a Copper sheet mounted on the inner surface of the inner cylinder. This ground path will connect the 2 endplates to the High Voltage return. All the pre-amps will merge their ground to this central ground.

Additional ground lugs have been included in the Anode wire pre-amplifier boards to strengthen the ground as each anode wire is readout out from both end. This may be used once the full detector is assembled to address possible ground loop issues.

[[File:imageservice19.png|439x409px]]

= Safety and Hazard considerations =

== Gas ==

The Ar and CO2 do not require special health safety system. The detector will be operated in the ventilated hall of the AD building at CERN.

Gas flow, detector gas pressure, and gas inlet temperature are constantly monitored and can trigger adequate response for shutdown or closure of the gas system for safety purpose.

== Cooling water ==

The water cooling system operates in negative pressure mode preventing any water leak to flood the detector outer surface. In addition water flow and inlet/outlet temperature are constantly monitored to allow fast response to off range parameter in order to shut down the electronics. The cooling water system requires a primary cooling tap water loop from the AD building. Water quality, water flow and water temperature are not under our responsibility. This aspect will be dealt during the initial phase of the rTPC commissioning at CERN.

== Power distribution ==

All electronics voltages are below 48V. No special safety circuits are envisaged.

The high-voltage circuit is fully protected outside the detector. The access to the high-voltage on the detector is restricted to the two end-plates. Interlock scheme will be put in place to ensure the shutdown of the HV in case of manual access to the detector itself.

The LV (Low Voltage) power units are also interlocked to prevent false powering sequence which can damage the downstream electronics modules.

[[File:imageservice20.png|571x411px]]

Figure 16- Alpha-g rTPC rack organization

File:20190109 165410.jpg

2019-01-09T17:29:09Z

Acapra:

Gas rack outside zone

Detector Services

2019-01-09T17:21:53Z

Acapra: /* Quick Start */

Back to [[Main Page]]

= Table of Figures =

[[#_Toc493684884|Figure 1 - Gas system schematic 6]]

[[#_Toc493684885|Figure 2 - A half-rack space is used for the gas control implementation. 7]]

[[#_Toc493684886|Figure 3 - Gas system half rack ready for installation 8]]

[[#_Toc493684887|Figure 4 - Cathode pad half-cylinder. Pairs of cooling pipes are visible 9]]

[[#_Toc493684888|Figure 5 - Water cooling system during test. 9]]

[[#_Toc493684889|Figure 6 - Model and 3D-printed realization of the dry-air manifold, very similar to the manifolds for the water cooling system. 12]]

[[#_Toc493684890|Figure 7 - Overall water cooling system 13]]

[[#_Toc493684891|Figure 8 - Temperature Monitor Board 14]]

[[#_Toc493684892|Figure 9 - End plate view of the layout of the different electronics boards & HV connections 15]]

[[#_Toc493684893|Figure 10 – Unpotted Anode Wire Card and Board with HV distribution 17]]

[[#_Toc493684894|Figure 11 - Anode wire pins and Field wire pins under yellow cover 17]]

[[#_Toc493684895|Figure 12 – New Anode Wire card with 16 potted HV capacitors, gold crimp pin Mill-max socket to the anode wire 18]]

[[#_Toc493684896|Figure 13 - Anode wire pre-amp power distribution & 16 channel temperature boards 19]]

[[#_Toc493684897|Figure 14 - Overall Low voltage distribution for anode wires and cathode pads 21]]

[[#_Toc493684898|Figure 15 - High Voltage & Grounding scheme for the rTPC 22]]

[[#_Toc493684899|Figure 16- Alpha-g rTPC rack organization 23]]

= Introduction =



This document is to describe the different services for the rTPC detector. 3 main sections cover the ''Gas system'', the ''Water cooling system'' and the ''Power distribution''.

= References and related Documents =

A dedicated TRIUMF Wiki site gathers all documentations for the rTPC detector system and equipment. Gas system, cooling water system and power distribution can be found there.

= Gas system =

The gas system as described below includes the entire gas infrastructure required to deliver gas to the detector, monitor and control the gas flow into the detector. It also includes the necessary equipment to notify any change from the nominal gas flow or gas mixture ratio delivered to the detector.

This type of mixed gas system is well understood by the Triumf detector group staff and no issue in its realization is expected.

This system is a stand-alone infrastructure assembled and delivered to CERN. It will be placed in the Alpha-g experimental area floor. The delivery of the different gases to the experimental area is under the responsibility of the Gas handling group from CERN.

The gas delivery system for the radial TPC (rTPC) is to provide Ar/CO2 gas mixture between 90/10 to 50/50 depending on the user requirements. This system does not recycle the gas as the environmental impact of either gas is minimal and the operation cost versus a recycling system would not be economic.

In the event of an additional quenching gas requirement, the overall gas system will have to be reconsidered and possible recycling system envisaged.

A simple gas mixer using digital flow meters will provide the precise mixture dialed by hand in the control system.

During the prototype phase, an ad-hoc gas system will be put in place. A portable gas system is available for such a test.

== Specifications ==

{|
! '''''Parameter'''''
! '''''Value'''''
|-
| Gas mixture
| Ar/CO2 (90/10..50/50)
|-
| Gas flow requirements
| 0.3l/min (50%), 0.3l/min (50%)
|-
| Detector gas Volume
| 0.18m3
|-
| Flushing time
| 5h @ 600cc/min
|-
| Flow rate
| 100 to 300 cc/minute
|}

== Operation ==

This gas system is quite simple as it takes 2 or 3 (spare) gas supplies and mixes them in a manifold before delivery to the rTPC detector. Necessary valves for purging lines and by-passing the detector are foreseen. The individual in-flows are controlled by gas dedicated mass flow controllers (MFC-x). The return flow is measured for leak assessment. An additional manometer monitors the internal detector pressure for evaluation of the electron drift characteristics.

[[File:imageservice2.png|575x315px|PA_doc-1.PNG]]

Figure 1 - Gas system schematic

The main flow controller and monitoring is performed by industrial devices from MKS. We added analog flowmeters on each gas line to provide a quick visual inspection of the gas mixing system.

This system will be hooked to the CERN AD-Gas facility. The CERN gas delivers gas through 2 switchable packs of 12 bottles for each gas.

The gas exhaust is evacuated to the exterior of the experimental area with an elevation of about 8m.

[[File:imageservice3.png|366x263px|PA_doc-2.PNG]]

Figure 2 - A half-rack space is used for the gas control implementation.



[[File:imageservice4.png|324x364px]]

Figure 3 - Gas system half rack ready for installation

== Quick Start ==
[[File:20181205 103217.jpg|thumb|Normal configuration of the manual valves of the GHS.]]
From-scratch (Neither gas supply lines to the GHS nor connections to and from the TPC connected) connection of the GHS should follow the following procedure:
# '''Flush supply lines:''' Gas lines from outside bottles going to the GHS should be briefly (5 min or so) flushed with a high gas flow ''before'' connecting to the GHS to remove possible dust, and to minimize the amount of air in the lines.
# '''Flush GHS:'''
## Close BV1 and BV3, open BV2 and BV4 to flush input lines, only briefly.
## Open BV1 and BV3, close BV2 and BV4 (normal operation configuration)
## With Midas frontend running, use the "Gas" custom page to set a desired flow, for flushing probably between 1l/min and 1.5l/min (Without Midas, use the telnet interface), then press the "Bypass Flow" button, and let flow for a while (5-15min?)
# '''Flush TPC input line:'''
## Connect the line that will go from the GHS to the TPC input-tee on the GHS side, but leave disconnected from the tee
## do the same for the pressure gauge line
## On the "Gas" custom page, press the "Flow TPC" button and let flow for 5 min (Without Midas, use the telnet interface).
# Make all remaining connections

== Telnet Interface ==
If the Midas frontend is not available the GHS can be controlled via telnet. Connect with
telnet algas
then use the following command to control solenoid valves:
cowr do i 1
to energize SVi, i.e. to switch it from its normal closed or open state to the other state,
cowr do i 0
to de-energize, i.e. switch to default state.

Gas flow is set with:
mfcwr ao i <desired flow>
where i is 1 for Ar and 2 for CO2, and the desired flow is in the internal units of the system.

= Water cooling system =

The outer surface of the detector has an arrangement of 8x8 electronics boards handling the readout of the cathode pads. These boards require operating at a controlled temperature to prevent heat build-up at the surface of the detector and improving the reliability of the system. In order to do so, 8 independent water pipe loops running from the bottom-up and down in the longitudinal axis of the detector are in a thermal contact to 8 successive readout boards. From the bottom, a pair of water hoses connect to 2 custom 3D-printed manifolds supplying the water feed and return to the 8 water loops.

These 8 loops are run in parallel to a negative pressure water cooling system. This negative pressure system is to ensure that no water leak will damage the electronics even during possible pipe or delivery hose breakage.

[[File:imageservice5.png|516x273px]]

Figure 4 - Cathode pad half-cylinder. Pairs of cooling pipes are visible

[[File:imageservice6.jpg|249x429px]]

Figure 5 - Water cooling system during test.

== Specifications ==

{|
! '''''Parameters'''''
! '''''Values'''''
|-
| Number of parallel water loops
| 8
|-
| Copper pipe OD/ID
| ¼” / 4.7mm
|-
| Length of one loop (on the rTPC)
| 4.6m
|-
| Water speed per loop
| …
|-
| Pressure drop per loop
| …
|-
| Power extraction per water loop
| 8x15W= 120W
|-
| Total cooling power
| 1KW
|-
| Inlet water temperature
| 16..20 degC (to be confirmed)
|-
| Outlet water temperature
| 20..24 degC
|-
| Total water flow
| 3..6 l/minute
|}

== Operation ==

''This description corresponds to how we think the system should work, but doesn't seem fully comatible with the behaviour of the system. E.g. the air pump only seems to turn on when PID1 falls below a certain value.''

The Pressure PID (proportional/integral/differential controller), PID2, monitors the air pressure in the vacuum vessel. When the pressure rises above a trigger value (perhaps 350mbar) it activates the aspirator vacuum pump to bring the pressure down to maintain a long time averaged pressure (say 300mbar).

This vacuum is then used to draw water through the detector into the vessel, from the Open Tank. The vacuum vessel's water level is monitored by PID1, using a differential pressure transducer measuring the weight of the water column by the difference between the air pressure in the top of the vessel and the water pressure at the bottom of the vessel. PID1 provides a variable signal to the Proportional Solenoid Valve, which varies the flow through the return loop around the (fixed flow) water pump, and thus changing the overall flow of water from the vacuum vessel to keep the water level in the vessel constant despite water being constantly drawn in by the vacuum. The water pulled from the vacuum vessel is pushed through the filter and into the Open Tank, completing the circulation.

Cooling is accomplished in the water pump leg of the system, as coolant pulled from the vacuum vessel passes through a heat exchanger coupled to CERN’s cooling water supply on its way to the pump. The rate that water flows through the system can be determined by many obstacles in the loop, but ideally it should be dominated by the position of the main Throttle Valve. Other valves should generally be set either fully open of fully closed, except for the Proportional Valve, whose position responds to the setting of the Throttle Valve.

Data outputs: there are five data outputs planned. The two process variables, the vacuum vessel absolute pressure, and differential water column pressure, will be sent to the Detector DAQ system. As well, the total flow rate is monitored by the flow meter, whose output is sent to the DAQ system. This data is not used by the circulation system itself. Likewise, the back pressure on the water filter is only sent to the DAQ system to indicate if the filter may need changing, and the temperature of the open tank is monitored and sent to the DAQ system, but currently there is no plan to use it in a temperature control feedback. The temperature of CERN's cooling water supply is not yet known, but we expect we can set the heat exchanger to run at a rate to hold the system temperature near 20°C without feedback control.

For the negative pressure to work effectively to prevent leaks, the water pressure in the detector ought to be down at least to half an atmosphere. With the detector cooling tubes 8x 4.6m of parallel 4.7mm ID copper tubes (2.3m down then 2.3 back-ups), and the supply using (perhaps 6m, 3 to the detector then 3 back) 9.7mm ID (=1/2" OD) poly tubing, the flow will need to be somewhat greater than the required 3 l/min in order to maintain that degree of vacuum in the detector, unless the Throttle Valve is placed upstream of the detector near the Open Tank, or an additional constriction is added in that region.

The water pump leg of the circuit residing in the water cooling/gas rack is placed far enough from the fringe field of the superconducting magnet to prevent disruptive effects on the solenoid valves and electric motors. Therefore the main 1/2" poly line to and from the water pump and its flow loop will add a pressure drop which is estimated at 0.4atm for a 10m line at 6 l/min (9.7mmID), which is within our requirements. As mentioned earlier, the splitting (and reassembly) of the line to the 8 cooling lines close to detector will be accomplished with the use of 2 separate manifolds equipped with temperature sensors for constant in/out global water temperature monitoring.

[[File:imageservice7.png|286x206px]][[File:imageservice8.jpg|286x206px]]

Figure 6 - Model and 3D-printed realization of the dry-air manifold, very similar to the manifolds for the water cooling system.

Currently, PVC valves with "true union" couplings are planned to be provided around the water pump to permit it to be swapped out from the system for servicing and a valved bypass to the water filter is provided for the same purpose to allow easily changing filter elements with the system running. True union valves are generally only available to 1/2NPT and larger, so an adapter fitting is required. The vacuum pump for the system is a small (2 x 5 x 7.5cm) cascaded dual Venturi aspirator, which is operated with site air, switched with two simultaneously operated normally closed solenoid valves, one for the site air, and one for the vessel vacuum input. If the system is leak free, the vacuum in the vessel should only be degraded by the dissolved air absorbed in the Open Tank and extracted in the vacuum vessel, which from past experience might mean running for under a minute every six hours or so.

The wetted surfaces of this system will be mainly PVC and Polyethylene for the valves, tubing and vessels, plus brass fittings, and the copper cooling tubes, and the surfaces of the heat exchanger, which will most likely be stainless steel. It may be possible to devise an equivalent of a bellows communicating with the Open Tank, so that it is only virtually open, thus sealing the cooling water from contaminants, which may allow us to run with just distilled water plus a strip of silver as a biocide (although I'm not sure if the mix of metals, copper plus stainless, may not interfere with the function of the silver, which is mostly used in computer cooling systems which are copper and plastic only). Otherwise, we will need a biocide and corrosion inhibitor. We have the latter, tolyltriazole, but the biocide is problematic. They tend to be short-lived; for systems which are open to atmosphere, typical industrial practice is chlorination, regularly refreshed, which builds up NaCl as it degrades, making the coolant corrosive. For T2K in Japan, we used the (human-nontoxic) "Germall", which has a two-year lifetime, after which when its effectiveness decays it turns into food, which is the case for any solute that is not actively biocidal. Eventually, any such additives build up and require the coolant to be flushed and replaced. Silver is a lovely method, but only works for extended periods with a sealed system which remains deoxygenated. My research so far has not turned up a good candidate which does not entail recurring changes of the coolant.

Power: the system needs to be able to run on European 220VAC power, but also operate here for testing purposes. The PIDs (Omega CN76160s) will accept both; for the rest of the system, I will have an a universal 90-240VAC input to 24VDC (5A) supply which will drive all the other components, including the pump, which will be a brushless 24VDC centrifugal pump, 1 - 1.5A range. The whole system will run about 125W.

Information on the temperature and flow rate of the CERN cooling water system will be provided later but we don’t anticipate any major issues about the sizing of the heat exchanger.

[[File:imageservice9.png|603x344px]]

Figure 7 - Overall water cooling system



== Temperature Measurement ==

[[File:imageservice10.jpeg|225x356px]] [[File:imageservice11.png|194x358px]]

Figure 8 Temperature Monitor Board

The water temperature is read by the Temperature Monitor Board (TEMPB). Thermistors located inside the manifold connect to a 16 bit ADC on the TEMPB which is read out by a Raspberry Pi mounted on board. In a similar manner the air system temperature is read with sensors in the air manifold. The TEMPB can provide temperature readout of 32 thermistors which connect to terminal blocks. The channels can also be readout differentially providing up to 16 channels for thermocouples. An on board thermistor allows for cold junction compensation. The board is powered via USB on the Raspberry Pi.

== Quick Start ==
This assumes all water lines and compressed air are connected, and both the blue water tank and the chiller are filled.

=== Begin running ===
# Close vent valve and main valve (3-way valve in T position)
# Switch on control box with power switch in the back
# confirm water pump is off, i.e. blue button is not illuminated
# Vacuum pump should now be running
# Open main valve, i.e. vertical connection open, connection to vent valve closed [[File:PWBmainOpen.jpg|thumb|Cooling system main valve open]]
# water should start flowing through the PWB cooling system and coming back through the clear pipe connected to the vacuum vessel
# Default settings are PID1 = 200, PID2 = 300, but may have been adjusted
# once the water level in the vacuum vessel has risen to ~20cm, turn on the water pump

=== Drain system ===
# from normal operation, i.e. vacuum pump and water pump running
# switch main valve to upside down T position
[[File:PWBmainVent.jpg|thumb|Cooling system main valve to vent valve]]
# open vent valve
# air should now enter the system, draining all water from the TPC pipes into the vacuum vessel
# water pump will drain the vacuum vessel into supply tank, turn it off before the vessel is completely empty to avoid air in the water pump
# switch off main power

== Troubleshooting ==
The water system is temperamental, and not intuitive at all. Common problems and solution attempts:

; water pump very loud, doesn't appear to pump water
: most likely air in the water pump. Open the vent of the vacuum vessel to allow water to enter pump by gravity alone, pump should stop screaming and start pumping. Close vent.
; vacuum pump comes on often (more then every 10-20 min)
: probably an air leak in the system. Check for bubbles and try to find leak.
; water level/flow/vacuum oscillate between extreme values
: unclear, try playing with PID settings, throttle valve, possibly orange ball valves by the water pump.

= Power distribution =

This section covers the different voltages and power requirements for the operation of the rTPC. This includes:

# High Voltage for the rTPC electron drift and amplification operation
# Low Voltage to the Anode wire frontend electronics
# Low Voltage to the Barrel Scintillator frontend electronics
# Low Voltage to the Cathode Pads electronics
# Grounding scheme of the overall rTPC

The provision of power to the different components of the detector is distributed from the top end plate of the rTPC.

[[File:imageservice12.png|507x354px]]

Figure 9 - End plate view of the layout of the different electronics boards & HV connections

== High Voltage for the rTPC ==

Three main components of the rTPC require High Voltage

* Main cathode wall (Negative ~5KV)
* Field wires (Negative ~ few hundred volts)
* Anode wires (Positive ~3.2KV )

These voltages are provided by a Quad High Voltage unit from CAEN. Three independent cables will be fed to the top of the rTPC.

High Voltage provision to the bottom endplate HV ring is taken from the central cathode near the bottom endplate.

The Field wire voltage is connected to all the field wire crimp-pins through a dedicated socket chain inserted on each of the crimp-pin on the top and bottom endplates. The field wires themselves interconnect the high voltage between the top and bottom pins.

The High Voltage for the anode wires is delivered to the Anode Wire Card (AWC) which will distribute it to its 16 wires, Figure 11. There are 16 AWC cards covering each end of the anode wires. All the top and bottom 32 AWC are connected serially. The anode wires themselves interconnect the high voltage between the top and bottom pins.

The anode wires pre-amplifiers boards (AWB) are sitting on the top of the Anode Wire Card (AWC) Figure 10. As the wire is powered by the HV, each anode wire has a high Voltage decoupling capacitor to isolate the pre-amp stage from the HV. This circuit resides on the AWC. The AWB will see the AC coupled anode signal only. With a new design of the AWC Figure 12, the decoupling capacitors are potted in epoxy to ensure proper isolation of the HV toward the surrounding circuit as well to prevent corona discharge due to relative humidity of the surrounding.

[[File:imageservice13.jpg|402x309px]]

Figure 10 – Unpotted Anode Wire Card and Board with HV distribution

[[File:imageservice14.jpg|402x309px]]

Figure 11 - Anode wire pins and Field wire pins under yellow cover

[[File:imageservice15.jpg|402x309px]]

Figure 12 – New Anode Wire card with 16 potted HV capacitors, gold crimp pin Mill-max socket to the anode wire

== Power distribution to the Anode wire frontend electronics ==

The AWB pre-amp board requires 2 voltages (+5V, -5V) which are fed through dedicated cable connections to each of the AWB. There are 16 AWB per end of the rTPC, therefore, 16 individual cable bundle will be reaching the top and another 16 the bottom of the rTPC. The cable bundle is to also provide the power return lines to the Power Supply.

The PL508 Low Voltage power Supply from Wiener will provide the main power source for all the 32 pre-amplifiers. A dedicated connection breakout board with current monitoring capability splits the source power to the 2 x 16 necessary connections to the Anode wire boards. Based on a RaspeberryPi card, a 16 power distribution board is already implemented for the prototype. This board has extra 4 channels of temperature monitoring which can be used later on. In the same process we produce a similar board for 16 channel of NTC, RTC temperature monitoring. They will be used for overall detector and cooling system monitoring.

{|
! '''''Component'''''
! '''''+5V Supply'''''
! '''''-5V Supply'''''
|-
| Anode Pre-amp
| 1.5W (0.3A)
| 1.5W (0.3A)
|}

[[File:imageservice16.jpg|239x363px]] [[File:imageservice17.png|239x363px]]

Figure 13 - Anode wire pre-amp power distribution & 16 channel temperature boards

== Power distribution to the Barrel Scintillator frontend electronics ==

Similar to the Anode wire pre-amplifier voltage distribution scheme, the Barrel Scintillators will require several low voltages for power and control. The Barrel Scintillator Boards (BSB) are mounted on both ends of the Scintillator bar with the same segmentation as the Anode wires i.e.: 16 BSB per end. The design of the electronic chain for the barrel scintillator is not fully designed yet.

The SiPM sensors require a bias voltage of the order of 28V. This voltage will be provided through the same mean as the low voltage power. Expected threshold voltage for the local signal level discriminator is also foreseen.

== Low Voltage to the Cathode Pads electronics ==

The cathode pads electronics mounted on the external cylinder surface is composed of the analog frontend and the digital conversion electronics. Two voltages are required (+5.3V, +2.3V). The delivery is done through dedicated “power bar” running the full length of the rTPC. A set of 3 bars covers the 8 axial cathode pad electronics boards. 8 of them cover the circumference of the rTPC.

{|
! '''''Component'''''
! '''''+5.3V Supply'''''
! '''''+2.3V Supply'''''
|-
| Padwing board
| 10W (1.8A)
| 2W (0.8A)
|}

Overall the low power supplies are provided by a PL508-3U with 8 modules of either single or dual channels from Wiener.

[[File:imageservice18.png|469x360px]]

Figure 14 - Overall Low voltage distribution for anode wires and cathode pads



== Grounding scheme of the overall rTPC ==

The grounding scheme challenge in this setup is the far distance between the 2 end of the rTPC (2.3m) and the lack of accessibility to the bottom of the detector. The proposal is to use a Copper sheet mounted on the inner surface of the inner cylinder. This ground path will connect the 2 endplates to the High Voltage return. All the pre-amps will merge their ground to this central ground.

Additional ground lugs have been included in the Anode wire pre-amplifier boards to strengthen the ground as each anode wire is readout out from both end. This may be used once the full detector is assembled to address possible ground loop issues.

[[File:imageservice19.png|439x409px]]

= Safety and Hazard considerations =

== Gas ==

The Ar and CO2 do not require special health safety system. The detector will be operated in the ventilated hall of the AD building at CERN.

Gas flow, detector gas pressure, and gas inlet temperature are constantly monitored and can trigger adequate response for shutdown or closure of the gas system for safety purpose.

== Cooling water ==

The water cooling system operates in negative pressure mode preventing any water leak to flood the detector outer surface. In addition water flow and inlet/outlet temperature are constantly monitored to allow fast response to off range parameter in order to shut down the electronics. The cooling water system requires a primary cooling tap water loop from the AD building. Water quality, water flow and water temperature are not under our responsibility. This aspect will be dealt during the initial phase of the rTPC commissioning at CERN.

== Power distribution ==

All electronics voltages are below 48V. No special safety circuits are envisaged.

The high-voltage circuit is fully protected outside the detector. The access to the high-voltage on the detector is restricted to the two end-plates. Interlock scheme will be put in place to ensure the shutdown of the HV in case of manual access to the detector itself.

The LV (Low Voltage) power units are also interlocked to prevent false powering sequence which can damage the downstream electronics modules.

[[File:imageservice20.png|571x411px]]

Figure 16- Alpha-g rTPC rack organization

Detector Services

2019-01-09T17:20:44Z

Acapra: /* Quick Start */

Back to [[Main Page]]

= Table of Figures =

[[#_Toc493684884|Figure 1 - Gas system schematic 6]]

[[#_Toc493684885|Figure 2 - A half-rack space is used for the gas control implementation. 7]]

[[#_Toc493684886|Figure 3 - Gas system half rack ready for installation 8]]

[[#_Toc493684887|Figure 4 - Cathode pad half-cylinder. Pairs of cooling pipes are visible 9]]

[[#_Toc493684888|Figure 5 - Water cooling system during test. 9]]

[[#_Toc493684889|Figure 6 - Model and 3D-printed realization of the dry-air manifold, very similar to the manifolds for the water cooling system. 12]]

[[#_Toc493684890|Figure 7 - Overall water cooling system 13]]

[[#_Toc493684891|Figure 8 - Temperature Monitor Board 14]]

[[#_Toc493684892|Figure 9 - End plate view of the layout of the different electronics boards & HV connections 15]]

[[#_Toc493684893|Figure 10 – Unpotted Anode Wire Card and Board with HV distribution 17]]

[[#_Toc493684894|Figure 11 - Anode wire pins and Field wire pins under yellow cover 17]]

[[#_Toc493684895|Figure 12 – New Anode Wire card with 16 potted HV capacitors, gold crimp pin Mill-max socket to the anode wire 18]]

[[#_Toc493684896|Figure 13 - Anode wire pre-amp power distribution & 16 channel temperature boards 19]]

[[#_Toc493684897|Figure 14 - Overall Low voltage distribution for anode wires and cathode pads 21]]

[[#_Toc493684898|Figure 15 - High Voltage & Grounding scheme for the rTPC 22]]

[[#_Toc493684899|Figure 16- Alpha-g rTPC rack organization 23]]

= Introduction =



This document is to describe the different services for the rTPC detector. 3 main sections cover the ''Gas system'', the ''Water cooling system'' and the ''Power distribution''.

= References and related Documents =

A dedicated TRIUMF Wiki site gathers all documentations for the rTPC detector system and equipment. Gas system, cooling water system and power distribution can be found there.

= Gas system =

The gas system as described below includes the entire gas infrastructure required to deliver gas to the detector, monitor and control the gas flow into the detector. It also includes the necessary equipment to notify any change from the nominal gas flow or gas mixture ratio delivered to the detector.

This type of mixed gas system is well understood by the Triumf detector group staff and no issue in its realization is expected.

This system is a stand-alone infrastructure assembled and delivered to CERN. It will be placed in the Alpha-g experimental area floor. The delivery of the different gases to the experimental area is under the responsibility of the Gas handling group from CERN.

The gas delivery system for the radial TPC (rTPC) is to provide Ar/CO2 gas mixture between 90/10 to 50/50 depending on the user requirements. This system does not recycle the gas as the environmental impact of either gas is minimal and the operation cost versus a recycling system would not be economic.

In the event of an additional quenching gas requirement, the overall gas system will have to be reconsidered and possible recycling system envisaged.

A simple gas mixer using digital flow meters will provide the precise mixture dialed by hand in the control system.

During the prototype phase, an ad-hoc gas system will be put in place. A portable gas system is available for such a test.

== Specifications ==

{|
! '''''Parameter'''''
! '''''Value'''''
|-
| Gas mixture
| Ar/CO2 (90/10..50/50)
|-
| Gas flow requirements
| 0.3l/min (50%), 0.3l/min (50%)
|-
| Detector gas Volume
| 0.18m3
|-
| Flushing time
| 5h @ 600cc/min
|-
| Flow rate
| 100 to 300 cc/minute
|}

== Operation ==

This gas system is quite simple as it takes 2 or 3 (spare) gas supplies and mixes them in a manifold before delivery to the rTPC detector. Necessary valves for purging lines and by-passing the detector are foreseen. The individual in-flows are controlled by gas dedicated mass flow controllers (MFC-x). The return flow is measured for leak assessment. An additional manometer monitors the internal detector pressure for evaluation of the electron drift characteristics.

[[File:imageservice2.png|575x315px|PA_doc-1.PNG]]

Figure 1 - Gas system schematic

The main flow controller and monitoring is performed by industrial devices from MKS. We added analog flowmeters on each gas line to provide a quick visual inspection of the gas mixing system.

This system will be hooked to the CERN AD-Gas facility. The CERN gas delivers gas through 2 switchable packs of 12 bottles for each gas.

The gas exhaust is evacuated to the exterior of the experimental area with an elevation of about 8m.

[[File:imageservice3.png|366x263px|PA_doc-2.PNG]]

Figure 2 - A half-rack space is used for the gas control implementation.



[[File:imageservice4.png|324x364px]]

Figure 3 - Gas system half rack ready for installation

== Quick Start ==
From-scratch (Neither gas supply lines to the GHS nor connections to and from the TPC connected) connection of the GHS should follow the following procedure:
# '''Flush supply lines:''' Gas lines from outside bottles going to the GHS should be briefly (5 min or so) flushed with a high gas flow ''before'' connecting to the GHS to remove possible dust, and to minimize the amount of air in the lines.
[[File:20181205 103217.jpg|thumb|Normal configuration of the manual valves of the GHS.]]
# '''Flush GHS:'''
## Close BV1 and BV3, open BV2 and BV4 to flush input lines, only briefly.
## Open BV1 and BV3, close BV2 and BV4 (normal operation configuration)
## With Midas frontend running, use the "Gas" custom page to set a desired flow, for flushing probably between 1l/min and 1.5l/min (Without Midas, use the telnet interface), then press the "Bypass Flow" button, and let flow for a while (5-15min?)
# '''Flush TPC input line:'''
## Connect the line that will go from the GHS to the TPC input-tee on the GHS side, but leave disconnected from the tee
## do the same for the pressure gauge line
## On the "Gas" custom page, press the "Flow TPC" button and let flow for 5 min (Without Midas, use the telnet interface).
# Make all remaining connections

== Telnet Interface ==
If the Midas frontend is not available the GHS can be controlled via telnet. Connect with
telnet algas
then use the following command to control solenoid valves:
cowr do i 1
to energize SVi, i.e. to switch it from its normal closed or open state to the other state,
cowr do i 0
to de-energize, i.e. switch to default state.

Gas flow is set with:
mfcwr ao i <desired flow>
where i is 1 for Ar and 2 for CO2, and the desired flow is in the internal units of the system.

= Water cooling system =

The outer surface of the detector has an arrangement of 8x8 electronics boards handling the readout of the cathode pads. These boards require operating at a controlled temperature to prevent heat build-up at the surface of the detector and improving the reliability of the system. In order to do so, 8 independent water pipe loops running from the bottom-up and down in the longitudinal axis of the detector are in a thermal contact to 8 successive readout boards. From the bottom, a pair of water hoses connect to 2 custom 3D-printed manifolds supplying the water feed and return to the 8 water loops.

These 8 loops are run in parallel to a negative pressure water cooling system. This negative pressure system is to ensure that no water leak will damage the electronics even during possible pipe or delivery hose breakage.

[[File:imageservice5.png|516x273px]]

Figure 4 - Cathode pad half-cylinder. Pairs of cooling pipes are visible

[[File:imageservice6.jpg|249x429px]]

Figure 5 - Water cooling system during test.

== Specifications ==

{|
! '''''Parameters'''''
! '''''Values'''''
|-
| Number of parallel water loops
| 8
|-
| Copper pipe OD/ID
| ¼” / 4.7mm
|-
| Length of one loop (on the rTPC)
| 4.6m
|-
| Water speed per loop
| …
|-
| Pressure drop per loop
| …
|-
| Power extraction per water loop
| 8x15W= 120W
|-
| Total cooling power
| 1KW
|-
| Inlet water temperature
| 16..20 degC (to be confirmed)
|-
| Outlet water temperature
| 20..24 degC
|-
| Total water flow
| 3..6 l/minute
|}

== Operation ==

''This description corresponds to how we think the system should work, but doesn't seem fully comatible with the behaviour of the system. E.g. the air pump only seems to turn on when PID1 falls below a certain value.''

The Pressure PID (proportional/integral/differential controller), PID2, monitors the air pressure in the vacuum vessel. When the pressure rises above a trigger value (perhaps 350mbar) it activates the aspirator vacuum pump to bring the pressure down to maintain a long time averaged pressure (say 300mbar).

This vacuum is then used to draw water through the detector into the vessel, from the Open Tank. The vacuum vessel's water level is monitored by PID1, using a differential pressure transducer measuring the weight of the water column by the difference between the air pressure in the top of the vessel and the water pressure at the bottom of the vessel. PID1 provides a variable signal to the Proportional Solenoid Valve, which varies the flow through the return loop around the (fixed flow) water pump, and thus changing the overall flow of water from the vacuum vessel to keep the water level in the vessel constant despite water being constantly drawn in by the vacuum. The water pulled from the vacuum vessel is pushed through the filter and into the Open Tank, completing the circulation.

Cooling is accomplished in the water pump leg of the system, as coolant pulled from the vacuum vessel passes through a heat exchanger coupled to CERN’s cooling water supply on its way to the pump. The rate that water flows through the system can be determined by many obstacles in the loop, but ideally it should be dominated by the position of the main Throttle Valve. Other valves should generally be set either fully open of fully closed, except for the Proportional Valve, whose position responds to the setting of the Throttle Valve.

Data outputs: there are five data outputs planned. The two process variables, the vacuum vessel absolute pressure, and differential water column pressure, will be sent to the Detector DAQ system. As well, the total flow rate is monitored by the flow meter, whose output is sent to the DAQ system. This data is not used by the circulation system itself. Likewise, the back pressure on the water filter is only sent to the DAQ system to indicate if the filter may need changing, and the temperature of the open tank is monitored and sent to the DAQ system, but currently there is no plan to use it in a temperature control feedback. The temperature of CERN's cooling water supply is not yet known, but we expect we can set the heat exchanger to run at a rate to hold the system temperature near 20°C without feedback control.

For the negative pressure to work effectively to prevent leaks, the water pressure in the detector ought to be down at least to half an atmosphere. With the detector cooling tubes 8x 4.6m of parallel 4.7mm ID copper tubes (2.3m down then 2.3 back-ups), and the supply using (perhaps 6m, 3 to the detector then 3 back) 9.7mm ID (=1/2" OD) poly tubing, the flow will need to be somewhat greater than the required 3 l/min in order to maintain that degree of vacuum in the detector, unless the Throttle Valve is placed upstream of the detector near the Open Tank, or an additional constriction is added in that region.

The water pump leg of the circuit residing in the water cooling/gas rack is placed far enough from the fringe field of the superconducting magnet to prevent disruptive effects on the solenoid valves and electric motors. Therefore the main 1/2" poly line to and from the water pump and its flow loop will add a pressure drop which is estimated at 0.4atm for a 10m line at 6 l/min (9.7mmID), which is within our requirements. As mentioned earlier, the splitting (and reassembly) of the line to the 8 cooling lines close to detector will be accomplished with the use of 2 separate manifolds equipped with temperature sensors for constant in/out global water temperature monitoring.

[[File:imageservice7.png|286x206px]][[File:imageservice8.jpg|286x206px]]

Figure 6 - Model and 3D-printed realization of the dry-air manifold, very similar to the manifolds for the water cooling system.

Currently, PVC valves with "true union" couplings are planned to be provided around the water pump to permit it to be swapped out from the system for servicing and a valved bypass to the water filter is provided for the same purpose to allow easily changing filter elements with the system running. True union valves are generally only available to 1/2NPT and larger, so an adapter fitting is required. The vacuum pump for the system is a small (2 x 5 x 7.5cm) cascaded dual Venturi aspirator, which is operated with site air, switched with two simultaneously operated normally closed solenoid valves, one for the site air, and one for the vessel vacuum input. If the system is leak free, the vacuum in the vessel should only be degraded by the dissolved air absorbed in the Open Tank and extracted in the vacuum vessel, which from past experience might mean running for under a minute every six hours or so.

The wetted surfaces of this system will be mainly PVC and Polyethylene for the valves, tubing and vessels, plus brass fittings, and the copper cooling tubes, and the surfaces of the heat exchanger, which will most likely be stainless steel. It may be possible to devise an equivalent of a bellows communicating with the Open Tank, so that it is only virtually open, thus sealing the cooling water from contaminants, which may allow us to run with just distilled water plus a strip of silver as a biocide (although I'm not sure if the mix of metals, copper plus stainless, may not interfere with the function of the silver, which is mostly used in computer cooling systems which are copper and plastic only). Otherwise, we will need a biocide and corrosion inhibitor. We have the latter, tolyltriazole, but the biocide is problematic. They tend to be short-lived; for systems which are open to atmosphere, typical industrial practice is chlorination, regularly refreshed, which builds up NaCl as it degrades, making the coolant corrosive. For T2K in Japan, we used the (human-nontoxic) "Germall", which has a two-year lifetime, after which when its effectiveness decays it turns into food, which is the case for any solute that is not actively biocidal. Eventually, any such additives build up and require the coolant to be flushed and replaced. Silver is a lovely method, but only works for extended periods with a sealed system which remains deoxygenated. My research so far has not turned up a good candidate which does not entail recurring changes of the coolant.

Power: the system needs to be able to run on European 220VAC power, but also operate here for testing purposes. The PIDs (Omega CN76160s) will accept both; for the rest of the system, I will have an a universal 90-240VAC input to 24VDC (5A) supply which will drive all the other components, including the pump, which will be a brushless 24VDC centrifugal pump, 1 - 1.5A range. The whole system will run about 125W.

Information on the temperature and flow rate of the CERN cooling water system will be provided later but we don’t anticipate any major issues about the sizing of the heat exchanger.

[[File:imageservice9.png|603x344px]]

Figure 7 - Overall water cooling system



== Temperature Measurement ==

[[File:imageservice10.jpeg|225x356px]] [[File:imageservice11.png|194x358px]]

Figure 8 Temperature Monitor Board

The water temperature is read by the Temperature Monitor Board (TEMPB). Thermistors located inside the manifold connect to a 16 bit ADC on the TEMPB which is read out by a Raspberry Pi mounted on board. In a similar manner the air system temperature is read with sensors in the air manifold. The TEMPB can provide temperature readout of 32 thermistors which connect to terminal blocks. The channels can also be readout differentially providing up to 16 channels for thermocouples. An on board thermistor allows for cold junction compensation. The board is powered via USB on the Raspberry Pi.

== Quick Start ==
This assumes all water lines and compressed air are connected, and both the blue water tank and the chiller are filled.

=== Begin running ===
# Close vent valve and main valve (3-way valve in T position)
# Switch on control box with power switch in the back
# confirm water pump is off, i.e. blue button is not illuminated
# Vacuum pump should now be running
# Open main valve, i.e. vertical connection open, connection to vent valve closed [[File:PWBmainOpen.jpg|thumb|Cooling system main valve open]]
# water should start flowing through the PWB cooling system and coming back through the clear pipe connected to the vacuum vessel
# Default settings are PID1 = 200, PID2 = 300, but may have been adjusted
# once the water level in the vacuum vessel has risen to ~20cm, turn on the water pump

=== Drain system ===
# from normal operation, i.e. vacuum pump and water pump running
# switch main valve to upside down T position
[[File:PWBmainVent.jpg|thumb|Cooling system main valve to vent valve]]
# open vent valve
# air should now enter the system, draining all water from the TPC pipes into the vacuum vessel
# water pump will drain the vacuum vessel into supply tank, turn it off before the vessel is completely empty to avoid air in the water pump
# switch off main power

== Troubleshooting ==
The water system is temperamental, and not intuitive at all. Common problems and solution attempts:

; water pump very loud, doesn't appear to pump water
: most likely air in the water pump. Open the vent of the vacuum vessel to allow water to enter pump by gravity alone, pump should stop screaming and start pumping. Close vent.
; vacuum pump comes on often (more then every 10-20 min)
: probably an air leak in the system. Check for bubbles and try to find leak.
; water level/flow/vacuum oscillate between extreme values
: unclear, try playing with PID settings, throttle valve, possibly orange ball valves by the water pump.

= Power distribution =

This section covers the different voltages and power requirements for the operation of the rTPC. This includes:

# High Voltage for the rTPC electron drift and amplification operation
# Low Voltage to the Anode wire frontend electronics
# Low Voltage to the Barrel Scintillator frontend electronics
# Low Voltage to the Cathode Pads electronics
# Grounding scheme of the overall rTPC

The provision of power to the different components of the detector is distributed from the top end plate of the rTPC.

[[File:imageservice12.png|507x354px]]

Figure 9 - End plate view of the layout of the different electronics boards & HV connections

== High Voltage for the rTPC ==

Three main components of the rTPC require High Voltage

* Main cathode wall (Negative ~5KV)
* Field wires (Negative ~ few hundred volts)
* Anode wires (Positive ~3.2KV )

These voltages are provided by a Quad High Voltage unit from CAEN. Three independent cables will be fed to the top of the rTPC.

High Voltage provision to the bottom endplate HV ring is taken from the central cathode near the bottom endplate.

The Field wire voltage is connected to all the field wire crimp-pins through a dedicated socket chain inserted on each of the crimp-pin on the top and bottom endplates. The field wires themselves interconnect the high voltage between the top and bottom pins.

The High Voltage for the anode wires is delivered to the Anode Wire Card (AWC) which will distribute it to its 16 wires, Figure 11. There are 16 AWC cards covering each end of the anode wires. All the top and bottom 32 AWC are connected serially. The anode wires themselves interconnect the high voltage between the top and bottom pins.

The anode wires pre-amplifiers boards (AWB) are sitting on the top of the Anode Wire Card (AWC) Figure 10. As the wire is powered by the HV, each anode wire has a high Voltage decoupling capacitor to isolate the pre-amp stage from the HV. This circuit resides on the AWC. The AWB will see the AC coupled anode signal only. With a new design of the AWC Figure 12, the decoupling capacitors are potted in epoxy to ensure proper isolation of the HV toward the surrounding circuit as well to prevent corona discharge due to relative humidity of the surrounding.

[[File:imageservice13.jpg|402x309px]]

Figure 10 – Unpotted Anode Wire Card and Board with HV distribution

[[File:imageservice14.jpg|402x309px]]

Figure 11 - Anode wire pins and Field wire pins under yellow cover

[[File:imageservice15.jpg|402x309px]]

Figure 12 – New Anode Wire card with 16 potted HV capacitors, gold crimp pin Mill-max socket to the anode wire

== Power distribution to the Anode wire frontend electronics ==

The AWB pre-amp board requires 2 voltages (+5V, -5V) which are fed through dedicated cable connections to each of the AWB. There are 16 AWB per end of the rTPC, therefore, 16 individual cable bundle will be reaching the top and another 16 the bottom of the rTPC. The cable bundle is to also provide the power return lines to the Power Supply.

The PL508 Low Voltage power Supply from Wiener will provide the main power source for all the 32 pre-amplifiers. A dedicated connection breakout board with current monitoring capability splits the source power to the 2 x 16 necessary connections to the Anode wire boards. Based on a RaspeberryPi card, a 16 power distribution board is already implemented for the prototype. This board has extra 4 channels of temperature monitoring which can be used later on. In the same process we produce a similar board for 16 channel of NTC, RTC temperature monitoring. They will be used for overall detector and cooling system monitoring.

{|
! '''''Component'''''
! '''''+5V Supply'''''
! '''''-5V Supply'''''
|-
| Anode Pre-amp
| 1.5W (0.3A)
| 1.5W (0.3A)
|}

[[File:imageservice16.jpg|239x363px]] [[File:imageservice17.png|239x363px]]

Figure 13 - Anode wire pre-amp power distribution & 16 channel temperature boards

== Power distribution to the Barrel Scintillator frontend electronics ==

Similar to the Anode wire pre-amplifier voltage distribution scheme, the Barrel Scintillators will require several low voltages for power and control. The Barrel Scintillator Boards (BSB) are mounted on both ends of the Scintillator bar with the same segmentation as the Anode wires i.e.: 16 BSB per end. The design of the electronic chain for the barrel scintillator is not fully designed yet.

The SiPM sensors require a bias voltage of the order of 28V. This voltage will be provided through the same mean as the low voltage power. Expected threshold voltage for the local signal level discriminator is also foreseen.

== Low Voltage to the Cathode Pads electronics ==

The cathode pads electronics mounted on the external cylinder surface is composed of the analog frontend and the digital conversion electronics. Two voltages are required (+5.3V, +2.3V). The delivery is done through dedicated “power bar” running the full length of the rTPC. A set of 3 bars covers the 8 axial cathode pad electronics boards. 8 of them cover the circumference of the rTPC.

{|
! '''''Component'''''
! '''''+5.3V Supply'''''
! '''''+2.3V Supply'''''
|-
| Padwing board
| 10W (1.8A)
| 2W (0.8A)
|}

Overall the low power supplies are provided by a PL508-3U with 8 modules of either single or dual channels from Wiener.

[[File:imageservice18.png|469x360px]]

Figure 14 - Overall Low voltage distribution for anode wires and cathode pads



== Grounding scheme of the overall rTPC ==

The grounding scheme challenge in this setup is the far distance between the 2 end of the rTPC (2.3m) and the lack of accessibility to the bottom of the detector. The proposal is to use a Copper sheet mounted on the inner surface of the inner cylinder. This ground path will connect the 2 endplates to the High Voltage return. All the pre-amps will merge their ground to this central ground.

Additional ground lugs have been included in the Anode wire pre-amplifier boards to strengthen the ground as each anode wire is readout out from both end. This may be used once the full detector is assembled to address possible ground loop issues.

[[File:imageservice19.png|439x409px]]

= Safety and Hazard considerations =

== Gas ==

The Ar and CO2 do not require special health safety system. The detector will be operated in the ventilated hall of the AD building at CERN.

Gas flow, detector gas pressure, and gas inlet temperature are constantly monitored and can trigger adequate response for shutdown or closure of the gas system for safety purpose.

== Cooling water ==

The water cooling system operates in negative pressure mode preventing any water leak to flood the detector outer surface. In addition water flow and inlet/outlet temperature are constantly monitored to allow fast response to off range parameter in order to shut down the electronics. The cooling water system requires a primary cooling tap water loop from the AD building. Water quality, water flow and water temperature are not under our responsibility. This aspect will be dealt during the initial phase of the rTPC commissioning at CERN.

== Power distribution ==

All electronics voltages are below 48V. No special safety circuits are envisaged.

The high-voltage circuit is fully protected outside the detector. The access to the high-voltage on the detector is restricted to the two end-plates. Interlock scheme will be put in place to ensure the shutdown of the HV in case of manual access to the detector itself.

The LV (Low Voltage) power units are also interlocked to prevent false powering sequence which can damage the downstream electronics modules.

[[File:imageservice20.png|571x411px]]

Figure 16- Alpha-g rTPC rack organization

Detector Services

2019-01-09T17:19:43Z

Acapra:

Back to [[Main Page]]

= Table of Figures =

[[#_Toc493684884|Figure 1 - Gas system schematic 6]]

[[#_Toc493684885|Figure 2 - A half-rack space is used for the gas control implementation. 7]]

[[#_Toc493684886|Figure 3 - Gas system half rack ready for installation 8]]

[[#_Toc493684887|Figure 4 - Cathode pad half-cylinder. Pairs of cooling pipes are visible 9]]

[[#_Toc493684888|Figure 5 - Water cooling system during test. 9]]

[[#_Toc493684889|Figure 6 - Model and 3D-printed realization of the dry-air manifold, very similar to the manifolds for the water cooling system. 12]]

[[#_Toc493684890|Figure 7 - Overall water cooling system 13]]

[[#_Toc493684891|Figure 8 - Temperature Monitor Board 14]]

[[#_Toc493684892|Figure 9 - End plate view of the layout of the different electronics boards & HV connections 15]]

[[#_Toc493684893|Figure 10 – Unpotted Anode Wire Card and Board with HV distribution 17]]

[[#_Toc493684894|Figure 11 - Anode wire pins and Field wire pins under yellow cover 17]]

[[#_Toc493684895|Figure 12 – New Anode Wire card with 16 potted HV capacitors, gold crimp pin Mill-max socket to the anode wire 18]]

[[#_Toc493684896|Figure 13 - Anode wire pre-amp power distribution & 16 channel temperature boards 19]]

[[#_Toc493684897|Figure 14 - Overall Low voltage distribution for anode wires and cathode pads 21]]

[[#_Toc493684898|Figure 15 - High Voltage & Grounding scheme for the rTPC 22]]

[[#_Toc493684899|Figure 16- Alpha-g rTPC rack organization 23]]

= Introduction =



This document is to describe the different services for the rTPC detector. 3 main sections cover the ''Gas system'', the ''Water cooling system'' and the ''Power distribution''.

= References and related Documents =

A dedicated TRIUMF Wiki site gathers all documentations for the rTPC detector system and equipment. Gas system, cooling water system and power distribution can be found there.

= Gas system =

The gas system as described below includes the entire gas infrastructure required to deliver gas to the detector, monitor and control the gas flow into the detector. It also includes the necessary equipment to notify any change from the nominal gas flow or gas mixture ratio delivered to the detector.

This type of mixed gas system is well understood by the Triumf detector group staff and no issue in its realization is expected.

This system is a stand-alone infrastructure assembled and delivered to CERN. It will be placed in the Alpha-g experimental area floor. The delivery of the different gases to the experimental area is under the responsibility of the Gas handling group from CERN.

The gas delivery system for the radial TPC (rTPC) is to provide Ar/CO2 gas mixture between 90/10 to 50/50 depending on the user requirements. This system does not recycle the gas as the environmental impact of either gas is minimal and the operation cost versus a recycling system would not be economic.

In the event of an additional quenching gas requirement, the overall gas system will have to be reconsidered and possible recycling system envisaged.

A simple gas mixer using digital flow meters will provide the precise mixture dialed by hand in the control system.

During the prototype phase, an ad-hoc gas system will be put in place. A portable gas system is available for such a test.

== Specifications ==

{|
! '''''Parameter'''''
! '''''Value'''''
|-
| Gas mixture
| Ar/CO2 (90/10..50/50)
|-
| Gas flow requirements
| 0.3l/min (50%), 0.3l/min (50%)
|-
| Detector gas Volume
| 0.18m3
|-
| Flushing time
| 5h @ 600cc/min
|-
| Flow rate
| 100 to 300 cc/minute
|}

== Operation ==

This gas system is quite simple as it takes 2 or 3 (spare) gas supplies and mixes them in a manifold before delivery to the rTPC detector. Necessary valves for purging lines and by-passing the detector are foreseen. The individual in-flows are controlled by gas dedicated mass flow controllers (MFC-x). The return flow is measured for leak assessment. An additional manometer monitors the internal detector pressure for evaluation of the electron drift characteristics.

[[File:imageservice2.png|575x315px|PA_doc-1.PNG]]

Figure 1 - Gas system schematic

The main flow controller and monitoring is performed by industrial devices from MKS. We added analog flowmeters on each gas line to provide a quick visual inspection of the gas mixing system.

This system will be hooked to the CERN AD-Gas facility. The CERN gas delivers gas through 2 switchable packs of 12 bottles for each gas.

The gas exhaust is evacuated to the exterior of the experimental area with an elevation of about 8m.

[[File:imageservice3.png|366x263px|PA_doc-2.PNG]]

Figure 2 - A half-rack space is used for the gas control implementation.



[[File:imageservice4.png|324x364px]]

Figure 3 - Gas system half rack ready for installation

== Quick Start ==
From-scratch (Neither gas supply lines to the GHS nor connections to and from the TPC connected) connection of the GHS should follow the following procedure:
# '''Flush supply lines:''' Gas lines from outside bottles going to the GHS should be briefly (5 min or so) flushed with a high gas flow ''before'' connecting to the GHS to remove possible dust, and to minimize the amount of air in the lines.
# '''Flush GHS:'''
## Close BV1 and BV3, open BV2 and BV4 to flush input lines, only briefly.
## Open BV1 and BV3, close BV2 and BV4 (normal operation configuration)
[[File:20181205 103217.jpg|thumb|Normal configuration of the manual valves of the GHS.]]
## With Midas frontend running, use the "Gas" custom page to set a desired flow, for flushing probably between 1l/min and 1.5l/min (Without Midas, use the telnet interface), then press the "Bypass Flow" button, and let flow for a while (5-15min?)
# '''Flush TPC input line:'''
## Connect the line that will go from the GHS to the TPC input-tee on the GHS side, but leave disconnected from the tee
## do the same for the pressure gauge line
## On the "Gas" custom page, press the "Flow TPC" button and let flow for 5 min (Without Midas, use the telnet interface).
# Make all remaining connections

== Telnet Interface ==
If the Midas frontend is not available the GHS can be controlled via telnet. Connect with
telnet algas
then use the following command to control solenoid valves:
cowr do i 1
to energize SVi, i.e. to switch it from its normal closed or open state to the other state,
cowr do i 0
to de-energize, i.e. switch to default state.

Gas flow is set with:
mfcwr ao i <desired flow>
where i is 1 for Ar and 2 for CO2, and the desired flow is in the internal units of the system.

= Water cooling system =

The outer surface of the detector has an arrangement of 8x8 electronics boards handling the readout of the cathode pads. These boards require operating at a controlled temperature to prevent heat build-up at the surface of the detector and improving the reliability of the system. In order to do so, 8 independent water pipe loops running from the bottom-up and down in the longitudinal axis of the detector are in a thermal contact to 8 successive readout boards. From the bottom, a pair of water hoses connect to 2 custom 3D-printed manifolds supplying the water feed and return to the 8 water loops.

These 8 loops are run in parallel to a negative pressure water cooling system. This negative pressure system is to ensure that no water leak will damage the electronics even during possible pipe or delivery hose breakage.

[[File:imageservice5.png|516x273px]]

Figure 4 - Cathode pad half-cylinder. Pairs of cooling pipes are visible

[[File:imageservice6.jpg|249x429px]]

Figure 5 - Water cooling system during test.

== Specifications ==

{|
! '''''Parameters'''''
! '''''Values'''''
|-
| Number of parallel water loops
| 8
|-
| Copper pipe OD/ID
| ¼” / 4.7mm
|-
| Length of one loop (on the rTPC)
| 4.6m
|-
| Water speed per loop
| …
|-
| Pressure drop per loop
| …
|-
| Power extraction per water loop
| 8x15W= 120W
|-
| Total cooling power
| 1KW
|-
| Inlet water temperature
| 16..20 degC (to be confirmed)
|-
| Outlet water temperature
| 20..24 degC
|-
| Total water flow
| 3..6 l/minute
|}

== Operation ==

''This description corresponds to how we think the system should work, but doesn't seem fully comatible with the behaviour of the system. E.g. the air pump only seems to turn on when PID1 falls below a certain value.''

The Pressure PID (proportional/integral/differential controller), PID2, monitors the air pressure in the vacuum vessel. When the pressure rises above a trigger value (perhaps 350mbar) it activates the aspirator vacuum pump to bring the pressure down to maintain a long time averaged pressure (say 300mbar).

This vacuum is then used to draw water through the detector into the vessel, from the Open Tank. The vacuum vessel's water level is monitored by PID1, using a differential pressure transducer measuring the weight of the water column by the difference between the air pressure in the top of the vessel and the water pressure at the bottom of the vessel. PID1 provides a variable signal to the Proportional Solenoid Valve, which varies the flow through the return loop around the (fixed flow) water pump, and thus changing the overall flow of water from the vacuum vessel to keep the water level in the vessel constant despite water being constantly drawn in by the vacuum. The water pulled from the vacuum vessel is pushed through the filter and into the Open Tank, completing the circulation.

Cooling is accomplished in the water pump leg of the system, as coolant pulled from the vacuum vessel passes through a heat exchanger coupled to CERN’s cooling water supply on its way to the pump. The rate that water flows through the system can be determined by many obstacles in the loop, but ideally it should be dominated by the position of the main Throttle Valve. Other valves should generally be set either fully open of fully closed, except for the Proportional Valve, whose position responds to the setting of the Throttle Valve.

Data outputs: there are five data outputs planned. The two process variables, the vacuum vessel absolute pressure, and differential water column pressure, will be sent to the Detector DAQ system. As well, the total flow rate is monitored by the flow meter, whose output is sent to the DAQ system. This data is not used by the circulation system itself. Likewise, the back pressure on the water filter is only sent to the DAQ system to indicate if the filter may need changing, and the temperature of the open tank is monitored and sent to the DAQ system, but currently there is no plan to use it in a temperature control feedback. The temperature of CERN's cooling water supply is not yet known, but we expect we can set the heat exchanger to run at a rate to hold the system temperature near 20°C without feedback control.

For the negative pressure to work effectively to prevent leaks, the water pressure in the detector ought to be down at least to half an atmosphere. With the detector cooling tubes 8x 4.6m of parallel 4.7mm ID copper tubes (2.3m down then 2.3 back-ups), and the supply using (perhaps 6m, 3 to the detector then 3 back) 9.7mm ID (=1/2" OD) poly tubing, the flow will need to be somewhat greater than the required 3 l/min in order to maintain that degree of vacuum in the detector, unless the Throttle Valve is placed upstream of the detector near the Open Tank, or an additional constriction is added in that region.

The water pump leg of the circuit residing in the water cooling/gas rack is placed far enough from the fringe field of the superconducting magnet to prevent disruptive effects on the solenoid valves and electric motors. Therefore the main 1/2" poly line to and from the water pump and its flow loop will add a pressure drop which is estimated at 0.4atm for a 10m line at 6 l/min (9.7mmID), which is within our requirements. As mentioned earlier, the splitting (and reassembly) of the line to the 8 cooling lines close to detector will be accomplished with the use of 2 separate manifolds equipped with temperature sensors for constant in/out global water temperature monitoring.

[[File:imageservice7.png|286x206px]][[File:imageservice8.jpg|286x206px]]

Figure 6 - Model and 3D-printed realization of the dry-air manifold, very similar to the manifolds for the water cooling system.

Currently, PVC valves with "true union" couplings are planned to be provided around the water pump to permit it to be swapped out from the system for servicing and a valved bypass to the water filter is provided for the same purpose to allow easily changing filter elements with the system running. True union valves are generally only available to 1/2NPT and larger, so an adapter fitting is required. The vacuum pump for the system is a small (2 x 5 x 7.5cm) cascaded dual Venturi aspirator, which is operated with site air, switched with two simultaneously operated normally closed solenoid valves, one for the site air, and one for the vessel vacuum input. If the system is leak free, the vacuum in the vessel should only be degraded by the dissolved air absorbed in the Open Tank and extracted in the vacuum vessel, which from past experience might mean running for under a minute every six hours or so.

The wetted surfaces of this system will be mainly PVC and Polyethylene for the valves, tubing and vessels, plus brass fittings, and the copper cooling tubes, and the surfaces of the heat exchanger, which will most likely be stainless steel. It may be possible to devise an equivalent of a bellows communicating with the Open Tank, so that it is only virtually open, thus sealing the cooling water from contaminants, which may allow us to run with just distilled water plus a strip of silver as a biocide (although I'm not sure if the mix of metals, copper plus stainless, may not interfere with the function of the silver, which is mostly used in computer cooling systems which are copper and plastic only). Otherwise, we will need a biocide and corrosion inhibitor. We have the latter, tolyltriazole, but the biocide is problematic. They tend to be short-lived; for systems which are open to atmosphere, typical industrial practice is chlorination, regularly refreshed, which builds up NaCl as it degrades, making the coolant corrosive. For T2K in Japan, we used the (human-nontoxic) "Germall", which has a two-year lifetime, after which when its effectiveness decays it turns into food, which is the case for any solute that is not actively biocidal. Eventually, any such additives build up and require the coolant to be flushed and replaced. Silver is a lovely method, but only works for extended periods with a sealed system which remains deoxygenated. My research so far has not turned up a good candidate which does not entail recurring changes of the coolant.

Power: the system needs to be able to run on European 220VAC power, but also operate here for testing purposes. The PIDs (Omega CN76160s) will accept both; for the rest of the system, I will have an a universal 90-240VAC input to 24VDC (5A) supply which will drive all the other components, including the pump, which will be a brushless 24VDC centrifugal pump, 1 - 1.5A range. The whole system will run about 125W.

Information on the temperature and flow rate of the CERN cooling water system will be provided later but we don’t anticipate any major issues about the sizing of the heat exchanger.

[[File:imageservice9.png|603x344px]]

Figure 7 - Overall water cooling system



== Temperature Measurement ==

[[File:imageservice10.jpeg|225x356px]] [[File:imageservice11.png|194x358px]]

Figure 8 Temperature Monitor Board

The water temperature is read by the Temperature Monitor Board (TEMPB). Thermistors located inside the manifold connect to a 16 bit ADC on the TEMPB which is read out by a Raspberry Pi mounted on board. In a similar manner the air system temperature is read with sensors in the air manifold. The TEMPB can provide temperature readout of 32 thermistors which connect to terminal blocks. The channels can also be readout differentially providing up to 16 channels for thermocouples. An on board thermistor allows for cold junction compensation. The board is powered via USB on the Raspberry Pi.

== Quick Start ==
This assumes all water lines and compressed air are connected, and both the blue water tank and the chiller are filled.

=== Begin running ===
# Close vent valve and main valve (3-way valve in T position)
# Switch on control box with power switch in the back
# confirm water pump is off, i.e. blue button is not illuminated
# Vacuum pump should now be running
# Open main valve, i.e. vertical connection open, connection to vent valve closed [[File:PWBmainOpen.jpg|thumb|Cooling system main valve open]]
# water should start flowing through the PWB cooling system and coming back through the clear pipe connected to the vacuum vessel
# Default settings are PID1 = 200, PID2 = 300, but may have been adjusted
# once the water level in the vacuum vessel has risen to ~20cm, turn on the water pump

=== Drain system ===
# from normal operation, i.e. vacuum pump and water pump running
# switch main valve to upside down T position
[[File:PWBmainVent.jpg|thumb|Cooling system main valve to vent valve]]
# open vent valve
# air should now enter the system, draining all water from the TPC pipes into the vacuum vessel
# water pump will drain the vacuum vessel into supply tank, turn it off before the vessel is completely empty to avoid air in the water pump
# switch off main power

== Troubleshooting ==
The water system is temperamental, and not intuitive at all. Common problems and solution attempts:

; water pump very loud, doesn't appear to pump water
: most likely air in the water pump. Open the vent of the vacuum vessel to allow water to enter pump by gravity alone, pump should stop screaming and start pumping. Close vent.
; vacuum pump comes on often (more then every 10-20 min)
: probably an air leak in the system. Check for bubbles and try to find leak.
; water level/flow/vacuum oscillate between extreme values
: unclear, try playing with PID settings, throttle valve, possibly orange ball valves by the water pump.

= Power distribution =

This section covers the different voltages and power requirements for the operation of the rTPC. This includes:

# High Voltage for the rTPC electron drift and amplification operation
# Low Voltage to the Anode wire frontend electronics
# Low Voltage to the Barrel Scintillator frontend electronics
# Low Voltage to the Cathode Pads electronics
# Grounding scheme of the overall rTPC

The provision of power to the different components of the detector is distributed from the top end plate of the rTPC.

[[File:imageservice12.png|507x354px]]

Figure 9 - End plate view of the layout of the different electronics boards & HV connections

== High Voltage for the rTPC ==

Three main components of the rTPC require High Voltage

* Main cathode wall (Negative ~5KV)
* Field wires (Negative ~ few hundred volts)
* Anode wires (Positive ~3.2KV )

These voltages are provided by a Quad High Voltage unit from CAEN. Three independent cables will be fed to the top of the rTPC.

High Voltage provision to the bottom endplate HV ring is taken from the central cathode near the bottom endplate.

The Field wire voltage is connected to all the field wire crimp-pins through a dedicated socket chain inserted on each of the crimp-pin on the top and bottom endplates. The field wires themselves interconnect the high voltage between the top and bottom pins.

The High Voltage for the anode wires is delivered to the Anode Wire Card (AWC) which will distribute it to its 16 wires, Figure 11. There are 16 AWC cards covering each end of the anode wires. All the top and bottom 32 AWC are connected serially. The anode wires themselves interconnect the high voltage between the top and bottom pins.

The anode wires pre-amplifiers boards (AWB) are sitting on the top of the Anode Wire Card (AWC) Figure 10. As the wire is powered by the HV, each anode wire has a high Voltage decoupling capacitor to isolate the pre-amp stage from the HV. This circuit resides on the AWC. The AWB will see the AC coupled anode signal only. With a new design of the AWC Figure 12, the decoupling capacitors are potted in epoxy to ensure proper isolation of the HV toward the surrounding circuit as well to prevent corona discharge due to relative humidity of the surrounding.

[[File:imageservice13.jpg|402x309px]]

Figure 10 – Unpotted Anode Wire Card and Board with HV distribution

[[File:imageservice14.jpg|402x309px]]

Figure 11 - Anode wire pins and Field wire pins under yellow cover

[[File:imageservice15.jpg|402x309px]]

Figure 12 – New Anode Wire card with 16 potted HV capacitors, gold crimp pin Mill-max socket to the anode wire

== Power distribution to the Anode wire frontend electronics ==

The AWB pre-amp board requires 2 voltages (+5V, -5V) which are fed through dedicated cable connections to each of the AWB. There are 16 AWB per end of the rTPC, therefore, 16 individual cable bundle will be reaching the top and another 16 the bottom of the rTPC. The cable bundle is to also provide the power return lines to the Power Supply.

The PL508 Low Voltage power Supply from Wiener will provide the main power source for all the 32 pre-amplifiers. A dedicated connection breakout board with current monitoring capability splits the source power to the 2 x 16 necessary connections to the Anode wire boards. Based on a RaspeberryPi card, a 16 power distribution board is already implemented for the prototype. This board has extra 4 channels of temperature monitoring which can be used later on. In the same process we produce a similar board for 16 channel of NTC, RTC temperature monitoring. They will be used for overall detector and cooling system monitoring.

{|
! '''''Component'''''
! '''''+5V Supply'''''
! '''''-5V Supply'''''
|-
| Anode Pre-amp
| 1.5W (0.3A)
| 1.5W (0.3A)
|}

[[File:imageservice16.jpg|239x363px]] [[File:imageservice17.png|239x363px]]

Figure 13 - Anode wire pre-amp power distribution & 16 channel temperature boards

== Power distribution to the Barrel Scintillator frontend electronics ==

Similar to the Anode wire pre-amplifier voltage distribution scheme, the Barrel Scintillators will require several low voltages for power and control. The Barrel Scintillator Boards (BSB) are mounted on both ends of the Scintillator bar with the same segmentation as the Anode wires i.e.: 16 BSB per end. The design of the electronic chain for the barrel scintillator is not fully designed yet.

The SiPM sensors require a bias voltage of the order of 28V. This voltage will be provided through the same mean as the low voltage power. Expected threshold voltage for the local signal level discriminator is also foreseen.

== Low Voltage to the Cathode Pads electronics ==

The cathode pads electronics mounted on the external cylinder surface is composed of the analog frontend and the digital conversion electronics. Two voltages are required (+5.3V, +2.3V). The delivery is done through dedicated “power bar” running the full length of the rTPC. A set of 3 bars covers the 8 axial cathode pad electronics boards. 8 of them cover the circumference of the rTPC.

{|
! '''''Component'''''
! '''''+5.3V Supply'''''
! '''''+2.3V Supply'''''
|-
| Padwing board
| 10W (1.8A)
| 2W (0.8A)
|}

Overall the low power supplies are provided by a PL508-3U with 8 modules of either single or dual channels from Wiener.

[[File:imageservice18.png|469x360px]]

Figure 14 - Overall Low voltage distribution for anode wires and cathode pads



== Grounding scheme of the overall rTPC ==

The grounding scheme challenge in this setup is the far distance between the 2 end of the rTPC (2.3m) and the lack of accessibility to the bottom of the detector. The proposal is to use a Copper sheet mounted on the inner surface of the inner cylinder. This ground path will connect the 2 endplates to the High Voltage return. All the pre-amps will merge their ground to this central ground.

Additional ground lugs have been included in the Anode wire pre-amplifier boards to strengthen the ground as each anode wire is readout out from both end. This may be used once the full detector is assembled to address possible ground loop issues.

[[File:imageservice19.png|439x409px]]

= Safety and Hazard considerations =

== Gas ==

The Ar and CO2 do not require special health safety system. The detector will be operated in the ventilated hall of the AD building at CERN.

Gas flow, detector gas pressure, and gas inlet temperature are constantly monitored and can trigger adequate response for shutdown or closure of the gas system for safety purpose.

== Cooling water ==

The water cooling system operates in negative pressure mode preventing any water leak to flood the detector outer surface. In addition water flow and inlet/outlet temperature are constantly monitored to allow fast response to off range parameter in order to shut down the electronics. The cooling water system requires a primary cooling tap water loop from the AD building. Water quality, water flow and water temperature are not under our responsibility. This aspect will be dealt during the initial phase of the rTPC commissioning at CERN.

== Power distribution ==

All electronics voltages are below 48V. No special safety circuits are envisaged.

The high-voltage circuit is fully protected outside the detector. The access to the high-voltage on the detector is restricted to the two end-plates. Interlock scheme will be put in place to ensure the shutdown of the HV in case of manual access to the detector itself.

The LV (Low Voltage) power units are also interlocked to prevent false powering sequence which can damage the downstream electronics modules.

[[File:imageservice20.png|571x411px]]

Figure 16- Alpha-g rTPC rack organization

File:20181205 103217.jpg

2019-01-09T17:18:37Z

Acapra:

Normal configuration of the manual valves of the GHS.

Detector Services

2018-12-07T13:20:02Z

Acapra:

Back to [[Main Page]]

= Table of Figures =

[[#_Toc493684884|Figure 1 - Gas system schematic 6]]

[[#_Toc493684885|Figure 2 - A half-rack space is used for the gas control implementation. 7]]

[[#_Toc493684886|Figure 3 - Gas system half rack ready for installation 8]]

[[#_Toc493684887|Figure 4 - Cathode pad half-cylinder. Pairs of cooling pipes are visible 9]]

[[#_Toc493684888|Figure 5 - Water cooling system during test. 9]]

[[#_Toc493684889|Figure 6 - Model and 3D-printed realization of the dry-air manifold, very similar to the manifolds for the water cooling system. 12]]

[[#_Toc493684890|Figure 7 - Overall water cooling system 13]]

[[#_Toc493684891|Figure 8 - Temperature Monitor Board 14]]

[[#_Toc493684892|Figure 9 - End plate view of the layout of the different electronics boards & HV connections 15]]

[[#_Toc493684893|Figure 10 – Unpotted Anode Wire Card and Board with HV distribution 17]]

[[#_Toc493684894|Figure 11 - Anode wire pins and Field wire pins under yellow cover 17]]

[[#_Toc493684895|Figure 12 – New Anode Wire card with 16 potted HV capacitors, gold crimp pin Mill-max socket to the anode wire 18]]

[[#_Toc493684896|Figure 13 - Anode wire pre-amp power distribution & 16 channel temperature boards 19]]

[[#_Toc493684897|Figure 14 - Overall Low voltage distribution for anode wires and cathode pads 21]]

[[#_Toc493684898|Figure 15 - High Voltage & Grounding scheme for the rTPC 22]]

[[#_Toc493684899|Figure 16- Alpha-g rTPC rack organization 23]]

= Introduction =



This document is to describe the different services for the rTPC detector. 3 main sections cover the ''Gas system'', the ''Water cooling system'' and the ''Power distribution''.

= References and related Documents =

A dedicated TRIUMF Wiki site gathers all documentations for the rTPC detector system and equipment. Gas system, cooling water system and power distribution can be found there.

= Gas system =

The gas system as described below includes the entire gas infrastructure required to deliver gas to the detector, monitor and control the gas flow into the detector. It also includes the necessary equipment to notify any change from the nominal gas flow or gas mixture ratio delivered to the detector.

This type of mixed gas system is well understood by the Triumf detector group staff and no issue in its realization is expected.

This system is a stand-alone infrastructure assembled and delivered to CERN. It will be placed in the Alpha-g experimental area floor. The delivery of the different gases to the experimental area is under the responsibility of the Gas handling group from CERN.

The gas delivery system for the radial TPC (rTPC) is to provide Ar/CO2 gas mixture between 90/10 to 50/50 depending on the user requirements. This system does not recycle the gas as the environmental impact of either gas is minimal and the operation cost versus a recycling system would not be economic.

In the event of an additional quenching gas requirement, the overall gas system will have to be reconsidered and possible recycling system envisaged.

A simple gas mixer using digital flow meters will provide the precise mixture dialed by hand in the control system.

During the prototype phase, an ad-hoc gas system will be put in place. A portable gas system is available for such a test.

== Specifications ==

{|
! '''''Parameter'''''
! '''''Value'''''
|-
| Gas mixture
| Ar/CO2 (90/10..50/50)
|-
| Gas flow requirements
| 0.3l/min (50%), 0.3l/min (50%)
|-
| Detector gas Volume
| 0.18m3
|-
| Flushing time
| 5h @ 600cc/min
|-
| Flow rate
| 100 to 300 cc/minute
|}

== Operation ==

This gas system is quite simple as it takes 2 or 3 (spare) gas supplies and mixes them in a manifold before delivery to the rTPC detector. Necessary valves for purging lines and by-passing the detector are foreseen. The individual in-flows are controlled by gas dedicated mass flow controllers (MFC-x). The return flow is measured for leak assessment. An additional manometer monitors the internal detector pressure for evaluation of the electron drift characteristics.

[[File:imageservice2.png|575x315px|PA_doc-1.PNG]]

Figure 1 - Gas system schematic

The main flow controller and monitoring is performed by industrial devices from MKS. We added analog flowmeters on each gas line to provide a quick visual inspection of the gas mixing system.

This system will be hooked to the CERN AD-Gas facility. The CERN gas delivers gas through 2 switchable packs of 12 bottles for each gas.

The gas exhaust is evacuated to the exterior of the experimental area with an elevation of about 8m.

[[File:imageservice3.png|366x263px|PA_doc-2.PNG]]

Figure 2 - A half-rack space is used for the gas control implementation.



[[File:imageservice4.png|324x364px]]

Figure 3 - Gas system half rack ready for installation

== Quick Start ==
From-scratch (Neither gas supply lines to the GHS nor connections to and from the TPC connected) connection of the GHS should follow the following procedure:
# '''Flush supply lines:''' Gas lines from outside bottles going to the GHS should be briefly (5 min or so) flushed with a high gas flow ''before'' connecting to the GHS to remove possible dust, and to minimize the amount of air in the lines.
# '''Flush GHS:'''
## Close BV3 and BV5, open BV2 and BV4 to flush input lines, only briefly.
## Open BV3 and BV5, close BV2 and BV4 (normal operation configuration)
## With Midas frontend running, use the "Gas" custom page to set a desired flow, for flushing probably between 1l/min and 1.5l/min (Without Midas, use the telnet interface), then press the "Bypass Flow" button, and let flow for a while (5-15min?)
# '''Flush TPC input line:'''
## Connect the line that will go from the GHS to the TPC input-tee on the GHS side, but leave disconnected from the tee
## do the same for the pressure gauge line
## On the "Gas" custom page, press the "Flow TPC" button and let flow for 5 min (Without Midas, use the telnet interface).
# Make all remaining connections

== Telnet Interface ==
If the Midas frontend is not available the GHS can be controlled via telnet. Connect with
telnet algas
then use the following command to control solenoid valves:
cowr do i 1
to energize SVi, i.e. to switch it from its normal closed or open state to the other state,
cowr do i 0
to de-energize, i.e. switch to default state.

Gas flow is set with:
mfcwr ao i <desired flow>
where i is 1 for Ar and 2 for CO2, and the desired flow is in the internal units of the system.

= Water cooling system =

The outer surface of the detector has an arrangement of 8x8 electronics boards handling the readout of the cathode pads. These boards require operating at a controlled temperature to prevent heat build-up at the surface of the detector and improving the reliability of the system. In order to do so, 8 independent water pipe loops running from the bottom-up and down in the longitudinal axis of the detector are in a thermal contact to 8 successive readout boards. From the bottom, a pair of water hoses connect to 2 custom 3D-printed manifolds supplying the water feed and return to the 8 water loops.

These 8 loops are run in parallel to a negative pressure water cooling system. This negative pressure system is to ensure that no water leak will damage the electronics even during possible pipe or delivery hose breakage.

[[File:imageservice5.png|516x273px]]

Figure 4 - Cathode pad half-cylinder. Pairs of cooling pipes are visible

[[File:imageservice6.jpg|249x429px]]

Figure 5 - Water cooling system during test.

== Specifications ==

{|
! '''''Parameters'''''
! '''''Values'''''
|-
| Number of parallel water loops
| 8
|-
| Copper pipe OD/ID
| ¼” / 4.7mm
|-
| Length of one loop (on the rTPC)
| 4.6m
|-
| Water speed per loop
| …
|-
| Pressure drop per loop
| …
|-
| Power extraction per water loop
| 8x15W= 120W
|-
| Total cooling power
| 1KW
|-
| Inlet water temperature
| 16..20 degC (to be confirmed)
|-
| Outlet water temperature
| 20..24 degC
|-
| Total water flow
| 3..6 l/minute
|}

== Operation ==

''This description corresponds to how we think the system should work, but doesn't seem fully comatible with the behaviour of the system. E.g. the air pump only seems to turn on when PID1 falls below a certain value.''

The Pressure PID (proportional/integral/differential controller), PID2, monitors the air pressure in the vacuum vessel. When the pressure rises above a trigger value (perhaps 350mbar) it activates the aspirator vacuum pump to bring the pressure down to maintain a long time averaged pressure (say 300mbar).

This vacuum is then used to draw water through the detector into the vessel, from the Open Tank. The vacuum vessel's water level is monitored by PID1, using a differential pressure transducer measuring the weight of the water column by the difference between the air pressure in the top of the vessel and the water pressure at the bottom of the vessel. PID1 provides a variable signal to the Proportional Solenoid Valve, which varies the flow through the return loop around the (fixed flow) water pump, and thus changing the overall flow of water from the vacuum vessel to keep the water level in the vessel constant despite water being constantly drawn in by the vacuum. The water pulled from the vacuum vessel is pushed through the filter and into the Open Tank, completing the circulation.

Cooling is accomplished in the water pump leg of the system, as coolant pulled from the vacuum vessel passes through a heat exchanger coupled to CERN’s cooling water supply on its way to the pump. The rate that water flows through the system can be determined by many obstacles in the loop, but ideally it should be dominated by the position of the main Throttle Valve. Other valves should generally be set either fully open of fully closed, except for the Proportional Valve, whose position responds to the setting of the Throttle Valve.

Data outputs: there are five data outputs planned. The two process variables, the vacuum vessel absolute pressure, and differential water column pressure, will be sent to the Detector DAQ system. As well, the total flow rate is monitored by the flow meter, whose output is sent to the DAQ system. This data is not used by the circulation system itself. Likewise, the back pressure on the water filter is only sent to the DAQ system to indicate if the filter may need changing, and the temperature of the open tank is monitored and sent to the DAQ system, but currently there is no plan to use it in a temperature control feedback. The temperature of CERN's cooling water supply is not yet known, but we expect we can set the heat exchanger to run at a rate to hold the system temperature near 20°C without feedback control.

For the negative pressure to work effectively to prevent leaks, the water pressure in the detector ought to be down at least to half an atmosphere. With the detector cooling tubes 8x 4.6m of parallel 4.7mm ID copper tubes (2.3m down then 2.3 back-ups), and the supply using (perhaps 6m, 3 to the detector then 3 back) 9.7mm ID (=1/2" OD) poly tubing, the flow will need to be somewhat greater than the required 3 l/min in order to maintain that degree of vacuum in the detector, unless the Throttle Valve is placed upstream of the detector near the Open Tank, or an additional constriction is added in that region.

The water pump leg of the circuit residing in the water cooling/gas rack is placed far enough from the fringe field of the superconducting magnet to prevent disruptive effects on the solenoid valves and electric motors. Therefore the main 1/2" poly line to and from the water pump and its flow loop will add a pressure drop which is estimated at 0.4atm for a 10m line at 6 l/min (9.7mmID), which is within our requirements. As mentioned earlier, the splitting (and reassembly) of the line to the 8 cooling lines close to detector will be accomplished with the use of 2 separate manifolds equipped with temperature sensors for constant in/out global water temperature monitoring.

[[File:imageservice7.png|286x206px]][[File:imageservice8.jpg|286x206px]]

Figure 6 - Model and 3D-printed realization of the dry-air manifold, very similar to the manifolds for the water cooling system.

Currently, PVC valves with "true union" couplings are planned to be provided around the water pump to permit it to be swapped out from the system for servicing and a valved bypass to the water filter is provided for the same purpose to allow easily changing filter elements with the system running. True union valves are generally only available to 1/2NPT and larger, so an adapter fitting is required. The vacuum pump for the system is a small (2 x 5 x 7.5cm) cascaded dual Venturi aspirator, which is operated with site air, switched with two simultaneously operated normally closed solenoid valves, one for the site air, and one for the vessel vacuum input. If the system is leak free, the vacuum in the vessel should only be degraded by the dissolved air absorbed in the Open Tank and extracted in the vacuum vessel, which from past experience might mean running for under a minute every six hours or so.

The wetted surfaces of this system will be mainly PVC and Polyethylene for the valves, tubing and vessels, plus brass fittings, and the copper cooling tubes, and the surfaces of the heat exchanger, which will most likely be stainless steel. It may be possible to devise an equivalent of a bellows communicating with the Open Tank, so that it is only virtually open, thus sealing the cooling water from contaminants, which may allow us to run with just distilled water plus a strip of silver as a biocide (although I'm not sure if the mix of metals, copper plus stainless, may not interfere with the function of the silver, which is mostly used in computer cooling systems which are copper and plastic only). Otherwise, we will need a biocide and corrosion inhibitor. We have the latter, tolyltriazole, but the biocide is problematic. They tend to be short-lived; for systems which are open to atmosphere, typical industrial practice is chlorination, regularly refreshed, which builds up NaCl as it degrades, making the coolant corrosive. For T2K in Japan, we used the (human-nontoxic) "Germall", which has a two-year lifetime, after which when its effectiveness decays it turns into food, which is the case for any solute that is not actively biocidal. Eventually, any such additives build up and require the coolant to be flushed and replaced. Silver is a lovely method, but only works for extended periods with a sealed system which remains deoxygenated. My research so far has not turned up a good candidate which does not entail recurring changes of the coolant.

Power: the system needs to be able to run on European 220VAC power, but also operate here for testing purposes. The PIDs (Omega CN76160s) will accept both; for the rest of the system, I will have an a universal 90-240VAC input to 24VDC (5A) supply which will drive all the other components, including the pump, which will be a brushless 24VDC centrifugal pump, 1 - 1.5A range. The whole system will run about 125W.

Information on the temperature and flow rate of the CERN cooling water system will be provided later but we don’t anticipate any major issues about the sizing of the heat exchanger.

[[File:imageservice9.png|603x344px]]

Figure 7 - Overall water cooling system



== Temperature Measurement ==

[[File:imageservice10.jpeg|225x356px]] [[File:imageservice11.png|194x358px]]

Figure 8 Temperature Monitor Board

The water temperature is read by the Temperature Monitor Board (TEMPB). Thermistors located inside the manifold connect to a 16 bit ADC on the TEMPB which is read out by a Raspberry Pi mounted on board. In a similar manner the air system temperature is read with sensors in the air manifold. The TEMPB can provide temperature readout of 32 thermistors which connect to terminal blocks. The channels can also be readout differentially providing up to 16 channels for thermocouples. An on board thermistor allows for cold junction compensation. The board is powered via USB on the Raspberry Pi.

== Quick Start ==
This assumes all water lines and compressed air are connected, and both the blue water tank and the chiller are filled.

=== Begin running ===
# Close vent valve and main valve (3-way valve in T position)
# Switch on control box with power switch in the back
# confirm water pump is off, i.e. blue button is not illuminated
# Vacuum pump should now be running
# Open main valve, i.e. vertical connection open, connection to vent valve closed [[File:PWBmainOpen.jpg|thumb|Cooling system main valve open]]
# water should start flowing through the PWB cooling system and coming back through the clear pipe connected to the vacuum vessel
# Default settings are PID1 = 200, PID2 = 300, but may have been adjusted
# once the water level in the vacuum vessel has risen to ~20cm, turn on the water pump

=== Drain system ===
# from normal operation, i.e. vacuum pump and water pump running
# switch main valve to upside down T position
[[File:PWBmainVent.jpg|thumb|Cooling system main valve to vent valve]]
# open vent valve
# air should now enter the system, draining all water from the TPC pipes into the vacuum vessel
# water pump will drain the vacuum vessel into supply tank, turn it off before the vessel is completely empty to avoid air in the water pump
# switch off main power

== Troubleshooting ==
The water system is temperamental, and not intuitive at all. Common problems and solution attempts:

; water pump very loud, doesn't appear to pump water
: most likely air in the water pump. Open the vent of the vacuum vessel to allow water to enter pump by gravity alone, pump should stop screaming and start pumping. Close vent.
; vacuum pump comes on often (more then every 10-20 min)
: probably an air leak in the system. Check for bubbles and try to find leak.
; water level/flow/vacuum oscillate between extreme values
: unclear, try playing with PID settings, throttle valve, possibly orange ball valves by the water pump.

= Power distribution =

This section covers the different voltages and power requirements for the operation of the rTPC. This includes:

# High Voltage for the rTPC electron drift and amplification operation
# Low Voltage to the Anode wire frontend electronics
# Low Voltage to the Barrel Scintillator frontend electronics
# Low Voltage to the Cathode Pads electronics
# Grounding scheme of the overall rTPC

The provision of power to the different components of the detector is distributed from the top end plate of the rTPC.

[[File:imageservice12.png|507x354px]]

Figure 9 - End plate view of the layout of the different electronics boards & HV connections

== High Voltage for the rTPC ==

Three main components of the rTPC require High Voltage

* Main cathode wall (Negative ~5KV)
* Field wires (Negative ~ few hundred volts)
* Anode wires (Positive ~3.2KV )

These voltages are provided by a Quad High Voltage unit from CAEN. Three independent cables will be fed to the top of the rTPC.

High Voltage provision to the bottom endplate HV ring is taken from the central cathode near the bottom endplate.

The Field wire voltage is connected to all the field wire crimp-pins through a dedicated socket chain inserted on each of the crimp-pin on the top and bottom endplates. The field wires themselves interconnect the high voltage between the top and bottom pins.

The High Voltage for the anode wires is delivered to the Anode Wire Card (AWC) which will distribute it to its 16 wires, Figure 11. There are 16 AWC cards covering each end of the anode wires. All the top and bottom 32 AWC are connected serially. The anode wires themselves interconnect the high voltage between the top and bottom pins.

The anode wires pre-amplifiers boards (AWB) are sitting on the top of the Anode Wire Card (AWC) Figure 10. As the wire is powered by the HV, each anode wire has a high Voltage decoupling capacitor to isolate the pre-amp stage from the HV. This circuit resides on the AWC. The AWB will see the AC coupled anode signal only. With a new design of the AWC Figure 12, the decoupling capacitors are potted in epoxy to ensure proper isolation of the HV toward the surrounding circuit as well to prevent corona discharge due to relative humidity of the surrounding.

[[File:imageservice13.jpg|402x309px]]

Figure 10 – Unpotted Anode Wire Card and Board with HV distribution

[[File:imageservice14.jpg|402x309px]]

Figure 11 - Anode wire pins and Field wire pins under yellow cover

[[File:imageservice15.jpg|402x309px]]

Figure 12 – New Anode Wire card with 16 potted HV capacitors, gold crimp pin Mill-max socket to the anode wire

== Power distribution to the Anode wire frontend electronics ==

The AWB pre-amp board requires 2 voltages (+5V, -5V) which are fed through dedicated cable connections to each of the AWB. There are 16 AWB per end of the rTPC, therefore, 16 individual cable bundle will be reaching the top and another 16 the bottom of the rTPC. The cable bundle is to also provide the power return lines to the Power Supply.

The PL508 Low Voltage power Supply from Wiener will provide the main power source for all the 32 pre-amplifiers. A dedicated connection breakout board with current monitoring capability splits the source power to the 2 x 16 necessary connections to the Anode wire boards. Based on a RaspeberryPi card, a 16 power distribution board is already implemented for the prototype. This board has extra 4 channels of temperature monitoring which can be used later on. In the same process we produce a similar board for 16 channel of NTC, RTC temperature monitoring. They will be used for overall detector and cooling system monitoring.

{|
! '''''Component'''''
! '''''+5V Supply'''''
! '''''-5V Supply'''''
|-
| Anode Pre-amp
| 1.5W (0.3A)
| 1.5W (0.3A)
|}

[[File:imageservice16.jpg|239x363px]] [[File:imageservice17.png|239x363px]]

Figure 13 - Anode wire pre-amp power distribution & 16 channel temperature boards

== Power distribution to the Barrel Scintillator frontend electronics ==

Similar to the Anode wire pre-amplifier voltage distribution scheme, the Barrel Scintillators will require several low voltages for power and control. The Barrel Scintillator Boards (BSB) are mounted on both ends of the Scintillator bar with the same segmentation as the Anode wires i.e.: 16 BSB per end. The design of the electronic chain for the barrel scintillator is not fully designed yet.

The SiPM sensors require a bias voltage of the order of 28V. This voltage will be provided through the same mean as the low voltage power. Expected threshold voltage for the local signal level discriminator is also foreseen.

== Low Voltage to the Cathode Pads electronics ==

The cathode pads electronics mounted on the external cylinder surface is composed of the analog frontend and the digital conversion electronics. Two voltages are required (+5.3V, +2.3V). The delivery is done through dedicated “power bar” running the full length of the rTPC. A set of 3 bars covers the 8 axial cathode pad electronics boards. 8 of them cover the circumference of the rTPC.

{|
! '''''Component'''''
! '''''+5.3V Supply'''''
! '''''+2.3V Supply'''''
|-
| Padwing board
| 10W (1.8A)
| 2W (0.8A)
|}

Overall the low power supplies are provided by a PL508-3U with 8 modules of either single or dual channels from Wiener.

[[File:imageservice18.png|469x360px]]

Figure 14 - Overall Low voltage distribution for anode wires and cathode pads



== Grounding scheme of the overall rTPC ==

The grounding scheme challenge in this setup is the far distance between the 2 end of the rTPC (2.3m) and the lack of accessibility to the bottom of the detector. The proposal is to use a Copper sheet mounted on the inner surface of the inner cylinder. This ground path will connect the 2 endplates to the High Voltage return. All the pre-amps will merge their ground to this central ground.

Additional ground lugs have been included in the Anode wire pre-amplifier boards to strengthen the ground as each anode wire is readout out from both end. This may be used once the full detector is assembled to address possible ground loop issues.

[[File:imageservice19.png|439x409px]]

= Safety and Hazard considerations =

== Gas ==

The Ar and CO2 do not require special health safety system. The detector will be operated in the ventilated hall of the AD building at CERN.

Gas flow, detector gas pressure, and gas inlet temperature are constantly monitored and can trigger adequate response for shutdown or closure of the gas system for safety purpose.

== Cooling water ==

The water cooling system operates in negative pressure mode preventing any water leak to flood the detector outer surface. In addition water flow and inlet/outlet temperature are constantly monitored to allow fast response to off range parameter in order to shut down the electronics. The cooling water system requires a primary cooling tap water loop from the AD building. Water quality, water flow and water temperature are not under our responsibility. This aspect will be dealt during the initial phase of the rTPC commissioning at CERN.

== Power distribution ==

All electronics voltages are below 48V. No special safety circuits are envisaged.

The high-voltage circuit is fully protected outside the detector. The access to the high-voltage on the detector is restricted to the two end-plates. Interlock scheme will be put in place to ensure the shutdown of the HV in case of manual access to the detector itself.

The LV (Low Voltage) power units are also interlocked to prevent false powering sequence which can damage the downstream electronics modules.

[[File:imageservice20.png|571x411px]]

Figure 16- Alpha-g rTPC rack organization

Detector Services

2018-12-07T13:18:28Z

Acapra: /* Drain system */

Back to [[Main Page]]

= Table of Figures =

[[#_Toc493684884|Figure 1 - Gas system schematic 6]]

[[#_Toc493684885|Figure 2 - A half-rack space is used for the gas control implementation. 7]]

[[#_Toc493684886|Figure 3 - Gas system half rack ready for installation 8]]

[[#_Toc493684887|Figure 4 - Cathode pad half-cylinder. Pairs of cooling pipes are visible 9]]

[[#_Toc493684888|Figure 5 - Water cooling system during test. 9]]

[[#_Toc493684889|Figure 6 - Model and 3D-printed realization of the dry-air manifold, very similar to the manifolds for the water cooling system. 12]]

[[#_Toc493684890|Figure 7 - Overall water cooling system 13]]

[[#_Toc493684891|Figure 8 - Temperature Monitor Board 14]]

[[#_Toc493684892|Figure 9 - End plate view of the layout of the different electronics boards & HV connections 15]]

[[#_Toc493684893|Figure 10 – Unpotted Anode Wire Card and Board with HV distribution 17]]

[[#_Toc493684894|Figure 11 - Anode wire pins and Field wire pins under yellow cover 17]]

[[#_Toc493684895|Figure 12 – New Anode Wire card with 16 potted HV capacitors, gold crimp pin Mill-max socket to the anode wire 18]]

[[#_Toc493684896|Figure 13 - Anode wire pre-amp power distribution & 16 channel temperature boards 19]]

[[#_Toc493684897|Figure 14 - Overall Low voltage distribution for anode wires and cathode pads 21]]

[[#_Toc493684898|Figure 15 - High Voltage & Grounding scheme for the rTPC 22]]

[[#_Toc493684899|Figure 16- Alpha-g rTPC rack organization 23]]

= Introduction =



This document is to describe the different services for the rTPC detector. 3 main sections cover the ''Gas system'', the ''Water cooling system'' and the ''Power distribution''.

= References and related Documents =

A dedicated TRIUMF Wiki site gathers all documentations for the rTPC detector system and equipment. Gas system, cooling water system and power distribution can be found there.

= Gas system =

The gas system as described below includes the entire gas infrastructure required to deliver gas to the detector, monitor and control the gas flow into the detector. It also includes the necessary equipment to notify any change from the nominal gas flow or gas mixture ratio delivered to the detector.

This type of mixed gas system is well understood by the Triumf detector group staff and no issue in its realization is expected.

This system is a stand-alone infrastructure assembled and delivered to CERN. It will be placed in the Alpha-g experimental area floor. The delivery of the different gases to the experimental area is under the responsibility of the Gas handling group from CERN.

The gas delivery system for the radial TPC (rTPC) is to provide Ar/CO2 gas mixture between 90/10 to 50/50 depending on the user requirements. This system does not recycle the gas as the environmental impact of either gas is minimal and the operation cost versus a recycling system would not be economic.

In the event of an additional quenching gas requirement, the overall gas system will have to be reconsidered and possible recycling system envisaged.

A simple gas mixer using digital flow meters will provide the precise mixture dialed by hand in the control system.

During the prototype phase, an ad-hoc gas system will be put in place. A portable gas system is available for such a test.

== Specifications ==

{|
! '''''Parameter'''''
! '''''Value'''''
|-
| Gas mixture
| Ar/CO2 (90/10..50/50)
|-
| Gas flow requirements
| 0.3l/min (50%), 0.3l/min (50%)
|-
| Detector gas Volume
| 0.18m3
|-
| Flushing time
| 5h @ 600cc/min
|-
| Flow rate
| 100 to 300 cc/minute
|}

== Operation ==

This gas system is quite simple as it takes 2 or 3 (spare) gas supplies and mixes them in a manifold before delivery to the rTPC detector. Necessary valves for purging lines and by-passing the detector are foreseen. The individual in-flows are controlled by gas dedicated mass flow controllers (MFC-x). The return flow is measured for leak assessment. An additional manometer monitors the internal detector pressure for evaluation of the electron drift characteristics.

[[File:imageservice2.png|575x315px|PA_doc-1.PNG]]

Figure 1 - Gas system schematic

The main flow controller and monitoring is performed by industrial devices from MKS. We added analog flowmeters on each gas line to provide a quick visual inspection of the gas mixing system.

This system will be hooked to the CERN AD-Gas facility. The CERN gas delivers gas through 2 switchable packs of 12 bottles for each gas.

The gas exhaust is evacuated to the exterior of the experimental area with an elevation of about 8m.

[[File:imageservice3.png|366x263px|PA_doc-2.PNG]]

Figure 2 - A half-rack space is used for the gas control implementation.



[[File:imageservice4.png|324x364px]]

Figure 3 - Gas system half rack ready for installation

== Quick Start ==
From-scratch (Neither gas supply lines to the GHS nor connections to and from the TPC connected) connection of the GHS should follow the following procedure:
# '''Flush supply lines:''' Gas lines from outside bottles going to the GHS should be briefly (5 min or so) flushed with a high gas flow ''before'' connecting to the GHS to remove possible dust, and to minimize the amount of air in the lines.
# '''Flush GHS:'''
## Close BV3 and BV5, open BV2 and BV4 to flush input lines, only briefly.
## Open BV3 and BV5, close BV2 and BV4 (normal operation configuration)
## With Midas frontend running, use the "Gas" custom page to set a desired flow, for flushing probably between 1l/min and 1.5l/min (Without Midas, use the telnet interface), then press the "Bypass Flow" button, and let flow for a while (5-15min?)
# '''Flush TPC input line:'''
## Connect the line that will go from the GHS to the TPC input-tee on the GHS side, but leave disconnected from the tee
## do the same for the pressure gauge line
## On the "Gas" custom page, press the "Flow TPC" button and let flow for 5 min (Without Midas, use the telnet interface).
# Make all remaining connections

== Telnet Interface ==
If the Midas frontend is not available the GHS can be controlled via telnet. Connect with
telnet algas
then use the following command to control solenoid valves:
cowr do i 1
to energize SVi, i.e. to switch it from its normal closed or open state to the other state,
cowr do i 0
to de-energize, i.e. switch to default state.

Gas flow is set with:
mfcwr ao i <desired flow>
where i is 1 for Ar and 2 for CO2, and the desired flow is in the internal units of the system.

= Water cooling system =

The outer surface of the detector has an arrangement of 8x8 electronics boards handling the readout of the cathode pads. These boards require operating at a controlled temperature to prevent heat build-up at the surface of the detector and improving the reliability of the system. In order to do so, 8 independent water pipe loops running from the bottom-up and down in the longitudinal axis of the detector are in a thermal contact to 8 successive readout boards. From the bottom, a pair of water hoses connect to 2 custom 3D-printed manifolds supplying the water feed and return to the 8 water loops.

These 8 loops are run in parallel to a negative pressure water cooling system. This negative pressure system is to ensure that no water leak will damage the electronics even during possible pipe or delivery hose breakage.

[[File:imageservice5.png|516x273px]]

Figure 4 - Cathode pad half-cylinder. Pairs of cooling pipes are visible

[[File:imageservice6.jpg|249x429px]]

Figure 5 - Water cooling system during test.

== Specifications ==

{|
! '''''Parameters'''''
! '''''Values'''''
|-
| Number of parallel water loops
| 8
|-
| Copper pipe OD/ID
| ¼” / 4.7mm
|-
| Length of one loop (on the rTPC)
| 4.6m
|-
| Water speed per loop
| …
|-
| Pressure drop per loop
| …
|-
| Power extraction per water loop
| 8x15W= 120W
|-
| Total cooling power
| 1KW
|-
| Inlet water temperature
| 16..20 degC (to be confirmed)
|-
| Outlet water temperature
| 20..24 degC
|-
| Total water flow
| 3..6 l/minute
|}

== Operation ==

''This description corresponds to how we think the system should work, but doesn't seem fully comatible with the behaviour of the system. E.g. the air pump only seems to turn on when PID1 falls below a certain value.''

The Pressure PID (proportional/integral/differential controller), PID2, monitors the air pressure in the vacuum vessel. When the pressure rises above a trigger value (perhaps 350mbar) it activates the aspirator vacuum pump to bring the pressure down to maintain a long time averaged pressure (say 300mbar).

This vacuum is then used to draw water through the detector into the vessel, from the Open Tank. The vacuum vessel's water level is monitored by PID1, using a differential pressure transducer measuring the weight of the water column by the difference between the air pressure in the top of the vessel and the water pressure at the bottom of the vessel. PID1 provides a variable signal to the Proportional Solenoid Valve, which varies the flow through the return loop around the (fixed flow) water pump, and thus changing the overall flow of water from the vacuum vessel to keep the water level in the vessel constant despite water being constantly drawn in by the vacuum. The water pulled from the vacuum vessel is pushed through the filter and into the Open Tank, completing the circulation.

Cooling is accomplished in the water pump leg of the system, as coolant pulled from the vacuum vessel passes through a heat exchanger coupled to CERN’s cooling water supply on its way to the pump. The rate that water flows through the system can be determined by many obstacles in the loop, but ideally it should be dominated by the position of the main Throttle Valve. Other valves should generally be set either fully open of fully closed, except for the Proportional Valve, whose position responds to the setting of the Throttle Valve.

Data outputs: there are five data outputs planned. The two process variables, the vacuum vessel absolute pressure, and differential water column pressure, will be sent to the Detector DAQ system. As well, the total flow rate is monitored by the flow meter, whose output is sent to the DAQ system. This data is not used by the circulation system itself. Likewise, the back pressure on the water filter is only sent to the DAQ system to indicate if the filter may need changing, and the temperature of the open tank is monitored and sent to the DAQ system, but currently there is no plan to use it in a temperature control feedback. The temperature of CERN's cooling water supply is not yet known, but we expect we can set the heat exchanger to run at a rate to hold the system temperature near 20°C without feedback control.

For the negative pressure to work effectively to prevent leaks, the water pressure in the detector ought to be down at least to half an atmosphere. With the detector cooling tubes 8x 4.6m of parallel 4.7mm ID copper tubes (2.3m down then 2.3 back-ups), and the supply using (perhaps 6m, 3 to the detector then 3 back) 9.7mm ID (=1/2" OD) poly tubing, the flow will need to be somewhat greater than the required 3 l/min in order to maintain that degree of vacuum in the detector, unless the Throttle Valve is placed upstream of the detector near the Open Tank, or an additional constriction is added in that region.

The water pump leg of the circuit residing in the water cooling/gas rack is placed far enough from the fringe field of the superconducting magnet to prevent disruptive effects on the solenoid valves and electric motors. Therefore the main 1/2" poly line to and from the water pump and its flow loop will add a pressure drop which is estimated at 0.4atm for a 10m line at 6 l/min (9.7mmID), which is within our requirements. As mentioned earlier, the splitting (and reassembly) of the line to the 8 cooling lines close to detector will be accomplished with the use of 2 separate manifolds equipped with temperature sensors for constant in/out global water temperature monitoring.

[[File:imageservice7.png|286x206px]][[File:imageservice8.jpg|286x206px]]

Figure 6 - Model and 3D-printed realization of the dry-air manifold, very similar to the manifolds for the water cooling system.

Currently, PVC valves with "true union" couplings are planned to be provided around the water pump to permit it to be swapped out from the system for servicing and a valved bypass to the water filter is provided for the same purpose to allow easily changing filter elements with the system running. True union valves are generally only available to 1/2NPT and larger, so an adapter fitting is required. The vacuum pump for the system is a small (2 x 5 x 7.5cm) cascaded dual Venturi aspirator, which is operated with site air, switched with two simultaneously operated normally closed solenoid valves, one for the site air, and one for the vessel vacuum input. If the system is leak free, the vacuum in the vessel should only be degraded by the dissolved air absorbed in the Open Tank and extracted in the vacuum vessel, which from past experience might mean running for under a minute every six hours or so.

The wetted surfaces of this system will be mainly PVC and Polyethylene for the valves, tubing and vessels, plus brass fittings, and the copper cooling tubes, and the surfaces of the heat exchanger, which will most likely be stainless steel. It may be possible to devise an equivalent of a bellows communicating with the Open Tank, so that it is only virtually open, thus sealing the cooling water from contaminants, which may allow us to run with just distilled water plus a strip of silver as a biocide (although I'm not sure if the mix of metals, copper plus stainless, may not interfere with the function of the silver, which is mostly used in computer cooling systems which are copper and plastic only). Otherwise, we will need a biocide and corrosion inhibitor. We have the latter, tolyltriazole, but the biocide is problematic. They tend to be short-lived; for systems which are open to atmosphere, typical industrial practice is chlorination, regularly refreshed, which builds up NaCl as it degrades, making the coolant corrosive. For T2K in Japan, we used the (human-nontoxic) "Germall", which has a two-year lifetime, after which when its effectiveness decays it turns into food, which is the case for any solute that is not actively biocidal. Eventually, any such additives build up and require the coolant to be flushed and replaced. Silver is a lovely method, but only works for extended periods with a sealed system which remains deoxygenated. My research so far has not turned up a good candidate which does not entail recurring changes of the coolant.

Power: the system needs to be able to run on European 220VAC power, but also operate here for testing purposes. The PIDs (Omega CN76160s) will accept both; for the rest of the system, I will have an a universal 90-240VAC input to 24VDC (5A) supply which will drive all the other components, including the pump, which will be a brushless 24VDC centrifugal pump, 1 - 1.5A range. The whole system will run about 125W.

Information on the temperature and flow rate of the CERN cooling water system will be provided later but we don’t anticipate any major issues about the sizing of the heat exchanger.

[[File:imageservice9.png|603x344px]]

Figure 7 - Overall water cooling system



== Temperature Measurement ==

[[File:imageservice10.jpeg|225x356px]] [[File:imageservice11.png|194x358px]]

Figure 8 Temperature Monitor Board

The water temperature is read by the Temperature Monitor Board (TEMPB). Thermistors located inside the manifold connect to a 16 bit ADC on the TEMPB which is read out by a Raspberry Pi mounted on board. In a similar manner the air system temperature is read with sensors in the air manifold. The TEMPB can provide temperature readout of 32 thermistors which connect to terminal blocks. The channels can also be readout differentially providing up to 16 channels for thermocouples. An on board thermistor allows for cold junction compensation. The board is powered via USB on the Raspberry Pi.

== Quick Start ==
This assumes all water lines and compressed air are connected, and both the blue water tank and the chiller are filled.

=== Begin running ===
# Close vent valve and main valve (3-way valve in T position)
# Switch on control box with power switch in the back
# confirm water pump is off, i.e. blue button is not illuminated
# Vacuum pump should now be running
# Open main valve, i.e. vertical connection open, connection to vent valve closed [[File:PWBmainOpen.jpg|thumb|Cooling system main valve open]]
# water should start flowing through the PWB cooling system and coming back through the clear pipe connected to the vacuum vessel
# Default settings are PID1 = 200, PID2 = 300, but may have been adjusted
# once the water level in the vacuum vessel has risen to ~20cm, turn on the water pump

=== Drain system ===
# from normal operation, i.e. vacuum pump and water pump running
# switch main valve to upside down T position
[[File:PWBmainVent.jpg|thumb]]
# open vent valve
# air should now enter the system, draining all water from the TPC pipes into the vacuum vessel
# water pump will drain the vacuum vessel into supply tank, turn it off before the vessel is completely empty to avoid air in the water pump
# switch off main power

== Troubleshooting ==
The water system is temperamental, and not intuitive at all. Common problems and solution attempts:

; water pump very loud, doesn't appear to pump water
: most likely air in the water pump. Open the vent of the vacuum vessel to allow water to enter pump by gravity alone, pump should stop screaming and start pumping. Close vent.
; vacuum pump comes on often (more then every 10-20 min)
: probably an air leak in the system. Check for bubbles and try to find leak.
; water level/flow/vacuum oscillate between extreme values
: unclear, try playing with PID settings, throttle valve, possibly orange ball valves by the water pump.

= Power distribution =

This section covers the different voltages and power requirements for the operation of the rTPC. This includes:

# High Voltage for the rTPC electron drift and amplification operation
# Low Voltage to the Anode wire frontend electronics
# Low Voltage to the Barrel Scintillator frontend electronics
# Low Voltage to the Cathode Pads electronics
# Grounding scheme of the overall rTPC

The provision of power to the different components of the detector is distributed from the top end plate of the rTPC.

[[File:imageservice12.png|507x354px]]

Figure 9 - End plate view of the layout of the different electronics boards & HV connections

== High Voltage for the rTPC ==

Three main components of the rTPC require High Voltage

* Main cathode wall (Negative ~5KV)
* Field wires (Negative ~ few hundred volts)
* Anode wires (Positive ~3.2KV )

These voltages are provided by a Quad High Voltage unit from CAEN. Three independent cables will be fed to the top of the rTPC.

High Voltage provision to the bottom endplate HV ring is taken from the central cathode near the bottom endplate.

The Field wire voltage is connected to all the field wire crimp-pins through a dedicated socket chain inserted on each of the crimp-pin on the top and bottom endplates. The field wires themselves interconnect the high voltage between the top and bottom pins.

The High Voltage for the anode wires is delivered to the Anode Wire Card (AWC) which will distribute it to its 16 wires, Figure 11. There are 16 AWC cards covering each end of the anode wires. All the top and bottom 32 AWC are connected serially. The anode wires themselves interconnect the high voltage between the top and bottom pins.

The anode wires pre-amplifiers boards (AWB) are sitting on the top of the Anode Wire Card (AWC) Figure 10. As the wire is powered by the HV, each anode wire has a high Voltage decoupling capacitor to isolate the pre-amp stage from the HV. This circuit resides on the AWC. The AWB will see the AC coupled anode signal only. With a new design of the AWC Figure 12, the decoupling capacitors are potted in epoxy to ensure proper isolation of the HV toward the surrounding circuit as well to prevent corona discharge due to relative humidity of the surrounding.

[[File:imageservice13.jpg|402x309px]]

Figure 10 – Unpotted Anode Wire Card and Board with HV distribution

[[File:imageservice14.jpg|402x309px]]

Figure 11 - Anode wire pins and Field wire pins under yellow cover

[[File:imageservice15.jpg|402x309px]]

Figure 12 – New Anode Wire card with 16 potted HV capacitors, gold crimp pin Mill-max socket to the anode wire

== Power distribution to the Anode wire frontend electronics ==

The AWB pre-amp board requires 2 voltages (+5V, -5V) which are fed through dedicated cable connections to each of the AWB. There are 16 AWB per end of the rTPC, therefore, 16 individual cable bundle will be reaching the top and another 16 the bottom of the rTPC. The cable bundle is to also provide the power return lines to the Power Supply.

The PL508 Low Voltage power Supply from Wiener will provide the main power source for all the 32 pre-amplifiers. A dedicated connection breakout board with current monitoring capability splits the source power to the 2 x 16 necessary connections to the Anode wire boards. Based on a RaspeberryPi card, a 16 power distribution board is already implemented for the prototype. This board has extra 4 channels of temperature monitoring which can be used later on. In the same process we produce a similar board for 16 channel of NTC, RTC temperature monitoring. They will be used for overall detector and cooling system monitoring.

{|
! '''''Component'''''
! '''''+5V Supply'''''
! '''''-5V Supply'''''
|-
| Anode Pre-amp
| 1.5W (0.3A)
| 1.5W (0.3A)
|}

[[File:imageservice16.jpg|239x363px]] [[File:imageservice17.png|239x363px]]

Figure 13 - Anode wire pre-amp power distribution & 16 channel temperature boards

== Power distribution to the Barrel Scintillator frontend electronics ==

Similar to the Anode wire pre-amplifier voltage distribution scheme, the Barrel Scintillators will require several low voltages for power and control. The Barrel Scintillator Boards (BSB) are mounted on both ends of the Scintillator bar with the same segmentation as the Anode wires i.e.: 16 BSB per end. The design of the electronic chain for the barrel scintillator is not fully designed yet.

The SiPM sensors require a bias voltage of the order of 28V. This voltage will be provided through the same mean as the low voltage power. Expected threshold voltage for the local signal level discriminator is also foreseen.

== Low Voltage to the Cathode Pads electronics ==

The cathode pads electronics mounted on the external cylinder surface is composed of the analog frontend and the digital conversion electronics. Two voltages are required (+5.3V, +2.3V). The delivery is done through dedicated “power bar” running the full length of the rTPC. A set of 3 bars covers the 8 axial cathode pad electronics boards. 8 of them cover the circumference of the rTPC.

{|
! '''''Component'''''
! '''''+5.3V Supply'''''
! '''''+2.3V Supply'''''
|-
| Padwing board
| 10W (1.8A)
| 2W (0.8A)
|}

Overall the low power supplies are provided by a PL508-3U with 8 modules of either single or dual channels from Wiener.

[[File:imageservice18.png|469x360px]]

Figure 14 - Overall Low voltage distribution for anode wires and cathode pads



== Grounding scheme of the overall rTPC ==

The grounding scheme challenge in this setup is the far distance between the 2 end of the rTPC (2.3m) and the lack of accessibility to the bottom of the detector. The proposal is to use a Copper sheet mounted on the inner surface of the inner cylinder. This ground path will connect the 2 endplates to the High Voltage return. All the pre-amps will merge their ground to this central ground.

Additional ground lugs have been included in the Anode wire pre-amplifier boards to strengthen the ground as each anode wire is readout out from both end. This may be used once the full detector is assembled to address possible ground loop issues.

[[File:imageservice19.png|439x409px]]

= Safety and Hazard considerations =

== Gas ==

The Ar and CO2 do not require special health safety system. The detector will be operated in the ventilated hall of the AD building at CERN.

Gas flow, detector gas pressure, and gas inlet temperature are constantly monitored and can trigger adequate response for shutdown or closure of the gas system for safety purpose.

== Cooling water ==

The water cooling system operates in negative pressure mode preventing any water leak to flood the detector outer surface. In addition water flow and inlet/outlet temperature are constantly monitored to allow fast response to off range parameter in order to shut down the electronics. The cooling water system requires a primary cooling tap water loop from the AD building. Water quality, water flow and water temperature are not under our responsibility. This aspect will be dealt during the initial phase of the rTPC commissioning at CERN.

== Power distribution ==

All electronics voltages are below 48V. No special safety circuits are envisaged.

The high-voltage circuit is fully protected outside the detector. The access to the high-voltage on the detector is restricted to the two end-plates. Interlock scheme will be put in place to ensure the shutdown of the HV in case of manual access to the detector itself.

The LV (Low Voltage) power units are also interlocked to prevent false powering sequence which can damage the downstream electronics modules.

[[File:imageservice20.png|571x411px]]

Figure 16- Alpha-g rTPC rack organization

File:PWBmainVent.jpg

2018-12-07T13:17:53Z

Acapra:

rTPC services rack, cooling system, main valve vent

Detector Services

2018-12-07T13:15:45Z

Acapra: /* Begin running */

Back to [[Main Page]]

= Table of Figures =

[[#_Toc493684884|Figure 1 - Gas system schematic 6]]

[[#_Toc493684885|Figure 2 - A half-rack space is used for the gas control implementation. 7]]

[[#_Toc493684886|Figure 3 - Gas system half rack ready for installation 8]]

[[#_Toc493684887|Figure 4 - Cathode pad half-cylinder. Pairs of cooling pipes are visible 9]]

[[#_Toc493684888|Figure 5 - Water cooling system during test. 9]]

[[#_Toc493684889|Figure 6 - Model and 3D-printed realization of the dry-air manifold, very similar to the manifolds for the water cooling system. 12]]

[[#_Toc493684890|Figure 7 - Overall water cooling system 13]]

[[#_Toc493684891|Figure 8 - Temperature Monitor Board 14]]

[[#_Toc493684892|Figure 9 - End plate view of the layout of the different electronics boards & HV connections 15]]

[[#_Toc493684893|Figure 10 – Unpotted Anode Wire Card and Board with HV distribution 17]]

[[#_Toc493684894|Figure 11 - Anode wire pins and Field wire pins under yellow cover 17]]

[[#_Toc493684895|Figure 12 – New Anode Wire card with 16 potted HV capacitors, gold crimp pin Mill-max socket to the anode wire 18]]

[[#_Toc493684896|Figure 13 - Anode wire pre-amp power distribution & 16 channel temperature boards 19]]

[[#_Toc493684897|Figure 14 - Overall Low voltage distribution for anode wires and cathode pads 21]]

[[#_Toc493684898|Figure 15 - High Voltage & Grounding scheme for the rTPC 22]]

[[#_Toc493684899|Figure 16- Alpha-g rTPC rack organization 23]]

= Introduction =



This document is to describe the different services for the rTPC detector. 3 main sections cover the ''Gas system'', the ''Water cooling system'' and the ''Power distribution''.

= References and related Documents =

A dedicated TRIUMF Wiki site gathers all documentations for the rTPC detector system and equipment. Gas system, cooling water system and power distribution can be found there.

= Gas system =

The gas system as described below includes the entire gas infrastructure required to deliver gas to the detector, monitor and control the gas flow into the detector. It also includes the necessary equipment to notify any change from the nominal gas flow or gas mixture ratio delivered to the detector.

This type of mixed gas system is well understood by the Triumf detector group staff and no issue in its realization is expected.

This system is a stand-alone infrastructure assembled and delivered to CERN. It will be placed in the Alpha-g experimental area floor. The delivery of the different gases to the experimental area is under the responsibility of the Gas handling group from CERN.

The gas delivery system for the radial TPC (rTPC) is to provide Ar/CO2 gas mixture between 90/10 to 50/50 depending on the user requirements. This system does not recycle the gas as the environmental impact of either gas is minimal and the operation cost versus a recycling system would not be economic.

In the event of an additional quenching gas requirement, the overall gas system will have to be reconsidered and possible recycling system envisaged.

A simple gas mixer using digital flow meters will provide the precise mixture dialed by hand in the control system.

During the prototype phase, an ad-hoc gas system will be put in place. A portable gas system is available for such a test.

== Specifications ==

{|
! '''''Parameter'''''
! '''''Value'''''
|-
| Gas mixture
| Ar/CO2 (90/10..50/50)
|-
| Gas flow requirements
| 0.3l/min (50%), 0.3l/min (50%)
|-
| Detector gas Volume
| 0.18m3
|-
| Flushing time
| 5h @ 600cc/min
|-
| Flow rate
| 100 to 300 cc/minute
|}

== Operation ==

This gas system is quite simple as it takes 2 or 3 (spare) gas supplies and mixes them in a manifold before delivery to the rTPC detector. Necessary valves for purging lines and by-passing the detector are foreseen. The individual in-flows are controlled by gas dedicated mass flow controllers (MFC-x). The return flow is measured for leak assessment. An additional manometer monitors the internal detector pressure for evaluation of the electron drift characteristics.

[[File:imageservice2.png|575x315px|PA_doc-1.PNG]]

Figure 1 - Gas system schematic

The main flow controller and monitoring is performed by industrial devices from MKS. We added analog flowmeters on each gas line to provide a quick visual inspection of the gas mixing system.

This system will be hooked to the CERN AD-Gas facility. The CERN gas delivers gas through 2 switchable packs of 12 bottles for each gas.

The gas exhaust is evacuated to the exterior of the experimental area with an elevation of about 8m.

[[File:imageservice3.png|366x263px|PA_doc-2.PNG]]

Figure 2 - A half-rack space is used for the gas control implementation.



[[File:imageservice4.png|324x364px]]

Figure 3 - Gas system half rack ready for installation

== Quick Start ==
From-scratch (Neither gas supply lines to the GHS nor connections to and from the TPC connected) connection of the GHS should follow the following procedure:
# '''Flush supply lines:''' Gas lines from outside bottles going to the GHS should be briefly (5 min or so) flushed with a high gas flow ''before'' connecting to the GHS to remove possible dust, and to minimize the amount of air in the lines.
# '''Flush GHS:'''
## Close BV3 and BV5, open BV2 and BV4 to flush input lines, only briefly.
## Open BV3 and BV5, close BV2 and BV4 (normal operation configuration)
## With Midas frontend running, use the "Gas" custom page to set a desired flow, for flushing probably between 1l/min and 1.5l/min (Without Midas, use the telnet interface), then press the "Bypass Flow" button, and let flow for a while (5-15min?)
# '''Flush TPC input line:'''
## Connect the line that will go from the GHS to the TPC input-tee on the GHS side, but leave disconnected from the tee
## do the same for the pressure gauge line
## On the "Gas" custom page, press the "Flow TPC" button and let flow for 5 min (Without Midas, use the telnet interface).
# Make all remaining connections

== Telnet Interface ==
If the Midas frontend is not available the GHS can be controlled via telnet. Connect with
telnet algas
then use the following command to control solenoid valves:
cowr do i 1
to energize SVi, i.e. to switch it from its normal closed or open state to the other state,
cowr do i 0
to de-energize, i.e. switch to default state.

Gas flow is set with:
mfcwr ao i <desired flow>
where i is 1 for Ar and 2 for CO2, and the desired flow is in the internal units of the system.

= Water cooling system =

The outer surface of the detector has an arrangement of 8x8 electronics boards handling the readout of the cathode pads. These boards require operating at a controlled temperature to prevent heat build-up at the surface of the detector and improving the reliability of the system. In order to do so, 8 independent water pipe loops running from the bottom-up and down in the longitudinal axis of the detector are in a thermal contact to 8 successive readout boards. From the bottom, a pair of water hoses connect to 2 custom 3D-printed manifolds supplying the water feed and return to the 8 water loops.

These 8 loops are run in parallel to a negative pressure water cooling system. This negative pressure system is to ensure that no water leak will damage the electronics even during possible pipe or delivery hose breakage.

[[File:imageservice5.png|516x273px]]

Figure 4 - Cathode pad half-cylinder. Pairs of cooling pipes are visible

[[File:imageservice6.jpg|249x429px]]

Figure 5 - Water cooling system during test.

== Specifications ==

{|
! '''''Parameters'''''
! '''''Values'''''
|-
| Number of parallel water loops
| 8
|-
| Copper pipe OD/ID
| ¼” / 4.7mm
|-
| Length of one loop (on the rTPC)
| 4.6m
|-
| Water speed per loop
| …
|-
| Pressure drop per loop
| …
|-
| Power extraction per water loop
| 8x15W= 120W
|-
| Total cooling power
| 1KW
|-
| Inlet water temperature
| 16..20 degC (to be confirmed)
|-
| Outlet water temperature
| 20..24 degC
|-
| Total water flow
| 3..6 l/minute
|}

== Operation ==

''This description corresponds to how we think the system should work, but doesn't seem fully comatible with the behaviour of the system. E.g. the air pump only seems to turn on when PID1 falls below a certain value.''

The Pressure PID (proportional/integral/differential controller), PID2, monitors the air pressure in the vacuum vessel. When the pressure rises above a trigger value (perhaps 350mbar) it activates the aspirator vacuum pump to bring the pressure down to maintain a long time averaged pressure (say 300mbar).

This vacuum is then used to draw water through the detector into the vessel, from the Open Tank. The vacuum vessel's water level is monitored by PID1, using a differential pressure transducer measuring the weight of the water column by the difference between the air pressure in the top of the vessel and the water pressure at the bottom of the vessel. PID1 provides a variable signal to the Proportional Solenoid Valve, which varies the flow through the return loop around the (fixed flow) water pump, and thus changing the overall flow of water from the vacuum vessel to keep the water level in the vessel constant despite water being constantly drawn in by the vacuum. The water pulled from the vacuum vessel is pushed through the filter and into the Open Tank, completing the circulation.

Cooling is accomplished in the water pump leg of the system, as coolant pulled from the vacuum vessel passes through a heat exchanger coupled to CERN’s cooling water supply on its way to the pump. The rate that water flows through the system can be determined by many obstacles in the loop, but ideally it should be dominated by the position of the main Throttle Valve. Other valves should generally be set either fully open of fully closed, except for the Proportional Valve, whose position responds to the setting of the Throttle Valve.

Data outputs: there are five data outputs planned. The two process variables, the vacuum vessel absolute pressure, and differential water column pressure, will be sent to the Detector DAQ system. As well, the total flow rate is monitored by the flow meter, whose output is sent to the DAQ system. This data is not used by the circulation system itself. Likewise, the back pressure on the water filter is only sent to the DAQ system to indicate if the filter may need changing, and the temperature of the open tank is monitored and sent to the DAQ system, but currently there is no plan to use it in a temperature control feedback. The temperature of CERN's cooling water supply is not yet known, but we expect we can set the heat exchanger to run at a rate to hold the system temperature near 20°C without feedback control.

For the negative pressure to work effectively to prevent leaks, the water pressure in the detector ought to be down at least to half an atmosphere. With the detector cooling tubes 8x 4.6m of parallel 4.7mm ID copper tubes (2.3m down then 2.3 back-ups), and the supply using (perhaps 6m, 3 to the detector then 3 back) 9.7mm ID (=1/2" OD) poly tubing, the flow will need to be somewhat greater than the required 3 l/min in order to maintain that degree of vacuum in the detector, unless the Throttle Valve is placed upstream of the detector near the Open Tank, or an additional constriction is added in that region.

The water pump leg of the circuit residing in the water cooling/gas rack is placed far enough from the fringe field of the superconducting magnet to prevent disruptive effects on the solenoid valves and electric motors. Therefore the main 1/2" poly line to and from the water pump and its flow loop will add a pressure drop which is estimated at 0.4atm for a 10m line at 6 l/min (9.7mmID), which is within our requirements. As mentioned earlier, the splitting (and reassembly) of the line to the 8 cooling lines close to detector will be accomplished with the use of 2 separate manifolds equipped with temperature sensors for constant in/out global water temperature monitoring.

[[File:imageservice7.png|286x206px]][[File:imageservice8.jpg|286x206px]]

Figure 6 - Model and 3D-printed realization of the dry-air manifold, very similar to the manifolds for the water cooling system.

Currently, PVC valves with "true union" couplings are planned to be provided around the water pump to permit it to be swapped out from the system for servicing and a valved bypass to the water filter is provided for the same purpose to allow easily changing filter elements with the system running. True union valves are generally only available to 1/2NPT and larger, so an adapter fitting is required. The vacuum pump for the system is a small (2 x 5 x 7.5cm) cascaded dual Venturi aspirator, which is operated with site air, switched with two simultaneously operated normally closed solenoid valves, one for the site air, and one for the vessel vacuum input. If the system is leak free, the vacuum in the vessel should only be degraded by the dissolved air absorbed in the Open Tank and extracted in the vacuum vessel, which from past experience might mean running for under a minute every six hours or so.

The wetted surfaces of this system will be mainly PVC and Polyethylene for the valves, tubing and vessels, plus brass fittings, and the copper cooling tubes, and the surfaces of the heat exchanger, which will most likely be stainless steel. It may be possible to devise an equivalent of a bellows communicating with the Open Tank, so that it is only virtually open, thus sealing the cooling water from contaminants, which may allow us to run with just distilled water plus a strip of silver as a biocide (although I'm not sure if the mix of metals, copper plus stainless, may not interfere with the function of the silver, which is mostly used in computer cooling systems which are copper and plastic only). Otherwise, we will need a biocide and corrosion inhibitor. We have the latter, tolyltriazole, but the biocide is problematic. They tend to be short-lived; for systems which are open to atmosphere, typical industrial practice is chlorination, regularly refreshed, which builds up NaCl as it degrades, making the coolant corrosive. For T2K in Japan, we used the (human-nontoxic) "Germall", which has a two-year lifetime, after which when its effectiveness decays it turns into food, which is the case for any solute that is not actively biocidal. Eventually, any such additives build up and require the coolant to be flushed and replaced. Silver is a lovely method, but only works for extended periods with a sealed system which remains deoxygenated. My research so far has not turned up a good candidate which does not entail recurring changes of the coolant.

Power: the system needs to be able to run on European 220VAC power, but also operate here for testing purposes. The PIDs (Omega CN76160s) will accept both; for the rest of the system, I will have an a universal 90-240VAC input to 24VDC (5A) supply which will drive all the other components, including the pump, which will be a brushless 24VDC centrifugal pump, 1 - 1.5A range. The whole system will run about 125W.

Information on the temperature and flow rate of the CERN cooling water system will be provided later but we don’t anticipate any major issues about the sizing of the heat exchanger.

[[File:imageservice9.png|603x344px]]

Figure 7 - Overall water cooling system



== Temperature Measurement ==

[[File:imageservice10.jpeg|225x356px]] [[File:imageservice11.png|194x358px]]

Figure 8 Temperature Monitor Board

The water temperature is read by the Temperature Monitor Board (TEMPB). Thermistors located inside the manifold connect to a 16 bit ADC on the TEMPB which is read out by a Raspberry Pi mounted on board. In a similar manner the air system temperature is read with sensors in the air manifold. The TEMPB can provide temperature readout of 32 thermistors which connect to terminal blocks. The channels can also be readout differentially providing up to 16 channels for thermocouples. An on board thermistor allows for cold junction compensation. The board is powered via USB on the Raspberry Pi.

== Quick Start ==
This assumes all water lines and compressed air are connected, and both the blue water tank and the chiller are filled.

=== Begin running ===
# Close vent valve and main valve (3-way valve in T position)
# Switch on control box with power switch in the back
# confirm water pump is off, i.e. blue button is not illuminated
# Vacuum pump should now be running
# Open main valve, i.e. vertical connection open, connection to vent valve closed [[File:PWBmainOpen.jpg|thumb|Cooling system main valve open]]
# water should start flowing through the PWB cooling system and coming back through the clear pipe connected to the vacuum vessel
# Default settings are PID1 = 200, PID2 = 300, but may have been adjusted
# once the water level in the vacuum vessel has risen to ~20cm, turn on the water pump

=== Drain system ===
# from normal operation, i.e. vacuum pump and water pump running
# switch main valve to upside down T position
# open vent valve
# air should now enter the system, draining all water from the TPC pipes into the vacuum vessel
# water pump will drain the vacuum vessel into supply tank, turn it off before the vessel is completely empty to avoid air in the water pump
# switch off main power

== Troubleshooting ==
The water system is temperamental, and not intuitive at all. Common problems and solution attempts:

; water pump very loud, doesn't appear to pump water
: most likely air in the water pump. Open the vent of the vacuum vessel to allow water to enter pump by gravity alone, pump should stop screaming and start pumping. Close vent.
; vacuum pump comes on often (more then every 10-20 min)
: probably an air leak in the system. Check for bubbles and try to find leak.
; water level/flow/vacuum oscillate between extreme values
: unclear, try playing with PID settings, throttle valve, possibly orange ball valves by the water pump.

= Power distribution =

This section covers the different voltages and power requirements for the operation of the rTPC. This includes:

# High Voltage for the rTPC electron drift and amplification operation
# Low Voltage to the Anode wire frontend electronics
# Low Voltage to the Barrel Scintillator frontend electronics
# Low Voltage to the Cathode Pads electronics
# Grounding scheme of the overall rTPC

The provision of power to the different components of the detector is distributed from the top end plate of the rTPC.

[[File:imageservice12.png|507x354px]]

Figure 9 - End plate view of the layout of the different electronics boards & HV connections

== High Voltage for the rTPC ==

Three main components of the rTPC require High Voltage

* Main cathode wall (Negative ~5KV)
* Field wires (Negative ~ few hundred volts)
* Anode wires (Positive ~3.2KV )

These voltages are provided by a Quad High Voltage unit from CAEN. Three independent cables will be fed to the top of the rTPC.

High Voltage provision to the bottom endplate HV ring is taken from the central cathode near the bottom endplate.

The Field wire voltage is connected to all the field wire crimp-pins through a dedicated socket chain inserted on each of the crimp-pin on the top and bottom endplates. The field wires themselves interconnect the high voltage between the top and bottom pins.

The High Voltage for the anode wires is delivered to the Anode Wire Card (AWC) which will distribute it to its 16 wires, Figure 11. There are 16 AWC cards covering each end of the anode wires. All the top and bottom 32 AWC are connected serially. The anode wires themselves interconnect the high voltage between the top and bottom pins.

The anode wires pre-amplifiers boards (AWB) are sitting on the top of the Anode Wire Card (AWC) Figure 10. As the wire is powered by the HV, each anode wire has a high Voltage decoupling capacitor to isolate the pre-amp stage from the HV. This circuit resides on the AWC. The AWB will see the AC coupled anode signal only. With a new design of the AWC Figure 12, the decoupling capacitors are potted in epoxy to ensure proper isolation of the HV toward the surrounding circuit as well to prevent corona discharge due to relative humidity of the surrounding.

[[File:imageservice13.jpg|402x309px]]

Figure 10 – Unpotted Anode Wire Card and Board with HV distribution

[[File:imageservice14.jpg|402x309px]]

Figure 11 - Anode wire pins and Field wire pins under yellow cover

[[File:imageservice15.jpg|402x309px]]

Figure 12 – New Anode Wire card with 16 potted HV capacitors, gold crimp pin Mill-max socket to the anode wire

== Power distribution to the Anode wire frontend electronics ==

The AWB pre-amp board requires 2 voltages (+5V, -5V) which are fed through dedicated cable connections to each of the AWB. There are 16 AWB per end of the rTPC, therefore, 16 individual cable bundle will be reaching the top and another 16 the bottom of the rTPC. The cable bundle is to also provide the power return lines to the Power Supply.

The PL508 Low Voltage power Supply from Wiener will provide the main power source for all the 32 pre-amplifiers. A dedicated connection breakout board with current monitoring capability splits the source power to the 2 x 16 necessary connections to the Anode wire boards. Based on a RaspeberryPi card, a 16 power distribution board is already implemented for the prototype. This board has extra 4 channels of temperature monitoring which can be used later on. In the same process we produce a similar board for 16 channel of NTC, RTC temperature monitoring. They will be used for overall detector and cooling system monitoring.

{|
! '''''Component'''''
! '''''+5V Supply'''''
! '''''-5V Supply'''''
|-
| Anode Pre-amp
| 1.5W (0.3A)
| 1.5W (0.3A)
|}

[[File:imageservice16.jpg|239x363px]] [[File:imageservice17.png|239x363px]]

Figure 13 - Anode wire pre-amp power distribution & 16 channel temperature boards

== Power distribution to the Barrel Scintillator frontend electronics ==

Similar to the Anode wire pre-amplifier voltage distribution scheme, the Barrel Scintillators will require several low voltages for power and control. The Barrel Scintillator Boards (BSB) are mounted on both ends of the Scintillator bar with the same segmentation as the Anode wires i.e.: 16 BSB per end. The design of the electronic chain for the barrel scintillator is not fully designed yet.

The SiPM sensors require a bias voltage of the order of 28V. This voltage will be provided through the same mean as the low voltage power. Expected threshold voltage for the local signal level discriminator is also foreseen.

== Low Voltage to the Cathode Pads electronics ==

The cathode pads electronics mounted on the external cylinder surface is composed of the analog frontend and the digital conversion electronics. Two voltages are required (+5.3V, +2.3V). The delivery is done through dedicated “power bar” running the full length of the rTPC. A set of 3 bars covers the 8 axial cathode pad electronics boards. 8 of them cover the circumference of the rTPC.

{|
! '''''Component'''''
! '''''+5.3V Supply'''''
! '''''+2.3V Supply'''''
|-
| Padwing board
| 10W (1.8A)
| 2W (0.8A)
|}

Overall the low power supplies are provided by a PL508-3U with 8 modules of either single or dual channels from Wiener.

[[File:imageservice18.png|469x360px]]

Figure 14 - Overall Low voltage distribution for anode wires and cathode pads



== Grounding scheme of the overall rTPC ==

The grounding scheme challenge in this setup is the far distance between the 2 end of the rTPC (2.3m) and the lack of accessibility to the bottom of the detector. The proposal is to use a Copper sheet mounted on the inner surface of the inner cylinder. This ground path will connect the 2 endplates to the High Voltage return. All the pre-amps will merge their ground to this central ground.

Additional ground lugs have been included in the Anode wire pre-amplifier boards to strengthen the ground as each anode wire is readout out from both end. This may be used once the full detector is assembled to address possible ground loop issues.

[[File:imageservice19.png|439x409px]]

= Safety and Hazard considerations =

== Gas ==

The Ar and CO2 do not require special health safety system. The detector will be operated in the ventilated hall of the AD building at CERN.

Gas flow, detector gas pressure, and gas inlet temperature are constantly monitored and can trigger adequate response for shutdown or closure of the gas system for safety purpose.

== Cooling water ==

The water cooling system operates in negative pressure mode preventing any water leak to flood the detector outer surface. In addition water flow and inlet/outlet temperature are constantly monitored to allow fast response to off range parameter in order to shut down the electronics. The cooling water system requires a primary cooling tap water loop from the AD building. Water quality, water flow and water temperature are not under our responsibility. This aspect will be dealt during the initial phase of the rTPC commissioning at CERN.

== Power distribution ==

All electronics voltages are below 48V. No special safety circuits are envisaged.

The high-voltage circuit is fully protected outside the detector. The access to the high-voltage on the detector is restricted to the two end-plates. Interlock scheme will be put in place to ensure the shutdown of the HV in case of manual access to the detector itself.

The LV (Low Voltage) power units are also interlocked to prevent false powering sequence which can damage the downstream electronics modules.

[[File:imageservice20.png|571x411px]]

Figure 16- Alpha-g rTPC rack organization

File:PWBmainOpen.jpg

2018-12-07T13:14:21Z

Acapra:

rTPC services rack, water cooling system, main valve open

Main Page

2018-11-09T04:55:20Z

Acapra:

This is the new wiki for the ALPHA-g detector and daq.

== Links ==

* agdaq at CERN: https://alphacpc05.cern.ch/agdaq/
* agmini midas status page: https://daq16.triumf.ca
* elog: https://daq.triumf.ca/elog-alphag/alphag/

== temporary index ==

* [[daq]] -- DAQ manual
* [[Equipment]] -- Equipment manuals

== Documentation ==

General information about the Alpha-g Radial Time Projection Chamber (rTPC)

* [[Detector Overview]] : Design note
* [[Detector Simulations]] : Simulation and track recontruction
* [[Mechanical Design]] : Mechanical design of the rTPC
* [[Detector Data Acquistion]] : Computer and Back-end considerations
* [[Detector Calibration]] : Laser calibration scheme
* [[Detector Electronics]] : Front-end and digitizers
* [[Detector Services]] : Gas, Water cooling, Power distribution
* [[Barrel Scintillator]] : Cosmic veto detector

* [[File:Top rTPC and BS Channel Numbering - RH.pdf|thumb|rTPC and BS Channel Numbering -- TOP view]]

* [[File:BV mapping.pdf|thumb|BV DAQ channels assignment]]

* [[Magnetometry]]

== Mediawiki default blurb ==

MediaWiki has been installed.

Consult the [https://www.mediawiki.org/wiki/Special:MyLanguage/Help:Contents User's Guide] for information on using the wiki software.

== Getting started ==
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Configuration_settings Configuration settings list]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:FAQ MediaWiki FAQ]
* [https://lists.wikimedia.org/mailman/listinfo/mediawiki-announce MediaWiki release mailing list]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Localisation#Translation_resources Localise MediaWiki for your language]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Combating_spam Learn how to combat spam on your wiki]

Main Page

2018-11-09T04:55:01Z

Acapra:

This is the new wiki for the ALPHA-g detector and daq.

== Links ==

* agdaq at CERN: https://alphacpc05.cern.ch/agdaq/
* agmini midas status page: https://daq16.triumf.ca
* elog: https://daq.triumf.ca/elog-alphag/alphag/

== temporary index ==

* [[daq]] -- DAQ manual
* [[Equipment]] -- Equipment manuals

== Documentation ==

General information about the Alpha-g Radial Time Projection Chamber (rTPC)

* [[Detector Overview]] : Design note
* [[Detector Simulations]] : Simulation and track recontruction
* [[Mechanical Design]] : Mechanical design of the rTPC
* [[Detector Data Acquistion]] : Computer and Back-end considerations
* [[Detector Calibration]] : Laser calibration scheme
* [[Detector Electronics]] : Front-end and digitizers
* [[Detector Services]] : Gas, Water cooling, Power distribution
* [[Barrel Scintillator]] : Cosmic veto detector

* [[File:Top rTPC and BS Channel Numbering - RH.pdf|thumb|rTPC and BS Channel Numbering -- TOP view]]

[[File:BV mapping.pdf|thumb|BV DAQ channels assignment]]

* [[Magnetometry]]

== Mediawiki default blurb ==

MediaWiki has been installed.

Consult the [https://www.mediawiki.org/wiki/Special:MyLanguage/Help:Contents User's Guide] for information on using the wiki software.

== Getting started ==
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Configuration_settings Configuration settings list]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:FAQ MediaWiki FAQ]
* [https://lists.wikimedia.org/mailman/listinfo/mediawiki-announce MediaWiki release mailing list]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Localisation#Translation_resources Localise MediaWiki for your language]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Combating_spam Learn how to combat spam on your wiki]

File:BV mapping.pdf

2018-11-09T04:54:38Z

Acapra:

BV or BSC channels assignment

Magnetometry

2018-11-09T04:41:02Z

Acapra: Created page with " {| class="wikitable sortable" |- ! NMR+Hall !! PWB !! Column !! Row !! Approx. Location |- | 6 || 19 || 0 || 7 || coilGprime/US |- | 1 || 4 || 4 || 7 || coilGprime/DS |-..."

{| class="wikitable sortable"
|-
! NMR+Hall !! PWB !! Column !! Row !! Approx. Location
|-
| 6 || 19 || 0 || 7 || coilGprime/US
|-
| 1 || 4 || 4 || 7 || coilGprime/DS
|-
| 4 ||33 ||2 ||5 ||coilAprime/Asacusa
|-
| 3 ||65 ||6 ||5 ||coilAprime/Laser-Hut
|-
| 5 ||2 ||0 ||3 ||coilG/US
|-
|8 ||47 ||4 ||3 ||coilG/DS
|-
| 7 ||29 ||2 ||1 ||coilA/Asuacusa
|-
| 2 ||0 ||6 ||1 ||coilA/Laser-Hut
|}

[[File:TPC magnetometer locations.pdf|thumb|Location of NMR+Hall in the TPC]]

Main Page

2018-11-09T04:30:09Z

Acapra:

This is the new wiki for the ALPHA-g detector and daq.

== Links ==

* agdaq at CERN: https://alphacpc05.cern.ch/agdaq/
* agmini midas status page: https://daq16.triumf.ca
* elog: https://daq.triumf.ca/elog-alphag/alphag/

== temporary index ==

* [[daq]] -- DAQ manual
* [[Equipment]] -- Equipment manuals

== Documentation ==

General information about the Alpha-g Radial Time Projection Chamber (rTPC)

* [[Detector Overview]] : Design note
* [[Detector Simulations]] : Simulation and track recontruction
* [[Mechanical Design]] : Mechanical design of the rTPC
* [[Detector Data Acquistion]] : Computer and Back-end considerations
* [[Detector Calibration]] : Laser calibration scheme
* [[Detector Electronics]] : Front-end and digitizers
* [[Detector Services]] : Gas, Water cooling, Power distribution
* [[Barrel Scintillator]] : Cosmic veto detector

* [[File:Top rTPC and BS Channel Numbering - RH.pdf|thumb|rTPC and BS Channel Numbering -- TOP view]]

* [[Magnetometry]]

== Mediawiki default blurb ==

MediaWiki has been installed.

Consult the [https://www.mediawiki.org/wiki/Special:MyLanguage/Help:Contents User's Guide] for information on using the wiki software.

== Getting started ==
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Configuration_settings Configuration settings list]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:FAQ MediaWiki FAQ]
* [https://lists.wikimedia.org/mailman/listinfo/mediawiki-announce MediaWiki release mailing list]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Localisation#Translation_resources Localise MediaWiki for your language]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Combating_spam Learn how to combat spam on your wiki]

Main Page

2018-11-09T04:28:38Z

Acapra:

This is the new wiki for the ALPHA-g detector and daq.

== Links ==

* agdaq at CERN: https://alphacpc05.cern.ch/agdaq/
* agmini midas status page: https://daq16.triumf.ca
* elog: https://daq.triumf.ca/elog-alphag/alphag/

== temporary index ==

* [[daq]] -- DAQ manual
* [[Equipment]] -- Equipment manuals

== Documentation ==

General information about the Alpha-g Radial Time Projection Chamber (rTPC)

* [[Detector Overview]] : Design note
* [[Detector Simulations]] : Simulation and track recontruction
* [[Mechanical Design]] : Mechanical design of the rTPC
* [[Detector Data Acquistion]] : Computer and Back-end considerations
* [[Detector Calibration]] : Laser calibration scheme
* [[Detector Electronics]] : Front-end and digitizers
* [[Detector Services]] : Gas, Water cooling, Power distribution
* [[Barrel Scintillator]] : Cosmic veto detector

* [[File:Top rTPC and BS Channel Numbering - RH.pdf|thumb|rTPC and BS Channel Numbering -- TOP view]]

* [[File:TPC magnetometers locations.pdf|thumb]]

== Mediawiki default blurb ==

MediaWiki has been installed.

Consult the [https://www.mediawiki.org/wiki/Special:MyLanguage/Help:Contents User's Guide] for information on using the wiki software.

== Getting started ==
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Configuration_settings Configuration settings list]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:FAQ MediaWiki FAQ]
* [https://lists.wikimedia.org/mailman/listinfo/mediawiki-announce MediaWiki release mailing list]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Localisation#Translation_resources Localise MediaWiki for your language]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Combating_spam Learn how to combat spam on your wiki]

Main Page

2018-11-09T04:28:20Z

Acapra:

This is the new wiki for the ALPHA-g detector and daq.

== Links ==

* agdaq at CERN: https://alphacpc05.cern.ch/agdaq/
* agmini midas status page: https://daq16.triumf.ca
* elog: https://daq.triumf.ca/elog-alphag/alphag/

== temporary index ==

* [[daq]] -- DAQ manual
* [[Equipment]] -- Equipment manuals

== Documentation ==

General information about the Alpha-g Radial Time Projection Chamber (rTPC)

* [[Detector Overview]] : Design note
* [[Detector Simulations]] : Simulation and track recontruction
* [[Mechanical Design]] : Mechanical design of the rTPC
* [[Detector Data Acquistion]] : Computer and Back-end considerations
* [[Detector Calibration]] : Laser calibration scheme
* [[Detector Electronics]] : Front-end and digitizers
* [[Detector Services]] : Gas, Water cooling, Power distribution
* [[Barrel Scintillator]] : Cosmic veto detector

* [[File:Top rTPC and BS Channel Numbering - RH.pdf|thumb|rTPC and BS Channel Numbering -- TOP view]]

* [[File:TPC magnetometers locations.pdf|thumb|NMR plus Hall location in the TPC]]

== Mediawiki default blurb ==

MediaWiki has been installed.

Consult the [https://www.mediawiki.org/wiki/Special:MyLanguage/Help:Contents User's Guide] for information on using the wiki software.

== Getting started ==
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Configuration_settings Configuration settings list]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:FAQ MediaWiki FAQ]
* [https://lists.wikimedia.org/mailman/listinfo/mediawiki-announce MediaWiki release mailing list]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Localisation#Translation_resources Localise MediaWiki for your language]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Combating_spam Learn how to combat spam on your wiki]

Main Page

2018-11-09T04:27:36Z

Acapra:

This is the new wiki for the ALPHA-g detector and daq.

== Links ==

* agdaq at CERN: https://alphacpc05.cern.ch/agdaq/
* agmini midas status page: https://daq16.triumf.ca
* elog: https://daq.triumf.ca/elog-alphag/alphag/

== temporary index ==

* [[daq]] -- DAQ manual
* [[Equipment]] -- Equipment manuals

== Documentation ==

General information about the Alpha-g Radial Time Projection Chamber (rTPC)

* [[Detector Overview]] : Design note
* [[Detector Simulations]] : Simulation and track recontruction
* [[Mechanical Design]] : Mechanical design of the rTPC
* [[Detector Data Acquistion]] : Computer and Back-end considerations
* [[Detector Calibration]] : Laser calibration scheme
* [[Detector Electronics]] : Front-end and digitizers
* [[Detector Services]] : Gas, Water cooling, Power distribution
* [[Barrel Scintillator]] : Cosmic veto detector

* [[File:Top rTPC and BS Channel Numbering - RH.pdf|thumb|rTPC and BS Channel Numbering -- TOP view]]

* [[File:TPC magnetometers locations.pdf|thumb|NMR+Hall location in the TPC]]

== Mediawiki default blurb ==

MediaWiki has been installed.

Consult the [https://www.mediawiki.org/wiki/Special:MyLanguage/Help:Contents User's Guide] for information on using the wiki software.

== Getting started ==
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Configuration_settings Configuration settings list]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:FAQ MediaWiki FAQ]
* [https://lists.wikimedia.org/mailman/listinfo/mediawiki-announce MediaWiki release mailing list]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Localisation#Translation_resources Localise MediaWiki for your language]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Combating_spam Learn how to combat spam on your wiki]

Main Page

2018-11-09T04:26:22Z

Acapra:

This is the new wiki for the ALPHA-g detector and daq.

== Links ==

* agdaq at CERN: https://alphacpc05.cern.ch/agdaq/
* agmini midas status page: https://daq16.triumf.ca
* elog: https://daq.triumf.ca/elog-alphag/alphag/

== temporary index ==

* [[daq]] -- DAQ manual
* [[Equipment]] -- Equipment manuals

== Documentation ==

General information about the Alpha-g Radial Time Projection Chamber (rTPC)

* [[Detector Overview]] : Design note
* [[Detector Simulations]] : Simulation and track recontruction
* [[Mechanical Design]] : Mechanical design of the rTPC
* [[Detector Data Acquistion]] : Computer and Back-end considerations
* [[Detector Calibration]] : Laser calibration scheme
* [[Detector Electronics]] : Front-end and digitizers
* [[Detector Services]] : Gas, Water cooling, Power distribution
* [[Barrel Scintillator]] : Cosmic veto detector

* [[File:Top rTPC and BS Channel Numbering - RH.pdf|thumb|rTPC and BS Channel Numbering -- TOP view]]

* [[File:TPC magnetometer locations.pdf|thumb|NMR+Hall location in the TPC]]

== Mediawiki default blurb ==

MediaWiki has been installed.

Consult the [https://www.mediawiki.org/wiki/Special:MyLanguage/Help:Contents User's Guide] for information on using the wiki software.

== Getting started ==
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Configuration_settings Configuration settings list]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:FAQ MediaWiki FAQ]
* [https://lists.wikimedia.org/mailman/listinfo/mediawiki-announce MediaWiki release mailing list]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Localisation#Translation_resources Localise MediaWiki for your language]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Combating_spam Learn how to combat spam on your wiki]

File:TPC magnetometer locations.pdf

2018-11-09T04:22:39Z

Acapra: Acapra uploaded a new version of File:TPC magnetometer locations.pdf

Sketch of NMR+Hall in the TPC

File:TPC magnetometer locations.pdf

2018-11-09T04:20:40Z

Acapra: Acapra uploaded a new version of File:TPC magnetometer locations.pdf

Sketch of NMR+Hall in the TPC

Main Page

2018-10-21T12:13:02Z

Acapra:

This is the new wiki for the ALPHA-g detector and daq.

== Links ==

* agdaq at CERN: https://alphacpc05.cern.ch/agdaq/
* agmini midas status page: https://daq16.triumf.ca
* elog: https://daq.triumf.ca/elog-alphag/alphag/

== temporary index ==

* [[daq]] -- DAQ manual
* [[Equipment]] -- Equipment manuals

== Documentation ==

General information about the Alpha-g Radial Time Projection Chamber (rTPC)

* [[Detector Overview]] : Design note
* [[Detector Simulations]] : Simulation and track recontruction
* [[Mechanical Design]] : Mechanical design of the rTPC
* [[Detector Data Acquistion]] : Computer and Back-end considerations
* [[Detector Calibration]] : Laser calibration scheme
* [[Detector Electronics]] : Front-end and digitizers
* [[Detector Services]] : Gas, Water cooling, Power distribution
* [[Barrel Scintillator]] : Cosmic veto detector

* [[File:Top rTPC and BS Channel Numbering - RH.pdf|thumb|rTPC and BS Channel Numbering -- TOP view]]

* [[File:TPC magnetometer locations.pdf|thumb]]

== Mediawiki default blurb ==

MediaWiki has been installed.

Consult the [https://www.mediawiki.org/wiki/Special:MyLanguage/Help:Contents User's Guide] for information on using the wiki software.

== Getting started ==
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Configuration_settings Configuration settings list]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:FAQ MediaWiki FAQ]
* [https://lists.wikimedia.org/mailman/listinfo/mediawiki-announce MediaWiki release mailing list]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Localisation#Translation_resources Localise MediaWiki for your language]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Combating_spam Learn how to combat spam on your wiki]

File:TPC magnetometer locations.pdf

2018-10-21T12:12:43Z

Acapra:

Sketch of NMR+Hall in the TPC