Daq

From AgWiki
Jump to navigation Jump to search

Links

Hardware manuals

Network configuration

The experiment private network number is: 192.168.1.x, gateway 192.168.1.1, netmask 255.255.255.0

network map: 

---- TRIUMF/CERN network --1gige-- alphagdaq --10gige-- juniper switch --1gige-- centrecom switch

alphagdaq network connections on the back of the machine: 

-----------------------------------------
|   | rj45 | rj45 |         | sfp | sfp |
-----------------------------------------

- left rj45 - copper 1gige - eno1 - spare (inactive)
- right rj45 - copper 1gige - eno2 - dhcp - cern/triumf network
- left sfp - 10gige - enp1s0f0 - static 192.168.1.1
- right sfp - 10gige - enp1s0f1 - spare (inactive)

connections between switches:

  • alphagdaq to juniper switch: direct attach (DAC) 10gige cable into any port of juniper switch
  • juniper switch to centrecom switch: rj45 sfp in any port of juniper switch - cat5e/cat6 cable - to any port of centrecom switch
  • cat5e/cat6 cable from any port of centrecom switch to the XXX port on the back of the juniper switch

connected to the juniper switch:

  • everything that sends event data to alphagdaq, specifically:
    • TRG (rj45 sfp)
    • 16x ADC (fiber sfp)
    • 64x PWB (fiber sfp)
  • 10gige uplink to alphagdaq (10gige DAC cable)
  • 1gige link to centrecom switch (rj45 sfp)
  • 40gige ports are not used

connected to the centrecom switch:

  • everything with copper rj45 connections, specifically:
    • CDM boards
    • HV, LV and VME power supplies
    • RaspberryPi3 boards

Clock and trigger distribution

Explanation

Trigger is generated by the trigger board (aka TRG, aka GRIF-C), from the TRG eSATA output through the eSATA splitter it is fed into the master CDM. The master CDM outputs the trigger on all MiniSAS output ports. One of the master CDM MiniSAS outputs is connected to the CDM slave eSATA input. The same as the master CDM, the slave CDM outputs the trigger on all MiniSAS output ports.

The master 62.5MHz clock is generated by the master CDM using the internal 10MHz oscillator or the external 10MHz AD clock. Also on MiniSAS port 6 the master CDM generates the 200MHz clock for the TDC. This clock is frequency-locked to the master 62.5MHz clock.

Master CDM clock selection (external CD 10MHz clock or internal oscillator 10MHz clock) is done by a magic esper command (see below).

From the master CDM, the clock is fed to: the slave CDM, the TRG (via the eSATA splitter) and to the TDC (via the RJ45 splitter).

The slave CDM sends the clock and the trigger to the ADCs and PWBs.

Schematic

TRG (GRIF-C)
------------
eSATA <------> eSATA splitter

eSATA splitter
--------------
trigger ---> eSATA ---> CDM-Master eSATA (trigger signal)
clock <--- (eSATA --- MiniSAS) <--- CDM MiniSAS (62.5MHz clock)

CDM-Master
----------
eSATA <--- eSATA splitter <--- eSATA from TRG (trigger signal)
MiniSAS 1 --> (MiniSAS --- eSATA) --> CDM-Slave eSATA (62.5 MHz)
MiniSAS 6 --> (MiniSAS --- eSATA) --> eSATA splitter --> RJ45 --> TDC (200 MHz)

CDM-Slave
---------
eSATA <--- trigger and 62.5MHz clock from CDM-Master
MiniSAS 1 --> ADC trigger and 62.5MHz clock
MiniSAS 2 --> same
MiniSAS 3 --> same
MiniSAS 4 --> same
MiniSAS 5 --> PWB trigger and 62.5MHz clock
MiniSAS 6 --> same

Setup of CDM boards

ALPHA-g uses the Rev3 CDM boards. The 4 CDM boards that belong to ALPHA-g have serial numbers 0, 1, 2 and 3 and are labeled cdm00, cdm01, cdm02, cdm03.

All the boards can be used as master and slave positions, but have to be configured appropriately.

Slave setup

  • install the board
  • connect ethernet
  • connect esata external clock
  • esper-tool cdmNN ### i.e. esper-tool cdm01
  • cd template
  • write current_setup 4
  • cd /cdm
  • read
  • observe clock frequency counters report correct values. "ext_clk" will only count if the lemo jumper is installed (see below)
11    esata_clk        uint32           R                 [62500246]                      
12    ext_clk          uint32           R                 [62500246]
  • cd /lmk
  • read
  • observe both PLL1 and PLL2 are locked: both values should be "1"
144   pll1_ld          uint8            R                 [1]                             
147   pll2_ld          uint8            R                 [1]                             
156   ld1_counter      uint32           RW                [3]                             
157   ld2_counter      uint32           RW                [3]

If needed, check correct operation of the pll lock monitoring:

  • unplug the esata cable from the master CDM
  • "red" light should go off
  • read of lmk should report "pll1_ld" value 0 and "ld1_counter" and "ld2_counter" should increment
  • reconnect the esata cable
  • "red" light should return
  • read of lmk should report both pll1_ld and pll2_ld locked (values "1").

Master setup

  • install the board
  • connect ethernet
  • esper-tool cdmNN ### i.e. esper-tool cdm01
  • cd template
  • write current_setup 3
  • cd /lmk
  • write clkin_sel_mode 0
  • cd /cdm
  • read
  • observe clock frequency counters report correct values. "ext_clk" will only count if the lemo jumper is installed (see below)
12    ext_clk          uint32           R                 [62500189]
  • cd /lmk
  • read
  • observe both PLL1 and PLL2 are locked: both values should be "1"
144   pll1_ld          uint8            R                 [1]                             
147   pll2_ld          uint8            R                 [1]                             
156   ld1_counter      uint32           RW                [3]                             
157   ld2_counter      uint32           RW                [3]

Check the 200MHz clock:

  • connect MiniSAS-eSATA splitter cable to MiniSAS output 6, connect eSATA end to CDM eSATA input
  • in esper-tool, read cdm: esata_clk should report 200MHz clock
11    esata_clk        uint32           R                 [200000603]                     
12    ext_clk          uint32           R                 [62500188]

clkin_sel_mode positions

  • 0 = 10MHz internal oscillator (use on master CDM in standalone mode)
  • 1 = eSATA clock (use on slave CDM)
  • 2 = external clock, should be 10MHz (use on master CDM to run from AD clock)

LEMO connections

  • external clock jumper: to read the frequency of the generated 62.5MHz clock, connect LEMO cable from left LEMO #2 to the right LEMO #1
  • 10MHz clock jumper: to feed 10MHz internal oscillator to the external clock, connect LEMO cable from right LEMO #3 to the right LEMO #1.

MIDAS frontends

UDP

feudp receives udp packets from the ADC and PWB boards. For each udp packet 1 midas event is created, with 1 data bank containing the udp data. Events are sent into the MIDAS event buffer BUFUDP. The names of the data banks are assigned in ODB /eq/feudp/settings.

{ADC,PWB} --> 1gige --> switch --> 10gige --> alphagdaq --> feudp -> BUFUDP

CTRL

fectrl controls all the ADC, PWB and trigger boards. It provides run start and stop sequencing, runs the event synchronization sequence at the begin of the run, and records slow controls data (temperatures, voltages, etc).

fectrl receives UDP data packets from the trigger board and sends them to the MIDAS event buffer BUFUDP.

fectrl receives commands from the control web pages (for ADC, PWB and trigger) to reboot, initialize, etc the boards.

fectrl configures the event builder via odb /eq/fectrl/evbconfig.

ADC <-> http esper <-> fectrl -> slow control data into midas history

PWB <-> http esper <-> fectrl -> slow control data into midas history

TRG <-> udp comm <-> fectrl -> BUFUDP, slow control and counters into midas history

fectrl <-> midas rpc <-> mhttpd <-> json rpc <-> control web pages for ADC, PWB and trigger

EVB

feevb receives udp packet data from the MIDAS event buffer BUFUDP, inspects the packet timestamps and collects the data with matching timestamps into physics events. feevb has provisions to do data suppression, reduction and compression in addition to the data reduction done in the ADC and PWB firmware.

feevb event synchronization relies on the event synchronization sequence done by fectrl when a run is started.

{ADC, PWB, TRG} -> BUFUDP -> feevb -> SYSTEM -> mlogger -> compression -> disk storage

ODB entries

  • /Equipment/
    • CTRL/Settings
      • TBW
    • CTRL/Settings/Trig - trigger settings
      • Enable - if set to "n", fectrl will not do anything with the trigger board. Normal value "y"
      • Modules - hostnames of the trigger board. Normal value "alphat01"
      • NimMask - 32-bit mask for the 32x NIM inputs (see TRG manual). Normal value 0, unless external trigger is in use.
      • EsataMask - 32-bit mask for the 32x eSATA signals (see TRG manual). Normal value 0.
      • adc16_mask[16] - 16x 16-bit mask for the 100MHz GRIF-16 onboard ADCs (see TRG manual). Normal value 0xFFFF for all 16 data links.
      • adc32_mask[16] - 16x 32-bit mask for the 62.5MHz FMC-ADC32 ADCs (see TRG manual). Normal value 0xFFFFFFFF for all 16 data links.
      • PassThrough - if set to "y", trigger is passed through the trigger module without generating any events and without causing deadtime.
      • CoincA, B, C and D - 32-bit masks for the adc16 coincidence trigger (see TRG manual)
    • CTRL/Settings/PWB - PWB settings
      • Enable - if set to "n", fectrl will not do anything with PWBs. Normal value "y".
      • enable_boot_user_page - if set to "n", fectrl will not try to reboot PWBs into the user page firmware. Normal value "y".
      • enable_trigger - if set to "n", all PWBs will be set to ignore the trigger. Normal value "y".
      • modules[64] - hostnames of PWB boards
      • boot_user_page[64] - if set to "y" and enable_boot_user_page is "y", fectrl will reboot the corresponding PWB to the user page firmware.
      • trigger[64] - if set to "y" and enable_trigget is "y", fectrl will set the PWB to accept the trigger.
      • ch_enable - if set to "n" disables all PWB channels. Normal value is "y".
      • ch_force - is set to "y" disables channel suppression (all PWB channels are read)
      • suppress_reset - if set to "y" enables channel suppression for reset channels (threshold has to be set correctly)
      • suppress_fpn - if set to "y" enables channel suppression for fpn channels (threshold has to be set correctly)
      • suppress_pads - set to "y" enables channel suppression for TPC pad channels (threshold has to be set correctly)
      • baseline_{reset,fpn,pads}[64] - waveform baseline for each PWB position
      • threshold_{reset,fpn,pads} - waveform suppression threshold, see below:
      • ch_threshold - PWB sca_N_ch_threshold, if set to 0, threshold is computed as "baseline_pads[seqpwb]-threshold_pads". Normal value is 0.

Trigger configuration

These trigger modes are implemented:

  • (manual trigger from web page)
  • software pulser trigger (from fectrl)
  • hardware pulser trigger (from the trigger board)
  • NIM and eSATA trigger from ADC front panel inputs
  • adc16 and adc32 discriminator triggers:
    • adc16 grand-or trigger
    • adc32 grand-or trigger
    • adc (adc16+adc32) grand-or trigger
    • adc16 multiplicity trigger (1ormore, 2ormore, 3ormore, 4ormore) for use with barrel scintillator
    • anode wire multiplicity trigger (1ormore, 2ormore, 3ormore, 4ormore) (anode wire signals can be fed from adc16 or from adc32 signals)
    • anode wire coincidence trigger
    • anode wire MLU (memory lookup) trigger

NIM trigger from ADC front panel

Trigger path: 

NIM signal --> LEMO input on ADC front panel --> data encoder --> data link to trigger board --> data decoder --> mask --> nim-esata-grand-or --> trigger

To enable NIM trigger, do this:

  • connect NIM signal to ADC front panel input
    • Note1: NIM signal must be inverted
    • Note2: LEMO connectors should be set to "NIM input" mode by ADC on-board jumpers (see ADC manual)
  • observe correct bit goes to zero in the TRG web page: data link high work should change:
0xNF00'0000 (N is the module id) to
0xN700'0000 (left lemo) or
0xNB00'0000 (right lemo)
  • Note3: if data link bits do not change, most likely the LEMO connectors are set to "DAC output" mode. Try to use LEMO inputs of a different ADC.
  • compute the NIM mask: there are 32 bits, 2 bits per ADC data link. Bit 0x1 is right LEMO, bit 0x2 is left LEMO
  • for example, to use left LEMO of ADC data link 12, set nim mask to (0x2)<<(2*12) = 0x02000000
  • initialize the trigger board (on the TRG web page press button "initialize"), or start a new run
  • on the history plot with "NIM grand or" counter, the rate should change from zero
  • if a run is active, if one of the ADC LEMO outputs is set to output the trigger signal as a NIM pulse, one should see it fire use a scope. The delay or jitter between NIM trigger in and NIM trigger out should be around 100-200 ns (FIXME - measure this!)
  • to set the DAQ to trigger on the NIM signal, on the run start page, check the box "TrigEsataNimGrandOr".

ZZZ

ZZZ

Retrieved from "https://daq00.triumf.ca/AgWiki/index.php?title=Daq&oldid=280"