VME-NIMIO32

VME-NIMIO32 - general purpose VME FPGA board

References

https://bitbucket.org/ttriumfdaq/vme-nimio32 VME-NIMIO32 firmware repository on bitbucket
File:VME-NIMIO32 Rev0.pdf rev0 schematics
File:VME-NIMIO32 Rev1.pdf rev1 schematics
File:VME-NIMIO32 Rev2.pdf rev2 schematics
File:VME-NIMIO32 REV1 - nCONFIG PCB Modification.pdf rev1 board required modification: "reset mod"

(dead) edev login: no public access, sorry!
(dead) ladd00 svn login: username: svn, password: svn

(dead) [1] VME-NIMIO32 (REA 131) project page on edev.triumf.ca
(dead) [2] Rev0 board schematics (link to edev)
(dead) [3] Rev0 board schematics
(dead) [4] Rev1 board schematics (link to edev)
(dead) [5] Rev1 board schematics
(dead) [6] Rev1 board required modification: "reset mod"
(dead) [7] Rev2 board schematics
(dead) [8] (obsolete!) Svn repository for VME-NIMIO32 firmware (obsolete!)
(dead) [9] Misc documentation

General characteristics

Photos

Available hardware

Altera cyclone 1 FPGA: EP1C6Q240C6N
Serial flash for FPGA configuration: Altera EPCS16
External SDRAM memory
VME interface: VME-D[31..0] bidirectional, VME-A[19..0] input only, DTACK output, no BERR, no RETRY/RESP. VME-A[31..20] input only connected only to address decoder FPGA (Altera MAX-something CPLD). This permits all single-word transfer modes, 32-bit DMA (BLT32) and 2eVME DMA (only drives D-lines, but still faster than BLT32). 64-bit DMA (MBLT64) and 2eSST are impossible.
16 NIM outputs
16 NIM inputs
16 ECL/LVDS inputs compatible with LeCroy 4616 ECL/NIM/ECL converter. (Except: Rev0 boards have an incompatible ECL connector)
16 "red" and 16 "green" LEDs
1 "VME access" LED
1 RJ-45 high speed serial I/O interface

Firmware functions

Generic IO32 firmware can include a number of functions. Not all functions are available all at the same time due to resource limitations on the FPGA. Refer to VME Registers for detailed information.

20 MHz 32-bit timestamp clock register
16 bits of NIM output register (first 4 NIM outputs are multi-function)
16 bits of NIM and 16 bits of ECL/LVDS input register with latch function.
32 scalers 32 bit, 300 MHz counting rate
20 MHz 32-bit timestamp clock register
4 timestamp registers using the 20 MHz timestamp clock. Any of the 32 inputs can be routed into any of the 4 timestamp registers.
trigger counter, trigger timestamp functions for NIM_IN[1] and DAQ busy functions for FP_NIMOUTPUT[1].
20 MHz clock for VF48 ADC and VT48 TDC
40 MHz clock for V1190 TDC.
example pulse generator (programmable period 32 bit 10 ns, fixed width 100 ns)
example L222-like delay and gate generators (NIM_IN[2]/NIM_OUT[2])
experimental 4-channel TDC. 10 ns base clock with 0.3 ns interpolation ladder (1 ns RMS).

Firmware update procedure

obtain and build the VME programmer (srunner_vme) and IO32 test program (test_VMENIMIO32):
- mkdir $HOME/packages
- cd $HOME/packages
- # !!!obsolete!!! svn checkout https://ladd00.triumf.ca/svn/daqsvn/trunk/vme (username svn, password svn)
- git clone https://bitbucket.org/ttriumfdaq/vme # get the repository, done once
- cd vme
- git pull # get latest version
- make -k srunner_vme.exe srunner_vme_gef.exe test_VMENIMIO32.exe test_VMENIMIO32_gef.exe

read the firmware revision from VME register 0:
- ./test_VMENIMIO32_gef.exe --addr 0x100000 --read 0

write Cyclone 1 FPGA firmware into the flash memory using the VME programmer:
- ./srunner_vme_gef.exe -program -64 VME-NIMIO32.pof 0x100020
- !!!be careful about using Rev0 pof files for Rev0 boards and Rev1 pof files for Rev1 boards!!!
- !!!be careful about using PPG32 pof files for IO32 boards and vice-versa!!!

reboot the Cyclone 1 FPGA into the new firmware (requires "fpga-reset" mod):
- ./test_VMENIMIO32_gef.exe --addr 0x100000 --reboot

Firmware revisions

Firmware for the main FPGA:

(NOTE: use Quartus 13.0.1 to compile VME-NIMIO32 firmware - support for Cyclone-1 FPGA is removed from newer versions)

(NOTE: on el7 linux, i.e. CentOS-7, use /daq/daqshare/olchansk/altera/12.1sp1/quartus/bin/quartus)

(NOTE: update July2020, on CentOS-7, DO NOT USE /daq/quartus/13.0sp1/quartus/bin/quartus it crashes on startup)

(NOTE: update July2020, on CentOS-7, use /daq/quartus/12.1sp1/quartus/bin/quartus)

(NOTE: update Mar2021, on Ubuntu LTS 20.04, use /daq/quartus/13.0sp1/quartus/bin/quartus or /daq/quartus/13.1.3.178/quartus/bin/quartus_pgmw)

firmware sources (obsolete) http://ladd00.triumf.ca/viewvc/daqsvn/trunk/VME-NIMIO32/VME-NIMIO32
firmware sources https://bitbucket.org/ttriumfdaq/vme-nimio32/src/master/VME-NIMIO32/
git clone git@bitbucket.org:ttriumfdaq/vme-nimio32.git
firmware schematics [[10]]
pof files for Rev0 [[11]]
pof files for Rev1 [[12]]
0x01100810 - DO NOT USE - first revision of generic firmware
0x01100818 - [[13]] - fixed bug in scaler[6], rearranged TSC4 registers.
0x01100914 - Rev1 - [[14]] - add NIM output functions: VME AS and DS outputs, example L222 delay generator
0x01101018 - Rev1 - [[15]] - add simplified trigger timestamp functions, add 40MHz V1190 clock output, add T2KM11 trigger function
0x01101019 - Rev0 - [[16]] - same as 0x01101018
0x01101105 - Rev0 - svn rev 109 - fixed problem with long pulses being miscounted by scalers
0x01101118 - Rev1 - svn rev 112 - improved S1249 trigger, output V1190 40 MHz clock on NIM_OUT[3]
0x01110702 - Rev1 - svn rev 144 - DO NOT USE - updated the VME interface - DO NOT USE - broken UniverseII VME access (V7805, V7750)
0x01110811 - Rev1 - svn rev 159 - no changes. fix for UniverseII VME access.
0x01110811 - PPG32-Rev1 - svn rev 160 - no changes. fix for UniverseII VME access.
0x01120309 for Rev0 and Rev1 boards - implement Dragon trigger function, additional ALPHA trigger function
0x01120523 - Rev0, Rev1 - svn rev 169 - implement new scalers (old scalers were lossy)
0x01120706 - Rev1 - svn rev 187 - NIM input/output 2 provides V1190 TDC trigger signal synchronized with the 40 MHz V1190 clock.
0x01120808 and 0x01120809 for Rev0 and Rev1 boards - implement external timestamp reset function
0x01120907 and 0x01120910 for Rev0 and Rev1 boards - DRAGON tail trigger logic
0x01120925 for Rev0 and Rev1 boards - final ALPHA trigger logic
0x01121026 for Rev1/Rev2 boards - final TRINAT trigger function
0x01121113 for Rev1/Rev2 boards - add register 16 to invert NIM and LVDS signals (for TRINAT inverted TTL inputs)
0x01121128 - change routing of scalers, implement timestamp clock synchronization. final firmware for TRINAT
0x01121205, 0x01121210, 0x01121212 - TRINAT trigger logic updates
0x01130215 - DRAGON - fix possible glitch in synchronizing the input signals
0x01131004 - S1249 - final revision
0x01131024 - cleanup, route ECL inputs to high LEDs (see command 6)
0x01140605 - final TRINAT trigger logic
0x01180917 - final TRINAT 2018 trigger logic
0x01181205 - (PPG32) EMMA clock distribution and PLL
0x01200721 - special version for IRIS, see the IRIS section on this page
0x01210430 - special version for IRIS, see "IRIS Clock" section in this page

Firmware for the VME address decoder:

firmware sources (obsolete) http://ladd00.triumf.ca/viewvc/daqsvn/trunk/VME-NIMIO32/MAX3000A_Addr_decode
firmware sources https://bitbucket.org/ttriumfdaq/vme-nimio32/src/master/MAX3000A_Addr_decode
board Rev0 [[17]]
board Rev1 [[18]]

Manual

VME interface

Firmware 0x01100810 implements VME A24/D32 access only. A24 addresses 0x00N0xxxx are decoded, where "N" is the A24 base address set by rotary switch SW3 "ADDRESS 20-23". Rotary switches SW1 and SW2 should be set to "0".

VME accessible registers are listed in the table below. VME address offsets are register numbers multiplied by 4, i.e. CSR register 1 is at address 0x00N00004. Flash programmer register 8 is at 0x00N00020 (use this address with srunner_vme flash programmer).

LEMO connectors

The LEMO connectors are NIM inputs (NIM_IN) and NIM outputs (FP_NIMOUT). NIM inputs correspond to green LEDs on the lower/right side of the board, NIM outputs correspond to red LEDs on the upper/left side of the board. The NIM channel numbering on the board schematics is counterintuitive, the channel numbering described here is implemented by the IO32 firmware:

                  O15 | ... | O1 | I15 | ... | I1
 VME_LED | RJ45 | O14 | ... | O0 | I14 | ... | I0
--------------------------------------------------- PCB

Registers

Number	Name	Access	FwRev	Description
0	FwRev	RO		Firmware revision (0x01YYMMDD)
1	Command	RW	0x01101104	Command register, see commands below
2	NIM output	RW		NIM output control: bits 15..0: NIM outputs, 31..16: NIM output function control, see below
3	NIM input	R		bits 15..0: status of NIM inputs, bits 31..16: latched NIM inputs
		W		bits 15..0: reset latch for corresponding NIM inputs (0xFFFF resets latch for all 16 bits)
4	example 1	RW		example 32-bit read-write register
5	example 2	RW	0x01100810	bits 15..0 control example scaledown, bits 31..16 control example delay, see NIM output function control
6	timestamp	R	0x01101105	read the 20 MHz timestamp clock counter
7	LVDS input	RW		ECL/LVDS inputs, works same as register 3
8	Flash programmer	RW		controls the 0xABCD ASMI Flash programmer. For use with "srunner_vme".
9	TDC4	RW		experimental 4 channel TDC. on read, reads the TDC FIFO. on write, resets the TDC.
10	TSC4 routing	RW	0x01100818	Controls signal routing for the 4 TSC units: bits 0..4 select one of the 32 channels routed into TSC[0], 5..10 for TSC[1], and so forth.
11	TSC4 fifo data	RW	0x01100818	Write: TSC4 reset (data fifo, etc). Read: reads the TSC4 data FIFO.
12	TSC4 control	W	0x01100818	on write: writes 32 bits of tsc_control
		R	0x01100818	on read: bits 14..0 - number of words in the TSC4 data fifo, bit 15 - TSC4 FIFO overflow latch (cleared by TSC4 reset), bits 31..16 - readback of tsc_control[31..16]
13	Serio control	R	0x01100818	on read: bit 0 - SMB_SCK, bit 1 - SMB_SDA output, bit 2 - SMB_SDA output enable, bit 3 - SMB_SDA input, bit 4 - SER_TX_LOCKn, bit 5 - SER_RX_LOCKn, bit 6 - SER_RX_CLK, bits 20..16 - SER_RX[4..0]
		W	0x01100818	on write: bit 0 - SMB_SCK, bit 1 - SMB_SDA output, bit 2 - SMB_SDA output enable, bit 6 - SER_TX_CLK, bits 12..8 - SER_TX[4..0]
14	SDRAM control	RW	0x01100818	Writes 32 bits of sdram_cmd: bits 23..0 - ADDR, 26..24 - CMD, 27..28 - DM
15	SDRAM data	RW	0x01100818	Write: low 16 bits of SDRAM input data, Read: bits 15..0 - read back 16 bits of SDRAM input data, bits 31..16 - 16 bits of SDRAM output data.
16	input inversion	RW	0x01121113	invert input signals: low 16 bits is NIM inputs, high 16 bits are LVDS inputs
17	scalers input routing	RW	0x01121123	routing of inputs to the scalers
18..47	unused		0x01120523	used to be scalers for NIM and LVDS. Moved to registers 60..63, see Scaler
48	example gate and delay generator	RW	0x01100914	example gate and delay generator: bits 15..0 control output pulse delay, bits 31..16 control output pulse width, also see NIM output function control, see #Example gate and delay generator
49	pulser	RW	0x01100928	pulse generator 32 bits 10 ns time step, see NIM output function control, see #Pulser
50	T2KM11 control	RW	0x01101018	see T2K M11 trigger control
51	T2KM11 FGD trigger	RW	0x01101018	T2K M11 FGD trigger output delay and width control
52	T2KM11 PIANO MA PMT trigger	RW	0x01101018	T2K M11 PIANO MA PMT gate delay and width control
53	trigger counter	R	0x01101018	NIM_IN[1] trigger counter
54	trigger timestamp	R	0x01101018	NIM_IN[1] trigger timestamp
55	unused		0x01120523
55	control external timestamp reset bitmap	RW	0x01120805	external timestamp reset is armed by writing into this register a 32-bit bitmap, 1 bit per external input: 0x80000000 is LVDS_IN_SYNC[15] through 0x00000001 is NIM_IN_SYNC[0]. Once armed, a signal on an enabled input will generate the ext_ResetTS signal resetting the timestamp counter. This control register is also reset, disarming the reset function.
56	S1249 control	RW	0x01101118	see S1249 trigger control
57	ALPHA control	RW	0x01120227	see ALPHA trigger control
58	DRAGON control	RW	0x01120302	see DRAGON trigger control
59	TRINAT control	RW	0x01121013	see TRINAT trigger control
60	Scaler status	R	0x01120523	see Scaler
61	Scaler data	R	0x01120523	see Scaler
62	Scaler channel disable bitmap	RW	0x01120523	32 bits, one per scaler input
63	Scaler latch enable bitmap	RW	0x01120523	32 bits, one per scaler input

Command register

0 - no-op
1 - reset
2 - FPGA reconfigure/reboot
3 - 0x01101105 - reset timestamp counter
4 - 0x01101105 - reset scalers
5 - 0x01101105 - latch scalers
6 - 0x01131019 - select LED routing according to bits 31,30

Functional units

NIM and LVDS/ECL Inputs

There are 16 NIM inputs via LEMO connectors on the front panel and there are 16 LVDS/ECL inputs via a 34-pin header connector on the board. This 34-pin connector is compatible with the LeCroy 4616 NIM-to-ECL converter.

Inside the FPGA firmware, inputs are split into 4 groups:

NIM_IN/LVDS_IN are the renamed/renumbered input pins;
NIM_IN_LATCH/LVDS_IN_LATCH are the latched input pins - input transition from 0 to 1 sets the latch bit to 1. latch bits are reset back to zero by writing to a VME register;
NIM_IN_SYNC/LVDS_IN_SYNC are the input signals synchronized with the 100 MHz system clock. This incurs a 20-30 ns delay (FIXME: need confirmation!) and introduces a 10 ns jitter relative to the unsynchronized NIM_IN/LVDS_IN signals;
NIM_IN_PULSE/LVDS_IN_PULSE are the synchronized signals with pulse length reduced to 1 clock.

There are 2 VME registers to control the inputs - one for NIM inputs, one for LVDS/ECL inputs. Reading the register returns 16 bits of NIM_IN/LVDS_IN (unsynchronised) and 16 bits of NIM_IN_LATCH/LVDS_IN_LATCH. Writing a 16-bit bit pattern clears the corresponding latch bits, i.e. write of 0x0000FFFF will clear all latch bits, write 0x00000001 will clear only the first latch bit.

LEDs

Onboard LEDs:

VME LED - indicates when module is addressed
RJ45 LEDs - not used
16 NIM input LEDs (green) (red on Rev2 boards - assembly mistake, wrong part, should be green)
16 NIM output LEDs (red)

The LEDs next to the NIM input and output connectors are normally connected to the respective NIM connectors. The green NIM input LEDs reflect the state of the NIM inputs, the red NIM output LEDs reflect the state of the NIM outputs.

The function of the LEDs can be changed by writing 0xN0000006 into the command register 3 (command 6):

n=0x0: function a: default routing: LEDs are connected to NIM inputs and NIM outputs
n=0x6: function b: LVDS inputs 0..7 are connected to the 8 upper "NIM input" LEDs, LVDS inputs 8..15 are connected to the 8 upper "NIM output" LEDs. Note that most IO32 boards do not have LEMO connectors installed on the upper 8 inputs and upper 8 outputs, making the corresponding LEDs available for use with the LVDS inputs.
n=0x8: function c: reserved
n=0xC: function d: reserved

NIM Outputs

There are 16 NIM outputs via LEMO connectors on the front panel of the module. Depending on firmware version, they can have arbitrary functions.

In the base firmware, the 16 NIM outputs are split into 4 sections, controlled by the nimout[31..0] VME register:

FP_NIMOUT[3..0]: multifunction outputs described in the table below
FP_NIMOUT[7..4]: "OR" of nimout[7..4] and experiment specific output functions (T2KM11, S1249, DRAGON, ALPHA)
FP_NIMOUT[15..8]: "OR" of nimout[15..8] and experiment specific output functions (ALPHA).

The multifunction outputs are controlled by nimout[31..16] in 4 groups of 2 bits selecting one of the 4 possible output functions of each output:

FP_NIMOUT[0] - 0x00030000 - bits 17,16
FP_NIMOUT[1] - 0x000C0000 - bits 19,18
FP_NIMOUT[2] - 0x00300000 - bits 21,20
FP_NIMOUT[3] - 0x00C00000 - bits 23,22

Output	FwRev	Func	Description
FP_NIMOUT[0]	0x01100818	0	nimout[0]
	0x01100818	1	clock 20 MHz - for use as VF48 clock
	0x01100818	2	NIM_IN_LATCH[0] - for use as alternate "daq busy" latch (use NIM_IN_LATCH[1] as primary daq busy latch)
	0x01120806	2	CMD_ResetTS - output the timestamp reset signal
	0x01100914	3	VME AS
FP_NIMOUT[1]	0x01100818	0	nimout[1]
	0x01101018	1	(NIM_IN_LATCH[1] OR nimout[1]) - for use as "daq busy" with software busy control
	0x01101018	2	pll1_40 MHz clock
	0x01100914	3	VME DS0
FP_NIMOUT[2]	0x01100818	0	nimout[2]
	0x01100818	1	scaledown2 - prescaler of NIM_IN[2]
	0x01100928	2	pulser
	0x01100914	3	VME DS1
	0x01120706	3	NIM_IN[2] synchronized with the clock 40 MHz for V1190 TDC
FP_NIMOUT[3]	0x01101118	0	clock 40 MHz for V1190 TDC
	0x01101118	1	nimout[3]
	0x01100914	2	output of example L222 delay generator
	0x01100914	3	VME DTACK (NOT the VME bus DTACK!)
FP_NIMOUT[15..4]	0x01100818		nimout[15..4]

DAQ Busy

FP_NIMIN[1] and FP_NIMOUT[1] implement a "DAQ busy" latch with software busy control, trigger timestamp and trigger counter.

Pulser

FP_NIMOUT[2] function 2 implements a programmable pulser. Output pulses are 100 ns wide with period programmed in the pulser VME register 49 in units of 10 ns plus 10 ns. Value 20 is 210 ns, value 40 is 410 ns, etc.

Scalers

The base firmware design allows for 32 scalers (16 NIM and 16 LVDS/ECL). Each scaler is implemented as a 28-bit LPM_COUNTER clocked directly by the corresponding asynchronous NIM_IN/LVDS_IN signal. Maximum counting rate is requested through "clock" definitions in the Quartus VME-NIMIO32.sdc file (300MHz) and the actually achieved counting rate is reported by Quartus in the "Fmax Summary" report.

Scaler data is written into a FIFO buffer (LPM_FIFO) and the module can be used as a timesliced multi-channel scaler.

To permit accurate measurement of counting rates, one of the input channels is replaced by a 20 MHz clock (see table below).

The scalers are controlled by the VME command register:

command 4 "reset scalers": reset all counters to zero, clear the data FIFO
command 5 "latch scalers": initiate the scaler readout sequence, write the data FIFO.

The scaler readout sequence is initiated by VME command or by signal on one of the inputs:

switch counting from the 28-bit "A" counter to the 4-bit "B" counter
transfer the contents of the 32 "A" counters into the data FIFO (except for disabled counters)
clear the "A" counters
switch counting from the "B" counter back to the "A" counter.
transfer the contents of the "B" counters into the data FIFO.
clear the "B" counters

Readout time of "A" counters is 360 ns. During this time, counting is done by the 4-bit "B" counters, which can only count from 0 to 15, limiting the maximum sustained counting rate to just above 40 MHz. The "B" counters do not wrap-around, but stay at the overflow value of 0xF making it easy to detect if the overflow condition happens in actual use.

If needed, it is easy to increase the width of "B" counters in the firmware, with the restriction that the width of "A" plus width of "B" (28+4=32) should not exceed the 32 bit width of the VME bus. Extension beyond the 32 bits total requires implementation of additional VME registers to read the extra bits.

The scaler readout sequence is intended to be deadtime-less and loss-less, with limitations:

width of 4-bit "B" register is intended for loss-less counting of the 40MHz V1190 TDC clock
if more than 16 pulses arrive within 360ns and it happens during the readout of "A" counters, the "B" counter can overflow (value 0xF) (but not wrap-around).
when switching between the "A" and "B" counters, if a pulse arrives exactly at the switching time, it may be lost. The probability of this happening is low, i.e. for a 4 MHz pulse train, the measured probability of losing one count per scaler readout is below 1e-3. If scalers are read once per second, loss is 1 count out of 4e6 with probability 1e-3.
if the input signals are in the same clock domain as the IO32 main clock (20MHz VF48 clock, 40MHz V1190 clock), there is no count loss when switching between "A" and "B" counters. The 40MHz V1190 clock is counted deadtime-less, loss-less.

VME registers:

Scaler status: bitmap: 0x8000 FIFO empty, 0x4000 FIFO overflow, 0x2000 busy, 0x0FFF FIFO word count.
Scaler data: VME read pops data from the FIFO. Format is 0xAAAAAAAB (28 bits of A counter, 4 bits of B counter). Values of "A" and "B" counters should be added by software.
Scaler channel disable bitmap: one bit per scaler, if set, data for this scaler is not written to the data FIFO. For example, default value 0 saves all 32 counters, FIFO word count increments by 32. Value 0xFFFFFFFC (!0x3) saves only the first 2 counters, FIFO word count increments by 2.
Scaler latch enable bitmap: one bit per scaler, if bit is set, signal on this input causes a "latch scalers" command)
Scaler routing register:

Due to FPGA space limitations, the base firmware design implements only 16 out of 32 possible scalers. All 16 NIM inputs, 16 ECL inputs and 16 NIM outputs can be routed into the 16 scalers using the scaler routing register 17:

Register 17: scaler routing
bits:
31..16: reserved for routing of inputs of scalers 31..16 (4x4 bits)
15..12: input of scalers 15..12 (4 bits)
11.8: input of scalers 11..8 (4 bits)
7..4: input of scalers 7..4 (4 bits)
3..0: input of scalers 3..0 (4 bits)

scaler 31 is permanently connected to the 20 MHz clock (the TSC4 internal or external clock)

Routing is done in blocks of 4 channels:
value 0: NIM inputs 3..0
value 1: NIM inputs 7..4
value 2: NIM inputs 11..8
value 3: NIM inputs 15..12
value 4: ECL inputs 3..0
value 5: ECL inputs 7..4
value 6: ECL inputs 11..8
value 7: ECL inputs 15..12
value 8: NIM outputs 3..0
value 9: NIM outputs 7..4
value 0xA: NIM outputs 11..8
value 0xB: NIM outputs 15..12
value 0xC: not connected
value 0xD: not connected
value 0xE: not connected
value 0xF: not connected

Examples:
- to connect 16 NIM inputs to the 16 scalers, write register 17 value 0x3210
- to connect 16 ECL inputs to the 16 scalers, write register 17 value 0x7654

Maximum counting rate according to Quartus is above 300 MHz.

To read the scalers:

reset the scalers (write VME command register 1 value command 4 "reset scalers").
the scalers are now counting
latch the scaler values (write VME command 5 "latch scalers")
poll the VME register "scaler status"
read the scalers data from the VME register "scaler data" until FIFO is empty
in the software, add the values of "A" and "B" counters
to compute counting rates, divide the counter value by the value of the 20MHz clock counter.
to compute total counts per run, add counter values in software.

Software:

test_VMENIMIO32 --addr 0x100000 --readscalers
C++: uint32_t buf[32]; int nwords = io32->io32->ReadScaler(buf, 32);

Scalers (pre FwRev 0x01121128)

Due to FPGA space limitations, the base firmware design does not actually implement all possible 32 scalers. Actually availble scalers are:

Firmware for Rev0 boards usually implements 16 NIM scalers, no ECL scalers, 20MHz clock is routed into the last ECL scaler.
Firmware for Rev1 boards usually implements 4 NIM scalers and 16 ECL scalers, 20 MHz clock is routed into the last NIM scaler.
maximum counting rate according to Quartus is above 300 MHz.

Scalers (pre FwRev 0x01120523)

The base firmware design allows for 32 scalers, one for each of the inputs. Each scaler is implemented as a 32-bit LPM_COUNTER clocked by the corresponding asynchronous NIM_IN/LVDS_IN signal. The maximum counting rate is limited by the switching time of the synthesized FPGA LPM_COUNTER and can change between different firmware revisions. For each revision, the maximum counting rate corresponds to the "Fmax" value reported by the Quartus FPGA compiler.

To permit accurate measurement of counting rates, readout of scalers is done in conjunction with the timestamp counter. First one should latch the scalers and the timestamp by writing the "latch scalers" command to the command registers, then one can read the values of scalers and the value of the scaler latch timestamp. This ensures that all scalers are "read at the same time". To compute counting rates, one should divide the increment of the scaler counters by the increment of the timestamp clock counter. For more information and for a code example, please refer to the "software" section.

Scaler counting is inhibited for 3 clock counts (30 ns) while latching scaler data into VME registers. This is done to avoid data corruption if an input signal increments the LPM_COUNTER at the same time as it's contents is being latched into the VME register. Deadtime-less scaler readout requires more FPGA resources and is not implemented in the base firmware. It can be implemented by special request.

Due to FPGA space limitations, the base firmware design does not actually implement all 32 scalers. Actually availble scalers are:

0x01100818 - 16 32-bit scalers for LVDS, 4 32-bit scalers for NIM (only the first 4 channels). Fmax is 300 MHz
Rev0 firmware usually implements 16 NIM scalers, no ECL scalers
Rev1 firmware usually implements 4 NIM scalers and 16 ECL scalers
maximum counting rate according to Quartus is 300 MHz.

To read the scalers:

write "latch scalers" command 5 into VME command register 1
read the values of scalers from VME registers 16..47
read the latch timestamp from VME register 55
compute counting rate as the difference between scaler counts divided by the difference in timestamp values (20MHz clock)

Software:

test_VMENIMIO32 --addr 0x100000 --readscalers33
C++: uint32_t buf[33]; ReadScalers33(buf); // buf[32] is the 20MHz timestamp

Timestamp

Register 6 is the 20 MHz timestamp counter. Reset by software (command 3) or by external input (write bitmap into register 55).

The timestamp counter is used by the DAQ trigger timestamp register and by the TSC4 block

Timestamp synchronization

To synchronize timestamps across multiple IO32 modules one should provide an external timestamp clock (20 MHz or 40 MHz) and an external timestamp reset pulse.

Provisions are made for simple synchronization between 2 IO32 modules:

40 MHz clock from the "master" module (FP_NIMOUT[3] function 0) is sent to the "slave" module (any NIM or ECL input)
timestamp reset from the "master" module (FP_NIMOUT[0] function 2) is sent to the "slave" module (any NIM or ECL input)
"slave" module is configured for external TSC clock: register 10 set bits 31 and 30 (external clock divided by 2), set bits 24..20 to the clock input (see section on TSC4)
to issue a timestamp reset
- arm the reset on the slave module: write register 55 with the correct encoding of timestamp reset input (see section on TSC4)
- issue command 3 (reset timestamp counter) to the master module (it will reset it's own timestamp and send the reset signal to the designated output - the armed slave module will see the reset signal, reset the timestamp and disarm the reset function.

NIM_IN[1] DAQ trigger

trigger counter
trigger timestamp

TSC4

The 4-channel timestamp device records the time of input pulses (leading edges). The time counter (TSC) is common with the trigger timestamp.

The time counter (TSC) can use the internal 20 MHz clock (the VF48 20 MHz clock which is the V1190 40 MHz clock divided by 2) or an external clock from any of the NIM or LVDS inputs.

When using 2 IO32 modules, the recommended configuration is to use the 40 MHz V1190 clock from one module to drive the time stamp clock on the other module using the clock selector 3 (external clock divided by 2).

The time stamp counter can be reset to zero by a VME command (see VME command register) or by an external signal on any NIM or LVDS input. Selection of the reset signal is done by the "TSC external reset mask" register (32 mask bits, one per input). When a reset signal is received, the time stamp counter is reset to zero and the external reset mask register is cleared. (reset only works once).

register 10 tsc4_route register bits:
- 4..0 - input of TSC[0]
- 9..5 - input of TSC[1]
- 14..10 - input of TSC[2]
- 19..16 - input of TSC[3]
- 24..20 - input of TSC external clock
- 25..29 - not used
- 31..30 - clock selector: 0=internal_20MHz, 1=reserved, 2=external clock from selected input, 3=external clock from selected input divided by 2 to form 50% duty cycle (for use with the 40 MHz V1190 clock)
- the 5 bit selection of TSC input is: values 0..15 - NIM inputs, values 16..31 - LVDS inputs

register 55 TSC external reset mask:
- bits 15..0 - mask of NIM inputs
- bits 31..16 - mask of LVDS inputs

The TSC4 output FIFO is 512 of 32-bit words.

TDC4

This function is experimental and not available for general use.

Example scaledown (prescaler)

The example scaledown (prescaler) is connected to NIM input 2 and is controlled by the lower 16 bits of VME register 5. The output of the scaledown can be routed to NIM output 2 (see NIM output functions). Scaledown value 0 makes the output follow the input. Value 1 makes the output skip 1 input pulse (scale down factor of 2), value 2 makes the output skip 2 input pulses (scale down factor of 3), and so forth.

Example gate and delay generator

The example delay and gate generator triggers on the leading edge of NIM_IN[3] input pulse and after a programmed delay, produces an output pulse of programmed width in NIM_OUT[3], output function 2.

The delay and width registers are 16 bit wide, running at 100 MHz, the delay range is about 100 ns to 650 us, the width range is 20 ns to 650 us, both in 10 ns increments. There is a jitter of exactly 10 ns between the input and output pulse caused by sychronization of the input pulse to the 100 MHz clock.

The minimum delay internally is 20 ns, externally about 100 ns because of delays through the VME board and through the FPGA input and output pins.

Special functions

The IO32 firmware is structured in a way to make it easy to implement special logic customised for different experiments and needs. This section describes some of these different functions.

All NIM and LVDS inputs are available for use by all special functions. (the first batch of IO32 boards has LEMO connectors only for NIM inputs 3..0).

NIM outputs are available to the special functions in 3 banks:

outputs 3..0 (4 total) are reserved for the standard functions ("busy" latch, 40MHz V1190 clock, 20MHz VF48 clock, etc).
outputs 7..4 (4 total) - nimout[7..4] is ORed with the outputs of all special functions, each gated by it's own "enable" bit (usually bit31 of the corresponding control register).
outputs 15..8 (8 total) - nimout[15..8] is ORed with ... (same thing as above)

Each of the NIM output "OR" functions has a large number of spare inputs available for adding new special functions.

To create a new special function, create the corresponding control register (cut-and-paste another register). Create a 32-bit "WIRE" function for mapping bits of this register to your functions. Use bit31 as "enable" bit for the NIM outputs. Create a 16-bit "WIRE" function to map outputs of the special function to the NIM outputs. Add your outputs to the NIM "OR" function through your enable signal. NIM inputs can be used directly or they can be mapped using "WIRE" functions.

VME Trigger latch and busy function

VME trigger latch and busy function uses NIM input 1 and NIM output 1. The leading-edge-sensitive NIM input 1 accepts a trigger pulse which is latched into the NIM input latch bit 1. When enabled, the value of latched NIM input 1 ORed with the value of the NIM output register 2 bit 1 is routed to the NIM output 1 ("busy signal"). The trigger pulse also latches the current time stamp into register 54 and increments the trigger counter in register 53. User frontend program shall poll bit 1 in the NIM input latch to detect an event. When ready for the next event, clear the busy latch by writing 0x00020000 into the NIM input register 3. At the end of run, set the "busy" NIM output 1 by setting bit 1 in the NIM output register 2 (bitmap value 0x00000002). At the begin of run, clear the "busy" output by clearing bit 1 in the NIM output register 2.

T2KM11 trigger control

Registers 50, 51, 52. Provides trigger and busy distribution for the M11 VME TOF system, the FGD detector and the PIANO detector. Provides programmable gate signals for the PIANO ADC gates. For more information, see T2KM11 elog and the IO32 schematics.

NIM_IN[1] - VME trigger (ADC gate daisy chained through the V1190 TDC and V792 ADC)
NIM_IN[3] - FGD CTM trigger input, at least 100 ns wide
NIM_IN[4] - FGD busy (from FGD SCM through TTL level converter)
NIM_IN[5] - PIANO busy
NIM_IN[6] - spare busy
NIM_IN[7] - internally connected to VME busy
NIM_IN[8..15] - scaler inputs 8 through 15

NIM_OUT[1] - VME busy output (n/c)
NIM_OUT[2] - broken, do not use
NIM_OUT[4] - DAQ busy - connect to main coincidence VETO input. Logic level 1 should block the ADC gate signal.
NIM_OUT[5] - FGD SCA STOP (with internal programmable delay) - connect to FGD SCM TTL trigger input through TTL
level converter
NIM_OUT[6] - PIANO MA PMT gate (with internal programmable delay and width)
NIM_OUT[7] - pulser output

T2KM11 trigger control register:

bit 31: t2km11_ena
bit 30: int_busy_disa
bit 29..11: not used
bit 10: ctm_bypass_ena
bit 9: ctm_in_ena
bit 8: tof_in_ena
bit 7..4: busy_ena[3..0]
bit 3: x3_ena
bit 2: x2_ena
bit 1: piano_mapmt_out_ena
bit 0: fgd_out_ena

S1249 trigger control

Register 56. Provides TDC trigger, ADC gate, ADC fast clear and busy signals for the S1249 electronics. For more information, see the S1249 elog and the IO32 schematics.

NIM inputs:
0 - s1249_gate_async - muon gate (100ns minimum width)
1 - s1249_daq_trig_async - DAQ trigger (starts the event readout)
2 - s1249_accept_async - EVTR - trigger accept pulse (edge-triggered)
3 - s1249_pileup_async - trigger reject pulse (edge-triggered)

NIM outputs:
0 - not used
1 - DAQ busy - should veto EVTR and the muon gate
2 - pulser (not used)
3 - 40MHz clock to V1190 TDC
4 - s1249_tdc_stop - fan out to V1190 TDC "trigger", IO32 input 1 "daq trigger" and to "postamp" pulser input
5 - s1249_fast_clear - fan out to all V792 "fast clear" inputs, should veto muon gate and EVTR
6 - s1249_gate_start - confirmation of muon gate start
7 - s1249_gate_x_async - extended muon gate

Register 56 bits:
31 - s1249_ena - enable S1249 NIM outputs
30 - s1249_force_accept - jams s1249_accept_async to "1", s1249_pileup_async still works (FIXME?)
29..16 - not used
15..0 - s1249_gate_x_width - muon gate extension in 25ns clocks

Register 51 bits: (t2km11_fgd_delay)
31..16: not used (was s1249_tdc_stop width, now fixed at 4 clocks)
15..0: s1249_tdc_stop delay (25ns units)

Register 52 bits: (t2km11_piano_delay)
31..16: s1249_fclear width (25ns units)
15..0: s1249_fclear delay (25ns units)

ALPHA trigger control

Register 57. Provides clock, trigger and busy distribution for TTC and VF48 modules, (in future) provides Si detector multiplicity trigger.

NIM inputs:
0 - alpha_veto1 (veto multA*B*C)
1 - alpha_veto2 (veto multAA*BB*CC), also alpha_ext_trig_in if enabled
2 - multA
3 - multB
4 - multC
5 - multAA
6 - multBB
7 - multCC
8..15 - VF48 busy signals (from VF48 AUX_OUT)

NIM outputs:
0 - spare 20 MHz clock
1 - not used
2 - programmable pulser (if enabled)
3 - not used
4 - busy output (to scope)
5 - trigger output (to scope)
6 - 20MHz clock (TTC1)
7 - 20MHz clock (TTC2)
8 - trigger output (TTC1)
9 - trigger output (TTC2)
10 - trigger output without VF48 busy for scaler
11 - VF48 busy output (to scope)

Register 57 bits:
31 - alpha_ena - enable ALPHA NIM outputs
30 - alpha_busy_force - ORed with the alpha_veto_in
29 - alpha_pulser_enable - send example pulser (register 49) into the trigger output
28 - alpha_veto2_pol
27 - alpha_veto1_pol
26 - alpha_veto2_ena
25 - alpha_veto1_ena
24 - alpha_ext_trig_ena
23..16 - not used
15..8 - alpha_busy_gate_width[7..0] - minimum busy signal width is set by width = 50ns + N*400ns.
7..0 - alpha_vf48_busy_ena[7..0] - enable busy signals from the 8 VF48 modules

DRAGON trigger control

Register 58. Provides trigger and ADC gate for Dragon head and tail DAQ.

NIM inputs:
0 - dragon cross timestamp
1 - dragon daq trigger input
2 - dragon_cfd1
3 - dragon_cfd2

ECL inputs:
7..0 - tail trigger inputs

NIM outputs:
0 - not used
1 - daq busy
2 - not used
3 - not used
4 - IO32 trigger
5 - TDC trigger (delay control through t2km11_fgd_delay register)
6 - ADC gate (CFD ORed with L222, width control through t2km11_fgd_delay register)
7 - tail grand OR output

Register 58 bits:
31 - dragon_ena - enable DRAGON NIM outputs
30 - dragon_busy_en - enable internal busy
29..24 - not used
23..16 - tail trigger input enable bit mask
15..8 - not used
7..0 - read: tail trigger input latch (tail trigger input signals are latched by NIM_IN[1] - dragon daq trigger input)

TRINAT trigger control

Provides trigger, ADC gate, TDC stop for TRINAT VME ADCs and TDC.

Firmware: 0x01121212

Registers:

reg 59: general control
reg 51: (t2km11_fgd_delay): low 16 bits: TDC trigger delay and Ion MCP coincidence gate; high 16 bits: ADC gate width
reg 52: (t2km11_piano_delay): high 16 bits: IonMCP ADC gate width

NIM inputs:
0 - not used
1 - DAQ trigger
2 - trigger input 1
3 - trigger input 2
4 - not used
5 - ion MCP
6 - not used (software: IonMCP trigger latch bit)
7 - not used

NIM outputs:
0 - not used
1 - DAQ busy
2 - not used
3 - not used
4 - IO32 trigger
5 - TDC trigger (stop), delay control through low 16 bits of t2km11_fgd_delay register, width 50 ns
6 - ADC gate, width control through high 16 bits of t2km11_fgd_delay register
7 - Ion MCP ADC gate, width control through high 16 bits of t2km11_piano_delay register.

Register 59 bits:
31 - trinat_ena - enable TRINAT NIM outputs
30 - trinat_ionmcp_adc_ena - enable gate to the Ion MCP ADC
29 - trinat_pulser_ena - enable pulser into the TRINAT trigger
28 - trinat_ionmcp_pulser_ena - enable TDC trigger pulses the Ion MCP ADC gate
27..0 - not used

TRINAT 2014 trigger control

This documentation is for VME-NIMIO32 firmware 0x01140605.

Provides multiple triggers based on SC, EMCP, RMCP, LED and UV signals, provides SC ADC gate (short and long), EMCP ADC gate, RMCP ADC gate, and TDC stop for TRINAT VME DAQ.

Trigger definitions:

A - scaled down SC
B - (scaled down SC) * LED
C - EMCP * SC
D - EMCP * UV
E - RMCP * UV_DELAYED
FF - (scaled down EMCP) fires the rmcp_coincidence_gate (about 5 usec), if EMCP fires inside the gate, the gate is restarted.
F - RMCP * rmcp_coincidence_gate
RR - RMCP * rmcp_gate -> fires the RMCP ADC gate. rmcp_gate length is same as rmcp_coincidence_gate (about 5 usec)

Trigger actions:

A -> SC ADC short gate, VF48, rmcp_coincidence_gate, TDC, DAQ trigger
B -> SC ADC long gate, rmcp_coincidence_gate, TDC, DAQ trigger
C -> SC ADC short gate, EMCP ADC gate, VF48, rmcp_coincidence_gate, TDC, DAQ trigger
D -> SC ADC short gate, EMCP ADC gate, rmcp_coincidence_gate, TDC, DAQ trigger
E -> RMCP ADC gate, rmcp_coincidence_gate (harmless?), TDC, DAQ trigger
F -> same as (E)
RR -> RMCP ADC gate

NIM inputs:
0 - not used
1 - DAQ trigger
2 - SC1
3 - SC2
4 - EMCP
5 - RMCP
6 - LED
7 - UV
8 - timestamp reset
9 - UV delayed for RMCP trigger (E)

NIM outputs:
0 - 20 MHz clock to VF48
1 - DAQ busy
2 - not used
3 - 40 MHz clock to second IO32 timestamp clock input
4 - TDC trigger (stop)
5 - SC ADC gate (short and long)
6 - EMCP ADC gate
7 - RMCP ADC gate
8 - IO32 DAQ trigger
9 - VF48 trigger
10 - timestamp reset 1
11 - timestamp reset 2
12 - SC1+SC2
13 - SC scaledown
14 - RMCP coincidence gate
15 - not used

Register 48 bits: (GDG example gate and delay generator)
7..0 - SC ADC gate width
15..8 - SC ADC long gate width (SC*LED)
23..16 - EMCP ADC gate width
31..24 - RMCP ADC gate width

Register 51 bits:
15..0 - TDC trigger delay (length of rmcp_gate and rmcp_coincidence_gate)
23..16 - Trigger A SC scaledown
31..24 - Trigger B SC*LED scaledowm

Register 52 bits:
15..0 - not used
23..16 - not used
31..24 - Trigger F EMCP*RMCP scaledown (EMCP scaledown)

Register 59 bits:
31 - trinat_ena - enable TRINAT NIM outputs
27..14 - not used
13..8 - enable bits for triggers A through F
7 - not used
6 - latched RMCP gate (tells us to read the RMCP ADC) (trigRR)
5..0 - latched bits for triggers A through F

TRINAT 2018 trigger control

This documentation is for VME-NIMIO32 firmware 0x01180914.