VME-NIMIO32

VME-NIMIO32 - general purpose VME FPGA board

References

• [1] VME-NIMIO32 (REA 131) project page on edev.triumf.ca
• [2] Rev0 board schematics
• [3] Rev1 board schematics
• [4] Svn repository for VME-NIMIO32 firmware
• [5] Misc documentation

General characteristics

Photos

Available hardware

• Altera cyclone 1 FPGA: EP1C6Q240C6N
• Serial flash for FPGA configuration: Altera EPCS16
• External SDRAM memory
• VME interface: VME-D[31..0] bidirectional, VME-A[19..0] input only, DTACK output, no BERR, no RETRY/RESP. VME-A[31..20] input only connected only to address decoder FPGA (Altera MAX-something CPLD). This permits all single-word transfer modes, 32-bit DMA (BLT32) and 2eVME DMA (only drives D-lines, but still faster than BLT32). 64-bit DMA (MBLT64) and 2eSST are impossible.
• 16 NIM outputs
• 16 NIM inputs
• 16 ECL/LVDS inputs compatible with LeCroy 4616 ECL/NIM/ECL converter. (Except: Rev0 boards have an incompatible ECL connector)
• 16 "red" and 16 "green" LEDs
• 1 "VME access" LED
• 1 RJ-45 high speed serial I/O interface

Firmware functions

Generic IO32 firmware can include a number of functions. Not all functions are available all at the same time due to resource limitations on the FPGA. Refer to VME Registers for detailed information.

• 20 MHz 32-bit timestamp clock register
• 16 bits of NIM output register (first 4 NIM outputs are multi-function)
• 16 bits of NIM and 16 bits of ECL/LVDS input register with latch function.
• 32 scalers 32 bit, up to 300 MHz counting rate
• 4 timestamp registers using the 20 MHz timestamp clock. Any of the 32 inputs can be routed into any of the 4 timestamp registers.
• trigger counter, trigger timestamp functions for NIM_IN[1] and DAQ busy functions for FP_NIMOUTPUT[1].
• 20 MHz clock for VF48 ADC and VT48 TDC; 40 MHz clock for V1190 TDC.
• example pulse generator (programmable period 32 bit 10 ns, fixed width 100 ns)
• example L222-like delay and gate generators (NIM_IN[2]/NIM_OUT[2])
• experimental 4-channel TDC. 10 ns base clock with 0.3 ns interpolation ladder (1 ns RMS).

Firmware update procedure

• obtain and build the VME programmer (srunner_vme) and IO32 test program (test_VMENIMIO32):
  • mkdir $HOME/packages
  • cd $HOME/packages
  • svn checkout https://ladd00.triumf.ca/svn/daqsvn/trunk/vme (username svn, password svn)
  • cd vme
  • make -k srunner_vme.exe srunner_vme_gef.exe test_VMENIMIO32.exe test_VMENIMIO32_gef.exe

• read the firmware revision from VME register 0:
  • ./test_VMENIMIO32_gef.exe --addr 0x100000 --read 0

• write Cyclone 1 FPGA firmware into the flash memory using the VME programmer:
  • ./srunner_vme_gef.exe -program -64 VME-NIMIO32.pof 0x100020
  • !!!be careful about using Rev0 pof files for Rev0 boards and Rev1 pof files for Rev1 boards!!!

• reboot the Cyclone 1 FPGA into the new firmware (requires "fpga-reset" mod):
  • ./test_VMENIMIO32_gef.exe --addr 0x100000 --reboot

Firmware revisions

Firmware for the main FPGA:

• firmware sources [[6]]
• firmware schematics [[7]]
• pof files for Rev0 [[8]]
• pof files for Rev1 [[9]]
• 0x01100810 - DO NOT USE - first revision of generic firmware
• 0x01100818 - [[10]] - fixed bug in scaler[6], rearranged TSC4 registers.
• 0x01100914 - Rev1 - [[11]] - add NIM output functions: VME AS and DS outputs, example L222 delay generator
• 0x01101018 - Rev1 - [[12]] - add simplified trigger timestamp functions, add 40MHz V1190 clock output, add T2KM11 trigger function
• 0x01101019 - Rev0 - [[13]] - same as 0x01101018
• 0x01101105 - Rev0 - svn rev 109 - fixed problem with long pulses being miscounted by scalers
• 0x01101118 - Rev1 - svn rev 112 - improved S1249 trigger, output V1190 40 MHz clock on NIM_OUT[3]
• 0x01110702 - Rev1 - svn rev 144 - updated the VME interface

Unless otherwise notes, because of FPGA resource constraints, firmware for Rev0 boards have 16 scalers for NIM_IN[15..0] and have no scalers for ECL/LVDS inputs. Firmware for Rev1 boards has 4 scalers for NIM_IN[3..0] and 16 scalers for ECL/LVDS inputs.

Firmware for the VME address decoder:

• firmware sources [[14]]
• board Rev0 [[15]]
• board Rev1 [[16]]

Manual

VME interface

Firmware 0x01100810 implements VME A24/D32 access only. A24 addresses 0x00N0xxxx are decoded, where "N" is the A24 base address set by rotary switch SW3 "ADDRESS 20-23". Rotary switches SW1 and SW2 should be set to "0".

VME accessible registers are listed in the table below. VME address offsets are register numbers multiplied by 4, i.e. CSR register 1 is at address 0x00N00004. Flash programmer register 8 is at 0x00N00020 (use this address with srunner_vme flash programmer).

LEMO connectors

The LEMO connectors are NIM inputs (NIM_IN) and NIM outputs (FP_NIMOUT). NIM inputs correspond to green LEDs on the lower/right side of the board, NIM outputs correspond to red LEDs on the upper/left side of the board. The NIM channel numbering on the board schematics is counterintuitive, the channel numbering described here is implemented by the IO32 firmware:

                  O15 | ... | O1 | I15 | ... | I1
 VME_LED | RJ45 | O14 | ... | O0 | I14 | ... | I0
--------------------------------------------------- PCB

Registers

Number	Name	Access	FwRev	Description
0	FwRev	RO		Firmware revision
1	CSR	RW	0x01100818	bit 0 - general reset, bit 1 - FPGA reconfigure
1	Command	RW	0x01101104	Command register, see commands below
2	NIM output	RW		NIM output control: bits 15..0: NIM outputs, 31..16: NIM output function control, see below
3	NIM input	R		bits 15..0: status of NIM inputs, bits 31..16: latched NIM inputs
		W		bits 15..0: reset latch for corresponding NIM inputs (0xFFFF resets latch for all 16 bits
4	example 1	RW		example 32-bit read-write register
5	example 2	RW	0x01100810	bits 15..0 control example scaledown, bits 31..16 control example delay, see NIM output function control
6	timestamp	R	0x01100810	on read, reads the timestamp clock and latches the 32 input scalers (causing scaler deadtime. FIXME!!!)
		W	0x01100810	on write, clears the 32 input scalers
6	timestamp	R	0x01101105	read the 20 MHz timestamp clock. No effect on scalers (use command 5 to latch the scalers and read latched timestamp from register 55
7	LVDS input	RW		ECL/LVDS inputs, works same as register 3
8	Flash programmer	RW		controls the 0xABCD ASMI Flash programmer. For use with "srunner_vme".
9	TDC4	RW		experimental 4 channel TDC. on read, reads the TDC FIFO. on write, resets the TDC.
10	TSC4 routing	RW	0x01100818	Controls signal routing for the 4 TSC units: bits 0..4 select one of the 32 channels routed into TSC[0], 5..10 for TSC[1], and so forth.
11	TSC4 fifo data	RW	0x01100818	Write: TSC4 reset (data fifo, etc). Read: reads the TSC4 data FIFO.
12	TSC4 control	W	0x01100818	on write: writes 32 bits of tsc_control
		R	0x01100818	on read: bits 14..0 - number of words in the TSC4 data fifo, bit 15 - TSC4 FIFO overflow latch (cleared by TSC4 reset), bits 31..16 - readback of tsc_control[31..16]
13	Serio control	R	0x01100818	on read: bit 0 - SMB_SCK, bit 1 - SMB_SDA output, bit 2 - SMB_SDA output enable, bit 3 - SMB_SDA input, bit 4 - SER_TX_LOCKn, bit 5 - SER_RX_LOCKn, bit 6 - SER_RX_CLK, bits 20..16 - SER_RX[4..0]
		W	0x01100818	on write: bit 0 - SMB_SCK, bit 1 - SMB_SDA output, bit 2 - SMB_SDA output enable, bit 6 - SER_TX_CLK, bits 12..8 - SER_TX[4..0]
14	SDRAM control	RW	0x01100818	Writes 32 bits of sdram_cmd: bits 23..0 - ADDR, 26..24 - CMD, 27..28 - DM
15	SDRAM data	RW	0x01100818	Write: low 16 bits of SDRAM input data, Read: bits 15..0 - read back 16 bits of SDRAM input data, bits 31..16 - 16 bits of SDRAM output data.
16..31	NIM scalers	R		read the latched value of NIM input scalers
32..47	LVDS scalers	R		read the latched value of LVDS input scalers
48	example L222	RW	0x01100914	example L222 delay generator: bits 15..0 control pulse delay, bits 31..16 control pulse width, also see NIM output function control
49	pulser	RW	0x01100928	pulse generator 32 bits 10 ns time step, see NIM output function control and T2K M11 trigger control
50	T2KM11 control	RW	0x01101018	see T2K M11 trigger control
51	T2KM11 FGD trigger	RW	0x01101018	T2K M11 FGD trigger output delay and width control
52	T2KM11 PIANO MA PMT trigger	RW	0x01101018	T2K M11 PIANO MA PMT gate delay and width control
53	trigger counter	R	0x01101018	NIM_IN[1] trigger counter
54	trigger timestamp	R	0x01101018	NIM_IN[1] trigger timestamp
55	scalers timestamp	R	0x01101105	time stamp latched by scalers latch command, read it together with scalers adn use as time base for precise scaler counting rate computations
56	S1249 control	RW	0x01101118	see S1249 trigger control
57..63	unused

Command register

• 0 - no-op
• 1 - reset
• 2 - FPGA reconfigure/reboot
• 3 - 0x01101105 - reset timestamp counter
• 4 - 0x01101105 - reset scalers
• 5 - 0x01101105 - latch scalers

Functional units

NIM and LVDS/ECL Inputs

There are 16 NIM inputs via LEMO connectors on the front panel and there are 16 LVDS/ECL inputs via a 34-pin header connector on the board. This 34-pin connector is compatible with the LeCroy 4616 NIM-to-ECL converter.

Inside the FPGA firmware, inputs are split into 4 groups:

• NIM_IN/LVDS_IN are the renamed/renumbered input pins;
• NIM_IN_LATCH/LVDS_IN_LATCH are the latched input pins - input transition from 0 to 1 sets the latch bit to 1. latch bits are reset back to zero by writing to a VME register;
• NIM_IN_SYNC/LVDS_IN_SYNC are the input signals synchronized with the 100 MHz system clock. This incurs a 20-30 ns delay (FIXME: need confirmation!) and introduces a 10 ns jitter relative to the unsynchronized NIM_IN/LVDS_IN signals;
• NIM_IN_PULSE/LVDS_IN_PULSE are the synchronized signals with pulse length reduced to 1 clock.

There are 2 VME registers to control the inputs - one for NIM inputs, one for LVDS/ECL inputs. Reading the register returns 16 bits of NIM_IN/LVDS_IN (unsynchronised) and 16 bits of NIM_IN_LATCH/LVDS_IN_LATCH. Writing a 16-bit bit pattern clears the corresponding latch bits, i.e. write of 0x0000FFFF will clear all latch bits, write 0x00000001 will clear only the first latch bit.

NIM Outputs

There are 16 NIM outputs via LEMO connectors on the front panel of the module. Depending on firmware version, they can have arbitrary functions.

In the base firmware, the 16 NIM outputs are connected to the lower 16 bits of the VME NIM output control register. In addition, the first 4 NIM outputs can have multiple functions selected by the upper 16 bits of the NIM output control register.

Upper 16 bits of the NIM output control register select the function of the output according to this table. The function bits are:

• FP_NIMOUT[0] - 0x00030000 - bits 17,16
• FP_NIMOUT[1] - 0x000C0000 - bits 19,18
• FP_NIMOUT[2] - 0x00300000 - bits 21,20
• FP_NIMOUT[3] - 0x00C00000 - bits 23,22

Output	FwRev	Func	Description
FP_NIMOUT[0]	0x01100818	0	nimout[0]
	0x01100818	1	clock 20 MHz - for use as VF48 clock
	0x01100818	2	NIM_IN_LATCH[0] - for use as "daq busy" latch
	0x01100818	3	NIM_IN_LATCH[0]
	0x01100914	3	VME AS
FP_NIMOUT[1]	0x01100818	0	nimout[1]
	0x01100818	1	NIM_IN_LATCH[1] - for use as "daq busy" latch
	0x01101018	1	(NIM_IN_LATCH[1] OR nimout[1]) - for use as "daq busy" with software busy control
	0x01100818	2	pll1_50MHz clock
	0x01101018	2	pll1_40 MHz clock
	0x01100818	3	pll1_50MHz clock
	0x01100914	3	VME DS0
FP_NIMOUT[2]	0x01100818	0	nimout[2]
	0x01100818	1	scaledown2 - prescaler of NIM_IN[2]
	0x01100818	2	scaledown2
	0x01100928	2	pulser
	0x01100818	3	scaledown2
	0x01100914	3	VME DS1
FP_NIMOUT[3]	0x01100818	0	nimout[3]
	0x01101118	0	clock 40 MHz for V1190 TDC
	0x01100818	1	delay3 - delay generator of NIM_IN_PULSE[3]
	0x01101018	1	pll1_40MHz clock
	0x01101118	1	nimout[3]
	0x01100818	2	delay3
	0x01100914	2	output of example L222 delay generator
	0x01100818	3	delay3
	0x01100914	3	VME DTACK (NOT the VME bus DTACK!)
FP_NIMOUT[15..4]	0x01100818		nimout[15..4]

Scalers

The base firmware design allows for 32 scalers, one for each of the inputs. Each scaler is implemented as a 32-bit LPM_COUNTER clocked by the corresponding asynchronous NIM_IN/LVDS_IN signal. The maximum counting rate is limited by the switching time of the synthesized FPGA LPM_COUNTER and can change between different firmware revisions. For each revision, the maximum counting rate corresponds to the "Fmax" value reported by the Quartus FPGA compiler.

To permit accurate measurement of counting rates, readout of scalers is done in conjunction with the timestamp counter. First one should latch the scalers and the timestamp by writing the "latch scalers" command to the command registers, then one can read the values of scalers and the value of the scaler latch timestamp. This ensures that all scalers are "read at the same time". To compute counting rates, one should divide the increment of the scaler counters by the increment of the timestamp clock counter. For more information and for a code example, please refer to the "software" section.

Scaler counting is inhibited for 3 clock counts (30 ns) while latching scaler data into VME registers. This is done to avoid data corruption if an input signal increments the LPM_COUNTER at the same time as it's contents is being latched into the VME register. Deadtime-less scaler readout requires more FPGA resources and is not implemented in the base firmware. It can be implemented by special request.

Due to FPGA space limitations, the base firmware design does not actually implement all 32 scalers. Actually availble scalers are:

• 0x01100818 - 16 32-bit scalers for LVDS, 4 32-bit scalers for NIM (only the first 4 channels). Fmax is 300 MHz
• Rev0 firmware usually implements 16 NIM scalers, no ECL scalers
• Rev1 firmware usually implements 4 NIM scalers and 16 ECL scalers
• maximum counting rate according to Quartus is 300 MHz.

TSC4

TBW

TDC4

This function is experimental and not available for general use.

Example scaledown (prescaler)

The example scaledown (prescaler) is connected to NIM input 2 and is controlled by the lower 16 bits of VME register 5. The output of the scaledown can be routed to NIM output 2 (see NIM output functions). Scaledown value 0 makes the output follow the input. Value 1 makes the output skip 1 input pulse (scale down factor of 2), value 2 makes the output skip 2 input pulses (scale down factor of 3), and so forth.

Example delay generator

TBW

Example L222 delay generator

The example L222 delay generator triggers on the leading edge of an input pulse and after a programmed delay, produces an output pulse of programmed width.

The delay and width registers are 16 bit wide, running at 100 MHz, the delay range is about 100 ns to 650 us, the width range is 10 ns to 650 us in 10 ns increments. Delay and width value of 0 disables the output. There is a jitter of exactly 10 ns between the input and output pulse caused by sychronization of the input pulse to the 100 MHz clock.

Internally, the L222 delay generator is implented as 2 cascaded lpm_counter megafunctions. The delay counter is started by the input pulse. It's cout pulse starts the width counter. The inverted cout output of the width counter forms the output of the generator.

If the generator is retriggered while the delay counter is still running, it will be restarted (as if the first trigger never happened). If the generator is retriggered while the width counter is still running, after the delay counter runs out, the width counter will be restarted, extending the output pulse (CHECKME: with a possible glitch) or producing a second output pulse.

SOFTWARE

test_VMENIMIO32.exe

Command line switches:

• --addr 0x00N00000 - module A24 base address, N is the setting of rotary switch SW3 "ADDRESS 20-23"
• --reboot - reboot/reconfigure the FPGA - use after upgrading the firmware or to reset the module to power-on state
• --read MMM - read register MMM, i.e. "--read 0" reads the firmware revision
• --write MMM VVV - write value VVV into register MMM, i.e. "--write 1 0" writes 0 to the CSR
• --readscalers - read the 32 input scalers, print scaler value, counting rate using the 20 MHz timestamp clock and counting rate using the computer clock.
• --pulsenim - pulse all NIM outputs
• --testbits MMM - test read/write of all 32 bits in register MMM
• --sleep SSS - sleep SSS seconds, i.e. "--sleep 1" sleeps for 1 second.
• --readtsc - example reading TSC register
• --readtsc4 - example reading and decoding the TSC4 data, programming TSC4 routing
• --nimout a b - select NIM output function (b) for NIM output (a)

New board startup and test procedure

• set VME address jumper A20-23 to "1", jumpers A24-27 and A28-31 to "0"
• connect JTAG USB blaster
• start Quartus programmer
• select correct USB blaster
• run "auto detect"
• flash the VME address decoder pof file into the EPM3032 part
• load the IO32 firmware sof file into the EP1C6 part
• run "vmescan_gef.exe", it should detect the IO32 board at A24 VME address 0x00100000, data should correspond to the sof file revision date code
• follow the firmware update instructions to flash the firmware pof file using the VME flash programmer
• confirm VME access LED is working
• confirm VME Data bus is okey: "./test_VMENIMIO32_gef.exe --addr 0x100000 --testbits 4"
• confirm FPGA reboot is working - "./test_VMENIMIO32_gef.exe --addr 0x100000 --reboot" prints 0xFFFFFFFF on the second read of firmware revision
• test NIM inputs: use NIM pulse generator or any NIM module inverted output, connect to each NIM input, observe that corresponding "green" LEDs is lighting up
• test ECL inputs: use NIM-ECL converter with a ribbon cable connected to the IO32 ECL input port, run "./test_VMENIMIO32_gef.exe --addr 0x100000 --readscalers", connect a NIM pulser to each ECL input, observe the counting rate on the corresponding scaler channel.
• test NIM output LEDs: "./test_VMENIMIO32_gef.exe --addr 0x100000 --pulsenim", observe all "red" LEDs are flashing, connect NIM outputs to NIM scaler, observe scaler counts at each LED flash
• unplug the board from VME, wait 10 sec, plug it back in, confirm that it is detected by vmescan (confirms the flash memory contents is good)

Board modifications

Rev0 modifications

• fpga-reset mod
• VME nAS mod
• second clock mod

Rev1 modifications

• fpga-reset-mod: see File:VME-NIMIO32 REV1 - nCONFIG PCB Modification.pdf

K.O.