PWB

From AgWiki
Revision as of 08:48, 3 October 2020 by Olchansk (talk | contribs) (→‎TODO)
Jump to navigation Jump to search

Links

Schematics

Manuals

Hardware

Power distribution

External power supply is:

  • +5V
  • +2V

Central board:

  • Ext_5V - F1 - RS1 (0.1R) - Fuse_5V
  • Ext_2V - F2/F3 - RS2 (0.1R) - Idig2-
  • LDO3 - Fuse_5V - DIG_3.3V - selfenable - ADM7154
  • LDO5 - Fuse_5V - DIG_2.5V - selfenable - ADM7154
  • LDO_CLN - Fuse5V - CLN_3.3V - selfenable -
  • VREG - DIG_3.3V - -2.5V SCA BIAS - selfenable - LTC1983ES6-3#TRMPBF
  • LDO7 - DIG_2.5V - DDR3_VRef - enable DDR3_1P5V, DDR3_PG, DDR3_VttEN
  • LDO2 - Idig2V- - DIG_1.1V - enable DIG_3.3V - LT3071
  • LDO6 - Idig2V- - DDR3_1P5V - enable DIG_3.3V - LT3083

SCA Wing:

  • Ext_5V - F1 - RS1 (0.3R) - Isca- - SCA_Fuse5V
  • LDO1 - SCA_Fuse5V - SCA3.3V - enable ADC_PWR_EN - ADM7154
  • LDO2 - SCA3.3V - ADC1.8V - selfenable - ADM7154
  • LDO3 - SCA3.3V - TRN2.5V - selfenable - LT1761
  • REF - SCA3.3V - 1.45V Vicm - selfenable - LM4121

Firmware

Firmware update

PWB firmware update is done using "esper-tool". 

esper-tool -v upload -f file.rpd http://pwbNN update factory_rpd
esper-tool -v upload -f file.rpd http://pwbNN update file_rpd

As of PWB firmware pwb_rev1_20200706_ko:

Permission to write into the flash memory is controlled by esper variables:

  • allow_write set to "false": "esper-tool upload" does a "verify": successful upload means epcq flash content is same as rpd file, error means content is not the same.
  • allow_write set to "true": "esper-tool upload" updates the epcq flash from the rpd file: for each data block, read the epcq flash, compare to rpd data, if different, erase the flash block, write the flash, read the flash, if there is a mismatch (bad write), return an error.
  • allow_factory_write set to "true" to enable writing to the "factory_rpd". DO NOT SET IT TO "true" UNLESS YOU MEAN TO UPDATE THE PWB BOOTLOADER. IF YOU FLASH BUM FIRMWARE or IF THERE IS A POWER OUTAGE WHILE UPDATING, THE PWB WILL BE BRICKED and has to be recovered by connecting a USB blaster.

To update firmware of multiple boards to correct version, recommended is the script "update_pwb.perl": 

cd ~/online/src
./update_pwb.perl pwb12 pwb06 pwb78

Factory page and user page firmware boot

Each PWB board has 2 firmware images: boot loader (factory page) and data acquisition (user page). On power up, the FPGA loads and runs the boot loader firmware from the factory page, later the control software reboots the FPGA into the data acquisition firmware from the user page.

Because loading defective firmware can brick https://en.wikipedia.org/wiki/Brick_(electronics) the PWB, new firmware images are loaded into the user page, if anything is wrong, the PWB can still boot from the factory page and one can use the firmware update function to load a good firmware image into the user page.

The boot loader firmware in the factory page is not intended for data acquisition, it is only meant to provide three functions:

  • boot enough hardware and software to communicate via the ethernet and the sata link
  • firmware update (both factory page and user page)
  • reboot into the user page

The boot loader firmware in the factory page is usually never updated unless absolutely needed to fix a problem with these three functions (i.e. compatibility with sata link communications).

If factory image is corrupted or the contains defective firmware and the PWB does not boot (no dhcp, no ping, no esper), the only way to unbrick it is by loading good firmware using a jtag flash programmer (usb-blaster). Connecting it requires physical access to the jtag connector on the PWB board. It is impossible when the detector is fully assembled in the experiment.

The PWB has a 32 Mbyte EPCQ flash memory chip, it is divided into 2 pages 16 Mbytes of factory page and 16 Mbytes of user page.

A complete firmware image generally contains 3 pieces:

  • FPGA firmware and NIOS "feam_bootloader" software RAM image (sof file)
  • NIOS "feam" software RAM image (feam.elf.flash.hex) - all the C code for esper communications and PWB data acquisition
  • NIOS filesystem image (feam.webpkg.hex) - esper web pages

This is the boot sequence:

  • on power-up the FPGA automatically loads the sof file from address 0 of the EPCQ flash (this is the factory image sof file)
  • the FPGA is "started"
  • the NIOS CPU starts executing the "feam_bootloader" C code embedded in the sof file (NIOS project hdl/software/feam_bootloader)
  • feam_bootloader initializes the hardware (clocks, DDR memory, etc)
  • feam_bootloader write-protects the EPCQ flash memory
  • feam_bootloader copies the "feam" software from EPCQ flash to the DDR memory (after checking for correct checkum)
  • feam_bootloader restarts the NIOS CPU
  • the NIOS CPU starts running from DDR memory, executes the "feam" software (NIOS project hdl/software/feam)
  • call main() in hdl/software/feam/src/task_init.c
  • call task_init() in the same file
  • infinite loop waiting for IP address via DHCP
  • after have IP address, call task_esper() from hdl/software/feam/src/task_esper.c
  • task_esper() creates all the esper modules, esper variables, etc,
  • mod_http.c starts the mongoose web server
  • PWB is open for business

This is the protection against booting corrupted firmware:

  • FPGA hardware loads the sof file from epcq flash and checks for correct checksum before "starting" it
  • NIOS CPU is part of the sof file, protected by sof file checksum
  • NIOS feam_bootloader C code is part of the sof file, protected by the sof checksum
  • feam_bootloader checks for correct signatures and checksums of the DDR memory image
  • "feam" C code is protected by the checksum of the DDR memory image

NIOS terminal

$ ssh agmini@daq16
$ /opt/intelFPGA/16.1/quartus/bin/jtagconfig
1) USB-Blaster [2-1.2]
  02B030DD   5CGTFD7(B5|C5|D5)/5CGXBC7B6/..
$ /opt/intelFPGA/17.0/quartus/bin/nios2-terminal
nios2-terminal: connected to hardware target using JTAG UART on cable
nios2-terminal: "USB-Blaster [2-1.2]", device 1, instance 0
nios2-terminal: (Use the IDE stop button or Ctrl-C to terminate)
PWB Revision 1 Boot Loader
Ver 2.0  Build 357 - Wed Jun  6 15:05:35 PDT 2018
...

Flash boot loader firmware via jtag

$ ssh agmini@daq16
$ /opt/intelFPGA/16.1/quartus/bin/jtagconfig
1) USB-Blaster [2-1.2]
  02B030DD   5CGTFD7(B5|C5|D5)/5CGXBC7B6/..
$ cd ~/online/firmware/pwb_rev1
$ ls -l
$ /opt/intelFPGA/17.1/quartus/bin/quartus_pgmw
... auto detect
... load the jic file
... in menu tools->programmer, enable "unprotect device"
... start program/configure operation

Flash user page firmware via esper-tool

$ ssh agmini@daq16
$ cd online/src
$ more update_pwb.perl ### check that $fw is set to the desired firmware file
$ ./update_pwb.perl pwb06 ### or give more PWB names or give "all"

Build firmware

NOTE: quartus 16.1 should be used for jtag (jtagconfig and jtagd)

NOTE: quartus 17.0 should be used to build the PWB firmware (17.1 is not compatible) 

$ ssh agmini@daq16
$ /opt/intelFPGA/17.0/nios2eds/nios2_command_shell.sh
------------------------------------------------
Altera Nios2 Command Shell [GCC 4]

Version 17.0, Build 602
------------------------------------------------
$ /opt/intelFPGA/17.0/quartus/linux64/lmgrd -c ~agmini/online/license-daq16.dat
$ cd online/firmware/git/pwb_rev1_firmware
$ git pull
$ git checkout alphag
$ git pull
$ ./scripts/compile_project.sh
$ ls -l bin/*.sof bin/*.jic bin/*.rpd
-rw-r--r-- 1 agmini alpha 12727389 Jan 24  2018 bin/feam_auto.rpd
-rw-r--r-- 1 agmini alpha 33554661 Jan 24  2018 bin/feam.jic
-rw-r--r-- 1 agmini alpha  6974754 Jan 24  2018 bin/feam.sof
$ ### feam.jic is loaded via jtag
$ ### feam_auto.rpd is loaded via esper
$ ### feam.sof is used to attach the signal tap

Partial build of firmware

  • scripts/compile_elf.sh - build the BSPs, the bootloader and the main elf firmware
  • scripts/update_elf.sh - bake the updated elf files into the sof (bootloader) and pof&jic&rpd files (main elf firmware)
  • scripts/load_jtag_sof.pl bin/rev1.sof - load sof file into the FPGA. this will boot the new fpga firmware, the new bootloader, but the old mail elf firmware (from epcq flash)
  • scripts/compile_nios.sh - regenerate the BSPs, then same as compile_elf.sh

ESPER Variables

  • Board
    • invert_ext_trig - invert trigger signal from CDM before it drives any logic (to undo incorrect signal polarity)
    • reset_nios - goggle up, them down to reset NIOS subsystem
  • Signalproc
    • test_mode - ADC data is replaced with a test pattern, see test mode bits in sca_x_ch_ctrl.
    • sca_a_ch_ctrl, sca_b_ch_ctrl, sca_c_ch_ctrl, sca_d_ch_ctrl - channel control bits:
11..0 - threshold - channel suppression threshold
14..12 - ctrl_test_mode - test mode:
         0=fixed patter 0xa5a,
         1=time bin counter,
         2=time bin counter with channel number,
         3=sequential adc sample counter,
         4={ch_crossed_out,trig_pos,trig_neg,adc[8:0]},
         5={trig,adc[10:0]},
         6={ch_crossed_min,adc[10:0]}
15 - ctrl_supp_mode - channel suppression mode: 0=adc<=(baseline-threshold), 1=adc<=threshold
  • Link
    • link_ctrl - sata link control. The bits are:
0 - sata_link_udp_stream_in_enable - permit data flow from sata link to OFFLOAD_SATA
1 - udp_stream_out_enable - permit sca data flow to sata link
2 - sata_to_eth_enable - permit ethernet data flow from sata link to TSE_MAC
3 - eth_to_sata_enable - permit ethernet data flow from TSE MAC to sata link
4 - enable_stop_our_tx - enable flow control: allow stop_tx
#5 - stop_our_tx - manually activate the flow control signal into link_tx
6 - enable_stop_remote_tx - enable flow control: allow send "stop_tx"
#7 - stop_remote_tx - manually activate the flow control signal into link_tx
8 - tx_test_pattern_udp - udp data is replaced by test pattern 0x11111111, 0x22222222, etc.
9 - tx_test_pattern_eth - nios-to-sata data is replaced by test pattern 0x11111111, 0x22222222, etc.
10 - udp_delay_enable - delay between udp packets, see udp_delay_value below
11 - reboot_tx - send K_REBOOT command to the sata link mate, where it has same effect as board/reset_nios.
12 - sata_to_nios_disable
13 - nios_to_sata_disable
14 - 
15 - 
16 -
24..31 - udp_delay_value - delay between udp packets (top 8 bits of a 20-bit counter)

Firmware data path

Main data path: 

feam_top
|
sca_sigproc
|
sca_event_control
4 * sca_control (sca_channel.sv)
4 * channel_fifo (1024 samples)

sca_control (sca_channel.sv)
|
state machine to control SCA read and write enables
|
sca_write_control (what does it do?!?)
sca_read_control

sca_read_control
|
state machine to store ADC samples into per-channel FIFOs (channel_fifo)
selector to replace ADC samples with a test pattern
79*sca_trig_one -> ch_crossed_out - channel hit detector

sca_event_control
|
event_fifo (hdl/mf/event_descriptor_fifo)
state machine to take ADC data from 4 per-channel FIFOs (channel_fifo) and store it into DDR memory (write addr increment is 510*8 (number of samples)*8 = 4080).
state machine to read transposed ADC data from DDR memory (read addr increment is 8+8=16 - 2 samples * 2 bytes * 4 sca = 16) and create 4 per-sca data streams

data stream:
4 per-sca data streams from sca_event_control -> sca_sigproc(event_dat)
|
packet_chunker (hdl/lib/packet_chunker.sv) (4*event_dat -> event_segment_dat) - multiplex 4 data streams into 1 stream of UDP-sized packets
|
packet_length_prepender (hdl/lib/packet_length_prepender.sv)  (event_segment_dat -> udp_event_val_out)
|
udp_event_val -> info qsys (udp_stream_sca) and into sata link (mux channel 2).

TODO

  • IMPOSSIBLE (ext clock only connected to clock cleaner) add frequency counter for the external clock (62.5 MHz)
  • DONE add random delay before memtest
  • DONE add 4 more bits to udp_delay
  • DONE fix crash on boot if run is active and triggers arrive (check that trigger is off on boot) (do not set trigger/enable_all, ext_trig_ena, inp_trig_ena, signal_proc/force_run to 1 after boot)
  • DONE (kludge jtag uart write()) increase size of jtag uart output buffer, make it unlimited if possible.
  • DONE (kludge jtag uart write()) on dhcp timeout, reboot the fpga - reboot of nios does not clear the jtag uart, which becomes full and nios stops.
  • DONE add NIOS watchdog timeout for fectrl
  • DONE do not use external clock until instructed by fectrl (clock cleaner default is pin select mode, default pin mode is input with pull down, so clock 0 is selected, this is the ext clock. but we can drive clock select pins from the fpga).
  • DONE change clock cleaner power up: use the 2 lines status_clkin0 and status_clkin1 to tell clock cleaner to use the internal clock (clkin2) - from the FPGA drive first line low, second line high. Also change all NIOS code that programs the clock cleaner to keep these two lines in the "input" mode. Current code switches them to "output" mode.
  • DONE clock cleaner power up state is wrong for both choices of clock: 62.5 MHz ext clock and 125 MHz internal clock, see explanation in feam_top.sv. It looks like I need to drive status_clkin0 and status_clkin1 to "holdover" mode, both logic level 1. Currently status_clkin0 is used as SPI read line, this has to be moved to a different line first.
  • add watchdog timeout that works in factory mode ("remote update" watchdog only works in user mode)
  • enable HTTP pipelining. it looks like it should work, but it does not. very old version of mongoose web server.
  • reset everything if it looks like NIOS TCP/IP code has crashed, but enough code is still running to prevent tripping of the fpga-remote-update watchdog timer. such crash tends to happens when the sata link mate is rebooted and incomplete data arrives from the disrupted ethernet data path.

ZZZ

ZZZ

Retrieved from "https://daq00.triumf.ca/AgWiki/index.php?title=PWB&oldid=607"