POL: Data format and analysis: Difference between revisions

From DaqWiki
Jump to navigation Jump to search
(Created page with "== Links == <div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> * Pol DAQ System * POL Experiment DAQ documentation * POL Data Banks </div> ==...")
 
 
(4 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Links ==
== Links ==
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* [[Pol DAQ System]]
* [[POL|POL overview]]
* [[POL Experiment DAQ documentation]]
* [[POL: Hardware and software]]
* [[POL Data Banks]]
* [[POL: Webpages]]
</div>
</div>


== Introduction ==
== Introduction ==


This document describes the MIDAS format data banks that are output by the POL frontend ''pol_fevme.exe''.  This frontend controls the hardware for the experiment, and programs the contents of the data banks. The analyzer reads the data from the data banks.
This document describes the MIDAS format data banks that are output by the POL frontend ''fepol_32bit.exe''.  This frontend controls the hardware for the experiment, and programs the contents of the data banks. The analyzer reads the data from the data banks.


== MIDAS Data ==
== MIDAS Data ==
The data are sent out as MIDAS events which contain various banks as shown in Table 1 below:
{|  style="text-align: left; width: 100%; background-color: rgb(255, 255, 255);" border="3" cellpadding="2" cellspacing="2"
|+ Table 1 : POL data banks
|-
|colspan="1" rowspan="1" style="vertical-align: top; background-color: azure; font-weight: bold;" |Equipment name
|colspan="1" rowspan="1" style="vertical-align: top; background-color: rgb(255, 255, 204); font-weight: bold;" |INFO
|colspan="1" rowspan="1" style="vertical-align: top; background-color: rgb(255, 255, 204); font-weight: bold;" |HISTO
|colspan="1" rowspan="1" style="vertical-align: top; background-color: rgb(255, 255, 204); font-weight: bold;" |VME
|-
| style=" background-color: azure; font-weight: bold;" | Event ID
| 3
| 5
| 11'''*'''
|-
| style=" background-color: azure; font-weight: bold;" | Sent out
|at end of SuperCycle
|at end of SuperCycle
|at end of SuperCycle
|-
| style=" background-color: azure; font-weight: bold;" |Banknames
| style="color:blue" | DBUG  CYCL '''**'''  SUMS
| style="color:blue" |HISI  HIS0 HIS1 HIS2 HIS3 HSUM  CYCL **
|style="color:blue" |MCS0
|-
| style=" background-color: azure; font-weight: bold;"|Contents
| Debug and other useful information
| Histograms and sums of the time-bins over each supercycle, ADC readout etc.
| Raw cycle scaler data (debugging)
|+
| style=" background-color: azure; font-weight: bold;" |Data sent to ODB
|YES
|NO
|NO
|}
;NOTES
*  For definitions of Cycle, Supercycle, Scan etc. see Nomenclature
*    In this version, threshold checks are not supported.
*    <span style="color:blue; font-weight:bold;">**</span> The  <span style="color:blue;">CYCL</span> banks in Event ID 3 and 5 are no longer identical
*    '''*''' Event 11 is for debugging and is normally disabled.
* Event 11 can be enabled by setting  the edit-on-start parameter ''send raw data bank'' to TRUE. This parameter points to  the ODB key <span style="color: purple; font-style:italic;">/Equipment/POL_ACQ/Settings/input/send raw data bank</span>.
The examples of data banks in this document have been generated by the utility {{Utility|name=mdump}} e.g.
<pre>
mdump -l 20      (dumps the data from the running experiment)
or
mdump -x /isdaq/data1/pol/data/2014_vmic/run00100.mid | more      (dumps the data from a saved data file)
</pre>
== Event ID 3  INFO Event Banks sent out once per cycle ==
The INFO event contains the following MIDAS Banks:  <span style="color:blue;"> DBUG  CYCL  SUMS </span>
;Bank Contents :
:Readout of debug and other useful information  (e.g. counters, DAC set/readback values/histo sums).
;NOTE
The INFO event (event ID 3) is sent to the ODB for display when running, and is also sent out to the data buffer.
=== DBUG bank ===
The DBUG bank is for debugging and need not concern the user too much. It may be disabled in later versions.
The contents of the <span style="color:blue;">DBUG</span> bank are shown in Table 2 below.
{|  style="text-align: left; width: 100%; background-color: whitesmoke;" border="3" cellpadding="2" cellspacing="2"
|+ Table 2 : <span style="font-weight:bold;color:blue;">DBUG</span>''': Debug bank  Type: FLOAT'''
|-
!Word !! Contents !!Explanation
|-
|1 || Data to read || Number of 32-bit words of data still buffered in the SIS3820
|-
|2 || LNE per cycle || Number of values in the SIS3820 buffer for one cycle (i.e. number of LNE)
|-
|3 || LNE per SC || LNE count from SIS3802 Acquisition Count Register i.e. Number of LNE pulses received from PPG
|-
|4 || LNE preset || Number of LNE set in SIS3820 Preset register. Once this number of LNE are received, SC is complete.
|-
|5 || Num bins || Number of bins sent out.
|-
|6 || Data bytes || 2 for 16-bit data.  [1 (8-bit data) and 2 (32-bit data) are not yet supported/tested in SC version.]
|-
|7 || Num Chan || Number of  input channels of the SIS3820 enabled. Always 4 for Pol experiment.
|-
|8 || Discard 1st Bin || True if first bin is discarded each cycle
|-
|9 || Discard 1st Cycle || True if first cycle is discarded each SC
|}
;KEY to Table 2
*    LNE Load Next Event  On receipt of LNE signal from PPG, SIS3820 moves to the next bin
*    SC  SuperCycle. The user selects how many PPG cycles taken at the '''same''' DAC value make up the SuperCycle
;Note
*Counters in this bank include
**    one extra cycle if ''Discard first cycle'' is set or
**    one extra time bin if ''Discard first time bin'' is set.
*''Discard first cycle'' and ''Discard first time bin'' are parameters available on the  <span style="color:darkcyan;font-style:italic">mhttpd</span> Pol expt PPG CYCLE custom web page.


=== CYCL Bank (EVID 3) ===
The data are sent out as MIDAS events which contain various banks: <code>HISI HIS0 HIS1 HIS2 HIS3 HSUM CYCL</code>. Some of these banks contain duplicated information (e.g. HISI vs CYCL), but they are all kept for backwards compatibility.
The contents of the <span style="color:blue;">CYCL</span> bank are shown in Table 3 below.
 
{|  style="text-align: left; width: 100%; background-color: whitesmoke;" border="3" cellpadding="2" cellspacing="2"
|+ Table 3 : <span style="font-weight:bold;color:blue;">CYCL</span>''': Cycle counter bank  Type: FLOAT    EventId: 3 '''
|-
!Word !! Contents !!Explanation
|-
| 1 || Scan type code || Always 1 -  indicates a DAC scan 
|-
| 2 || Cycle counter || Increments, counting cycles. Cycle Counter will count by N
|-
| 3 || SuperCycle counter || Increments, counting supercycles. If there are N cycles in a supercycle, the Cycle Counter will count N times as fast as the Supercycle Counter.
|-
| 4 || Cycles/SC counter || Counts cycles within each supercycle. If there are N cycles per supercycle, this counter should always show N.
|-
| 5 || Sweep counter || Increments, counting the number of complete DAC sweeps (or scans).
|-
| 6 || Skipped-cycle counter || Counts number of cycles skipped. If first cycle is discarded each SC, this number will be same as SuperCycle Counter.
|-
| 7 || Num cycles histogrammed || Should be the same as Cycle counter.
|-
| 8 || DAC Increment counter || Counts DAC increments within a sweep. If the number of DAC increments selected per sweep is M, counter will vary from 0 to M.
|-
| 9 || DAC Set Value || DAC Set value in Volts  (DAC Output  0)
|-
|  10 || DAC Readback || DAC Readback value in Volts (i.e. direct readback via Input 0 of Galil ADC connected to DAC Output 0 )
|-
| 11 || ADC0 Average || Average value of ADC Input 0 ( DAC Readback in Volts)
|-
| 12 || ADC1 Average || Average value of ADC Input 1 (Volts)
|-
| 13 || ADC2 Average || Average value of ADC Input 2 (Volts)
|-
| 14 || ADC3 Average || Average value of ADC Input 3 (Volts)
|-
| 15 || spare ||not used
|-
|
|}
;Notes:
*Counters in this bank include
**    one extra cycle if ''Discard first cycle'' is set or
**    one extra time bin if ''Discard first time bin'' is set.
*''Discard first cycle'' and ''Discard first time bin'' are parameters available on the  <span style="color:darkcyan;font-style:italic">mhttpd</span> Pol expt PPG CYCLE custom web page.
*      "N" in the table below  is the '''calculated''' number of cycles per supercycle, i.e.
**        N = number of requested cycles per supercycle if ''Discard first cycle'' is false.
**        N = (number of requested cycles + 1)  if  ''Discard first cycle'' is true.
*      "DAC Readback" reads back the DAC Set Value from ADC Input 0.  There is a second user-defined readback on ADC Input 1 which can be found in bank [[#HISI]].
*      Contents of this <span style="color:blue;">CYCL</span> bank (Event 3) is now different from the <span style="color:blue;">CYCL</span> bank in Event 5
 
 
 
=== SUMS Bank ===
 
The <span style="color:blue;">SUMS</span> bank is a copy of [[#HSUM]] bank.
 
 
=== Example of Event ID 3 ===
 
Since Event ID 3 is sent to the ODB, you can see the contents of the INFO Event by clicking on "Info" on the MIDAS status page, e.g. Figure 1
 
[[image]]
 
In this case, the input parameters were:
 
*    100 bins per cycle
*    100 cycles per supercycle
*    Discard first cycle per SC and first bin per cycle
*    DAC Sweep 0-0.1V  10 increments of 0.01V step
 
This data can also be found in the data buffer, e.g.:
<pre>
------------------------- Event# 2 ------------------------
Evid:0003- Mask:0008- Serial:4- Time:0x53dc4735- Dsize:180/0xb4
#banks:3 - Bank list:-DBUGCYCLSUMS-
 
Bank:DBUG Length: 36(I*1)/9(I*4)/9(Type) Type:Real*4 (FMT machine dependent)
  1-> 0.000e+00 1.010e+02 2.030e+04 2.030e+04 1.010e+02 2.000e+00 4.000e+00 1.000e+00
  9-> 1.000e+00
 
Bank:CYCL Length: 60(I*1)/15(I*4)/15(Type) Type:Real*4 (FMT machine dependent)
  1-> 1.000e+00 1.000e+03 5.000e+00 2.000e+02 1.000e+00 5.000e+00 1.000e+03 4.000e+00
  9-> 4.000e-02 4.300e-02 4.150e-02 3.913e-01 0.000e+00 9.263e+00 0.000e+00
 
Bank:SUMS Length: 32(I*1)/8(I*4)/4(Type) Type:double*8
  1->    0.00000e+00        9.99990e+04        0.00000e+00        0.00000e+00
</pre>
 
== Event ID 5: Histograms sent out once per Supercycle ==
The INFO event contains the following MIDAS Banks:  <span style="color:blue;"> HISI  HIS0 HIS1 HIS2 HIS3  HSUM CYCL  </span>
Bank Names :  HISI  HIS0 HIS1 HIS2 HIS3  HSUM CYCL 
Bank Contents: [[Example]]


=== HISI Bank ===
=== HISI Bank ===
Line 229: Line 30:
|5 || DAC Increment counter || Counts from 0 to Num Dac Increments
|5 || DAC Increment counter || Counts from 0 to Num Dac Increments
|-
|-
|6 || NumCycles Summed || Number of cycles summed in these histograms. Should be equal to the number of cycles per supercycle
|6 || NumCycles Summed || Number of iterations summed in these histograms.
|-
|-
|7 || Scaler buffer 1st word || DAC Set Value in Volts for this event stored in Scaler data buffer as the first word for checking.<br>
|7 || Scaler buffer 1st word || DAC Set Value in Volts for this event stored in Scaler data buffer as the first word for checking.<br>
Line 286: Line 87:
|}
|}


 
=== CYCL bank ===
 
=== CYCL Bank (EVID 5) ===
Note that this bank  (EVID 5) is now DIFFERENT from  the  <span style="font-weight:bold;color:blue;">CYCL</span> Bank in Event ID 3.
Words 1-10 inclusive are identical in both cases, but  this bank also contains the readback of  the first 4 ADC Inputs (two of which were formerly in  <span style="font-weight:bold;color:blue;">ADCR</span> bank, now removed).


{|  style="text-align: left; width: 100%; background-color: lightgoldenrodyellow;" border="3" cellpadding="2" cellspacing="2"
{|  style="text-align: left; width: 100%; background-color: lightgoldenrodyellow;" border="3" cellpadding="2" cellspacing="2"
Line 299: Line 96:
|1 || Scan type code || Always 1 -  indicates a DAC scan   
|1 || Scan type code || Always 1 -  indicates a DAC scan   
|-
|-
|2 || Cycle counter || Increments, counting cycles. Cycle Counter will count by N
|2 || Iteration counter || Increments, counting PPG iterations.
|-
|-
|3 || SuperCycle counter || Increments, counting supercycles. If there are N cycles in a supercycle, the Cycle Counter will count N times as fast as the Supercycle Counter.
|3 || Cycle counter || Increments, counting cycles. If there are N iterations in a cycle, the Cycle Counter will count N times as fast as the Iteration Counter.
|-
|-
|4 || Cycles/SC counter || Counts cycles within each supercycle. If there are N cycles per supercycle, this counter should always show N.
|4 || Iterations/Cycle counter || Counts PPG iterations within each cycle. If there are N iterations per cycle, this counter should always show N.
|-
|-
|5 || Sweep counter || Increments, counting the number of complete DAC sweeps (or scans).
|5 || Sweep counter || Increments, counting the number of complete DAC sweeps (or scans).
|-
|-
|6 || Skipped-cycle counter || Counts number of cycles skipped. If first cycle is discarded each SC, this number will be same as SuperCycle Counter.
|6 || Skipped-iteration counter || Counts number of iterations skipped. If first iteration is discarded each cycle, this number will be same as Cycle Counter.
|-
|-
|7 || Num cycles histogrammed || Should be the same as Cycle counter.
|7 || Num iterations histogrammed || Should be the same as Iteration counter.
|-
|-
|8 || DAC Increment counter|| Counts DAC increments within a sweep. If the number of DAC increments selected per sweep is M, counter will vary from 0 to M.
|8 || DAC Increment counter|| Counts DAC increments within a sweep. If the number of DAC increments selected per sweep is M, counter will vary from 0 to M.
Line 339: Line 136:




== Analysis ==


=== Python library for parsing data ===


The [https://bitbucket.org/ttriumfdaq/pol_data pol_data] package contains tools for reading midas data and converting it to objects that can be used in python code. It is able to parse data from both the old DAQ and new DAQ. It is able to read both midas files and data from the live experiment. It is used as a basis for both the online and offline tools described below.


=== Example of EventID 5 ===
=== Online analysis - polana ===
In this example, only Scaler Input 1 has been connected to a clock, so only the <span style="font-weight:bold;color:blue;">HIS1</span> bank has counts.
<pre>
------------------------ Event# 7 ------------------------
Evid:0005- Mask:0020- Serial:1- Time:0x5339eea8- Dsize:352/0x160
#banks:8 - Bank list:-CYCLHISIHIS0HIS1HIS2HIS3HSUM-


Bank:CYCL Length: 68(I*1)/17(I*4)/17(Type) Type:Real*4 (FMT machine dependent)
The main POL DAQ repository contains a python-based analyzer that can generate histograms and display them on the "Web plots" webpage. The program is called "polana" and can be started using the relevant button the main DAQ webpage.
  1-> 1.000e+00 1.000e+03 5.000e+00 2.000e+02 1.000e+00 5.000e+00 1.000e+03 4.000e+00
  9-> 4.000e-02 4.150e-02 3.943e-01 9.000e-04 9.263e+00 4.150e-02 3.913e-01 0.000e+00
  17-> 9.263e+00


Bank:HISI Length: 28(I*1)/7(I*4)/7(Type) Type:Real*4 (FMT machine dependent)
The "Web plots" webpage can show the following plots:
  1-> 1.000e+03 5.000e+00 4.000e-02 3.958e-01 4.000e+00 1.000e+00 4.000e-02


Bank:HIS0 Length: 400(I*1)/100(I*4)/100(Type) Type:Unsigned Integer*4
* Sum of counts vs event time
  1-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
* Sum of counts vs DAC voltage (scanned variable)
  9-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
* Time spectrum (cumulative over the whole run)
  17-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
* Time spectrum (most recent cycle only)
  25-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  33-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  41-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  49-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  57-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  65-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  73-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  81-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  89-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  97-> 0x00000000 0x00000000 0x00000000 0x00000000


Bank:HIS1 Length: 400(I*1)/100(I*4)/100(Type) Type:Unsigned Integer*4
You can select which scaler channels to display, and limit to only showing data for specific DAC voltages (by limiting which "scan steps" to display).
  1-> 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8
  9-> 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8
  17-> 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8
  25-> 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8
  33-> 0x000003e7 0x000003e9 0x000003e7 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8
  41-> 0x000003e8 0x000003e9 0x000003e7 0x000003e8 0x000003e9 0x000003e8 0x000003e7 0x000003e9
  49-> 0x000003e8 0x000003e8 0x000003e7 0x000003e9 0x000003e8 0x000003e8 0x000003e8 0x000003e8
  57-> 0x000003e8 0x000003e8 0x000003e9 0x000003e7 0x000003e9 0x000003e8 0x000003e8 0x000003e7
  65-> 0x000003e9 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e7
  73-> 0x000003e8 0x000003e8 0x000003e9 0x000003e8 0x000003e7 0x000003e9 0x000003e7 0x000003e8
  81-> 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e7
  89-> 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8
  97-> 0x000003e8 0x000003e8 0x000003e8 0x000003e8


Bank:HIS2 Length: 400(I*1)/100(I*4)/100(Type) Type:Unsigned Integer*4
=== Offline analysis - gui ===
  1-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  9-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  17-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  25-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  33-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  41-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  49-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  57-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  65-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  73-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  81-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  89-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  97-> 0x00000000 0x00000000 0x00000000 0x00000000


Bank:HIS3 Length: 400(I*1)/100(I*4)/100(Type) Type:Unsigned Integer*4
An interactive GUI for interacting with midas files and converting to CSV files etc was originally created by Tom Proctor and called "main.py". That version required the user to run "mdump" to convert the midas files to text, then parsed those text files. This was error-prone and required tweaks to the parsing code for each experimental campaign.
  1-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  9-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  17-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  25-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  33-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  41-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  49-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  57-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  65-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  73-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  81-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  89-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  97-> 0x00000000 0x00000000 0x00000000 0x00000000
 
Bank:HSUM Length: 32(I*1)/8(I*4)/4(Type) Type:double*8
  1->    0.00000e+00        9.99990e+04        0.00000e+00        0.00000e+00
 
</pre>
 
 
 
 
== Event ID 11: Raw data sent out each cycle ==
Event ID 11 contains the bank <span style="font-weight:bold;color:blue;">MCS0</span>.
 
=== MSC0 Bank (EVID 11) ===
 
Bank Contents: Example
 
Presently the <span style="font-weight:bold;color:blue;">MCS0</span> Bank IS sent out for debugging purposes. It may later be disabled by default, in which case it can be enabled by setting  the ODB Key
      {{Odbpath|path=/Equipment/POL_ACQ/Settings/input/send debug banks/send raw data bank}} to "y".
The  <span style="font-weight:bold;color:blue;">MCS0</span>Bank is always in 32-bit format. However, the data packing depends on the SIS3820 data format selected. This can be 8-bit, 16-bit**, 32-bit  (see SIS3820 manual).
With 4 channels enabled, the MCS0 bank format is shown in Table 8 for the three supported cases of SIS3820 data format.
 
;Note
:Only the '''16-bit format''' is implemented/tested for the SuperCycle mode POL is currently using.
 
{|  style="text-align: left; width: 100%; background-color: white;" border="3" cellpadding="2" cellspacing="2"
|+ Table 8 : <span style="font-weight:bold;color:blue;">MCS0</span>'''  Raw Data bank  Type: Unsigned Integer (32-bit)  EventId: 11'''
|-
| style="text-align:center; font-weight:bold; background-color:lightgreen;" | Table 8a : 8-BIT FORMAT DATA
| style="text-align:center; font-weight:bold; color:white; background-color: green;"| Table 8b : 16-BIT FORMAT DATA
| style="text-align:center; font-weight:bold;  background-color: springgreen;"| Table 8c : 32-BIT FORMAT DATA
|-
|
 
{|  style="text-align: left; width: 100%; background-color: lightgreen;" border="3" cellpadding="2" cellspacing="2"
|-
!Word !! Byte!! Time Bin !! Input
|-
|  colspan="1" rowspan="4" | 1
| 0 || 0 || 0
|-
| 1 || 0 || 1
|-
| 2 || 0 || 2
|-
| 3 || 0 || 3
|-
|  colspan="1" rowspan="4" | 2
| 0 || 1 || 0
|-
| 1 || 1 || 1
|-
| 2 || 1 || 2
|-
| 3 || 1 || 3
|-
|.........||..........||.......||.........
|-
|  colspan="1" rowspan="4" | Nbins
| 0 || Nbins || 0
|-
| 1 || Nbins || 1
|-
| 2 || Nbins || 2
|-
| 3 || Nbins || 3
|-
|}
<br><br><br><br>
|
{|  style="text-align: left; width: 100%; background-color: green; color:white;" border="3" cellpadding="2" cellspacing="2"
|-
!Word !! Byte!! Time Bin !! Input
|-
|  colspan="1" rowspan="4" | 1
| 0
| rowspan="2"| 0
| rowspan="2" | 0
|-
| 1
|-
| 2
|rowspan="2"| 0
| rowspan="2"| 2
|-
| 3
|-
 
|  colspan="1" rowspan="4" | 2
| 0
|rowspan="2"| 0
|rowspan="2"| 2
|-
| 1
|-
| 2
|rowspan="2"| 0
|rowspan="2"| 3
|-
| 3
|-
|.........||..........||.......||.........
|-
|  colspan="1" rowspan="4" | 2*Nbins-1
| 0
|rowspan="2"| Nbins
|rowspan="2"| 0
|-
| 1
|-
| 2
|rowspan="2"| Nbins
|rowspan="2"| 2
|-
| 3
|-
 
|  colspan="1" rowspan="4" | 2*Nbins
| 0
|rowspan="2"| Nbins
|rowspan="2"| 2
|-
| 1
|-
| 2
|rowspan="2"| Nbins
|rowspan="2"| 3
|-
| 3
|}
 
|
{|  style="text-align: left; width: 100%; background-color: springgreen;" border="3" cellpadding="2" cellspacing="2"
|-
!Word !! Byte!! Time Bin !! Input
|-
|1 || 0-3 || Time bin 0 || 0
|-
|2 || 0-3 || Time bin 0 || 1
|-
|3 || 0-3 || Time bin 0 || 2
|-
|4 || 0-3 || Time bin 0 || 3
|-
|5 || 0-3 || Time bin 1 || 0
|-
|6 || 0-3 || Time bin 1 || 1
|-
|7 || 0-3 || Time bin 1 || 2
|-
|8 || 0-3 || Time bin 1 || 3
|-
|.... ||.... || .... || ....
|-
|4*Nbins-3 || 0-3 || Last Time bin|| 0
|-
|4*Nbins-2 || 0-3 || Last Time bin|| 1
|-
|4*Nbins-1 || 0-3 || Last Time bin|| 2
|-
|4*Nbins || 0-3 || Last Time bin|| 3
|}
<br><br><br><br>
 
|}
 
;Notes:
 
* The data are buffered in the SIS3820 scaler. The data are read out periodically into a data buffer.
* At the end of a SuperCycle, the data buffer is decoded. The data from each cycle are summed to form the data for a SuperCycle. The histograms are sent out, the DAC incremented, and the next SuperCycle is started.
 
 
=== Examples of Event ID 11 ===
The pol experiment uses the 16-bit SIS3820 data format. The following data in 16-bit SIS3820 format are for 4 inputs with 10 bins selected.
''Discard First Bin'' and ''Discard First Cycle'' are selected.
 
The first word of the MCS0 bin is always the DAC Voltage * 1000 i.e. the DAC Voltage in mV.
This is to convert the DAC Voltage (type float) to integer for the (integer) data bank.
 
The Time Bin number for the first few data words has been added in yellow, and the data for
 
*    Input 0 is coloured  <span style="font-weight:bold;color:blue;">blue</span>
*    Input 1 is coloured  <span style="font-weight:bold;color:green;">green</span>
*    Input 2 is coloured  <span style="font-weight:bold;color:red;">red</span>
*    Input 3 is coloured  <span style="font-weight:bold;color:purple;">purple</span>
 
 
Input 3 is connected to a clock. The counts are constant during the cycle, but at the end of each cycle the counts increase as the scaler remains on the same bin for a longer period (waiting for next TOF). This longer time is discarded when the first bin/cycle is discarded.
 
;Programming Note
Currently the scaler is ENABLEd rather than ARMed. In the future, ENABLE may be changed to ARM, in which case an extra bin must be added and the first bin need no longer be discarded.
 
{|  style="text-align: left; width: 100%; background-color: silver;" border="0" cellpadding="1" cellspacing="1"
|+ Table 9 : Example of <span style="font-weight:bold;color:blue;">MCS0</span>''' data in 16-bit format 
|-
| colspan="9" |------------------------ Event# 5 ------------------------
Evid:000b- Mask:0800- Serial:2- Time:0x5339eea7- Dsize:556/0x22c #banks:1 - Bank list:-MCS0-
|- 
| colspan="9" |Bank:MCS0 Length: 536(I*1)/134(I*4)/134(Type) Type:Unsigned Integer*4
|-  style="color: saddlebrown;font-weight:bold; text-align:center;"
|
|style="font-weight:bold; color:fuchsia;"| DAC(mv) 
| 0 || 0 || 1 || 1 || 2 || 2 || 3  || Time bin
|-
| 1->  ||0x000001f4
||0x<span style="font-weight:bold;color:green;">0000</span><span style="font-weight:bold;color:blue;">0000</span>
||0x<span style="font-weight:bold;color:purple;">0057</span><span style="font-weight:bold;color:red;">0000</span>
|0x00000000 ||0x00050000 ||0x00000000 ||0x00050000 ||0x00000000 ||
|-  style="color: saddlebrown;font-weight:bold;text-align:center;"
| || 3 || 4 || 4 || 5 || 5 || 6 || 6 || 7 ||  First Cycle
|-
|9->  ||0x00050000||0x00000000||0x00050000||0x00000000||0x00050000||0x00000000||0x00050000||0x00000000||
|-  style="color: saddlebrown;font-weight:bold;text-align:center;"
| || 7 || 8 || 8 || 9 || 9 ||10 || 10 |
|style="color: lightcoral;"|    0
|style="color: lightcoral;"|  Next Cycle
|-
|17-> ||0x00050000 ||0x00000000 ||0x00050000 ||0x00000000 ||0x00050000 ||0x00000000 ||0x00050000
|0x<span style="font-weight:bold;color:green;">0000</span><span style="font-weight:bold;color:blue;">0000</span> ||
|-  style="color: lightcoral;font-weight:bold;text-align:center;"
| || 0 || 1 ||  1 || 2 || 2 || 3 || 3 || 4 || 
|-
|25->||0x<span style="font-weight:bold;color:purple;">8a7c</span><span style="font-weight:bold;color:red;">0000</span>
|0x00000000||0x00050000||0x00000000||0x00050000||0x00000000||0x00050000||0x00000000 ||
|-
|33->||0x00050000||0x00000000||0x00050000||0x00000000||0x00050000||0x00000000||0x00050000||0x00000000 ||
|-
|41->||0x00050000||0x00000000||0x00050000||0x00000000||0x00050000||0x00000000||0x85470000||0x00000000 ||
|-
|49->||0x00050000||0x00000000||0x00050000||0x00000000||0x00050000||0x00000000||0x00050000||0x00000000 ||
|-
| 57->||0x00050000||0x00000000||0x00050000||0x00000000||0x00050000||0x00000000||0x00050000||0x00000000 ||
|-
|65->||0x00050000||0x00000000||0x00050000||0x00000000||0x84ca0000||0x00000000||0x00050000||0x00000000 ||
|-
|  73->||0x00050000||0x00000000||0x00050000||0x00000000||0x00050000||0x00000000||0x00050000||0x00000000 ||
|-
|  81->||0x00050000||0x00000000||0x00050000||0x00000000||0x00050000||0x00000000||0x00050000||0x00000000 ||
|-
|  89->||0x00050000||0x00000000||0x84960000||0x00000000||0x00050000||0x00000000||0x00050000||0x00000000 ||
|-
|  97->||0x00050000||0x00000000||0x00050000||0x00000000||0x00050000||0x00000000||0x00050000||0x00000000 ||
|-
| 105->||0x00050000||0x00000000||0x00050000||0x00000000||0x00050000||0x00000000||0x00050000||0x00000000 ||
|-
| 113->||0x84770000||0x00000000||0x00050000||0x00000000||0x00050000||0x00000000||0x00050000||0x00000000 ||
|-
| 121->||0x00050000||0x00000000||0x00050000||0x00000000||0x00050000||0x00000000||0x00050000||0x00000000 ||
|-
| 129->||0x00050000||0x00000000||0x00050000||0x00000000||0x00050000||0x00000000||  ||  ||
|}


The new code reads midas files directly in python, and does not require the user to do any pre-processing of the data with mdump.


The program is found in the [https://bitbucket.org/ttriumfdaq/pol_data pol_data] repository and is called <code>gui.py</code>. It can be started using the <code>gui</code> alias on midpol, or can be installed on your own laptop.


------------------------ Event# 5 ------------------------
If running on midpol, you should make sure to connect with X-forwarding enabled, e.g.
Evid:000b- Mask:0800- Serial:2- Time:0x5339eea7- Dsize:556/0x22c
#banks:1 - Bank list:-MCS0-


Bank:MCS0 Length: 536(I*1)/134(I*4)/134(Type) Type:Unsigned Integer*4
  $ ssh -Y pol@midpol
        <span style="font-weight:bold;color:cyan">DAC(mv)</span>            0                0                1                1            2              2              3          Time bin
  $ bash
  1->  0x000001f4 0x00000000 0x00570000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000
  $ gui
                3                4                4                5                5            6              6              7                  First Cycle
  9-> 0x00050000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000
                7                8                8                9                9            10            10            0                  Next Cycle
  17-> 0x00050000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000
                0                1                1                2                2              3              3              4         
  25-> 0x8a7c0000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000
  33-> 0x00050000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000
  41-> 0x00050000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000 0x85470000 0x00000000
  49-> 0x00050000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000
  57-> 0x00050000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000
  65-> 0x00050000 0x00000000 0x00050000 0x00000000 0x84ca0000 0x00000000 0x00050000 0x00000000
  73-> 0x00050000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000
  81-> 0x00050000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000
  89-> 0x00050000 0x00000000 0x84960000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000
  97-> 0x00050000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000
  105-> 0x00050000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000
  113-> 0x84770000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000
121-> 0x00050000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000
129-> 0x00050000 0x00000000 0x00050000 0x00000000 0x00050000 0x00000000


A screenshot of the GUI is shown below.


== Updates ==
[[Image:Pol_gui.png|800px]]
19Aug2019 document moved to wiki
Original document: dasdevpc2:/home/suz/www/pol/vmicbanks.html
Version 4 scaler read out at end of supercycle rather than each cycle. Galil Rio DAC incremented after histograms are sent
Version 5 Galil Rio averaging program for 4 ADC channels is loaded (pgm and RIO loaded by Donald). Averages of ADC0-3 are added to CYCL banks which are now different in Events 3 and 5.


[[Category:POL]]
[[Category:POL]]

Latest revision as of 14:56, 17 February 2023

Links

Introduction

This document describes the MIDAS format data banks that are output by the POL frontend fepol_32bit.exe. This frontend controls the hardware for the experiment, and programs the contents of the data banks. The analyzer reads the data from the data banks.

MIDAS Data

The data are sent out as MIDAS events which contain various banks: HISI HIS0 HIS1 HIS2 HIS3 HSUM CYCL. Some of these banks contain duplicated information (e.g. HISI vs CYCL), but they are all kept for backwards compatibility.

HISI Bank

Table 4 : HISI Histo counter bank Type: FLOAT EventId: 5
Word Contents Explanation
1 Cycle counter Increments, counting cycles. This will count by N where N is the requested number of cycles per supercycle.
2 SuperCycle counter Increments, counting supercycles.
3 Set Value DAC Set Value in Volts for this event
4 Set Value Readback Users' Readback (ADC Channel 1 Average value - see also CYCL bank ). See DAC Set value Readback **.
5 DAC Increment counter Counts from 0 to Num Dac Increments
6 NumCycles Summed Number of iterations summed in these histograms.
7 Scaler buffer 1st word DAC Set Value in Volts for this event stored in Scaler data buffer as the first word for checking.

Should always be equal to word 3 (but may have rounding errors due to conversion to integer mV and back).


Notes:

  • Cycle counter contains the user-requested number of cycles, so if parameter Discard first cycle is set, the first cycle will already have been discarded. Compare with #CYCL bank, where the actual number of cycles is recorded.
  • ** The DAC is used to drive a high voltage power supply. The Set Value Readback is read from ADC Channel 1, which is connected by the users to their own voltage readback. The actual readback of the DAC itself is found in #CYCL bank, read from ADC Channel 0.


HIS0,HIS1,HIS2,HIS3 Banks

The banks HIS0...HIS3 are the histogram banks for Scaler Input channels 0-3. These contain the cumulative sums of each time bin summed over the number of cycles/supercycle at a particular DAC increment value. Summing all the bins together for each channel should give the same value as in the HSUM bank. The histograms are cleared after each supercycle.

Table 5 : HIS0..3 Histogram Sums for Scaler Inputs 0..3 Type: Unsigned Integer EventId: 5
Word Contents
1 Time bin 0 summed for one SuperCycle
2 Time bin 1 summed for one SuperCycle
... ............
... ............
N Last time bin summed for one SuperCycle

HSUM Bank

The HSUM bank contains the sums of all the time-bins (output as histograms in Event 5) for each Scaler Channel. The sums are cleared after each SuperCycle.


Table 6 : HSUM Histogram Sums for Scaler Inputs 0..3 Type: FLOAT EventId: 5
Word Contents
1 Sum of all the time bins of Scaler input 0 for one SuperCycle
2 Sum of all the time bins of Scaler input 1 for one SuperCycle
3 Sum of all the time bins of Scaler input 2 for one SuperCycle
4 Sum of all the time bins of Scaler input 3 for one SuperCycle

CYCL bank

Table 7 : CYCL Cycle counter/ADC Readback bank Type: FLOAT EventId: 5
Word Contents Explanation
1 Scan type code Always 1 - indicates a DAC scan
2 Iteration counter Increments, counting PPG iterations.
3 Cycle counter Increments, counting cycles. If there are N iterations in a cycle, the Cycle Counter will count N times as fast as the Iteration Counter.
4 Iterations/Cycle counter Counts PPG iterations within each cycle. If there are N iterations per cycle, this counter should always show N.
5 Sweep counter Increments, counting the number of complete DAC sweeps (or scans).
6 Skipped-iteration counter Counts number of iterations skipped. If first iteration is discarded each cycle, this number will be same as Cycle Counter.
7 Num iterations histogrammed Should be the same as Iteration counter.
8 DAC Increment counter Counts DAC increments within a sweep. If the number of DAC increments selected per sweep is M, counter will vary from 0 to M.
9 DAC Set Value DAC Set value in Volts (DAC Output 0)
10 ADC0 Galil ADC0 Input (connected to DAC Output 0 ) i.e. DAC Readback in Volts
11 ADC1 Galil ADC1 Input (Volts)
12 ADC2 Galil ADC2 Input (Volts)
13 ADC3 Galil ADC3 Input (Volts)
14 ADC0 Average Average value of ADC Input 0 (DAC Readback in Volts)
15 ADC1 Average Average value of ADC Input 1 (Volts)
16 ADC2 Average Average value of ADC Input 2 (Volts)
17 ADC3 Average Average value of ADC Input 3(Volts)


NOTES

ADC Channel 1 (user-defined DAC readback) is also output in the #HISI Bank. Since the HISI bank is assembled at a slightly different time to the CYCL bank, the values in the two banks may differ because of jitter on the ADC readback. The average values are generated to average out this jitter.


Analysis

Python library for parsing data

The pol_data package contains tools for reading midas data and converting it to objects that can be used in python code. It is able to parse data from both the old DAQ and new DAQ. It is able to read both midas files and data from the live experiment. It is used as a basis for both the online and offline tools described below.

Online analysis - polana

The main POL DAQ repository contains a python-based analyzer that can generate histograms and display them on the "Web plots" webpage. The program is called "polana" and can be started using the relevant button the main DAQ webpage.

The "Web plots" webpage can show the following plots:

  • Sum of counts vs event time
  • Sum of counts vs DAC voltage (scanned variable)
  • Time spectrum (cumulative over the whole run)
  • Time spectrum (most recent cycle only)

You can select which scaler channels to display, and limit to only showing data for specific DAC voltages (by limiting which "scan steps" to display).

Offline analysis - gui

An interactive GUI for interacting with midas files and converting to CSV files etc was originally created by Tom Proctor and called "main.py". That version required the user to run "mdump" to convert the midas files to text, then parsed those text files. This was error-prone and required tweaks to the parsing code for each experimental campaign.

The new code reads midas files directly in python, and does not require the user to do any pre-processing of the data with mdump.

The program is found in the pol_data repository and is called gui.py. It can be started using the gui alias on midpol, or can be installed on your own laptop.

If running on midpol, you should make sure to connect with X-forwarding enabled, e.g.

$ ssh -Y pol@midpol
$ bash
$ gui

A screenshot of the GUI is shown below.

Pol gui.png