POL: Data format and analysis: Difference between revisions

From DaqWiki
Jump to navigation Jump to search
No edit summary
 
(2 intermediate revisions by the same user not shown)
Line 30: Line 30:
|5 || DAC Increment counter || Counts from 0 to Num Dac Increments
|5 || DAC Increment counter || Counts from 0 to Num Dac Increments
|-
|-
|6 || NumCycles Summed || Number of cycles summed in these histograms. Should be equal to the number of cycles per supercycle
|6 || NumCycles Summed || Number of iterations summed in these histograms.
|-
|-
|7 || Scaler buffer 1st word || DAC Set Value in Volts for this event stored in Scaler data buffer as the first word for checking.<br>
|7 || Scaler buffer 1st word || DAC Set Value in Volts for this event stored in Scaler data buffer as the first word for checking.<br>
Line 87: Line 87:
|}
|}


 
=== CYCL bank ===
 
=== CYCL Bank ===


{|  style="text-align: left; width: 100%; background-color: lightgoldenrodyellow;" border="3" cellpadding="2" cellspacing="2"
{|  style="text-align: left; width: 100%; background-color: lightgoldenrodyellow;" border="3" cellpadding="2" cellspacing="2"
Line 98: Line 96:
|1 || Scan type code || Always 1 -  indicates a DAC scan   
|1 || Scan type code || Always 1 -  indicates a DAC scan   
|-
|-
|2 || Cycle counter || Increments, counting cycles. Cycle Counter will count by N
|2 || Iteration counter || Increments, counting PPG iterations.
|-
|-
|3 || SuperCycle counter || Increments, counting supercycles. If there are N cycles in a supercycle, the Cycle Counter will count N times as fast as the Supercycle Counter.
|3 || Cycle counter || Increments, counting cycles. If there are N iterations in a cycle, the Cycle Counter will count N times as fast as the Iteration Counter.
|-
|-
|4 || Cycles/SC counter || Counts cycles within each supercycle. If there are N cycles per supercycle, this counter should always show N.
|4 || Iterations/Cycle counter || Counts PPG iterations within each cycle. If there are N iterations per cycle, this counter should always show N.
|-
|-
|5 || Sweep counter || Increments, counting the number of complete DAC sweeps (or scans).
|5 || Sweep counter || Increments, counting the number of complete DAC sweeps (or scans).
|-
|-
|6 || Skipped-cycle counter || Counts number of cycles skipped. If first cycle is discarded each SC, this number will be same as SuperCycle Counter.
|6 || Skipped-iteration counter || Counts number of iterations skipped. If first iteration is discarded each cycle, this number will be same as Cycle Counter.
|-
|-
|7 || Num cycles histogrammed || Should be the same as Cycle counter.
|7 || Num iterations histogrammed || Should be the same as Iteration counter.
|-
|-
|8 || DAC Increment counter|| Counts DAC increments within a sweep. If the number of DAC increments selected per sweep is M, counter will vary from 0 to M.
|8 || DAC Increment counter|| Counts DAC increments within a sweep. If the number of DAC increments selected per sweep is M, counter will vary from 0 to M.
Line 138: Line 136:




=== Example of EventID 5 ===
== Analysis ==
In this example, only Scaler Input 1 has been connected to a clock, so only the <span style="font-weight:bold;color:blue;">HIS1</span> bank has counts.  
 
<pre>
=== Python library for parsing data ===
------------------------ Event# 7 ------------------------
 
Evid:0005- Mask:0020- Serial:1- Time:0x5339eea8- Dsize:352/0x160
The [https://bitbucket.org/ttriumfdaq/pol_data pol_data] package contains tools for reading midas data and converting it to objects that can be used in python code. It is able to parse data from both the old DAQ and new DAQ. It is able to read both midas files and data from the live experiment. It is used as a basis for both the online and offline tools described below.
#banks:8 - Bank list:-CYCLHISIHIS0HIS1HIS2HIS3HSUM-
 
=== Online analysis - polana ===
 
The main POL DAQ repository contains a python-based analyzer that can generate histograms and display them on the "Web plots" webpage. The program is called "polana" and can be started using the relevant button the main DAQ webpage.


Bank:CYCL Length: 68(I*1)/17(I*4)/17(Type) Type:Real*4 (FMT machine dependent)
The "Web plots" webpage can show the following plots:
  1-> 1.000e+00 1.000e+03 5.000e+00 2.000e+02 1.000e+00 5.000e+00 1.000e+03 4.000e+00
  9-> 4.000e-02 4.150e-02 3.943e-01 9.000e-04 9.263e+00 4.150e-02 3.913e-01 0.000e+00
  17-> 9.263e+00


Bank:HISI Length: 28(I*1)/7(I*4)/7(Type) Type:Real*4 (FMT machine dependent)
* Sum of counts vs event time
  1-> 1.000e+03 5.000e+00 4.000e-02 3.958e-01 4.000e+00 1.000e+00 4.000e-02
* Sum of counts vs DAC voltage (scanned variable)
* Time spectrum (cumulative over the whole run)
* Time spectrum (most recent cycle only)


Bank:HIS0 Length: 400(I*1)/100(I*4)/100(Type) Type:Unsigned Integer*4
You can select which scaler channels to display, and limit to only showing data for specific DAC voltages (by limiting which "scan steps" to display).
  1-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  9-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  17-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  25-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  33-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  41-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  49-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  57-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  65-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  73-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  81-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  89-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  97-> 0x00000000 0x00000000 0x00000000 0x00000000


Bank:HIS1 Length: 400(I*1)/100(I*4)/100(Type) Type:Unsigned Integer*4
=== Offline analysis - gui ===
  1-> 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8
  9-> 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8
  17-> 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8
  25-> 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8
  33-> 0x000003e7 0x000003e9 0x000003e7 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8
  41-> 0x000003e8 0x000003e9 0x000003e7 0x000003e8 0x000003e9 0x000003e8 0x000003e7 0x000003e9
  49-> 0x000003e8 0x000003e8 0x000003e7 0x000003e9 0x000003e8 0x000003e8 0x000003e8 0x000003e8
  57-> 0x000003e8 0x000003e8 0x000003e9 0x000003e7 0x000003e9 0x000003e8 0x000003e8 0x000003e7
  65-> 0x000003e9 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e7
  73-> 0x000003e8 0x000003e8 0x000003e9 0x000003e8 0x000003e7 0x000003e9 0x000003e7 0x000003e8
  81-> 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e7
  89-> 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8 0x000003e8
  97-> 0x000003e8 0x000003e8 0x000003e8 0x000003e8


Bank:HIS2 Length: 400(I*1)/100(I*4)/100(Type) Type:Unsigned Integer*4
An interactive GUI for interacting with midas files and converting to CSV files etc was originally created by Tom Proctor and called "main.py". That version required the user to run "mdump" to convert the midas files to text, then parsed those text files. This was error-prone and required tweaks to the parsing code for each experimental campaign.
  1-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  9-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  17-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  25-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  33-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  41-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  49-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  57-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  65-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  73-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  81-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  89-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  97-> 0x00000000 0x00000000 0x00000000 0x00000000


Bank:HIS3 Length: 400(I*1)/100(I*4)/100(Type) Type:Unsigned Integer*4
The new code reads midas files directly in python, and does not require the user to do any pre-processing of the data with mdump.
  1-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  9-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  17-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  25-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  33-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  41-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  49-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  57-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  65-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  73-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  81-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  89-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  97-> 0x00000000 0x00000000 0x00000000 0x00000000


Bank:HSUM Length: 32(I*1)/8(I*4)/4(Type) Type:double*8
The program is found in the [https://bitbucket.org/ttriumfdaq/pol_data pol_data] repository and is called <code>gui.py</code>. It can be started using the <code>gui</code> alias on midpol, or can be installed on your own laptop.
  1->     0.00000e+00        9.99990e+04        0.00000e+00        0.00000e+00


</pre>
If running on midpol, you should make sure to connect with X-forwarding enabled, e.g.


== Analysis ==
$ ssh -Y pol@midpol
$ bash
$ gui


=== Online analysis ===
A screenshot of the GUI is shown below.


=== Offline analysis ===
[[Image:Pol_gui.png|800px]]


[[Category:POL]]
[[Category:POL]]

Latest revision as of 13:56, 17 February 2023

Links

Introduction

This document describes the MIDAS format data banks that are output by the POL frontend fepol_32bit.exe. This frontend controls the hardware for the experiment, and programs the contents of the data banks. The analyzer reads the data from the data banks.

MIDAS Data

The data are sent out as MIDAS events which contain various banks: HISI HIS0 HIS1 HIS2 HIS3 HSUM CYCL. Some of these banks contain duplicated information (e.g. HISI vs CYCL), but they are all kept for backwards compatibility.

HISI Bank

Table 4 : HISI Histo counter bank Type: FLOAT EventId: 5
Word Contents Explanation
1 Cycle counter Increments, counting cycles. This will count by N where N is the requested number of cycles per supercycle.
2 SuperCycle counter Increments, counting supercycles.
3 Set Value DAC Set Value in Volts for this event
4 Set Value Readback Users' Readback (ADC Channel 1 Average value - see also CYCL bank ). See DAC Set value Readback **.
5 DAC Increment counter Counts from 0 to Num Dac Increments
6 NumCycles Summed Number of iterations summed in these histograms.
7 Scaler buffer 1st word DAC Set Value in Volts for this event stored in Scaler data buffer as the first word for checking.

Should always be equal to word 3 (but may have rounding errors due to conversion to integer mV and back).


Notes:

  • Cycle counter contains the user-requested number of cycles, so if parameter Discard first cycle is set, the first cycle will already have been discarded. Compare with #CYCL bank, where the actual number of cycles is recorded.
  • ** The DAC is used to drive a high voltage power supply. The Set Value Readback is read from ADC Channel 1, which is connected by the users to their own voltage readback. The actual readback of the DAC itself is found in #CYCL bank, read from ADC Channel 0.


HIS0,HIS1,HIS2,HIS3 Banks

The banks HIS0...HIS3 are the histogram banks for Scaler Input channels 0-3. These contain the cumulative sums of each time bin summed over the number of cycles/supercycle at a particular DAC increment value. Summing all the bins together for each channel should give the same value as in the HSUM bank. The histograms are cleared after each supercycle.

Table 5 : HIS0..3 Histogram Sums for Scaler Inputs 0..3 Type: Unsigned Integer EventId: 5
Word Contents
1 Time bin 0 summed for one SuperCycle
2 Time bin 1 summed for one SuperCycle
... ............
... ............
N Last time bin summed for one SuperCycle

HSUM Bank

The HSUM bank contains the sums of all the time-bins (output as histograms in Event 5) for each Scaler Channel. The sums are cleared after each SuperCycle.


Table 6 : HSUM Histogram Sums for Scaler Inputs 0..3 Type: FLOAT EventId: 5
Word Contents
1 Sum of all the time bins of Scaler input 0 for one SuperCycle
2 Sum of all the time bins of Scaler input 1 for one SuperCycle
3 Sum of all the time bins of Scaler input 2 for one SuperCycle
4 Sum of all the time bins of Scaler input 3 for one SuperCycle

CYCL bank

Table 7 : CYCL Cycle counter/ADC Readback bank Type: FLOAT EventId: 5
Word Contents Explanation
1 Scan type code Always 1 - indicates a DAC scan
2 Iteration counter Increments, counting PPG iterations.
3 Cycle counter Increments, counting cycles. If there are N iterations in a cycle, the Cycle Counter will count N times as fast as the Iteration Counter.
4 Iterations/Cycle counter Counts PPG iterations within each cycle. If there are N iterations per cycle, this counter should always show N.
5 Sweep counter Increments, counting the number of complete DAC sweeps (or scans).
6 Skipped-iteration counter Counts number of iterations skipped. If first iteration is discarded each cycle, this number will be same as Cycle Counter.
7 Num iterations histogrammed Should be the same as Iteration counter.
8 DAC Increment counter Counts DAC increments within a sweep. If the number of DAC increments selected per sweep is M, counter will vary from 0 to M.
9 DAC Set Value DAC Set value in Volts (DAC Output 0)
10 ADC0 Galil ADC0 Input (connected to DAC Output 0 ) i.e. DAC Readback in Volts
11 ADC1 Galil ADC1 Input (Volts)
12 ADC2 Galil ADC2 Input (Volts)
13 ADC3 Galil ADC3 Input (Volts)
14 ADC0 Average Average value of ADC Input 0 (DAC Readback in Volts)
15 ADC1 Average Average value of ADC Input 1 (Volts)
16 ADC2 Average Average value of ADC Input 2 (Volts)
17 ADC3 Average Average value of ADC Input 3(Volts)


NOTES

ADC Channel 1 (user-defined DAC readback) is also output in the #HISI Bank. Since the HISI bank is assembled at a slightly different time to the CYCL bank, the values in the two banks may differ because of jitter on the ADC readback. The average values are generated to average out this jitter.


Analysis

Python library for parsing data

The pol_data package contains tools for reading midas data and converting it to objects that can be used in python code. It is able to parse data from both the old DAQ and new DAQ. It is able to read both midas files and data from the live experiment. It is used as a basis for both the online and offline tools described below.

Online analysis - polana

The main POL DAQ repository contains a python-based analyzer that can generate histograms and display them on the "Web plots" webpage. The program is called "polana" and can be started using the relevant button the main DAQ webpage.

The "Web plots" webpage can show the following plots:

  • Sum of counts vs event time
  • Sum of counts vs DAC voltage (scanned variable)
  • Time spectrum (cumulative over the whole run)
  • Time spectrum (most recent cycle only)

You can select which scaler channels to display, and limit to only showing data for specific DAC voltages (by limiting which "scan steps" to display).

Offline analysis - gui

An interactive GUI for interacting with midas files and converting to CSV files etc was originally created by Tom Proctor and called "main.py". That version required the user to run "mdump" to convert the midas files to text, then parsed those text files. This was error-prone and required tweaks to the parsing code for each experimental campaign.

The new code reads midas files directly in python, and does not require the user to do any pre-processing of the data with mdump.

The program is found in the pol_data repository and is called gui.py. It can be started using the gui alias on midpol, or can be installed on your own laptop.

If running on midpol, you should make sure to connect with X-forwarding enabled, e.g.

$ ssh -Y pol@midpol
$ bash
$ gui

A screenshot of the GUI is shown below.

Pol gui.png