Data Formats: Difference between revisions

From MidasWiki
Jump to navigation Jump to search
(Created page with "== Data Formats supported by the frontend == Only two formats are now supported by the frontend, MIDAS and @ref FE_FIXED_event_readout "FIXED" @ref FE_MIDA...")
 
No edit summary
 
(4 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Data Formats supported by the frontend ==
{{Pagelinks}}
= Links =
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* [[Frontend Operation]]
* [[Event Structure]]
* [[mlogger]]
</div>


Only two formats are now supported by the frontend,  [[MIDAS format|MIDAS]] and @ref FE_FIXED_event_readout "FIXED"
== Data Formats in MIDAS ==
@ref  FE_MIDAS_event_construction has been described
previously. This section gives a detailed description of the MIDAS format.


Note that a frontend cannot write data directly into ROOT format. A conversion to ROOT may be done (e.g. by the data logger) from one of the
Only two formats are now supported by the  [[Frontend Operation#Frontend|Frontend]],  [[Event Structure|"MIDAS"]] and [[Event Structure#FIXED Format Event|"FIXED"]]. See [[MIDAS Event Construction]] for details.
supported formats (see @ref FE_tbl_Format "equipment list" for details).
[[Event Structure]] gives a detailed description of the available formats.


Note that a frontend cannot write data directly into ROOT format. A conversion to ROOT may be done from one of the supported Data Formats (e.g. by the data logger  - see [[mlogger#logging in MIDAS|logging in MIDAS]]). See also [[Equipment List Parameters#Format]] for details).


== MIDAS format ==
[[Category:Data Logging]]
Special formats are used in MIDAS for the event header, banks and when writing to disk
or tape.
This section explains these formats in detail.
Each event carries a 16-byte header. The header is generated by the front-end with the
'''bm_compose_event()''' routine and is used by consumers to distinguish between
different events.
The header is defined in the EVENT_HEADER structure in the file '''midas.h''' in the midas package.
It has following structure:
 
[[File:MEHBH.jpg|Figure 1:  Structure of Event and Bank headers with data block.]]
 
 
* The @ref  FE_tbl_EventID "event ID" describes the type of event. Usually 1 is used for triggered events, 2 for scaler events, 3 for HV events etc.
* The  @ref FE_tbl_TrigMask "trigger mask" can be used to describe the sub-type of an event. A trigger event can have different trigger sources like "physics event", "calibration event", "clock event". These trigger sources are usually read in by the front-end in a pattern unit. Consumers can request events with a specific triggering mask.
* The <b>serial number</b> starts at 1 and is incremented by the front-end for each event.
* The <b>time stamp</b> is written by the front-end before an event is read out. It uses the time() function which returns the time in seconds since 1.1.1970 00:00:00 UTC.
* The <b>data size</b> contains the number of bytes that follows the event header.
* The <b>data area</b> of the event can contain information in any user format (integer, real etc.), although only certain formats are supported when events are copied to the ODB or written by the logger in ASCII format.
 
<div id="byte-ordering"></div>
Event headers are always kept in the <b>byte ordering</b> of the local machine. If events are sent over the network between computers with different byte
ordering, the event header is swapped automatically, but not the event contents.
 
=== Bank Format ===
Events in MIDAS format contain "MIDAS banks". A bank is a substructure of an event
and can contain only one type of data, either a single value or an array of values.
Banks have a name of exactly four characters, which are treated as a <b>bank ID</b>.
Banks in an event consist of a \b global bank header and an \b individual bank header
for each bank.
 
Figure 2 shows a MIDAS event containing banks coloured to match the structure in Figure 1. This has been
obtained from a MIDAS data file using the  @ref RC_mdump_utility "mdump utility".
 
<br><br>
<span class="image"> Figure 2: Example of MIDAS banks dumped by <span class="client">mdump</span>.</span>
<br>
------------------------ Event# 2 --------------------------------<br>
<span style="background-color: rgb(204, 204, 255);">Evid:000d- Mask:0000- Serial:0- Time:0x4c7a6869- Dsize:48/0x30</span><br>
<span style="background-color: lightgreen;">\#banks:1 - Bank list:-SDAS-</span><br>
<br>
<span style="background-color: orange;">Bank:SDAS Length: 32(I*1)/8(I*4)/8(Type)Type:Real*4 (FMT machine dependent)</span><br>
<span style="background-color: pink;">  1-> 4.000e+00 1.000e+01 1.000e+00 3.400e+00 3.400e+00 3.400e+00 3.400e+00 3.400e+00</span><br>
<br>
------------------------ Event# 3 --------------------------------<br>
<span style="background-color: rgb(204, 204, 255);">Evid:0001- Mask:0000- Serial:0- Time:0x4c7a686b- Dsize:344/0x158</span><br>
<span style="background-color: lightgreen;">\#banks:2 - Bank list:-MPETMCPP-</span><br>
<br>
<span style="background-color: orange;">Bank:MPET Length: 304(I*1)/76(I*4)/76(Type) Type:Unsigned Integer*4</span><br>
<span style="background-color: pink;">    1-> 0x80010000 0x00000002 0x10010000 0x00004e21 0x80020000 0x00000002 0x20020000 0x000015f4<br>
  9-> 0x20020000 0x00001660 0x20020000 0x0000185f 0x20020000 0x0000191e 0x20020000 0x000019d6<br>
  17-> 0x40020000 0x00001a37 0x20020000 0x00001a77 0x20020000 0x00001ba2 0x10020000 0x00004e22<br>
  25-> 0x80030000 0x00000002 0x20030000 0x00001637 0x20030000 0x000018d1 0x20030000 0x000019bc<br>
  33-> 0x20030000 0x00001b35 0x20030000 0x00001bb2 0x10030000 0x00004e21 0x80040000 0x00000002<br>
  41-> 0x10040000 0x00004e22 0x80050000 0x00000002 0x20050000 0x000013c5 0x20050000 0x000017f2<br>
  49-> 0x20050000 0x0000185f 0x20050000 0x00001976 0x20050000 0x00001aa8 0x10050000 0x00004e21<br>
  57-> 0x80060000 0x00000002 0x20060000 0x000015c3 0x20060000 0x000018d8 0x20060000 0x0000198d<br>
  65-> 0x20060000 0x00001ac4 0x10060000 0x00004e22 0x80070000 0x00000002 0x20070000 0x00001747<br>
  73-> 0x20070000 0x000019ae 0x10070000 0x00004e21</span><br>
<br>
<span style="background-color: orange;">Bank:MCPP Length: 16(I*1)/4(I*4)/4(Type) Type:Unsigned Integer*4</span><br>
<span style="background-color: pink;">  1-> 0x00005e4c 0x0000352d 0x00006453 0x00006d5b</span><br>
<br>
 
The "data size total" is the size in bytes of all bank headers and bank data.
Flags are currently not used. The bank header contains four characters as
identification, a bank type that is one of the TID_xxx values defined in
midas.h, and the data size in bytes.
If the byte ordering of the contents of a complete event has to be swapped,
the routine bk_swap() can be used.
 
<br><hr><br>
\anchor idx_format_Midas_Tape
@subsection Tape Format
Events are written to disk files without any reformatting.
For tapes, however, a fixed block size is used. The block size TAPE_BUFFER_SIZE
is defined in midas.h and usually 32kB. Three special events are produced
by the system. A begin-of-run (BOR) and end-of-run (EOR)
event is produced which contains an ASCII dump of the ODB in its data area.
Their IDs is 0x8000 (BOR) and 0x8001 (EOR). A message event (ID 0x8002) is created
if Log messages is enabled in the logger channel setting.
The message is contained in the data area as an ASCII string.
The BOR event has the number MIDAS_MAGIC (0x494d or 'MI') as the trigger mask
and the current run number as the serial number.
A tape can therefore be identified as a MIDAS formatted tape.
The routine tape_copy() in the utility mtape.c is an example of how to read
a tape in MIDAS format.

Latest revision as of 14:32, 5 August 2015


Links

Data Formats in MIDAS

Only two formats are now supported by the Frontend, "MIDAS" and "FIXED". See MIDAS Event Construction for details. Event Structure gives a detailed description of the available formats.

Note that a frontend cannot write data directly into ROOT format. A conversion to ROOT may be done from one of the supported Data Formats (e.g. by the data logger - see logging in MIDAS). See also Equipment List Parameters#Format for details).

Retrieved from "https://daq00.triumf.ca/MidasWiki/index.php?title=Data_Formats&oldid=2094"