Midas supports two differents data format so far. A possible new candidate would be the NeXus format, but presently no implementation has been developed.
The event ID describes the type of event. Usually 1 is used for triggered events, 2 for scaler events, 3 for HV events etc. The trigger mask can be used to describe the sub-type of an event. A trigger event can have different trigger sources like "physics event", "calibration event", "clock event". These trigger sources are usually read in by the front-end in a pattern unit. Consumers can request events with a specific triggering mask. The serial number starts at one and is incremented by the front-end for each event. The time stamp is written by the front-end before an event is read out. It uses the time() function which returns the time in seconds since 1.1.1970 00:00:00 UTC. The data size contains the number of bytes that follows the event header. The data area of the event can contain information in any user format, although only certain formats are supported when events are copied to the ODB or written by the logger in ASCII format. Event headers are always kept in the byte ordering of the local machine. If events are sent over the network between computers with different byte ordering, the event header is swapped automatically, but not the event contents.
YBOS is organized on a 4-byte alignment structure.
The YBOS library function provides all the tools for manipulation of the above mentioned elements in a independent Operating System like. But the implementation of the YBOS part in Midas does not use any reference to the YBOS library code. Instead only the strict necessary functions have to be re-written in C and incorporated into the Midas package. This has been motivated by the fact that only a sub-set of function is essential to the operation of:
This Midas/YBOS implementation restricts the user to a subset of the YBOS package only for the front-end part. It doesn't prevent him/her to use the full YBOS library for stand alone program accessing data file written by Midas.
The YBOS implementation under Midas has the following restrictions:
YBOS uses "double word" unit for all length references.
The bank length parameter corresponds to the size of the data section in double word count + 1. The supported bank type are defined in the ybos.h file see YBOS Bank Types.
The Offset is computed with the following rules:
This is the internal format of MIDAS .hst files (reverse engineered by K.Olchanski).
History file consists of definition records and data records. Definition records for each history event always preceed data records for this event. Other than that, definition and data records are present in the file in no particular order. If data definition changes at some point, the new definition record is written to the file and new data records follow using the new format.
The data is written in native-endian format and big-endian history files (i.e. written on PowerPC-based computer) would not read on little-endian computers (i.e. Intel/AMD CPUs).
When decoding history data records, be aware that the data is written on the format returned by db_get_data(), and follow certain alignement rules. Misaligned data may contain empty padding bytes.
definition record: struct HIST_RECORD: uint32_t record_type = 0x46445348 (RT_DEF) uint32_t event_id = history event id uint32_t time = unix time uint32_t def_offset = 0 uint32_t data_size = size in bytes of following data = ntags*sizeof(TAG) event name: char name[NAME_LENGTH] = name of this event (fixed size, 32 bytes) struct TAG [0] char name[NAME_LENGTH] = tag name (fixed size, 32 bytes) uint32_t type = type of the data (TID_DOUBLE, etc) uint32_t n_data = 1 for single variables, number of array elements for arrays struct TAG [1] ... struct TAG [ntags-1] ...
data record: struct HIST_RECORD: (same as above) uint32_t record_type = 0x41445348 (RT_DATA) uint32_t event_id = history event id uint32_t time = unix time uint32_t def_offset = ??? uint32_t data_size = size in bytes of following data event data char[data_size] = history data for this event (see mhdump.cxx and examples below on decoding this data
To decode the "event data" blob in a data record, one should parse the data definition for this event and remember the contents of all tag definitions. One way to parse the data is by computing the "offset" for each tag by adding up the tag size for each tag. For each tag, the corresponding data size is the product of n_data and the size of the MIDAS type, given in this table (extracted from midas.c):
/* data type sizes */ int tid_size[] = { 0, /* tid == 0 not defined */ 1, /* TID_BYTE unsigned byte 0 255 */ 1, /* TID_SBYTE signed byte -128 127 */ 1, /* TID_CHAR single character 0 255 */ 2, /* TID_WORD two bytes 0 65535 */ 2, /* TID_SHORT signed word -32768 32767 */ 4, /* TID_DWORD four bytes 0 2^32-1 */ 4, /* TID_INT signed dword -2^31 2^31-1 */ 4, /* TID_BOOL four bytes bool 0 1 */ 4, /* TID_FLOAT 4 Byte float format */ 8, /* TID_DOUBLE 8 Byte float format */ 1, /* TID_BITFIELD 8 Bits Bitfield 00000000 11111111 */ 0, /* TID_STRING zero terminated string */ 0, /* TID_ARRAY variable length array of unkown type */ 0, /* TID_STRUCT C structure */ 0, /* TID_KEY key in online database */ 0 /* TID_LINK link in online database */ };
Example code for computing tag offsets:
int offset = 0; for (int itag=0; itag<ntags; itag++) { int size = tags[itag].n_data * tid_size[tags[itag].type]; type_for_tag[itag] = tags[itag].type; offset_for_tag[itag] = offset; offset += size; }
Example code for decoding the event data blob:
char *buf = new char[data_size]; fread(buf, 1, data_size, f); // read data blob from file int itag = ...; // the tag we want to look at int j = ...; // array index inside the tag int offset = offset_for_tag[itag]; void* ptr = (void*)(buf+offset); switch (type_for_tag[itag]) { default: printf("unknownType%d ",type_for_tag[itag]); break; case 6: /* DWORD */ printf("%u ",((uint32_t*)ptr)[j]); break; }
Utilities - Top - Supported hardware