Some notes on the MIDAS history.
MIDAS documentation at
- midas equipment concepts
- midas equipment event ids
- midas data banks
- midas history concepts
- history records (correspond to data banks)
- history record ids (correspond to equipment ids)
- history tags (describe the structure
- describes the code path from the user read function through odb to the mlogger to the history file
- midas history file internal data format
- documents the tool for looking inside history files - mhdump
But some things remain unclear after reading the documentation - where are the history definitions
saved? what happens if an equipment is deleted or renamed? what's all the mumbling about
/History/Events and /History/Tags? what's this /History/PerVariableHistory?
As I go through my review of the MIDAS history code, I will attempt to clarify some of this information.
The default value of 0 is intended to operate the midas history in "traditional" mode. In this mode:
- there is one history record for each equipment
- history record id is equal to the equipment id
- /History/Events and /History/Tags are not required and can be safely deleted
The downside of this history mode is that there is only one history record per equipment. If some
equipment has many banks not all of which are updated all at the same time, every time one bank is
updated, data for all banks is written to the history file, even if data in all those other banks had not
changed. The result is undesired duplication of data in midas history files. In turn, this leads to slow
down while making history plots (mhttpd has to read more data from bigger data files, which takes time)
and for long running experiments may pose problems with disk space for storing history files.
In addition, when logging history data into an SQL database, each history record is mapped into an SQL
table, so all variables from all banks of an equipment end up in the same SQL table - and in addition to
data duplication described above, a data presentation problem is created - database users and
administrators dislike having SQL tables with "too many" columns!
To solve both problems - reduce data duplication and avoid creating over-large SQL tables - per-
variable history has been implemented.
to be continued...