BNMR: Data Logging: Difference between revisions
No edit summary |
No edit summary |
||
Line 115: | Line 115: | ||
Only a single number gets written to the run description for each variable - the average value throughout the run. | Only a single number gets written to the run description for each variable - the average value throughout the run. | ||
NOTE: Automatic CAMP variables to be logged must be set ''loggable'' in the CAMP CUI - see [[#Set CAMP variable loggable]]. | |||
==== Independent variables ==== | ==== Independent variables ==== | ||
Line 188: | Line 188: | ||
=== ODB variables === | === ODB variables === | ||
==== Independent variables ==== | |||
ODB variables specifying the configuration of the run are stored as independent variables in the MUD file. The user does not specify directly which variables log for each mode; the decision is made by the [[BNMR:_RF_calculator|RF calculator]] program. | ODB variables specifying the configuration of the run are stored as independent variables in the MUD file. The user does not specify directly which variables log for each mode; the decision is made by the [[BNMR:_RF_calculator|RF calculator]] program. | ||
Line 204: | Line 206: | ||
See the [[BNMR:_Mode_changer|mode changer]] page for details about how the user may load settings from a MUD file to replicate a previous experimental configuration. | See the [[BNMR:_Mode_changer|mode changer]] page for details about how the user may load settings from a MUD file to replicate a previous experimental configuration. | ||
==== Run description ==== | |||
Some ODB variables are also used to build the run description in the MUD file (along with any [[#Run_description|CAMP variables]] being used for the run description). | |||
The variables used are: | |||
* {{Odbpath|path=/Experiment/Edit on start/experiment number}} for the <code>exptNumber</code> | |||
* {{Odbpath|path=/Experiment/Edit on start/experimenter}} for the <code>experimenter</code> | |||
* {{Odbpath|path=/Experiment/Edit on start/run_title}} for the <code>run_title</code> | |||
* {{Odbpath|path=/Experiment/Edit on start/sample}} for the <code>sample</code> | |||
* {{Odbpath|path=/Experiment/Edit on start/orientation}} for the <code>orient</code> | |||
* {{Odbpath|path=/Experiment/Name}} for the <code>area</code> and <code>apparatus</code>, | |||
* {{Odbpath|path=/CurrentMode/Name}} for the <code>insert</code> (hijacking a field used by MUSR but not {{bnmqr|join=or}}) | |||
* {{Odbpath|path=/Runinfo/Start time binary}} for the <code>timeBegin</code> | |||
* {{Odbpath|path=/Runinfo/Stop time binary}} for the <code>timeEnd</code> | |||
* {{Odbpath|path=/MUD Logging/Run description/Settings/Manual field value}} if not reading from CAMP | |||
* {{Odbpath|path=/MUD Logging/Run description/Settings/Manual temperature value}} if not reading from CAMP | |||
See [[BNMR:_Edit-on-start_parameters|edit-on-start parameters]] for more details on those variables. | |||
[[Category:BNMR]] [[Category:Logging]] | [[Category:BNMR]] [[Category:Logging]] |
Revision as of 11:52, 27 April 2022
MUD files
bnmr and bnqr do not save their data as Midas files, but use MUD files instead (.msr
extension).
Therefore we only use the Midas mlogger for recording history data, not the regular event data.
The key feature to note is that MUD files are not event-based. One can save statistics and histograms, but not complex per-event data. Histograms may be used to a single number per event (the X-axis is event number) or the total counts over an entire run (the X-axis is time-of-flight in ns, for example). Data saved as statistics can include the mean/min/max/std.dev. and more information.
Users analyze MUD files using:
- the mudpy and bdata python packages (with some GUIs built on top of them)
- the MUSR/BNMR analyzer program physica
- the MUD utility program mud_util
A couple of python utilities based on bdata
are provided in the bnmr repository, which may be useful when adding new features to the data logger:
bnxr_common/tests/dump_mud_file.py
dumps the content of a MUD file to screenbnxr_common/tests/diff_mud_files.py
compares the content of 2 MUD files
DAQ experts may enable dumping of Midas banks to a Midas file for debugging purposes. The banks written will come from the BNMR:_frontend frontend program only, and will not contain any EPICS variables, CAMP variables, or any manipulation of histograms performed in the bnxr_logger.exe
program.
bnxr_logger.exe introduction
The bnxr_logger.exe
program is responsible for collecting data and writing to MUD files.
Most of the parameters controlling the data logging are found in the ODB at /MUD Logging. Most settings are configured manually, though some are set/computed by the RF calculator program and the beginning of each run.
The program gets data from the following sources:
- Histograms - read from midas banks sent by the frontend
- Camp - read directly from CAMP
- EPICS - read directly from EPICS
- ODB variables - read directly from the ODB
- Run comments - read from a text file that is edited via a web interface
More details on each of these is presented further down this page.
Saved data files
The bnmr and bnqr experiments use the MUSR-style Run Numbering scheme. The data files are named according to the run number. Real data files are archived in the MUSR Data Archive at the end of the run.
During a run, several versions of the .msr (or MUD) data file are retained with a version number appended (in case of any problems with the most recent saved data). A symlink to the most recent version is updated during the run, so analysis programs do not have to concern themselves with the versioning information.
For example, for BNMR run 40362, while the run is in progress, the saved data directory would contain files such as
lrwxrwxrwx 1 bnmr bnmr 48 Feb 26 12:09 040362.msr -> /isdaq/data1/bnmr/dlog/current/040362.msr_v27 -rw-r--r-- 1 bnmr bnmr 4403 Feb 26 12:09 040362.msr_v27 -rw-r--r-- 1 bnmr bnmr 4403 Feb 26 12:09 040362.msr_v26 -rw-r--r-- 1 bnmr bnmr 4403 Feb 26 12:09 040362.msr_v25 -rw-r--r-- 1 bnmr bnmr 4403 Feb 26 12:08 040362.msr_v24
At end-of-run, normally all earlier versions are purged and the final data file is renamed without the appended version number, and the file archived to the MUSR Data Archive. These steps can be suppressed by setting /MUD Logging/Global/Settings/Purge and archive at end of run to false, for example if the user decides it would be better to keep the version 26 of a file instead of the most recent version 27.
Using the same example as before, after the run the saved data directory would contain:
-rw-r--r-- 1 bnmr bnmr 4399 Feb 26 12:10 040362.msr
Archiving
The ODB keys Archiver task and Archiver dir in /MUD Logging/Global/Settings determine where files for "real" runs get copied to after a run is complete. This is normally just an scp
to a directory on cmms.triumf.ca
.
Nuking a run
If the user decides that a run is irredeemable, they may choose to "nuke" the run and start again. This involves deleting ALL the files that have been written, pretending that the run never happened. The same run number will then be used for the next run. Users appreciate this feature as it allows analysis to occur on contiguous ranges of run numbers.
Indicating that a run should be nuked is done with the /MUD Logging/Global/Settings/Nuke this run flag in the ODB, which is generally achieved by the user clicking the "Nuke ⚠️" button on the Status page
Cleaning up if end-of-run fails
The logger_cleanup.exe program can be used if bnxr_logger.exe crashes during a run and fails to perform the standard cleanup/archive duties at the end of a run.
It effectively just runs the same code as the main MUD logger runs at the end of the run, but allows the user to specify any run number in the past if they wish.
It will:
- Delete any temporary files that are no longer needed
- Archive the final file if this was a "real" run (not a "test" run)
Usage is:
logger_cleanup.exe [-r Run number] Run number of run to be cleaned up and/or archived. Defaults to the most recent run. [-h Hostname ] hostname is supplied only for a remote host [-e Experiment] experiment name (default is given by environment variable MIDAS_EXPT_NAME)
bnxr_logger.exe details
The code for the MUD logger is found in the bnmr repository at bnxr_common/logging
. bnmr and bnqr are very similar, so the executable is almost identical for both (just an #ifdef
for the default ODB settings to use). The main configuration is all done through ODB keys.
The program is built as part of the main build (cd build; make install
from the bnmr repository).
The general functions performed by the program are:
- Read data sent by the frontend and process it. This includes both the data from the scalers as well as metadata like the current cycle number.
- Read data from EPICS and CAMP, either periodically or each time we get data from the frontend.
- Write data to a MUD file periodically during the run.
Histograms
TODO
CAMP
CAMP data is logged in two separate sections of the MUD file - in the run description and as independent variables. The bnxr_logger.exe program connects directly to the CAMP server and reads the values from there.
Run description
The MUD run description section has entries for the magnetic field strength and temperature of the sample. This data can be exposed to users when searching for data files taken previously.
ODB settings in /MUD Logging/Run description/Settings specify whether to read the magnetic field and/or temperature from CAMP, and if so, which instrument to read from. CAMP can automatically compute the standard deviation of variables throughout a run, and the user can choose to record this value in the file too.
Only a single number gets written to the run description for each variable - the average value throughout the run.
NOTE: Automatic CAMP variables to be logged must be set loggable in the CAMP CUI - see #Set CAMP variable loggable.
Independent variables
Other CAMP variables can be logged during a run too.
ODB settings in /MUD Logging/Camp/Settings define whether CAMP variables should be written as histograms of readings or just average values. The general configuration is to log a per-cycle histograms for T1 (TI) modes and just average values for T2 (TD) modes.
The user does not list each individual CAMP variable to be logged in the ODB, but instead lists which "categories" of variables should be included in the MUD file (e.g. log_mdarc
). The mapping between CAMP variable and "category" is done via CAMP itself (see next section).
This CAMP data is stored as "independent variables" in the MUD file, keyed by the CAMP path (of format like /Magnet/mag_field
).
Set CAMP variable loggable
All CAMP variables to be logged must be set loggable in the CAMP user interface. This includes the parameters to be used in the run description.
The loggable categories are shown in Table 1.
Set logging | Variable Logged for | |
---|---|---|
Type1 (I-MUSR) | Type2 (TD-MUSR) | |
log_mdarc | Y | Y |
log_mdarc_td | N | Y |
log_td_musr | ||
log_mdarc_ti | Y | N |
log_i_musr |
Checking which variables will be logged
The "Logging" custom webpage includes a feature that allows the user to see which CAMP variables will be logged given the current ODB and CAMP settings. This is implemented using the midas JRPC system to talk to the bnxr_logger.exe
program, which returns the list of variables.
CAMP error file
CAMP keeps a log file for errors. This can be found in camp_srv.log at path /home/musrdaq/musr/camp/[camphost]/log/ where camphost is bnmrvw for bnmr or polvw for bnqr. Error counts can also be found on the CAMP CUI under R.Errors and W.Errors in Figure 1 above.
e.g.
tail -f ~musrdaq/musr/camp/bnmrvw/log/camp_srv.log [JUN 26 17:46:41:2018] camp_srv: instrument 'biasV' not online in insIfWriteVerify /biasV | failed 2 attempts [JUN 26 17:46:43:2018] camp_srv: instrument 'biasV' not online in insIfWriteVerify /biasV | failed 2 attempts [JUN 26 17:46:45:2018] camp_srv: instrument 'biasV' not online in insIfWriteVerify /biasV | failed 2 attempts
EPICS
See the EPICS page for an overview of EPICS in bnmr and bnqr.
Users directly specify which EPICS variables should be logged in /MUD Logging/Epics/Settings (or by using the relevant bit of the Custom Logging page). The bnxr_logger.exe program connects directly to EPICS and reads the values itself.
The code allows for deciding whether to store a histogram of readings throughout the run or just summary statistics. In general, a histogram is written for TI runs, where the scaler data is being store in a per-cycle style; the histograms for the EPICS data and scaler data will be the same length, with one entry per PPG cycle. To avoid overloading the EPICS server, we can read from it periodically rather than every time a new PPG cycle starts. Each time we read a value, we will apply it to all the cycles that happened since we last read.
EPICS data is stored as "independent variables" in the MUD file, keyed by the PV name (of format like ILE2:BIAS15:RDVOL
).
Run comments
When a run is in progress, the user is presented with a textarea
element on the status page. This allows the user to note any critical information about the run that should be stored in the file, not just in a separate logbook. The textarea is hooked up to the run_comment_editor.py program using Midas JRPC. The python program just writes a text file on disk in the directory specified by /Logger/Data dir. Other settings for the python program are in /Equipment/RunComments/Settings.
When it is time for the bnxr_logger.exe program to write a new MUD file, it just reads the file that was written by the python program. There are a couple of extra settings in /MUD Logging/TextFileComments/Settings.
MUD files have a section dedicated to storing run comments.
ODB variables
Independent variables
ODB variables specifying the configuration of the run are stored as independent variables in the MUD file. The user does not specify directly which variables log for each mode; the decision is made by the RF calculator program.
In particular, the variables to be saved are stored as ODB links in the /Equipment/RFCalculator/Computed/Mud links/PPG/PPG20 directory (for mode 20; similar names for other modes). The definition of the links is in mud_ind_var_links.py
, which is included by rf_calculator_fe.py. This keeps all the mode-specific complexity in one place, and in a language that is more expressive. The C++ code in bnxr_logger.exe simply has to iterate over all the links in the relevant directory.
Note the following conversions between C++/ODB data types and MUD independent variables:
- Arrays/vectors are not supported (in principle we could store a histogram, but users have not requested that yet)
- Numbers are stored in the
mean
member of the MUD variable. - Strings are stored in the
units
member of the MUD variable. - Booleans are stored with 1/0 in the
mean
member and yes/no in theunits
member of the MUD variable. - If the ODB link name contains parentheses, the string between the parenthesis is stored in the
units
member of the MUD variable (e.g. a link calledFrequency (Hz)
will end up withHz
as the units). - If the ODB link name does not contain parentheses (e.g.
Frequency
), but there is another ODB key calledFrequency ((UNITS))
, the string in the latter key is stored as theunits
for the former. This is for backwards-compatibility with older files that were not consistent about whether MUD variables contain units in thename
, or only in theunits
.
ODB settings are stored as "independent variables" in the MUD file, keyed by the ODB link name (of format like PPG/PPG20/Enable RF
to be backwards-compatible with older files). To be clear, the name of the link itself is used, not the name of the target of the link.
See the mode changer page for details about how the user may load settings from a MUD file to replicate a previous experimental configuration.
Run description
Some ODB variables are also used to build the run description in the MUD file (along with any CAMP variables being used for the run description).
The variables used are:
- /Experiment/Edit on start/experiment number for the
exptNumber
- /Experiment/Edit on start/experimenter for the
experimenter
- /Experiment/Edit on start/run_title for the
run_title
- /Experiment/Edit on start/sample for the
sample
- /Experiment/Edit on start/orientation for the
orient
- /Experiment/Name for the
area
andapparatus
, - /CurrentMode/Name for the
insert
(hijacking a field used by MUSR but not bnmr or bnqr) - /Runinfo/Start time binary for the
timeBegin
- /Runinfo/Stop time binary for the
timeEnd
- /MUD Logging/Run description/Settings/Manual field value if not reading from CAMP
- /MUD Logging/Run description/Settings/Manual temperature value if not reading from CAMP
See edit-on-start parameters for more details on those variables.