BNMR: Data Logging
Links
Data Logging
Data Logging in the bnmr and bnqr experiments is performed by
- the custom data archiver client BNMR: mdarc (for both run types),
- the standard MIDAS logger mlogger (Type 1 only)
with input from BNMR: mheader (Type 1 only) and data read from the CAMP and EPICS slow control systems (see BNMR: Camp Logging andBNMR: Epics Logging).
The data are saved in MUD format which can be read the the analyzer program physica. Data files are archived at the end of run to the MUSR archive area.
The bnmr and bnqr experiments also use the MUSR BNMR: Run numbering scheme. The data files are named according to the run number (see #Naming of saved files).
Most of the parameters controlling the data logging are found in the mdarc ODB subtree at /Equipment/FIFO_acq/mdarc. Note that this area is also accessed by other clients (i.e. rf_config and mheader). The contents of the ODB is also saved at end-of-run by mlogger.
The data is saved differently for Type 1 (I-MUSR) and Type 2 (TD-MUSR) as explained below.
Type 2 (TD-MUSR) runs
For Type 2 runs, the client BNMR: mdarc is required to save the data as it is saved directly into MUD format. Client [[BNMR: mheader] is inactive. Mdarc receives histograms and scaler data from the frontend in the form of MIDAS banks. These MIDAS banks are not saved (c.f. Type 1), but the data they contain is converted to MUD format, and saved in .msr files. If BNMR: Camp Logging and/or BNMR: Epics Logging is enabled, data from CAMP and EPICS logged variables are read directly from CAMP and EPICS and also saved in the *.msr file in standard MUD TD-musr format.
The frequency of saving the data is controlled by the ODB key save_interval(sec). A new version of the data file is saved each time. At the end-of-run, the versions are purged and the final data file is archived.
Type 1 (I-MUSR) runs
In addition to BNMR: mdarc, for Type 1 runs the standard MIDAS logger mlogger and client BNMR: mheader are also needed. Scaler data from the frontend are sent out in the form of MIDAS banks. These banks are named differently and have different content to those sent for a Type 2 run. The client mheader is also required for a Type 1 run. At begin and end-of-run, it sends an extra bank into the data stream, containing the header information required to write the MUD file. If CAMP and/or EPICS logging is enabled, CAMP and EPICS header banks are also sent containing the names of the EPICS and CAMP variables that are to be logged. These banks must be received at begin-of-run, or the data cannot be converted to MUD format during the run. However, the data can still be converted from the .mid file by the program midbnmr after the end-of-run.
The standard Midas logger mlogger saves all the data banks in standard MIDAS format, i.e. into a *.mid file. The client mdarc also receives the data banks, and converts the data to MUD format (using midbnmr routines), saving a .msr file. The data in this file is saved in standard MUSR I-musr format. Thus two data files are saved for each Type 1 run, a MIDAS-format (.mid) file and a MUD-format (.msr) file.
Naming of saved files
The data are written into a saved file in the directory given by the ODB parameter saved_data_directory. Type 1 (I-Musr) runs result in two saved data files, a .mid and a .msr file. Type 2 (TD-Musr) runs result in only one saved file, the .msr file (see above). Both run types also produce a .odb file, containing a copy of the MIDAS online data base (ODB). In the event of corruption of the ODB, it can be regenerated from this file.
The name of the files is dependent on the run number. For Type 2 runs, several versions of the .msr (or MUD) data file are retained with a version number appended (in case of any problems with the most recent saved data). At end-of-run, normally all earlier versions are purged and the final data fileis renamed without the appended version number, then copied to an archive according to ODB parameter archived_data_directory. The version numbers of the intermediate files are hidden from analysis programs by the use of a softlink, so that these programs always get the latest data. The final data filename is used as the soft link name to refer to the latest data file. So during the run this filename is actually a soft link to the latest version.
For example, for BNMR run 40362 (running Type 2 (TD-MUSR) ), while the run is in progress, the saved data directory would contain files such as
lrwxrwxrwx 1 bnmr bnmr 48 Feb 26 12:09 040362.msr -> /isdaq/data1/bnmr/dlog/current/040362.msr_v27 -rw-r--r-- 1 bnmr bnmr 4403 Feb 26 12:09 040362.msr_v27 -rw-r--r-- 1 bnmr bnmr 4403 Feb 26 12:09 040362.msr_v26 -rw-r--r-- 1 bnmr bnmr 4403 Feb 26 12:09 040362.msr_v25 -rw-r--r-- 1 bnmr bnmr 4403 Feb 26 12:08 040362.msr_v24
And after the run had ended:
-rw-r--r-- 1 bnmr bnmr 4399 Feb 26 12:10 040362.msr -rw-r--r-- 1 bnmr bnmr 19990 Feb 26 12:10 040362.odb
The number of intermediate versions of the run files kept during a run is controlled by the ODB parameter
num_versions_before_purge. The action of renaming the files to a final saved version and archiving can be suppressed by the ODB parameter endrun_purge_and_archive. This may be useful if the data in the latest saved file is bad for some reason. After the run is ended, the user can delete the bad data file by hand.
Running the client |cleanup will then purge, rename and archive the final data file.
In the case of a Type 1 run three files are saved, e.g.
$ ls /isdaq/data1/bnqr/dlog/current/045238.* -lt -rw-r--r-- 1 bnqr users 123088 Aug 18 11:00 /isdaq/data1/bnqr/dlog/current/045238.odb -rw-r--r-- 1 bnqr users 2156308 Aug 18 11:00 /isdaq/data1/bnqr/dlog/current/045238.mid -rw-r--r-- 1 bnqr users 93341 Aug 18 11:00 /isdaq/data1/bnqr/dlog/current/045238.msr
Archiving the data
In both cases (Types 1(I-MUSR) and 2(TD-MUSR)), the final *.msr data file for real runs (see BNMR: Run numbering) is automatically copied by BNMR:mdarc to the archive directory, given by ODB parameter archived_data_directory. Currently, this directory is
<file_path>. SSH keys have been set up so that the copy can be performed without need of a password.
Should a file not be archived for some reason at end-of-run, the client |cleanup will archive any files missing from the MUSR data archive.