BNMR: Data Logging

• BNMR overview
• Getting started
• Troubleshooting
• All BNMR pages
• Midas

Links

• BNMR: mdarc
• BNMR: mheader
• BNMR: Camp Logging
• BNMR: Epics Logging
• Getting Started
• Nomenclature

Data Format

For both run types, the frontend sends the data in the form of MIDAS data banks. The data banks are processed and saved as *.msr files in MUD format. These files are then analyzed by the MUSR/BNMR analyzer program physica, and can be dumped/examined by the mud utility program mud_util.

For Type 1 (I-Musr) runs only, the incoming MIDAS data banks are also saved in a .mid file in MIDAS format. Both the saved *.mid file and the incoming data banks can be dumped using the MIDAS utility program mdump].

Data Logging

Data Logging in the bnmr and bnqr experiments is performed by

• the custom data archiver client mdarc (for both run types),
• the standard MIDAS logger mlogger (Type 1 only)

The bnmr and bnqr experiments use the MUSR-style Run Numbering scheme. The data files are named according to the run number (see #Naming of saved files).

Most of the parameters controlling the data logging are found in the mdarc ODB subtree at /Equipment/FIFO_acq/mdarc. Note that this area is also accessed by other clients (i.e. rf_config and mheader).

The data are saved differently for Type 1 (I-MUSR) and Type 2 (TD-MUSR) as explained below. For both run types, the contents of the ODB is also saved at the end of every run by mlogger in an ASCII file. In the event of corruption of the ODB, it can be regenerated from this file.

Real data files are also archived in the MUSR Data Archive at the end of run.

Type 1 (I-MUSR) runs

In addition to mdarc, the standard MIDAS logger mlogger and client mheader are also required to save Type 1 data.

Scaler data from the frontend are sent out in the form of MIDAS banks. These banks are named differently and have different content to those sent for a Type 2 run. They contain the scaler data for each PPG cycle. At begin and end-of-run, mheader sends an extra bank into the data stream, containing the header information required to write the MUD file. If CAMP and/or EPICS logging is enabled, CAMP and EPICS header banks are also sent by mheader. These contain the names of the EPICS and CAMP variables that are to be logged. These banks must be received at begin-of-run, or the data cannot be converted to MUD format during the run. However, the data can still be converted to MUD format by the program midbnmr after the end-of-run. During the run, mheader periodically sends banks containing the values of the CAMP and EPICS logged variables.

The standard Midas logger mlogger saves all the data banks in standard MIDAS format, i.e. into a *.mid file. The client mdarc also receives the data banks, and converts the data to MUD format (using midbnmr routines), saving an .msr file. The data in this file are saved in standard MUSR I-musr format. Thus two data files are saved for each Type 1 run, a MIDAS-format (.mid) file and a MUD-format (.msr) file.

Type 2 (TD-MUSR) runs

For Type 2 runs, the client frontend sends out data in the form of MIDAS banks (named differently from the Type 1 data banks). These banks contain the histograms accumulated and stored in the frontend. mdarc receives these banks, and saves the histogram data directly in MUD format. Client mheader is inactive. If CAMP logging and/or EPICS Logging is enabled, data from CAMP and EPICS logged variables are read directly from the CAMP and EPICS slow control system, and also saved in the *.msr file in standard MUD TD-musr format. In this case, no intermediate *.mid file is produced. The frequency of saving the data is controlled by the ODB key save_interval(sec). A new version of the data file is saved each time. At the end-of-run, the versions are purged and the final data file is archived.

Saved data files

The data are written by mdarc and the standard MIDAS logger mlogger to the same directory (currently /isdaq/data1/bn[mq]r/dlog/current where current is a soft-link to the current year). By using a soft-link, the path in the ODB does not need to be updated each year.

The same path ( /isdaq/data1/bn[mq]r/dlog/current) is entered both in the ODB parameter saved_data_directory used by mdarc and in the ODB parameter /logger/data dir used by the standard MIDAS logger mlogger.

The saved data files are named by their run number (see Run Numbering) and their #Data Format.

Type 1 (I-Musr) runs result in two saved data files, a .mid and an .msr file (see #Data Format). Type 2 (TD-Musr) runs result in only one saved data file, the .msr file (see above). In addition, both run types also produce an .odb file at the end-of-run, containing a copy of the MIDAS online data base (ODB). The .odb files also named by the current run number.

Type 1 data files

In the case of a Type 1 run, three files are saved, e.g.

$ ls /isdaq/data1/bnqr/dlog/current/045238.* -lt
-rw-r--r-- 1 bnqr users  123088 Aug 18 11:00 /isdaq/data1/bnqr/dlog/current/045238.odb
-rw-r--r-- 1 bnqr users 2156308 Aug 18 11:00 /isdaq/data1/bnqr/dlog/current/045238.mid
-rw-r--r-- 1 bnqr users   93341 Aug 18 11:00 /isdaq/data1/bnqr/dlog/current/045238.msr

Type 2 data files

For Type 2 runs, several versions of the .msr (or MUD) data file are retained with a version number appended (in case of any problems with the most recent saved data). At end-of-run, normally all earlier versions are purged and the final data fileis renamed without the appended version number, then copied to an archive according to ODB parameter archived_data_directory. The version numbers of the intermediate files are hidden from analysis programs by the use of a softlink, so that these programs always get the latest data. The final data file name is used as the soft-link name to refer to the latest data file. So during the run this filename is actually a soft-link to the latest version.

For example, for BNMR run 40362 (running Type 2 (TD-MUSR) ), while the run is in progress, the saved data directory would contain files such as

lrwxrwxrwx     1 bnmr     bnmr           48      Feb 26 12:09 040362.msr -> /isdaq/data1/bnmr/dlog/current/040362.msr_v27
-rw-r--r--    1 bnmr     bnmr         4403   Feb 26 12:09 040362.msr_v27
-rw-r--r--    1 bnmr     bnmr         4403   Feb 26 12:09 040362.msr_v26
-rw-r--r--    1 bnmr     bnmr         4403   Feb 26 12:09 040362.msr_v25
-rw-r--r--    1 bnmr     bnmr         4403   Feb 26 12:08 040362.msr_v24

And after the run had ended:

-rw-r--r--    1 bnmr     bnmr         4399  Feb 26 12:10 040362.msr
-rw-r--r--    1 bnmr     bnmr        19990 Feb 26 12:10 040362.odb

The number of intermediate versions of the run files kept during a run is controlled by the ODB parameter num_versions_before_purge. The action of renaming the files to a final saved version and archiving can be suppressed by the ODB parameter endrun_purge_and_archive. This may be useful if the data in the latest saved file is bad for some reason. After the run is ended, the user can delete the bad data file by hand. Running the client cleanup will then purge, rename and archive the final data file.

Archiving the data

For both run types), the final *.msr data file for real runs (see run numbering) is automatically copied by mdarc to the archive directory, given by ODB parameter archived_data_directory. Currently, this directory is musrarc@cmms:dlog. SSH keys have been set up so that the copy can be performed without need of a password.

Should a file not be archived for some reason at end-of-run, the client cleanup will archive any files missing from the MUSR data archive.