/Lazy ODB tree: Difference between revisions

From MidasWiki
Jump to navigation Jump to search
mNo edit summary
(The midas documentation has moved to https://midas.triumf.ca/docs/)
Tag: Replaced
 
Line 1: Line 1:
{{Pagelinks}}
The midas documentation has moved. See [https://midas.triumf.ca/docs/develop/reference/reference/odb_keys/#lazy /Lazy] on the ODB keys page and [https://midas.triumf.ca/docs/develop/features/storing_data/data_logging/archiving/ Data logging: archiving (lazylogger)] for the new documentation about the /Lazy ODB keys.
 
= Links =
<div style="column-count:4;-moz-column-count:4;-webkit-column-count:4">
* [[Data Logger]]
* [[Lazylogger]]
</div>
 
 
== Creating the /Lazy tree ==
This tree is created with default parameters on the first activation of the <span style="color:darkcyan;">''lazylogger''</span>  utility . This task connects to a defined channel (e.g. Tape, Disk) when started. Multiple instances of the program can run concurrently, connecting to a different channel (see [[lazylogger]]).
 
The following example shows a  <span style="color:purple; font-style:italic">/Lazy</span> tree with a <channel-name> of "Disk". A sub-tree <span style="color:purple; font-style:italic">/Lazy/Disk</span> has been created, containing sub-trees "Settings" and "Statistics". A third sub-tree "List" will be created after the first successful operation of the channel. The contents of these sub-trees is discussed below (see
[[#Explanation of the Keys in the ODB /Lazy/<channel_name>/Settings subtree|Settings]]
[[#The ODB /Lazy/<channel_name>/Statistics subtree|Statistics]] and
[[#The ODB /Lazy/<channel_name>/List subtree|List]] subtrees.)
 
[local:pol:S]/>ls /lazy/disk
Settings
Statistics
List
 
== The ODB /Lazy/<channel_name>/Settings subtree ==
=== Example ===
The following example shows a typical <span style="color:purple; font-style:italic">/Lazy</span>  ODB tree obtained using the [[odbedit]] "ls" command. See keys in the ODB /Lazy tree for more information on each key.
 
  Settings                            DIR
        Maintain free space(%)          INT    1    4    3m  0  RWD  0
        Stay behind                    INT    1    4    3m  0  RWD  -3
        Alarm Class                    STRING  1    32    3m  0  RWD 
        Running condition              STRING  1    128  3m  0  RWD  ALWAYS
        Data dir                        STRING  1    256  3m  0  RWD  /home/midas/online
        Data format                    STRING  1    8    3m  0  RWD  MIDAS
        Filename format                STRING  1    128  3m  0  RWD  run%05d.mid
        Backup type                    STRING  1    8    3m  0  RWD  Tape
        Execute after rewind            STRING  1    64    3m  0  RWD 
        Path                            STRING  1    128  3m  0  RWD 
        Capacity (Bytes)                FLOAT  1    4    3m  0  RWD  5e+09
        List label                      STRING  1    128  3m  0  RWD 
        Execute before writing file    STRING  1    64    11h  0  RWD  lazy_prewrite.csh
        Execute after writing file      STRING  1    64    11h  0  RWD  rundb_addrun.pl
        Modulo.Position                STRING  1    8    11h  0  RWD  2.1
        Tape Data Append                BOOL    1    4    11h  0  RWD  y
 
 
=== Explanation of the Keys in the ODB /Lazy/<channel_name>/Settings subtree  ===
 
An example of the subtree is shown above.
 
====  <span  style="color: purple;">''Maintain free space(%)''</span>  ====
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' INT
* '''Default:'''  0
</div>
 
 
As the data Logger (<span style="color:darkcyan;">''mlogger''</span>) runs independently from the <span style="color:darkcyan;">''Lazylogger''</span>, the data logging device (e.g. disk) contains all the recorded data files. Under this condition, Lazylogger can be instructed to "purge" the disk after successful backup of the data onto a "slow device" (e.g. tape).
 
This parameter <span style="color:purple; font-style:italic">Maintain free space(%)</span> in the ODB <span style="color:purple; font-style:italic">/Lazy/<channel_name>/Settings</span> subtree controls the percentage of disk space required to be maintained as free (for "none" set to 0).
 
* The conditions that must ALL be satisfied before removing a data file are defined as:
#The data file corresponding to the given run number following the format declared under [[#Filename format#Filename format]] IS PRESENT on the Settings/Data Dir path.
#The given run number MUST appear anywhere under the [[#The ODB /Lazy/<channel_name>/List subtree|List]] directory of ALL the Lazy channels having the same
[[#Filename format#Filename format]] as this channel.
#The given run number MUST appear anywhere under the [[#The ODB /Lazy/<channel_name>/List subtree|List]] directory of that channel
 
 
====  <span  style="color: purple;">''Stay behind''</span>  ====
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' INT
* '''Default:'''  0
</div>
 
 
This parameter  in the ODB <span style="color:purple; font-style:italic">/Lazy/<channel_name>/Settings</span> subtree defines how many consecutive data files should be kept between the current run and the last lazylogger run.
 
; Example with "Stay behind = -3"
: Current acquisition run number 253 -> run00253.mid is being logged by mlogger.
: Files available on the disk corresponding to runs #248, #249, #250, #251, #252.
: Lazylogger will start backing up run #250 as soon as the new run 254 starts.
: "Stay behind = -3" corresponds to 3 files untouched on the disk (#251, #252, #253).
 
; If the '''sign''' (+/-) of the "Stay behind" parameter is
: negative: the lazylogger will always scan the entire disk at the "Data Dir" path, from the oldest to the most recent file present for backup.
: positive: lazylogger will backup starting from -x behind the current acquisition run number. Run order will be ignored.
 
 
 
====  <span  style="color: purple;">''Alarm Class''</span>  ====
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' STRING
* '''Default:'''  ""
</div>
 
 
This parameter  in the ODB <span style="color:purple; font-style:italic">/Lazy/<channel_name>/Settings</span> subtree defines
specify the [[Alarm class]] to be used in case of an alarm being triggered.
 
 
 
====  <span  style="color: purple;">''Running Condition''</span>  ====
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' STRING
* '''Default:'''  "ALWAYS"
</div>
 
 
This parameter  in the ODB <span style="color:purple; font-style:italic">/Lazy/<channel_name>/Settings</span> subtree
specifies the type of condition for which the lazylogger should be actived. By default the lazylogger is ALWAYS running.
 
;The choices are
: ALWAYS, NEVER, WHILE_NO_ACQ_RUNNING,
: or a condition based on a single field of the ODB key<=>value
 
'''Example''' : In the case of high data rate acquisition it could be necessary to activate the lazylogger only
 
* when the run is either paused or stopped (i.e. WHILE_NO_ACQ_RUNNING)
      [[odbedit]] > set "Running condition" WHILE_ACQ_NOT_RUNNING
* or when some external condition is satisfied such as "low beam intensity".
      odbedit> set "Running condition" "/alias/max_rate < 200"
 
 
====  <span  style="color: purple;">''Data Dir''</span>  ====
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' STRING
* '''Default:'''  ""
</div>
 
 
This parameter  in the ODB <span style="color:purple; font-style:italic">/Lazy/<channel_name>/Settings</span> subtree
specifies the data directory path of the data logging device (e.g. disk). If this key is present, the contents is taken as the path, otherwise the current directory is used (i.e. the path where the lazylogger client was started).
 
 
====  <span  style="color: purple;">''Data format''</span>  ====
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' STRING
* '''Default:'''  "MIDAS"
</div>
 
 
This parameter  in the ODB <span style="color:purple; font-style:italic">/Lazy/<channel_name>/Settings</span> subtree
specifies the data format of the data files. See [[Keys in the ODB /Logger/Channels subtree#/Logger/Channels/0/Settings/Format|Format]].
 
 
====  <span  style="color: purple;">''Filename format''</span>  ====
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' STRING
* '''Default:'''  "run%05d.mid"
</div>
 
 
This parameter  in the ODB <span style="color:purple; font-style:italic">/Lazy/<channel_name>/Settings</span> subtree
specifies the filename format of the data files on the data logging device (i.e. disk). This should be the same  filename format as used by the data logger to save the files
(see [[Keys in the ODB /Logger/Channels subtree#/Logger/Channels/0/Settings/Filename | Filename]]).
 
 
 
====  <span  style="color: purple;">''Backup type''</span>  ====
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' STRING
* '''Default:'''  "Tape"
</div>
 
 
This parameter  in the ODB <span style="color:purple; font-style:italic">/Lazy/<channel_name>/Settings</span> subtree  specifies the type of the "backup device" to be used for the lazylog backup.
It can be "Disk","Tape", "Script" or "Ftp".
 
==== <span  style="color: purple;">''Execute after rewind''</span>  ====
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' STRING
* '''Default:'''  ""
</div>
 
 
This parameter  in the ODB <span style="color:purple; font-style:italic">/Lazy/<channel_name>/Settings</span> subtree  specifies
an optional script to run after completion of a lazylogger backup set (see below [[#Capacity (Bytes)|Capacity]]).
 
 
====  <span  style="color: purple;">''Path''</span>  ====
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' STRING
* '''Default:'''  ""
</div>
 
 
This parameter  in the ODB <span style="color:purple; font-style:italic">/Lazy/<channel_name>/Settings</span> subtree  specifies
the [[#Backup type|backup device]] path.
 
There are three possible types of Path:
 
* For Tape : e.g. /dev/nst0- (UNIX like).
* For Disk : e.g. /data1/myexpt
* For Script: the path to the script file to run.  An example file for copying to dcache is here [[lazy_dcache.perl]].
* For Ftp :  host,port,user,password,directory
 
====  <span  style="color: purple;">''Capacity (Bytes)''</span>  ====
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' FLOAT
* '''Default:'''  5e9
</div>
 
 
This parameter only matters for TAPE backup type.  This parameter  in the ODB <span style="color:purple; font-style:italic">/Lazy/<channel_name>/Settings</span> subtree  specifies
the maximum [[#Backup type|backup device]]  capacity in bytes. When this capacity is reached,the lazylogger will close the backup device and clear the [[#List label|List label]] field to prevent further backup. It will also rewind the stream device if possible.
 
====  <span  style="color: purple;">''List label''</span>  ====
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' STRING
* '''Default:'''  ""
</div>
 
 
This parameter  in the ODB <span style="color:purple; font-style:italic">/Lazy/<channel_name>/Settings</span> subtree  specifies
a label for a set of backed up files to the [[#Backup type|backup device]]. This label is used internally by the lazylogger for creating a new array composed of the backed up runs under the [[#The ODB /Lazy/<channel_name>/List subtree|List]] subdirectory, until the [[#Capacity (Bytes)|Capacity]] value has been reached. Once the backup set is complete, lazylogger will clear the "List label" field and therefore prevent any further backup until a non-empty "List label" is re-entered. However, the list label will remain under the
[[#The ODB /Lazy/<channel_name>/List subtree|List]]key to display all run being backed up until the corresponding files have been removed from the disk.
 
====  <span  style="color: purple;">''Execute before writing file''</span>  ====
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' STRING
* '''Default:'''  ""
</div>
 
 
This parameter  in the ODB <span style="color:purple; font-style:italic">/Lazy/<channel_name>/Settings</span> subtree  permits a script to be run before the beginning of
the lazy job. The arguments passed to the script are: input file name , output file name, current block number.
 
====  <span  style="color: purple;">''Execute after writing file''</span>  ====
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' STRING
* '''Default:'''  ""
</div>
 
 
This parameter  in the ODB <span style="color:purple; font-style:italic">/Lazy/<channel_name>/Settings</span> subtree  permits
a script to be run after the completion of the lazy job. The arguments passed to the scripts are: list label, current job number, source path, file name, file size in MB, current block number.
 
 
====  <span  style="color: purple;">''Modulo.Position''</span>  ====
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' STRING
* '''Default:'''  ""
</div>
 
 
This field  in the ODB <span style="color:purple; font-style:italic">/Lazy/<channel_name>/Settings</span> subtree
is for multiple instances of the lazylogger where each instance works on a sub-set of run number. By specifying the Modulo.Position you're telling the current lazy instance how many instances are simultaneously running (3.) and the position of which this instance is assigned to (.1) As an example for 3 lazyloggers running simultaneously the field assignment should be :
 
  Channel    Field    Run#
  Lazy_1    3.0      21, 24, 27, ...
  Lazy_2    3.1      22, 25, 28, ...
  Lazy_3    3.2      23, 26, 29, ...
 
In the [[#Example of the ODB /Lazy/<channel_name>/Settings subtree|example above]], the Modulo.Position is set to 2.1 , indicating this is instance 1, and two lazyloggers are running.
 
 
====  <span  style="color: purple;">''Tape Data Append''</span>  ====
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' BOOL
* '''Default:'''  "y"
</div>
 
 
If this key in the ODB <span style="color:purple; font-style:italic">/Lazy/<channel_name>/Settings</span> subtree
is set to "y", the spooling of the Tape device to the End_of_Device (EOD) before starting the lazy job is enabled. This command is valid only for [[#Backup type|Backup type]] Tape. If this flag is not enabled the lazy job starts at the current tape position.
 
 
== The ODB /Lazy/<channel_name>/Statistics subtree ==
 
The  Statistics subtree contains general information about the status of the current lazylogger channel.
 
An example of the Statistics subtree for a lazylogger channel is shown below:
 
=== Example ===
[local:pol:S]/>ls /lazy/disk/Statistics/
Backup file                    000321.mid
File size (Bytes)              0
KBytes copied                  0
Total Bytes copied              0
Copy progress (%)              0
Copy Rate (Bytes per s)        0
Backup status (%)              0
Number of Files                173
Current Lazy run                321
 
== The ODB /Lazy/<channel_name>/List subtree ==
 
 
The List subtree will contain arrays of run number associated with the array name backup-set label (see [[#List label|List label]]) . Any run number appearing in any of the arrays is considered to have been backed up.
 
=== Example ===
An example of the List subtree for a lazylogger channel is shown below. In this case, the list label was set to "archive".
 
[local:pol:S]/>ls /lazy/disk/List
archive
                                230
                                231
                                232
                                233
                                234
                                235
                                236
                                237
 
[[Category:ODB Tree]]

Latest revision as of 22:00, 30 October 2026

The midas documentation has moved. See /Lazy on the ODB keys page and Data logging: archiving (lazylogger) for the new documentation about the /Lazy ODB keys.

Retrieved from "https://daq00.triumf.ca/MidasWiki/index.php?title=/Lazy_ODB_tree&oldid=3655"