/Lazy ODB tree: Difference between revisions
No edit summary |
No edit summary |
||
Line 2: | Line 2: | ||
* [[Data Logger]] | * [[Data Logger]] | ||
* [[Lazylogger]] | * [[Lazylogger]] | ||
</div> | </div> | ||
== Creating the /Lazy tree == | |||
This tree is created with default parameters on the first activation of the <span style="color:darkcyan;">''lazylogger''</span> utility . This task connects to a defined channel (e.g. Tape, Disk) when started. Multiple instances of the program can run concurrently, connecting to a different channel (see lazylogger | This tree is created with default parameters on the first activation of the <span style="color:darkcyan;">''lazylogger''</span> utility . This task connects to a defined channel (e.g. Tape, Disk) when started. Multiple instances of the program can run concurrently, connecting to a different channel (see [[lazylogger]]). | ||
The following example shows a <span style="color:purple; font-style:italic">/Lazy</span> tree with a <channel-name> of "Disk". A sub-tree <span style="color:purple; font-style:italic">/Lazy/Disk</span> has been created, containing sub-trees "Settings" and "Statistics". A third sub-tree "List" will be created after the first successful operation of the channel. The contents of these sub-trees is discussed below (see Settings | The following example shows a <span style="color:purple; font-style:italic">/Lazy</span> tree with a <channel-name> of "Disk". A sub-tree <span style="color:purple; font-style:italic">/Lazy/Disk</span> has been created, containing sub-trees "Settings" and "Statistics". A third sub-tree "List" will be created after the first successful operation of the channel. The contents of these sub-trees is discussed below (see | ||
[[#Explanation of the Keys in the ODB /Lazy/<channel_name>/Settings subtree|Settings]] | |||
[[#The ODB /Lazy/<channel_name>/Statistics subtree|Statistics]] and | |||
[[#The ODB /Lazy/<channel_name>/List subtree|List]] subtrees.) | |||
[local:pol:S]/>ls /lazy/disk | [local:pol:S]/>ls /lazy/disk | ||
Line 16: | Line 18: | ||
List | List | ||
== | == The ODB /Lazy/<channel_name>/Settings subtree == | ||
=== Example === | |||
The following example shows a typical <span style="color:purple; font-style:italic">/Lazy</span> ODB tree obtained using the [[odbedit]] "ls" command. See keys in the ODB /Lazy tree for more information on each key. | The following example shows a typical <span style="color:purple; font-style:italic">/Lazy</span> ODB tree obtained using the [[odbedit]] "ls" command. See keys in the ODB /Lazy tree for more information on each key. | ||
Settings | Settings DIR | ||
Maintain free space(%) | Maintain free space(%) INT 1 4 3m 0 RWD 0 | ||
Stay behind | Stay behind INT 1 4 3m 0 RWD -3 | ||
Alarm Class | Alarm Class STRING 1 32 3m 0 RWD | ||
Running condition | Running condition STRING 1 128 3m 0 RWD ALWAYS | ||
Data dir | Data dir STRING 1 256 3m 0 RWD /home/midas/online | ||
Data format | Data format STRING 1 8 3m 0 RWD MIDAS | ||
Filename format | Filename format STRING 1 128 3m 0 RWD run%05d.mid | ||
Backup type | Backup type STRING 1 8 3m 0 RWD Tape | ||
Execute after rewind | Execute after rewind STRING 1 64 3m 0 RWD | ||
Path | Path STRING 1 128 3m 0 RWD | ||
Capacity (Bytes) | Capacity (Bytes) FLOAT 1 4 3m 0 RWD 5e+09 | ||
List label | List label STRING 1 128 3m 0 RWD | ||
Execute before writing file STRING 1 64 11h 0 RWD lazy_prewrite.csh | |||
Execute after writing file STRING 1 64 11h 0 RWD rundb_addrun.pl | |||
Modulo.Position STRING 1 8 11h 0 RWD 2.1 | |||
Tape Data Append BOOL 1 4 11h 0 RWD y | |||
Line 42: | Line 45: | ||
An example of the subtree is shown above. | An example of the subtree is shown above. | ||
== <span style="color: purple;">''Maintain free space(%)''</span> == | ==== <span style="color: purple;">''Maintain free space(%)''</span> ==== | ||
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | <div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | ||
* '''Type:''' INT | * '''Type:''' INT | ||
Line 54: | Line 57: | ||
* The conditions that must ALL be satisfied before removing a data file are defined as: | * The conditions that must ALL be satisfied before removing a data file are defined as: | ||
#The data file corresponding to the given run number following the format declared under | #The data file corresponding to the given run number following the format declared under [[#Filename format#Filename format]] IS PRESENT on the Settings/Data Dir path. | ||
#The given run number MUST appear anywhere under the < | #The given run number MUST appear anywhere under the [[#The ODB /Lazy/<channel_name>/List subtree|List]] directory of ALL the Lazy channels having the same | ||
#The given run number MUST appear anywhere under the < | [[#Filename format#Filename format]] as this channel. | ||
#The given run number MUST appear anywhere under the [[#The ODB /Lazy/<channel_name>/List subtree|List]] directory of that channel | |||
== <span style="color: purple;">''Stay behind''</span> == | ==== <span style="color: purple;">''Stay behind''</span> ==== | ||
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | <div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | ||
* '''Type:''' INT | * '''Type:''' INT | ||
Line 80: | Line 84: | ||
== <span style="color: purple;">''Alarm Class''</span> == | ==== <span style="color: purple;">''Alarm Class''</span> ==== | ||
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | <div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | ||
* '''Type:''' STRING | * '''Type:''' STRING | ||
Line 92: | Line 96: | ||
== <span style="color: purple;">''Running Condition''</span> == | ==== <span style="color: purple;">''Running Condition''</span> ==== | ||
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | <div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | ||
* '''Type:''' STRING | * '''Type:''' STRING | ||
Line 114: | Line 118: | ||
== <span style="color: purple;">''Data Dir''</span> == | ==== <span style="color: purple;">''Data Dir''</span> ==== | ||
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | <div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | ||
* '''Type:''' STRING | * '''Type:''' STRING | ||
Line 125: | Line 129: | ||
== <span style="color: purple;">''Data format''</span> == | ==== <span style="color: purple;">''Data format''</span> ==== | ||
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | <div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | ||
* '''Type:''' STRING | * '''Type:''' STRING | ||
Line 136: | Line 140: | ||
== <span style="color: purple;">''Filename format''</span> == | ==== <span style="color: purple;">''Filename format''</span> ==== | ||
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | <div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | ||
* '''Type:''' STRING | * '''Type:''' STRING | ||
Line 149: | Line 153: | ||
== <span style="color: purple;">''Backup | ==== <span style="color: purple;">''Backup type''</span> ==== | ||
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | <div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | ||
* '''Type:''' STRING | * '''Type:''' STRING | ||
Line 160: | Line 164: | ||
== | ==== <span style="color: purple;">''Execute after rewind''</span> ==== | ||
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | <div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | ||
* '''Type:''' STRING | * '''Type:''' STRING | ||
Line 168: | Line 172: | ||
This parameter in the ODB <span style="color:purple; font-style:italic">/Lazy/<channel_name>/Settings</span> subtree specifies | This parameter in the ODB <span style="color:purple; font-style:italic">/Lazy/<channel_name>/Settings</span> subtree specifies | ||
an optional script to run after completion of a lazylogger backup set (see below [[#Capacity (Bytes)]]). | an optional script to run after completion of a lazylogger backup set (see below [[#Capacity (Bytes)|Capacity]]). | ||
== <span style="color: purple;">''Path''</span> == | ==== <span style="color: purple;">''Path''</span> ==== | ||
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | <div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | ||
* '''Type:''' STRING | * '''Type:''' STRING | ||
Line 189: | Line 193: | ||
== <span style="color: purple;">''Capacity (Bytes)''</span> == | ==== <span style="color: purple;">''Capacity (Bytes)''</span> ==== | ||
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | <div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | ||
* '''Type:''' FLOAT | * '''Type:''' FLOAT | ||
Line 199: | Line 203: | ||
the maximum [[#Backup type|backup device]] capacity in bytes. When this capacity is reached,the lazylogger will close the backup device and clear the [[#List label|List label]] field to prevent further backup. It will also rewind the stream device if possible. | the maximum [[#Backup type|backup device]] capacity in bytes. When this capacity is reached,the lazylogger will close the backup device and clear the [[#List label|List label]] field to prevent further backup. It will also rewind the stream device if possible. | ||
== <span style="color: purple;">''List label''</span> == | ==== <span style="color: purple;">''List label''</span> ==== | ||
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | <div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | ||
* '''Type:''' STRING | * '''Type:''' STRING | ||
Line 207: | Line 211: | ||
This parameter in the ODB <span style="color:purple; font-style:italic">/Lazy/<channel_name>/Settings</span> subtree specifies | This parameter in the ODB <span style="color:purple; font-style:italic">/Lazy/<channel_name>/Settings</span> subtree specifies | ||
a label for a set of backed up files to the [[#Backup type|backup device]]. This label is used internally by the lazylogger for creating a new array composed of the backed up runs under the [[/ | a label for a set of backed up files to the [[#Backup type|backup device]]. This label is used internally by the lazylogger for creating a new array composed of the backed up runs under the [[#The ODB /Lazy/<channel_name>/List subtree|List]] subdirectory, until the [[#Capacity (Bytes)|Capacity]] value has been reached. Once the backup set is complete, lazylogger will clear the "List label" field and therefore prevent any further backup until a non-empty "List label" is re-entered. However, the list label will remain under the | ||
[[#The ODB /Lazy/<channel_name>/List subtree|List]]key to display all run being backed up until the corresponding files have been removed from the disk. | |||
== <span style="color: purple;">''Execute before writing file''</span> == | ==== <span style="color: purple;">''Execute before writing file''</span> ==== | ||
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | <div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | ||
* '''Type:''' STRING | * '''Type:''' STRING | ||
Line 219: | Line 224: | ||
the lazy job. The arguments passed to the script are: input file name , output file name, current block number. | the lazy job. The arguments passed to the script are: input file name , output file name, current block number. | ||
== <span style="color: purple;">''Execute after writing file''</span> == | ==== <span style="color: purple;">''Execute after writing file''</span> ==== | ||
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | <div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | ||
* '''Type:''' STRING | * '''Type:''' STRING | ||
Line 230: | Line 235: | ||
== <span style="color: purple;">''Modulo.Position''</span> == | ==== <span style="color: purple;">''Modulo.Position''</span> ==== | ||
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | <div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | ||
* '''Type:''' STRING | * '''Type:''' STRING | ||
Line 248: | Line 253: | ||
== <span style="color: purple;">''Tape Data Append''</span> == | ==== <span style="color: purple;">''Tape Data Append''</span> ==== | ||
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | <div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> | ||
* '''Type:''' BOOL | * '''Type:''' BOOL | ||
Line 256: | Line 261: | ||
If this key in the ODB <span style="color:purple; font-style:italic">/Lazy/<channel_name>/Settings</span> subtree | If this key in the ODB <span style="color:purple; font-style:italic">/Lazy/<channel_name>/Settings</span> subtree | ||
is set to "y", the spooling of the Tape device to the End_of_Device (EOD) before starting the lazy job is enabled. This command is valid only for [[Backup | is set to "y", the spooling of the Tape device to the End_of_Device (EOD) before starting the lazy job is enabled. This command is valid only for [[#Backup type|Backup type]] Tape. If this flag is not enabled the lazy job starts at the current tape position. | ||
== The ODB /Lazy/<channel_name>/Statistics subtree == | |||
The | The Statistics subtree contains general information about the status of the current lazylogger channel. | ||
An example of the Statistics subtree for a lazylogger channel is shown below: | An example of the Statistics subtree for a lazylogger channel is shown below: | ||
[local:pol:S]/>ls /lazy/disk/Statistics/ | === Example === | ||
Backup file 000321.mid | [local:pol:S]/>ls /lazy/disk/Statistics/ | ||
File size (Bytes) 0 | Backup file 000321.mid | ||
KBytes copied 0 | File size (Bytes) 0 | ||
Total Bytes copied 0 | KBytes copied 0 | ||
Copy progress (%) 0 | Total Bytes copied 0 | ||
Copy Rate (Bytes per s) 0 | Copy progress (%) 0 | ||
Backup status (%) 0 | Copy Rate (Bytes per s) 0 | ||
Number of Files 173 | Backup status (%) 0 | ||
Current Lazy run 321 | Number of Files 173 | ||
Current Lazy run 321 | |||
== The ODB /Lazy/<channel_name>/List subtree == | |||
The List | The List subtree will contain arrays of run number associated with the array name backup-set label (see [[#List label|List label]]) . Any run number appearing in any of the arrays is considered to have been backed up. | ||
=== Example === | |||
An example of the List subtree for a lazylogger channel is shown below. In this case, the list label was set to "archive". | An example of the List subtree for a lazylogger channel is shown below. In this case, the list label was set to "archive". | ||
[local:pol:S]/>ls /lazy/disk/List | [local:pol:S]/>ls /lazy/disk/List | ||
archive | archive | ||
230 | 230 | ||
231 | 231 | ||
Line 295: | Line 301: | ||
237 | 237 | ||
[[Category:ODB Tree]] | |||
[ | |||
[ | |||
Revision as of 14:20, 5 June 2014
Creating the /Lazy tree
This tree is created with default parameters on the first activation of the lazylogger utility . This task connects to a defined channel (e.g. Tape, Disk) when started. Multiple instances of the program can run concurrently, connecting to a different channel (see lazylogger).
The following example shows a /Lazy tree with a <channel-name> of "Disk". A sub-tree /Lazy/Disk has been created, containing sub-trees "Settings" and "Statistics". A third sub-tree "List" will be created after the first successful operation of the channel. The contents of these sub-trees is discussed below (see Settings Statistics and List subtrees.)
[local:pol:S]/>ls /lazy/disk Settings Statistics List
The ODB /Lazy/<channel_name>/Settings subtree
Example
The following example shows a typical /Lazy ODB tree obtained using the odbedit "ls" command. See keys in the ODB /Lazy tree for more information on each key.
Settings DIR Maintain free space(%) INT 1 4 3m 0 RWD 0 Stay behind INT 1 4 3m 0 RWD -3 Alarm Class STRING 1 32 3m 0 RWD Running condition STRING 1 128 3m 0 RWD ALWAYS Data dir STRING 1 256 3m 0 RWD /home/midas/online Data format STRING 1 8 3m 0 RWD MIDAS Filename format STRING 1 128 3m 0 RWD run%05d.mid Backup type STRING 1 8 3m 0 RWD Tape Execute after rewind STRING 1 64 3m 0 RWD Path STRING 1 128 3m 0 RWD Capacity (Bytes) FLOAT 1 4 3m 0 RWD 5e+09 List label STRING 1 128 3m 0 RWD Execute before writing file STRING 1 64 11h 0 RWD lazy_prewrite.csh Execute after writing file STRING 1 64 11h 0 RWD rundb_addrun.pl Modulo.Position STRING 1 8 11h 0 RWD 2.1 Tape Data Append BOOL 1 4 11h 0 RWD y
Explanation of the Keys in the ODB /Lazy/<channel_name>/Settings subtree
An example of the subtree is shown above.
Maintain free space(%)
- Type: INT
- Default: 0
As the data Logger (mlogger) runs independently from the Lazylogger, the data logging device (e.g. disk) contains all the recorded data files. Under this condition, Lazylogger can be instructed to "purge" the disk after successful backup of the data onto a "slow device" (e.g. tape).
This parameter Maintain free space(%) in the ODB /Lazy/<channel_name>/Settings subtree controls the percentage of disk space required to be maintained as free (for "none" set to 0).
- The conditions that must ALL be satisfied before removing a data file are defined as:
- The data file corresponding to the given run number following the format declared under #Filename format#Filename format IS PRESENT on the Settings/Data Dir path.
- The given run number MUST appear anywhere under the List directory of ALL the Lazy channels having the same
#Filename format#Filename format as this channel.
- The given run number MUST appear anywhere under the List directory of that channel
Stay behind
- Type: INT
- Default: 0
This parameter in the ODB /Lazy/<channel_name>/Settings subtree defines how many consecutive data files should be kept between the current run and the last lazylogger run.
- Example with "Stay behind = -3"
- Current acquisition run number 253 -> run00253.mid is being logged by mlogger.
- Files available on the disk corresponding to runs #248, #249, #250, #251, #252.
- Lazylogger will start backing up run #250 as soon as the new run 254 starts.
- "Stay behind = -3" corresponds to 3 files untouched on the disk (#251, #252, #253).
- If the sign (+/-) of the "Stay behind" parameter is
- negative: the lazylogger will always scan the entire disk at the "Data Dir" path, from the oldest to the most recent file present for backup.
- positive: lazylogger will backup starting from -x behind the current acquisition run number. Run order will be ignored.
Alarm Class
- Type: STRING
- Default: ""
This parameter in the ODB /Lazy/<channel_name>/Settings subtree defines
specify the Alarm class to be used in case of an alarm being triggered.
Running Condition
- Type: STRING
- Default: "ALWAYS"
This parameter in the ODB /Lazy/<channel_name>/Settings subtree
specifies the type of condition for which the lazylogger should be actived. By default the lazylogger is ALWAYS running.
- The choices are
- ALWAYS, NEVER, WHILE_NO_ACQ_RUNNING,
- or a condition based on a single field of the ODB key<=>value
Example : In the case of high data rate acquisition it could be necessary to activate the lazylogger only
- when the run is either paused or stopped (i.e. WHILE_NO_ACQ_RUNNING)
odbedit > set "Running condition" WHILE_ACQ_NOT_RUNNING
- or when some external condition is satisfied such as "low beam intensity".
odbedit> set "Running condition" "/alias/max_rate < 200"
Data Dir
- Type: STRING
- Default: ""
This parameter in the ODB /Lazy/<channel_name>/Settings subtree
specifies the data directory path of the data logging device (e.g. disk). If this key is present, the contents is taken as the path, otherwise the current directory is used (i.e. the path where the lazylogger client was started).
Data format
- Type: STRING
- Default: "MIDAS"
This parameter in the ODB /Lazy/<channel_name>/Settings subtree
specifies the data format of the data files. See Format.
Filename format
- Type: STRING
- Default: "run%05d.mid"
This parameter in the ODB /Lazy/<channel_name>/Settings subtree
specifies the filename format of the data files on the data logging device (i.e. disk). This should be the same filename format as used by the data logger to save the files
(see Filename).
Backup type
- Type: STRING
- Default: "Tape"
This parameter in the ODB /Lazy/<channel_name>/Settings subtree specifies the type of the "backup device" to be used for the lazylog backup.
It can be "Disk","Tape" or "Ftp".
Execute after rewind
- Type: STRING
- Default: ""
This parameter in the ODB /Lazy/<channel_name>/Settings subtree specifies
an optional script to run after completion of a lazylogger backup set (see below Capacity).
Path
- Type: STRING
- Default: ""
This parameter in the ODB /Lazy/<channel_name>/Settings subtree specifies
the backup device path.
There are three possible types of Path:
- For Tape : e.g. /dev/nst0- (UNIX like).
- For Disk : e.g. /data1/myexpt
- For Ftp : host,port,user,password,directory
Capacity (Bytes)
- Type: FLOAT
- Default: 5e9
This parameter in the ODB /Lazy/<channel_name>/Settings subtree specifies
the maximum backup device capacity in bytes. When this capacity is reached,the lazylogger will close the backup device and clear the List label field to prevent further backup. It will also rewind the stream device if possible.
List label
- Type: STRING
- Default: ""
This parameter in the ODB /Lazy/<channel_name>/Settings subtree specifies
a label for a set of backed up files to the backup device. This label is used internally by the lazylogger for creating a new array composed of the backed up runs under the List subdirectory, until the Capacity value has been reached. Once the backup set is complete, lazylogger will clear the "List label" field and therefore prevent any further backup until a non-empty "List label" is re-entered. However, the list label will remain under the
Listkey to display all run being backed up until the corresponding files have been removed from the disk.
Execute before writing file
- Type: STRING
- Default: ""
This parameter in the ODB /Lazy/<channel_name>/Settings subtree permits a script to be run before the beginning of
the lazy job. The arguments passed to the script are: input file name , output file name, current block number.
Execute after writing file
- Type: STRING
- Default: ""
This parameter in the ODB /Lazy/<channel_name>/Settings subtree permits
a script to be run after the completion of the lazy job. The arguments passed to the scripts are: list label, current job number, source path, file name, file size in MB, current block number.
Modulo.Position
- Type: STRING
- Default: ""
This field in the ODB /Lazy/<channel_name>/Settings subtree
is for multiple instances of the lazylogger where each instance works on a sub-set of run number. By specifying the Modulo.Position you're telling the current lazy instance how many instances are simultaneously running (3.) and the position of which this instance is assigned to (.1) As an example for 3 lazyloggers running simultaneously the field assignment should be :
Channel Field Run# Lazy_1 3.0 21, 24, 27, ... Lazy_2 3.1 22, 25, 28, ... Lazy_3 3.2 23, 26, 29, ...
In the example above, the Modulo.Position is set to 2.1 , indicating this is instance 1, and two lazyloggers are running.
Tape Data Append
- Type: BOOL
- Default: "y"
If this key in the ODB /Lazy/<channel_name>/Settings subtree
is set to "y", the spooling of the Tape device to the End_of_Device (EOD) before starting the lazy job is enabled. This command is valid only for Backup type Tape. If this flag is not enabled the lazy job starts at the current tape position.
The ODB /Lazy/<channel_name>/Statistics subtree
The Statistics subtree contains general information about the status of the current lazylogger channel.
An example of the Statistics subtree for a lazylogger channel is shown below:
Example
[local:pol:S]/>ls /lazy/disk/Statistics/ Backup file 000321.mid File size (Bytes) 0 KBytes copied 0 Total Bytes copied 0 Copy progress (%) 0 Copy Rate (Bytes per s) 0 Backup status (%) 0 Number of Files 173 Current Lazy run 321
The ODB /Lazy/<channel_name>/List subtree
The List subtree will contain arrays of run number associated with the array name backup-set label (see List label) . Any run number appearing in any of the arrays is considered to have been backed up.
Example
An example of the List subtree for a lazylogger channel is shown below. In this case, the list label was set to "archive".
[local:pol:S]/>ls /lazy/disk/List archive 230 231 232 233 234 235 236 237