Lazylogger: Difference between revisions

From MidasWiki
Jump to navigation Jump to search
No edit summary
No edit summary
 
(26 intermediate revisions by 2 users not shown)
Line 1: Line 1:
<div style="column-count:4;-moz-column-count:4;-webkit-column-count:4">
{{Pagelinks}}
== Links ==
<div style="column-count:2;-moz-column-count:2;-webkit-column-count:2">
* [[Data Logger]]
* [[Data Logger]]
* [[/Lazy ODB tree]]
* [[/Lazy ODB tree]]
Line 7: Line 9:


== Purpose ==
== Purpose ==
The <span style="color:darkcyan;">''lazylogger''</span> utility decouples the data acquisition from the data logging mechanism. The need for such an application has been dictated by the slow response time of some of the media logging devices (i.e. tape devices). Delay due to tape mounting, re-tension and re-positioning implies that the data acquisition has to be held until operation completion. By using '''[[mlogger]]''' to log data to disk in a first stage and then using  <span style="color:darkcyan;">''lazylogger''</span> to copy or move the stored files to the "slow device", the data acquisition can be kept running without interruption.
The <span style="color:darkcyan;">''lazylogger''</span> utility decouples the data acquisition from the data logging mechanism. The need for such an application has been dictated by the slow response time of the old media logging devices, such as tape devices. Delay due to tape mounting, re-tension and re-positioning implies that the data acquisition has to be held until operation completion. By using '''[[mlogger]]''' to log data to disk in a first stage and then using  <span style="color:darkcyan;"> ''lazylogger''</span> to copy or move the stored files, the data acquisition can be kept running without interruption. These devices are no longer supported internally by Midas but can still be managed through scripts. Lazylogger is still much used for getting a copy of the original data to a remote data storage. While there are many ways to do this independently of the acquisition system, the lazylogger provides a simple way to manage this task. Lazylogger features are listed below. 


* Multiple  <span style="color:darkcyan;">''lazyloggers''</span> can be running simultaneously on the same computer, each one taking care of a particular channel.
* Multiple  <span style="color:darkcyan;">''lazyloggers''</span> can be running simultaneously on the same computer, each one taking care of a particular channel.
* Each  <span style="color:darkcyan;">''lazylogger''</span> channel will have a dedicated ODB tree (see ODB /Lazy Tree ) containing its own information.
* Each  <span style="color:darkcyan;">''lazylogger''</span> channel will have a dedicated ODB tree (see ODB /Lazy Tree ) containing its own information.
* All the  <span style="color:darkcyan;">''lazylogger''</span> channels will be in the ODB under [[/Lazy ODB tree|/Lazy/<channel-name>/...]].
* All the  <span style="color:darkcyan;">''lazylogger''</span> channels will be in the ODB under [[/Lazy ODB tree|/Lazy/<channel-name>/...]].
* Each channel tree is composed of three sub-trees [[/Lazy ODB tree#Explanation of the Keys in the ODB /Lazy/<channel_name>/Settings subtree|Settings]],[[/Lazy ODB tree#The ODB /Lazy/<channel_name>/Statistics subtree|Statistics]] and [[/Lazy ODB tree#The ODB /Lazy/<channel_name>/List subtree|List]].
* Each channel tree is composed of two sub-trees [[/Lazy ODB tree#Explanation of the Keys in the ODB /Lazy/<channel_name>/Settings subtree|Settings]],[[/Lazy ODB tree#The ODB /Lazy/<channel_name>/Statistics subtree|Statistics]] and [[/Lazy ODB tree#The ODB /Lazy/<channel_name>]].
* Dynamic directory destination based on run number or date.
* Dynamic directory destination based on run number or date.
* Script copy support.
* Script copy support.
* Compression copy.
* Compression copy.


The [[/Lazy ODB tree#Explanation of the Keys in the ODB /Lazy/<channel_name>/Settings subtree|Settings]] and [[/Lazy ODB tree#The ODB /Lazy/<channel_name>/Statistics subtree|Statistics]] trees are self-explanatory, and contain the operational parameters of the channel(s). The  [[/Lazy ODB tree#The ODB /Lazy/<channel_name>/List subtree|List]] key will have a dynamic list of run numbers which have been successfully dealt with by the lazylogger channel. This list won't exist until the first successful operation of the channel is completed. While the lazylogger was developed specifically for tape devices, it also supports data file transfer to an FTP repository system. Improvements towards more generic support (including scripting and pipe compression) have been added.
The [[/Lazy ODB tree#Explanation of the Keys in the ODB /Lazy/<channel_name>/Settings subtree|Settings]] and [[/Lazy ODB tree#The ODB /Lazy/<channel_name>/Statistics subtree|Statistics]] trees are self-explanatory, and contain the operational parameters of the channel(s). Two hidden log files for each channel are created and updated ans resides in the main experiment directory
(.<channel_name>.dirlist, .<chanel_name>.donelist), these files reflect the current state of the data copy. Lazylogger support internally data file transfer through FTP repository system. Improvements towards more generic support (including scripting and pipe compression) have been added as well.


== Usage ==
== Usage ==
Usage: lazylogger [-D] [-h <Hostname>] [-e <Experiment>] -c <channel name>


== Arguments ==
== Arguments ==
Line 28: Line 33:
  [-c channel ]        : logging channel ''<channel-name>''. Specify the lazylogger to activate.
  [-c channel ]        : logging channel ''<channel-name>''. Specify the lazylogger to activate.
  [-z ]                : zap statistics. Clear the statistics tree of all the defined lazylogger channels.
  [-z ]                : zap statistics. Clear the statistics tree of all the defined lazylogger channels.
[-t ]                : permit lazylogger to TALK msg.
[-n ]                : No file purging on the source dir.
[-d ]                : Additional debug messages


== ODB setting fields ==


The settings field covers different operation mode of the lazylogger. Some of which may not be used at the present time.


Period                        : Lazylogger check during idle phase (ex: 10) seconds
Maintain free space (%)      : Space on the source side (if exceeded, file will be purged based on status of copy (ex: 0)
Stay behind                  : Keep lazylogger copying runs with run# - "stay behind" (ex: 1)
Alarm Class                  : Assign Alarm class 
Running condition            : running condition of lazylogger (ALWAYS, NEVER, WHILE_ACQ_NOT_RUNNING) (ex: ALWAYS)
Data dir                      : Data source directory (ex: /home/midas/online)
Data format                  : Data format (def: MIDAS)
Filename format              : Data source file name format (ex: run%05d.mid)
                                In case of script, the file can be of any type (ex: iris_%08d%04d.mid.gz, run%05dsub%03d.mid.lz4)
Backup type                  : Copy method (Disk, Tape, Script, SFTP, FTP)
Execute after rewind          : Script to be performed by lazylogger  side in "backup Type=TAPE"
Path                          : Destination path (ex:
                                DISK  : /data/dragon/Summer-2019/
                                SCRIPT: /home/dsproto/online/dsproto_daq/lazy_to_analyzer.sh
                                SCRIPT: /home/agdaq/online/bin/lazy_eos.perl
                                SFTP  : sftp://user@host.domain.ca/home/user/data
                                public ssh-key is to be setup on the destination host.
Capacity (Bytes)              : Maximum capacity of the destination device, mainly for "TAPE" (ex: 5e+09)
List label                    : Lazylogger channel name (no default, to be filled by user to activate lazylogger) (ex: Mybackup)
Execute before writing file    : Script to run before each lazylogger copy operation 
Execute after writing file    : Script to run after each completion of lazylogger copy operation
Modulo.Position                : Enable multiple lazy on same source. Ex: 3ch : 3.0, 3.1, 3.2
Copy Delay                    : Run transition handling during script operation (def: 0)
Tape Data Append              : Enable positioning of the "TAPE" to EOD before each lazy copy (def: y)


== Setting up the Lazylogger ==
== Setting up the Lazylogger ==
Line 36: Line 70:
The Lazylogger requires to be set up before the data file can be moved. This setup consists of the following steps:
The Lazylogger requires to be set up before the data file can be moved. This setup consists of the following steps:


: '''1.''' Invoke the lazylogger once for setting up the appropriate ODB ''<channel-name>'' tree and exit. In this example, the ''<channel-name>'' is "Tape".
: '''1.''' Invoke the lazylogger once for setting up the appropriate ODB ''<channel-name>'' tree and exit. In this example, the ''<channel-name>'' is "Pion".
   >lazylogger -c Tape
   >lazylogger -c pion


: '''2.''' Edit the newly created ODB tree. Set the [[/Lazy ODB tree#Explanation of the Keys in the ODB /Lazy/''<channel_name>''/Settings subtree|Settings]] field(s) to match your requirements.
: '''2.''' Edit the newly created ODB tree. Set the [[/Lazy ODB tree#Explanation of the Keys in the ODB /Lazy/''<channel_name>''/Settings subtree|Settings]] field(s) to match your requirements.
::  The following example is shown using [[Odbedit|odbedit]] :
::  The following example is shown using [[Odbedit|odbedit]] :
  > odbedit -e midas
  > odbedit -e midas
  [local:midas:Stopped]/>cd /Lazy/tape/
  [local:midas:Stopped]/>cd /Lazy/pion/
  [local:midas:Stopped]tape>ls
  [local:midas:Stopped]pion>ls
  [local:midas:Stopped]tape>ls -lr
  [local:midas:Stopped]pion>ls -lr
  Key name                        Type    #Val  Size  Last Opn Mode Value
  Key name                        Type    #Val  Size  Last Opn Mode Value
  ---------------------------------------------------------------------------
  ---------------------------------------------------------------------------
  tape                            DIR
  pion                          DIR
     Settings                    DIR
     Settings                    DIR
         Maintain free space(%)  INT    1    4    3m  0  RWD  0
         Maintain free space(%)  INT    1    4    3m  0  RWD  0
Line 56: Line 90:
         Data format            STRING  1    8    3m  0  RWD  MIDAS
         Data format            STRING  1    8    3m  0  RWD  MIDAS
         Filename format        STRING  1    128  3m  0  RWD  run%05d.mid
         Filename format        STRING  1    128  3m  0  RWD  run%05d.mid
         Backup type            STRING  1    8    3m  0  RWD  Tape
         Backup type            STRING  1    8    3m  0  RWD  Pion
         Execute after rewind    STRING  1    64    3m  0  RWD   
         Execute after rewind    STRING  1    64    3m  0  RWD   
         Path                    STRING  1    128  3m  0  RWD   
         Path                    STRING  1    128  3m  0  RWD   
Line 71: Line 105:
         Number of Files        INT    1    4    3m  0  RWD  0
         Number of Files        INT    1    4    3m  0  RWD  0
         Current Lazy run        INT    1    4    3m  0  RWD  0
         Current Lazy run        INT    1    4    3m  0  RWD  0
  [local:midas:Stopped]tape>cd Settings/
  [local:midas:Stopped]pion>cd Settings/
  [local:midas:Stopped]Settings>set "Data dir" /data
  [local:midas:Stopped]Settings>set "Data dir" /data
  [local:midas:Stopped]Settings>set "Capacity (Bytes)" 15e9
  [local:midas:Stopped]Settings>set "Capacity (Bytes)" 15e9
Line 77: Line 111:


: '''3.''' Start lazylogger in the background
: '''3.''' Start lazylogger in the background
  >lazylogger -c Tape -D
  >lazylogger -c Pion -D




Line 83: Line 117:
:: At this point, the lazylogger is running and waiting for the [[/Lazy ODB tree#List label|List label]] key to be defined before starting the copy procedure.  
:: At this point, the lazylogger is running and waiting for the [[/Lazy ODB tree#List label|List label]] key to be defined before starting the copy procedure.  
  > odbedit -e midas
  > odbedit -e midas
  [local:midas:Stopped]/>cd /Lazy/tape/Settings
  [local:midas:Stopped]/>cd /Lazy/pion/Settings
  [local:midas:Stopped]Settings>set "List label" cni-043
  [local:midas:Stopped]Settings>set "List label" cni-043


''NOTE:'' The [[Mstat|mstat]] utility will display information regarding the status of the lazylogger.
''NOTE:'' The [[Mstat|mstat]] utility will display information regarding the status of the lazylogger.
== Using a script ==
If you set a "Backup type" of "Script", then your custom script (specified as the "Path") will be called for each file that gets written. This script could implement custom logic for moving or renaming files, or do some quick processing of the file. The script can be written in any language.
The script will be called with one argument - the full path to the midas file to process. It should return 0 if it processed the file successfully, and >0 if it failed. lazylogger will automatically retry files that failed. Anything written by the script to stdout will appear in the midas message log.
Note that only 1 instance of the script is ever run at once - if you are writing new files every 60s, but your script takes 90s to execute, you will get a backlog of data and should consider using a proper queue system to implement parallel processing, rather than using the lazylogger.


== Operation of the Lazylogger ==
== Operation of the Lazylogger ==


For every major operation of the <span style="color:darkcyan;">''lazylogger''</span> a message is sent to the [[Message System|Message buffer]] and will be appended to the default MIDAS log file (''midas.log'') See [[Keys in the ODB /Logger tree#/Logger/Message File|Message file]]  . These messages are the only means of finding out What/When/Where/How the lazylogger has operated on a data file. A fragment of the ''midas.log'' for the chaos experiment is shown below. In this case, the [[/Lazy ODB tree#Maintain free space(%)|Maintain free space]] field was enabled which produces the cleanup of the data files and the entry in the  [[/Lazy ODB tree#The ODB /Lazy/<channel_name>/List subtree|List]]  tree after copy.
For every major operation of the <span style="color:darkcyan;">''lazylogger''</span> a message is sent to the [[Message System|Message buffer]] and will be appended to a default Lazy log file (''lazy.log''). These messages are the only means of finding out What/When/Where/How the lazylogger has operated on a data file. A fragment of the ''lazy.log'' for the iris experiment is shown below. In this case, the [[/Lazy ODB tree#Maintain free space(%)|Maintain free space]] field was enabled which produces the cleanup of the data files and the entry in the  [[/Lazy ODB tree#The ODB /Lazy/<channel_name>/List subtree|List]]  tree after copy. In this particular case, the lazylogger uses a perl script to push the data file to a data storage system using dCache.


  Fri Mar 24 14:40:08 2000 [Lazy_Tape] 8351 (rm:16050ms) /scr0/spring2000/run08351.ybs file REMOVED
  [local:iris:Stopped]/Lazy>
  Fri Mar 24 14:40:08 2000 [Lazy_Tape] Tape run#8351 entry REMOVED
  Key name                        Type    #Val  Size  Last Opn Mode Value
  Fri Mar 24 14:59:55 2000 [Logger] stopping run after having received 1200000 events
  ---------------------------------------------------------------------------
  Fri Mar 24 14:59:56 2000 [CHAOS] Run 8366 stopped
  Lazy                            DIR
  Fri Mar 24 14:59:56 2000 [Logger] Run #8366 stopped
    Dcache                      DIR
  Fri Mar 24 14:59:57 2000 [SUSI] saving info in run log
        Settings                DIR
  Fri Mar 24 15:00:07 2000 [Logger] starting new run
            Period              INT    1    4    51h  0  RWD 2
  Fri Mar 24 15:00:07 2000 [CHAOS] Run 8367 started
            Maintain free space INT    1    4    51h 0  RWD  15
  Fri Mar 24 15:00:07 2000 [Logger] Run #8367 started
            Stay behind        INT    1    4    51h 0  RWD  1
  Fri Mar 24 15:06:59 2000 [Lazy_Tape] cni-043[15] (cp:410.6s) /dev/nst0/run08365.ybs 864.020MB file NEW
            Alarm Class        STRING  1    32    51h  0  RWD  
  Fri Mar 24 15:07:35 2000 [Lazy_Tape] 8352 (rm:25854ms) /scr0/spring2000/run08352.ybs file REMOVED
            Running condition  STRING 1    128  51h  0  RWD  ALWAYS
  Fri Mar 24 15:07:35 2000 [Lazy_Tape] Tape run#8352 entry REMOVED
            Data dir            STRING  1    256  51h  0  RWD /iris_data0/iris/current
  Fri Mar 24 15:27:09 2000 [Lazy_Tape] 8353 (rm:23693ms) /scr0/spring2000/run08353.ybs file REMOVED
            Data format        STRING  1    8    51h  0  RWD  MIDAS
  Fri Mar 24 15:27:09 2000 [Lazy_Tape] Tape run#8353 entry REMOVED
            Filename format    STRING  1    128  51h  0  RWD  iris_%08d%04d.mid.gz
  Fri Mar 24 15:33:22 2000 [Logger] stopping run after having received 1200000 events
            Backup type        STRING  1    8    51h  0  RWD Script
  Fri Mar 24 15:33:22 2000 [CHAOS] Run 8367 stopped
            Execute after rewindSTRING  1    64    51h  0  RWD  
  Fri Mar 24 15:33:23 2000 [Logger] Run #8367 stopped
            Path                STRING  1    128  51h 0  RWD  /home/iris/bin/lazy_dcache.perl
  Fri Mar 24 15:33:24 2000 [SUSI] saving info in run log
            Capacity (Bytes)    FLOAT  1    4    51h 0  RWD  5e+09
  Fri Mar 24 15:33:33 2000 [Logger] starting new run
            List label          STRING  1    128  51h  0  RWD  Dcache
  Fri Mar 24 15:33:34 2000 [CHAOS] Run 8368 started
            Execute before writiSTRING 1    64    51h  0  RWD 
  Fri Mar 24 15:33:34 2000 [Logger] Run #8368 started
            Execute after writinSTRING  1    64    51h  0  RWD 
Fri Mar 24 15:40:18 2000 [Lazy_Tape] cni-043[16] (cp:395.4s) /dev/nst0/run08366.ybs 857.677MB file NEW
            Modulo.Position    STRING  1    8    51h  0  RWD  
  Fri Mar 24 15:50:15 2000 [Lazy_Tape] 8354 (rm:28867ms) /scr0/spring2000/run08354.ybs file REMOVED
            Tape Data Append    BOOL    1    4    51h  0  RWD y
  Fri Mar 24 15:50:15 2000 [Lazy_Tape] Tape run#8354 entry REMOVED
          Statistics              DIR
            Backup file        STRING 1    128  15h  0  RWDE iris_000035270000.mid.gz
            File size (Bytes)  DOUBLE 1    8    15h  0  RWDE 326345493
            KBytes copied      DOUBLE 1    8    15h  0  RWDE 0
            Total Bytes copied  DOUBLE  1    8    15h 0  RWDE 0
            Copy progress (%)   DOUBLE  1    8    15h  0  RWDE 0
            Copy Rate (Bytes perDOUBLE 1    8    15h  0  RWDE 0
            Backup status (%)   DOUBLE  1    8    15h  0  RWDE 0
            Number of Files    INT    1    4    15h  0  RWDE 247
            Current Lazy run    INT    1    4    15h  0  RWDE 35270000
 
  Excerpt of lazy.log:
  ...
  ...
Wed May  6 17:26:00 2015 [Lazy_Dcache] Dcache[30] (cp:1.0s) iris_000032140000.mid.gz 6.146MB  file NEW
Wed May  6 17:26:02 2015 [Lazy_Dcache] Dcache[31] (cp:0.1s) iris_000032150000.mid.gz 22.894MB  file NEW
Wed May  6 17:26:04 2015 [Lazy_Dcache] Dcache[32] (cp:0.1s) iris_000032160000.mid.gz 116.110MB  file NEW
Wed May  6 17:26:07 2015 [Lazy_Dcache] Dcache[33] (cp:0.1s) iris_000032170000.mid.gz 110.228MB  file NEW
Wed May  6 17:26:09 2015 [Lazy_Dcache] Dcache[34] (cp:0.0s) iris_000032180000.mid.gz 101.333MB  file NEW
Wed May  6 17:26:12 2015 [Lazy_Dcache] Dcache[35] (cp:0.1s) iris_000032190000.mid.gz 90.928MB  file NEW
Wed May  6 17:26:14 2015 [Lazy_Dcache] Dcache[36] (cp:0.1s) iris_000032200000.mid.gz 88.461MB  file NEW
Thu May  7 15:47:02 2015 [Lazy_Dcache] Dcache[1] (cp:3.2s) iris_000032210000.mid.gz 87.862MB  file NEW
Thu May  7 15:47:06 2015 [Lazy_Dcache] Dcache[2] (cp:2.2s) iris_000032220000.mid.gz 74.789MB  file NEW
Thu May  7 15:47:10 2015 [Lazy_Dcache] Dcache[3] (cp:1.2s) iris_000032230000.mid.gz 0.204MB  file NEW
Thu May  7 15:47:16 2015 [Lazy_Dcache] Dcache[4] (cp:4.1s) iris_000032240000.mid.gz 105.346MB  file NEW
Thu May  7 15:47:20 2015 [Lazy_Dcache] Dcache[5] (cp:1.2s) iris_000032250000.mid.gz 14.048MB  file NEW
Thu May  7 15:47:23 2015 [Lazy_Dcache] Dcache[6] (cp:1.2s) iris_000032260000.mid.gz 5.011MB  file NEW
Thu May  7 15:47:30 2015 [Lazy_Dcache] Dcache[7] (cp:4.2s) iris_000032270000.mid.gz 94.028MB  file NEW
Thu May  7 15:47:36 2015 [Lazy_Dcache] Dcache[8] (cp:4.2s) iris_000032280000.mid.gz 110.765MB  file NEW
Thu May  7 15:47:42 2015 [Lazy_Dcache] Dcache[9] (cp:3.0s) iris_000032290000.mid.gz 88.982MB  file NEW
Thu May  7 15:47:46 2015 [Lazy_Dcache] Dcache[10] (cp:2.0s) iris_000032300000.mid.gz 51.735MB  file NEW
...
Once the lazylogger has started a job on a data file, trying to terminate the application will result in producing a log message informing the user of the actual percentage of the backup completed so far. This message will repeat itself until completion of the backup, and only then the lazylogger application will terminate.
Once the lazylogger has started a job on a data file, trying to terminate the application will result in producing a log message informing the user of the actual percentage of the backup completed so far. This message will repeat itself until completion of the backup, and only then the lazylogger application will terminate.
If an interruption of the lazylogger is forced (kill...), the state of the backup device is undetermined. Recovery is not possible, and the full backup set has to be redone. In order to do this, you need:
#To rewind the backup device.
#Delete the /Lazy/<channel_name>/List/<list label> array.
#Restart the lazylogger with the -z switch which will "zap" the statistics entries.
In order to facilitate the recovery procedure, lazylogger produces an ODB ASCII file of the lazy channel tree after completion of successful operation. This file (''Tape_recover.odb'') stored in [[/Lazy ODB tree#Data dir|Data dir]] can be used for ODB as well as lazylogger recovery.


[[Category:Utilities]]
[[Category:Utilities]] [[Category:Data Logging]]

Latest revision as of 10:23, 1 December 2020


Links


Purpose

The lazylogger utility decouples the data acquisition from the data logging mechanism. The need for such an application has been dictated by the slow response time of the old media logging devices, such as tape devices. Delay due to tape mounting, re-tension and re-positioning implies that the data acquisition has to be held until operation completion. By using mlogger to log data to disk in a first stage and then using lazylogger to copy or move the stored files, the data acquisition can be kept running without interruption. These devices are no longer supported internally by Midas but can still be managed through scripts. Lazylogger is still much used for getting a copy of the original data to a remote data storage. While there are many ways to do this independently of the acquisition system, the lazylogger provides a simple way to manage this task. Lazylogger features are listed below.

  • Multiple lazyloggers can be running simultaneously on the same computer, each one taking care of a particular channel.
  • Each lazylogger channel will have a dedicated ODB tree (see ODB /Lazy Tree ) containing its own information.
  • All the lazylogger channels will be in the ODB under /Lazy/<channel-name>/....
  • Each channel tree is composed of two sub-trees Settings,Statistics and /Lazy ODB tree#The ODB /Lazy/<channel_name>.
  • Dynamic directory destination based on run number or date.
  • Script copy support.
  • Compression copy.

The Settings and Statistics trees are self-explanatory, and contain the operational parameters of the channel(s). Two hidden log files for each channel are created and updated ans resides in the main experiment directory (.<channel_name>.dirlist, .<chanel_name>.donelist), these files reflect the current state of the data copy. Lazylogger support internally data file transfer through FTP repository system. Improvements towards more generic support (including scripting and pipe compression) have been added as well.

Usage

Usage: lazylogger [-D] [-h <Hostname>] [-e <Experiment>] -c <channel name>

Arguments

[-h ] : help.
[-h hostname ]        : see common parameters
[-e experiment_name ] : see common parameters
[-D ]                 : start program as a daemon.
[-c channel ]         : logging channel <channel-name>. Specify the lazylogger to activate.
[-z ]                 : zap statistics. Clear the statistics tree of all the defined lazylogger channels.
[-t ]                 : permit lazylogger to TALK msg.
[-n ]                 : No file purging on the source dir.
[-d ]                 : Additional debug messages

ODB setting fields

The settings field covers different operation mode of the lazylogger. Some of which may not be used at the present time. 

Period                        : Lazylogger check during idle phase (ex: 10) seconds
Maintain free space (%)       : Space on the source side (if exceeded, file will be purged based on status of copy (ex: 0) 
Stay behind                   : Keep lazylogger copying runs with run# - "stay behind" (ex: 1)
Alarm Class                   : Assign Alarm class  
Running condition             : running condition of lazylogger (ALWAYS, NEVER, WHILE_ACQ_NOT_RUNNING) (ex: ALWAYS)
Data dir                      : Data source directory (ex: /home/midas/online)
Data format                   : Data format (def: MIDAS)
Filename format               : Data source file name format (ex: run%05d.mid)
                                In case of script, the file can be of any type (ex: iris_%08d%04d.mid.gz, run%05dsub%03d.mid.lz4)
Backup type                   : Copy method (Disk, Tape, Script, SFTP, FTP)
Execute after rewind          : Script to be performed by lazylogger  side in "backup Type=TAPE"
Path                          : Destination path (ex:
                                DISK  : /data/dragon/Summer-2019/
                                SCRIPT: /home/dsproto/online/dsproto_daq/lazy_to_analyzer.sh
                                SCRIPT: /home/agdaq/online/bin/lazy_eos.perl
                                SFTP  : sftp://user@host.domain.ca/home/user/data
                                public ssh-key is to be setup on the destination host.
Capacity (Bytes)               : Maximum capacity of the destination device, mainly for "TAPE" (ex: 5e+09)
List label                     : Lazylogger channel name (no default, to be filled by user to activate lazylogger) (ex: Mybackup)
Execute before writing file    : Script to run before each lazylogger copy operation  
Execute after writing file     : Script to run after each completion of lazylogger copy operation
Modulo.Position                : Enable multiple lazy on same source. Ex: 3ch : 3.0, 3.1, 3.2 
Copy Delay                     : Run transition handling during script operation (def: 0)
Tape Data Append               : Enable positioning of the "TAPE" to EOD before each lazy copy (def: y)

Setting up the Lazylogger

The Lazylogger requires to be set up before the data file can be moved. This setup consists of the following steps:

1. Invoke the lazylogger once for setting up the appropriate ODB <channel-name> tree and exit. In this example, the <channel-name> is "Pion".
 >lazylogger -c pion
2. Edit the newly created ODB tree. Set the Settings field(s) to match your requirements.
The following example is shown using odbedit :
> odbedit -e midas
[local:midas:Stopped]/>cd /Lazy/pion/
[local:midas:Stopped]pion>ls
[local:midas:Stopped]pion>ls -lr
Key name                        Type    #Val  Size  Last Opn Mode Value
---------------------------------------------------------------------------
pion                           DIR
   Settings                    DIR
       Maintain free space(%)  INT     1     4     3m   0   RWD  0
       Stay behind             INT     1     4     3m   0   RWD  -3
       Alarm Class             STRING  1     32    3m   0   RWD  
       Running condition       STRING  1     128   3m   0   RWD  ALWAYS
       Data dir                STRING  1     256   3m   0   RWD  /home/midas/online
       Data format             STRING  1     8     3m   0   RWD  MIDAS
       Filename format         STRING  1     128   3m   0   RWD  run%05d.mid
       Backup type             STRING  1     8     3m   0   RWD  Pion
       Execute after rewind    STRING  1     64    3m   0   RWD  
       Path                    STRING  1     128   3m   0   RWD  
       Capacity (Bytes)        FLOAT   1     4     3m   0   RWD  5e+09
       List label              STRING  1     128   3m   0   RWD  
   Statistics                  DIR
       Backup file             STRING  1     128   3m   0   RWD  none 
       File size [Bytes]       DOUBLE  1     4     3m   0   RWD  0
       KBytes copied           DOUBLE  1     4     3m   0   RWD  0
       Total Bytes copied      DOUBLE  1     4     3m   0   RWD  0
       Copy progress [%]       DOUBLE  1     4     3m   0   RWD  0
       Copy Rate [bytes per s] DOUBLE  1     4     3m   0   RWD  0
       Backup status [%]       DOUBLE  1     4     3m   0   RWD  0
       Number of Files         INT     1     4     3m   0   RWD  0
       Current Lazy run        INT     1     4     3m   0   RWD  0
[local:midas:Stopped]pion>cd Settings/
[local:midas:Stopped]Settings>set "Data dir" /data
[local:midas:Stopped]Settings>set "Capacity (Bytes)" 15e9


3. Start lazylogger in the background
>lazylogger -c Pion -D


4. Define List label key
At this point, the lazylogger is running and waiting for the List label key to be defined before starting the copy procedure.
> odbedit -e midas
[local:midas:Stopped]/>cd /Lazy/pion/Settings
[local:midas:Stopped]Settings>set "List label" cni-043

NOTE: The mstat utility will display information regarding the status of the lazylogger.

Using a script

If you set a "Backup type" of "Script", then your custom script (specified as the "Path") will be called for each file that gets written. This script could implement custom logic for moving or renaming files, or do some quick processing of the file. The script can be written in any language.

The script will be called with one argument - the full path to the midas file to process. It should return 0 if it processed the file successfully, and >0 if it failed. lazylogger will automatically retry files that failed. Anything written by the script to stdout will appear in the midas message log.

Note that only 1 instance of the script is ever run at once - if you are writing new files every 60s, but your script takes 90s to execute, you will get a backlog of data and should consider using a proper queue system to implement parallel processing, rather than using the lazylogger.

Operation of the Lazylogger

For every major operation of the lazylogger a message is sent to the Message buffer and will be appended to a default Lazy log file (lazy.log). These messages are the only means of finding out What/When/Where/How the lazylogger has operated on a data file. A fragment of the lazy.log for the iris experiment is shown below. In this case, the Maintain free space field was enabled which produces the cleanup of the data files and the entry in the List tree after copy. In this particular case, the lazylogger uses a perl script to push the data file to a data storage system using dCache. 

[local:iris:Stopped]/Lazy>
Key name                        Type    #Val  Size  Last Opn Mode Value
---------------------------------------------------------------------------
Lazy                            DIR
    Dcache                      DIR
        Settings                DIR
            Period              INT     1     4     51h  0   RWD  2
            Maintain free space INT     1     4     51h  0   RWD  15
            Stay behind         INT     1     4     51h  0   RWD  1
            Alarm Class         STRING  1     32    51h  0   RWD  
            Running condition   STRING  1     128   51h  0   RWD  ALWAYS
            Data dir            STRING  1     256   51h  0   RWD  /iris_data0/iris/current
            Data format         STRING  1     8     51h  0   RWD  MIDAS
            Filename format     STRING  1     128   51h  0   RWD  iris_%08d%04d.mid.gz
            Backup type         STRING  1     8     51h  0   RWD  Script
            Execute after rewindSTRING  1     64    51h  0   RWD  
            Path                STRING  1     128   51h  0   RWD  /home/iris/bin/lazy_dcache.perl
            Capacity (Bytes)    FLOAT   1     4     51h  0   RWD  5e+09
            List label          STRING  1     128   51h  0   RWD  Dcache
            Execute before writiSTRING  1     64    51h  0   RWD   
            Execute after writinSTRING  1     64    51h  0   RWD  
            Modulo.Position     STRING  1     8     51h  0   RWD  
            Tape Data Append    BOOL    1     4     51h  0   RWD  y
         Statistics              DIR
            Backup file         STRING  1     128   15h  0   RWDE iris_000035270000.mid.gz
            File size (Bytes)   DOUBLE  1     8     15h  0   RWDE 326345493
            KBytes copied       DOUBLE  1     8     15h  0   RWDE 0
            Total Bytes copied  DOUBLE  1     8     15h  0   RWDE 0
            Copy progress (%)   DOUBLE  1     8     15h  0   RWDE 0
            Copy Rate (Bytes perDOUBLE  1     8     15h  0   RWDE 0
            Backup status (%)   DOUBLE  1     8     15h  0   RWDE 0
            Number of Files     INT     1     4     15h  0   RWDE 247
            Current Lazy run    INT     1     4     15h  0   RWDE 35270000

Excerpt of lazy.log:
...
Wed May  6 17:26:00 2015 [Lazy_Dcache] Dcache[30] (cp:1.0s) iris_000032140000.mid.gz 6.146MB  file NEW
Wed May  6 17:26:02 2015 [Lazy_Dcache] Dcache[31] (cp:0.1s) iris_000032150000.mid.gz 22.894MB  file NEW
Wed May  6 17:26:04 2015 [Lazy_Dcache] Dcache[32] (cp:0.1s) iris_000032160000.mid.gz 116.110MB  file NEW
Wed May  6 17:26:07 2015 [Lazy_Dcache] Dcache[33] (cp:0.1s) iris_000032170000.mid.gz 110.228MB  file NEW
Wed May  6 17:26:09 2015 [Lazy_Dcache] Dcache[34] (cp:0.0s) iris_000032180000.mid.gz 101.333MB  file NEW
Wed May  6 17:26:12 2015 [Lazy_Dcache] Dcache[35] (cp:0.1s) iris_000032190000.mid.gz 90.928MB  file NEW
Wed May  6 17:26:14 2015 [Lazy_Dcache] Dcache[36] (cp:0.1s) iris_000032200000.mid.gz 88.461MB  file NEW
Thu May  7 15:47:02 2015 [Lazy_Dcache] Dcache[1] (cp:3.2s) iris_000032210000.mid.gz 87.862MB  file NEW
Thu May  7 15:47:06 2015 [Lazy_Dcache] Dcache[2] (cp:2.2s) iris_000032220000.mid.gz 74.789MB  file NEW
Thu May  7 15:47:10 2015 [Lazy_Dcache] Dcache[3] (cp:1.2s) iris_000032230000.mid.gz 0.204MB  file NEW
Thu May  7 15:47:16 2015 [Lazy_Dcache] Dcache[4] (cp:4.1s) iris_000032240000.mid.gz 105.346MB  file NEW
Thu May  7 15:47:20 2015 [Lazy_Dcache] Dcache[5] (cp:1.2s) iris_000032250000.mid.gz 14.048MB  file NEW
Thu May  7 15:47:23 2015 [Lazy_Dcache] Dcache[6] (cp:1.2s) iris_000032260000.mid.gz 5.011MB  file NEW
Thu May  7 15:47:30 2015 [Lazy_Dcache] Dcache[7] (cp:4.2s) iris_000032270000.mid.gz 94.028MB  file NEW
Thu May  7 15:47:36 2015 [Lazy_Dcache] Dcache[8] (cp:4.2s) iris_000032280000.mid.gz 110.765MB  file NEW
Thu May  7 15:47:42 2015 [Lazy_Dcache] Dcache[9] (cp:3.0s) iris_000032290000.mid.gz 88.982MB  file NEW
Thu May  7 15:47:46 2015 [Lazy_Dcache] Dcache[10] (cp:2.0s) iris_000032300000.mid.gz 51.735MB  file NEW

... Once the lazylogger has started a job on a data file, trying to terminate the application will result in producing a log message informing the user of the actual percentage of the backup completed so far. This message will repeat itself until completion of the backup, and only then the lazylogger application will terminate.

Retrieved from "https://daq00.triumf.ca/MidasWiki/index.php?title=Lazylogger&oldid=3056"