ODB Access and Use: Difference between revisions

From MidasWiki
Jump to navigation Jump to search
(Created page with "==== Links ==== <div style="column-count:3;-moz-column-count:3;-webkit-column-count:3"> * odbedit * mhttpd ODB page * * </div> == Introduction == The MIDAS Online D...")
 
No edit summary
Line 10: Line 10:
The MIDAS Online Data Base (ODB) is the main communication platform between MIDAS applications. It contains all the information needed to run an experiment.  
The MIDAS Online Data Base (ODB) is the main communication platform between MIDAS applications. It contains all the information needed to run an experiment.  


The ODB is hierarchically structured, similar to a file system, with directories and sub-directories (or trees and subtrees). The data are stored in key/data pairs, similar to the Windows NT registry. Keys can be dynamically created and deleted. The data associated with a key can be of different types such as: byte, words, double words, float, strings, etc. or arrays of any of those. A key can also be a directory or a symbolic link.
== ODB Keys ==
The ODB is hierarchically structured, similar to a file system, with directories and sub-directories (or trees and subtrees). The data are stored in key/data pairs, similar to the Windows NT registry. ODB keys can be dynamically created and deleted. The data associated with a key can be of different types such as: byte, words, double words, float, strings, etc. or arrays of any of those. A key can also be a directory or a symbolic link. ODB Key names are case-independent, and may contain spaces.


== Accessing the ODB ==
== Accessing the ODB ==
Line 17: Line 18:
Scripts can easily access the odb using the [[odbedit]] -c argument.
Scripts can easily access the odb using the [[odbedit]] -c argument.


Each field or '''KEY''' in the MIDAS ODB can be accessed '''interactively''' in two possible ways:
The MIDAS ODB can be accessed interactively in two ways:
* Using the [[odbedit|odbedit application]] command line.
* Using the [[odbedit|odbedit application]] command line.
* Using a web browser accessing the URL of the [[mhttpd|midas server]]
* Using a web browser accessing the URL of the [[mhttpd|midas server]]




== The ODB Structure ==
= Creating the ODB =
The data in the ODB are structured in a '''tree form''' with each "directory" dealing with a specific section of the acquisition.  
After installation of MIDAS, before any other applications are started, the ODB must be created by the user (see [[odbedit #Creating the ODB|creating the ODB ]]).  Running [[odbedit]] to create the ODB also causes the trees [[/Runinfo ODB tree||/Runinfo]], [[/Experiment ODB tree|/Experiment]], [[/System ODB tree|/System]] to be created and filled with default values. Other system applications will automatically create their own initial ODB structure
Particular system applications will automatically create their own initial ODB structure
filled with default values (for example, the [[mlogger|data logger]] creates the  [[/Logger ODB tree]]. The user may then modify these configurations to fit  his/her requirements  (see [[#Customizing|customizing]]). A list of the trees in the ODB  used by the system can be found [[Online Database|here]].  
filled with default values (for example, the [[mlogger|data logger]] creates the  [[/Logger ODB tree]]. The user may then modify these configurations to fit  his/her requirements  (see [[#Customizing|customizing]]). A list of the trees in the ODB  used by the system can be found [[Online Database|here]].  


== Shared-memory files ==
[[odbedit #Creating the ODB|Creating the ODB ]]also creates
all the shared-memory files needed for the experiment (see list below). By default, these files will be created in the area indicated in the [[exptab]] file for your experiment.
<pre>
.ALARM.SHM  .ELOG.SHM  .HISTORY.SHM  .MSG.SHM  .ODB.SHM  .SYSMSG.SHM  .SYSTEM.SHM
</pre>


 
= Customizing the ODB =
== Customizing ==
The user customizes the ODB for their experiment by modifying the values of "system" parameters (e.g. [[Keys in the ODB /Logger tree]], and adds extra keys under existing trees as needed, e.g. "Edit on Start" parameters under the [[/Experiment ODB tree]].  The user may creates "optional" trees e.g. (/Script, /Custom) populating them with keys as required.
The user customizes the ODB for their experiment by modifying the values of "system" parameters (e.g. [[Keys in the ODB /Logger tree]], and adds extra keys under existing trees as needed, e.g. "Edit on Start" parameters under the [[/Experiment ODB tree]].  The user may creates "optional" trees e.g. (/Script, /Custom) populating them with keys as required.
The user is also free to create his/her own tree structure under root (/) for his/her own purposes.  
The user is also free to create his/her own tree structure under root (/) for his/her own purposes.  


== Creating ODB keys ==
ODB Key names are case-independent and may contain spaces. Key names with spaces must be enclosed in quotes.
The easiest way to create, set (and delete if necessary) ODB keys is to use the [[mhttpd]] ODB page "Create" button. Alternatively, [[odbedit]] can be used as demonstrated in the following example. [[Odbedit]] is more powerful, also allowing the user to re-order, copy and rename the keys.
Once a directory is created, keys can be created of the types supported by MIDAS i.e.
INT DWORD BOOL FLOAT DOUBLE STRING, defined in midas.h.  Arrays of all these types can also be created.
Values are assigned to the keys e.g. with the [[odbedit]] set command.
=== Example ===
This example shows how to create an ODB tree (<span style="color:purple; font-style:italic">/Custom</span>) using [[odbedit]].
The [[odbedit]] commands are shown in red. Comments are shown in green and preceded by a "#".
$ odbedit 
[local:midas:S]/>pwd    <span style="color:green; font-style:normal"># show current directory (tree) </span>
/
[local:midas:S]/><span style="color:red; font-style:normal">mkdir custom/images</span>    <span style="color:green; font-style:normal"># make directory "custom" and subdirectory "images"</span>
[local:midas:S]/><span style="color:red; font-style:normal">ls</span>  <span style="color:green; font-style:normal"># list all keys</span>
System
Programs
Experiment
Runinfo
Alarms
Custom
[local:midas:S]/><span style="color:red; font-style:normal">cd custom </span>  <span style="color:green; font-style:normal"># change directory to "/custom" </span>
[local:midas:S]/custom><span style="color:red; font-style:normal">ls</span>
images
[local:midas:S]/custom><span style="color:red; font-style:normal">create string "custom functions&"</span>  <span style="color:green; font-style:normal"># Create a key of type STRING. Note use of quotes as key name contains a space </span>
String length [32]: 128
[local:midas:S]/custom><span style="color:red; font-style:normal">set "custom functions&" /home/midas/custom/custom.html</span>  <span style="color:green; font-style:normal"># Set key to a value. Could type "set cus" followed by tab to demonstrate tab-completion </span>
[local:midas:S]/custom><span style="color:red; font-style:normal">ls -r</span>  <span style="color:green; font-style:normal"># list recursively </span>
custom
    images
    custom functions&          /home/midas/custom/custom.html
== Creating arrays with odbedit ==
The following example demonstrates the use of [[odbedit]] to create an array, set its elements to various value, and expand it. An array can be truncated using the [[odbedit]] trunc command.
[local:pol:S]/test>create int fred[5]
[local:pol:S]/test>set fred[*] 5
[local:pol:S]/test>set fred[1..3] 6
[local:pol:S]/test>set fred[8] 9
[local:pol:S]/test>ls
fred
                                5
                                6
                                6
                                6
                                5
                                0
                                0
                                0
                                9
== Reorder ODB keys ==
Sometimes it may be desired to reorder the ODB keys once created. The [odbedit] move command is used for this purpose as in the following example:
The key "custom" can be moved to the top (or bottom) of the list, e.g.
[local:midas:S]/>ls
System
Programs
Experiment
Runinfo
Alarms
Custom
The key "Custom" can be moved to the top
[local:midas:S]/>move custom top
[local:midas:S]/>ls
Custom
System
Programs
Experiment
Runinfo
Alarms
or to any position, e.g.
[local:midas:Stopped]/>move custom 1
[local:midas:Stopped]/>ls
System
Custom
Programs
Experiment
Runinfo
Alarms
== Create a link in the ODB ==
An example of creating  softlinks in the odb can be found under [[Edit-on-start Parameters#Creating edit-on-start parameters|Creating edit-on-start parameters]].


= Creating the ODB =
= Save and reload the ODB =
See [[odbedit #Creating the ODB|creating the ODB ]]
The ODB can be saved at its current position in several formats (ascii, xml as a c structure) using the [[odbedit]] save command, and reloaded from a saved file with the [[odbedit]] load command.
 
Since the ODB may become corrupted,  a copy is saved automatically at the beginning of each run. Since this file is overwritten each time, users often also save a copy (that will not be overwritten) at the end of each run (see [[Keys in the ODB /Logger tree#ODB Dump|ODB Dump]]) along with the data for that run.
Users can then return their ODB to the state it was in for any particular run.
 
The entire database need not be loaded. Saved ASCII files can be made of just a part of the database, and these can be reloaded into the database. Since the full path is given in the saved file, the file can be loaded from any position in the database.
 
The saved ASCII file may of course be edited prior to loading, if keynames or values need to be changed. If the keys in the load file do not exist, they will be created. If they do exist, the values from the file will be loaded.
 
 
 
 
= '''experim.h''' include file =
 
The [[odbedit]] make command creates "experim.h", a file containing C structures which can be included into frontend and analyzer code to enable easy access to the ODB parameters (see also [[Frontend]]). The file is created in the current directory. The file experim.h contains C structures for the ODB trees <span style="color:purple; font-style:italic">/Experiment, /Analyzer/Parameters</span>, and <span style="color:purple; font-style:italic">/Equipment/<equipment-name>/Settings</span> if these trees are present.  All C structures are accompanied with a string representation which can be used in the ''db_create_record'' function to setup an ODB structure which matches the C structure.  
 
In order to include the analyzer section, the ODB key <span style="color:purple; font-style:italic">/<Analyzer>/Parameters</span> has to be present, where <Analyzer> is the name of the analyzer. The command used is then "make <Analyzer> ".
 
Examples using ''experim.h'' can be found [[Event Notification (Hot-Link)#How to set up a Hot-Link|How to set up a Hot-Link]] and [[Frontend user code#Frontend code|Frontend code]].
 
 
 
 
= Corrupted ODB =
If the Online Database becomes corrupted, [[odbedit]] may no longer work, and other clients will also fail to open the database. In this case, the old ODB should be deleted and a new one created. The contents of the ODB can be reloaded from a (see #Save and reload the ODB|saved ODB file]].
 
To delete the corrupted ODB, delete the  [[#Shared-memory files|*.SHM files]] for your experiment.


= Creating Keys in the ODB =
[mpet@titan01 ~/online]$ ls .*.SHM
The first time [[odbedit]] is run, the trees [[/Runinfo ODB tree||/Runinfo]], [[/Experiment ODB tree|/Experiment]], [[/System ODB tree|/System]] will be created and filled with default values.
.ALARM.SHM  .ELOG.SHM  .HISTORY.SHM  .MSG.SHM .ODB.SHM  .SYSMSG.SHM  .SYSTEM.SHM
Other applications will add their own trees (see [[#The ODB Structure|above]]).
[mpet@titan01 ~/online]$ rm .*.SHM


The user may customize the ODB by adding more keys.
Create new *.SHM files by [[#Creating the ODB|re-creating the ODB]], then [[#Save and reload the ODB|load a saved file]] containing the latest copy of the ODB contents.

Revision as of 16:26, 27 June 2014

Links

Introduction

The MIDAS Online Data Base (ODB) is the main communication platform between MIDAS applications. It contains all the information needed to run an experiment.

ODB Keys

The ODB is hierarchically structured, similar to a file system, with directories and sub-directories (or trees and subtrees). The data are stored in key/data pairs, similar to the Windows NT registry. ODB keys can be dynamically created and deleted. The data associated with a key can be of different types such as: byte, words, double words, float, strings, etc. or arrays of any of those. A key can also be a directory or a symbolic link. ODB Key names are case-independent, and may contain spaces.

Accessing the ODB

The MIDAS library provides a complete set of functions to manage and operate on these keys (in odb.c) (see odb functions). Scripts can easily access the odb using the odbedit -c argument.

The MIDAS ODB can be accessed interactively in two ways:


Creating the ODB

After installation of MIDAS, before any other applications are started, the ODB must be created by the user (see creating the ODB ). Running odbedit to create the ODB also causes the trees |/Runinfo, /Experiment, /System to be created and filled with default values. Other system applications will automatically create their own initial ODB structure filled with default values (for example, the data logger creates the /Logger ODB tree. The user may then modify these configurations to fit his/her requirements (see customizing). A list of the trees in the ODB used by the system can be found here.

Shared-memory files

Creating the ODB also creates all the shared-memory files needed for the experiment (see list below). By default, these files will be created in the area indicated in the exptab file for your experiment.

.ALARM.SHM  .ELOG.SHM  .HISTORY.SHM  .MSG.SHM  .ODB.SHM  .SYSMSG.SHM  .SYSTEM.SHM

Customizing the ODB

The user customizes the ODB for their experiment by modifying the values of "system" parameters (e.g. Keys in the ODB /Logger tree, and adds extra keys under existing trees as needed, e.g. "Edit on Start" parameters under the /Experiment ODB tree. The user may creates "optional" trees e.g. (/Script, /Custom) populating them with keys as required. The user is also free to create his/her own tree structure under root (/) for his/her own purposes.

Creating ODB keys

ODB Key names are case-independent and may contain spaces. Key names with spaces must be enclosed in quotes.

The easiest way to create, set (and delete if necessary) ODB keys is to use the mhttpd ODB page "Create" button. Alternatively, odbedit can be used as demonstrated in the following example. Odbedit is more powerful, also allowing the user to re-order, copy and rename the keys.

Once a directory is created, keys can be created of the types supported by MIDAS i.e. INT DWORD BOOL FLOAT DOUBLE STRING, defined in midas.h. Arrays of all these types can also be created.

Values are assigned to the keys e.g. with the odbedit set command.

Example

This example shows how to create an ODB tree (/Custom) using odbedit. The odbedit commands are shown in red. Comments are shown in green and preceded by a "#".

$ odbedit   
[local:midas:S]/>pwd    # show current directory (tree) 
/
[local:midas:S]/>mkdir custom/images    # make directory "custom" and subdirectory "images"
[local:midas:S]/>ls  # list all keys
System
Programs
Experiment
Runinfo
Alarms
Custom
[local:midas:S]/>cd custom   # change directory to "/custom" 
[local:midas:S]/custom>ls
images
[local:midas:S]/custom>create string "custom functions&"  # Create a key of type STRING. Note use of quotes as key name contains a space 
String length [32]: 128
[local:midas:S]/custom>set "custom functions&" /home/midas/custom/custom.html  # Set key to a value. Could type "set cus" followed by tab to demonstrate tab-completion 
[local:midas:S]/custom>ls -r  # list recursively 
custom
   images
   custom functions&           /home/midas/custom/custom.html

Creating arrays with odbedit

The following example demonstrates the use of odbedit to create an array, set its elements to various value, and expand it. An array can be truncated using the odbedit trunc command.

[local:pol:S]/test>create int fred[5]
[local:pol:S]/test>set fred[*] 5
[local:pol:S]/test>set fred[1..3] 6
[local:pol:S]/test>set fred[8] 9
[local:pol:S]/test>ls
fred
                               5
                               6
                               6
                               6
                               5
                               0
                               0
                               0
                               9


Reorder ODB keys

Sometimes it may be desired to reorder the ODB keys once created. The [odbedit] move command is used for this purpose as in the following example:

The key "custom" can be moved to the top (or bottom) of the list, e.g.

[local:midas:S]/>ls
System
Programs
Experiment
Runinfo
Alarms
Custom

The key "Custom" can be moved to the top

[local:midas:S]/>move custom top
[local:midas:S]/>ls
Custom
System
Programs
Experiment
Runinfo
Alarms

or to any position, e.g.

[local:midas:Stopped]/>move custom 1
[local:midas:Stopped]/>ls
System
Custom
Programs
Experiment
Runinfo
Alarms

Create a link in the ODB

An example of creating softlinks in the odb can be found under Creating edit-on-start parameters.

Save and reload the ODB

The ODB can be saved at its current position in several formats (ascii, xml as a c structure) using the odbedit save command, and reloaded from a saved file with the odbedit load command.

Since the ODB may become corrupted, a copy is saved automatically at the beginning of each run. Since this file is overwritten each time, users often also save a copy (that will not be overwritten) at the end of each run (see ODB Dump) along with the data for that run. Users can then return their ODB to the state it was in for any particular run.

The entire database need not be loaded. Saved ASCII files can be made of just a part of the database, and these can be reloaded into the database. Since the full path is given in the saved file, the file can be loaded from any position in the database.

The saved ASCII file may of course be edited prior to loading, if keynames or values need to be changed. If the keys in the load file do not exist, they will be created. If they do exist, the values from the file will be loaded.



experim.h include file

The odbedit make command creates "experim.h", a file containing C structures which can be included into frontend and analyzer code to enable easy access to the ODB parameters (see also Frontend). The file is created in the current directory. The file experim.h contains C structures for the ODB trees /Experiment, /Analyzer/Parameters, and /Equipment/<equipment-name>/Settings if these trees are present. All C structures are accompanied with a string representation which can be used in the db_create_record function to setup an ODB structure which matches the C structure.

In order to include the analyzer section, the ODB key /<Analyzer>/Parameters has to be present, where <Analyzer> is the name of the analyzer. The command used is then "make <Analyzer> ".

Examples using experim.h can be found How to set up a Hot-Link and Frontend code.



Corrupted ODB

If the Online Database becomes corrupted, odbedit may no longer work, and other clients will also fail to open the database. In this case, the old ODB should be deleted and a new one created. The contents of the ODB can be reloaded from a (see #Save and reload the ODB|saved ODB file]].

To delete the corrupted ODB, delete the *.SHM files for your experiment.

[mpet@titan01 ~/online]$ ls .*.SHM .ALARM.SHM .ELOG.SHM .HISTORY.SHM .MSG.SHM .ODB.SHM .SYSMSG.SHM .SYSTEM.SHM [mpet@titan01 ~/online]$ rm .*.SHM

Create new *.SHM files by re-creating the ODB, then load a saved file containing the latest copy of the ODB contents.