View source for ODB Access and Use

{{Pagelinks}}
= Links =
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* [[odbedit]]
* [[Odbedit command list]]
* [[mhttpd]] ODB page
* [[Online Database|ODB structure]]
* 
</div>
= Purpose =
This page gives details on how users may access and use the ODB. 
 
= Introduction =
The MIDAS Online Data Base (ODB) is the main communication platform between MIDAS applications. It contains all the information needed to run an experiment. A description of [[Online Database#The ODB Structure|the ODB Structure]] and a list of all the ODB trees used by the system can be found in [[Online Database]].

= ODB Keys =

The ODB is hierarchically structured, similar to a file system, with directories and sub-directories (or trees and subtrees). The data are stored in key/data pairs, similar to the Windows NT registry. ODB keys can be dynamically created and deleted. The data associated with a key can be of different types such as: byte, words, double words, float, strings, etc. or arrays of any of those.

A key can also be a directory or a symbolic link. ODB Key names are '''case-independent''', and '''may contain spaces'''.
<br>

= Accessing the ODB =

The MIDAS ODB can be accessed '''interactively''' in two ways:
*  [[odbedit|odbedit application]] command line.
*  Midas server [[mhttpd]] ODB page

For non-interactive access,
* The MIDAS library provides a complete '''set of functions''' (in C/C++) to manage and operate on ODB keys (see [http://ladd00.triumf.ca/~daqweb/doc/midas-devel/doc/html/group__odbfunctionc.html odb functions]). 
Examples of many of these routines in use can be found in [[Frontend user code]].

* Scripts can easily access the odb using the [[odbedit]] ''-c'' argument. Examples of this can be found in [[#Accessing the ODB from a script|Script Access]]

* A set of Javascript functions are provided to allow the web server [[Custom Page|Custom Pages]] to access the ODB.





= Creating the ODB =

After installation of MIDAS, before any other applications are started, the ODB must be created by the user (see [[odbedit #Creating the ODB|creating the ODB ]]).  Running [[odbedit]] to create the ODB also causes the trees [[/Runinfo ODB tree|/Runinfo]], [[/Experiment ODB tree|/Experiment]], [[/System ODB tree|/System]] to be created and filled with default values. Other system applications will automatically create their own initial ODB structure
filled with default values (for example, the [[mlogger|data logger]] creates the  [[/Logger ODB tree]]. The user may then modify these configurations to fit  his/her requirements  (see [[#Customizing|customizing]]). A list of the trees in the ODB  used by the system can be found [[Online Database|here]]. 

== Shared-memory files ==
[[odbedit #Creating the ODB|Creating the ODB ]]also creates
all the shared-memory files needed for the experiment (see list below). By default, these files will be created in the area indicated in the [[exptab]] file for your experiment. 
<pre>
.ALARM.SHM  .ELOG.SHM  .HISTORY.SHM  .MSG.SHM  .ODB.SHM  .SYSMSG.SHM  .SYSTEM.SHM
</pre>

= Customizing the ODB =
The user customizes the ODB for their experiment by modifying the values of "system" parameters (e.g. [[Keys in the ODB /Logger tree]], or adding extra keys under existing trees as needed, e.g. "Edit on Start" parameters under the [[/Experiment ODB tree]].  The user may creates "optional" trees e.g. ([[/Script ODB tree]], [[/Custom ODB tree]]) populating them with keys as required.
The user is also free to create his/her own tree structure under root (/) for his/her own purposes. 

== Creating ODB keys ==

The easiest way to create, set (and delete if necessary) [[#ODB Keys|ODB keys]] is to use the [[ODB Page]] of the web server [[mhttpd]]. This also includes creating '''subdirectories''' and '''symbolic links'''. See [[ODB Page#Create an ODB key]] for details.


Alternatively, [[odbedit]] can be used as demonstrated in the following example. [[Odbedit]] is more powerful, also allowing the user to for example re-order, copy or rename  the keys (see [[odbedit command list]]).

Once a directory is created, keys can be created of the types supported by MIDAS (see [[Midas Data Types]]), e.g. INT DWORD BOOL FLOAT DOUBLE STRING.  Arrays of all these types can also be created. Values are then assigned to the keys using the {{Odbedit cmd|cmd=set}}.

=== Example ===
This example shows how to create an ODB tree (<span style="color:purple; font-style:italic">/Custom</span>) using [[odbedit]]. Creating the same tree using the [[ODB Page]] is easy (see [[ODB Page#Create an ODB key]] for more information).
 
The [[odbedit]] commands are shown in red. Comments are shown in green and preceded by a "#".
 $ odbedit   
 [local:midas:S]/>pwd    <span style="color:green; font-style:normal"># show current directory (tree) </span>
 /
 [local:midas:S]/><span style="color:red; font-style:normal">mkdir custom/images</span>    <span style="color:green; font-style:normal"># make directory "custom" and subdirectory "images"</span>
 [local:midas:S]/><span style="color:red; font-style:normal">ls</span>  <span style="color:green; font-style:normal"># list all keys</span>
 System
 Programs
 Experiment
 Runinfo
 Alarms
 Custom
 [local:midas:S]/><span style="color:red; font-style:normal">cd custom </span>  <span style="color:green; font-style:normal"># change directory to "/custom" </span>
 [local:midas:S]/custom><span style="color:red; font-style:normal">ls</span>
 images
 [local:midas:S]/custom><span style="color:red; font-style:normal">create string "custom page&"</span>  <span style="color:green; font-style:normal"># Create a key of type STRING. Note use of quotes as key name contains a space </span>
 String length [32]: 128
 [local:midas:S]/custom><span style="color:red; font-style:normal">set "custom page&" /home/midas/custom/custom.html</span>  <span style="color:green; font-style:normal"># Set key to a value. 
 #             Could type instead "set cus" followed by tab to demonstrate tab-completion </span>
 [local:midas:S]/custom><span style="color:red; font-style:normal">ls -r</span>  <span style="color:green; font-style:normal"># list recursively </span>
 custom
    images
    custom page&           /home/midas/custom/custom.html


== Creating arrays ==
Creating arrays in the ODB is simple using the [[ODB Page]] (see [[ODB Page#Create an ODB key]] for more information).

The following example demonstrates the use of [[odbedit]] to create an array, set its elements to various value, and expand it. An array can be truncated using the [[odbedit]] ''trunc'' command.
 [local:pol:S]/test>create int fred[5]
 [local:pol:S]/test>set fred[*] 5
 [local:pol:S]/test>set fred[1..3] 6
 [local:pol:S]/test>set fred[8] 9
 [local:pol:S]/test>ls
 fred
                                5
                                6
                                6
                                6
                                5
                                0
                                0
                                0
                                9


== Reordering ODB keys ==
Sometimes it may be desired to reorder the ODB keys once created. This must be done using [[odbedit]]. 

The  {{Odbedit cmd|cmd=move}} is used for this purpose as in the following example:

The key "custom" can be moved to the top (or bottom) of the list, e.g.
 
 [local:midas:S]/>ls
 System
 Programs
 Experiment
 Runinfo
 Alarms
 Custom

The key "Custom" can be moved to the top

 [local:midas:S]/>move custom top
 [local:midas:S]/>ls
 Custom
 System
 Programs
 Experiment
 Runinfo
 Alarms

or to any position, e.g.

 [local:midas:Stopped]/>move custom 1
 [local:midas:Stopped]/>ls
 System
 Custom
 Programs
 Experiment
 Runinfo
 Alarms

== Create a subdirectory in the ODB ==
Creating a subdirectory is very easy to do using the [[ODB Page]]. See [[ODB Page#Create an ODB key|Create an ODB key]].

Using [[odbedit]], select the desired directory using {{Odbedit cmd|cmd=cd}} then make the directory with {{Odbedit cmd|cmd=mkdir}} e.g.

 [local:test:S]cd /Experiment
 [local:test:S]mkdir "edit on start"

== Create a symbolic link in the ODB ==
Creating a symbolic link is easy to do using the [[ODB Page]]. See [[ODB Page#Create an ODB key|Create an ODB key]].


Creating the same symbolic link with [[odbedit]]
 [local:test:S]cd /Experiment/edit on start
 [local:test:S]Edit on Start> ln "/logger/write data" "write data"
 [local:test:S]Edit on Start> ls
 write data -> /logger/write data
                                y

= Save and reload the ODB =
The ODB can be saved at its current position in several formats (ascii, xml as a c structure) using the [[odbedit]] ''save'' command, and reloaded from a saved file with the [[odbedit]] ''load'' command.

Since the ODB may become [[#Corrupted ODB|corrupted]],  a copy is saved automatically at the beginning of each run. Since this file is overwritten each time, users often also save a copy (that will not be overwritten) at the end of each run (see [[Keys in the ODB /Logger tree#ODB Dump|ODB Dump]]) along with the data for that run. (The ODB contents can also be sent to the logging channel and saved with the run data - see [[Keys in the ODB /Logger/Channels tree #ODB Dump|ODB Dump in /Logger/Channels subtree]]).

Users can then return their ODB to the state it was in for any particular run.

The entire database need not be loaded. Saved ASCII files can be made of just a part of the database, and these can be reloaded into the database. Since the full path is given in the saved file, the file can be loaded from any position in the database.

The saved ASCII file may of course be edited prior to loading, if keynames or values need to be changed. If the keys in the load file do not exist, they will be created. If they do exist, the values from the file will be loaded.

= '''experim.h''' include file =

The [[odbedit]] '''make''' command creates "''experim.h''", a file containing a C structure and ascii representation of part of the ODB. The file is created in the current directory. 
e.g.
 [local:midas:S]/>make
 Analyzer "Analyzer" not found in ODB, skipping analyzer parameters.
 "experim.h" has been written to /home/midas/online

The file ''experim.h'' contains C structures for the ODB trees <span style="color:purple; font-style:italic">/Experiment, /Analyzer/Parameters</span>, and <span style="color:purple; font-style:italic">/Equipment/<equipment-name>/Settings</span> if these trees are present.   All C structures are accompanied by a string representation which can be used in the ''db_create_record'' function to setup an ODB structure which matches the required C structure, ensuring that the required keys are present in the ODB (see
[[Frontend user code#Function begin_of_run|Frontend]] or [[Event Notification (Hot-Link)#How to set up a Hot-Link|setting up a hot link]]). 

If the name of the analyzer is not the default (i.e. "Analyzer"), then in order to include the analyzer section, the ODB key <span style="color:purple; font-style:italic">/<Analyzer-name>/Parameters</span> has to be present, where <Analyzer-name> is the name of the analyzer. 

The command used is then "make <Analyzer-name> ".

An example ''experim.h'' can be found at
[http://ladd00.triumf.ca/~daqweb/doc/midas/examples/Triumf/c/experim.h].



= Corrupted ODB =
If the ODB becomes corrupted, [[odbedit]] may no longer work, and other clients will also fail to open the database. In this case, the old ODB should be deleted and a new one created. The contents of the ODB can be reloaded from a [[#Save and reload the ODB|saved ODB file]].
Follow the instructions in [[Recovery from Corrupted ODB]].

= Accessing the ODB from a script =
This external command feature of [[odbedit]] allows for sophisticated scripts to be created that can manipulate the odb.
Such scripts can for example

*    check ODB parameters prior to beginning of run
*    send run parameters to the electronic logbook
*    set some ODB parameters when a script button on the [[mhttpd]] main status page is pressed
*    load a particular saved odb file when a button on a custom page is pressed (e.g. load a "tune") 


Shell scripts sending [[odbedit]] commands

*    script run at end-of-run or begin-of-run
*    script run when a [[/Script ODB tree#Script-button|script-button]] or [[/Customscript ODB tree#Script-button|script-button]]/Customscript button is pressed


== Example script ==
The following is part of a script run at end-of-run, where it reads some parameters from the odb and saves them in a temporary file to be sent to the elog.

To make the script run at end of run, the name of the script is entered in the "Execute on stop run" key in the The ODB /Programs tree .

 #!/bin/tcsh
 # This script is started at the end of each run. It takes some parameters
 # from the odb and creates an entry in the elog 
 .....
 # Start collecting information from ODB first
 set Run_number = `odb -e $MIDAS_EXPT_NAME -c 'ls "/Runinfo/Run number"'`
 set number = `echo $Run_number | awk '{print $3}'`
 set sample = `odb -e $MIDAS_EXPT_NAME -c 'ls "/Experiment/Edit on Start/sample"'`
 set Sample = `echo $sample | awk '{print $2}'`
 .....
 # Now create the temporary file to be sent to the elog
 echo "Run # $number" >> $fin
 odb -e $MIDAS_EXPT_NAME -c 'ls "/Runinfo/Start time"' >> $fin
 odb -e $MIDAS_EXPT_NAME -c 'ls "/Runinfo/Stop time"' >> $fin
 echo "$Sample at T = $T K, H = $H T and RF = $RF mW">> $fin
 .....




[[Category:ODB]]