MidasWiki - User contributions [en]

Mhttpd.js

2026-03-14T00:41:45Z

Lindner: /* About */

{{Pagelinks}}
= Links =
* [[Mhttpd|mhttpd MIDAS web server]]
* [[Custom Page]]
* [[mjsonrpc|JSON-RPC functions]]

= About =

{{File|name=mhttpd.js}} is a file containing javascript function wrappers for most MIDAS AJAX functions typically used for implementing interactive [[Custom Page]]s for MIDAS experiments.
Since January 2016, it also contains [[#JSON-RPC interface]] functions (see also [[mjsonrpc]]).

AJAX is a set of web development techniques used to construct interactive web applications:
* http://en.wikipedia.org/wiki/Ajax_(programming)

<div id="include js lib"></div>
{{File|name=mhttpd.js}} is typically served by [[mhttpd]] from {{Filepath|path=$MIDASSYS/resources/mhttpd.js}}, and is included into an html file as follows:
<pre>
<html>
<head>

...
<script src='mhttpd.js'></script>
...
</head>
...
</html>
</pre>

A mostly up to date copy of mhttpd.js is linked here:
* https://daq.triumf.ca/~daqweb/doc/midas-develop/resources/mhttpd.js
* https://bitbucket.org/tmidas/midas/src/HEAD/resources/mhttpd.js?at=develop

An example for using all mhttpd.js functions is included in the MIDAS distribution under examples/javascript1:

* https://daq.triumf.ca/~daqweb/doc/midas-develop/examples/javascript1/
* https://daq.triumf.ca/~daqweb/doc/midas-develop/examples/javascript1/example1.html

MIDAS AJAX functions can be accessed directly without using mhttpd.js wrappers (i.e. using the JSON-P script-tag method)

* [[mjsonrpc]] - MIDAS JSON-RPC functions
* [[AJAX]] - MIDAS AJAX functions

By default, MIDAS AJAX functions assume that the web page was loaded from mhttpd and use relative URLs to access AJAX functions of the same mhttpd. If the web page is loaded from some other source, or needs to talk to a different mhttpd, use the ODBSetURL() function to set the base URL of the target mhttpd. Such cross-site access is normally blocked by web browsers, but recent versions of mhttpd implement the "cross-origin resource sharing" http header to permit such access.

= JSON-RPC interface =

The original AJAX interface to MIDAS is being replaced by a JSON-RPC based interface.

The Javascript functions for use with MIDAS will continue to be documented on this page.

For additional information about the JSON-RPC interface, see [[mjsonrpc]]

= Functions =

This is a partial list of available javascript functions.

For the full list, and for the full documentation, please go to the doxygen-generated documentation at https://daq.triumf.ca/~daqweb/doc/midas-devel/html/mhttpd_8js.html

Functions to access the MIDAS JSON-RPC API:

* function mjsonrpc_set_url (url) - set the URL of the MIDAS JSON-RPC server, if different from web page URL. Cross-site access is fully supported (see CORS).
* function mjsonrpc_call (method, params, id) - call arbitrary RPC method
* function mjsonrpc_start_program (name, id, callback, error_callback)
* function mjsonrpc_stop_program (name, unique, id, callback, error_callback)
* function mjsonrpc_db_get_values (paths, id, callback, error_callback) - read ODB values
* function mjsonrpc_db_paste (paths, values, id, callback, error_callback) - write ODB values

Functions listed below will be replaced by JSON-RPC functions and will become obsolete:

Helper functions:

* function getMouseXY(e) --- removed Feb 2014
* function XMLHttpRequestGeneric()
* function ODBExtractRecord(record, key)
* function ODBEdit(path)
* function ODBFinishInlineEdit(...)
* function ODBInlineEditKeydown(...)
* function ODBInlineEdit(...)
* function ODBCall(...)

ODB access functions:

* function ODBSetURL(url_of_mhttpd) - set the URL of target mhttpd (if web page is loaded from different source)
* function ODBSet(path, value, pwdname) - write into ODB
* function ODBMCopy(paths, callback, format) - read multiple ODB subtrees in ODB, XML or JSON format, asynchronously
* function ODBMLs(paths, callback) - return ODB directory listings (same as odbedit ls -l)

Obsolete ODB access functions replaced by ODBMCopy():

* function ODBGet(path, format, defval, len, type) - read from ODB
* function ODBMGet(paths, callback, formats) - read multiple ODB entries
* function ODBGetRecord(path) - read from ODB
* function ODBKey(path) - get the description of an ODB entry --- replaced by ODBMKey()
* function ODBCopy(path, format) - read an ODB subtree in ODB, XML or JSON format

ODB entries creation and modification:

* function ODBMCreate(paths, types, arraylengths, stringlengths, callback) - create multiple ODB entries using db_create_key(path, type)
* function ODBMResize(paths, arraylengths, stringlengths, callback) - resize multiple ODB arrays, including element size of strings and string arrays
* function ODBMKey(paths, callback) - read multiple ODB KEYs
* function ODBMDelete(paths, callback) - delete multiple ODB entries using db_delete_key(path)
* function ODBMRename(paths, names, callback) - rename multiple ODB entries using db_rename_key(path, name)
* function ODBMLink(paths, links, callback) - create symbolic links using db_create_link(link, path)
* function ODBMReorder(paths, indices, callback) - reorder ODB entries using db_reorder_key(path, index)

MIDAS access functions:

* function ODBRpc_rev0(name, rpc, args) --- replaced by ODBRpc()
* function ODBRpc_rev1(name, rpc, max_reply_length, args) --- replaced by ODBRpc()
* function ODBRpc(name, command, args, callback, max_reply_length)
* **function ODBGetMsg(facility,start,number) - get midas messages from midas.log
* function ODBGenerateMsg(m) - write a midas message
* function ODBGetAlarms() - get list of active alarms

;Note
: ** marks functions whose parameters have changed recently (August 2015)

= Summary table =

;Notes
: "*" marks obsoleted functions. Please avoid using them for new designs.
: "**" marks functions whose parameters have changed recently (August 2015)

{| class="wikitable"
|+ Summary of mhttpd.js functions
! Function !! XML !! JSON !! JSON-P !! async !! AJAX !! URI encoding !! replaced by (AJAX) !! replaced by (mjsonrpc)
|-
| *ODBSet(path, value, pwdname) || no || no || no || no || [[AJAX#jset|cmd=jset]] || by caller, except for "value" || ||mjsonrpc_db_paste
|-
| *ODBGet(path, format, defval, len, type) || no || no || no || no || [[AJAX#jget|cmd=jget]] || by caller, except for "format"||ODBMCopy ||mjsonrpc_db_get_values
|-
| *ODBMGet(paths, callback, formats) || no || no || no || yes || [[AJAX#jget|cmd=jget]] || by caller, including "formats"||ODBMCopy ||mjsonrpc_db_get_values
|-
| *ODBGetRecord(path) || no || no || no || no || [[AJAX#jget|cmd=jget]] with "name=1" || by caller||ODBMCopy ||mjsonrpc_db_get_values
|-
| *ODBKey(path) || no || yes || no || no || [[AJAX#jkey|cmd=jkey]] || by caller||ODBMKey|| mjsonrpc_db_ls
|-
| *ODBCopy(path, format) || yes || yes || yes || no || [[AJAX#jcopy|cmd=jcopy]] || by caller, except for "format"||ODBMCopy ||mjsonrpc_db_copy
|-
| ODBMCopy(paths, callback, format) || yes || yes || yes || yes || [[AJAX#jcopy|cmd=jcopy]] || internal || ||mjsonrpc_db_copy
|-
| ODBMLs(paths, callback) || no || yes || yes || yes || [[AJAX#jcopy|cmd=jcopy]] || internal || ||mjsonrpc_db_ls
|-
| ODBMCreate(paths, types, arraylengths, stringlengths, callback) || no || yes || yes || yes || [[AJAX#jcreate|cmd=jcreate]] || internal || ||mjsonrpc_db_create
|-
| ODBMResize(paths, arraylengths, stringlengths, callback) || no || yes || yes || yes || [[AJAX#jresize|cmd=jresize]] || internal|| ||mjsonrpc_db_resize
|-
| ODBMRename(paths, names, callback) || no || yes || yes || yes || [[AJAX#jrename|cmd=jrename]] || internal || ||
|-
| ODBMKey(paths, callback) || no || yes || yes || yes || [[AJAX#jkey|cmd=jkey&encoding=json]] || internal || ||mjsonrpc_db_ls
|-
| ODBMDelete(paths, callback) || no || yes || yes || yes || [[AJAX#jdelete|cmd=jdelete]] || internal || ||
|-
| ODBMLink(paths, links, callback) || no || yes || yes || yes || [[AJAX#jlink|cmd=jlink]] || internal || ||
|-
| ODBMReorder(paths, indices, callback) || no || yes || yes || yes || [[AJAX#jreorder|cmd=jreorder]] || internal || ||
|-
| *ODBRpc_rev0(name, rpc, args) || no || no || no || no || cmd=jrpc_rev0 || || ODBRpc ||mjsonrpc_call
|-
| *ODBRpc_rev1(name, rpc, max_reply_length, args) || no || no || no || no || cmd=jrpc_rev1|| ODBRpc || ||mjsonrpc_call
|-
| ODBRpc(...) || yes || yes || yes || yes || cmd=jrpc || || ||mjsonrpc_call
|-
| **ODBGetMsg(n) || no || no || no || no || [[AJAX#jmsg|cmd=jmsg]] || not needed || ||
|-
| *ODBGenerateMsg(m) || no || no || no || no || [[AJAX#jgenmsg|cmd=jgenmsg]] || by caller|| ||mjsonrpc_cm_msg
|-
| ODBGetAlarms() || no || no || no || no || [[AJAX#jalm|cmd=jalm]] || - || ||
|}

= ODBSetURL =

Prototype:
* function ODBSetURL(url)

Arguments:
* url - URL of mhttpd we want to talk to.]

Returns: nothing

Example:
<pre>
ODBSetURL("http://localhost:8080/");
</pre>

= ODBSet =

See: [[AJAX#jset]]

See: http://ladd00.triumf.ca/~daqweb/doc/midas-old/html/RC_mhttpd_custom_ODB_access.html#RC_mhttpd_custom_odbset

Prototype:
* function ODBSet(path, value, pwdname)

Arguments:
* path - ODB path, must be URI-encoded
* value - value to write, must NOT be URI-encoded (encoding is done inside ODBSet())
* pwdname - see [[AJAX#jset]]

Returns: nothing

Error handling: issues an alert() if there are errors.

BUG: may call alert() and require user interaction.

= ODBGet =

See: [[AJAX#jset]]

See: http://ladd00.triumf.ca/~daqweb/doc/midas-old/html/RC_mhttpd_custom_ODB_access.html#RC_mhttpd_custom_odbget

Prototype:
* function ODBGet(path, format, defval, len, type)

Arguments:
* path - ODB path to read, must be URI-encoded
* format - sprintf format for the data, must NOT be URI-encoded
* defval - if ODB path does not exist, it is created using [[AJAX#jset]] with this value, must be URI-encoded. Disable this by using "undefined"
* len - data length for jset
* type - data type for jset

* BUG: read the BUG sections in [[AJAX#jget]]

= ODBMGet =

See: [[AJAX#jget]]

See: http://ladd00.triumf.ca/~daqweb/doc/midas-old/html/RC_mhttpd_custom_ODB_access.html#RC_mhttpd_custom_odbmget

Prototype:
* function ODBMGet(paths, callback, formats)

Arguments:
* paths - array of strings - ODB paths to read, must be URI-encoded
* callback - name of javascript function for asynchronous callback
* formats - array of strings - sprintf() format strings for each ODB path, see BUGS in [[AJAX#jget]], must by URI-encoded

Valid ODB path targets are: simple variables and arrays (specified as "/eq/foo/var/array[*]").

ODBMGet for subdirectories does not work (AJAX jget returns correct data, but incorrectly decoded by ODBMGet() javascript code).

ODBMGet for arrays specified without "[*]" is unknown.

* BUG: read the BUG sections in [[AJAX#jget]]

= ODBGetRecord =

See: [[AJAX#jget]] with parameter "name=1"

See: http://ladd00.triumf.ca/~daqweb/doc/midas-old/html/RC_mhttpd_custom_ODB_access.html#RC_mhttpd_custom_odbgetrecord

Prototype:
* function ODBGetRecord(path)

Arguments:
* path - ODB path to read, must be URI-encoded

Returns:
* request.responseText
* use "function ODBExtractRecord(record, key)" to extract ODB values from data retuned by ODBGetRecord().

Valid ODB path targets are: simple variables, array elements ("/eq/foo/var/array[0]"), arrays (specified as "/eq/foo/var/array[*]") and subdirectories ("/experiment")

Usage:
<pre>
var rec1 = ODBGetRecord(encodeURIComponent("/Equipment/RpcExample/Variables"));
document.getElementById('foo').innerHTML = rec1;
document.getElementById('bar').innerHTML = ODBExtractRecord(rec1, 'SLOW[2]');
</pre>

* BUG: read the BUG sections in [[AJAX#jget]]

= ODBKey =

See: [[AJAX#jkey]]

Prototype:
* function ODBKey(path)

Arguments:
* path - ODB path, must be URI-encoded

Returns: javascript object with data fields corresponding to the data fields of MIDAS KEY (midas.h):
<pre>
{"name":"SLOW","type":"TID_DOUBLE","num_values":"3","item_size":"8","last_written":"1376633931"}
</pre>

Usage:
<pre>
var key = new ODBKey(encodeURIComponent("/Equipment/RpcExample/Variables/SLOW"));
document.getElementById('foo').innerHTML = "name: " + key.name + " type: " + key.type;
</pre>

= ODBCopy =

See: [[AJAX#jcopy]]

Prototype:
* function ODBCopy(path, format)

Arguments:
* path - ODB path, must be URI-encoded
* format - data format (same as "encoding"), must NOT be URI-encoded

Returns: request.responseText, use JSON.parse() to decode JSON data.

Usage:
<pre>
var data_json = ODBCopy("/Equipment/RpcExample/Variables", "json");
var obj = JSON.parse(data_json);
document.getElementById('json_data0').innerHTML = obj.SLOW[0];
</pre>

Return value:
<pre>
{
"SLOW/key" : { "type" : 10, "num_values" : 3, "last_written" : 1376634320 },
"SLOW" : [ 2357, 1376634320, -8.6602540236010100e+01 ]
}
</pre>

= ODBMCopy =

See: [[AJAX#jcopy]]

Prototype:
* function ODBMCopy(paths, callback, encoding)

Arguments:
* paths - array of strings - odb paths to read
* callback - name of javascript function for asynchronous callback, or "undefined" for synchronous return
* encoding - requested data encoding - ODB, XML or JSON, see [[AJAX#jcopy]]

Returns:
* synchronous: returns request.responseText, use JSON.parse() to decode JSON data
* asynchronous: calls the callback function with a single argument set to request.responseText, use JSON.parse() to decode JSON data

Usage:
<pre>
var paths = [
"/Equipment/deapvme01/Settings/EnableControl",
"/Equipment/deapvme01/Settings/mainSwitch",
"/Equipment/deapvme01/Readback/sysMainSwitch.0",
"/Equipment/deapvme01/Variables",
"/Equipment/deapvme02/Settings/EnableControl",
"/Equipment/deapvme02/Settings/mainSwitch",
"/Equipment/deapvme02/Readback/sysMainSwitch.0",
"/Equipment/deapvme02/Variables",
"/Equipment/deapvme03/Settings/EnableControl",
"/Equipment/deapvme03/Settings/mainSwitch",
"/Equipment/deapvme03/Readback/sysMainSwitch.0",
"/Equipment/deapvme03/Variables",
];

ODBMCopy(paths, load_callback, "json");
...
function load_callback(data)
{
var obj = JSON.parse(data);
document.getElementById('enableControl1').innerHTML = obj[0].EnableControl;
document.getElementById('mainSwitch1').innerHTML = obj[1]['mainSwitch'] + " / " + obj[2]['sysMainSwitch.0'];
document.getElementById('p5v1').innerHTML = obj[3].current[0];
...
</pre>

JSON data returned from ODBMCopy():

<pre>
[ { "EnableControl/key" : { "type" : 7, "last_written" : 1375145722 }, "EnableControl" : 1 } , { "mainSwitch/key" : { "type" : 7, "last_written" : 1376610870 }, "mainSwitch" : 1 } , { "sysMainSwitch.0/key" : { "type" : 7, "last_written" : 1376681747 }, "sysMainSwitch.0" : 1 } , { "switch/key" : { "type" : 7, "num_values" : 3, "last_written" : 1376681747 }, "switch" : [ 1, 1, 1 ], "status/key" : { "type" : 7, "num_values" : 3, "last_written" : 1376681747 }, "status" : [ 1, 1, 1 ], "demandVoltage/key" : { "type" : 9, "num_values" : 3, "last_written" : 1376681747 }, "demandVoltage" : [ 5, 12, 12 ], "senseVoltage/key" : { "type" : 9, "num_values" : 3, "last_written" : 1376681747 }, "senseVoltage" : [ 5.0500002e+00, 1.1980000e+01, 1.1990000e+01 ], "current/key" : { "type" : 9, "num_values" : 3, "last_written" : 1376681747 }, "current" : [ 9.0229996e+01, 4.2700000e+00, 5.9899998e+00 ], "sparkCount/key" : { "type" : 7, "num_values" : 3, "last_written" : 1376681747 }, "sparkCount" : [ 0, 0, 0 ], "sensorTemperature/key" : { "type" : 7, "num_values" : 8, "last_written" : 1376681747 }, "sensorTemperature" : [ -128, -128, 25, -128, -128, 25, -128, -128 ], "fanSpeed/key" : { "type" : 7, "num_values" : 3, "last_written" : 1376681747 }, "fanSpeed" : [ 3180, 3112, 3236 ] } , { "EnableControl/key" : { "type" : 7, "last_written" : 1375145722 }, "EnableControl" : 1 } , { "mainSwitch/key" : { "type" : 7, "last_written" : 1376603253 }, "mainSwitch" : 1 } , { "sysMainSwitch.0/key" : { "type" : 7, "last_written" : 1376681750 }, "sysMainSwitch.0" : 1 } , { "switch/key" : { "type" : 7, "num_values" : 3, "last_written" : 1376681750 }, "switch" : [ 1, 1, 1 ], "status/key" : { "type" : 7, "num_values" : 3, "last_written" : 1376681750 }, "status" : [ 1, 1, 1 ], "demandVoltage/key" : { "type" : 9, "num_values" : 3, "last_written" : 1376681750 }, "demandVoltage" : [ 5, 12, 12 ], "senseVoltage/key" : { "type" : 9, "num_values" : 3, "last_written" : 1376681750 }, "senseVoltage" : [ 5, 1.1950000e+01, 1.1980000e+01 ], "current/key" : { "type" : 9, "num_values" : 3, "last_written" : 1376681750 }, "current" : [ 1.4169999e+01, 1.1000000e+00, 8.0000001e-01 ], "sparkCount/key" : { "type" : 7, "num_values" : 3, "last_written" : 1376681750 }, "sparkCount" : [ 0, 0, 0 ], "sensorTemperature/key" : { "type" : 7, "num_values" : 8, "last_written" : 1376681750 }, "sensorTemperature" : [ -128, -128, -128, -128, -128, -128, -128, 22 ], "fanSpeed/key" : { "type" : 7, "num_values" : 3, "last_written" : 1376681750 }, "fanSpeed" : [ 3217, 3225, 3213 ] } , { "EnableControl/key" : { "type" : 7, "last_written" : 1375145722 }, "EnableControl" : 1 } , { "mainSwitch/key" : { "type" : 7, "last_written" : 1376596430 }, "mainSwitch" : 1 } , { "sysMainSwitch.0/key" : { "type" : 7, "last_written" : 1376681752 }, "sysMainSwitch.0" : 1 } , { "switch/key" : { "type" : 7, "num_values" : 3, "last_written" : 1376681752 }, "switch" : [ 1, 1, 1 ], "status/key" : { "type" : 7, "num_values" : 3, "last_written" : 1376681752 }, "status" : [ 1, 1, 1 ], "demandVoltage/key" : { "type" : 9, "num_values" : 3, "last_written" : 1376681752 }, "demandVoltage" : [ 5, 12, 12 ], "senseVoltage/key" : { "type" : 9, "num_values" : 3, "last_written" : 1376681752 }, "senseVoltage" : [ 4.9800000e+00, 1.2009999e+01, 1.1969999e+01 ], "current/key" : { "type" : 9, "num_values" : 3, "last_written" : 1376681752 }, "current" : [ 9.1139999e+01, 4.1999998e+00, 6.0300002e+00 ], "sparkCount/key" : { "type" : 7, "num_values" : 3, "last_written" : 1376681752 }, "sparkCount" : [ 0, 0, 0 ], "sensorTemperature/key" : { "type" : 7, "num_values" : 8, "last_written" : 1376681752 }, "sensorTemperature" : [ -128, -128, 24, -128, -128, 27, -128, -128 ], "fanSpeed/key" : { "type" : 7, "num_values" : 3, "last_written" : 1376681752 }, "fanSpeed" : [ 3153, 3067, 3180 ] } ]
</pre>

= ODBMLs =

Description: return ODB directory listing with full ODB KEY information (same as odbedit ls -l). It uses the jcopy AJAX method with the JSON "norecurse" option. Same as with odbedit "ls -l", symlinks are followed and the linked-to values are reported, with the symlink path reported as "link" entry in the ODB KEY ("xxx/key") section.

See: [[AJAX#jcopy]]

Prototype:
* function ODBMLs(paths, callback)

Arguments:
* paths - array of strings - odb paths to report
* callback - name of javascript function for asynchronous callback, or "undefined" for synchronous return

Returns:
* array of JSON encoded ODB entries
* synchronous: returns request.responseText, use JSON.parse() to decode JSON data
* asynchronous: calls the callback function with a single argument set to request.responseText, use JSON.parse() to decode JSON data

Usage:
<pre>
var paths = [ "/", "/Test/noexistant", "/Experiment" ];
var data = ODBMLs(paths);
var obj = JSON.Parse(data);
</pre>

= ODBMCreate =

See: [[AJAX#jcreate]]

Prototype:
* function ODBMCreate(paths, types, arraylengths, stringlengths, callback)

Arguments:
* paths - array of strings - odb paths to create
* types - array of integers - midas data types of created keys (TID_INT, etc)
* arraylengths - array of integers - "undefined" - or array size to create
* stringlengths - array of integers - "undefined" - or array of lengths of strings to create
* callback - name of javascript function for asynchronous callback, or "undefined" for synchronous return

Returns:
* array of integers: status of db_create_key() for each element of "paths"
* synchronous: returns request.responseText, use JSON.parse() to decode JSON data
* asynchronous: calls the callback function with a single argument set to request.responseText, use JSON.parse() to decode JSON data

Usage:
<pre>
var paths = [ "/Test/foo", "/Test/noexistant", "/Test/bar" ];
var types = [ TID_INT, 0, TID_STRING ];
ODBMCreate(paths, types);
</pre>

= ODBMResize =

See: [[AJAX#jresize]]

Prototype:
* function ODBMResize(paths, arraylengths, stringlengths, callback)

Arguments:
* paths - array of strings - odb paths to resize
* arraylengths - array of integers - new array size, use "0" for "no change"
* stringlengths - array of integers - new string length or new element length for string arrays, use "0" for "no change"
* callback - name of javascript function for asynchronous callback, or "undefined" for synchronous return

Returns:
* array of integers: status of db_set_num_values() for each element of "paths"
* synchronous: returns request.responseText, use JSON.parse() to decode JSON data
* asynchronous: calls the callback function with a single argument set to request.responseText, use JSON.parse() to decode JSON data

Usage:
<pre>
var paths = [ "/Test/foo", "/Test/noexistant", "/Test/bar" ];
var arraylengths = [ 10, 0, 15 ];
var stringarrays = [ 0, 0, 16 ];
ODBMResize(paths, arraylengths, stringlengths);
</pre>

= ODBMKey =

See: [[AJAX#jkey]]

Prototype:
* function ODBMKey(paths, callback)

Arguments:
* paths - array of strings - odb paths to read ODB KEYs
* callback - name of javascript function for asynchronous callback, or "undefined" for synchronous return

Returns:
* array of ODB KEYs
* synchronous: returns request.responseText, use JSON.parse() to decode JSON data
* asynchronous: calls the callback function with a single argument set to request.responseText, use JSON.parse() to decode JSON data

Usage:
<pre>
var paths = [ "/Test/foo", "/Test/noexistant", "/Test/bar" ];
var data = JSON.parse(ODBMKey(paths));
</pre>

Example:
<pre>
var paths = [ "/Test/foo", "/Test/noexistant", "/Test/bar", "/Test/foofoo", "/Test/barlink", "/Test/foo10", "/Test/bar15" ];
var data = ODBMKey(paths);
xprint data;
[ { "name":"foo","type":7,"type_name":"TID_INT","num_values":1,"item_size":4,"last_written":0 }, { "/error":312 }, { "name":"bar","type":12,"type_name":"TID_STRING","num_values":1,"item_size":0,"last_written":0 }, { "/error":312 }, { "name":"bar","type":12,"type_name":"TID_STRING","num_values":1,"item_size":0,"last_written":0 }, { "name":"foo10","type":7,"type_name":"TID_INT","num_values":5,"item_size":4,"last_written":1380134916 }, { "name":"bar15","type":12,"type_name":"TID_STRING","num_values":7,"item_size":12,"last_written":1380134916 } ]
</pre>

= ODBMDelete =

See: [[AJAX#jdelete]]

Prototype:
* function ODBMDelete(paths, callback)

Arguments:
* paths - array of strings - odb paths to delete
* callback - name of javascript function for asynchronous callback, or "undefined" for synchronous return

Returns:
* array of integer: status of db_delete_key() for each element of "paths"
* synchronous: returns request.responseText, use JSON.parse() to decode JSON data
* asynchronous: calls the callback function with a single argument set to request.responseText, use JSON.parse() to decode JSON data

Usage:
<pre>
var paths = [ "/Test/foo", "/Test/noexistant", "/Test/bar" ];
ODBMDelete(paths);
</pre>

= ODBMRename =

See: [[AJAX#jrename]]

Prototype:
* function ODBMRename(paths, names, callback)

Arguments:
* paths - array of strings - odb paths to rename
* names - array of strings - new names (subject to ODB limitation on key names)
* callback - name of javascript function for asynchronous callback, or "undefined" for synchronous return

Returns:
* array of integer: status of db_rename_key() for each element of "paths"
* synchronous: returns request.responseText, use JSON.parse() to decode JSON data
* asynchronous: calls the callback function with a single argument set to request.responseText, use JSON.parse() to decode JSON data

Usage:
<pre>
var paths = [ "/Test/foo", "/Test/noexistant", "/Test/bar" ];
var names = [ "fooxxx", "zzz", "barxxx" ];
ODBMRename(paths, names);
</pre>

= ODBMLink =

See: [[AJAX#jlink]]

Prototype:
* function ODBMLink(paths, links, callback)

Arguments:
* paths - array of strings - odb paths create as symlinks
* links - array of strings - odb paths of existing odb entries that we link to
* callback - name of javascript function for asynchronous callback, or "undefined" for synchronous return

Returns:
* array of integer: status of db_create_link() for each element of "paths"
* synchronous: returns request.responseText, use JSON.parse() to decode JSON data
* asynchronous: calls the callback function with a single argument set to request.responseText, use JSON.parse() to decode JSON data

Usage:
<pre>
var paths = [ "/Test/foolink", "/Test/barlink" ];
var links = [ "/Test/foo", "/Test/bar" ];
ODBMLink(paths, links);
</pre>

= ODBGetMsg =
Gets message(s) from the MIDAS [[Message System]] or from the [[Chat Page|chat]].

The parameters of this command have been changed August 2015. See [[#odbgetmsg_old|below]] for older version.

Prototype:

* function ODBGetMsg(facility, start, number)

Arguments:

* facility - message facility in which messages are written
: facility can be "midas" for normal messages or "chat" to load the chat messages
* start - start time (0=latest message)
: the absolute time from when to retrieve messages. Time is defined in unix standard time, i.e. seconds since 1.1.1970. A value of zero has a special meaning of now.
* number - number of midas messages to return

Returns: array of strings, if number==1, a single string.

Usage:

Retrieve the last 100 messages:
var data = ODBGetMsg("midas",0,100);
document.getElementById("foo").innerHTML = data.join("\n");
Each message returned in a separate line in this code contains its UNIX time in the first position, e.g.
1449015258 01:14:18.006 2015/12/02 [SC Frontend,TALK] General alarm: COBRA SC quenched
Now the UNIX time stamp can be saved in the JavaScript code on the page, e.g.
last_tstamp = 1449015258
and in another call we can retrieve the additional 100 message which occurred BEFORE the previous block, i.e.
ODBGetMsg(midas, last_tstamp-1, 100)
The time stamping ensures that no message is lost between the two calls, even if the second call is made many seconds later than the first call. So start
means start searching for messages at the given UNIX timestamp, and go backwards for n messages.

The function ODBGetMsg returns an array of strings, with one entry per message. This way they can be easily parsed in JavaScript:
var msg = ODBGetMsg(midas, 0, 100);
for (i=0 ; i<msg.length ; i++)
alert(msg[i]);

To see how individual messages are parsed and extracted see function ''msg_append(msg)'' in [[Mhttpd.js|MIDAS Javascript Library]].

<div style="font-size:90%; background-color:whitesmoke">
<div id="odbgetmsg_old"></div>
; ODBGetMsg (pre August 2015)
See: [[AJAX#jgenmsg]]
Prototype:
* function ODBGetMsg(n)

Arguments:
* n - number of midas messages to return

Returns: array of strings, if n==1, as single string.

Usage:
var data = ODBGetMsg(10);
document.getElementById("foo").innerHTML = data.join("\n");

</div>

= ODBGenMsg =
Send a message to the MIDAS [[Message System]] or the [[Chat Page|Chat]].

The parameters of this command have been changed August 2015. See [[#odbgenmsg_old|below]] for older version.

Prototype:

* function ODBGenMsg(type, facility, user, msg)

Arguments:
* type - midas message type, numerical value of one of MT_ERROR, MT_INFO, MT_DEBUG, MT_USER, as defined in midas.h.
: i.e. 1 for MT_ERROR, 2 for MT_INFO etc.
* facility - facility can be "midas" for the MIDAS message system, or "chat" for the chat message system
* user - name of the user who generated the message
: The user name ends up in the message file inside the [] section to tag each message with an associated user name.
* message - text written into midas messages

Usage:
ODBGenerateMsg(2, midas, fred, "message text");

<div style="font-size:90%; background-color:whitesmoke">
<div id="odbgenmsg_old"></div>
; ODBGenerateMsg (pre August 2015)

See: [[AJAX#jgenmsg]]

Prototype:
* function ODBGenerateMsg(m)

Arguments:
* m - text written into midas messages, must be URI-encoded.

Usage:
<pre>
ODBGenerateMsg(encodeURIComponent("message text"));
</pre>
</div>

= ODBGetAlarms =

See: [[AJAX#jalm]]

Prototype:
* function ODBGetAlarms()

Returns: array of strings

Usage:
<pre>
var data = ODBGetAlarms();
document.getElementById("foo").innerHTML = data.join("\n");
</pre>

[[Category:Custom]] [[Category:Javascript library]] [[Category:Ajax]]

Mhttpd.js

2026-03-14T00:41:20Z

Lindner: /* About */

{{Pagelinks}}
= Links =
* [[Mhttpd|mhttpd MIDAS web server]]
* [[Custom Page]]
* [[mjsonrpc|JSON-RPC functions]]

= About =

{{File|name=mhttpd.js}} is a file containing javascript function wrappers for most MIDAS AJAX functions typically used for implementing interactive [[Custom Page]]s for MIDAS experiments.
Since January 2016, it also contains [[#JSON-RPC interface]] functions (see also [[mjsonrpc]]).

AJAX is a set of web development techniques used to construct interactive web applications:
* http://en.wikipedia.org/wiki/Ajax_(programming)

<div id="include js lib"></div>
{{File|name=mhttpd.js}} is typically served by [[mhttpd]] from {{Filepath|path=$MIDASSYS/resources/mhttpd.js}}, and is included into an html file as follows:
<pre>
<html>
<head>

...
<script src='mhttpd.js'></script>
...
</head>
...
</html>
</pre>

A mostly up to date copy of mhttpd.js is linked here:
* https://daq.triumf.ca/~daqweb/doc/midas-develop/resources/mhttpd.js
* https://bitbucket.org/tmidas/midas/src/HEAD/resources/mhttpd.js?at=develop

An example for using all mhttpd.js functions is included in the MIDAS distribution under examples/javascript1:

* https://daq.triumf.ca/~daqweb/doc/midas-develop/examples/javascript1/
* https://daq.triumf.ca/~daqweb/doc/midas-develop/examples/javascript1/example.html

MIDAS AJAX functions can be accessed directly without using mhttpd.js wrappers (i.e. using the JSON-P script-tag method)

* [[mjsonrpc]] - MIDAS JSON-RPC functions
* [[AJAX]] - MIDAS AJAX functions

By default, MIDAS AJAX functions assume that the web page was loaded from mhttpd and use relative URLs to access AJAX functions of the same mhttpd. If the web page is loaded from some other source, or needs to talk to a different mhttpd, use the ODBSetURL() function to set the base URL of the target mhttpd. Such cross-site access is normally blocked by web browsers, but recent versions of mhttpd implement the "cross-origin resource sharing" http header to permit such access.

= JSON-RPC interface =

The original AJAX interface to MIDAS is being replaced by a JSON-RPC based interface.

The Javascript functions for use with MIDAS will continue to be documented on this page.

For additional information about the JSON-RPC interface, see [[mjsonrpc]]

= Functions =

This is a partial list of available javascript functions.

For the full list, and for the full documentation, please go to the doxygen-generated documentation at https://daq.triumf.ca/~daqweb/doc/midas-devel/html/mhttpd_8js.html

Functions to access the MIDAS JSON-RPC API:

* function mjsonrpc_set_url (url) - set the URL of the MIDAS JSON-RPC server, if different from web page URL. Cross-site access is fully supported (see CORS).
* function mjsonrpc_call (method, params, id) - call arbitrary RPC method
* function mjsonrpc_start_program (name, id, callback, error_callback)
* function mjsonrpc_stop_program (name, unique, id, callback, error_callback)
* function mjsonrpc_db_get_values (paths, id, callback, error_callback) - read ODB values
* function mjsonrpc_db_paste (paths, values, id, callback, error_callback) - write ODB values

Functions listed below will be replaced by JSON-RPC functions and will become obsolete:

Helper functions:

* function getMouseXY(e) --- removed Feb 2014
* function XMLHttpRequestGeneric()
* function ODBExtractRecord(record, key)
* function ODBEdit(path)
* function ODBFinishInlineEdit(...)
* function ODBInlineEditKeydown(...)
* function ODBInlineEdit(...)
* function ODBCall(...)

ODB access functions:

* function ODBSetURL(url_of_mhttpd) - set the URL of target mhttpd (if web page is loaded from different source)
* function ODBSet(path, value, pwdname) - write into ODB
* function ODBMCopy(paths, callback, format) - read multiple ODB subtrees in ODB, XML or JSON format, asynchronously
* function ODBMLs(paths, callback) - return ODB directory listings (same as odbedit ls -l)

Obsolete ODB access functions replaced by ODBMCopy():

* function ODBGet(path, format, defval, len, type) - read from ODB
* function ODBMGet(paths, callback, formats) - read multiple ODB entries
* function ODBGetRecord(path) - read from ODB
* function ODBKey(path) - get the description of an ODB entry --- replaced by ODBMKey()
* function ODBCopy(path, format) - read an ODB subtree in ODB, XML or JSON format

ODB entries creation and modification:

* function ODBMCreate(paths, types, arraylengths, stringlengths, callback) - create multiple ODB entries using db_create_key(path, type)
* function ODBMResize(paths, arraylengths, stringlengths, callback) - resize multiple ODB arrays, including element size of strings and string arrays
* function ODBMKey(paths, callback) - read multiple ODB KEYs
* function ODBMDelete(paths, callback) - delete multiple ODB entries using db_delete_key(path)
* function ODBMRename(paths, names, callback) - rename multiple ODB entries using db_rename_key(path, name)
* function ODBMLink(paths, links, callback) - create symbolic links using db_create_link(link, path)
* function ODBMReorder(paths, indices, callback) - reorder ODB entries using db_reorder_key(path, index)

MIDAS access functions:

* function ODBRpc_rev0(name, rpc, args) --- replaced by ODBRpc()
* function ODBRpc_rev1(name, rpc, max_reply_length, args) --- replaced by ODBRpc()
* function ODBRpc(name, command, args, callback, max_reply_length)
* **function ODBGetMsg(facility,start,number) - get midas messages from midas.log
* function ODBGenerateMsg(m) - write a midas message
* function ODBGetAlarms() - get list of active alarms

;Note
: ** marks functions whose parameters have changed recently (August 2015)

= Summary table =

;Notes
: "*" marks obsoleted functions. Please avoid using them for new designs.
: "**" marks functions whose parameters have changed recently (August 2015)

{| class="wikitable"
|+ Summary of mhttpd.js functions
! Function !! XML !! JSON !! JSON-P !! async !! AJAX !! URI encoding !! replaced by (AJAX) !! replaced by (mjsonrpc)
|-
| *ODBSet(path, value, pwdname) || no || no || no || no || [[AJAX#jset|cmd=jset]] || by caller, except for "value" || ||mjsonrpc_db_paste
|-
| *ODBGet(path, format, defval, len, type) || no || no || no || no || [[AJAX#jget|cmd=jget]] || by caller, except for "format"||ODBMCopy ||mjsonrpc_db_get_values
|-
| *ODBMGet(paths, callback, formats) || no || no || no || yes || [[AJAX#jget|cmd=jget]] || by caller, including "formats"||ODBMCopy ||mjsonrpc_db_get_values
|-
| *ODBGetRecord(path) || no || no || no || no || [[AJAX#jget|cmd=jget]] with "name=1" || by caller||ODBMCopy ||mjsonrpc_db_get_values
|-
| *ODBKey(path) || no || yes || no || no || [[AJAX#jkey|cmd=jkey]] || by caller||ODBMKey|| mjsonrpc_db_ls
|-
| *ODBCopy(path, format) || yes || yes || yes || no || [[AJAX#jcopy|cmd=jcopy]] || by caller, except for "format"||ODBMCopy ||mjsonrpc_db_copy
|-
| ODBMCopy(paths, callback, format) || yes || yes || yes || yes || [[AJAX#jcopy|cmd=jcopy]] || internal || ||mjsonrpc_db_copy
|-
| ODBMLs(paths, callback) || no || yes || yes || yes || [[AJAX#jcopy|cmd=jcopy]] || internal || ||mjsonrpc_db_ls
|-
| ODBMCreate(paths, types, arraylengths, stringlengths, callback) || no || yes || yes || yes || [[AJAX#jcreate|cmd=jcreate]] || internal || ||mjsonrpc_db_create
|-
| ODBMResize(paths, arraylengths, stringlengths, callback) || no || yes || yes || yes || [[AJAX#jresize|cmd=jresize]] || internal|| ||mjsonrpc_db_resize
|-
| ODBMRename(paths, names, callback) || no || yes || yes || yes || [[AJAX#jrename|cmd=jrename]] || internal || ||
|-
| ODBMKey(paths, callback) || no || yes || yes || yes || [[AJAX#jkey|cmd=jkey&encoding=json]] || internal || ||mjsonrpc_db_ls
|-
| ODBMDelete(paths, callback) || no || yes || yes || yes || [[AJAX#jdelete|cmd=jdelete]] || internal || ||
|-
| ODBMLink(paths, links, callback) || no || yes || yes || yes || [[AJAX#jlink|cmd=jlink]] || internal || ||
|-
| ODBMReorder(paths, indices, callback) || no || yes || yes || yes || [[AJAX#jreorder|cmd=jreorder]] || internal || ||
|-
| *ODBRpc_rev0(name, rpc, args) || no || no || no || no || cmd=jrpc_rev0 || || ODBRpc ||mjsonrpc_call
|-
| *ODBRpc_rev1(name, rpc, max_reply_length, args) || no || no || no || no || cmd=jrpc_rev1|| ODBRpc || ||mjsonrpc_call
|-
| ODBRpc(...) || yes || yes || yes || yes || cmd=jrpc || || ||mjsonrpc_call
|-
| **ODBGetMsg(n) || no || no || no || no || [[AJAX#jmsg|cmd=jmsg]] || not needed || ||
|-
| *ODBGenerateMsg(m) || no || no || no || no || [[AJAX#jgenmsg|cmd=jgenmsg]] || by caller|| ||mjsonrpc_cm_msg
|-
| ODBGetAlarms() || no || no || no || no || [[AJAX#jalm|cmd=jalm]] || - || ||
|}

= ODBSetURL =

Prototype:
* function ODBSetURL(url)

Arguments:
* url - URL of mhttpd we want to talk to.]

Returns: nothing

Example:
<pre>
ODBSetURL("http://localhost:8080/");
</pre>

= ODBSet =

See: [[AJAX#jset]]

See: http://ladd00.triumf.ca/~daqweb/doc/midas-old/html/RC_mhttpd_custom_ODB_access.html#RC_mhttpd_custom_odbset

Prototype:
* function ODBSet(path, value, pwdname)

Arguments:
* path - ODB path, must be URI-encoded
* value - value to write, must NOT be URI-encoded (encoding is done inside ODBSet())
* pwdname - see [[AJAX#jset]]

Returns: nothing

Error handling: issues an alert() if there are errors.

BUG: may call alert() and require user interaction.

= ODBGet =

See: [[AJAX#jset]]

See: http://ladd00.triumf.ca/~daqweb/doc/midas-old/html/RC_mhttpd_custom_ODB_access.html#RC_mhttpd_custom_odbget

Prototype:
* function ODBGet(path, format, defval, len, type)

Arguments:
* path - ODB path to read, must be URI-encoded
* format - sprintf format for the data, must NOT be URI-encoded
* defval - if ODB path does not exist, it is created using [[AJAX#jset]] with this value, must be URI-encoded. Disable this by using "undefined"
* len - data length for jset
* type - data type for jset

* BUG: read the BUG sections in [[AJAX#jget]]

= ODBMGet =

See: [[AJAX#jget]]

See: http://ladd00.triumf.ca/~daqweb/doc/midas-old/html/RC_mhttpd_custom_ODB_access.html#RC_mhttpd_custom_odbmget

Prototype:
* function ODBMGet(paths, callback, formats)

Arguments:
* paths - array of strings - ODB paths to read, must be URI-encoded
* callback - name of javascript function for asynchronous callback
* formats - array of strings - sprintf() format strings for each ODB path, see BUGS in [[AJAX#jget]], must by URI-encoded

Valid ODB path targets are: simple variables and arrays (specified as "/eq/foo/var/array[*]").

ODBMGet for subdirectories does not work (AJAX jget returns correct data, but incorrectly decoded by ODBMGet() javascript code).

ODBMGet for arrays specified without "[*]" is unknown.

* BUG: read the BUG sections in [[AJAX#jget]]

= ODBGetRecord =

See: [[AJAX#jget]] with parameter "name=1"

See: http://ladd00.triumf.ca/~daqweb/doc/midas-old/html/RC_mhttpd_custom_ODB_access.html#RC_mhttpd_custom_odbgetrecord

Prototype:
* function ODBGetRecord(path)

Arguments:
* path - ODB path to read, must be URI-encoded

Returns:
* request.responseText
* use "function ODBExtractRecord(record, key)" to extract ODB values from data retuned by ODBGetRecord().

Valid ODB path targets are: simple variables, array elements ("/eq/foo/var/array[0]"), arrays (specified as "/eq/foo/var/array[*]") and subdirectories ("/experiment")

Usage:
<pre>
var rec1 = ODBGetRecord(encodeURIComponent("/Equipment/RpcExample/Variables"));
document.getElementById('foo').innerHTML = rec1;
document.getElementById('bar').innerHTML = ODBExtractRecord(rec1, 'SLOW[2]');
</pre>

* BUG: read the BUG sections in [[AJAX#jget]]

= ODBKey =

See: [[AJAX#jkey]]

Prototype:
* function ODBKey(path)

Arguments:
* path - ODB path, must be URI-encoded

Returns: javascript object with data fields corresponding to the data fields of MIDAS KEY (midas.h):
<pre>
{"name":"SLOW","type":"TID_DOUBLE","num_values":"3","item_size":"8","last_written":"1376633931"}
</pre>

Usage:
<pre>
var key = new ODBKey(encodeURIComponent("/Equipment/RpcExample/Variables/SLOW"));
document.getElementById('foo').innerHTML = "name: " + key.name + " type: " + key.type;
</pre>

= ODBCopy =

See: [[AJAX#jcopy]]

Prototype:
* function ODBCopy(path, format)

Arguments:
* path - ODB path, must be URI-encoded
* format - data format (same as "encoding"), must NOT be URI-encoded

Returns: request.responseText, use JSON.parse() to decode JSON data.

Usage:
<pre>
var data_json = ODBCopy("/Equipment/RpcExample/Variables", "json");
var obj = JSON.parse(data_json);
document.getElementById('json_data0').innerHTML = obj.SLOW[0];
</pre>

Return value:
<pre>
{
"SLOW/key" : { "type" : 10, "num_values" : 3, "last_written" : 1376634320 },
"SLOW" : [ 2357, 1376634320, -8.6602540236010100e+01 ]
}
</pre>

= ODBMCopy =

See: [[AJAX#jcopy]]

Prototype:
* function ODBMCopy(paths, callback, encoding)

Arguments:
* paths - array of strings - odb paths to read
* callback - name of javascript function for asynchronous callback, or "undefined" for synchronous return
* encoding - requested data encoding - ODB, XML or JSON, see [[AJAX#jcopy]]

Returns:
* synchronous: returns request.responseText, use JSON.parse() to decode JSON data
* asynchronous: calls the callback function with a single argument set to request.responseText, use JSON.parse() to decode JSON data

Usage:
<pre>
var paths = [
"/Equipment/deapvme01/Settings/EnableControl",
"/Equipment/deapvme01/Settings/mainSwitch",
"/Equipment/deapvme01/Readback/sysMainSwitch.0",
"/Equipment/deapvme01/Variables",
"/Equipment/deapvme02/Settings/EnableControl",
"/Equipment/deapvme02/Settings/mainSwitch",
"/Equipment/deapvme02/Readback/sysMainSwitch.0",
"/Equipment/deapvme02/Variables",
"/Equipment/deapvme03/Settings/EnableControl",
"/Equipment/deapvme03/Settings/mainSwitch",
"/Equipment/deapvme03/Readback/sysMainSwitch.0",
"/Equipment/deapvme03/Variables",
];

ODBMCopy(paths, load_callback, "json");
...
function load_callback(data)
{
var obj = JSON.parse(data);
document.getElementById('enableControl1').innerHTML = obj[0].EnableControl;
document.getElementById('mainSwitch1').innerHTML = obj[1]['mainSwitch'] + " / " + obj[2]['sysMainSwitch.0'];
document.getElementById('p5v1').innerHTML = obj[3].current[0];
...
</pre>

JSON data returned from ODBMCopy():

<pre>
[ { "EnableControl/key" : { "type" : 7, "last_written" : 1375145722 }, "EnableControl" : 1 } , { "mainSwitch/key" : { "type" : 7, "last_written" : 1376610870 }, "mainSwitch" : 1 } , { "sysMainSwitch.0/key" : { "type" : 7, "last_written" : 1376681747 }, "sysMainSwitch.0" : 1 } , { "switch/key" : { "type" : 7, "num_values" : 3, "last_written" : 1376681747 }, "switch" : [ 1, 1, 1 ], "status/key" : { "type" : 7, "num_values" : 3, "last_written" : 1376681747 }, "status" : [ 1, 1, 1 ], "demandVoltage/key" : { "type" : 9, "num_values" : 3, "last_written" : 1376681747 }, "demandVoltage" : [ 5, 12, 12 ], "senseVoltage/key" : { "type" : 9, "num_values" : 3, "last_written" : 1376681747 }, "senseVoltage" : [ 5.0500002e+00, 1.1980000e+01, 1.1990000e+01 ], "current/key" : { "type" : 9, "num_values" : 3, "last_written" : 1376681747 }, "current" : [ 9.0229996e+01, 4.2700000e+00, 5.9899998e+00 ], "sparkCount/key" : { "type" : 7, "num_values" : 3, "last_written" : 1376681747 }, "sparkCount" : [ 0, 0, 0 ], "sensorTemperature/key" : { "type" : 7, "num_values" : 8, "last_written" : 1376681747 }, "sensorTemperature" : [ -128, -128, 25, -128, -128, 25, -128, -128 ], "fanSpeed/key" : { "type" : 7, "num_values" : 3, "last_written" : 1376681747 }, "fanSpeed" : [ 3180, 3112, 3236 ] } , { "EnableControl/key" : { "type" : 7, "last_written" : 1375145722 }, "EnableControl" : 1 } , { "mainSwitch/key" : { "type" : 7, "last_written" : 1376603253 }, "mainSwitch" : 1 } , { "sysMainSwitch.0/key" : { "type" : 7, "last_written" : 1376681750 }, "sysMainSwitch.0" : 1 } , { "switch/key" : { "type" : 7, "num_values" : 3, "last_written" : 1376681750 }, "switch" : [ 1, 1, 1 ], "status/key" : { "type" : 7, "num_values" : 3, "last_written" : 1376681750 }, "status" : [ 1, 1, 1 ], "demandVoltage/key" : { "type" : 9, "num_values" : 3, "last_written" : 1376681750 }, "demandVoltage" : [ 5, 12, 12 ], "senseVoltage/key" : { "type" : 9, "num_values" : 3, "last_written" : 1376681750 }, "senseVoltage" : [ 5, 1.1950000e+01, 1.1980000e+01 ], "current/key" : { "type" : 9, "num_values" : 3, "last_written" : 1376681750 }, "current" : [ 1.4169999e+01, 1.1000000e+00, 8.0000001e-01 ], "sparkCount/key" : { "type" : 7, "num_values" : 3, "last_written" : 1376681750 }, "sparkCount" : [ 0, 0, 0 ], "sensorTemperature/key" : { "type" : 7, "num_values" : 8, "last_written" : 1376681750 }, "sensorTemperature" : [ -128, -128, -128, -128, -128, -128, -128, 22 ], "fanSpeed/key" : { "type" : 7, "num_values" : 3, "last_written" : 1376681750 }, "fanSpeed" : [ 3217, 3225, 3213 ] } , { "EnableControl/key" : { "type" : 7, "last_written" : 1375145722 }, "EnableControl" : 1 } , { "mainSwitch/key" : { "type" : 7, "last_written" : 1376596430 }, "mainSwitch" : 1 } , { "sysMainSwitch.0/key" : { "type" : 7, "last_written" : 1376681752 }, "sysMainSwitch.0" : 1 } , { "switch/key" : { "type" : 7, "num_values" : 3, "last_written" : 1376681752 }, "switch" : [ 1, 1, 1 ], "status/key" : { "type" : 7, "num_values" : 3, "last_written" : 1376681752 }, "status" : [ 1, 1, 1 ], "demandVoltage/key" : { "type" : 9, "num_values" : 3, "last_written" : 1376681752 }, "demandVoltage" : [ 5, 12, 12 ], "senseVoltage/key" : { "type" : 9, "num_values" : 3, "last_written" : 1376681752 }, "senseVoltage" : [ 4.9800000e+00, 1.2009999e+01, 1.1969999e+01 ], "current/key" : { "type" : 9, "num_values" : 3, "last_written" : 1376681752 }, "current" : [ 9.1139999e+01, 4.1999998e+00, 6.0300002e+00 ], "sparkCount/key" : { "type" : 7, "num_values" : 3, "last_written" : 1376681752 }, "sparkCount" : [ 0, 0, 0 ], "sensorTemperature/key" : { "type" : 7, "num_values" : 8, "last_written" : 1376681752 }, "sensorTemperature" : [ -128, -128, 24, -128, -128, 27, -128, -128 ], "fanSpeed/key" : { "type" : 7, "num_values" : 3, "last_written" : 1376681752 }, "fanSpeed" : [ 3153, 3067, 3180 ] } ]
</pre>

= ODBMLs =

Description: return ODB directory listing with full ODB KEY information (same as odbedit ls -l). It uses the jcopy AJAX method with the JSON "norecurse" option. Same as with odbedit "ls -l", symlinks are followed and the linked-to values are reported, with the symlink path reported as "link" entry in the ODB KEY ("xxx/key") section.

See: [[AJAX#jcopy]]

Prototype:
* function ODBMLs(paths, callback)

Arguments:
* paths - array of strings - odb paths to report
* callback - name of javascript function for asynchronous callback, or "undefined" for synchronous return

Returns:
* array of JSON encoded ODB entries
* synchronous: returns request.responseText, use JSON.parse() to decode JSON data
* asynchronous: calls the callback function with a single argument set to request.responseText, use JSON.parse() to decode JSON data

Usage:
<pre>
var paths = [ "/", "/Test/noexistant", "/Experiment" ];
var data = ODBMLs(paths);
var obj = JSON.Parse(data);
</pre>

= ODBMCreate =

See: [[AJAX#jcreate]]

Prototype:
* function ODBMCreate(paths, types, arraylengths, stringlengths, callback)

Arguments:
* paths - array of strings - odb paths to create
* types - array of integers - midas data types of created keys (TID_INT, etc)
* arraylengths - array of integers - "undefined" - or array size to create
* stringlengths - array of integers - "undefined" - or array of lengths of strings to create
* callback - name of javascript function for asynchronous callback, or "undefined" for synchronous return

Returns:
* array of integers: status of db_create_key() for each element of "paths"
* synchronous: returns request.responseText, use JSON.parse() to decode JSON data
* asynchronous: calls the callback function with a single argument set to request.responseText, use JSON.parse() to decode JSON data

Usage:
<pre>
var paths = [ "/Test/foo", "/Test/noexistant", "/Test/bar" ];
var types = [ TID_INT, 0, TID_STRING ];
ODBMCreate(paths, types);
</pre>

= ODBMResize =

See: [[AJAX#jresize]]

Prototype:
* function ODBMResize(paths, arraylengths, stringlengths, callback)

Arguments:
* paths - array of strings - odb paths to resize
* arraylengths - array of integers - new array size, use "0" for "no change"
* stringlengths - array of integers - new string length or new element length for string arrays, use "0" for "no change"
* callback - name of javascript function for asynchronous callback, or "undefined" for synchronous return

Returns:
* array of integers: status of db_set_num_values() for each element of "paths"
* synchronous: returns request.responseText, use JSON.parse() to decode JSON data
* asynchronous: calls the callback function with a single argument set to request.responseText, use JSON.parse() to decode JSON data

Usage:
<pre>
var paths = [ "/Test/foo", "/Test/noexistant", "/Test/bar" ];
var arraylengths = [ 10, 0, 15 ];
var stringarrays = [ 0, 0, 16 ];
ODBMResize(paths, arraylengths, stringlengths);
</pre>

= ODBMKey =

See: [[AJAX#jkey]]

Prototype:
* function ODBMKey(paths, callback)

Arguments:
* paths - array of strings - odb paths to read ODB KEYs
* callback - name of javascript function for asynchronous callback, or "undefined" for synchronous return

Returns:
* array of ODB KEYs
* synchronous: returns request.responseText, use JSON.parse() to decode JSON data
* asynchronous: calls the callback function with a single argument set to request.responseText, use JSON.parse() to decode JSON data

Usage:
<pre>
var paths = [ "/Test/foo", "/Test/noexistant", "/Test/bar" ];
var data = JSON.parse(ODBMKey(paths));
</pre>

Example:
<pre>
var paths = [ "/Test/foo", "/Test/noexistant", "/Test/bar", "/Test/foofoo", "/Test/barlink", "/Test/foo10", "/Test/bar15" ];
var data = ODBMKey(paths);
xprint data;
[ { "name":"foo","type":7,"type_name":"TID_INT","num_values":1,"item_size":4,"last_written":0 }, { "/error":312 }, { "name":"bar","type":12,"type_name":"TID_STRING","num_values":1,"item_size":0,"last_written":0 }, { "/error":312 }, { "name":"bar","type":12,"type_name":"TID_STRING","num_values":1,"item_size":0,"last_written":0 }, { "name":"foo10","type":7,"type_name":"TID_INT","num_values":5,"item_size":4,"last_written":1380134916 }, { "name":"bar15","type":12,"type_name":"TID_STRING","num_values":7,"item_size":12,"last_written":1380134916 } ]
</pre>

= ODBMDelete =

See: [[AJAX#jdelete]]

Prototype:
* function ODBMDelete(paths, callback)

Arguments:
* paths - array of strings - odb paths to delete
* callback - name of javascript function for asynchronous callback, or "undefined" for synchronous return

Returns:
* array of integer: status of db_delete_key() for each element of "paths"
* synchronous: returns request.responseText, use JSON.parse() to decode JSON data
* asynchronous: calls the callback function with a single argument set to request.responseText, use JSON.parse() to decode JSON data

Usage:
<pre>
var paths = [ "/Test/foo", "/Test/noexistant", "/Test/bar" ];
ODBMDelete(paths);
</pre>

= ODBMRename =

See: [[AJAX#jrename]]

Prototype:
* function ODBMRename(paths, names, callback)

Arguments:
* paths - array of strings - odb paths to rename
* names - array of strings - new names (subject to ODB limitation on key names)
* callback - name of javascript function for asynchronous callback, or "undefined" for synchronous return

Returns:
* array of integer: status of db_rename_key() for each element of "paths"
* synchronous: returns request.responseText, use JSON.parse() to decode JSON data
* asynchronous: calls the callback function with a single argument set to request.responseText, use JSON.parse() to decode JSON data

Usage:
<pre>
var paths = [ "/Test/foo", "/Test/noexistant", "/Test/bar" ];
var names = [ "fooxxx", "zzz", "barxxx" ];
ODBMRename(paths, names);
</pre>

= ODBMLink =

See: [[AJAX#jlink]]

Prototype:
* function ODBMLink(paths, links, callback)

Arguments:
* paths - array of strings - odb paths create as symlinks
* links - array of strings - odb paths of existing odb entries that we link to
* callback - name of javascript function for asynchronous callback, or "undefined" for synchronous return

Returns:
* array of integer: status of db_create_link() for each element of "paths"
* synchronous: returns request.responseText, use JSON.parse() to decode JSON data
* asynchronous: calls the callback function with a single argument set to request.responseText, use JSON.parse() to decode JSON data

Usage:
<pre>
var paths = [ "/Test/foolink", "/Test/barlink" ];
var links = [ "/Test/foo", "/Test/bar" ];
ODBMLink(paths, links);
</pre>

= ODBGetMsg =
Gets message(s) from the MIDAS [[Message System]] or from the [[Chat Page|chat]].

The parameters of this command have been changed August 2015. See [[#odbgetmsg_old|below]] for older version.

Prototype:

* function ODBGetMsg(facility, start, number)

Arguments:

* facility - message facility in which messages are written
: facility can be "midas" for normal messages or "chat" to load the chat messages
* start - start time (0=latest message)
: the absolute time from when to retrieve messages. Time is defined in unix standard time, i.e. seconds since 1.1.1970. A value of zero has a special meaning of now.
* number - number of midas messages to return

Returns: array of strings, if number==1, a single string.

Usage:

Retrieve the last 100 messages:
var data = ODBGetMsg("midas",0,100);
document.getElementById("foo").innerHTML = data.join("\n");
Each message returned in a separate line in this code contains its UNIX time in the first position, e.g.
1449015258 01:14:18.006 2015/12/02 [SC Frontend,TALK] General alarm: COBRA SC quenched
Now the UNIX time stamp can be saved in the JavaScript code on the page, e.g.
last_tstamp = 1449015258
and in another call we can retrieve the additional 100 message which occurred BEFORE the previous block, i.e.
ODBGetMsg(midas, last_tstamp-1, 100)
The time stamping ensures that no message is lost between the two calls, even if the second call is made many seconds later than the first call. So start
means start searching for messages at the given UNIX timestamp, and go backwards for n messages.

The function ODBGetMsg returns an array of strings, with one entry per message. This way they can be easily parsed in JavaScript:
var msg = ODBGetMsg(midas, 0, 100);
for (i=0 ; i<msg.length ; i++)
alert(msg[i]);

To see how individual messages are parsed and extracted see function ''msg_append(msg)'' in [[Mhttpd.js|MIDAS Javascript Library]].

<div style="font-size:90%; background-color:whitesmoke">
<div id="odbgetmsg_old"></div>
; ODBGetMsg (pre August 2015)
See: [[AJAX#jgenmsg]]
Prototype:
* function ODBGetMsg(n)

Arguments:
* n - number of midas messages to return

Returns: array of strings, if n==1, as single string.

Usage:
var data = ODBGetMsg(10);
document.getElementById("foo").innerHTML = data.join("\n");

</div>

= ODBGenMsg =
Send a message to the MIDAS [[Message System]] or the [[Chat Page|Chat]].

The parameters of this command have been changed August 2015. See [[#odbgenmsg_old|below]] for older version.

Prototype:

* function ODBGenMsg(type, facility, user, msg)

Arguments:
* type - midas message type, numerical value of one of MT_ERROR, MT_INFO, MT_DEBUG, MT_USER, as defined in midas.h.
: i.e. 1 for MT_ERROR, 2 for MT_INFO etc.
* facility - facility can be "midas" for the MIDAS message system, or "chat" for the chat message system
* user - name of the user who generated the message
: The user name ends up in the message file inside the [] section to tag each message with an associated user name.
* message - text written into midas messages

Usage:
ODBGenerateMsg(2, midas, fred, "message text");

<div style="font-size:90%; background-color:whitesmoke">
<div id="odbgenmsg_old"></div>
; ODBGenerateMsg (pre August 2015)

See: [[AJAX#jgenmsg]]

Prototype:
* function ODBGenerateMsg(m)

Arguments:
* m - text written into midas messages, must be URI-encoded.

Usage:
<pre>
ODBGenerateMsg(encodeURIComponent("message text"));
</pre>
</div>

= ODBGetAlarms =

See: [[AJAX#jalm]]

Prototype:
* function ODBGetAlarms()

Returns: array of strings

Usage:
<pre>
var data = ODBGetAlarms();
document.getElementById("foo").innerHTML = data.join("\n");
</pre>

[[Category:Custom]] [[Category:Javascript library]] [[Category:Ajax]]

Main Page

2025-11-04T22:54:22Z

Lindner: /* News and Support */

= Introduction =

MIDAS is a modern data acquisition system developed at [http://www.psi.ch PSI] and [http://www.triumf.ca TRIUMF]. Supported hardware includes VME, Fastbus, CAMAC, RS232, GPIB, USB, ethernet, fiber optic and MSCB attached data acquisition devices. MIDAS is written in C/C++ and runs on Linux, MacOS and MS Windows. It is licensed under the [http://www.gnu.org/copyleft/gpl.html GNU Public License].
[[Introduction|'''more...''']]

= Documentation =

* [[Midas documentation| Main Documentation]]
* [[Installation | Installation Instructions]]
* [[Changelog | Changelog / Upgrade Instructions]]
* [https://daq.triumf.ca/~daqweb/doc/midas-develop/html/modules.html <i>Doxygen</i>] Code structure, functions, etc...
* [https://bitbucket.com/tmidas/midas git repository on bitbucket] Source code, bug tracking, etc
* [https://midas.triumf.ca/forum MIDAS forum] Discussion, announcements, problem reports, requests for improvements

= News and Support=

For news about bug fixes and new releases, go to the
[https://midas.triumf.ca/forum MIDAS forum] hosted by TRIUMF using
the [http://elog.psi.ch/elog ELOG] system. Users can register in this system to be notified automatically via
E-mail when new entries are submitted.

Another source of information is the [https://bitbucket.com/tmidas/midas Bitbucket Repository] of MIDAS, where one can see the latest changes to the software.

Finally, you can read presentations from
* [https://indico.psi.ch/conferenceDisplay.py?ovw=True&confId=3793 2015 MIDAS Workshop]
* [https://meetings.triumf.ca/event/428/ 2017 MIDAS Workshop]
* [https://meetings.triumf.ca/event/429/ 2019 MIDAS Workshop]
* [[2023 MIDAS Workshop]]
* [[2025 MIDAS Workshop]]

= Related packages =

* [http://elog.psi.ch/elog ELOG] - Electronics Logbook from PSI
* [https://bitbucket.org/tmidas/manalyzer/src/master/ manalyzer] - data analysis from TRIUMF
* [[ROOT]] - data analysis package from CERN
* [[ROOTANA]] - ROOT-based analyzer for MIDAS
* [[ROODY]] - viewer for online histograms
* http://elog.psi.ch/rome/ - MEG/PSI data analysis package

2025 MIDAS Workshop

2025-11-04T22:53:54Z

Lindner:

The agenda and posted presentation are on the [https://indico.psi.ch/event/17580/timetable/ Indico page for 2025 workshop].

[https://triumfoffice365-my.sharepoint.com/:v:/g/personal/lindner_triumf_ca/EYDh_iPG8CpEqUvKFiNUYXsBin4pC53fgtiAmfvnQscC7Q?nav=eyJyZWZlcnJhbEluZm8iOnsicmVmZXJyYWxBcHAiOiJPbmVEcml2ZUZvckJ1c2luZXNzIiwicmVmZXJyYWxBcHBQbGF0Zm9ybSI6IldlYiIsInJlZmVycmFsTW9kZSI6InZpZXciLCJyZWZlcnJhbFZpZXciOiJNeUZpbGVzTGlua0NvcHkifX0&e=iey0zn Zoom recording of day 1]

[https://triumfoffice365-my.sharepoint.com/:v:/g/personal/lindner_triumf_ca/ETKLMOgo5vVDrW6cvohWBDMBUmYNzMBg2BRZdRk1RJEqug?nav=eyJyZWZlcnJhbEluZm8iOnsicmVmZXJyYWxBcHAiOiJPbmVEcml2ZUZvckJ1c2luZXNzIiwicmVmZXJyYWxBcHBQbGF0Zm9ybSI6IldlYiIsInJlZmVycmFsTW9kZSI6InZpZXciLCJyZWZlcnJhbFZpZXciOiJNeUZpbGVzTGlua0NvcHkifX0&e=Pcy5lY Zoom recording of day 2]

2025 MIDAS Workshop

2025-11-04T22:52:08Z

Lindner:

Main Page

2025-11-04T22:51:02Z

Lindner: /* News and Support */

= Introduction =

MIDAS is a modern data acquisition system developed at [http://www.psi.ch PSI] and [http://www.triumf.ca TRIUMF]. Supported hardware includes VME, Fastbus, CAMAC, RS232, GPIB, USB, ethernet, fiber optic and MSCB attached data acquisition devices. MIDAS is written in C/C++ and runs on Linux, MacOS and MS Windows. It is licensed under the [http://www.gnu.org/copyleft/gpl.html GNU Public License].
[[Introduction|'''more...''']]

= Documentation =

* [[Midas documentation| Main Documentation]]
* [[Installation | Installation Instructions]]
* [[Changelog | Changelog / Upgrade Instructions]]
* [https://daq.triumf.ca/~daqweb/doc/midas-develop/html/modules.html <i>Doxygen</i>] Code structure, functions, etc...
* [https://bitbucket.com/tmidas/midas git repository on bitbucket] Source code, bug tracking, etc
* [https://midas.triumf.ca/forum MIDAS forum] Discussion, announcements, problem reports, requests for improvements

= News and Support=

For news about bug fixes and new releases, go to the
[https://midas.triumf.ca/forum MIDAS forum] hosted by TRIUMF using
the [http://elog.psi.ch/elog ELOG] system. Users can register in this system to be notified automatically via
E-mail when new entries are submitted.

Another source of information is the [https://bitbucket.com/tmidas/midas Bitbucket Repository] of MIDAS, where one can see the latest changes to the software.

Finally, you can read presentations from
* [https://indico.psi.ch/conferenceDisplay.py?ovw=True&confId=3793 2015 MIDAS Workshop]
* [https://meetings.triumf.ca/event/428/ 2017 MIDAS Workshop]
* [https://meetings.triumf.ca/event/429/ 2019 MIDAS Workshop]
* [[2023 MIDAS Workshop]]
* [[2025 MIDAS Workshop]]
* [https://indico.psi.ch/event/17580/ 2025 MIDAS Workshop]

= Related packages =

* [http://elog.psi.ch/elog ELOG] - Electronics Logbook from PSI
* [https://bitbucket.org/tmidas/manalyzer/src/master/ manalyzer] - data analysis from TRIUMF
* [[ROOT]] - data analysis package from CERN
* [[ROOTANA]] - ROOT-based analyzer for MIDAS
* [[ROODY]] - viewer for online histograms
* http://elog.psi.ch/rome/ - MEG/PSI data analysis package

2025 MIDAS Workshop

2025-09-24T20:43:02Z

Lindner: Created page with " The agenda and posted presentation are on the [https://indico.psi.ch/event/17580/timetable/ Indico page for 2025 workshop]."

The agenda and posted presentation are on the [https://indico.psi.ch/event/17580/timetable/ Indico page for 2025 workshop].

Main Page

2025-09-24T20:42:01Z

Lindner: /* News and Support */

= Introduction =

MIDAS is a modern data acquisition system developed at [http://www.psi.ch PSI] and [http://www.triumf.ca TRIUMF]. Supported hardware includes VME, Fastbus, CAMAC, RS232, GPIB, USB, ethernet, fiber optic and MSCB attached data acquisition devices. MIDAS is written in C/C++ and runs on Linux, MacOS and MS Windows. It is licensed under the [http://www.gnu.org/copyleft/gpl.html GNU Public License].
[[Introduction|'''more...''']]

= Documentation =

* [[Midas documentation| Main Documentation]]
* [[Installation | Installation Instructions]]
* [[Changelog | Changelog / Upgrade Instructions]]
* [https://daq.triumf.ca/~daqweb/doc/midas-develop/html/modules.html <i>Doxygen</i>] Code structure, functions, etc...
* [https://bitbucket.com/tmidas/midas git repository on bitbucket] Source code, bug tracking, etc
* [https://midas.triumf.ca/forum MIDAS forum] Discussion, announcements, problem reports, requests for improvements

= News and Support=

For news about bug fixes and new releases, go to the
[https://midas.triumf.ca/forum MIDAS forum] hosted by TRIUMF using
the [http://elog.psi.ch/elog ELOG] system. Users can register in this system to be notified automatically via
E-mail when new entries are submitted.

Another source of information is the [https://bitbucket.com/tmidas/midas Bitbucket Repository] of MIDAS, where one can see the latest changes to the software.

Finally, you can read presentations from
* [https://indico.psi.ch/conferenceDisplay.py?ovw=True&confId=3793 2015 MIDAS Workshop]
* [https://meetings.triumf.ca/event/428/ 2017 MIDAS Workshop]
* [https://meetings.triumf.ca/event/429/ 2019 MIDAS Workshop]
* [[2023 MIDAS Workshop]]
* [[2025 MIDAS Workshop]]
* [https://indico.psi.ch/event/17580/ 2025 MIDAS Workshop]

= Related packages =

* [http://elog.psi.ch/elog ELOG] - Electronics Logbook from PSI
* [https://bitbucket.org/tmidas/manalyzer/src/master/ manalyzer] - data analysis from TRIUMF
* [[ROOT]] - data analysis package from CERN
* [[ROOTANA]] - ROOT-based analyzer for MIDAS
* [[ROODY]] - viewer for online histograms
* http://elog.psi.ch/rome/ - MEG/PSI data analysis package

Main Page

2025-09-18T23:16:30Z

Lindner: /* News and Support */

= Introduction =

MIDAS is a modern data acquisition system developed at [http://www.psi.ch PSI] and [http://www.triumf.ca TRIUMF]. Supported hardware includes VME, Fastbus, CAMAC, RS232, GPIB, USB, ethernet, fiber optic and MSCB attached data acquisition devices. MIDAS is written in C/C++ and runs on Linux, MacOS and MS Windows. It is licensed under the [http://www.gnu.org/copyleft/gpl.html GNU Public License].
[[Introduction|'''more...''']]

= Documentation =

* [[Midas documentation| Main Documentation]]
* [[Installation | Installation Instructions]]
* [[Changelog | Changelog / Upgrade Instructions]]
* [https://daq.triumf.ca/~daqweb/doc/midas-develop/html/modules.html <i>Doxygen</i>] Code structure, functions, etc...
* [https://bitbucket.com/tmidas/midas git repository on bitbucket] Source code, bug tracking, etc
* [https://midas.triumf.ca/forum MIDAS forum] Discussion, announcements, problem reports, requests for improvements

= News and Support=

For news about bug fixes and new releases, go to the
[https://midas.triumf.ca/forum MIDAS forum] hosted by TRIUMF using
the [http://elog.psi.ch/elog ELOG] system. Users can register in this system to be notified automatically via
E-mail when new entries are submitted.

Another source of information is the [https://bitbucket.com/tmidas/midas Bitbucket Repository] of MIDAS, where one can see the latest changes to the software.

Finally, you can read presentations from
* [https://indico.psi.ch/conferenceDisplay.py?ovw=True&confId=3793 2015 MIDAS Workshop]
* [https://meetings.triumf.ca/event/428/ 2017 MIDAS Workshop]
* [https://meetings.triumf.ca/event/429/ 2019 MIDAS Workshop]
* [[2023 MIDAS Workshop]]
* [https://indico.psi.ch/event/17580/ 2025 MIDAS Workshop]

= Related packages =

* [http://elog.psi.ch/elog ELOG] - Electronics Logbook from PSI
* [https://bitbucket.org/tmidas/manalyzer/src/master/ manalyzer] - data analysis from TRIUMF
* [[ROOT]] - data analysis package from CERN
* [[ROOTANA]] - ROOT-based analyzer for MIDAS
* [[ROODY]] - viewer for online histograms
* http://elog.psi.ch/rome/ - MEG/PSI data analysis package

Main Page

2025-09-18T23:16:05Z

Lindner: /* News and Support */

= Introduction =

MIDAS is a modern data acquisition system developed at [http://www.psi.ch PSI] and [http://www.triumf.ca TRIUMF]. Supported hardware includes VME, Fastbus, CAMAC, RS232, GPIB, USB, ethernet, fiber optic and MSCB attached data acquisition devices. MIDAS is written in C/C++ and runs on Linux, MacOS and MS Windows. It is licensed under the [http://www.gnu.org/copyleft/gpl.html GNU Public License].
[[Introduction|'''more...''']]

= Documentation =

* [[Midas documentation| Main Documentation]]
* [[Installation | Installation Instructions]]
* [[Changelog | Changelog / Upgrade Instructions]]
* [https://daq.triumf.ca/~daqweb/doc/midas-develop/html/modules.html <i>Doxygen</i>] Code structure, functions, etc...
* [https://bitbucket.com/tmidas/midas git repository on bitbucket] Source code, bug tracking, etc
* [https://midas.triumf.ca/forum MIDAS forum] Discussion, announcements, problem reports, requests for improvements

= News and Support=

For news about bug fixes and new releases, go to the
[https://midas.triumf.ca/forum MIDAS forum] hosted by TRIUMF using
the [http://elog.psi.ch/elog ELOG] system. Users can register in this system to be notified automatically via
E-mail when new entries are submitted.

Another source of information is the [https://bitbucket.com/tmidas/midas Bitbucket Repository] of MIDAS, where one can see the latest changes to the software.

Finally, you can read presentations from
* [https://indico.psi.ch/conferenceDisplay.py?ovw=True&confId=3793 2015 MIDAS Workshop]
* [https://meetings.triumf.ca/event/428/ 2017 MIDAS Workshop]
* [https://meetings.triumf.ca/event/429/ 2019 MIDAS Workshop]
* [[2023 MIDAS Workshop]]
* [https://indico.psi.ch/event/17580/ [2025 MIDAS Workshop]]

= Related packages =

* [http://elog.psi.ch/elog ELOG] - Electronics Logbook from PSI
* [https://bitbucket.org/tmidas/manalyzer/src/master/ manalyzer] - data analysis from TRIUMF
* [[ROOT]] - data analysis package from CERN
* [[ROOTANA]] - ROOT-based analyzer for MIDAS
* [[ROODY]] - viewer for online histograms
* http://elog.psi.ch/rome/ - MEG/PSI data analysis package

Equipment Page

2025-04-29T17:55:55Z

Lindner: /* Configuring the equipment page */

{{Pagelinks}}

= Links =
{{mhttpdpages}}

= Purpose =
The purpose of the [[mhttpd]] equipment pages is to allow the user to view any Variables related to an equipment.

= Access the equipment page =

The main [[Status Page]] contains a list of all the equipment in an experiment (with each equipment being defined by a frontend). The name of each equipment on that page is a link to the relevant equipment page.

= Configuring the equipment page =

There are several ODB keys that can be defined in {{Odbpath|path=/Equipment/<equip_name>/}} that affect the behaviour of this webpage. See [[/Equipment_ODB_tree#Settings_subtree|the Equipment ODB tree]] documentation for full details, but in brief:

* "/Equipment/<equip_name>/Variables/" contains the Variables to show on this page (and to record in the history system).
* "/Equipment/<equip_name>/Settings/Names" or "/Equipment/<equip_name>/Settings/Names <variable>" contains the name of each element in the Variables arrays (e.g. channel names if recording the current of multiple channels).
* "/Equipment/<equip_name>/Settings/Unit <variable>" contains the units in which a variable is measured.
* "/Equipment/<equip_name>/Settings/Editable" contains a comma-separated list of variables that can be edited on the Equipment page.
* "/Equipment/<equip_name>/Settings/Grid display" is a boolean which allows you to switch between linear display and "grid" display. These options are:
** Grid display: This is useful if there are “channels” with different attributes. Such as HV channels with current and voltage. In this case, we have several array under “Variables” which “belong” together, such as a voltage array and a current array. In this case one should have /Equpiment/<name>/Settings/Grid display = true, which cases the display like at the top of this page (Output on, Power good, Fault, Current). One also should then add “Names” under settings to have proper labels for each channel.
** Non-grid display: Here we have arrays not related to each other. Here the grid display should be off, then you get the display directly above, where all banks are shown below each other.

All the settings are optional, and must be created by the user if desired.

= Example configuration and display =

Click to enlarge each image...

<gallery widths="300px" mode="packed">
Equipment_Variables.png|Variables
Equipment_Settings.png|Settings
Equipment_Page.png|Resulting display
</gallery>

Equipment Page

2025-04-29T17:54:44Z

Lindner: /* Configuring the equipment page */

Setup MySQL database for MIDAS

2025-04-25T22:47:05Z

Lindner: /* Setting up MySQL using mariadb on macbook */

This page provides instructions for setting up a MySQL database to use with the MIDAS history logging.

Setting up the MySQL database is highly dependent on what OS and OS version you are using. Most of the setup work is just MySQL related. For this reason we provide below instructions for the MySQL database setup on a couple different OS. You will probably need to modify these instructions for your OS. Here's the some installation instructions:

* [[#Setting up MySQL on Ubuntu 24.04 LTS | Setting up MySQL on Ubuntu 24.04 LTS ]]
* [[#Setting up MySQL using mariadb on macbook | Setting up MySQL using mariadb on macbook]]

== Setting up MySQL on Ubuntu 24.04 LTS ==

This instructions assume you have root privileges and install MySQL as ROOT.

=== Do these steps as root user ===

Install the MySQL server software.

<pre>
apt install mysql-server
</pre>

You will find mysql has started after installation

<pre>
root@mpmt-daq03:/home1/wheel# service mysql status
mysql.service - MySQL Community Server
Loaded: loaded (/usr/lib/systemd/system/mysql.service; enabled; preset: enabled)
Active: active (running) since Fri 2025-04-25 15:12:58 PDT; 4s ago
Process: 153129 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=0>
Main PID: 153137 (mysqld)
Status: "Server is operational"
</pre>

Default configuration only allows MySQL connections to localhost. Good
<pre>
root@mpmt-daq03:/home1/wheel# grep bind-add /etc/mysql/mysql.conf.d/mysqld.cnf
bind-address = 127.0.0.1
mysqlx-bind-address = 127.0.0.1
</pre>

Login to MySQL as root and create the MySQL accounts

<pre>
mysql -h localhost
...
mysql> CREATE DATABASE IF NOT EXISTS history;
mysql> CREATE USER history_reader@'localhost' IDENTIFIED BY 'CHANGE_THIS_PASSWORD!';
mysql> GRANT SELECT ON history.* TO history_reader@'localhost';
mysql> CREATE USER 'history_writer'@'localhost' IDENTIFIED BY 'CHANGE_THIS_PASSWORD!';
mysql> GRANT SELECT,INSERT,CREATE,ALTER,INDEX ON history.* TO 'history_writer'@'localhost';
mysql> flush privileges;
</pre>

ATTENTION: change the account password listed above to something different!

Check the socket to connect to:

<pre>
root@mpmt-daq03:/home1/wheel# grep sock /etc/mysql/my.cnf
# Port or socket location where to connect
socket = /run/mysqld/mysqld.sock
</pre>

=== Do these steps as the experiment user ===

Rebuild MIDAS with support for MySQL. If this works correctly the build will notice MySQL is installed. For instance, with cmake we will see

<pre>
-- MIDAS: Found MySQL version 8.0.41
</pre>

Go to online directory and create files so MIDAS knows accounts to talk to ODB

<pre>

testmysql@mpmt-daq03:~/online$ cd ~/online
testmysql@mpmt-daq03:~/online$ emacs -nw mysql_reader.txt
testmysql@mpmt-daq03:~/online$ emacs -nw mysql_writer.txt
testmysql@mpmt-daq03:~/online$ more mysql_*.txt
::::::::::::::
mysql_reader.txt
::::::::::::::
database=history
socket=/run/mysqld/mysqld.sock
user=history_reader
password=CHANGE_THIS_PASSWORD!
::::::::::::::
mysql_writer.txt
::::::::::::::
database=history
socket=/run/mysqld/mysqld.sock
user=history_writer
password=CHANGE_THIS_PASSWORD!
</pre>

Again, set the correct password in these files.

Change the mlogger settings so we use the MySQL for the history logging:
* disable the previous history logger
* enable the MySQL history logging:
<pre>
set "/Logger/History/MYSQL/Active" y
</pre>

Restart the mlogger on the command line and see that some MySQL tables are created and columns added:

<pre>
testmysql@mpmt-daq03:~/online$ mlogger
[Logger,INFO] Per-variable history is enabled
[Logger,ERROR] [history_schema.cxx:1258:Mysql::Prepare,ERROR] mysql_query(SELECT event_name, table_name, itimestamp FROM _history_index WHERE table_name!='';) error 1146 (Table 'history._history_index' doesn't exist)
[Logger,INFO] Adding SQL table "myperiodicequipment_pydt"
[Logger,INFO] Adding SQL table "_history_index"
[Logger,INFO] Adding column "event_name" to SQL table "_history_index", status 1
[Logger,INFO] Adding column "table_name" to SQL table "_history_index", status 1
...
[Logger,INFO] Adding column "fluxgate0" to SQL table "myperiodicequipment_pydt", status 1
[Logger,INFO] Adding column "fluxgate1" to SQL table "myperiodicequipment_pydt", status 1
[Logger,INFO] Adding column "valvex" to SQL table "myperiodicequipment_pydt", status 1
[Logger,INFO] Adding column "valvey" to SQL table "myperiodicequipment_pydt", status 1
[Logger,INFO] Adding column "t5" to SQL table "myperiodicequipment_pydt", status 1
...
</pre>

Login directly to the MySQL database and make sure the data is actually being logged

<pre>
mysql> select _i_time,fluxgate0 from myperiodicequipment_pydt where _i_time <1745620795;
+------------+-----------+
| _i_time | fluxgate0 |
+------------+-----------+
| 1745620782 | 6.20209 |
| 1745620783 | 6.3519 |
| 1745620784 | 6.5677 |
| 1745620785 | 7.81413 |
| 1745620786 | 5.41127 |
| 1745620787 | 7.31112 |
| 1745620788 | 6.14285 |
| 1745620789 | 7.20425 |
| 1745620790 | 6.9303 |
| 1745620791 | 7.08178 |
| 1745620792 | 7.44422 |
| 1745620794 | 5.32677 |
+------------+-----------+
12 rows in set (0.00 sec)
</pre>

Restart mhttpd and you will see the history plots starting to get data from MySQL database.

== Setting up MySQL using mariadb on macbook ==

* install mariadb-10.4
<pre>
macos:
sudo port install mariadb-10.4
ln -s ln -fs /opt/local/lib/mariadb-10.4/bin/mysql_config ~/bin/
mysql_config --version
10.4.13
</pre>
* rebuild midas, check the cmake finds the correct mysql: MIDAS: Found MySQL version 10.4.13
* create $HOME/.my.cnf
<pre>
[mysqld]
#port=3306
socket=/Users/olchansk/mysql/mysql.sock
datadir=/Users/olchansk/mysql/db
pid-file=/Users/olchansk/mysql/mysqld.pid

[client]
#port=3306
socket=/Users/olchansk/mysql/mysql.sock
</pre>
* mkdir $HOME/mysql
* start mysql:
<pre>
macos: /opt/local/lib/mariadb-10.4/bin/mysqld
</pre>
* configure the database: start the "mysql" utility (/opt/local/lib/mariadb-10.4/bin/mysql)
<pre>
NOTE: instructions thanks to Ben S.
NOTE: use some other passwords instead of the example "reader_password" and "writer_password"

MariaDB [(none)]> CREATE DATABASE IF NOT EXISTS history;
Query OK, 1 row affected (0.003 sec)

MariaDB [(none)]> CREATE USER history_reader@'localhost' IDENTIFIED BY 'reader_password';
Query OK, 0 rows affected (0.027 sec)

MariaDB [(none)]> GRANT SELECT ON history.* TO history_reader@'localhost';
Query OK, 0 rows affected (0.016 sec)

MariaDB [(none)]> CREATE USER 'history_writer'@'localhost' IDENTIFIED BY 'writer_password';
Query OK, 0 rows affected (0.014 sec)

MariaDB [(none)]> GRANT SELECT,INSERT,CREATE,ALTER,INDEX ON history.* TO 'history_writer'@'localhost';
Query OK, 0 rows affected (0.017 sec)

MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.001 sec)
</pre>
* try connecting to both users:
<pre>
/opt/local/lib/mariadb-10.4/bin/mysql -u history_reader -p ### should ask for reader password
/opt/local/lib/mariadb-10.4/bin/mysql -u history_writer -p ### should ask for writer password
</pre>
* create midas config files:
** mysql_writer.txt
<pre>
#server=localhost
#port=xxx
database=history
socket=/Users/olchansk/mysql/mysql.sock
user=history_writer
password=writer_password ### change this!!!
#buffer=1000
</pre>
** mysql_reader.txt
<pre>
#server=localhost
#port=xxx
database=history
socket=/Users/olchansk/mysql/mysql.sock
user=history_reader
password=reader_password ### change this!!!
</pre>
* test MIDAS connection
<pre>
mh2sql --mysql mysql_writer.txt
(there should be no errors printed)
</pre>
* if desired, import midas history from .hst files:
<pre>
mh2sql --mysql mysql_writer.txt *.hst
Reading 130813.hst
[mh2sql,ERROR] [history_schema.cxx:1165:Mysql::Prepare,ERROR] mysql_query(SELECT event_name, table_name, itimestamp FROM _history_index WHERE table_name!='';) error 1146 (Table 'history._history_index' doesn't exist)
[mh2sql,INFO] Adding SQL table "rpcexample", status 311
[mh2sql,INFO] Adding SQL table "rpcexample_1596127680", status 1
[mh2sql,INFO] Adding SQL table "_history_index", status 1
[mh2sql,INFO] Adding column "event_name" to SQL table "_history_index", status 1
...
[mh2sql,INFO] Adding SQL table "run_transitions", status 311
[mh2sql,INFO] Adding SQL table "run_transitions_1596127681", status 1
[mh2sql,INFO] Adding column "state" to SQL table "run_transitions_1596127681", status 1
[mh2sql,INFO] Adding column "run_number" to SQL table "run_transitions_1596127681", status 1
Reading 130815.hst
</pre>

** change ODB /Logger/History/MYSQL/Active to "y", "mysql writer" and "mysql reader" to location of these files (leave the default values if these files are in the experiment directory).
* enable MYSQL history in mhttpd:
** change ODB /History/LoggerHistoryChannel to "MYSQL"

History System

2025-04-25T22:46:53Z

Lindner: /* Write MYSQL-history events */

{{Pagelinks}}

== Links ==
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* [[/History ODB tree]]
* [[History Page]]
* [[Mhformat|MIDAS history file format]]
* [[Keys in the ODB /Logger tree|/Logger ODB tree]]
* [[Keys in the ODB /Logger/History subtree|/Logger/History subtree]]
* [[Image History]]
</div>

<div id="history file"></div>
= Introduction =

The history system is an add-on capability built into the MIDAS data logger [[mlogger]] to record useful information in parallel to the data logging.
This information is recorded with a time stamp and saved into a '''history file''' (see [[#History drivers|History drivers]]) for later retrieval. One set of files is created per day containing all the requested history events. The history data may be displayed in graphical form using the [[mhttpd]] MIDAS webserver utility, giving the user an easy way of seeing how experimental variables have changed with time.
The history logging will be in action only if the MIDAS data logger is running, but it is not necessary to have any [[Keys in the ODB /Logger tree#/Logger/Channels subtree|data logging channel]] enabled.

Since June 2020, it is possible to also use the midas history system to record images (png/jpeg etc) periodically, for example to later produce a timelapse from a webcam - see the [[Image History]] page for more details.

= MidasHistoryInterface =

The new history configuration defines history channels similar to logger channels. Each history channel creates a history interface class (class MidasHistoryInterface). The [[mlogger|data logger]] can use this interface to write data into the history, mhttpd, mhist & co can use this interface to read data from the history. The interface is defined in history.h

* http://ladd00.triumf.ca/~daqweb/doc/midas/include/history.h

= History drivers =

Presently, the MIDAS distribution contains these drivers:

; MIDAS-History
: this is the traditional history driver, with data stored in the .hst binary files. (format is documented here [[mhformat]], see also [[mhist]] and [[mhdump]])
: By default ODB subtree '''[[Keys in the ODB /Logger/History subtree#MIDAS subtree |"MIDAS"]]''' is set up to write MIDAS-history.

; ODBC-History
: stores history data in SQL database using the ODBC abstraction layer. See [[#MIDAS ODBC SQL History system|ODBC SQL History system]].

: By default ODB subtree '''[[Keys in the ODB /Logger/History subtree#ODBC subtree|"ODBC"]]''' is set up to write ODBC-history.

; SQLITE-History
: stores data in SQL database as SQLITE3 .sqlite3 files. The SQL table format is similar to ODBC-history, performance is similar, no database server required.
: By default ODB subtree '''[[Keys in the ODB /Logger/History subtree#SQLITE subtree|"SQLITE"]]''' is set up to write SQLITE-history.

; MYSQL-History
:
: By default ODB subtree '''[[Keys in the ODB /Logger/History subtree#MYSQL subtree|"MYSQL"]]''' is set up to write MYSQL-history.

; FILE-History
: stores each variable in its own file, making it much easier to trace the history of a particular variable.
: By default ODB subtree '''[[Keys in the ODB /Logger/History subtree#FILE subtree|FILE]]''' is set up to write FILE-history.

More drivers can be easily added by the user (create history_xxx.cxx, add constructor to hs_get_history(), create the ODB subtree <span style="color:purple; font-style:italic">/Logger/History/xxx/channel</span>).

= Location of History Files =
;MIDAS-history
:By default, the traditional MIDAS-history is enabled, writing history files into the MIDAS data directory path given by the ODB key [[Keys in the ODB /Logger tree#Data dir|Data dir]].
:If history data is important, it is recommended to write it to a directory that is backed up or archived. (The MIDAS data directory is usually located on a scratch data disk without backups).

:The location of the MIDAS-history data files can be changed by use of the ODB Key [[Keys in the ODB /Logger tree#History dir|History dir]].

;ODBC-History
:The location of ODBC-History files is ...

;SQLITE-History
:The location of SQLITE-History files is the directory path given by the ODB key [[Keys in the ODB /Logger/History subtree#Sqlite dir|Sqlite dir]].

;MYSQL-History
:The location of MYSQL-History files is ...

;FILE-History
:The location of FILE-History files is the directory path given by the ODB key [[Keys in the ODB /Logger/History subtree#History dir|History dir]].

= Types of History Events =

There are two basic types of history events, which are defined in different ways:

* "[[#Frontend History Event|Frontend]]" (or "Equipment") History event composed in the frontend. See [[#Frontend History Event|Frontend History Event]] for details.
* "Virtual" (or "Links") History event composed within the ODB under the [[/History ODB tree#Links subtree|/History/Links subtree]]. See [[#Virtual History Event|Virtual History Event]] for details.

Both these history event definitions take effect when the data logger gets a start run [[Run States and Transitions|transition]]. Any modifications made during the run are not applied until the start of the next run.

== Virtual History Event ==
The history of any ODB variable can be recorded in a "virtual" history event by creating a link to that variable in the [[/History ODB tree#Links subtree|/History/Links subtree]]. This is used for ODB variables that are not in an Equipment.

; To enable Virtual History Events
: Create links as desired in [[/History ODB tree#Links subtree|/History/Links subtree]].

History events are created for each entry under ODB subdirectory {{Odbpath|path=/History/Links}}.

Two types of links are permitted:
<dl>
<dt> ODB key {{Odbpath|path=/history/links/aaa}} is a link to a subdirectory</dt>
<dd><ol>
<li> db_watch() is setup to watch this subdirectory</li>
<li> tags are created for each subdirectory entry (1 tag per entry)</li>
<li> There is no possibility for naming array elements, so 1 tag per array, regardless of the number of elements.</li> </ol></dd>
<dt>ODB key {{Odbpath|path=/history/links/bbb}} is a subdirectory with links to ODB values</dt>
<dd> <ol>
<li>db_watch is setup to watch each link target</li>
<li> tags are created for each link (1 tag per link)</li>
<li> tag name is the link name (NOT the target name). There is no possibility for naming array elements</li>
</ol></dd>
</dl>

Mixing links and subdirectories is not permitted.

The update period of history events created for {{Odbpath|path=/history/links}} is controlled by entries in {{Odbpath|path=/history/links periods}}. Numeric values of periods are same as for a [[#Frontend History Event|Frontend History event]]. Numeric value 0 disables the history for a particular event.

See [https://midas.triumf.ca/elog/Midas/1087 MidasElog#1087] for more information.

== Frontend History Event ==

Each [[Frontend Operation#Equipments and Events|Equipment]] has the capability to generate "history data".

; To enable Frontend History Events

: Set the [[/Equipment ODB tree#Log history|Log history]] ODB key of each equipment to be logged '''non-zero'''. (See [[Equipment List Parameters#Log History|Log History parameter]]). (You may need to restart the frontend.)

This causes the event contents to be copied to the ODB [[/Equipment ODB tree#Variables subtree|/Equipment/''<equipment-name>''/Variables subtree]] for the use of the History system. history events are created by parsing the content of {{Odbpath|path=/Equipment/<equipment-name>/variables}}.

For each equipment, history is controlled by the value of {{Odbpath|path=/Equipment/<equipment-name>/common/period}} as follows:
* 0 = history disabled
* 1 = history is enabled
* >1 = history is enabled, throttled down

The throttling is implemented in log_history()/watch_history() by this algorithm:
the very first history event is recorded, then all changed to the data are ignored until "period" seconds has elapsed. Then the next history event will be recorded, and following
changes will be ignored until "period" second elapses, and so forth. Period value "1" has special meaning - there is no throttling, all history events are logged.

This information is from [https://midas.triumf.ca/elog/Midas/1087 MidasElog#1087].

=== Sequence for frontend History Event ===
The main steps for the frontend History Event is as follows:

* The user code in the frontend equipment Event Readout routines reads the data, placing it into a MIDAS data bank
* In mfe.c, if the key [[/Equipment ODB tree#Log history|Log history]] > 0 (i.e. the history system is enabled for this equipment), this data bank is written into ODB (by update_odb()).
* This ODB write triggers an ODB hot-link into mlogger
* The hot-link calls mlogger.c::log_history(), which calls hs_write() to write the data into the history file. The frequency of the history writes is specified by the number of seconds between writes stored in the ODB key [[/Equipment ODB tree#Log history|Log history]].
* history.c::hs_write() or history_odbc::hs_write_odbc() writes the data into a history file or into an SQL database.

= How to Write History events =
* Enable desired [[#To enable Frontend History Events|Frontend]] and/or [[#To enable Virtual History Events|Virtual]] history events
* [[#Select the history driver|Select the history driver]]
* Follow instructions below for selected driver:
** [[#Write MIDAS-History events|Write MIDAS-history events]]
** [[#Write ODBC-History events|Write ODBC-history events]]
** [[#Write SQLITE-History events|Write SQLITE-history events]]
** [[#Write MYSQL-History events|Write MYSQL-history events]]
** [[#Write FILE-History events|Write FILE-history events]]

== Select the history driver ==

* Select the '''[[#History drivers|history driver]]''' and therefore the [[Keys in the ODB /Logger/History subtree#<history-logging-channel> subtrees|history-logging-channel]] to use to write the MIDAS-history files.
* set [[Keys in the ODB /Logger/History subtree#Active|Active]] ODB key in the selected [[Keys in the ODB /Logger/History subtree#<history-logging-channel> subtrees|history-logging-channel]] to "y". All other logging channels should be disabled.
* set [[/History ODB tree#LoggerHistoryChannel|LoggerHistoryChannel]] ODB key to the selected [[#History drivers|History channel]].

== Write MIDAS-History events ==
* Enable desired [[#To enable Frontend History Events|Frontend]] and/or [[#To enable Virtual History Events|Virtual]] history events
* Perform steps in [[#Select the history driver|Select History Driver]] with [[#History drivers|history driver]] "MIDAS".
* Restart [[mhttpd]] and [[mlogger]], start the run.

MIDAS-history files will be saved into the [[#Location of History Files|MIDAS-history data directory]].

== Write ODBC-History events ==

See [[#MIDAS ODBC SQL History system|MIDAS ODBC SQL History system]].
* If necessary, [[#Set up a MySQL database|set up a MySQL database]].
* Enable desired virtual and frontend history events (see [[#Virtual History Event|Virtual history events]] and [[#Frontend History Event|Frontend history events]]).
* Perform steps in [[#Select the history driver|Select History Driver]] with [[#History drivers|history driver]] "ODBC".
* Check ODB Key [[Keys in the ODB /Logger/History subtree#Writer_ODBC_DSN|Writer_ODBC_DSN]] is set to "history writer" and Key [[Keys in the ODB /Logger/History subtree#Reader_ODBC_DSN|Reader_ODBC_DSN]] is set to "history reader" (the default values)
* set [[Keys in the ODB /Logger/History subtree#Debug|Debug]] to 1 if desired
* restart [[mlogger]] in verbose mode (i.e. mlogger -v), and observe how it issues SQL commands to create the tables and columns corresponding to MIDAS-history events and tags.
* restart [[mhttpd]]

'''Notes'''
# [[mlogger]] is programmed to raise alarms if connection to SQL database is interrupted or if some events cannot be written into the database (i.e. data type mismatch, SQL syntax errors, etc). The traditional MIDAS-history never raised alarms because it "never failed" - other than from "disk full" errors, which are immediately obvious.
# With ODBC History enabled, [[mhttpd]] will only use history information from the SQL database to make history plots and to extract history variable names for the history plot editor. The ODB {{Odbpath|path=/History}} subtrees[[/History ODB tree#Tags subtree|Tags]] and [[/History ODB tree#Events subtree|Events]] are not used.
# variable names presented to the user may change from MIDAS names to SQL names (use the history panel "Label" text fields to create permanent plot labels).

== Write SQLITE-history events ==

No special configuration required.

* Enable desired virtual and frontend history events (see [[#Virtual History Event|Virtual history events]] and [[#Frontend History Event|Frontend history events]]).
* Perform steps to [[#Select the history driver|Select History Driver]] with [[#History drivers|history driver]] "SQLITE".

* Create a directory for storage of sqlite history files
* save its path in the ODB key [[Keys in the ODB /Logger/History subtree#Sqlite dir|Sqlite dir]]
* restart [[mhttpd]] and [[mlogger]]

In the history data directory (in [[Keys in the ODB /Logger/History subtree#Sqlite dir|Sqlite dir]]) you should see files with name "mh_xxx.sqlite3" corresponding to your equipment and variable names.

== Write MYSQL-history events ==

* Enable desired virtual and frontend history events (see [[#Virtual History Event|Virtual history events]] and [[#Frontend History Event|Frontend history events]]).
* Perform steps to [[#Select the history driver|Select History Driver]] with [[#History drivers|history driver]] "MYSQL".
* Read the following instructions to [[setup MySQL database for MIDAS]]:

* enable MYSQL history in mlogger:
* restart mlogger, restart mhttpd
...........

== Write FILE-history events ==

* Enable desired virtual and frontend history events (see [[#Virtual History Event|Virtual history events]] and [[#Frontend History Event|Frontend history events]]).
* Perform steps to [[#Select the history driver|Select History Driver]] with [[#History drivers|history driver]] "SQLITE".
* Assign directory path for saved files using the ODB key [[Keys in the ODB /Logger/History subtree#History dir|History dir]].
...........

<br>
---------------
<br>

= History Events =
The following information is derived from [https://midas.triumf.ca/elog/Midas/1087 MidasElog#1087].

A "history event" is a history atomic unit of data. Associated with each history event is a timestamp (unix time),
a name (limited to NAME_LENGTH in the old history) and a list of history tags that describe the individual data
values inside the history event.

When making history plots in [[mhttpd]] (see [[History Page]]) for each curve on the plot, one selects a history event (from the list
of currently active events, recently active events or the list of all events that ever existed), then from the list of tags
inside the history event one selects the particular variable that will be plotted.

== Old MIDAS history events ==
In the old MIDAS history, all history events are written into one history file (.hst file + optional .def and .idx event definition and time index files
which can be/are regenerated automatically from the .hst file). History events are identified by 16-bit history event IDs, the persistent mapping
from history event names and the 16-bit history event IDs is stored in ODB subtree {{Odbpath|path=/History/Events}}. In addition the list of all known history event tags is
stored in ODB subtree {{Odbpath|path=/History/Tags}}. For per-equipment history, the 16-bit history event ID is the value of ODB key [[/Equipment ODB tree#Event ID|/Equipment/<equipment-name>/Common/Event ID]].

== SQL history events ==
In the SQL history (MySQL, SQLITE, etc), each history event is an SQL table. The history event tags are the SQL table columns.

== FILE history events ==
In the new FILE history, each history event is written into a separate file, tag definition are recorded in text format in the file header, history event
data is appended to the file in binary format (fixed record size). If the history event definition is changed, a new file will be started.

= History Event Construction =
The [[mlogger]] creates [[#Frontend history event]]s in open_history() by parsing the ODB subtree {{Odbpath|path=/equipment/<equipment-name>/variables}}. Each ODB entry under the {{Odbpath|path=variables/}} subtree is referred to as a "variable". Each variable can be a single ODB value, an array of ODB values, or a subdirectory (corresponding to TID_STRUCT structured data banks). As each variable is processed, one or more tags are created to describe it.

Single ODB values will generally produce a single tag, while arrays can produce (depending whether the array is "[[#Named Arrays|Named]]" or not)
* multiple tags - one per array element ("Named" array) or
* one single tag - describing the whole array

The code can generate two types of history:
<dl>
<dt> "per-equipment" history</dt>
<dd> will have the tags for all variables concatenated together into one single history event</dd>
<dt> "per-variable" history</dt>
<dd> will have one history event defined for each variable. Inside could be one tag (for single odb values and unnamed array) or multiple tags (for named arrays and structured data
banks).</dd>
</dl>

Per-equipment history is the original MIDAS history implementation. Per-variable history was added to permit efficient data storage in SQL tables. Its initial implementation used 1 ODB hotlink for each variable and it was easy to exceed the maximum permitted number of ODB hotlinks (db_open_record()). To reduce consumption of hotlinks, db_watch() has been implemented and now per-variable history only uses 1 ODB hotlink per equipment.

;NOTE
: With db_watch, per-equipment history is no longer available. '''Per-variable history is the new default''' (and the only option).

[https://midas.triumf.ca/elog/Midas/1087 MidasElog#1087]

= Construction of history event tags =
Currently,
* single ODB values are treated as arrays of length "1"
* string ODB entries are not permitted

History event tags are constructed in the [[mlogger]] add_equipment(). Presently there are some quirks with the generation of history event tags (see [https://midas.triumf.ca/elog/Midas/1088]),
e.g. single odb values are treated as arrays of length "1" and "string" ODB entries are not permitted.

For variables of type TID_KEY (subdirectories, corresponding to TID_STRUCT structured banks), one tag is generated for each subdirectory entry. Tag names for {{Odbpath|path=/equipment/<equipment-name>/variables/aaa/bbb}} will be "aaa_bbb" (with an underscore). Subdirectory entries of type TID_KEY and TID_LINK are currently not supported.

For "[[#Named Arrays|Named Arrays]]", individual tags are generated for each array element. Tag names are taken from the ''Names'' array. For empty tag names (empty names array), tags are "aaa_0", "aaa_1", etc (for {{Odbpath|path=/equipment/<equipment-name>/variables/aaa}}).

For "[[#Named Arrays|Single Names Arrays]]", tag names have the variable name appended (with a space), for {{Odbpath|path=/equipment/<equipment-name>/variables/aaa}} and an empty names array, tags will be "aaa_0 aaa", "aaa_1 aaa", etc. For a populated names array, the tags will be "name0 aaa", "name1 aaa", etc.

For unnamed arrays and single odb variables (in ODB, single odb variables are arrays of length 1), a single tag is generated.

== Named Arrays ==

'''Named Arrays''' have a "Names" entry in {{Odbpath|path=/equipment/<equipment-name>/settings}}.

For example, to add names to {{Odbpath|path=/equipment/<equipment-name>/variables/aaa}}, create a string array {{Odbpath|path=/equipment/<equipment-name>/settings/names aaa}}. The names array should be at least as long as the corresponding data array. Individual entries in the names array can be left blank (tag names will be "aaa_0", "aaa_1", etc). Duplicate tag names are not permitted.

A single "Names" entry can be created to name all arrays in variables with the same names ('''"Single Names"''').

Create {{Odbpath|path=/equipment/<equipment-name>/settings/names}} and arrays {{Odbpath|path=/equipment/<equipment-name>/variables/aaa}} and {{Odbpath|path=/equipment/<equipment-name>/variables/bbb}} will have
history tags "name0 aaa", "name1 aaa", "name0 bbb", "name1 bbb", etc. If "names" are left blank, tag names will be "aaa_0 aaa", "aaa_1 aaa", "bbb_0 bbb", "bbb_1 bbb", etc.

In the [[mhttpd]] web server [[History Page]] variables viewer, "single name" arrays are displayed in a 2D table.

This information is from [https://midas.triumf.ca/elog/Midas/1088].

= Historical note on MIDAS-History Tag names =
Tags are only used for MIDAS-history events.

The mlogger/mhttpd implementation of <span style="color:purple; font-style:italic">/History/Tags</span> has proved troublesome and we are moving away from it.

;During the transition period:
: Implementations starting with Version 4435 of [[mlogger]] and [[mhttpd]] (i.e. "new" versions) will now work '''without''' <span style="color:purple; font-style:italic">/History/Tags</span>. The history tags are read directly from the history files themselves. To run in this mode, set ODB key [[/History ODB tree#DisableTags|/History/DisableTags]] to "y". Existing [[/History ODB tree#Tags subtree|/History/Tags subtree]] will be automatically deleted.

Two downsides to running like this are
* it is slower and tags become non-persistent
* if some frontends have not been running for a while, their variables may vanish from the history panel editor.

== New-style History Tags ==
For the above two reasons, using <span style="color:purple; font-style:italic">/History/Tags</span> is still recommended.
However, the format of the tags is now changed to simplify management and reduce ODB size.

; For NEW implementations
: the key [[/History ODB tree#CreateOldTags|CreateOldTags]] will not be present. "New-style" history tags will be created automatically in the [[/History ODB tree#Tags subtree|Tags subtree]].

;If you have old-style tags
: [[mlogger]] will automatically convert the old tags to this new format (this is why you should make a backup of your ODB).

** using "old" (i.e. pre Version 4435) mlogger with "new" mhttpd is OK: new mhttpd understands both formats of /History/Tags.
** using "old" mhttpd with "new" mlogger is OK provided the ODB key [[/History ODB tree#CreateOldTags|/History/CreateOldTags]] is set to "y" before starting mlogger.

== Old-style History Tags ==
Prior to versions 4435 of [[mlogger]] and [[mhttpd]], old-style history tags were generated.
See
* http://ladd00.triumf.ca/~daqweb/doc/midas-old/html/F_History_logging.html
for details.

<br>
---------------
<br>

= MIDAS ODBC SQL History system =

This section describes the internal workings of the MIDAS SQL history system.

The SQL history system implements the MIDAS history using a SQL database for data storage instead of flat binary files.

The SQL history code lives in a separate files history.h and history_sql.cxx. The present implementation uses the ODBC API for accessing SQL databases.
MySQL is fully supported, Pgsql support is partially implemented, support for arbitrary ODBC drivers should be easy to add in history_sql and most SQL operations are done using an interface class.

Preliminary testing with MySQL and PgSQL indicates that disk space requirements are the same for both storage mechanisms using flat binary files or either of the 2 SQL databases. All 3 tested storage mechanisms store the data uncompressed. (Observed gzip-1 compression ratios from CERN-ALPHA history files are 50% or better).

Preliminary testing also indicates that MySQL database is "not too slow" for use by T2K/ND280 slow control system. Using MySQL, performance seems to improve somewhat compared to traditional MIDAS-history because each SQL table is stored into a separate file, compared to everything-in-one-file in traditional MIDAS-history.

== Set up a MySQL database ==

In order to setup a MySQL database, the following is needed:

* the server hostname and port number
* database name
* user name and password.

It is recommended to create 3 different users:

* a "root" user with full priveleges,
* a "reader" user with read-only priveleges
* and a "writer" user with "insert, create table, and add column" priveleges (no permission to drop table, remove columns, etc)

=== Example ===
#Setup a MySQL database (RHEL/SL 5)

ssh root@localhost
service mysqld start
/usr/bin/mysql_secure_installation (enter current password for root: press <enter>, then accept default answer "Y" to all questions)
mysqladmin -p create history
using mysql-administrator, create 2 users:
history_writer, with database "history" schema privileges: select, insert, create, alter, index
history_reader, with database "history" schema privileges: select

#Setup $HOME/.odbc.ini file:

cat $HOME/.odbc.ini
[history_writer]
Description = history_writer
Driver = /usr/lib64/libmyodbc3.so
SERVER = ladd05.triumf.ca
PORT =
Database = history
OPTION = 3
SOCKET =
User = history_writer
Password = ******
[history_reader]
Description = history_reader
Driver = /usr/lib64/libmyodbc3.so
SERVER = ladd05.triumf.ca
PORT =
Database = history
OPTION = 3
SOCKET =
User = history_reader
Password =

Note: if you are using mhist you will need a separate .odbc.ini file for each user that you have connecting to the database (through MIDAS).

== Layout of SQL tables ==

* One table is created for each history event. Table name is the same as the history event name (as reported by [[mhdump]]).
* one column is created for each history tag (history variable). Column name is the same as the tag name (as reported by mhdump). For array tags, one column is created for each array element: array a[3] will produce columns a_0, a_1 and a_2.
* two special columns are always created: _i_time and _t_time are the timestamps in the UNIX-time integer format and in the SQL "time" format. (A priori, it is not clear which timestamp format is more useful for end users. mhttpd uses the _i_time integer time format). Note that this timestamp is the time when mlogger receives the hot-link signal (see above) and it can be a few seconds behind the time when the user placed the data into the MIDAS bank. For high precision (and sub-second time resolution) time stamps, users should generate their own timestamps and include them as part of the data itself.
* a special table called _history_index is created to remember the mapping between SQL names and MIDAS names for event names, variable name and tag names.

=== Schema Evolution ===

* when new data fields are added, new SQL columns are created automatically
* existing SQL columns are never deleted
* arrays are expanded automatically, and arrays never shrink.
* if the type of a data field changes, a conflict may occur with existing SQL columns, for example if previous type was "FLOAT" and new type is "DOUBLE", the existing SQL columns would have the SQL data type "float", too narrow to store new values. mlogger will complain about this. Such conflicts may be resolved by changing the type of the SQL column using SQL tools. In general, SQL names are created from MIDAS names by replacing all non-alphanumerical characters to an underscore "_". If this procedure creates duplicate column names, the column names and made unique by appending them with a random number, i.e. MIDAS names "A+B" and "A-B" will be translated to "A_B" and "A_B_12345" (random integer). True MIDAS names are saved into the _history_index.

[[Category:history]]

Setup MySQL database for MIDAS

2025-04-25T22:45:44Z

Lindner: /* Do these steps as the experiment user */

Setup MySQL database for MIDAS

2025-04-25T22:44:39Z

Lindner: /* Do these steps as the experiment user */

This page provides instructions for setting up a MySQL database to use with the MIDAS history logging.

Setting up the MySQL database is highly dependent on what OS and OS version you are using. Most of the setup work is just MySQL related. For this reason we provide below instructions for the MySQL database setup on a couple different OS. You will probably need to modify these instructions for your OS. Here's the some installation instructions:

* [[#Setting up MySQL on Ubuntu 24.04 LTS | Setting up MySQL on Ubuntu 24.04 LTS ]]
* [[#Setting up MySQL using mariadb on macbook | Setting up MySQL using mariadb on macbook]]

== Setting up MySQL on Ubuntu 24.04 LTS ==

This instructions assume you have root privileges and install MySQL as ROOT.

=== Do these steps as root user ===

Install the MySQL server software.

<pre>
apt install mysql-server
</pre>

You will find mysql has started after installation

<pre>
root@mpmt-daq03:/home1/wheel# service mysql status
mysql.service - MySQL Community Server
Loaded: loaded (/usr/lib/systemd/system/mysql.service; enabled; preset: enabled)
Active: active (running) since Fri 2025-04-25 15:12:58 PDT; 4s ago
Process: 153129 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=0>
Main PID: 153137 (mysqld)
Status: "Server is operational"
</pre>

Default configuration only allows MySQL connections to localhost. Good
<pre>
root@mpmt-daq03:/home1/wheel# grep bind-add /etc/mysql/mysql.conf.d/mysqld.cnf
bind-address = 127.0.0.1
mysqlx-bind-address = 127.0.0.1
</pre>

Login to MySQL as root and create the MySQL accounts

<pre>
mysql -h localhost
...
mysql> CREATE DATABASE IF NOT EXISTS history;
mysql> CREATE USER history_reader@'localhost' IDENTIFIED BY 'CHANGE_THIS_PASSWORD!';
mysql> GRANT SELECT ON history.* TO history_reader@'localhost';
mysql> CREATE USER 'history_writer'@'localhost' IDENTIFIED BY 'CHANGE_THIS_PASSWORD!';
mysql> GRANT SELECT,INSERT,CREATE,ALTER,INDEX ON history.* TO 'history_writer'@'localhost';
mysql> flush privileges;
</pre>

ATTENTION: change the account password listed above to something different!

Check the socket to connect to:

<pre>
root@mpmt-daq03:/home1/wheel# grep sock /etc/mysql/my.cnf
# Port or socket location where to connect
socket = /run/mysqld/mysqld.sock
</pre>

=== Do these steps as the experiment user ===

Rebuild MIDAS with support for MySQL. If this works correctly the build will notice MySQL is installed. For instance, with cmake we will see

<pre>
-- MIDAS: Found MySQL version 8.0.41
</pre>

Go to online directory and create files so MIDAS knows accounts to talk to ODB

<pre>

testmysql@mpmt-daq03:~/online$ cd ~/online
testmysql@mpmt-daq03:~/online$ emacs -nw mysql_reader.txt
testmysql@mpmt-daq03:~/online$ emacs -nw mysql_writer.txt
testmysql@mpmt-daq03:~/online$ more mysql_*.txt
::::::::::::::
mysql_reader.txt
::::::::::::::
database=history
socket=/run/mysqld/mysqld.sock
user=history_reader
password=CHANGE_THIS_PASSWORD!
::::::::::::::
mysql_writer.txt
::::::::::::::
database=history
socket=/run/mysqld/mysqld.sock
user=history_writer
password=CHANGE_THIS_PASSWORD!
</pre>

Again, set the correct password in these files.

Change the mlogger settings so we use the MySQL for the history logging:
* disable the previous history logger
* enable the MySQL history logging:
<pre>
set "/Logger/History/MYSQL/Active" y
</pre>

Restart the mlogger on the command line and see that some MySQL tables are created and columns added:

<pre>
testmysql@mpmt-daq03:~/online$ mlogger
[Logger,INFO] Per-variable history is enabled
[Logger,ERROR] [history_schema.cxx:1258:Mysql::Prepare,ERROR] mysql_query(SELECT event_name, table_name, itimestamp FROM _history_index WHERE table_name!='';) error 1146 (Table 'history._history_index' doesn't exist)
[Logger,INFO] Adding SQL table "myperiodicequipment_pydt"
[Logger,INFO] Adding SQL table "_history_index"
[Logger,INFO] Adding column "event_name" to SQL table "_history_index", status 1
[Logger,INFO] Adding column "table_name" to SQL table "_history_index", status 1
...
[Logger,INFO] Adding column "fluxgate0" to SQL table "myperiodicequipment_pydt", status 1
[Logger,INFO] Adding column "fluxgate1" to SQL table "myperiodicequipment_pydt", status 1
[Logger,INFO] Adding column "valvex" to SQL table "myperiodicequipment_pydt", status 1
[Logger,INFO] Adding column "valvey" to SQL table "myperiodicequipment_pydt", status 1
[Logger,INFO] Adding column "t5" to SQL table "myperiodicequipment_pydt", status 1
...
</pre>

Login directly to the MySQL database and make sure the data is actually being logged

<pre>
mysql> select _i_time,fluxgate0 from myperiodicequipment_pydt where _i_time <1745620795;
+------------+-----------+
| _i_time | fluxgate0 |
+------------+-----------+
| 1745620782 | 6.20209 |
| 1745620783 | 6.3519 |
| 1745620784 | 6.5677 |
| 1745620785 | 7.81413 |
| 1745620786 | 5.41127 |
| 1745620787 | 7.31112 |
| 1745620788 | 6.14285 |
| 1745620789 | 7.20425 |
| 1745620790 | 6.9303 |
| 1745620791 | 7.08178 |
| 1745620792 | 7.44422 |
| 1745620794 | 5.32677 |
+------------+-----------+
12 rows in set (0.00 sec)
</pre>

== Setting up MySQL using mariadb on macbook ==

* install mariadb-10.4
<pre>
macos:
sudo port install mariadb-10.4
ln -s ln -fs /opt/local/lib/mariadb-10.4/bin/mysql_config ~/bin/
mysql_config --version
10.4.13
</pre>
* rebuild midas, check the cmake finds the correct mysql: MIDAS: Found MySQL version 10.4.13
* create $HOME/.my.cnf
<pre>
[mysqld]
#port=3306
socket=/Users/olchansk/mysql/mysql.sock
datadir=/Users/olchansk/mysql/db
pid-file=/Users/olchansk/mysql/mysqld.pid

[client]
#port=3306
socket=/Users/olchansk/mysql/mysql.sock
</pre>
* mkdir $HOME/mysql
* start mysql:
<pre>
macos: /opt/local/lib/mariadb-10.4/bin/mysqld
</pre>
* configure the database: start the "mysql" utility (/opt/local/lib/mariadb-10.4/bin/mysql)
<pre>
NOTE: instructions thanks to Ben S.
NOTE: use some other passwords instead of the example "reader_password" and "writer_password"

MariaDB [(none)]> CREATE DATABASE IF NOT EXISTS history;
Query OK, 1 row affected (0.003 sec)

MariaDB [(none)]> CREATE USER history_reader@'localhost' IDENTIFIED BY 'reader_password';
Query OK, 0 rows affected (0.027 sec)

MariaDB [(none)]> GRANT SELECT ON history.* TO history_reader@'localhost';
Query OK, 0 rows affected (0.016 sec)

MariaDB [(none)]> CREATE USER 'history_writer'@'localhost' IDENTIFIED BY 'writer_password';
Query OK, 0 rows affected (0.014 sec)

MariaDB [(none)]> GRANT SELECT,INSERT,CREATE,ALTER,INDEX ON history.* TO 'history_writer'@'localhost';
Query OK, 0 rows affected (0.017 sec)

MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.001 sec)
</pre>
* try connecting to both users:
<pre>
/opt/local/lib/mariadb-10.4/bin/mysql -u history_reader -p ### should ask for reader password
/opt/local/lib/mariadb-10.4/bin/mysql -u history_writer -p ### should ask for writer password
</pre>
* create midas config files:
** mysql_writer.txt
<pre>
#server=localhost
#port=xxx
database=history
socket=/Users/olchansk/mysql/mysql.sock
user=history_writer
password=writer_password ### change this!!!
#buffer=1000
</pre>
** mysql_reader.txt
<pre>
#server=localhost
#port=xxx
database=history
socket=/Users/olchansk/mysql/mysql.sock
user=history_reader
password=reader_password ### change this!!!
</pre>
* test MIDAS connection
<pre>
mh2sql --mysql mysql_writer.txt
(there should be no errors printed)
</pre>
* if desired, import midas history from .hst files:
<pre>
mh2sql --mysql mysql_writer.txt *.hst
Reading 130813.hst
[mh2sql,ERROR] [history_schema.cxx:1165:Mysql::Prepare,ERROR] mysql_query(SELECT event_name, table_name, itimestamp FROM _history_index WHERE table_name!='';) error 1146 (Table 'history._history_index' doesn't exist)
[mh2sql,INFO] Adding SQL table "rpcexample", status 311
[mh2sql,INFO] Adding SQL table "rpcexample_1596127680", status 1
[mh2sql,INFO] Adding SQL table "_history_index", status 1
[mh2sql,INFO] Adding column "event_name" to SQL table "_history_index", status 1
...
[mh2sql,INFO] Adding SQL table "run_transitions", status 311
[mh2sql,INFO] Adding SQL table "run_transitions_1596127681", status 1
[mh2sql,INFO] Adding column "state" to SQL table "run_transitions_1596127681", status 1
[mh2sql,INFO] Adding column "run_number" to SQL table "run_transitions_1596127681", status 1
Reading 130815.hst
</pre>

Setup MySQL database for MIDAS

2025-04-25T22:41:58Z

Lindner: /* Do these steps as the experiment user */

This page provides instructions for setting up a MySQL database to use with the MIDAS history logging.

Setting up the MySQL database is highly dependent on what OS and OS version you are using. Most of the setup work is just MySQL related. For this reason we provide below instructions for the MySQL database setup on a couple different OS. You will probably need to modify these instructions for your OS. Here's the some installation instructions:

* [[#Setting up MySQL on Ubuntu 24.04 LTS | Setting up MySQL on Ubuntu 24.04 LTS ]]
* [[#Setting up MySQL using mariadb on macbook | Setting up MySQL using mariadb on macbook]]

== Setting up MySQL on Ubuntu 24.04 LTS ==

This instructions assume you have root privileges and install MySQL as ROOT.

=== Do these steps as root user ===

Install the MySQL server software.

<pre>
apt install mysql-server
</pre>

You will find mysql has started after installation

<pre>
root@mpmt-daq03:/home1/wheel# service mysql status
mysql.service - MySQL Community Server
Loaded: loaded (/usr/lib/systemd/system/mysql.service; enabled; preset: enabled)
Active: active (running) since Fri 2025-04-25 15:12:58 PDT; 4s ago
Process: 153129 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=0>
Main PID: 153137 (mysqld)
Status: "Server is operational"
</pre>

Default configuration only allows MySQL connections to localhost. Good
<pre>
root@mpmt-daq03:/home1/wheel# grep bind-add /etc/mysql/mysql.conf.d/mysqld.cnf
bind-address = 127.0.0.1
mysqlx-bind-address = 127.0.0.1
</pre>

Login to MySQL as root and create the MySQL accounts

<pre>
mysql -h localhost
...
mysql> CREATE DATABASE IF NOT EXISTS history;
mysql> CREATE USER history_reader@'localhost' IDENTIFIED BY 'CHANGE_THIS_PASSWORD!';
mysql> GRANT SELECT ON history.* TO history_reader@'localhost';
mysql> CREATE USER 'history_writer'@'localhost' IDENTIFIED BY 'CHANGE_THIS_PASSWORD!';
mysql> GRANT SELECT,INSERT,CREATE,ALTER,INDEX ON history.* TO 'history_writer'@'localhost';
mysql> flush privileges;
</pre>

ATTENTION: change the account password listed above to something different!

Check the socket to connect to:

<pre>
root@mpmt-daq03:/home1/wheel# grep sock /etc/mysql/my.cnf
# Port or socket location where to connect
socket = /run/mysqld/mysqld.sock
</pre>

=== Do these steps as the experiment user ===

Rebuild MIDAS with support for MySQL. If this works correctly the build will notice MySQL is installed. For instance, with cmake we will see

<pre>
-- MIDAS: Found MySQL version 8.0.41
</pre>

Go to online directory and create files so MIDAS knows accounts to talk to ODB

<pre>

testmysql@mpmt-daq03:~/online$ cd ~/online
testmysql@mpmt-daq03:~/online$ emacs -nw mysql_reader.txt
testmysql@mpmt-daq03:~/online$ emacs -nw mysql_writer.txt
testmysql@mpmt-daq03:~/online$ more mysql_*.txt
::::::::::::::
mysql_reader.txt
::::::::::::::
database=history
socket=/run/mysqld/mysqld.sock
user=history_reader
password=CHANGE_THIS_PASSWORD!
::::::::::::::
mysql_writer.txt
::::::::::::::
database=history
socket=/run/mysqld/mysqld.sock
user=history_writer
password=CHANGE_THIS_PASSWORD!
</pre>

Again, set the correct password in these files.

Change the mlogger settings so we use the MySQL for the history logging:
* disable the previous history logger
* enable the MySQL history logging:
<pre>
set "/Logger/History/MYSQL/Active" y
</pre>

Restart the mlogger on the command line and see that some MySQL tables are created and columns added:

<pre>
testmysql@mpmt-daq03:~/online$ mlogger
[Logger,INFO] Per-variable history is enabled
[Logger,ERROR] [history_schema.cxx:1258:Mysql::Prepare,ERROR] mysql_query(SELECT event_name, table_name, itimestamp FROM _history_index WHERE table_name!='';) error 1146 (Table 'history._history_index' doesn't exist)
[Logger,INFO] Adding SQL table "myperiodicequipment_pydt"
[Logger,INFO] Adding SQL table "_history_index"
[Logger,INFO] Adding column "event_name" to SQL table "_history_index", status 1
[Logger,INFO] Adding column "table_name" to SQL table "_history_index", status 1
...
[Logger,INFO] Adding column "fluxgate0" to SQL table "myperiodicequipment_pydt", status 1
[Logger,INFO] Adding column "fluxgate1" to SQL table "myperiodicequipment_pydt", status 1
[Logger,INFO] Adding column "valvex" to SQL table "myperiodicequipment_pydt", status 1
[Logger,INFO] Adding column "valvey" to SQL table "myperiodicequipment_pydt", status 1
[Logger,INFO] Adding column "t5" to SQL table "myperiodicequipment_pydt", status 1
...
</pre>

== Setting up MySQL using mariadb on macbook ==

* install mariadb-10.4
<pre>
macos:
sudo port install mariadb-10.4
ln -s ln -fs /opt/local/lib/mariadb-10.4/bin/mysql_config ~/bin/
mysql_config --version
10.4.13
</pre>
* rebuild midas, check the cmake finds the correct mysql: MIDAS: Found MySQL version 10.4.13
* create $HOME/.my.cnf
<pre>
[mysqld]
#port=3306
socket=/Users/olchansk/mysql/mysql.sock
datadir=/Users/olchansk/mysql/db
pid-file=/Users/olchansk/mysql/mysqld.pid

[client]
#port=3306
socket=/Users/olchansk/mysql/mysql.sock
</pre>
* mkdir $HOME/mysql
* start mysql:
<pre>
macos: /opt/local/lib/mariadb-10.4/bin/mysqld
</pre>
* configure the database: start the "mysql" utility (/opt/local/lib/mariadb-10.4/bin/mysql)
<pre>
NOTE: instructions thanks to Ben S.
NOTE: use some other passwords instead of the example "reader_password" and "writer_password"

MariaDB [(none)]> CREATE DATABASE IF NOT EXISTS history;
Query OK, 1 row affected (0.003 sec)

MariaDB [(none)]> CREATE USER history_reader@'localhost' IDENTIFIED BY 'reader_password';
Query OK, 0 rows affected (0.027 sec)

MariaDB [(none)]> GRANT SELECT ON history.* TO history_reader@'localhost';
Query OK, 0 rows affected (0.016 sec)

MariaDB [(none)]> CREATE USER 'history_writer'@'localhost' IDENTIFIED BY 'writer_password';
Query OK, 0 rows affected (0.014 sec)

MariaDB [(none)]> GRANT SELECT,INSERT,CREATE,ALTER,INDEX ON history.* TO 'history_writer'@'localhost';
Query OK, 0 rows affected (0.017 sec)

MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.001 sec)
</pre>
* try connecting to both users:
<pre>
/opt/local/lib/mariadb-10.4/bin/mysql -u history_reader -p ### should ask for reader password
/opt/local/lib/mariadb-10.4/bin/mysql -u history_writer -p ### should ask for writer password
</pre>
* create midas config files:
** mysql_writer.txt
<pre>
#server=localhost
#port=xxx
database=history
socket=/Users/olchansk/mysql/mysql.sock
user=history_writer
password=writer_password ### change this!!!
#buffer=1000
</pre>
** mysql_reader.txt
<pre>
#server=localhost
#port=xxx
database=history
socket=/Users/olchansk/mysql/mysql.sock
user=history_reader
password=reader_password ### change this!!!
</pre>
* test MIDAS connection
<pre>
mh2sql --mysql mysql_writer.txt
(there should be no errors printed)
</pre>
* if desired, import midas history from .hst files:
<pre>
mh2sql --mysql mysql_writer.txt *.hst
Reading 130813.hst
[mh2sql,ERROR] [history_schema.cxx:1165:Mysql::Prepare,ERROR] mysql_query(SELECT event_name, table_name, itimestamp FROM _history_index WHERE table_name!='';) error 1146 (Table 'history._history_index' doesn't exist)
[mh2sql,INFO] Adding SQL table "rpcexample", status 311
[mh2sql,INFO] Adding SQL table "rpcexample_1596127680", status 1
[mh2sql,INFO] Adding SQL table "_history_index", status 1
[mh2sql,INFO] Adding column "event_name" to SQL table "_history_index", status 1
...
[mh2sql,INFO] Adding SQL table "run_transitions", status 311
[mh2sql,INFO] Adding SQL table "run_transitions_1596127681", status 1
[mh2sql,INFO] Adding column "state" to SQL table "run_transitions_1596127681", status 1
[mh2sql,INFO] Adding column "run_number" to SQL table "run_transitions_1596127681", status 1
Reading 130815.hst
</pre>

Setup MySQL database for MIDAS

2025-04-25T22:40:43Z

Lindner: /* Do these steps as the experiment user */

This page provides instructions for setting up a MySQL database to use with the MIDAS history logging.

Setting up the MySQL database is highly dependent on what OS and OS version you are using. Most of the setup work is just MySQL related. For this reason we provide below instructions for the MySQL database setup on a couple different OS. You will probably need to modify these instructions for your OS. Here's the some installation instructions:

* [[#Setting up MySQL on Ubuntu 24.04 LTS | Setting up MySQL on Ubuntu 24.04 LTS ]]
* [[#Setting up MySQL using mariadb on macbook | Setting up MySQL using mariadb on macbook]]

== Setting up MySQL on Ubuntu 24.04 LTS ==

This instructions assume you have root privileges and install MySQL as ROOT.

=== Do these steps as root user ===

Install the MySQL server software.

<pre>
apt install mysql-server
</pre>

You will find mysql has started after installation

<pre>
root@mpmt-daq03:/home1/wheel# service mysql status
mysql.service - MySQL Community Server
Loaded: loaded (/usr/lib/systemd/system/mysql.service; enabled; preset: enabled)
Active: active (running) since Fri 2025-04-25 15:12:58 PDT; 4s ago
Process: 153129 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=0>
Main PID: 153137 (mysqld)
Status: "Server is operational"
</pre>

Default configuration only allows MySQL connections to localhost. Good
<pre>
root@mpmt-daq03:/home1/wheel# grep bind-add /etc/mysql/mysql.conf.d/mysqld.cnf
bind-address = 127.0.0.1
mysqlx-bind-address = 127.0.0.1
</pre>

Login to MySQL as root and create the MySQL accounts

<pre>
mysql -h localhost
...
mysql> CREATE DATABASE IF NOT EXISTS history;
mysql> CREATE USER history_reader@'localhost' IDENTIFIED BY 'CHANGE_THIS_PASSWORD!';
mysql> GRANT SELECT ON history.* TO history_reader@'localhost';
mysql> CREATE USER 'history_writer'@'localhost' IDENTIFIED BY 'CHANGE_THIS_PASSWORD!';
mysql> GRANT SELECT,INSERT,CREATE,ALTER,INDEX ON history.* TO 'history_writer'@'localhost';
mysql> flush privileges;
</pre>

ATTENTION: change the account password listed above to something different!

Check the socket to connect to:

<pre>
root@mpmt-daq03:/home1/wheel# grep sock /etc/mysql/my.cnf
# Port or socket location where to connect
socket = /run/mysqld/mysqld.sock
</pre>

=== Do these steps as the experiment user ===

Rebuild MIDAS with support for MySQL. If this works correctly the build will notice MySQL is installed. For instance, with cmake we will see

<pre>
-- MIDAS: Found MySQL version 8.0.41
</pre>

Go to online directory and create files so MIDAS knows accounts to talk to ODB

<pre>

testmysql@mpmt-daq03:~/online$ cd ~/online
testmysql@mpmt-daq03:~/online$ emacs -nw mysql_reader.txt
testmysql@mpmt-daq03:~/online$ emacs -nw mysql_writer.txt
testmysql@mpmt-daq03:~/online$ more mysql_*.txt
::::::::::::::
mysql_reader.txt
::::::::::::::
database=history
socket=/run/mysqld/mysqld.sock
user=history_reader
password=CHANGE_THIS_PASSWORD!
::::::::::::::
mysql_writer.txt
::::::::::::::
database=history
socket=/run/mysqld/mysqld.sock
user=history_writer
password=CHANGE_THIS_PASSWORD!
</pre>

Again, set the correct password in these files.

Change the mlogger settings so we use the MySQL for the history logging:
* disable the previous history logger
* enable the MySQL history logging:
<pre>
set "/Logger/History/MYSQL/Active" y
</pre>

Restart the mlogger on the command line and see that some MySQL tables are created and columns added:

<pre>
testmysql@mpmt-daq03:~/online$ mlogger
[Logger,INFO] Per-variable history is enabled
[Logger,ERROR] [history_schema.cxx:1258:Mysql::Prepare,ERROR] mysql_query(SELECT event_name, table_name, itimestamp FROM _history_index WHERE table_name!='';) error 1146 (Table 'history._history_index' doesn't exist)
[Logger,INFO] Adding SQL table "myperiodicequipment_pydt"
[Logger,INFO] Adding SQL table "_history_index"
[Logger,INFO] Adding column "event_name" to SQL table "_history_index", status 1
[Logger,INFO] Adding column "table_name" to SQL table "_history_index", status 1
[Logger,INFO] Adding column "tag_name" to SQL table "_history_index", status 1
[Logger,INFO] Adding column "tag_type" to SQL table "_history_index", status 1
...
</pre>

== Setting up MySQL using mariadb on macbook ==

* install mariadb-10.4
<pre>
macos:
sudo port install mariadb-10.4
ln -s ln -fs /opt/local/lib/mariadb-10.4/bin/mysql_config ~/bin/
mysql_config --version
10.4.13
</pre>
* rebuild midas, check the cmake finds the correct mysql: MIDAS: Found MySQL version 10.4.13
* create $HOME/.my.cnf
<pre>
[mysqld]
#port=3306
socket=/Users/olchansk/mysql/mysql.sock
datadir=/Users/olchansk/mysql/db
pid-file=/Users/olchansk/mysql/mysqld.pid

[client]
#port=3306
socket=/Users/olchansk/mysql/mysql.sock
</pre>
* mkdir $HOME/mysql
* start mysql:
<pre>
macos: /opt/local/lib/mariadb-10.4/bin/mysqld
</pre>
* configure the database: start the "mysql" utility (/opt/local/lib/mariadb-10.4/bin/mysql)
<pre>
NOTE: instructions thanks to Ben S.
NOTE: use some other passwords instead of the example "reader_password" and "writer_password"

MariaDB [(none)]> CREATE DATABASE IF NOT EXISTS history;
Query OK, 1 row affected (0.003 sec)

MariaDB [(none)]> CREATE USER history_reader@'localhost' IDENTIFIED BY 'reader_password';
Query OK, 0 rows affected (0.027 sec)

MariaDB [(none)]> GRANT SELECT ON history.* TO history_reader@'localhost';
Query OK, 0 rows affected (0.016 sec)

MariaDB [(none)]> CREATE USER 'history_writer'@'localhost' IDENTIFIED BY 'writer_password';
Query OK, 0 rows affected (0.014 sec)

MariaDB [(none)]> GRANT SELECT,INSERT,CREATE,ALTER,INDEX ON history.* TO 'history_writer'@'localhost';
Query OK, 0 rows affected (0.017 sec)

MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.001 sec)
</pre>
* try connecting to both users:
<pre>
/opt/local/lib/mariadb-10.4/bin/mysql -u history_reader -p ### should ask for reader password
/opt/local/lib/mariadb-10.4/bin/mysql -u history_writer -p ### should ask for writer password
</pre>
* create midas config files:
** mysql_writer.txt
<pre>
#server=localhost
#port=xxx
database=history
socket=/Users/olchansk/mysql/mysql.sock
user=history_writer
password=writer_password ### change this!!!
#buffer=1000
</pre>
** mysql_reader.txt
<pre>
#server=localhost
#port=xxx
database=history
socket=/Users/olchansk/mysql/mysql.sock
user=history_reader
password=reader_password ### change this!!!
</pre>
* test MIDAS connection
<pre>
mh2sql --mysql mysql_writer.txt
(there should be no errors printed)
</pre>
* if desired, import midas history from .hst files:
<pre>
mh2sql --mysql mysql_writer.txt *.hst
Reading 130813.hst
[mh2sql,ERROR] [history_schema.cxx:1165:Mysql::Prepare,ERROR] mysql_query(SELECT event_name, table_name, itimestamp FROM _history_index WHERE table_name!='';) error 1146 (Table 'history._history_index' doesn't exist)
[mh2sql,INFO] Adding SQL table "rpcexample", status 311
[mh2sql,INFO] Adding SQL table "rpcexample_1596127680", status 1
[mh2sql,INFO] Adding SQL table "_history_index", status 1
[mh2sql,INFO] Adding column "event_name" to SQL table "_history_index", status 1
...
[mh2sql,INFO] Adding SQL table "run_transitions", status 311
[mh2sql,INFO] Adding SQL table "run_transitions_1596127681", status 1
[mh2sql,INFO] Adding column "state" to SQL table "run_transitions_1596127681", status 1
[mh2sql,INFO] Adding column "run_number" to SQL table "run_transitions_1596127681", status 1
Reading 130815.hst
</pre>

Setup MySQL database for MIDAS

2025-04-25T22:34:04Z

Lindner: /* Do these steps as the experiment user */

This page provides instructions for setting up a MySQL database to use with the MIDAS history logging.

Setting up the MySQL database is highly dependent on what OS and OS version you are using. Most of the setup work is just MySQL related. For this reason we provide below instructions for the MySQL database setup on a couple different OS. You will probably need to modify these instructions for your OS. Here's the some installation instructions:

* [[#Setting up MySQL on Ubuntu 24.04 LTS | Setting up MySQL on Ubuntu 24.04 LTS ]]
* [[#Setting up MySQL using mariadb on macbook | Setting up MySQL using mariadb on macbook]]

== Setting up MySQL on Ubuntu 24.04 LTS ==

This instructions assume you have root privileges and install MySQL as ROOT.

=== Do these steps as root user ===

Install the MySQL server software.

<pre>
apt install mysql-server
</pre>

You will find mysql has started after installation

<pre>
root@mpmt-daq03:/home1/wheel# service mysql status
mysql.service - MySQL Community Server
Loaded: loaded (/usr/lib/systemd/system/mysql.service; enabled; preset: enabled)
Active: active (running) since Fri 2025-04-25 15:12:58 PDT; 4s ago
Process: 153129 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=0>
Main PID: 153137 (mysqld)
Status: "Server is operational"
</pre>

Default configuration only allows MySQL connections to localhost. Good
<pre>
root@mpmt-daq03:/home1/wheel# grep bind-add /etc/mysql/mysql.conf.d/mysqld.cnf
bind-address = 127.0.0.1
mysqlx-bind-address = 127.0.0.1
</pre>

Login to MySQL as root and create the MySQL accounts

<pre>
mysql -h localhost
...
mysql> CREATE DATABASE IF NOT EXISTS history;
mysql> CREATE USER history_reader@'localhost' IDENTIFIED BY 'CHANGE_THIS_PASSWORD!';
mysql> GRANT SELECT ON history.* TO history_reader@'localhost';
mysql> CREATE USER 'history_writer'@'localhost' IDENTIFIED BY 'CHANGE_THIS_PASSWORD!';
mysql> GRANT SELECT,INSERT,CREATE,ALTER,INDEX ON history.* TO 'history_writer'@'localhost';
mysql> flush privileges;
</pre>

ATTENTION: change the account password listed above to something different!

Check the socket to connect to:

<pre>
root@mpmt-daq03:/home1/wheel# grep sock /etc/mysql/my.cnf
# Port or socket location where to connect
socket = /run/mysqld/mysqld.sock
</pre>

=== Do these steps as the experiment user ===

Rebuild MIDAS with support for MySQL. If this works correctly the build will notice MySQL is installed. For instance, with cmake we will see

<pre>
-- MIDAS: Found MySQL version 8.0.41
</pre>

Go to online directory and create files so MIDAS knows accounts to talk to ODB

<pre>

testmysql@mpmt-daq03:~/online$ cd ~/online
testmysql@mpmt-daq03:~/online$ emacs -nw mysql_reader.txt
testmysql@mpmt-daq03:~/online$ emacs -nw mysql_writer.txt
testmysql@mpmt-daq03:~/online$ more mysql_*.txt
::::::::::::::
mysql_reader.txt
::::::::::::::
database=history
socket=/run/mysqld/mysqld.sock
user=history_reader
password=CHANGE_THIS_PASSWORD!
::::::::::::::
mysql_writer.txt
::::::::::::::
database=history
socket=/run/mysqld/mysqld.sock
user=history_writer
password=CHANGE_THIS_PASSWORD!
</pre>

Again, set the correct password in these files.

== Setting up MySQL using mariadb on macbook ==

* install mariadb-10.4
<pre>
macos:
sudo port install mariadb-10.4
ln -s ln -fs /opt/local/lib/mariadb-10.4/bin/mysql_config ~/bin/
mysql_config --version
10.4.13
</pre>
* rebuild midas, check the cmake finds the correct mysql: MIDAS: Found MySQL version 10.4.13
* create $HOME/.my.cnf
<pre>
[mysqld]
#port=3306
socket=/Users/olchansk/mysql/mysql.sock
datadir=/Users/olchansk/mysql/db
pid-file=/Users/olchansk/mysql/mysqld.pid

[client]
#port=3306
socket=/Users/olchansk/mysql/mysql.sock
</pre>
* mkdir $HOME/mysql
* start mysql:
<pre>
macos: /opt/local/lib/mariadb-10.4/bin/mysqld
</pre>
* configure the database: start the "mysql" utility (/opt/local/lib/mariadb-10.4/bin/mysql)
<pre>
NOTE: instructions thanks to Ben S.
NOTE: use some other passwords instead of the example "reader_password" and "writer_password"

MariaDB [(none)]> CREATE DATABASE IF NOT EXISTS history;
Query OK, 1 row affected (0.003 sec)

MariaDB [(none)]> CREATE USER history_reader@'localhost' IDENTIFIED BY 'reader_password';
Query OK, 0 rows affected (0.027 sec)

MariaDB [(none)]> GRANT SELECT ON history.* TO history_reader@'localhost';
Query OK, 0 rows affected (0.016 sec)

MariaDB [(none)]> CREATE USER 'history_writer'@'localhost' IDENTIFIED BY 'writer_password';
Query OK, 0 rows affected (0.014 sec)

MariaDB [(none)]> GRANT SELECT,INSERT,CREATE,ALTER,INDEX ON history.* TO 'history_writer'@'localhost';
Query OK, 0 rows affected (0.017 sec)

MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.001 sec)
</pre>
* try connecting to both users:
<pre>
/opt/local/lib/mariadb-10.4/bin/mysql -u history_reader -p ### should ask for reader password
/opt/local/lib/mariadb-10.4/bin/mysql -u history_writer -p ### should ask for writer password
</pre>
* create midas config files:
** mysql_writer.txt
<pre>
#server=localhost
#port=xxx
database=history
socket=/Users/olchansk/mysql/mysql.sock
user=history_writer
password=writer_password ### change this!!!
#buffer=1000
</pre>
** mysql_reader.txt
<pre>
#server=localhost
#port=xxx
database=history
socket=/Users/olchansk/mysql/mysql.sock
user=history_reader
password=reader_password ### change this!!!
</pre>
* test MIDAS connection
<pre>
mh2sql --mysql mysql_writer.txt
(there should be no errors printed)
</pre>
* if desired, import midas history from .hst files:
<pre>
mh2sql --mysql mysql_writer.txt *.hst
Reading 130813.hst
[mh2sql,ERROR] [history_schema.cxx:1165:Mysql::Prepare,ERROR] mysql_query(SELECT event_name, table_name, itimestamp FROM _history_index WHERE table_name!='';) error 1146 (Table 'history._history_index' doesn't exist)
[mh2sql,INFO] Adding SQL table "rpcexample", status 311
[mh2sql,INFO] Adding SQL table "rpcexample_1596127680", status 1
[mh2sql,INFO] Adding SQL table "_history_index", status 1
[mh2sql,INFO] Adding column "event_name" to SQL table "_history_index", status 1
...
[mh2sql,INFO] Adding SQL table "run_transitions", status 311
[mh2sql,INFO] Adding SQL table "run_transitions_1596127681", status 1
[mh2sql,INFO] Adding column "state" to SQL table "run_transitions_1596127681", status 1
[mh2sql,INFO] Adding column "run_number" to SQL table "run_transitions_1596127681", status 1
Reading 130815.hst
</pre>

Setup MySQL database for MIDAS

2025-04-25T22:33:37Z

Lindner: /* Do these steps as the experiment user */

This page provides instructions for setting up a MySQL database to use with the MIDAS history logging.

Setting up the MySQL database is highly dependent on what OS and OS version you are using. Most of the setup work is just MySQL related. For this reason we provide below instructions for the MySQL database setup on a couple different OS. You will probably need to modify these instructions for your OS. Here's the some installation instructions:

* [[#Setting up MySQL on Ubuntu 24.04 LTS | Setting up MySQL on Ubuntu 24.04 LTS ]]
* [[#Setting up MySQL using mariadb on macbook | Setting up MySQL using mariadb on macbook]]

== Setting up MySQL on Ubuntu 24.04 LTS ==

This instructions assume you have root privileges and install MySQL as ROOT.

=== Do these steps as root user ===

Install the MySQL server software.

<pre>
apt install mysql-server
</pre>

You will find mysql has started after installation

<pre>
root@mpmt-daq03:/home1/wheel# service mysql status
mysql.service - MySQL Community Server
Loaded: loaded (/usr/lib/systemd/system/mysql.service; enabled; preset: enabled)
Active: active (running) since Fri 2025-04-25 15:12:58 PDT; 4s ago
Process: 153129 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=0>
Main PID: 153137 (mysqld)
Status: "Server is operational"
</pre>

Default configuration only allows MySQL connections to localhost. Good
<pre>
root@mpmt-daq03:/home1/wheel# grep bind-add /etc/mysql/mysql.conf.d/mysqld.cnf
bind-address = 127.0.0.1
mysqlx-bind-address = 127.0.0.1
</pre>

Login to MySQL as root and create the MySQL accounts

<pre>
mysql -h localhost
...
mysql> CREATE DATABASE IF NOT EXISTS history;
mysql> CREATE USER history_reader@'localhost' IDENTIFIED BY 'CHANGE_THIS_PASSWORD!';
mysql> GRANT SELECT ON history.* TO history_reader@'localhost';
mysql> CREATE USER 'history_writer'@'localhost' IDENTIFIED BY 'CHANGE_THIS_PASSWORD!';
mysql> GRANT SELECT,INSERT,CREATE,ALTER,INDEX ON history.* TO 'history_writer'@'localhost';
mysql> flush privileges;
</pre>

ATTENTION: change the account password listed above to something different!

Check the socket to connect to:

<pre>
root@mpmt-daq03:/home1/wheel# grep sock /etc/mysql/my.cnf
# Port or socket location where to connect
socket = /run/mysqld/mysqld.sock
</pre>

=== Do these steps as the experiment user ===

Rebuild MIDAS with support for MySQL. If this works correctly the build will notice MySQL is installed. For instance, with cmake we will see

<pre>
-- MIDAS: Found MySQL version 8.0.41
</pre>

Go to online directory and create files so MIDAS knows accounts to talk to ODB

<pre>

testmysql@mpmt-daq03:~/online$ cd ~/online
testmysql@mpmt-daq03:~/online$ emacs -nw mysql_reader.txt
testmysql@mpmt-daq03:~/online$ emacs -nw mysql_writer.txt
testmysql@mpmt-daq03:~/online$ more mysql_*.txt
::::::::::::::
mysql_reader.txt
::::::::::::::
database=history
socket=/run/mysqld/mysqld.sock
user=history_reader
password=reader_PsFtKL5kR3
::::::::::::::
mysql_writer.txt
::::::::::::::
database=history
socket=/run/mysqld/mysqld.sock
user=history_writer
password=writer_MQgoHDLqN3
</pre>

== Setting up MySQL using mariadb on macbook ==

* install mariadb-10.4
<pre>
macos:
sudo port install mariadb-10.4
ln -s ln -fs /opt/local/lib/mariadb-10.4/bin/mysql_config ~/bin/
mysql_config --version
10.4.13
</pre>
* rebuild midas, check the cmake finds the correct mysql: MIDAS: Found MySQL version 10.4.13
* create $HOME/.my.cnf
<pre>
[mysqld]
#port=3306
socket=/Users/olchansk/mysql/mysql.sock
datadir=/Users/olchansk/mysql/db
pid-file=/Users/olchansk/mysql/mysqld.pid

[client]
#port=3306
socket=/Users/olchansk/mysql/mysql.sock
</pre>
* mkdir $HOME/mysql
* start mysql:
<pre>
macos: /opt/local/lib/mariadb-10.4/bin/mysqld
</pre>
* configure the database: start the "mysql" utility (/opt/local/lib/mariadb-10.4/bin/mysql)
<pre>
NOTE: instructions thanks to Ben S.
NOTE: use some other passwords instead of the example "reader_password" and "writer_password"

MariaDB [(none)]> CREATE DATABASE IF NOT EXISTS history;
Query OK, 1 row affected (0.003 sec)

MariaDB [(none)]> CREATE USER history_reader@'localhost' IDENTIFIED BY 'reader_password';
Query OK, 0 rows affected (0.027 sec)

MariaDB [(none)]> GRANT SELECT ON history.* TO history_reader@'localhost';
Query OK, 0 rows affected (0.016 sec)

MariaDB [(none)]> CREATE USER 'history_writer'@'localhost' IDENTIFIED BY 'writer_password';
Query OK, 0 rows affected (0.014 sec)

MariaDB [(none)]> GRANT SELECT,INSERT,CREATE,ALTER,INDEX ON history.* TO 'history_writer'@'localhost';
Query OK, 0 rows affected (0.017 sec)

MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.001 sec)
</pre>
* try connecting to both users:
<pre>
/opt/local/lib/mariadb-10.4/bin/mysql -u history_reader -p ### should ask for reader password
/opt/local/lib/mariadb-10.4/bin/mysql -u history_writer -p ### should ask for writer password
</pre>
* create midas config files:
** mysql_writer.txt
<pre>
#server=localhost
#port=xxx
database=history
socket=/Users/olchansk/mysql/mysql.sock
user=history_writer
password=writer_password ### change this!!!
#buffer=1000
</pre>
** mysql_reader.txt
<pre>
#server=localhost
#port=xxx
database=history
socket=/Users/olchansk/mysql/mysql.sock
user=history_reader
password=reader_password ### change this!!!
</pre>
* test MIDAS connection
<pre>
mh2sql --mysql mysql_writer.txt
(there should be no errors printed)
</pre>
* if desired, import midas history from .hst files:
<pre>
mh2sql --mysql mysql_writer.txt *.hst
Reading 130813.hst
[mh2sql,ERROR] [history_schema.cxx:1165:Mysql::Prepare,ERROR] mysql_query(SELECT event_name, table_name, itimestamp FROM _history_index WHERE table_name!='';) error 1146 (Table 'history._history_index' doesn't exist)
[mh2sql,INFO] Adding SQL table "rpcexample", status 311
[mh2sql,INFO] Adding SQL table "rpcexample_1596127680", status 1
[mh2sql,INFO] Adding SQL table "_history_index", status 1
[mh2sql,INFO] Adding column "event_name" to SQL table "_history_index", status 1
...
[mh2sql,INFO] Adding SQL table "run_transitions", status 311
[mh2sql,INFO] Adding SQL table "run_transitions_1596127681", status 1
[mh2sql,INFO] Adding column "state" to SQL table "run_transitions_1596127681", status 1
[mh2sql,INFO] Adding column "run_number" to SQL table "run_transitions_1596127681", status 1
Reading 130815.hst
</pre>

Setup MySQL database for MIDAS

2025-04-25T22:24:43Z

Lindner: /* Do these steps as root user */

Setup MySQL database for MIDAS

2025-04-25T22:23:36Z

Lindner: /* Do these steps as root user */

Setup MySQL database for MIDAS

2025-04-25T22:19:59Z

Lindner: /* Setting up MySQL on Ubuntu 24.04 LTS */

Setup MySQL database for MIDAS

2025-04-25T22:18:34Z

Lindner: /* Setting up MySQL on Ubuntu 24.04 LTS */

Setup MySQL database for MIDAS

2025-04-25T22:16:03Z

Lindner: /* Setting up MySQL on Ubuntu 24.04 LTS */

Setup MySQL database for MIDAS

2025-04-25T22:13:56Z

Lindner: /* Setting up MySQL on Ubuntu 24.04 LTS */

Setup MySQL database for MIDAS

2025-04-23T21:32:46Z

Lindner: /* Setting up MySQL on Ubuntu 24.04 LTS */

Setup MySQL database for MIDAS

2025-04-23T21:29:49Z

Lindner: /* Setting up MySQL on Ubuntu 24.04 LTS */

Setup MySQL database for MIDAS

2025-04-23T21:28:07Z

Lindner: /* Setting up MySQL using mariadb on macbook */

History System

2025-04-23T21:27:43Z

Lindner: /* Write MYSQL-history events */

{{Pagelinks}}

== Links ==
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* [[/History ODB tree]]
* [[History Page]]
* [[Mhformat|MIDAS history file format]]
* [[Keys in the ODB /Logger tree|/Logger ODB tree]]
* [[Keys in the ODB /Logger/History subtree|/Logger/History subtree]]
* [[Image History]]
</div>

<div id="history file"></div>
= Introduction =

The history system is an add-on capability built into the MIDAS data logger [[mlogger]] to record useful information in parallel to the data logging.
This information is recorded with a time stamp and saved into a '''history file''' (see [[#History drivers|History drivers]]) for later retrieval. One set of files is created per day containing all the requested history events. The history data may be displayed in graphical form using the [[mhttpd]] MIDAS webserver utility, giving the user an easy way of seeing how experimental variables have changed with time.
The history logging will be in action only if the MIDAS data logger is running, but it is not necessary to have any [[Keys in the ODB /Logger tree#/Logger/Channels subtree|data logging channel]] enabled.

Since June 2020, it is possible to also use the midas history system to record images (png/jpeg etc) periodically, for example to later produce a timelapse from a webcam - see the [[Image History]] page for more details.

= MidasHistoryInterface =

The new history configuration defines history channels similar to logger channels. Each history channel creates a history interface class (class MidasHistoryInterface). The [[mlogger|data logger]] can use this interface to write data into the history, mhttpd, mhist & co can use this interface to read data from the history. The interface is defined in history.h

* http://ladd00.triumf.ca/~daqweb/doc/midas/include/history.h

= History drivers =

Presently, the MIDAS distribution contains these drivers:

; MIDAS-History
: this is the traditional history driver, with data stored in the .hst binary files. (format is documented here [[mhformat]], see also [[mhist]] and [[mhdump]])
: By default ODB subtree '''[[Keys in the ODB /Logger/History subtree#MIDAS subtree |"MIDAS"]]''' is set up to write MIDAS-history.

; ODBC-History
: stores history data in SQL database using the ODBC abstraction layer. See [[#MIDAS ODBC SQL History system|ODBC SQL History system]].

: By default ODB subtree '''[[Keys in the ODB /Logger/History subtree#ODBC subtree|"ODBC"]]''' is set up to write ODBC-history.

; SQLITE-History
: stores data in SQL database as SQLITE3 .sqlite3 files. The SQL table format is similar to ODBC-history, performance is similar, no database server required.
: By default ODB subtree '''[[Keys in the ODB /Logger/History subtree#SQLITE subtree|"SQLITE"]]''' is set up to write SQLITE-history.

; MYSQL-History
:
: By default ODB subtree '''[[Keys in the ODB /Logger/History subtree#MYSQL subtree|"MYSQL"]]''' is set up to write MYSQL-history.

; FILE-History
: stores each variable in its own file, making it much easier to trace the history of a particular variable.
: By default ODB subtree '''[[Keys in the ODB /Logger/History subtree#FILE subtree|FILE]]''' is set up to write FILE-history.

More drivers can be easily added by the user (create history_xxx.cxx, add constructor to hs_get_history(), create the ODB subtree <span style="color:purple; font-style:italic">/Logger/History/xxx/channel</span>).

= Location of History Files =
;MIDAS-history
:By default, the traditional MIDAS-history is enabled, writing history files into the MIDAS data directory path given by the ODB key [[Keys in the ODB /Logger tree#Data dir|Data dir]].
:If history data is important, it is recommended to write it to a directory that is backed up or archived. (The MIDAS data directory is usually located on a scratch data disk without backups).

:The location of the MIDAS-history data files can be changed by use of the ODB Key [[Keys in the ODB /Logger tree#History dir|History dir]].

;ODBC-History
:The location of ODBC-History files is ...

;SQLITE-History
:The location of SQLITE-History files is the directory path given by the ODB key [[Keys in the ODB /Logger/History subtree#Sqlite dir|Sqlite dir]].

;MYSQL-History
:The location of MYSQL-History files is ...

;FILE-History
:The location of FILE-History files is the directory path given by the ODB key [[Keys in the ODB /Logger/History subtree#History dir|History dir]].

= Types of History Events =

There are two basic types of history events, which are defined in different ways:

* "[[#Frontend History Event|Frontend]]" (or "Equipment") History event composed in the frontend. See [[#Frontend History Event|Frontend History Event]] for details.
* "Virtual" (or "Links") History event composed within the ODB under the [[/History ODB tree#Links subtree|/History/Links subtree]]. See [[#Virtual History Event|Virtual History Event]] for details.

Both these history event definitions take effect when the data logger gets a start run [[Run States and Transitions|transition]]. Any modifications made during the run are not applied until the start of the next run.

== Virtual History Event ==
The history of any ODB variable can be recorded in a "virtual" history event by creating a link to that variable in the [[/History ODB tree#Links subtree|/History/Links subtree]]. This is used for ODB variables that are not in an Equipment.

; To enable Virtual History Events
: Create links as desired in [[/History ODB tree#Links subtree|/History/Links subtree]].

History events are created for each entry under ODB subdirectory {{Odbpath|path=/History/Links}}.

Two types of links are permitted:
<dl>
<dt> ODB key {{Odbpath|path=/history/links/aaa}} is a link to a subdirectory</dt>
<dd><ol>
<li> db_watch() is setup to watch this subdirectory</li>
<li> tags are created for each subdirectory entry (1 tag per entry)</li>
<li> There is no possibility for naming array elements, so 1 tag per array, regardless of the number of elements.</li> </ol></dd>
<dt>ODB key {{Odbpath|path=/history/links/bbb}} is a subdirectory with links to ODB values</dt>
<dd> <ol>
<li>db_watch is setup to watch each link target</li>
<li> tags are created for each link (1 tag per link)</li>
<li> tag name is the link name (NOT the target name). There is no possibility for naming array elements</li>
</ol></dd>
</dl>

Mixing links and subdirectories is not permitted.

The update period of history events created for {{Odbpath|path=/history/links}} is controlled by entries in {{Odbpath|path=/history/links periods}}. Numeric values of periods are same as for a [[#Frontend History Event|Frontend History event]]. Numeric value 0 disables the history for a particular event.

See [https://midas.triumf.ca/elog/Midas/1087 MidasElog#1087] for more information.

== Frontend History Event ==

Each [[Frontend Operation#Equipments and Events|Equipment]] has the capability to generate "history data".

; To enable Frontend History Events

: Set the [[/Equipment ODB tree#Log history|Log history]] ODB key of each equipment to be logged '''non-zero'''. (See [[Equipment List Parameters#Log History|Log History parameter]]). (You may need to restart the frontend.)

This causes the event contents to be copied to the ODB [[/Equipment ODB tree#Variables subtree|/Equipment/''<equipment-name>''/Variables subtree]] for the use of the History system. history events are created by parsing the content of {{Odbpath|path=/Equipment/<equipment-name>/variables}}.

For each equipment, history is controlled by the value of {{Odbpath|path=/Equipment/<equipment-name>/common/period}} as follows:
* 0 = history disabled
* 1 = history is enabled
* >1 = history is enabled, throttled down

The throttling is implemented in log_history()/watch_history() by this algorithm:
the very first history event is recorded, then all changed to the data are ignored until "period" seconds has elapsed. Then the next history event will be recorded, and following
changes will be ignored until "period" second elapses, and so forth. Period value "1" has special meaning - there is no throttling, all history events are logged.

This information is from [https://midas.triumf.ca/elog/Midas/1087 MidasElog#1087].

=== Sequence for frontend History Event ===
The main steps for the frontend History Event is as follows:

* The user code in the frontend equipment Event Readout routines reads the data, placing it into a MIDAS data bank
* In mfe.c, if the key [[/Equipment ODB tree#Log history|Log history]] > 0 (i.e. the history system is enabled for this equipment), this data bank is written into ODB (by update_odb()).
* This ODB write triggers an ODB hot-link into mlogger
* The hot-link calls mlogger.c::log_history(), which calls hs_write() to write the data into the history file. The frequency of the history writes is specified by the number of seconds between writes stored in the ODB key [[/Equipment ODB tree#Log history|Log history]].
* history.c::hs_write() or history_odbc::hs_write_odbc() writes the data into a history file or into an SQL database.

= How to Write History events =
* Enable desired [[#To enable Frontend History Events|Frontend]] and/or [[#To enable Virtual History Events|Virtual]] history events
* [[#Select the history driver|Select the history driver]]
* Follow instructions below for selected driver:
** [[#Write MIDAS-History events|Write MIDAS-history events]]
** [[#Write ODBC-History events|Write ODBC-history events]]
** [[#Write SQLITE-History events|Write SQLITE-history events]]
** [[#Write MYSQL-History events|Write MYSQL-history events]]
** [[#Write FILE-History events|Write FILE-history events]]

== Select the history driver ==

* Select the '''[[#History drivers|history driver]]''' and therefore the [[Keys in the ODB /Logger/History subtree#<history-logging-channel> subtrees|history-logging-channel]] to use to write the MIDAS-history files.
* set [[Keys in the ODB /Logger/History subtree#Active|Active]] ODB key in the selected [[Keys in the ODB /Logger/History subtree#<history-logging-channel> subtrees|history-logging-channel]] to "y". All other logging channels should be disabled.
* set [[/History ODB tree#LoggerHistoryChannel|LoggerHistoryChannel]] ODB key to the selected [[#History drivers|History channel]].

== Write MIDAS-History events ==
* Enable desired [[#To enable Frontend History Events|Frontend]] and/or [[#To enable Virtual History Events|Virtual]] history events
* Perform steps in [[#Select the history driver|Select History Driver]] with [[#History drivers|history driver]] "MIDAS".
* Restart [[mhttpd]] and [[mlogger]], start the run.

MIDAS-history files will be saved into the [[#Location of History Files|MIDAS-history data directory]].

== Write ODBC-History events ==

See [[#MIDAS ODBC SQL History system|MIDAS ODBC SQL History system]].
* If necessary, [[#Set up a MySQL database|set up a MySQL database]].
* Enable desired virtual and frontend history events (see [[#Virtual History Event|Virtual history events]] and [[#Frontend History Event|Frontend history events]]).
* Perform steps in [[#Select the history driver|Select History Driver]] with [[#History drivers|history driver]] "ODBC".
* Check ODB Key [[Keys in the ODB /Logger/History subtree#Writer_ODBC_DSN|Writer_ODBC_DSN]] is set to "history writer" and Key [[Keys in the ODB /Logger/History subtree#Reader_ODBC_DSN|Reader_ODBC_DSN]] is set to "history reader" (the default values)
* set [[Keys in the ODB /Logger/History subtree#Debug|Debug]] to 1 if desired
* restart [[mlogger]] in verbose mode (i.e. mlogger -v), and observe how it issues SQL commands to create the tables and columns corresponding to MIDAS-history events and tags.
* restart [[mhttpd]]

'''Notes'''
# [[mlogger]] is programmed to raise alarms if connection to SQL database is interrupted or if some events cannot be written into the database (i.e. data type mismatch, SQL syntax errors, etc). The traditional MIDAS-history never raised alarms because it "never failed" - other than from "disk full" errors, which are immediately obvious.
# With ODBC History enabled, [[mhttpd]] will only use history information from the SQL database to make history plots and to extract history variable names for the history plot editor. The ODB {{Odbpath|path=/History}} subtrees[[/History ODB tree#Tags subtree|Tags]] and [[/History ODB tree#Events subtree|Events]] are not used.
# variable names presented to the user may change from MIDAS names to SQL names (use the history panel "Label" text fields to create permanent plot labels).

== Write SQLITE-history events ==

No special configuration required.

* Enable desired virtual and frontend history events (see [[#Virtual History Event|Virtual history events]] and [[#Frontend History Event|Frontend history events]]).
* Perform steps to [[#Select the history driver|Select History Driver]] with [[#History drivers|history driver]] "SQLITE".

* Create a directory for storage of sqlite history files
* save its path in the ODB key [[Keys in the ODB /Logger/History subtree#Sqlite dir|Sqlite dir]]
* restart [[mhttpd]] and [[mlogger]]

In the history data directory (in [[Keys in the ODB /Logger/History subtree#Sqlite dir|Sqlite dir]]) you should see files with name "mh_xxx.sqlite3" corresponding to your equipment and variable names.

== Write MYSQL-history events ==

* Enable desired virtual and frontend history events (see [[#Virtual History Event|Virtual history events]] and [[#Frontend History Event|Frontend history events]]).
* Perform steps to [[#Select the history driver|Select History Driver]] with [[#History drivers|history driver]] "MYSQL".
* Read the following instructions to [[setup MySQL database for MIDAS]]:

* enable MYSQL history in mlogger:
** change ODB /Logger/History/MYSQL/Active to "y", "mysql writer" and "mysql reader" to location of these files (leave the default values if these files are in the experiment directory).
* enable MYSQL history in mhttpd:
** change ODB /History/LoggerHistoryChannel to "MYSQL"
* restart mlogger, restart mhttpd
...........

== Write FILE-history events ==

* Enable desired virtual and frontend history events (see [[#Virtual History Event|Virtual history events]] and [[#Frontend History Event|Frontend history events]]).
* Perform steps to [[#Select the history driver|Select History Driver]] with [[#History drivers|history driver]] "SQLITE".
* Assign directory path for saved files using the ODB key [[Keys in the ODB /Logger/History subtree#History dir|History dir]].
...........

<br>
---------------
<br>

= History Events =
The following information is derived from [https://midas.triumf.ca/elog/Midas/1087 MidasElog#1087].

A "history event" is a history atomic unit of data. Associated with each history event is a timestamp (unix time),
a name (limited to NAME_LENGTH in the old history) and a list of history tags that describe the individual data
values inside the history event.

When making history plots in [[mhttpd]] (see [[History Page]]) for each curve on the plot, one selects a history event (from the list
of currently active events, recently active events or the list of all events that ever existed), then from the list of tags
inside the history event one selects the particular variable that will be plotted.

== Old MIDAS history events ==
In the old MIDAS history, all history events are written into one history file (.hst file + optional .def and .idx event definition and time index files
which can be/are regenerated automatically from the .hst file). History events are identified by 16-bit history event IDs, the persistent mapping
from history event names and the 16-bit history event IDs is stored in ODB subtree {{Odbpath|path=/History/Events}}. In addition the list of all known history event tags is
stored in ODB subtree {{Odbpath|path=/History/Tags}}. For per-equipment history, the 16-bit history event ID is the value of ODB key [[/Equipment ODB tree#Event ID|/Equipment/<equipment-name>/Common/Event ID]].

== SQL history events ==
In the SQL history (MySQL, SQLITE, etc), each history event is an SQL table. The history event tags are the SQL table columns.

== FILE history events ==
In the new FILE history, each history event is written into a separate file, tag definition are recorded in text format in the file header, history event
data is appended to the file in binary format (fixed record size). If the history event definition is changed, a new file will be started.

= History Event Construction =
The [[mlogger]] creates [[#Frontend history event]]s in open_history() by parsing the ODB subtree {{Odbpath|path=/equipment/<equipment-name>/variables}}. Each ODB entry under the {{Odbpath|path=variables/}} subtree is referred to as a "variable". Each variable can be a single ODB value, an array of ODB values, or a subdirectory (corresponding to TID_STRUCT structured data banks). As each variable is processed, one or more tags are created to describe it.

Single ODB values will generally produce a single tag, while arrays can produce (depending whether the array is "[[#Named Arrays|Named]]" or not)
* multiple tags - one per array element ("Named" array) or
* one single tag - describing the whole array

The code can generate two types of history:
<dl>
<dt> "per-equipment" history</dt>
<dd> will have the tags for all variables concatenated together into one single history event</dd>
<dt> "per-variable" history</dt>
<dd> will have one history event defined for each variable. Inside could be one tag (for single odb values and unnamed array) or multiple tags (for named arrays and structured data
banks).</dd>
</dl>

Per-equipment history is the original MIDAS history implementation. Per-variable history was added to permit efficient data storage in SQL tables. Its initial implementation used 1 ODB hotlink for each variable and it was easy to exceed the maximum permitted number of ODB hotlinks (db_open_record()). To reduce consumption of hotlinks, db_watch() has been implemented and now per-variable history only uses 1 ODB hotlink per equipment.

;NOTE
: With db_watch, per-equipment history is no longer available. '''Per-variable history is the new default''' (and the only option).

[https://midas.triumf.ca/elog/Midas/1087 MidasElog#1087]

= Construction of history event tags =
Currently,
* single ODB values are treated as arrays of length "1"
* string ODB entries are not permitted

History event tags are constructed in the [[mlogger]] add_equipment(). Presently there are some quirks with the generation of history event tags (see [https://midas.triumf.ca/elog/Midas/1088]),
e.g. single odb values are treated as arrays of length "1" and "string" ODB entries are not permitted.

For variables of type TID_KEY (subdirectories, corresponding to TID_STRUCT structured banks), one tag is generated for each subdirectory entry. Tag names for {{Odbpath|path=/equipment/<equipment-name>/variables/aaa/bbb}} will be "aaa_bbb" (with an underscore). Subdirectory entries of type TID_KEY and TID_LINK are currently not supported.

For "[[#Named Arrays|Named Arrays]]", individual tags are generated for each array element. Tag names are taken from the ''Names'' array. For empty tag names (empty names array), tags are "aaa_0", "aaa_1", etc (for {{Odbpath|path=/equipment/<equipment-name>/variables/aaa}}).

For "[[#Named Arrays|Single Names Arrays]]", tag names have the variable name appended (with a space), for {{Odbpath|path=/equipment/<equipment-name>/variables/aaa}} and an empty names array, tags will be "aaa_0 aaa", "aaa_1 aaa", etc. For a populated names array, the tags will be "name0 aaa", "name1 aaa", etc.

For unnamed arrays and single odb variables (in ODB, single odb variables are arrays of length 1), a single tag is generated.

== Named Arrays ==

'''Named Arrays''' have a "Names" entry in {{Odbpath|path=/equipment/<equipment-name>/settings}}.

For example, to add names to {{Odbpath|path=/equipment/<equipment-name>/variables/aaa}}, create a string array {{Odbpath|path=/equipment/<equipment-name>/settings/names aaa}}. The names array should be at least as long as the corresponding data array. Individual entries in the names array can be left blank (tag names will be "aaa_0", "aaa_1", etc). Duplicate tag names are not permitted.

A single "Names" entry can be created to name all arrays in variables with the same names ('''"Single Names"''').

Create {{Odbpath|path=/equipment/<equipment-name>/settings/names}} and arrays {{Odbpath|path=/equipment/<equipment-name>/variables/aaa}} and {{Odbpath|path=/equipment/<equipment-name>/variables/bbb}} will have
history tags "name0 aaa", "name1 aaa", "name0 bbb", "name1 bbb", etc. If "names" are left blank, tag names will be "aaa_0 aaa", "aaa_1 aaa", "bbb_0 bbb", "bbb_1 bbb", etc.

In the [[mhttpd]] web server [[History Page]] variables viewer, "single name" arrays are displayed in a 2D table.

This information is from [https://midas.triumf.ca/elog/Midas/1088].

= Historical note on MIDAS-History Tag names =
Tags are only used for MIDAS-history events.

The mlogger/mhttpd implementation of <span style="color:purple; font-style:italic">/History/Tags</span> has proved troublesome and we are moving away from it.

;During the transition period:
: Implementations starting with Version 4435 of [[mlogger]] and [[mhttpd]] (i.e. "new" versions) will now work '''without''' <span style="color:purple; font-style:italic">/History/Tags</span>. The history tags are read directly from the history files themselves. To run in this mode, set ODB key [[/History ODB tree#DisableTags|/History/DisableTags]] to "y". Existing [[/History ODB tree#Tags subtree|/History/Tags subtree]] will be automatically deleted.

Two downsides to running like this are
* it is slower and tags become non-persistent
* if some frontends have not been running for a while, their variables may vanish from the history panel editor.

== New-style History Tags ==
For the above two reasons, using <span style="color:purple; font-style:italic">/History/Tags</span> is still recommended.
However, the format of the tags is now changed to simplify management and reduce ODB size.

; For NEW implementations
: the key [[/History ODB tree#CreateOldTags|CreateOldTags]] will not be present. "New-style" history tags will be created automatically in the [[/History ODB tree#Tags subtree|Tags subtree]].

;If you have old-style tags
: [[mlogger]] will automatically convert the old tags to this new format (this is why you should make a backup of your ODB).

** using "old" (i.e. pre Version 4435) mlogger with "new" mhttpd is OK: new mhttpd understands both formats of /History/Tags.
** using "old" mhttpd with "new" mlogger is OK provided the ODB key [[/History ODB tree#CreateOldTags|/History/CreateOldTags]] is set to "y" before starting mlogger.

== Old-style History Tags ==
Prior to versions 4435 of [[mlogger]] and [[mhttpd]], old-style history tags were generated.
See
* http://ladd00.triumf.ca/~daqweb/doc/midas-old/html/F_History_logging.html
for details.

<br>
---------------
<br>

= MIDAS ODBC SQL History system =

This section describes the internal workings of the MIDAS SQL history system.

The SQL history system implements the MIDAS history using a SQL database for data storage instead of flat binary files.

The SQL history code lives in a separate files history.h and history_sql.cxx. The present implementation uses the ODBC API for accessing SQL databases.
MySQL is fully supported, Pgsql support is partially implemented, support for arbitrary ODBC drivers should be easy to add in history_sql and most SQL operations are done using an interface class.

Preliminary testing with MySQL and PgSQL indicates that disk space requirements are the same for both storage mechanisms using flat binary files or either of the 2 SQL databases. All 3 tested storage mechanisms store the data uncompressed. (Observed gzip-1 compression ratios from CERN-ALPHA history files are 50% or better).

Preliminary testing also indicates that MySQL database is "not too slow" for use by T2K/ND280 slow control system. Using MySQL, performance seems to improve somewhat compared to traditional MIDAS-history because each SQL table is stored into a separate file, compared to everything-in-one-file in traditional MIDAS-history.

== Set up a MySQL database ==

In order to setup a MySQL database, the following is needed:

* the server hostname and port number
* database name
* user name and password.

It is recommended to create 3 different users:

* a "root" user with full priveleges,
* a "reader" user with read-only priveleges
* and a "writer" user with "insert, create table, and add column" priveleges (no permission to drop table, remove columns, etc)

=== Example ===
#Setup a MySQL database (RHEL/SL 5)

ssh root@localhost
service mysqld start
/usr/bin/mysql_secure_installation (enter current password for root: press <enter>, then accept default answer "Y" to all questions)
mysqladmin -p create history
using mysql-administrator, create 2 users:
history_writer, with database "history" schema privileges: select, insert, create, alter, index
history_reader, with database "history" schema privileges: select

#Setup $HOME/.odbc.ini file:

cat $HOME/.odbc.ini
[history_writer]
Description = history_writer
Driver = /usr/lib64/libmyodbc3.so
SERVER = ladd05.triumf.ca
PORT =
Database = history
OPTION = 3
SOCKET =
User = history_writer
Password = ******
[history_reader]
Description = history_reader
Driver = /usr/lib64/libmyodbc3.so
SERVER = ladd05.triumf.ca
PORT =
Database = history
OPTION = 3
SOCKET =
User = history_reader
Password =

Note: if you are using mhist you will need a separate .odbc.ini file for each user that you have connecting to the database (through MIDAS).

== Layout of SQL tables ==

* One table is created for each history event. Table name is the same as the history event name (as reported by [[mhdump]]).
* one column is created for each history tag (history variable). Column name is the same as the tag name (as reported by mhdump). For array tags, one column is created for each array element: array a[3] will produce columns a_0, a_1 and a_2.
* two special columns are always created: _i_time and _t_time are the timestamps in the UNIX-time integer format and in the SQL "time" format. (A priori, it is not clear which timestamp format is more useful for end users. mhttpd uses the _i_time integer time format). Note that this timestamp is the time when mlogger receives the hot-link signal (see above) and it can be a few seconds behind the time when the user placed the data into the MIDAS bank. For high precision (and sub-second time resolution) time stamps, users should generate their own timestamps and include them as part of the data itself.
* a special table called _history_index is created to remember the mapping between SQL names and MIDAS names for event names, variable name and tag names.

=== Schema Evolution ===

* when new data fields are added, new SQL columns are created automatically
* existing SQL columns are never deleted
* arrays are expanded automatically, and arrays never shrink.
* if the type of a data field changes, a conflict may occur with existing SQL columns, for example if previous type was "FLOAT" and new type is "DOUBLE", the existing SQL columns would have the SQL data type "float", too narrow to store new values. mlogger will complain about this. Such conflicts may be resolved by changing the type of the SQL column using SQL tools. In general, SQL names are created from MIDAS names by replacing all non-alphanumerical characters to an underscore "_". If this procedure creates duplicate column names, the column names and made unique by appending them with a random number, i.e. MIDAS names "A+B" and "A-B" will be translated to "A_B" and "A_B_12345" (random integer). True MIDAS names are saved into the _history_index.

[[Category:history]]

Setup MySQL database for MIDAS

2025-04-23T21:26:22Z

Lindner:

Setup MySQL database for MIDAS

2025-04-23T21:25:40Z

Lindner:

Setup MySQL database for MIDAS

2025-04-23T21:24:53Z

Lindner:

Setup MySQL database for MIDAS

2025-04-23T21:24:29Z

Lindner:

Setup MySQL database for MIDAS

2025-04-23T21:24:17Z

Lindner:

Setup MySQL database for MIDAS

2025-04-23T21:23:45Z

Lindner:

Setup MySQL database for MIDAS

2025-04-23T21:22:35Z

Lindner:

Setup MySQL database for MIDAS

2025-04-23T21:21:15Z

Lindner:

This page provides instructions for setting up a MySQL database to use with the MIDAS history logging.

Setting up the

== Setting up MySQL using mariadb on macbook ==

* install mariadb-10.4
<pre>
macos:
sudo port install mariadb-10.4
ln -s ln -fs /opt/local/lib/mariadb-10.4/bin/mysql_config ~/bin/
mysql_config --version
10.4.13
</pre>
* rebuild midas, check the cmake finds the correct mysql: MIDAS: Found MySQL version 10.4.13
* create $HOME/.my.cnf
<pre>
[mysqld]
#port=3306
socket=/Users/olchansk/mysql/mysql.sock
datadir=/Users/olchansk/mysql/db
pid-file=/Users/olchansk/mysql/mysqld.pid

[client]
#port=3306
socket=/Users/olchansk/mysql/mysql.sock
</pre>
* mkdir $HOME/mysql
* start mysql:
<pre>
macos: /opt/local/lib/mariadb-10.4/bin/mysqld
</pre>
* configure the database: start the "mysql" utility (/opt/local/lib/mariadb-10.4/bin/mysql)
<pre>
NOTE: instructions thanks to Ben S.
NOTE: use some other passwords instead of the example "reader_password" and "writer_password"

MariaDB [(none)]> CREATE DATABASE IF NOT EXISTS history;
Query OK, 1 row affected (0.003 sec)

MariaDB [(none)]> CREATE USER history_reader@'localhost' IDENTIFIED BY 'reader_password';
Query OK, 0 rows affected (0.027 sec)

MariaDB [(none)]> GRANT SELECT ON history.* TO history_reader@'localhost';
Query OK, 0 rows affected (0.016 sec)

MariaDB [(none)]> CREATE USER 'history_writer'@'localhost' IDENTIFIED BY 'writer_password';
Query OK, 0 rows affected (0.014 sec)

MariaDB [(none)]> GRANT SELECT,INSERT,CREATE,ALTER,INDEX ON history.* TO 'history_writer'@'localhost';
Query OK, 0 rows affected (0.017 sec)

MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.001 sec)
</pre>
* try connecting to both users:
<pre>
/opt/local/lib/mariadb-10.4/bin/mysql -u history_reader -p ### should ask for reader password
/opt/local/lib/mariadb-10.4/bin/mysql -u history_writer -p ### should ask for writer password
</pre>
* create midas config files:
** mysql_writer.txt
<pre>
#server=localhost
#port=xxx
database=history
socket=/Users/olchansk/mysql/mysql.sock
user=history_writer
password=writer_password ### change this!!!
#buffer=1000
</pre>
** mysql_reader.txt
<pre>
#server=localhost
#port=xxx
database=history
socket=/Users/olchansk/mysql/mysql.sock
user=history_reader
password=reader_password ### change this!!!
</pre>
* test MIDAS connection
<pre>
mh2sql --mysql mysql_writer.txt
(there should be no errors printed)
</pre>
* if desired, import midas history from .hst files:
<pre>
mh2sql --mysql mysql_writer.txt *.hst
Reading 130813.hst
[mh2sql,ERROR] [history_schema.cxx:1165:Mysql::Prepare,ERROR] mysql_query(SELECT event_name, table_name, itimestamp FROM _history_index WHERE table_name!='';) error 1146 (Table 'history._history_index' doesn't exist)
[mh2sql,INFO] Adding SQL table "rpcexample", status 311
[mh2sql,INFO] Adding SQL table "rpcexample_1596127680", status 1
[mh2sql,INFO] Adding SQL table "_history_index", status 1
[mh2sql,INFO] Adding column "event_name" to SQL table "_history_index", status 1
...
[mh2sql,INFO] Adding SQL table "run_transitions", status 311
[mh2sql,INFO] Adding SQL table "run_transitions_1596127681", status 1
[mh2sql,INFO] Adding column "state" to SQL table "run_transitions_1596127681", status 1
[mh2sql,INFO] Adding column "run_number" to SQL table "run_transitions_1596127681", status 1
Reading 130815.hst
...
</pre>
* enable MYSQL history in mlogger:
** change ODB /Logger/History/MYSQL/Active to "y", "mysql writer" and "mysql reader" to location of these files (leave the default values if these files are in the experiment directory).
* enable MYSQL history in mhttpd:
** change ODB /History/LoggerHistoryChannel to "MYSQL"
* restart mlogger, restart mhttpd
...........

Setup MySQL database for MIDAS

2025-04-23T21:20:18Z

Lindner: Created page with "This page provides instructions for setting up a MySQL database to use with the MIDAS history logging. Setting up the # MyAQ"

This page provides instructions for setting up a MySQL database to use with the MIDAS history logging.

Setting up the

# MyAQ

History System

2025-04-23T21:19:21Z

Lindner: /* Write MYSQL-history events */

{{Pagelinks}}

== Links ==
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* [[/History ODB tree]]
* [[History Page]]
* [[Mhformat|MIDAS history file format]]
* [[Keys in the ODB /Logger tree|/Logger ODB tree]]
* [[Keys in the ODB /Logger/History subtree|/Logger/History subtree]]
* [[Image History]]
</div>

<div id="history file"></div>
= Introduction =

The history system is an add-on capability built into the MIDAS data logger [[mlogger]] to record useful information in parallel to the data logging.
This information is recorded with a time stamp and saved into a '''history file''' (see [[#History drivers|History drivers]]) for later retrieval. One set of files is created per day containing all the requested history events. The history data may be displayed in graphical form using the [[mhttpd]] MIDAS webserver utility, giving the user an easy way of seeing how experimental variables have changed with time.
The history logging will be in action only if the MIDAS data logger is running, but it is not necessary to have any [[Keys in the ODB /Logger tree#/Logger/Channels subtree|data logging channel]] enabled.

Since June 2020, it is possible to also use the midas history system to record images (png/jpeg etc) periodically, for example to later produce a timelapse from a webcam - see the [[Image History]] page for more details.

= MidasHistoryInterface =

The new history configuration defines history channels similar to logger channels. Each history channel creates a history interface class (class MidasHistoryInterface). The [[mlogger|data logger]] can use this interface to write data into the history, mhttpd, mhist & co can use this interface to read data from the history. The interface is defined in history.h

* http://ladd00.triumf.ca/~daqweb/doc/midas/include/history.h

= History drivers =

Presently, the MIDAS distribution contains these drivers:

; MIDAS-History
: this is the traditional history driver, with data stored in the .hst binary files. (format is documented here [[mhformat]], see also [[mhist]] and [[mhdump]])
: By default ODB subtree '''[[Keys in the ODB /Logger/History subtree#MIDAS subtree |"MIDAS"]]''' is set up to write MIDAS-history.

; ODBC-History
: stores history data in SQL database using the ODBC abstraction layer. See [[#MIDAS ODBC SQL History system|ODBC SQL History system]].

: By default ODB subtree '''[[Keys in the ODB /Logger/History subtree#ODBC subtree|"ODBC"]]''' is set up to write ODBC-history.

; SQLITE-History
: stores data in SQL database as SQLITE3 .sqlite3 files. The SQL table format is similar to ODBC-history, performance is similar, no database server required.
: By default ODB subtree '''[[Keys in the ODB /Logger/History subtree#SQLITE subtree|"SQLITE"]]''' is set up to write SQLITE-history.

; MYSQL-History
:
: By default ODB subtree '''[[Keys in the ODB /Logger/History subtree#MYSQL subtree|"MYSQL"]]''' is set up to write MYSQL-history.

; FILE-History
: stores each variable in its own file, making it much easier to trace the history of a particular variable.
: By default ODB subtree '''[[Keys in the ODB /Logger/History subtree#FILE subtree|FILE]]''' is set up to write FILE-history.

More drivers can be easily added by the user (create history_xxx.cxx, add constructor to hs_get_history(), create the ODB subtree <span style="color:purple; font-style:italic">/Logger/History/xxx/channel</span>).

= Location of History Files =
;MIDAS-history
:By default, the traditional MIDAS-history is enabled, writing history files into the MIDAS data directory path given by the ODB key [[Keys in the ODB /Logger tree#Data dir|Data dir]].
:If history data is important, it is recommended to write it to a directory that is backed up or archived. (The MIDAS data directory is usually located on a scratch data disk without backups).

:The location of the MIDAS-history data files can be changed by use of the ODB Key [[Keys in the ODB /Logger tree#History dir|History dir]].

;ODBC-History
:The location of ODBC-History files is ...

;SQLITE-History
:The location of SQLITE-History files is the directory path given by the ODB key [[Keys in the ODB /Logger/History subtree#Sqlite dir|Sqlite dir]].

;MYSQL-History
:The location of MYSQL-History files is ...

;FILE-History
:The location of FILE-History files is the directory path given by the ODB key [[Keys in the ODB /Logger/History subtree#History dir|History dir]].

= Types of History Events =

There are two basic types of history events, which are defined in different ways:

* "[[#Frontend History Event|Frontend]]" (or "Equipment") History event composed in the frontend. See [[#Frontend History Event|Frontend History Event]] for details.
* "Virtual" (or "Links") History event composed within the ODB under the [[/History ODB tree#Links subtree|/History/Links subtree]]. See [[#Virtual History Event|Virtual History Event]] for details.

Both these history event definitions take effect when the data logger gets a start run [[Run States and Transitions|transition]]. Any modifications made during the run are not applied until the start of the next run.

== Virtual History Event ==
The history of any ODB variable can be recorded in a "virtual" history event by creating a link to that variable in the [[/History ODB tree#Links subtree|/History/Links subtree]]. This is used for ODB variables that are not in an Equipment.

; To enable Virtual History Events
: Create links as desired in [[/History ODB tree#Links subtree|/History/Links subtree]].

History events are created for each entry under ODB subdirectory {{Odbpath|path=/History/Links}}.

Two types of links are permitted:
<dl>
<dt> ODB key {{Odbpath|path=/history/links/aaa}} is a link to a subdirectory</dt>
<dd><ol>
<li> db_watch() is setup to watch this subdirectory</li>
<li> tags are created for each subdirectory entry (1 tag per entry)</li>
<li> There is no possibility for naming array elements, so 1 tag per array, regardless of the number of elements.</li> </ol></dd>
<dt>ODB key {{Odbpath|path=/history/links/bbb}} is a subdirectory with links to ODB values</dt>
<dd> <ol>
<li>db_watch is setup to watch each link target</li>
<li> tags are created for each link (1 tag per link)</li>
<li> tag name is the link name (NOT the target name). There is no possibility for naming array elements</li>
</ol></dd>
</dl>

Mixing links and subdirectories is not permitted.

The update period of history events created for {{Odbpath|path=/history/links}} is controlled by entries in {{Odbpath|path=/history/links periods}}. Numeric values of periods are same as for a [[#Frontend History Event|Frontend History event]]. Numeric value 0 disables the history for a particular event.

See [https://midas.triumf.ca/elog/Midas/1087 MidasElog#1087] for more information.

== Frontend History Event ==

Each [[Frontend Operation#Equipments and Events|Equipment]] has the capability to generate "history data".

; To enable Frontend History Events

: Set the [[/Equipment ODB tree#Log history|Log history]] ODB key of each equipment to be logged '''non-zero'''. (See [[Equipment List Parameters#Log History|Log History parameter]]). (You may need to restart the frontend.)

This causes the event contents to be copied to the ODB [[/Equipment ODB tree#Variables subtree|/Equipment/''<equipment-name>''/Variables subtree]] for the use of the History system. history events are created by parsing the content of {{Odbpath|path=/Equipment/<equipment-name>/variables}}.

For each equipment, history is controlled by the value of {{Odbpath|path=/Equipment/<equipment-name>/common/period}} as follows:
* 0 = history disabled
* 1 = history is enabled
* >1 = history is enabled, throttled down

The throttling is implemented in log_history()/watch_history() by this algorithm:
the very first history event is recorded, then all changed to the data are ignored until "period" seconds has elapsed. Then the next history event will be recorded, and following
changes will be ignored until "period" second elapses, and so forth. Period value "1" has special meaning - there is no throttling, all history events are logged.

This information is from [https://midas.triumf.ca/elog/Midas/1087 MidasElog#1087].

=== Sequence for frontend History Event ===
The main steps for the frontend History Event is as follows:

* The user code in the frontend equipment Event Readout routines reads the data, placing it into a MIDAS data bank
* In mfe.c, if the key [[/Equipment ODB tree#Log history|Log history]] > 0 (i.e. the history system is enabled for this equipment), this data bank is written into ODB (by update_odb()).
* This ODB write triggers an ODB hot-link into mlogger
* The hot-link calls mlogger.c::log_history(), which calls hs_write() to write the data into the history file. The frequency of the history writes is specified by the number of seconds between writes stored in the ODB key [[/Equipment ODB tree#Log history|Log history]].
* history.c::hs_write() or history_odbc::hs_write_odbc() writes the data into a history file or into an SQL database.

= How to Write History events =
* Enable desired [[#To enable Frontend History Events|Frontend]] and/or [[#To enable Virtual History Events|Virtual]] history events
* [[#Select the history driver|Select the history driver]]
* Follow instructions below for selected driver:
** [[#Write MIDAS-History events|Write MIDAS-history events]]
** [[#Write ODBC-History events|Write ODBC-history events]]
** [[#Write SQLITE-History events|Write SQLITE-history events]]
** [[#Write MYSQL-History events|Write MYSQL-history events]]
** [[#Write FILE-History events|Write FILE-history events]]

== Select the history driver ==

* Select the '''[[#History drivers|history driver]]''' and therefore the [[Keys in the ODB /Logger/History subtree#<history-logging-channel> subtrees|history-logging-channel]] to use to write the MIDAS-history files.
* set [[Keys in the ODB /Logger/History subtree#Active|Active]] ODB key in the selected [[Keys in the ODB /Logger/History subtree#<history-logging-channel> subtrees|history-logging-channel]] to "y". All other logging channels should be disabled.
* set [[/History ODB tree#LoggerHistoryChannel|LoggerHistoryChannel]] ODB key to the selected [[#History drivers|History channel]].

== Write MIDAS-History events ==
* Enable desired [[#To enable Frontend History Events|Frontend]] and/or [[#To enable Virtual History Events|Virtual]] history events
* Perform steps in [[#Select the history driver|Select History Driver]] with [[#History drivers|history driver]] "MIDAS".
* Restart [[mhttpd]] and [[mlogger]], start the run.

MIDAS-history files will be saved into the [[#Location of History Files|MIDAS-history data directory]].

== Write ODBC-History events ==

See [[#MIDAS ODBC SQL History system|MIDAS ODBC SQL History system]].
* If necessary, [[#Set up a MySQL database|set up a MySQL database]].
* Enable desired virtual and frontend history events (see [[#Virtual History Event|Virtual history events]] and [[#Frontend History Event|Frontend history events]]).
* Perform steps in [[#Select the history driver|Select History Driver]] with [[#History drivers|history driver]] "ODBC".
* Check ODB Key [[Keys in the ODB /Logger/History subtree#Writer_ODBC_DSN|Writer_ODBC_DSN]] is set to "history writer" and Key [[Keys in the ODB /Logger/History subtree#Reader_ODBC_DSN|Reader_ODBC_DSN]] is set to "history reader" (the default values)
* set [[Keys in the ODB /Logger/History subtree#Debug|Debug]] to 1 if desired
* restart [[mlogger]] in verbose mode (i.e. mlogger -v), and observe how it issues SQL commands to create the tables and columns corresponding to MIDAS-history events and tags.
* restart [[mhttpd]]

'''Notes'''
# [[mlogger]] is programmed to raise alarms if connection to SQL database is interrupted or if some events cannot be written into the database (i.e. data type mismatch, SQL syntax errors, etc). The traditional MIDAS-history never raised alarms because it "never failed" - other than from "disk full" errors, which are immediately obvious.
# With ODBC History enabled, [[mhttpd]] will only use history information from the SQL database to make history plots and to extract history variable names for the history plot editor. The ODB {{Odbpath|path=/History}} subtrees[[/History ODB tree#Tags subtree|Tags]] and [[/History ODB tree#Events subtree|Events]] are not used.
# variable names presented to the user may change from MIDAS names to SQL names (use the history panel "Label" text fields to create permanent plot labels).

== Write SQLITE-history events ==

No special configuration required.

* Enable desired virtual and frontend history events (see [[#Virtual History Event|Virtual history events]] and [[#Frontend History Event|Frontend history events]]).
* Perform steps to [[#Select the history driver|Select History Driver]] with [[#History drivers|history driver]] "SQLITE".

* Create a directory for storage of sqlite history files
* save its path in the ODB key [[Keys in the ODB /Logger/History subtree#Sqlite dir|Sqlite dir]]
* restart [[mhttpd]] and [[mlogger]]

In the history data directory (in [[Keys in the ODB /Logger/History subtree#Sqlite dir|Sqlite dir]]) you should see files with name "mh_xxx.sqlite3" corresponding to your equipment and variable names.

== Write MYSQL-history events ==

* Enable desired virtual and frontend history events (see [[#Virtual History Event|Virtual history events]] and [[#Frontend History Event|Frontend history events]]).
* Perform steps to [[#Select the history driver|Select History Driver]] with [[#History drivers|history driver]] "MYSQL".
* Read the following instructions to [[setup MySQL database for MIDAS]]:
* install mariadb-10.4
<pre>
macos:
sudo port install mariadb-10.4
ln -s ln -fs /opt/local/lib/mariadb-10.4/bin/mysql_config ~/bin/
mysql_config --version
10.4.13
</pre>
* rebuild midas, check the cmake finds the correct mysql: MIDAS: Found MySQL version 10.4.13
* create $HOME/.my.cnf
<pre>
[mysqld]
#port=3306
socket=/Users/olchansk/mysql/mysql.sock
datadir=/Users/olchansk/mysql/db
pid-file=/Users/olchansk/mysql/mysqld.pid

[client]
#port=3306
socket=/Users/olchansk/mysql/mysql.sock
</pre>
* mkdir $HOME/mysql
* start mysql:
<pre>
macos: /opt/local/lib/mariadb-10.4/bin/mysqld
</pre>
* configure the database: start the "mysql" utility (/opt/local/lib/mariadb-10.4/bin/mysql)
<pre>
NOTE: instructions thanks to Ben S.
NOTE: use some other passwords instead of the example "reader_password" and "writer_password"

MariaDB [(none)]> CREATE DATABASE IF NOT EXISTS history;
Query OK, 1 row affected (0.003 sec)

MariaDB [(none)]> CREATE USER history_reader@'localhost' IDENTIFIED BY 'reader_password';
Query OK, 0 rows affected (0.027 sec)

MariaDB [(none)]> GRANT SELECT ON history.* TO history_reader@'localhost';
Query OK, 0 rows affected (0.016 sec)

MariaDB [(none)]> CREATE USER 'history_writer'@'localhost' IDENTIFIED BY 'writer_password';
Query OK, 0 rows affected (0.014 sec)

MariaDB [(none)]> GRANT SELECT,INSERT,CREATE,ALTER,INDEX ON history.* TO 'history_writer'@'localhost';
Query OK, 0 rows affected (0.017 sec)

MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.001 sec)
</pre>
* try connecting to both users:
<pre>
/opt/local/lib/mariadb-10.4/bin/mysql -u history_reader -p ### should ask for reader password
/opt/local/lib/mariadb-10.4/bin/mysql -u history_writer -p ### should ask for writer password
</pre>
* create midas config files:
** mysql_writer.txt
<pre>
#server=localhost
#port=xxx
database=history
socket=/Users/olchansk/mysql/mysql.sock
user=history_writer
password=writer_password ### change this!!!
#buffer=1000
</pre>
** mysql_reader.txt
<pre>
#server=localhost
#port=xxx
database=history
socket=/Users/olchansk/mysql/mysql.sock
user=history_reader
password=reader_password ### change this!!!
</pre>
* test MIDAS connection
<pre>
mh2sql --mysql mysql_writer.txt
(there should be no errors printed)
</pre>
* if desired, import midas history from .hst files:
<pre>
mh2sql --mysql mysql_writer.txt *.hst
Reading 130813.hst
[mh2sql,ERROR] [history_schema.cxx:1165:Mysql::Prepare,ERROR] mysql_query(SELECT event_name, table_name, itimestamp FROM _history_index WHERE table_name!='';) error 1146 (Table 'history._history_index' doesn't exist)
[mh2sql,INFO] Adding SQL table "rpcexample", status 311
[mh2sql,INFO] Adding SQL table "rpcexample_1596127680", status 1
[mh2sql,INFO] Adding SQL table "_history_index", status 1
[mh2sql,INFO] Adding column "event_name" to SQL table "_history_index", status 1
...
[mh2sql,INFO] Adding SQL table "run_transitions", status 311
[mh2sql,INFO] Adding SQL table "run_transitions_1596127681", status 1
[mh2sql,INFO] Adding column "state" to SQL table "run_transitions_1596127681", status 1
[mh2sql,INFO] Adding column "run_number" to SQL table "run_transitions_1596127681", status 1
Reading 130815.hst
...
</pre>
* enable MYSQL history in mlogger:
** change ODB /Logger/History/MYSQL/Active to "y", "mysql writer" and "mysql reader" to location of these files (leave the default values if these files are in the experiment directory).
* enable MYSQL history in mhttpd:
** change ODB /History/LoggerHistoryChannel to "MYSQL"
* restart mlogger, restart mhttpd
...........

== Write FILE-history events ==

* Enable desired virtual and frontend history events (see [[#Virtual History Event|Virtual history events]] and [[#Frontend History Event|Frontend history events]]).
* Perform steps to [[#Select the history driver|Select History Driver]] with [[#History drivers|history driver]] "SQLITE".
* Assign directory path for saved files using the ODB key [[Keys in the ODB /Logger/History subtree#History dir|History dir]].
...........

<br>
---------------
<br>

= History Events =
The following information is derived from [https://midas.triumf.ca/elog/Midas/1087 MidasElog#1087].

A "history event" is a history atomic unit of data. Associated with each history event is a timestamp (unix time),
a name (limited to NAME_LENGTH in the old history) and a list of history tags that describe the individual data
values inside the history event.

When making history plots in [[mhttpd]] (see [[History Page]]) for each curve on the plot, one selects a history event (from the list
of currently active events, recently active events or the list of all events that ever existed), then from the list of tags
inside the history event one selects the particular variable that will be plotted.

== Old MIDAS history events ==
In the old MIDAS history, all history events are written into one history file (.hst file + optional .def and .idx event definition and time index files
which can be/are regenerated automatically from the .hst file). History events are identified by 16-bit history event IDs, the persistent mapping
from history event names and the 16-bit history event IDs is stored in ODB subtree {{Odbpath|path=/History/Events}}. In addition the list of all known history event tags is
stored in ODB subtree {{Odbpath|path=/History/Tags}}. For per-equipment history, the 16-bit history event ID is the value of ODB key [[/Equipment ODB tree#Event ID|/Equipment/<equipment-name>/Common/Event ID]].

== SQL history events ==
In the SQL history (MySQL, SQLITE, etc), each history event is an SQL table. The history event tags are the SQL table columns.

== FILE history events ==
In the new FILE history, each history event is written into a separate file, tag definition are recorded in text format in the file header, history event
data is appended to the file in binary format (fixed record size). If the history event definition is changed, a new file will be started.

= History Event Construction =
The [[mlogger]] creates [[#Frontend history event]]s in open_history() by parsing the ODB subtree {{Odbpath|path=/equipment/<equipment-name>/variables}}. Each ODB entry under the {{Odbpath|path=variables/}} subtree is referred to as a "variable". Each variable can be a single ODB value, an array of ODB values, or a subdirectory (corresponding to TID_STRUCT structured data banks). As each variable is processed, one or more tags are created to describe it.

Single ODB values will generally produce a single tag, while arrays can produce (depending whether the array is "[[#Named Arrays|Named]]" or not)
* multiple tags - one per array element ("Named" array) or
* one single tag - describing the whole array

The code can generate two types of history:
<dl>
<dt> "per-equipment" history</dt>
<dd> will have the tags for all variables concatenated together into one single history event</dd>
<dt> "per-variable" history</dt>
<dd> will have one history event defined for each variable. Inside could be one tag (for single odb values and unnamed array) or multiple tags (for named arrays and structured data
banks).</dd>
</dl>

Per-equipment history is the original MIDAS history implementation. Per-variable history was added to permit efficient data storage in SQL tables. Its initial implementation used 1 ODB hotlink for each variable and it was easy to exceed the maximum permitted number of ODB hotlinks (db_open_record()). To reduce consumption of hotlinks, db_watch() has been implemented and now per-variable history only uses 1 ODB hotlink per equipment.

;NOTE
: With db_watch, per-equipment history is no longer available. '''Per-variable history is the new default''' (and the only option).

[https://midas.triumf.ca/elog/Midas/1087 MidasElog#1087]

= Construction of history event tags =
Currently,
* single ODB values are treated as arrays of length "1"
* string ODB entries are not permitted

History event tags are constructed in the [[mlogger]] add_equipment(). Presently there are some quirks with the generation of history event tags (see [https://midas.triumf.ca/elog/Midas/1088]),
e.g. single odb values are treated as arrays of length "1" and "string" ODB entries are not permitted.

For variables of type TID_KEY (subdirectories, corresponding to TID_STRUCT structured banks), one tag is generated for each subdirectory entry. Tag names for {{Odbpath|path=/equipment/<equipment-name>/variables/aaa/bbb}} will be "aaa_bbb" (with an underscore). Subdirectory entries of type TID_KEY and TID_LINK are currently not supported.

For "[[#Named Arrays|Named Arrays]]", individual tags are generated for each array element. Tag names are taken from the ''Names'' array. For empty tag names (empty names array), tags are "aaa_0", "aaa_1", etc (for {{Odbpath|path=/equipment/<equipment-name>/variables/aaa}}).

For "[[#Named Arrays|Single Names Arrays]]", tag names have the variable name appended (with a space), for {{Odbpath|path=/equipment/<equipment-name>/variables/aaa}} and an empty names array, tags will be "aaa_0 aaa", "aaa_1 aaa", etc. For a populated names array, the tags will be "name0 aaa", "name1 aaa", etc.

For unnamed arrays and single odb variables (in ODB, single odb variables are arrays of length 1), a single tag is generated.

== Named Arrays ==

'''Named Arrays''' have a "Names" entry in {{Odbpath|path=/equipment/<equipment-name>/settings}}.

For example, to add names to {{Odbpath|path=/equipment/<equipment-name>/variables/aaa}}, create a string array {{Odbpath|path=/equipment/<equipment-name>/settings/names aaa}}. The names array should be at least as long as the corresponding data array. Individual entries in the names array can be left blank (tag names will be "aaa_0", "aaa_1", etc). Duplicate tag names are not permitted.

A single "Names" entry can be created to name all arrays in variables with the same names ('''"Single Names"''').

Create {{Odbpath|path=/equipment/<equipment-name>/settings/names}} and arrays {{Odbpath|path=/equipment/<equipment-name>/variables/aaa}} and {{Odbpath|path=/equipment/<equipment-name>/variables/bbb}} will have
history tags "name0 aaa", "name1 aaa", "name0 bbb", "name1 bbb", etc. If "names" are left blank, tag names will be "aaa_0 aaa", "aaa_1 aaa", "bbb_0 bbb", "bbb_1 bbb", etc.

In the [[mhttpd]] web server [[History Page]] variables viewer, "single name" arrays are displayed in a 2D table.

This information is from [https://midas.triumf.ca/elog/Midas/1088].

= Historical note on MIDAS-History Tag names =
Tags are only used for MIDAS-history events.

The mlogger/mhttpd implementation of <span style="color:purple; font-style:italic">/History/Tags</span> has proved troublesome and we are moving away from it.

;During the transition period:
: Implementations starting with Version 4435 of [[mlogger]] and [[mhttpd]] (i.e. "new" versions) will now work '''without''' <span style="color:purple; font-style:italic">/History/Tags</span>. The history tags are read directly from the history files themselves. To run in this mode, set ODB key [[/History ODB tree#DisableTags|/History/DisableTags]] to "y". Existing [[/History ODB tree#Tags subtree|/History/Tags subtree]] will be automatically deleted.

Two downsides to running like this are
* it is slower and tags become non-persistent
* if some frontends have not been running for a while, their variables may vanish from the history panel editor.

== New-style History Tags ==
For the above two reasons, using <span style="color:purple; font-style:italic">/History/Tags</span> is still recommended.
However, the format of the tags is now changed to simplify management and reduce ODB size.

; For NEW implementations
: the key [[/History ODB tree#CreateOldTags|CreateOldTags]] will not be present. "New-style" history tags will be created automatically in the [[/History ODB tree#Tags subtree|Tags subtree]].

;If you have old-style tags
: [[mlogger]] will automatically convert the old tags to this new format (this is why you should make a backup of your ODB).

** using "old" (i.e. pre Version 4435) mlogger with "new" mhttpd is OK: new mhttpd understands both formats of /History/Tags.
** using "old" mhttpd with "new" mlogger is OK provided the ODB key [[/History ODB tree#CreateOldTags|/History/CreateOldTags]] is set to "y" before starting mlogger.

== Old-style History Tags ==
Prior to versions 4435 of [[mlogger]] and [[mhttpd]], old-style history tags were generated.
See
* http://ladd00.triumf.ca/~daqweb/doc/midas-old/html/F_History_logging.html
for details.

<br>
---------------
<br>

= MIDAS ODBC SQL History system =

This section describes the internal workings of the MIDAS SQL history system.

The SQL history system implements the MIDAS history using a SQL database for data storage instead of flat binary files.

The SQL history code lives in a separate files history.h and history_sql.cxx. The present implementation uses the ODBC API for accessing SQL databases.
MySQL is fully supported, Pgsql support is partially implemented, support for arbitrary ODBC drivers should be easy to add in history_sql and most SQL operations are done using an interface class.

Preliminary testing with MySQL and PgSQL indicates that disk space requirements are the same for both storage mechanisms using flat binary files or either of the 2 SQL databases. All 3 tested storage mechanisms store the data uncompressed. (Observed gzip-1 compression ratios from CERN-ALPHA history files are 50% or better).

Preliminary testing also indicates that MySQL database is "not too slow" for use by T2K/ND280 slow control system. Using MySQL, performance seems to improve somewhat compared to traditional MIDAS-history because each SQL table is stored into a separate file, compared to everything-in-one-file in traditional MIDAS-history.

== Set up a MySQL database ==

In order to setup a MySQL database, the following is needed:

* the server hostname and port number
* database name
* user name and password.

It is recommended to create 3 different users:

* a "root" user with full priveleges,
* a "reader" user with read-only priveleges
* and a "writer" user with "insert, create table, and add column" priveleges (no permission to drop table, remove columns, etc)

=== Example ===
#Setup a MySQL database (RHEL/SL 5)

ssh root@localhost
service mysqld start
/usr/bin/mysql_secure_installation (enter current password for root: press <enter>, then accept default answer "Y" to all questions)
mysqladmin -p create history
using mysql-administrator, create 2 users:
history_writer, with database "history" schema privileges: select, insert, create, alter, index
history_reader, with database "history" schema privileges: select

#Setup $HOME/.odbc.ini file:

cat $HOME/.odbc.ini
[history_writer]
Description = history_writer
Driver = /usr/lib64/libmyodbc3.so
SERVER = ladd05.triumf.ca
PORT =
Database = history
OPTION = 3
SOCKET =
User = history_writer
Password = ******
[history_reader]
Description = history_reader
Driver = /usr/lib64/libmyodbc3.so
SERVER = ladd05.triumf.ca
PORT =
Database = history
OPTION = 3
SOCKET =
User = history_reader
Password =

Note: if you are using mhist you will need a separate .odbc.ini file for each user that you have connecting to the database (through MIDAS).

== Layout of SQL tables ==

* One table is created for each history event. Table name is the same as the history event name (as reported by [[mhdump]]).
* one column is created for each history tag (history variable). Column name is the same as the tag name (as reported by mhdump). For array tags, one column is created for each array element: array a[3] will produce columns a_0, a_1 and a_2.
* two special columns are always created: _i_time and _t_time are the timestamps in the UNIX-time integer format and in the SQL "time" format. (A priori, it is not clear which timestamp format is more useful for end users. mhttpd uses the _i_time integer time format). Note that this timestamp is the time when mlogger receives the hot-link signal (see above) and it can be a few seconds behind the time when the user placed the data into the MIDAS bank. For high precision (and sub-second time resolution) time stamps, users should generate their own timestamps and include them as part of the data itself.
* a special table called _history_index is created to remember the mapping between SQL names and MIDAS names for event names, variable name and tag names.

=== Schema Evolution ===

* when new data fields are added, new SQL columns are created automatically
* existing SQL columns are never deleted
* arrays are expanded automatically, and arrays never shrink.
* if the type of a data field changes, a conflict may occur with existing SQL columns, for example if previous type was "FLOAT" and new type is "DOUBLE", the existing SQL columns would have the SQL data type "float", too narrow to store new values. mlogger will complain about this. Such conflicts may be resolved by changing the type of the SQL column using SQL tools. In general, SQL names are created from MIDAS names by replacing all non-alphanumerical characters to an underscore "_". If this procedure creates duplicate column names, the column names and made unique by appending them with a random number, i.e. MIDAS names "A+B" and "A-B" will be translated to "A_B" and "A_B_12345" (random integer). True MIDAS names are saved into the _history_index.

[[Category:history]]

MidasWiki:Community portal

2025-01-20T20:45:42Z

Lindner:

Our main forum for exchanging information about MIDAS is an ELOG list, available here

[https://midas.triumf.ca/forum https://midas.triumf.ca/forum]

All users of MIDAS are encouraged to sign-up for the ELOG forum.

Another source of information is the [https://bitbucket.com/tmidas/midas Bitbucket Repository] of MIDAS, where one can see the latest changes to the software.

There is a [https://midas.triumf.ca/MidasWiki/index.php/FAQ FAQ on this wiki].

Finally, you can read presentations from the [https://indico.psi.ch/conferenceDisplay.py?ovw=True&confId=3793 2015 MIDAS Workshop], [https://indico.triumf.ca/event/428/ 2017 MIDAS Workshop], [https://indico.triumf.ca/event/429/ 2019 MIDAS Workshop] and [https://indico.psi.ch/event/15025/timetable/#20230912 2023 MIDAS workshop]

MidasWiki:Community portal

2025-01-20T20:45:21Z

Lindner:

Our main forum for exchanging information about MIDAS is an ELOG list, available here

[https://midas.triumf.ca/forum https://midas.triumf.ca/forum]

All users of MIDAS are encouraged to sign-up for the ELOG forum.

Another source of information is the [https://bitbucket.com/tmidas/midas Bitbucket Repository] of MIDAS, where one can see the latest changes to the software.

There is a [https://midas.triumf.ca/MidasWiki/index.php/FAQ FAQ on this wiki].

Finally, you can read presentations from the [https://indico.psi.ch/conferenceDisplay.py?ovw=True&confId=3793 2015 MIDAS Workshop], [https://indico.triumf.ca/conferenceDisplay.py?confId=2342 2017 MIDAS Workshop], [https://indico.triumf.ca/event/429/ 2019 MIDAS Workshop] and [https://indico.psi.ch/event/15025/timetable/#20230912 2023 MIDAS workshop]

MidasWiki:Community portal

2025-01-20T20:44:44Z

Lindner:

Our main forum for exchanging information about MIDAS is an ELOG list, available here

[https://midas.triumf.ca/forum https://midas.triumf.ca/forum]

All users of MIDAS are encouraged to sign-up for the ELOG forum.

Another source of information is the [https://bitbucket.com/tmidas/midas Bitbucket Repository] of MIDAS, where one can see the latest changes to the software.

There is a [https://midas.triumf.ca/MidasWiki/index.php/FAQ FAQ on this wiki].

Finally, you can read presentations from the [https://indico.psi.ch/conferenceDisplay.py?ovw=True&confId=3793 2015 MIDAS Workshop], [https://indico.triumf.ca/conferenceDisplay.py?confId=2342 2017 MIDAS Workshop], [https://indico.triumf.ca/conferenceDisplay.py?confId=2438 2019 MIDAS Workshop] and [https://indico.psi.ch/event/15025/timetable/#20230912 2023 MIDAS workshop]

Main Page

2023-06-09T22:52:58Z

Lindner: /* News and Support */

= Introduction =

MIDAS is a modern data acquisition system developed at [http://www.psi.ch PSI] and [http://www.triumf.ca TRIUMF]. Supported hardware includes VME, Fastbus, CAMAC, RS232, GPIB, USB, ethernet, fiber optic and MSCB attached data acquisition devices. MIDAS is written in C/C++ and runs on Linux, MacOS and MS Windows. It is licensed under the [http://www.gnu.org/copyleft/gpl.html GNU Public License].

= Documentation =

* [[Midas documentation| Main Documentation]]
* [[Installation | Installation Instructions]]
* [[Changelog | Changelog / Upgrade Instructions]]
* [https://daq.triumf.ca/~daqweb/doc/midas-develop/html/modules.html <i>Doxygen</i>] Code structure, functions, etc...
* [https://bitbucket.com/tmidas/midas git repository on bitbucket] Source code, bug tracking, etc
* [https://midas.triumf.ca/forum MIDAS forum] Discussion, announcements, problem reports, requests for improvements

= News and Support=

For news about bug fixes and new releases, go to the
[https://midas.triumf.ca/forum MIDAS forum] hosted by TRIUMF using
the [http://elog.psi.ch/elog ELOG] system. Users can register in this system to be notified automatically via
E-mail when new entries are submitted.

Another source of information is the [https://bitbucket.com/tmidas/midas Bitbucket Repository] of MIDAS, where one can see the latest changes to the software.

Finally, you can read presentations from
* [https://indico.psi.ch/conferenceDisplay.py?ovw=True&confId=3793 2015 MIDAS Workshop]
* [https://meetings.triumf.ca/event/428/ 2017 MIDAS Workshop]
* [https://meetings.triumf.ca/event/429/ 2019 MIDAS Workshop]

= Related packages =

* [http://elog.psi.ch/elog ELOG] - Electronics Logbook from PSI
* [[ROOT]] - data analysis package from CERN
* [[ROOTANA]] - ROOT-based analyzer for MIDAS
* [[ROODY]] - viewer for online histograms
* http://elog.psi.ch/rome/ - MEG/PSI data analysis package

Main Page

2023-06-09T20:01:06Z

Lindner: /* News and Support */

= Introduction =

MIDAS is a modern data acquisition system developed at [http://www.psi.ch PSI] and [http://www.triumf.ca TRIUMF]. Supported hardware includes VME, Fastbus, CAMAC, RS232, GPIB, USB, ethernet, fiber optic and MSCB attached data acquisition devices. MIDAS is written in C/C++ and runs on Linux, MacOS and MS Windows. It is licensed under the [http://www.gnu.org/copyleft/gpl.html GNU Public License].

= Documentation =

* [[Midas documentation| Main Documentation]]
* [[Installation | Installation Instructions]]
* [[Changelog | Changelog / Upgrade Instructions]]
* [https://daq.triumf.ca/~daqweb/doc/midas-develop/html/modules.html <i>Doxygen</i>] Code structure, functions, etc...
* [https://bitbucket.com/tmidas/midas git repository on bitbucket] Source code, bug tracking, etc
* [https://midas.triumf.ca/forum MIDAS forum] Discussion, announcements, problem reports, requests for improvements

= News and Support=

For news about bug fixes and new releases, go to the
[https://midas.triumf.ca/forum MIDAS forum] hosted by TRIUMF using
the [http://elog.psi.ch/elog ELOG] system. Users can register in this system to be notified automatically via
E-mail when new entries are submitted.

Another source of information is the [https://bitbucket.com/tmidas/midas Bitbucket Repository] of MIDAS, where one can see the latest changes to the software.

Finally, you can read presentations from
* [https://indico.psi.ch/conferenceDisplay.py?ovw=True&confId=3793 2015 MIDAS Workshop]
* [https://meetings.triumf.ca/event/428/ 2017 MIDAS Workshop]
* [https://indico.triumf.ca/conferenceDisplay.py?confId=2438 2019 MIDAS Workshop]

= Related packages =

* [http://elog.psi.ch/elog ELOG] - Electronics Logbook from PSI
* [[ROOT]] - data analysis package from CERN
* [[ROOTANA]] - ROOT-based analyzer for MIDAS
* [[ROODY]] - viewer for online histograms
* http://elog.psi.ch/rome/ - MEG/PSI data analysis package

MIDAS Workshops

2023-06-09T19:48:48Z

Lindner: /* 2017 Midas workshop */

Copy of 2017 and 2019 MIDAS workshop slides

=== 2017 Midas workshop ===

Stefan Ritt -
[[File:Stefan_introduction.pdf|thumb|introduction]]

MIDAS Workshops

2023-06-09T19:48:30Z

Lindner: /* 2017 Midas workshop */

Copy of 2017 and 2019 MIDAS workshop slides

=== 2017 Midas workshop ===

Stefan Ritt -
[[File:Stefan_introduction.pdf|thumb|introduction]]

https://daq00.triumf.ca/MidasWiki/index.php/File:Stefan_introduction.pdf

File:Stefan introduction.pdf

2023-06-09T19:46:50Z

Lindner:

MIDAS Workshops

2023-06-09T19:43:49Z

Lindner: /* 2017 Midas workshop */

Copy of 2017 and 2019 MIDAS workshop slides

=== 2017 Midas workshop ===

Stefan Ritt -
[[File:Stefan introduction|thumb|introduction]]

MIDAS Workshops

2023-06-09T17:58:25Z

Lindner: /* 2017 Midas workshop */

Copy of 2017 and 2019 MIDAS workshop slides

=== 2017 Midas workshop ===

Stefan Ritt - Introduction