Hs config: Difference between revisions

From MidasWiki
Jump to navigation Jump to search
No edit summary
 
(8 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{Pagelinks}}
This information has been added to documentation under [[History System]], [[/Logger ODB tree]], [[/History ODB tree]]
= MidasHistoryInterface =
= MidasHistoryInterface =


New history configuration defines history channels similar to logger channels. Each history channel creates a history interface class (class MidasHistoryInterface). The mlogger can use this interface to write data into the history, mhttpd, mhist & co can use this interface to read data from the history. The interface is defined in history.h
New history configuration defines history channels similar to logger channels. Each history channel creates a history interface class (class MidasHistoryInterface). The mlogger can use this interface to write data into the history, mhttpd, mhist & co can use this interface to read data from the history. The interface is defined in history.h
* http://ladd00.triumf.ca/~daqweb/doc/midas/include/history.h


= History drivers =
= History drivers =
Line 12: Line 12:
* "ODBC" stores history data in SQL database using the ODBC abstraction layer. MySQL is fully supported, Pgsql support is partially implemented, support for arbitrary ODBC driver is easy to add.
* "ODBC" stores history data in SQL database using the ODBC abstraction layer. MySQL is fully supported, Pgsql support is partially implemented, support for arbitrary ODBC driver is easy to add.
* "SQLITE" stores data in SQLITE3 .sqlite3 files. The SQL table format is similar to ODBC history, performance is similar, no database server required.
* "SQLITE" stores data in SQLITE3 .sqlite3 files. The SQL table format is similar to ODBC history, performance is similar, no database server required.
* "MYSQL" stores history data in a MySQL database using MySQL client libraries.
* "FILE" is like "MIDAS", but stores each variable in its own file, making it much easier to trace the history of a particular variable.


= ODB tree =
= ODB tree =


* /Logger/
* /Logger/
** History/0/ --- history channel 0 (by default: "MIDAS")
** History/MIDAS/
*** Type --- driver type ("MIDAS")
*** Type --- driver type ("MIDAS")
*** Active --- y/n --- tells mlogger to write data into this channel
*** Active --- y/n --- tells mlogger to write data into this channel
*** Debug --- debug level
*** Debug --- debug level
*** PerVariableHistory --- tells mlogger to enable the Per-variable history logging mode
*** PerVariableHistory --- tells mlogger to enable the Per-variable history logging mode
** History/1/ --- history channel 1 (by default: "ODBC")
** History/ODBC/
*** Type --- driver type ("ODBC")
*** Type --- driver type ("ODBC")
*** Active --- y/n
*** Active --- y/n
Line 27: Line 29:
*** Writer_ODBC_DSN --- ODBC Data Source Name for writing (used by mlogger)
*** Writer_ODBC_DSN --- ODBC Data Source Name for writing (used by mlogger)
*** Reader_ODBC_DSN --- ODBC Data Source Name for reading (used by mhttpd, mhist)
*** Reader_ODBC_DSN --- ODBC Data Source Name for reading (used by mhttpd, mhist)
** History/2/ --- history channel 2 (by default: "SQLITE")
** History/SQLITE/ --- history channel 2 (by default: "SQLITE")
*** Type --- driver type ("SQLITE")
*** Type --- driver type ("SQLITE")
*** Active --- y/n
*** Active --- y/n
*** Debug --- debug level
*** Debug --- debug level
*** Sqlite dir --- "/file/system/path" location of SQLITE database files
*** Sqlite dir --- "/file/system/path" location of SQLITE database files
** History/MYSQL/
*** Type --- driver type ("MYSQL")
*** Active --- y/n
*** Debug --- debug level
*** MYSQL Writer --- File defining connection details for writing (used by mlogger)
*** MYSQL Reader --- File defining connection details for reading (used by mhttpd, mhist)
** History/FILE/
*** Type --- driver type ("FILE")
*** Active --- y/n --- tells mlogger to write data into this channel
*** Debug --- debug level
*** History dir --- where to store history files
* /History/
* /History/
** LoggerHistoryChannel --- "0" --- default history channel for mhttpd, mhist & co
** LoggerHistoryChannel --- "MIDAS" --- default history channel for mhttpd, mhist & co


Obsoleted ODB settings:
Obsoleted ODB settings:


* /Logger/WriteFileHistory (y) --- use /Logger/History/0/Active (enable/disable writing of .hst history files)
* /Logger/WriteFileHistory (y) --- use /Logger/History/*/Active (enable/disable writing of .hst history files)
* /Logger/ODBC* --- use /Logger/History/1/Writer_ODBC_DSN
* /Logger/ODBC* --- use /Logger/History/ODBC/Writer_ODBC_DSN
* /History/PerVariableHistory --- use /Logger/History/0/PerVariableHistory (is ODBC or SQLITE history is enable, PerVariableHistory mode is forced to "y")
* /History/PerVariableHistory --- use /Logger/History/MIDAS/PerVariableHistory
* /History/ODBC* --- use /Logger/History/1/Reader_ODBC_DSN
* /History/ODBC* --- use /Logger/History/ODBC/Reader_ODBC_DSN


Other ODB settings:
Other ODB settings:


* /Logger/History dir --- location of "MIDAS" history data files
* /Logger/History dir --- location of "MIDAS" history data files (optional; if not preset, "/Logger/Data dir" will be used instead)
* /History/Events, /History/Tags, /History/DisableTags --- private data storage by "MIDAS" history. (not used by ODBC and SQLITE history)
* /History/Events, /History/Tags, /History/DisableTags --- private data storage by "MIDAS" history. (not used by ODBC/SQLITE/MYSQL history)
 
= Using the MIDAS history =
 
No special configuration required. MIDAS history is enabled by default, writing history files into the MIDAS data directory. If history data is important, it is recommended to write it to a directory that is backed up or archived. (the MIDAS data directory is usually located on a scratch data disk without backups). Create a directory for storage of midas history files, save it's path in ODB "/Logger/History dir", set "/Logger/History/MIDAS/Active" to "y", set "/History/LoggerHistoryChannel" to "MIDAS", restart mhttpd, restart mlogger. In the history data directory you should see files with name "YYMMDD.hst" corresponding to each day of midas operation. Contents of .hsr files can be examined directly using [[mhdump]] or indirectly using [[mhist]] or the history display page of [[mhttpd]]
 
= Changing which history driver is used =
 
In most experiments you probably only want one history system to be active. However, Midas does allow you to write history data to multiple locations by having multiple drivers active at once. The other core Midas tools (mhttpd etc) will only ever read from one history driver though.
 
To define which history channels data is logged to, set "/Logger/History/*/Active" to "y". For example, if you previously were using the MIDAS driver and want to migrate to the MYSQL driver, you would set "/Logger/History/MIDAS/Active" to "n" and "/Logger/History/MYSQL/Active" to "y".
 
To tell mhttpd etc which channel to get its data from, set "/History/LoggerHistoryChannel" (e.g. to "MYSQL").
 
After changing which history channels are active, you must **restart mlogger and mhttpd**.
 
= Using the ODBC history =
 
See [[Keys_in_the_ODB_/Logger/History_subtree#ODBC_subtree]].
 
= Using the SQLITE history =
 
No special configuration required. Create a directory for storage of sqlite history files, save it's path in ODB "/Logger/History/SQLITE/Sqlite dir". In the history data directory you should see files with name "mh_xxx.sqlite3" corresponding to your equipment and variable names.
 
= Using the MYSQL history =
 
After creating a MySQL database (and accounts with read/write and read-only permissions), create files that tell Midas how to connect to the database (see [[Keys_in_the_ODB_/Logger/History_subtree#MYSQL_subtree]] for syntax). Specify the name of these files in "/Logger/History/MYSQL/MYSQL Writer" and "/Logger/History/MYSQL/MYSQL Reader".
 
[[Category:History]]

Latest revision as of 14:35, 29 August 2019


This information has been added to documentation under History System, /Logger ODB tree, /History ODB tree

MidasHistoryInterface

New history configuration defines history channels similar to logger channels. Each history channel creates a history interface class (class MidasHistoryInterface). The mlogger can use this interface to write data into the history, mhttpd, mhist & co can use this interface to read data from the history. The interface is defined in history.h

History drivers

The MIDAS distribution contains these drivers. More drivers can be easily added by the user (create history_xxx.cxx, add constructor to hs_get_histrory(), create the ODB /Logger/History/x/channel).

  • "MIDAS" is the traditional MIDAS history, data is stored in the .hst binary files. (format is documented here mhformat, see also mhist and mhdump)
  • "ODBC" stores history data in SQL database using the ODBC abstraction layer. MySQL is fully supported, Pgsql support is partially implemented, support for arbitrary ODBC driver is easy to add.
  • "SQLITE" stores data in SQLITE3 .sqlite3 files. The SQL table format is similar to ODBC history, performance is similar, no database server required.
  • "MYSQL" stores history data in a MySQL database using MySQL client libraries.
  • "FILE" is like "MIDAS", but stores each variable in its own file, making it much easier to trace the history of a particular variable.

ODB tree

  • /Logger/
    • History/MIDAS/
      • Type --- driver type ("MIDAS")
      • Active --- y/n --- tells mlogger to write data into this channel
      • Debug --- debug level
      • PerVariableHistory --- tells mlogger to enable the Per-variable history logging mode
    • History/ODBC/
      • Type --- driver type ("ODBC")
      • Active --- y/n
      • Debug --- debug level
      • Writer_ODBC_DSN --- ODBC Data Source Name for writing (used by mlogger)
      • Reader_ODBC_DSN --- ODBC Data Source Name for reading (used by mhttpd, mhist)
    • History/SQLITE/ --- history channel 2 (by default: "SQLITE")
      • Type --- driver type ("SQLITE")
      • Active --- y/n
      • Debug --- debug level
      • Sqlite dir --- "/file/system/path" location of SQLITE database files
    • History/MYSQL/
      • Type --- driver type ("MYSQL")
      • Active --- y/n
      • Debug --- debug level
      • MYSQL Writer --- File defining connection details for writing (used by mlogger)
      • MYSQL Reader --- File defining connection details for reading (used by mhttpd, mhist)
    • History/FILE/
      • Type --- driver type ("FILE")
      • Active --- y/n --- tells mlogger to write data into this channel
      • Debug --- debug level
      • History dir --- where to store history files
  • /History/
    • LoggerHistoryChannel --- "MIDAS" --- default history channel for mhttpd, mhist & co

Obsoleted ODB settings:

  • /Logger/WriteFileHistory (y) --- use /Logger/History/*/Active (enable/disable writing of .hst history files)
  • /Logger/ODBC* --- use /Logger/History/ODBC/Writer_ODBC_DSN
  • /History/PerVariableHistory --- use /Logger/History/MIDAS/PerVariableHistory
  • /History/ODBC* --- use /Logger/History/ODBC/Reader_ODBC_DSN

Other ODB settings:

  • /Logger/History dir --- location of "MIDAS" history data files (optional; if not preset, "/Logger/Data dir" will be used instead)
  • /History/Events, /History/Tags, /History/DisableTags --- private data storage by "MIDAS" history. (not used by ODBC/SQLITE/MYSQL history)

Using the MIDAS history

No special configuration required. MIDAS history is enabled by default, writing history files into the MIDAS data directory. If history data is important, it is recommended to write it to a directory that is backed up or archived. (the MIDAS data directory is usually located on a scratch data disk without backups). Create a directory for storage of midas history files, save it's path in ODB "/Logger/History dir", set "/Logger/History/MIDAS/Active" to "y", set "/History/LoggerHistoryChannel" to "MIDAS", restart mhttpd, restart mlogger. In the history data directory you should see files with name "YYMMDD.hst" corresponding to each day of midas operation. Contents of .hsr files can be examined directly using mhdump or indirectly using mhist or the history display page of mhttpd

Changing which history driver is used

In most experiments you probably only want one history system to be active. However, Midas does allow you to write history data to multiple locations by having multiple drivers active at once. The other core Midas tools (mhttpd etc) will only ever read from one history driver though.

To define which history channels data is logged to, set "/Logger/History/*/Active" to "y". For example, if you previously were using the MIDAS driver and want to migrate to the MYSQL driver, you would set "/Logger/History/MIDAS/Active" to "n" and "/Logger/History/MYSQL/Active" to "y".

To tell mhttpd etc which channel to get its data from, set "/History/LoggerHistoryChannel" (e.g. to "MYSQL").

After changing which history channels are active, you must **restart mlogger and mhttpd**.

Using the ODBC history

See Keys_in_the_ODB_/Logger/History_subtree#ODBC_subtree.

Using the SQLITE history

No special configuration required. Create a directory for storage of sqlite history files, save it's path in ODB "/Logger/History/SQLITE/Sqlite dir". In the history data directory you should see files with name "mh_xxx.sqlite3" corresponding to your equipment and variable names.

Using the MYSQL history

After creating a MySQL database (and accounts with read/write and read-only permissions), create files that tell Midas how to connect to the database (see Keys_in_the_ODB_/Logger/History_subtree#MYSQL_subtree for syntax). Specify the name of these files in "/Logger/History/MYSQL/MYSQL Writer" and "/Logger/History/MYSQL/MYSQL Reader".