MidasWiki - User contributions [en]

Quickstart Linux

2026-09-29T16:17:26Z

Dfujimoto: /* ROODY Package Installation */

{{Pagelinks}}
== Introduction ==
This quickstart shows you how to install the MIDAS packages on a linux box and create a '''MIDAS experiment'''.
The linux box will be called the MIDAS '''''experiment host'''''.

==Installation==
Log on to the ''[[#Introduction|Experiment Host]]'' and decide on a name for the MIDAS experiment (the ''experiment name'').
In these instructions, the MIDAS packages and experiment will be installed under the username "johnfoo".
The ODB and other shared memory buffers for the MIDAS experiment will reside on the ''experiment host'' (localhost).
MIDAS and the other required packages will be installed under directory {{Filepath|path=$HOME/packages}}. The MIDAS ''experiment directory'' will be {{Filepath|path=$HOME/online}} and the MIDAS ''experiment name'' will be "e777". The name of the ''experiment host'' will be "mhostpc".

;NOTE
: The user should substitute directory paths for the MIDAS packages and experiment directory as required, and names for ''experiment host'', username, ''experiment name'' appropriate to his/her own setup.

== Experimental setup ==
These instructions describe setting up an experiment that will run ONLY on the MIDAS ''experiment host'' computer (i.e. '''localhost only'''). In this case, the MIDAS ''experiment host'' has access to any hardware required, and all frontends, logger, analyzer etc. run on the one computer.

Modifications to run an experiment with additional frontend(s) running '''remotely''' (i.e. on remote cpu(s)) will also be described.
Follow the instructions for setting up a localhost-only experiment until indicated.

In both cases, the main MIDAS applications (e.g. [[mlogger]], [[mhttpd]]) run on the MIDAS ''experiment host''.

==Environment Variables==
The following Environment variables should to be added to the {{File|name=.cshrc}} or {{File|name=.bashrc}} file (depending on your shell) in the {{Filepath|path=$HOME}} directory on the ''experiment host'', so that they will be defined at login. '''Substitute appropriate values for your own setup'''.

{| class="wikitable"
|-
! csh !! bash !! comment
|-
| setenv MIDASSYS $HOME/packages/midas || export MIDASSYS=$HOME/packages/midas ||Base directory of the MIDAS package
|-
| setenv MIDAS_EXPTAB $HOME/online/exptab || export MIDAS_EXPTAB=$HOME/online/exptab || MIDAS experiment table
|-
| setenv MIDAS_EXPT_NAME e777 || export MIDAS_EXPT_NAME=e777 || MIDAS experiment name
|-
| setenv PATH .:$MIDASSYS/bin:$PATH || export PATH=$PATH:$MIDASSYS/bin || path
|}

Logout and login again, or source {{File|name=.cshrc}} (source {{File|name=.bashrc}}) for the changes to take effect.
;NOTE
: See [[MIDAS environment variables]] for a list of environment variables used by MIDAS.

== ROOT Package Installation ==
MIDAS has optional integration with [https://root.cern.ch ROOT]. ROOT is not required for acquiring data or storing it in midas files, but is used by some of the analysis tools (including the example analyzer that will be built later in this Quickstart). It is also possible to write midas data directly to ROOT files if desired.

If you require ROOT integration, follow the installation instructions on the [https://root.cern.ch ROOT website] and set the ROOTSYS environment variable.

If you do not require ROOT integration, leave the ROOTSYS variable undefined.

== Adding SSL support ==

We recommend running mhttpd (the MIDAS web server) behind an industry-grade web proxy like apache httpd or nginx. httpd/nginx will then handle the SSL security. It is however also possible to build mhttpd with built-in SSL support. See [[Mhttpd|the mhttpd documentation]] for more details about web security.

If you wish to enable SSL support in mhttpd itself:


[mhostpc] cd $MIDASSYS
[mhostpc] make mbedtls


== MIDAS Package Installation ==
The MIDAS package will be installed on the ''experiment host'' in the directory given by MIDASSYS - see [[#Environment Variables]].

Install the MIDAS and other package(s) from the MIDAS git repository in an xterm as follows:

[mhostpc] mkdir $HOME/packages
[mhostpc] cd $HOME/packages
[mhostpc] git clone https://gitlab.triumf.ca/midas/midas.git --recurse-submodules


If you already cloned MIDAS and forgot the "--recurse-submodules", you can do a "git submodule update --init --recursive" later. If you want to pull MIDAS later to update to the current version, you need either "git pull --recurse-submodules". To tell git to always recurse, you can set a global flag via "git config submodule.recurse true".

The MIDAS Makefile will auto-detect whether MSCB, ROOT, MYSQL, SQLITE, ODBC etc. are available (or have been installed). If you need these features, make sure they are available before building the MIDAS package.

[mhostpc] cd $MIDASSYS
[mhostpc] mkdir build
[mhostpc] cd build
[mhostpc] cmake .. ### configure build process (see note below about cmake/cmake3)
[mhostpc] make install ### build and install midas to bin/ and lib/ directories
[mhostpc] cd ..
[mhostpc] ls -l bin/odbedit ### check that odbedit has been created (do not run it yet)


MIDAS requires cmake version 3.10 or later. On some systems, you have to call "cmake3 ..", on others just "cmake ..". Some systems might require you to install cmake first (see the [https://cmake.org/download/ cmake website; installation is very simple).

In case of problems see
* [[Installation#Installing MIDAS|gitlab is down]] or
* [[Installation/Compilation problems#SSL certificate errors|SSL certificate errors]].

==ROOTANA Package Installation==

rootana is an optional framework for analyzing midas data in realtime and displaying plots (either by a graphical interface or via webpages). Most of the features are now available in manalyzer, which is included in the base midas install.

If you require the legacy ROOTANA package, you can install it with:

[mhostpc] cd $HOME/packages
[mhostpc] git clone https://gitlab.triumf.ca/midas/rootana.git
[mhostpc] cd rootana
[mhostpc] make


add ROOTANASYS=$HOME/packages/rootana to the login file.

Install the JSROOT package:
<pre>
cd $HOME/packages
git clone https://github.com/linev/jsroot.git
</pre>

add JSROOTSYS=$HOME/packages/jsroot to the login file.

==ROODY Package Installation==

ROODY is a legacy package for displaying live ROOT histograms. It is generally not used by new experiments.

If ROODY is needed,

[mhostpc] cd $HOME/packages
[mhostpc] git clone https://gitlab.triumf.ca/midas/roody.git
[mhostpc] cd roody
[mhostpc] make


== Create the Experiment file exptab ==
In this example, the experimental directory is {{Filepath|path=$HOME/online}}. Create this directory on the ''experiment host'' (localhost):

[mhostpc] mkdir $HOME/online
[mhostpc] cd $HOME/online
Then create the [[exptab]] file {{Filepath|path=$HOME/online/exptab}} containing the three parameters:
# the experiment name
# location of MIDAS shared memory buffers
# username
as follows (using parameters for your own experiment):

[mhostpc] cat > exptab
e777 /home/johnfoo/online johnfoo

The path of the {{File|name=exptab}} file is given by MIDAS_EXPTAB - see [[#Environment Variables]].

== Create shared memory files ==
At this point you should be able to run [[odbedit]] on the ''experiment host'' (localhost). The first time {{Utility|name=odbedit}} is run, it will create the required {{File|name=.*.SHM}} files in the MIDAS experiment directory {{Filepath|path=$HOME/online}}. These are the saved files for the MIDAS shared memory.

The default directory {{Filepath|path=$HOME/online}} will contain the MIDAS messages file ({{File|name=midas.log}}) and
any data files you may create. If you want an alternative location for the data files see [[#Customizing your experiment]].

Run {{Utility|name=odbedit}}, and type {{Odbedit cmd|cmd=ls}} to list the default directories.
[mhostpc] odbedit
[local:e777:S]/>ls
Experiment
System
Programs
Logger
Runinfo
Alarms

== Run the MIDAS Web Server ==

=== Using apache ===

Recommended for production systems with maximum security is to run mhttpd behind an industry-standard SSL/https proxy with password protection. We document instructions for apache here, but you could nginx instead.

To do this using apache httpd on CentOS-7, do this:
* configure apache httpd per instructions for [https://daq00.triumf.ca/DaqWiki/index.php/Ubuntu#Install_apache_httpd_proxy_for_midas_and_elog Ubuntu] or [https://daq00.triumf.ca/DaqWiki/index.php/SLinstall#Configure_HTTPS_server_.28CentOS7.29 CentOS 7]
* run mhttpd
* use web browser to connect to https://mhostpc/
* enter the user name and password configured in apache httpd
* the midas status page should open

=== Standalone (less secure) ===

For simplified installation, one can use standalone mhttpd with built-in https and password protection. This configuration
is only as secure as the underlying mongoose and OpenSSL web server and encryption packages. Use the odbedit tool to edit settings in the [[Webserver_ODB_tree|Webserver ODB tree]]:

* Set "Enable https port" to "y"
* Set "https port passwords" to "y"

Many current operating systems implement a built-in firewall which will block outside access to mhttpd. See instructions for [https://daq00.triumf.ca/DaqWiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29 CentOS 7].

Start [[mhttpd]] on the ''experiment host'' (localhost) like this:
[mhostpc] mhttpd

You will get the following messages:

[mhttpd,ERROR] [mhttpd.cxx:13880:mongoose,ERROR] mongoose web server cannot find password file "/home/johnfoo/online/htpasswd.txt"
[mhttpd,ERROR] [mhttpd.cxx:13881:mongoose,ERROR] please create password file: touch /home/johnfoo/online/htpasswd.txt
[mhttpd,ERROR] [mhttpd.cxx:13818:mongoose,ERROR] mongoose web server password file "/home/johnfoo/online/htpasswd.txt" has no passwords for realm "exptname"
[mhttpd,ERROR] [mhttpd.cxx:13819:mongoose,ERROR] please add passwords by running: htdigest /home/johnfoo/online/htpasswd.txt exptname midas
mongoose_init: Error: Cannot initialize authorization object!
Error: Could not start the mongoose web server, see messages and midas.log, bye!


Create the password file by following the instructions printed by mhttpd. The http digest domain name is the experiment name, suggested default user name is "midas". You will be asked to type in a password. (htdigest is part of the httpd-tools package)


[mhostpc] htdigest -c /home/johnfoo/online/htpasswd.txt exptname midas
Adding password for midas in realm exptname.
New password:
Re-type new password:

It is a good idea to set the password file {{Filepath|path=htpasswd.txt}} readable and writable by owner only.

Run {{Utility|name=mhttpd}} again.

If you get error messages like <code>https port "[::]:8443" requested, but mhttpd compiled without MG_ENABLE_SSL</code> then you need to re-compile MIDAS after [[Mhttpd#Build_mhttpd_with_SSL_support|adding SSL support]].

Otherwise, you will get the following messages:

HTTP Digest authentication with realm "exptname" and password file "/home/johnfoo/online/htpasswd.txt"
Hostlist off, connections from anywhere will be accepted
Listening on "http://localhost:8080", passwords OFF, hostlist OFF
Listening on "http://[::1]:8080", passwords OFF, hostlist OFF
[mhttpd,ERROR] [mhttpd.cxx:15739:mongoose_listen,ERROR] cannot find SSL certificate file "/home/johnfoo/online/ssl_cert.pem"
[mhttpd,ERROR] [mhttpd.cxx:15740:mongoose_listen,ERROR] please create SSL certificate file using openssl: cd $MIDASSYS; openssl req -new -nodes -newkey rsa:2048 -sha256 -out ssl_cert.csr -keyout ssl_cert.key -subj "/C=/ST=/L=/O=midas/OU=mhttpd/CN=localhost"; openssl x509 -req -days 365 -sha256 -in ssl_cert.csr -signkey ssl_cert.key -out ssl_cert.pem; cat ssl_cert.key >> ssl_cert.pem
[mhttpd,ERROR] [mhttpd.cxx:15741:mongoose_listen,ERROR] or using certbot (recommened): setup certbot per Let's Encrypt instructions, certificates are typically saved in /etc/letsencrypt/live/$HOSTNAME/, copy fullchain.pem and privkey.pem to $MIDASSYS; cd $MIDASSYS; cat fullchain.pem privkey.pem > ssl_cert.pem
[mhttpd,ERROR] [mhttpd.cxx:15918:mongoose_init,ERROR] Failed to listen on a TCP port enabled in ODB /WebServer
Error: Could not start the mongoose web server, see messages and midas.log, bye!


Create a self-signed certificate suitable for initial testing by executing the commands printed by mhttpd.

For production use, you should create a properly signed certificate, see [[Mhttpd#Create an SSL certificate|create your own SSL certificate]] or you should run mhttpd behind an SSL/https proxy (apache httpd).

Now restart {{Utility|name=mhttpd}}
[mhostpc] mhttpd
HTTP Digest authentication with realm "test_pol" and password file "/home/johnfoo/online/htpasswd.txt"
Hostlist off, connections from anywhere will be accepted
Listening on "http://localhost:8080", passwords OFF, hostlist OFF
Listening on "http://[::1]:8080", passwords OFF, hostlist OFF
Mongoose web server will use https certificate file "./ssl_cert.pem"
Listening on "https://[::]:8443", passwords enabled, hostlist OFF
Mongoose web server is using 1 threads
Mongoose web server is using 2 threads
Mongoose web server is using 3 threads
Mongoose web server is using 4 threads

Now point a web browser running on the same host computer (localhost) to https://localhost:8443.
If the web browser is running on a different computer, go to URL of the form

https://mhostpc.triumf.ca:8443 (substitute your host machine name and domain for "mhostpc.triumf.ca")

If you are using the default SSL certificate you will probably get a message: "This Connection is Untrusted". Click "I understand the risks" and add an exception. This is because the test certificate is self-signed. Then confirm an exception.

You should then see an authentication box asking you for the user name and password. The user name is "midas". Enter the password you just created. The Midas [[Status Page]] should appear with multiple buttons for run control as well as equipment listing (no equipments will be listed as yet) and application listings. Please refer to [[mhttpd]] (the MIDAS Web-based Run Control utility) for further information. You can start and stop runs from the main status page, and use the [[ODB Page]] to access the database (ODB).

; Note
: Default ports of 8080 and 8443 are used by [[mhttpd]]. To use some other ports, change the relevant values in the [[Webserver_ODB_tree|Webserver ODB tree]].

== Run the MIDAS logger ==
Start the MIDAS logger [[mlogger]] on the ''experiment host'' (localhost) :
[mhostpc] mlogger
This should start without error. It is usually run as a daemon, however in this case it is run in a terminal to check for errors. Starting the midas logger [[mlogger]] will automatically create more keys in the {{Odbpath|path=/Logger}} ODB tree.

From the MIDAS status page, go to the "Programs" page, then the "Logger" page and set ODB variables:
* Required -> set to "y"
* Start command -> set to "mlogger -D"
* Alarm Class -> set to "Alarm"

To test this, go to the "Programs" page, press "stop logger", then "start logger".

Start a run, wait 10 seconds, stop the run.

From the MIDAS "Status" page, go to the "logging channels -> channel #0" and set ODB variables:
* Output -> "FILE"

Start a run, go to the "Status" page, observe that the output file is written with lz4 compression and has the "mid.lz4" extension.

Stop the run.

== Run test frontend ==

Run <code>fetest</code>

On the MIDAS "Status" page there will be 5 new equipment entries:

{| class="wikitable"
|+ Equipment created by the fetest program
|-
! Equipment name !! Data it sends !! How often data is sent
|-
| test_random || Random data banks || 1Hz
|-
| test_slow || Example slow control data for testing the midas history plots || 1Hz, even when a run isn't in progress
|-
| test_rare || Data that look like a sine wave || 0.1Hz, even when a run isn't in progress
|-
| test_bulk || Large events || 1Hz
|-
| test_rpc || None. It shows an example of a "remote procedure call" (RPC) handler. Javascript/C++/Python code can connect to the program, and it will respond with the current time (in this toy example) || never
|}

Note the event counters for test_slow and test_rare are already incrementing.

Start a run.

Observe that the event counters for test_bulk and test_random are also incrementing now, and that the output data file size is growing.

Click on the "Event dump" page and then the "Run" button. You'll see the raw data that is being written to disk. (The data from most of the equipment is double-precision floating point numbers; the data from the "bulk" equipment is just a bunch of zeros).

Return the status page and stop the run.

== Create a history plot ==

Go to the MIDAS "History" page.

Say "new", panel name "test". In the history editor select event "test_slow", and click the "save" button (you can ignore the rest of the options for now). If the "test_slow" event isn't listed, make sure that the mlogger program is running, as described above.

History plot "test" should open with a sine-wave plot. Data on the plot should update as long as "fetest" is running (there is no need to start a run).

== Clients run on Localhost only ==
;NOTE:
* If creating a MIDAS experiment with a '''REMOTE''' frontend, '''continue by following the instructions''' [[#Running with one or more REMOTE frontends]].
* If all clients are running on the Experiment Host (i.e. '''localhost'''), continue with the following instructions:

=== Frontend and Analyzer (localhost) ===

THIS SECTION IS VERY OBSOLETE

There are several examples of [[Frontend user code]] ({{File|name=frontend.c}}) in the MIDAS package available under {{Filepath|path=$MIDASSYS/examples/}}. Choose a suitable example from this directory that you can later modify for your own particular setup (e.g. for a slow control, choose the {{Filepath|path=../slowcont/}} example).

A '''frontend''' is a program that usually reads data from the hardware and sends it to a buffer to be logged and analyzed. You can find documentation about the frontend structure under [[Frontend Application]], [[Frontend Operation]] and [[Frontend user code]].

The example chosen here is from {{Filepath|path=$MIDASSYS/examples/experiment/}}. It does '''not need any hardware''' and produces simulated events.

On the MIDAS ''experiment host'' (localhost), copy the example to the experiment directory and build it.

[mhostpc] cd $HOME/online
[mhostpc] cp $MIDASSYS/examples/experiment/* .
[mhostpc] make
The analyzer will only build if ROOT has previously been installed.
At this point the frontend and the analyzer should be ready if no errors were generated during the build. Try the frontend and analyzer by starting them in xterms.

[mhostpc] frontend
[mhostpc] analyzer

=== Equipment ODB tree ===
Starting the frontend will automatically create the {{Odbpath|path=/Equipment}} ODB tree and one or more equipments. In this case, the equipments "trigger" and "scaler" have been defined by code in {{File|name=frontend.c}} and created the first time frontend runs.

[mhostpc] odbedit
[local:e777:S]/>ls /equipment
Trigger
Scaler 

View the MIDAS main status web page (by pointing your browser to https://localhost:8443/ or appropriate port if not using the default).
Observe on the [[Status Page]] that the two Equipments (Trigger, Scaler) have appeared in the Equipment section (Figure 1) and are coloured green.
 
'''Click thumbnail to enlarge'''
[[File:quickstart_status_page.png|thumb|left|Figure 1: web server status page showing example frontend equipments]]
<div style="clear: both"></div> 

Rather than using odbedit, you can view the contents of these equipments by clicking on the ODB button, then on {{Odbpath|path=/Equipment}}, then {{Odbpath|path=Trigger}} or {{Odbpath|path=Scaler}}. See [[ODB Page]] for more information.

=== Start a run ===
Start a run by pressing the Start button on the [[Status Page]].
The Equipment display on the Status Page will now show some event statistics (Figure 2)
'''Click thumbnail to enlarge'''
[[File:sample_frontend_status.png|thumb|left|Figure 2: Equipments shown on Status Page showing event statistics]]
<div style="clear: both"></div> 

The frontend will show an event display that will update when a run is started, e.g.

Sample Frontend connected to <local>. Press "!" to exit 18:48:2546:08
==========================================================================
Run status: Running Run number 2 |/ /
===========================================================================
Equipment Status Events Events/sec Rate[B/s] ODB->FE FE->OD
---------------------------------------------------------------------------
Trigger OK 13948 99.0 5413.0 0 140
Scaler OK 15 0.3 15.4 0 15

The [[Status Page]] will also show events being generated (click the refresh button if necessary).

=== Event Dump ===
While a run is in progress, the midas application [[mdump]] will provide you an event dump of the collected data from the running frontend, e.g.

mhostpc> mdump -l 50
blank
- MIDAS revision: Mon Nov 2 11:50:51 2015 -0800 - 3b66779 -- Enter <!> to Exit ------- Midas Dump ---
------------------------ Event# 1 ------------------------
Evid:0001- Mask:0000- Serial:0- Time:0x567a0c5d- Dsize:40/0x28
#banks:2 - Bank list:-ADC0TDC0-
blank
Bank:ADC0 Length: 8(I*1)/2(I*4)/4(Type) Type:Unsigned Integer*2
1-> 0x0167 0x03c6 0x0069 0x0073
blank
Bank:TDC0 Length: 8(I*1)/2(I*4)/4(Type) Type:Unsigned Integer*2
1-> 0x0051 0x00ff 0x004a 0x00ec

For further data processing/analysis, either the midas analyzer or the rootana can used for data display as well.

=== Create a startup script (localhost) ===
It is useful to create a script that will automatically start all the required clients for the experiment. This is called a startup script. Here, the startup script will be {{File|name=start_daq.sh}} (any name can be used). Create {{Filepath|path=$HOME/online/bin/start_daq.sh}} on the ''experiment host'' as follows (supply the [[mhttpd]] ports if default is not used):

#!/bin/sh
# start_daq.sh
cd $HOME/online
odbedit -c clean
# start mhttpd on default port. (Mongoose https version)
mhttpd -D # you can optionally restrict access to localhost and other specified hosts - see [[mhttpd]]
xterm -e ./frontend &
xterm -e ./analyzer &
mlogger -D
#end file

Before running this script, you will need to shutdown any running clients:
mhostpc> odbedit -c "sh all"
or you can use the {{Utility|name=mhttpd}} [[Programs Page]] to shut them down.

The script will start them as daemons.
By running the startup script start_daq.sh, several MIDAS applications will be started in sequence:
# Cleanup previous MIDAS application (if any).
# Start the MIDAS web server [mhttpd]
# Start the frontend application in its own xterm (for debugging purpose).
# Start the analyzer application in its own xterm (for debugging purpose).
# Start the MIDAS Data logger [mlogger]
[mhostpc] sh ./start_daq

You may wish to modify the script to restart missing clients only.

Next [[#Customizing your experiment|customize your experiment]].

== Running with one or more REMOTE frontends ==
These instructions assume you have already followed the instructions to setup an experiment running on the MIDAS experiment host (localhost), i.e. you have setup the MIDAS Environment variables, downloaded MIDAS packages, setup the [[mhttpd]] password and have {{Utility|name=mhttpd}} running successfully.

In the case of '''remote''' frontend(s), the remote cpu(s) typically have access to some or all of the hardware. They might be VMIC cpus running in VME crates for example, connected to the MIDAS Host via Ethernet. They read data from the hardware and send it back to the MIDAS ''experiment host'' to be logged and analyzed.

=== Experimental setup (Remote) ===
These instructions assume that the MIDAS ''experiment host'' computer is 64-bit (if not see [[#Build 32-bit MIDAS libraries|note below]]), and there is one ''remote host'', a 32-bit machine named "rlxhost".

The ''remote host'' mounts the {{Filepath|path=/home}} disk of the ''experiment host'', so that it '''has access to the MIDAS packages''', the {{Filepath|path=$HOME/online}} directory and shares the .cshrc (or .bashrc) script. It is convenient to set up ssh keys (use ssh-keygen) so that the ''remote host'' can be accessed without supplying the password from the ''experiment host''.

If your setup is different, you will have to make changes as appropriate.

=== Build 32-bit MIDAS libraries ===
NOTE

If the MIDAS ''experiment host'' computer is 32-bit (i.e. the same as the ''remote host''), which is a simpler setup, you will have built the 32-bit MIDAS libraries already, and can skip the rest of this section. Note however, that in the .cshrc file in the [[#Modify .cshrc (.bashrc) for Remote host|following section]] you will not need the lines starting "# select 64-bit or 32-bit MIDAS and ROOT". The PATH should contain $MIDASSYS/bin on both experiment host and remote host. This will point to the 32-bit MIDAS libraries.
------------------------------

 
Assuming the MIDAS host computer is 64-bit, you will have already built the 64-bit MIDAS libraries.
In this case, if the remote frontend is 32-bit, you will need to build the 32-bit MIDAS libraries on the 64-bit machine, i.e.

[mhostpc] cd /home/packages/midas
[mhostpc] make linux32 ### build the 32-bit MIDAS libraries on 64-bit machine
[mhostpc] ls -l linux-m32/bin/odbedit ### check that the 32-bit odbedit has been created 
Do not run {{Utility|name=odbedit}} yet on the ''remote host''!

If the "make linux32" command fails, you may need to install 32bit support for your compiler (e.g. "apt install g++-multilib").

=== Modify .cshrc (.bashrc) for Remote host ===
Add the following to the {{File|name=.cshrc}} (modified appropriately if using {{File|name=.bashrc}}). When running with a REMOTE Host, the application [[mserver]] will be started (using a particular port) on the MIDAS ''experiment host'' machine only. Note that the environment variable MIDAS_SERVER_HOST is defined on the remote machine(s) but NOT on the ''experiment host''.

If ROOT has been installed properly and the script <ROOT installation directory>/bin/thisroot.sh is called from .cshrc or .bashrc, the ROOTSYS environment variable should be defined and the ROOT bin directory should be in the PATH environment variable. Make sure that this is the case.


# setenv MIDAS_HOST mhostpc # substitute your experiment host name
# setenv REMOTE_HOST rlxhost # substitute your remote host name
# setup the MIDAS mserver host
#
switch (`hostname`)
case $MIDAS_HOST*:
unsetenv MIDAS_SERVER_HOST ## MIDAS_SERVER_HOST not defined
breaksw
default:
setenv MIDAS_SERVER_HOST $MIDAS_HOST.triumf.ca:1175 ## for remote host, define MIDAS_SERVER_HOST, use your domain here
## as hostname + domain name + default mserver port
## port must match that of [[mserver]] used in start_daq.sh
endsw
#
# select 64-bit or 32-bit MIDAS
#
switch (`uname -i`)
case i386:
setenv PATH .:$MIDASSYS/linux-m32/bin:$PATH
breaksw
default:
setenv PATH .:$MIDASSYS/linux/bin:$PATH
endsw
#
setenv PATH .:$HOME/online/bin:$HOME/packages/roody/bin:$PATH
#

After executing {{File|name=.cshrc}} (or {{File|name=.bashrc}}) or logging out, on the MIDAS host (mhostpc) make sure environment variable MIDAS_SERVER_HOST is NOT defined
[mhostpc] echo $MIDAS_SERVER_HOST
[mhostpc] MIDAS_SERVER_HOST: Undefined variable.

and on the '''''remote host''''', make sure MIDAS_SERVER_HOST IS defined:
[rlxhost] echo $MIDAS_SERVER_HOST
[rlxhost] MIDAS_SERVER_HOST

Also on the ''remote host'', make sure that the correct (32-bit) {{Utility|name=odbedit}} will be used (do not run it yet!)
[rlxhost] which odbedit
/home/johnfoo/packages/midas/linux-m32/bin/odbedit

=== Grant Remote host access permission ===
Give permission for the remote host(s) to access the experiment by following the instructions to [[Security#MIDAS programs on remote machines|allow MIDAS programs on remote machines]]. If the MIDAS web server [[mhttpd]] is running, use [[ODB Page]] to edit the ODB keys listed, otherwise use [[odbedit]] on the ''experiment host'' machine.

=== Start mserver on Experiment Host ===
The application [[mserver]] is not required when running an experiment on the ''experiment host'' (localhost) only, but it '''IS required''' for access from a '''REMOTE''' frontend.
Start [[mserver]] on MIDAS host (default port 1175) or use the "-p" option for a different port.
The port must match the port defined for environment variable MIDAS_SERVER_HOST in {{File|name=.cshrc}} (or {{File|name=.bashrc}}) [[#Modify .cshrc (.bashrc) for Remote host|above]].
[mhostpc] mserver -D

The environment variables on the REMOTE host should be:

[rlxhost] printenv | grep MIDAS
MIDASSYS=/home/e777/packages/midas
MIDAS_EXPT_NAME=e777
MIDAS_SERVER_HOST=mhostpc.triumf.ca:1175
(a different port may be used - see above)

=== Check access to ODB from Remote host ===
Run {{Utility|name=odbedit}} on the remote host to check access to ODB.
[rlxhost] odbedit
[mhostpc.triumf.ca:1175:e777:Stopped]/>ls 
If it does not work correctly, check that
* MIDAS_SERVER_HOST is defined on REMOTE host with same port as [[mserver]]
* mserver is running on MIDAS host with the correct port
* MIDAS_SERVER_HOST is NOT defined on MIDAS host
* remote access permission to REMOTE host has been granted as described above.

=== Build the analyzer on Experiment Host ===
The analyzer will be built and run on the MIDAS ''experiment host'' machine.
Select an example from $MIDASSYS/examples/ (see [[#Frontend and Analyzer (localhost)]]) and copy it to $HOME/online on the MIDAS host machine. Since we are assuming a 64-bit MIDAS host and a 32-bit remote host, edit the example Makefile so only the analyzer will be built.

=== Build the frontend (32bit) for Remote host ===
The frontend runs on a 32-bit remote host in this example. It therefore must either
* be built 32-bit on the 64-bit MIDAS host with the "-m32" flag
* or built on the REMOTE host
and in either case the [[#Build 32-bit MIDAS libraries|32-bit MIDAS libraries]] used for linking.

To avoid confusion, create a different subdirectory for the frontend, e.g.
[rlxhost] mkdir $HOME/online/fe_32
Copy the Makefile and frontend code from the same example to this directory.
Modify the Makefile as needed, and only build the frontend.

Once built, start the frontend in an xterm on the REMOTE host.
[rlxhost] $HOME/online/fe_32/frontend
Check that it connects successfully to the experiment and the Equipments appear on the [[Status Page]] as described under [[#Frontend and Analyzer (localhost)]]. The Equipment Status (Figure 2) should show that the Equipments are now running on the remote cpu, i.e. "SampleFrontend@rlxhost.triumf.ca" rather than "Sample Frontend@localhost".

=== Run the Analyzer on Experiment host ===
Start the analyzer in an xterm on the MIDAS ''experiment host''
[johnfoo@mhostpc e777] $HOME/online/analyzer

Continue by starting a run and dumping the data as described under [[#Frontend and Analyzer (localhost)]].

=== Create experiment startup script ===
It is useful to create a script that will automatically start all the required clients for the experiment. This is called a startup script (any name can be used). Create {{Filepath|path=$HOME/online/bin/start_daq.sh}} on the ''experiment host'' as follows, (supply the [[mhttpd]] ports if default is not used) :

#!/bin/sh
# start_daq.sh
cd $HOME/online
#
switch (`hostname`)
case $MIDAS_HOST*: ## MIDAS_HOST is assigned in .cshrc (.bashrc)
echo "Good, we are on $MIDAS_HOST"
breaksw
case $REMOTE_HOST*: ## REMOTE_HOST is assigned in .cshrc (.bashrc)
echo "startup script should be executed on $MIDAS_HOST"
endsw
#
odbedit -c clean
# start mhttpd on default port (Mongoose https version)
mhttpd -D # you can optionally restrict access to localhost and/or other specified hosts - see [[mhttpd]]
# start mserver on default port (use argument -p to use a different port)
mserver -D # start mserver on default port (1175) or a different port using -p option - see [[mserver]]
# the port must match that of MIDAS_SERVER_HOST in .cshrc (.bashrc) - see above
# access must be specifically allowed - see [[#Grant Remote host access permission||above]]
xterm -e ./analyzer &
## start frontend on remote host
#ssh $REMOTE_HOST $HOME/online/bin/start_frontend -O >& $HOME/online/bin/start_frontend.log &
mlogger -D
#end file

Note that command to start the frontend on the remote host is commented out. It can be uncommented once the start_frontend script is written and tested.

Shut down any clients that are running already by running odbedit on the MIDAS ''experiment host'' and execute the new startup script {{File|name=start_daq.sh}} (make it executable if desired)

[mhostpc] odbedit
[local:e777:S]/>sh all
[mhostpc] ./start_daq.sh 
and check that {{Utility|name=odbedit}} still works on the ''REMOTE host''

[rlxhost] odbedit
[mhostpc.triumf.ca:1175:e777:Stopped]/>ls 

You may wish to modify the script to restart missing clients only.

=== Create script to start frontend ===
The startup file {{File|name=start_daq.sh}} invoked a script {{File|name=start_frontend}} to start the remote frontend. This line was commented out (see [[#Create experiment startup script|above]]). Once the frontend is working, create a script to start the frontend (e.g. file {{Filepath|path=$HOME/online/bin/start_frontend}}).

#!/bin/tcsh
# Script to start frontend running on $REMOTE_HOST
#
# This script runs on the REMOTE HOST
#
echo "starting frontend for experiment $MIDAS_EXPT_NAME "
xterm -geometry 150x50+800+0 -fg white -bg blue -title "Frontend" -e "$HOME/online/fe_32/frontend" &
echo "Frontend task has been started"
exit

Check that this script works by running it on the REMOTE host. If successful,
run the commented command in an xterm on the ''experiment host''
[mhostpc] ssh $REMOTE_HOST $HOME/online/bin/start_frontend -O >& $HOME/online/bin/start_frontend.log & 
This command relies on password-less access set up with SSH keys (see [[#Experimental setup (Remote)]]).
If successful, uncomment line in {{File|name=start_daq.sh}}, and run the script to check that it can start the remote client(s) from the MIDAS ''experiment host'' (first shutting down all running clients) :

[mhostpc] odbedit -c "sh all"
[mhostpc] ./start_daq.sh 

Next [[#Customizing your experiment|customize your experiment]].

== Customizing your experiment ==

Here are some of the more common options provided for users to customize their experiments according to their own preferences:

; Write frontend(s) to read out and control your hardware
: Modify the example frontend and Makefile (from $MIDASSYS/examples/) you have already copied and built so it accesses your hardware.

: The example Frontend.c is an example of [[Frontend user code]]. You can find documentation about the frontend structure under [[Frontend Application]], [[Frontend Operation]] and [[Frontend user code]]. You may need to set up some [[Event Notification (Hot-Link)|Hot-Links]] in the frontend. The MIDAS package includes device drivers for some of the more common hardware. See [[MIDAS Driver Library]] for more information.

; Modify analyzer(s) for your data
: Modify the example analyzer code as necessary to analyze the data from your frontends.

; Slow Controls
: Control your slow-controls equipment with slow-control frontend(s) optionally using the [[#MIDAS Package Installation|MSCB]] system.

; Data logger
: Set up the data logger [[mlogger]] to log data to a storage device. Many options are available - see [[Data Logger]], [[mlogger]] and [[/Logger ODB tree]] for instructions.

; Allow programs to be restarted from the [[Programs Page]]
: See mhttpd [[Programs Page]] and [[/Programs ODB tree]]

; Write scripts which run at start and end of run
: See [[/Programs ODB tree#Execute on start run]] and [[/Programs ODB tree#Execute on stop run]]

; Create a shutdown script
: Create a script similar to the startup script to shutdown the experiment

; Lazy logger
: Set up the [[lazylogger]] for archiving data files for storage.

; Create edit-on-start parameters
: See [[Edit-on-start Parameters]]

; Set up the alarm system
: See [[Alarm System]] and [[Alarms Page]]

; Set up history system
: See [[History System]] and [[History Page]]

; Set up the Electronic logbook (Elog)
: See [[Elog]] for details

; Set up the Sequencer
: The [[Sequencer Page#Sequencer]] can be used to run a sequence of runs, changing parameters as needed.

; Write a webserver Custom page
: To better control and monitor your experiment, you may want to write a [[Custom Page]].

== Start MIDAS on boot ==

To start MIDAS mhttpd, mlogger and elogd at boot time, add following cron entries:
<pre>
[iris@iris00 ~]$ crontab -l
@reboot /home/iris/packages/midas/bin/mhttpd -D
@reboot /home/iris/packages/midas/bin/mlogger -D
@reboot /home/iris/packages/elog/elogd -c /home/iris/packages/elog/elogd.cfgiris -D
[iris@iris00 ~]$
</pre>

[[Category:Installation]]

Quickstart Linux

2026-09-29T16:17:02Z

Dfujimoto: /* ROOTANA Package Installation */

{{Pagelinks}}
== Introduction ==
This quickstart shows you how to install the MIDAS packages on a linux box and create a '''MIDAS experiment'''.
The linux box will be called the MIDAS '''''experiment host'''''.

==Installation==
Log on to the ''[[#Introduction|Experiment Host]]'' and decide on a name for the MIDAS experiment (the ''experiment name'').
In these instructions, the MIDAS packages and experiment will be installed under the username "johnfoo".
The ODB and other shared memory buffers for the MIDAS experiment will reside on the ''experiment host'' (localhost).
MIDAS and the other required packages will be installed under directory {{Filepath|path=$HOME/packages}}. The MIDAS ''experiment directory'' will be {{Filepath|path=$HOME/online}} and the MIDAS ''experiment name'' will be "e777". The name of the ''experiment host'' will be "mhostpc".

;NOTE
: The user should substitute directory paths for the MIDAS packages and experiment directory as required, and names for ''experiment host'', username, ''experiment name'' appropriate to his/her own setup.

== Experimental setup ==
These instructions describe setting up an experiment that will run ONLY on the MIDAS ''experiment host'' computer (i.e. '''localhost only'''). In this case, the MIDAS ''experiment host'' has access to any hardware required, and all frontends, logger, analyzer etc. run on the one computer.

Modifications to run an experiment with additional frontend(s) running '''remotely''' (i.e. on remote cpu(s)) will also be described.
Follow the instructions for setting up a localhost-only experiment until indicated.

In both cases, the main MIDAS applications (e.g. [[mlogger]], [[mhttpd]]) run on the MIDAS ''experiment host''.

==Environment Variables==
The following Environment variables should to be added to the {{File|name=.cshrc}} or {{File|name=.bashrc}} file (depending on your shell) in the {{Filepath|path=$HOME}} directory on the ''experiment host'', so that they will be defined at login. '''Substitute appropriate values for your own setup'''.

{| class="wikitable"
|-
! csh !! bash !! comment
|-
| setenv MIDASSYS $HOME/packages/midas || export MIDASSYS=$HOME/packages/midas ||Base directory of the MIDAS package
|-
| setenv MIDAS_EXPTAB $HOME/online/exptab || export MIDAS_EXPTAB=$HOME/online/exptab || MIDAS experiment table
|-
| setenv MIDAS_EXPT_NAME e777 || export MIDAS_EXPT_NAME=e777 || MIDAS experiment name
|-
| setenv PATH .:$MIDASSYS/bin:$PATH || export PATH=$PATH:$MIDASSYS/bin || path
|}

Logout and login again, or source {{File|name=.cshrc}} (source {{File|name=.bashrc}}) for the changes to take effect.
;NOTE
: See [[MIDAS environment variables]] for a list of environment variables used by MIDAS.

== ROOT Package Installation ==
MIDAS has optional integration with [https://root.cern.ch ROOT]. ROOT is not required for acquiring data or storing it in midas files, but is used by some of the analysis tools (including the example analyzer that will be built later in this Quickstart). It is also possible to write midas data directly to ROOT files if desired.

If you require ROOT integration, follow the installation instructions on the [https://root.cern.ch ROOT website] and set the ROOTSYS environment variable.

If you do not require ROOT integration, leave the ROOTSYS variable undefined.

== Adding SSL support ==

We recommend running mhttpd (the MIDAS web server) behind an industry-grade web proxy like apache httpd or nginx. httpd/nginx will then handle the SSL security. It is however also possible to build mhttpd with built-in SSL support. See [[Mhttpd|the mhttpd documentation]] for more details about web security.

If you wish to enable SSL support in mhttpd itself:


[mhostpc] cd $MIDASSYS
[mhostpc] make mbedtls


== MIDAS Package Installation ==
The MIDAS package will be installed on the ''experiment host'' in the directory given by MIDASSYS - see [[#Environment Variables]].

Install the MIDAS and other package(s) from the MIDAS git repository in an xterm as follows:

[mhostpc] mkdir $HOME/packages
[mhostpc] cd $HOME/packages
[mhostpc] git clone https://gitlab.triumf.ca/midas/midas.git --recurse-submodules


If you already cloned MIDAS and forgot the "--recurse-submodules", you can do a "git submodule update --init --recursive" later. If you want to pull MIDAS later to update to the current version, you need either "git pull --recurse-submodules". To tell git to always recurse, you can set a global flag via "git config submodule.recurse true".

The MIDAS Makefile will auto-detect whether MSCB, ROOT, MYSQL, SQLITE, ODBC etc. are available (or have been installed). If you need these features, make sure they are available before building the MIDAS package.

[mhostpc] cd $MIDASSYS
[mhostpc] mkdir build
[mhostpc] cd build
[mhostpc] cmake .. ### configure build process (see note below about cmake/cmake3)
[mhostpc] make install ### build and install midas to bin/ and lib/ directories
[mhostpc] cd ..
[mhostpc] ls -l bin/odbedit ### check that odbedit has been created (do not run it yet)


MIDAS requires cmake version 3.10 or later. On some systems, you have to call "cmake3 ..", on others just "cmake ..". Some systems might require you to install cmake first (see the [https://cmake.org/download/ cmake website; installation is very simple).

In case of problems see
* [[Installation#Installing MIDAS|gitlab is down]] or
* [[Installation/Compilation problems#SSL certificate errors|SSL certificate errors]].

==ROOTANA Package Installation==

rootana is an optional framework for analyzing midas data in realtime and displaying plots (either by a graphical interface or via webpages). Most of the features are now available in manalyzer, which is included in the base midas install.

If you require the legacy ROOTANA package, you can install it with:

[mhostpc] cd $HOME/packages
[mhostpc] git clone https://gitlab.triumf.ca/midas/rootana.git
[mhostpc] cd rootana
[mhostpc] make


add ROOTANASYS=$HOME/packages/rootana to the login file.

Install the JSROOT package:
<pre>
cd $HOME/packages
git clone https://github.com/linev/jsroot.git
</pre>

add JSROOTSYS=$HOME/packages/jsroot to the login file.

==ROODY Package Installation==

ROODY is a legacy package for displaying live ROOT histograms. It is generally not used by new experiments.

If ROODY is needed,

[mhostpc] cd $HOME/packages
[mhostpc] git clone https://bitbucket.org/tmidas/roody
[mhostpc] cd roody
[mhostpc] make


== Create the Experiment file exptab ==
In this example, the experimental directory is {{Filepath|path=$HOME/online}}. Create this directory on the ''experiment host'' (localhost):

[mhostpc] mkdir $HOME/online
[mhostpc] cd $HOME/online
Then create the [[exptab]] file {{Filepath|path=$HOME/online/exptab}} containing the three parameters:
# the experiment name
# location of MIDAS shared memory buffers
# username
as follows (using parameters for your own experiment):

[mhostpc] cat > exptab
e777 /home/johnfoo/online johnfoo

The path of the {{File|name=exptab}} file is given by MIDAS_EXPTAB - see [[#Environment Variables]].

== Create shared memory files ==
At this point you should be able to run [[odbedit]] on the ''experiment host'' (localhost). The first time {{Utility|name=odbedit}} is run, it will create the required {{File|name=.*.SHM}} files in the MIDAS experiment directory {{Filepath|path=$HOME/online}}. These are the saved files for the MIDAS shared memory.

The default directory {{Filepath|path=$HOME/online}} will contain the MIDAS messages file ({{File|name=midas.log}}) and
any data files you may create. If you want an alternative location for the data files see [[#Customizing your experiment]].

Run {{Utility|name=odbedit}}, and type {{Odbedit cmd|cmd=ls}} to list the default directories.
[mhostpc] odbedit
[local:e777:S]/>ls
Experiment
System
Programs
Logger
Runinfo
Alarms

== Run the MIDAS Web Server ==

=== Using apache ===

Recommended for production systems with maximum security is to run mhttpd behind an industry-standard SSL/https proxy with password protection. We document instructions for apache here, but you could nginx instead.

To do this using apache httpd on CentOS-7, do this:
* configure apache httpd per instructions for [https://daq00.triumf.ca/DaqWiki/index.php/Ubuntu#Install_apache_httpd_proxy_for_midas_and_elog Ubuntu] or [https://daq00.triumf.ca/DaqWiki/index.php/SLinstall#Configure_HTTPS_server_.28CentOS7.29 CentOS 7]
* run mhttpd
* use web browser to connect to https://mhostpc/
* enter the user name and password configured in apache httpd
* the midas status page should open

=== Standalone (less secure) ===

For simplified installation, one can use standalone mhttpd with built-in https and password protection. This configuration
is only as secure as the underlying mongoose and OpenSSL web server and encryption packages. Use the odbedit tool to edit settings in the [[Webserver_ODB_tree|Webserver ODB tree]]:

* Set "Enable https port" to "y"
* Set "https port passwords" to "y"

Many current operating systems implement a built-in firewall which will block outside access to mhttpd. See instructions for [https://daq00.triumf.ca/DaqWiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29 CentOS 7].

Start [[mhttpd]] on the ''experiment host'' (localhost) like this:
[mhostpc] mhttpd

You will get the following messages:

[mhttpd,ERROR] [mhttpd.cxx:13880:mongoose,ERROR] mongoose web server cannot find password file "/home/johnfoo/online/htpasswd.txt"
[mhttpd,ERROR] [mhttpd.cxx:13881:mongoose,ERROR] please create password file: touch /home/johnfoo/online/htpasswd.txt
[mhttpd,ERROR] [mhttpd.cxx:13818:mongoose,ERROR] mongoose web server password file "/home/johnfoo/online/htpasswd.txt" has no passwords for realm "exptname"
[mhttpd,ERROR] [mhttpd.cxx:13819:mongoose,ERROR] please add passwords by running: htdigest /home/johnfoo/online/htpasswd.txt exptname midas
mongoose_init: Error: Cannot initialize authorization object!
Error: Could not start the mongoose web server, see messages and midas.log, bye!


Create the password file by following the instructions printed by mhttpd. The http digest domain name is the experiment name, suggested default user name is "midas". You will be asked to type in a password. (htdigest is part of the httpd-tools package)


[mhostpc] htdigest -c /home/johnfoo/online/htpasswd.txt exptname midas
Adding password for midas in realm exptname.
New password:
Re-type new password:

It is a good idea to set the password file {{Filepath|path=htpasswd.txt}} readable and writable by owner only.

Run {{Utility|name=mhttpd}} again.

If you get error messages like <code>https port "[::]:8443" requested, but mhttpd compiled without MG_ENABLE_SSL</code> then you need to re-compile MIDAS after [[Mhttpd#Build_mhttpd_with_SSL_support|adding SSL support]].

Otherwise, you will get the following messages:

HTTP Digest authentication with realm "exptname" and password file "/home/johnfoo/online/htpasswd.txt"
Hostlist off, connections from anywhere will be accepted
Listening on "http://localhost:8080", passwords OFF, hostlist OFF
Listening on "http://[::1]:8080", passwords OFF, hostlist OFF
[mhttpd,ERROR] [mhttpd.cxx:15739:mongoose_listen,ERROR] cannot find SSL certificate file "/home/johnfoo/online/ssl_cert.pem"
[mhttpd,ERROR] [mhttpd.cxx:15740:mongoose_listen,ERROR] please create SSL certificate file using openssl: cd $MIDASSYS; openssl req -new -nodes -newkey rsa:2048 -sha256 -out ssl_cert.csr -keyout ssl_cert.key -subj "/C=/ST=/L=/O=midas/OU=mhttpd/CN=localhost"; openssl x509 -req -days 365 -sha256 -in ssl_cert.csr -signkey ssl_cert.key -out ssl_cert.pem; cat ssl_cert.key >> ssl_cert.pem
[mhttpd,ERROR] [mhttpd.cxx:15741:mongoose_listen,ERROR] or using certbot (recommened): setup certbot per Let's Encrypt instructions, certificates are typically saved in /etc/letsencrypt/live/$HOSTNAME/, copy fullchain.pem and privkey.pem to $MIDASSYS; cd $MIDASSYS; cat fullchain.pem privkey.pem > ssl_cert.pem
[mhttpd,ERROR] [mhttpd.cxx:15918:mongoose_init,ERROR] Failed to listen on a TCP port enabled in ODB /WebServer
Error: Could not start the mongoose web server, see messages and midas.log, bye!


Create a self-signed certificate suitable for initial testing by executing the commands printed by mhttpd.

For production use, you should create a properly signed certificate, see [[Mhttpd#Create an SSL certificate|create your own SSL certificate]] or you should run mhttpd behind an SSL/https proxy (apache httpd).

Now restart {{Utility|name=mhttpd}}
[mhostpc] mhttpd
HTTP Digest authentication with realm "test_pol" and password file "/home/johnfoo/online/htpasswd.txt"
Hostlist off, connections from anywhere will be accepted
Listening on "http://localhost:8080", passwords OFF, hostlist OFF
Listening on "http://[::1]:8080", passwords OFF, hostlist OFF
Mongoose web server will use https certificate file "./ssl_cert.pem"
Listening on "https://[::]:8443", passwords enabled, hostlist OFF
Mongoose web server is using 1 threads
Mongoose web server is using 2 threads
Mongoose web server is using 3 threads
Mongoose web server is using 4 threads

Now point a web browser running on the same host computer (localhost) to https://localhost:8443.
If the web browser is running on a different computer, go to URL of the form

https://mhostpc.triumf.ca:8443 (substitute your host machine name and domain for "mhostpc.triumf.ca")

If you are using the default SSL certificate you will probably get a message: "This Connection is Untrusted". Click "I understand the risks" and add an exception. This is because the test certificate is self-signed. Then confirm an exception.

You should then see an authentication box asking you for the user name and password. The user name is "midas". Enter the password you just created. The Midas [[Status Page]] should appear with multiple buttons for run control as well as equipment listing (no equipments will be listed as yet) and application listings. Please refer to [[mhttpd]] (the MIDAS Web-based Run Control utility) for further information. You can start and stop runs from the main status page, and use the [[ODB Page]] to access the database (ODB).

; Note
: Default ports of 8080 and 8443 are used by [[mhttpd]]. To use some other ports, change the relevant values in the [[Webserver_ODB_tree|Webserver ODB tree]].

== Run the MIDAS logger ==
Start the MIDAS logger [[mlogger]] on the ''experiment host'' (localhost) :
[mhostpc] mlogger
This should start without error. It is usually run as a daemon, however in this case it is run in a terminal to check for errors. Starting the midas logger [[mlogger]] will automatically create more keys in the {{Odbpath|path=/Logger}} ODB tree.

From the MIDAS status page, go to the "Programs" page, then the "Logger" page and set ODB variables:
* Required -> set to "y"
* Start command -> set to "mlogger -D"
* Alarm Class -> set to "Alarm"

To test this, go to the "Programs" page, press "stop logger", then "start logger".

Start a run, wait 10 seconds, stop the run.

From the MIDAS "Status" page, go to the "logging channels -> channel #0" and set ODB variables:
* Output -> "FILE"

Start a run, go to the "Status" page, observe that the output file is written with lz4 compression and has the "mid.lz4" extension.

Stop the run.

== Run test frontend ==

Run <code>fetest</code>

On the MIDAS "Status" page there will be 5 new equipment entries:

{| class="wikitable"
|+ Equipment created by the fetest program
|-
! Equipment name !! Data it sends !! How often data is sent
|-
| test_random || Random data banks || 1Hz
|-
| test_slow || Example slow control data for testing the midas history plots || 1Hz, even when a run isn't in progress
|-
| test_rare || Data that look like a sine wave || 0.1Hz, even when a run isn't in progress
|-
| test_bulk || Large events || 1Hz
|-
| test_rpc || None. It shows an example of a "remote procedure call" (RPC) handler. Javascript/C++/Python code can connect to the program, and it will respond with the current time (in this toy example) || never
|}

Note the event counters for test_slow and test_rare are already incrementing.

Start a run.

Observe that the event counters for test_bulk and test_random are also incrementing now, and that the output data file size is growing.

Click on the "Event dump" page and then the "Run" button. You'll see the raw data that is being written to disk. (The data from most of the equipment is double-precision floating point numbers; the data from the "bulk" equipment is just a bunch of zeros).

Return the status page and stop the run.

== Create a history plot ==

Go to the MIDAS "History" page.

Say "new", panel name "test". In the history editor select event "test_slow", and click the "save" button (you can ignore the rest of the options for now). If the "test_slow" event isn't listed, make sure that the mlogger program is running, as described above.

History plot "test" should open with a sine-wave plot. Data on the plot should update as long as "fetest" is running (there is no need to start a run).

== Clients run on Localhost only ==
;NOTE:
* If creating a MIDAS experiment with a '''REMOTE''' frontend, '''continue by following the instructions''' [[#Running with one or more REMOTE frontends]].
* If all clients are running on the Experiment Host (i.e. '''localhost'''), continue with the following instructions:

=== Frontend and Analyzer (localhost) ===

THIS SECTION IS VERY OBSOLETE

There are several examples of [[Frontend user code]] ({{File|name=frontend.c}}) in the MIDAS package available under {{Filepath|path=$MIDASSYS/examples/}}. Choose a suitable example from this directory that you can later modify for your own particular setup (e.g. for a slow control, choose the {{Filepath|path=../slowcont/}} example).

A '''frontend''' is a program that usually reads data from the hardware and sends it to a buffer to be logged and analyzed. You can find documentation about the frontend structure under [[Frontend Application]], [[Frontend Operation]] and [[Frontend user code]].

The example chosen here is from {{Filepath|path=$MIDASSYS/examples/experiment/}}. It does '''not need any hardware''' and produces simulated events.

On the MIDAS ''experiment host'' (localhost), copy the example to the experiment directory and build it.

[mhostpc] cd $HOME/online
[mhostpc] cp $MIDASSYS/examples/experiment/* .
[mhostpc] make
The analyzer will only build if ROOT has previously been installed.
At this point the frontend and the analyzer should be ready if no errors were generated during the build. Try the frontend and analyzer by starting them in xterms.

[mhostpc] frontend
[mhostpc] analyzer

=== Equipment ODB tree ===
Starting the frontend will automatically create the {{Odbpath|path=/Equipment}} ODB tree and one or more equipments. In this case, the equipments "trigger" and "scaler" have been defined by code in {{File|name=frontend.c}} and created the first time frontend runs.

[mhostpc] odbedit
[local:e777:S]/>ls /equipment
Trigger
Scaler 

View the MIDAS main status web page (by pointing your browser to https://localhost:8443/ or appropriate port if not using the default).
Observe on the [[Status Page]] that the two Equipments (Trigger, Scaler) have appeared in the Equipment section (Figure 1) and are coloured green.
 
'''Click thumbnail to enlarge'''
[[File:quickstart_status_page.png|thumb|left|Figure 1: web server status page showing example frontend equipments]]
<div style="clear: both"></div> 

Rather than using odbedit, you can view the contents of these equipments by clicking on the ODB button, then on {{Odbpath|path=/Equipment}}, then {{Odbpath|path=Trigger}} or {{Odbpath|path=Scaler}}. See [[ODB Page]] for more information.

=== Start a run ===
Start a run by pressing the Start button on the [[Status Page]].
The Equipment display on the Status Page will now show some event statistics (Figure 2)
'''Click thumbnail to enlarge'''
[[File:sample_frontend_status.png|thumb|left|Figure 2: Equipments shown on Status Page showing event statistics]]
<div style="clear: both"></div> 

The frontend will show an event display that will update when a run is started, e.g.

Sample Frontend connected to <local>. Press "!" to exit 18:48:2546:08
==========================================================================
Run status: Running Run number 2 |/ /
===========================================================================
Equipment Status Events Events/sec Rate[B/s] ODB->FE FE->OD
---------------------------------------------------------------------------
Trigger OK 13948 99.0 5413.0 0 140
Scaler OK 15 0.3 15.4 0 15

The [[Status Page]] will also show events being generated (click the refresh button if necessary).

=== Event Dump ===
While a run is in progress, the midas application [[mdump]] will provide you an event dump of the collected data from the running frontend, e.g.

mhostpc> mdump -l 50
blank
- MIDAS revision: Mon Nov 2 11:50:51 2015 -0800 - 3b66779 -- Enter <!> to Exit ------- Midas Dump ---
------------------------ Event# 1 ------------------------
Evid:0001- Mask:0000- Serial:0- Time:0x567a0c5d- Dsize:40/0x28
#banks:2 - Bank list:-ADC0TDC0-
blank
Bank:ADC0 Length: 8(I*1)/2(I*4)/4(Type) Type:Unsigned Integer*2
1-> 0x0167 0x03c6 0x0069 0x0073
blank
Bank:TDC0 Length: 8(I*1)/2(I*4)/4(Type) Type:Unsigned Integer*2
1-> 0x0051 0x00ff 0x004a 0x00ec

For further data processing/analysis, either the midas analyzer or the rootana can used for data display as well.

=== Create a startup script (localhost) ===
It is useful to create a script that will automatically start all the required clients for the experiment. This is called a startup script. Here, the startup script will be {{File|name=start_daq.sh}} (any name can be used). Create {{Filepath|path=$HOME/online/bin/start_daq.sh}} on the ''experiment host'' as follows (supply the [[mhttpd]] ports if default is not used):

#!/bin/sh
# start_daq.sh
cd $HOME/online
odbedit -c clean
# start mhttpd on default port. (Mongoose https version)
mhttpd -D # you can optionally restrict access to localhost and other specified hosts - see [[mhttpd]]
xterm -e ./frontend &
xterm -e ./analyzer &
mlogger -D
#end file

Before running this script, you will need to shutdown any running clients:
mhostpc> odbedit -c "sh all"
or you can use the {{Utility|name=mhttpd}} [[Programs Page]] to shut them down.

The script will start them as daemons.
By running the startup script start_daq.sh, several MIDAS applications will be started in sequence:
# Cleanup previous MIDAS application (if any).
# Start the MIDAS web server [mhttpd]
# Start the frontend application in its own xterm (for debugging purpose).
# Start the analyzer application in its own xterm (for debugging purpose).
# Start the MIDAS Data logger [mlogger]
[mhostpc] sh ./start_daq

You may wish to modify the script to restart missing clients only.

Next [[#Customizing your experiment|customize your experiment]].

== Running with one or more REMOTE frontends ==
These instructions assume you have already followed the instructions to setup an experiment running on the MIDAS experiment host (localhost), i.e. you have setup the MIDAS Environment variables, downloaded MIDAS packages, setup the [[mhttpd]] password and have {{Utility|name=mhttpd}} running successfully.

In the case of '''remote''' frontend(s), the remote cpu(s) typically have access to some or all of the hardware. They might be VMIC cpus running in VME crates for example, connected to the MIDAS Host via Ethernet. They read data from the hardware and send it back to the MIDAS ''experiment host'' to be logged and analyzed.

=== Experimental setup (Remote) ===
These instructions assume that the MIDAS ''experiment host'' computer is 64-bit (if not see [[#Build 32-bit MIDAS libraries|note below]]), and there is one ''remote host'', a 32-bit machine named "rlxhost".

The ''remote host'' mounts the {{Filepath|path=/home}} disk of the ''experiment host'', so that it '''has access to the MIDAS packages''', the {{Filepath|path=$HOME/online}} directory and shares the .cshrc (or .bashrc) script. It is convenient to set up ssh keys (use ssh-keygen) so that the ''remote host'' can be accessed without supplying the password from the ''experiment host''.

If your setup is different, you will have to make changes as appropriate.

=== Build 32-bit MIDAS libraries ===
NOTE

If the MIDAS ''experiment host'' computer is 32-bit (i.e. the same as the ''remote host''), which is a simpler setup, you will have built the 32-bit MIDAS libraries already, and can skip the rest of this section. Note however, that in the .cshrc file in the [[#Modify .cshrc (.bashrc) for Remote host|following section]] you will not need the lines starting "# select 64-bit or 32-bit MIDAS and ROOT". The PATH should contain $MIDASSYS/bin on both experiment host and remote host. This will point to the 32-bit MIDAS libraries.
------------------------------

 
Assuming the MIDAS host computer is 64-bit, you will have already built the 64-bit MIDAS libraries.
In this case, if the remote frontend is 32-bit, you will need to build the 32-bit MIDAS libraries on the 64-bit machine, i.e.

[mhostpc] cd /home/packages/midas
[mhostpc] make linux32 ### build the 32-bit MIDAS libraries on 64-bit machine
[mhostpc] ls -l linux-m32/bin/odbedit ### check that the 32-bit odbedit has been created 
Do not run {{Utility|name=odbedit}} yet on the ''remote host''!

If the "make linux32" command fails, you may need to install 32bit support for your compiler (e.g. "apt install g++-multilib").

=== Modify .cshrc (.bashrc) for Remote host ===
Add the following to the {{File|name=.cshrc}} (modified appropriately if using {{File|name=.bashrc}}). When running with a REMOTE Host, the application [[mserver]] will be started (using a particular port) on the MIDAS ''experiment host'' machine only. Note that the environment variable MIDAS_SERVER_HOST is defined on the remote machine(s) but NOT on the ''experiment host''.

If ROOT has been installed properly and the script <ROOT installation directory>/bin/thisroot.sh is called from .cshrc or .bashrc, the ROOTSYS environment variable should be defined and the ROOT bin directory should be in the PATH environment variable. Make sure that this is the case.


# setenv MIDAS_HOST mhostpc # substitute your experiment host name
# setenv REMOTE_HOST rlxhost # substitute your remote host name
# setup the MIDAS mserver host
#
switch (`hostname`)
case $MIDAS_HOST*:
unsetenv MIDAS_SERVER_HOST ## MIDAS_SERVER_HOST not defined
breaksw
default:
setenv MIDAS_SERVER_HOST $MIDAS_HOST.triumf.ca:1175 ## for remote host, define MIDAS_SERVER_HOST, use your domain here
## as hostname + domain name + default mserver port
## port must match that of [[mserver]] used in start_daq.sh
endsw
#
# select 64-bit or 32-bit MIDAS
#
switch (`uname -i`)
case i386:
setenv PATH .:$MIDASSYS/linux-m32/bin:$PATH
breaksw
default:
setenv PATH .:$MIDASSYS/linux/bin:$PATH
endsw
#
setenv PATH .:$HOME/online/bin:$HOME/packages/roody/bin:$PATH
#

After executing {{File|name=.cshrc}} (or {{File|name=.bashrc}}) or logging out, on the MIDAS host (mhostpc) make sure environment variable MIDAS_SERVER_HOST is NOT defined
[mhostpc] echo $MIDAS_SERVER_HOST
[mhostpc] MIDAS_SERVER_HOST: Undefined variable.

and on the '''''remote host''''', make sure MIDAS_SERVER_HOST IS defined:
[rlxhost] echo $MIDAS_SERVER_HOST
[rlxhost] MIDAS_SERVER_HOST

Also on the ''remote host'', make sure that the correct (32-bit) {{Utility|name=odbedit}} will be used (do not run it yet!)
[rlxhost] which odbedit
/home/johnfoo/packages/midas/linux-m32/bin/odbedit

=== Grant Remote host access permission ===
Give permission for the remote host(s) to access the experiment by following the instructions to [[Security#MIDAS programs on remote machines|allow MIDAS programs on remote machines]]. If the MIDAS web server [[mhttpd]] is running, use [[ODB Page]] to edit the ODB keys listed, otherwise use [[odbedit]] on the ''experiment host'' machine.

=== Start mserver on Experiment Host ===
The application [[mserver]] is not required when running an experiment on the ''experiment host'' (localhost) only, but it '''IS required''' for access from a '''REMOTE''' frontend.
Start [[mserver]] on MIDAS host (default port 1175) or use the "-p" option for a different port.
The port must match the port defined for environment variable MIDAS_SERVER_HOST in {{File|name=.cshrc}} (or {{File|name=.bashrc}}) [[#Modify .cshrc (.bashrc) for Remote host|above]].
[mhostpc] mserver -D

The environment variables on the REMOTE host should be:

[rlxhost] printenv | grep MIDAS
MIDASSYS=/home/e777/packages/midas
MIDAS_EXPT_NAME=e777
MIDAS_SERVER_HOST=mhostpc.triumf.ca:1175
(a different port may be used - see above)

=== Check access to ODB from Remote host ===
Run {{Utility|name=odbedit}} on the remote host to check access to ODB.
[rlxhost] odbedit
[mhostpc.triumf.ca:1175:e777:Stopped]/>ls 
If it does not work correctly, check that
* MIDAS_SERVER_HOST is defined on REMOTE host with same port as [[mserver]]
* mserver is running on MIDAS host with the correct port
* MIDAS_SERVER_HOST is NOT defined on MIDAS host
* remote access permission to REMOTE host has been granted as described above.

=== Build the analyzer on Experiment Host ===
The analyzer will be built and run on the MIDAS ''experiment host'' machine.
Select an example from $MIDASSYS/examples/ (see [[#Frontend and Analyzer (localhost)]]) and copy it to $HOME/online on the MIDAS host machine. Since we are assuming a 64-bit MIDAS host and a 32-bit remote host, edit the example Makefile so only the analyzer will be built.

=== Build the frontend (32bit) for Remote host ===
The frontend runs on a 32-bit remote host in this example. It therefore must either
* be built 32-bit on the 64-bit MIDAS host with the "-m32" flag
* or built on the REMOTE host
and in either case the [[#Build 32-bit MIDAS libraries|32-bit MIDAS libraries]] used for linking.

To avoid confusion, create a different subdirectory for the frontend, e.g.
[rlxhost] mkdir $HOME/online/fe_32
Copy the Makefile and frontend code from the same example to this directory.
Modify the Makefile as needed, and only build the frontend.

Once built, start the frontend in an xterm on the REMOTE host.
[rlxhost] $HOME/online/fe_32/frontend
Check that it connects successfully to the experiment and the Equipments appear on the [[Status Page]] as described under [[#Frontend and Analyzer (localhost)]]. The Equipment Status (Figure 2) should show that the Equipments are now running on the remote cpu, i.e. "SampleFrontend@rlxhost.triumf.ca" rather than "Sample Frontend@localhost".

=== Run the Analyzer on Experiment host ===
Start the analyzer in an xterm on the MIDAS ''experiment host''
[johnfoo@mhostpc e777] $HOME/online/analyzer

Continue by starting a run and dumping the data as described under [[#Frontend and Analyzer (localhost)]].

=== Create experiment startup script ===
It is useful to create a script that will automatically start all the required clients for the experiment. This is called a startup script (any name can be used). Create {{Filepath|path=$HOME/online/bin/start_daq.sh}} on the ''experiment host'' as follows, (supply the [[mhttpd]] ports if default is not used) :

#!/bin/sh
# start_daq.sh
cd $HOME/online
#
switch (`hostname`)
case $MIDAS_HOST*: ## MIDAS_HOST is assigned in .cshrc (.bashrc)
echo "Good, we are on $MIDAS_HOST"
breaksw
case $REMOTE_HOST*: ## REMOTE_HOST is assigned in .cshrc (.bashrc)
echo "startup script should be executed on $MIDAS_HOST"
endsw
#
odbedit -c clean
# start mhttpd on default port (Mongoose https version)
mhttpd -D # you can optionally restrict access to localhost and/or other specified hosts - see [[mhttpd]]
# start mserver on default port (use argument -p to use a different port)
mserver -D # start mserver on default port (1175) or a different port using -p option - see [[mserver]]
# the port must match that of MIDAS_SERVER_HOST in .cshrc (.bashrc) - see above
# access must be specifically allowed - see [[#Grant Remote host access permission||above]]
xterm -e ./analyzer &
## start frontend on remote host
#ssh $REMOTE_HOST $HOME/online/bin/start_frontend -O >& $HOME/online/bin/start_frontend.log &
mlogger -D
#end file

Note that command to start the frontend on the remote host is commented out. It can be uncommented once the start_frontend script is written and tested.

Shut down any clients that are running already by running odbedit on the MIDAS ''experiment host'' and execute the new startup script {{File|name=start_daq.sh}} (make it executable if desired)

[mhostpc] odbedit
[local:e777:S]/>sh all
[mhostpc] ./start_daq.sh 
and check that {{Utility|name=odbedit}} still works on the ''REMOTE host''

[rlxhost] odbedit
[mhostpc.triumf.ca:1175:e777:Stopped]/>ls 

You may wish to modify the script to restart missing clients only.

=== Create script to start frontend ===
The startup file {{File|name=start_daq.sh}} invoked a script {{File|name=start_frontend}} to start the remote frontend. This line was commented out (see [[#Create experiment startup script|above]]). Once the frontend is working, create a script to start the frontend (e.g. file {{Filepath|path=$HOME/online/bin/start_frontend}}).

#!/bin/tcsh
# Script to start frontend running on $REMOTE_HOST
#
# This script runs on the REMOTE HOST
#
echo "starting frontend for experiment $MIDAS_EXPT_NAME "
xterm -geometry 150x50+800+0 -fg white -bg blue -title "Frontend" -e "$HOME/online/fe_32/frontend" &
echo "Frontend task has been started"
exit

Check that this script works by running it on the REMOTE host. If successful,
run the commented command in an xterm on the ''experiment host''
[mhostpc] ssh $REMOTE_HOST $HOME/online/bin/start_frontend -O >& $HOME/online/bin/start_frontend.log & 
This command relies on password-less access set up with SSH keys (see [[#Experimental setup (Remote)]]).
If successful, uncomment line in {{File|name=start_daq.sh}}, and run the script to check that it can start the remote client(s) from the MIDAS ''experiment host'' (first shutting down all running clients) :

[mhostpc] odbedit -c "sh all"
[mhostpc] ./start_daq.sh 

Next [[#Customizing your experiment|customize your experiment]].

== Customizing your experiment ==

Here are some of the more common options provided for users to customize their experiments according to their own preferences:

; Write frontend(s) to read out and control your hardware
: Modify the example frontend and Makefile (from $MIDASSYS/examples/) you have already copied and built so it accesses your hardware.

: The example Frontend.c is an example of [[Frontend user code]]. You can find documentation about the frontend structure under [[Frontend Application]], [[Frontend Operation]] and [[Frontend user code]]. You may need to set up some [[Event Notification (Hot-Link)|Hot-Links]] in the frontend. The MIDAS package includes device drivers for some of the more common hardware. See [[MIDAS Driver Library]] for more information.

; Modify analyzer(s) for your data
: Modify the example analyzer code as necessary to analyze the data from your frontends.

; Slow Controls
: Control your slow-controls equipment with slow-control frontend(s) optionally using the [[#MIDAS Package Installation|MSCB]] system.

; Data logger
: Set up the data logger [[mlogger]] to log data to a storage device. Many options are available - see [[Data Logger]], [[mlogger]] and [[/Logger ODB tree]] for instructions.

; Allow programs to be restarted from the [[Programs Page]]
: See mhttpd [[Programs Page]] and [[/Programs ODB tree]]

; Write scripts which run at start and end of run
: See [[/Programs ODB tree#Execute on start run]] and [[/Programs ODB tree#Execute on stop run]]

; Create a shutdown script
: Create a script similar to the startup script to shutdown the experiment

; Lazy logger
: Set up the [[lazylogger]] for archiving data files for storage.

; Create edit-on-start parameters
: See [[Edit-on-start Parameters]]

; Set up the alarm system
: See [[Alarm System]] and [[Alarms Page]]

; Set up history system
: See [[History System]] and [[History Page]]

; Set up the Electronic logbook (Elog)
: See [[Elog]] for details

; Set up the Sequencer
: The [[Sequencer Page#Sequencer]] can be used to run a sequence of runs, changing parameters as needed.

; Write a webserver Custom page
: To better control and monitor your experiment, you may want to write a [[Custom Page]].

== Start MIDAS on boot ==

To start MIDAS mhttpd, mlogger and elogd at boot time, add following cron entries:
<pre>
[iris@iris00 ~]$ crontab -l
@reboot /home/iris/packages/midas/bin/mhttpd -D
@reboot /home/iris/packages/midas/bin/mlogger -D
@reboot /home/iris/packages/elog/elogd -c /home/iris/packages/elog/elogd.cfgiris -D
[iris@iris00 ~]$
</pre>

[[Category:Installation]]

Quickstart Linux

2026-09-29T16:15:38Z

Dfujimoto: /* MIDAS Package Installation */

{{Pagelinks}}
== Introduction ==
This quickstart shows you how to install the MIDAS packages on a linux box and create a '''MIDAS experiment'''.
The linux box will be called the MIDAS '''''experiment host'''''.

==Installation==
Log on to the ''[[#Introduction|Experiment Host]]'' and decide on a name for the MIDAS experiment (the ''experiment name'').
In these instructions, the MIDAS packages and experiment will be installed under the username "johnfoo".
The ODB and other shared memory buffers for the MIDAS experiment will reside on the ''experiment host'' (localhost).
MIDAS and the other required packages will be installed under directory {{Filepath|path=$HOME/packages}}. The MIDAS ''experiment directory'' will be {{Filepath|path=$HOME/online}} and the MIDAS ''experiment name'' will be "e777". The name of the ''experiment host'' will be "mhostpc".

;NOTE
: The user should substitute directory paths for the MIDAS packages and experiment directory as required, and names for ''experiment host'', username, ''experiment name'' appropriate to his/her own setup.

== Experimental setup ==
These instructions describe setting up an experiment that will run ONLY on the MIDAS ''experiment host'' computer (i.e. '''localhost only'''). In this case, the MIDAS ''experiment host'' has access to any hardware required, and all frontends, logger, analyzer etc. run on the one computer.

Modifications to run an experiment with additional frontend(s) running '''remotely''' (i.e. on remote cpu(s)) will also be described.
Follow the instructions for setting up a localhost-only experiment until indicated.

In both cases, the main MIDAS applications (e.g. [[mlogger]], [[mhttpd]]) run on the MIDAS ''experiment host''.

==Environment Variables==
The following Environment variables should to be added to the {{File|name=.cshrc}} or {{File|name=.bashrc}} file (depending on your shell) in the {{Filepath|path=$HOME}} directory on the ''experiment host'', so that they will be defined at login. '''Substitute appropriate values for your own setup'''.

{| class="wikitable"
|-
! csh !! bash !! comment
|-
| setenv MIDASSYS $HOME/packages/midas || export MIDASSYS=$HOME/packages/midas ||Base directory of the MIDAS package
|-
| setenv MIDAS_EXPTAB $HOME/online/exptab || export MIDAS_EXPTAB=$HOME/online/exptab || MIDAS experiment table
|-
| setenv MIDAS_EXPT_NAME e777 || export MIDAS_EXPT_NAME=e777 || MIDAS experiment name
|-
| setenv PATH .:$MIDASSYS/bin:$PATH || export PATH=$PATH:$MIDASSYS/bin || path
|}

Logout and login again, or source {{File|name=.cshrc}} (source {{File|name=.bashrc}}) for the changes to take effect.
;NOTE
: See [[MIDAS environment variables]] for a list of environment variables used by MIDAS.

== ROOT Package Installation ==
MIDAS has optional integration with [https://root.cern.ch ROOT]. ROOT is not required for acquiring data or storing it in midas files, but is used by some of the analysis tools (including the example analyzer that will be built later in this Quickstart). It is also possible to write midas data directly to ROOT files if desired.

If you require ROOT integration, follow the installation instructions on the [https://root.cern.ch ROOT website] and set the ROOTSYS environment variable.

If you do not require ROOT integration, leave the ROOTSYS variable undefined.

== Adding SSL support ==

We recommend running mhttpd (the MIDAS web server) behind an industry-grade web proxy like apache httpd or nginx. httpd/nginx will then handle the SSL security. It is however also possible to build mhttpd with built-in SSL support. See [[Mhttpd|the mhttpd documentation]] for more details about web security.

If you wish to enable SSL support in mhttpd itself:


[mhostpc] cd $MIDASSYS
[mhostpc] make mbedtls


== MIDAS Package Installation ==
The MIDAS package will be installed on the ''experiment host'' in the directory given by MIDASSYS - see [[#Environment Variables]].

Install the MIDAS and other package(s) from the MIDAS git repository in an xterm as follows:

[mhostpc] mkdir $HOME/packages
[mhostpc] cd $HOME/packages
[mhostpc] git clone https://gitlab.triumf.ca/midas/midas.git --recurse-submodules


If you already cloned MIDAS and forgot the "--recurse-submodules", you can do a "git submodule update --init --recursive" later. If you want to pull MIDAS later to update to the current version, you need either "git pull --recurse-submodules". To tell git to always recurse, you can set a global flag via "git config submodule.recurse true".

The MIDAS Makefile will auto-detect whether MSCB, ROOT, MYSQL, SQLITE, ODBC etc. are available (or have been installed). If you need these features, make sure they are available before building the MIDAS package.

[mhostpc] cd $MIDASSYS
[mhostpc] mkdir build
[mhostpc] cd build
[mhostpc] cmake .. ### configure build process (see note below about cmake/cmake3)
[mhostpc] make install ### build and install midas to bin/ and lib/ directories
[mhostpc] cd ..
[mhostpc] ls -l bin/odbedit ### check that odbedit has been created (do not run it yet)


MIDAS requires cmake version 3.10 or later. On some systems, you have to call "cmake3 ..", on others just "cmake ..". Some systems might require you to install cmake first (see the [https://cmake.org/download/ cmake website; installation is very simple).

In case of problems see
* [[Installation#Installing MIDAS|gitlab is down]] or
* [[Installation/Compilation problems#SSL certificate errors|SSL certificate errors]].

==ROOTANA Package Installation==

rootana is an optional framework for analyzing midas data in realtime and displaying plots (either by a graphical interface or via webpages). Most of the features are now available in manalyzer, which is included in the base midas install.

If you require the legacy ROOTANA package, you can install it with:

[mhostpc] cd $HOME/packages
[mhostpc] git clone https://bitbucket.org/tmidas/rootana
[mhostpc] cd rootana
[mhostpc] make


add ROOTANASYS=$HOME/packages/rootana to the login file.

Install the JSROOT package:
<pre>
cd $HOME/packages
git clone https://github.com/linev/jsroot.git
</pre>

add JSROOTSYS=$HOME/packages/jsroot to the login file.

==ROODY Package Installation==

ROODY is a legacy package for displaying live ROOT histograms. It is generally not used by new experiments.

If ROODY is needed,

[mhostpc] cd $HOME/packages
[mhostpc] git clone https://bitbucket.org/tmidas/roody
[mhostpc] cd roody
[mhostpc] make


== Create the Experiment file exptab ==
In this example, the experimental directory is {{Filepath|path=$HOME/online}}. Create this directory on the ''experiment host'' (localhost):

[mhostpc] mkdir $HOME/online
[mhostpc] cd $HOME/online
Then create the [[exptab]] file {{Filepath|path=$HOME/online/exptab}} containing the three parameters:
# the experiment name
# location of MIDAS shared memory buffers
# username
as follows (using parameters for your own experiment):

[mhostpc] cat > exptab
e777 /home/johnfoo/online johnfoo

The path of the {{File|name=exptab}} file is given by MIDAS_EXPTAB - see [[#Environment Variables]].

== Create shared memory files ==
At this point you should be able to run [[odbedit]] on the ''experiment host'' (localhost). The first time {{Utility|name=odbedit}} is run, it will create the required {{File|name=.*.SHM}} files in the MIDAS experiment directory {{Filepath|path=$HOME/online}}. These are the saved files for the MIDAS shared memory.

The default directory {{Filepath|path=$HOME/online}} will contain the MIDAS messages file ({{File|name=midas.log}}) and
any data files you may create. If you want an alternative location for the data files see [[#Customizing your experiment]].

Run {{Utility|name=odbedit}}, and type {{Odbedit cmd|cmd=ls}} to list the default directories.
[mhostpc] odbedit
[local:e777:S]/>ls
Experiment
System
Programs
Logger
Runinfo
Alarms

== Run the MIDAS Web Server ==

=== Using apache ===

Recommended for production systems with maximum security is to run mhttpd behind an industry-standard SSL/https proxy with password protection. We document instructions for apache here, but you could nginx instead.

To do this using apache httpd on CentOS-7, do this:
* configure apache httpd per instructions for [https://daq00.triumf.ca/DaqWiki/index.php/Ubuntu#Install_apache_httpd_proxy_for_midas_and_elog Ubuntu] or [https://daq00.triumf.ca/DaqWiki/index.php/SLinstall#Configure_HTTPS_server_.28CentOS7.29 CentOS 7]
* run mhttpd
* use web browser to connect to https://mhostpc/
* enter the user name and password configured in apache httpd
* the midas status page should open

=== Standalone (less secure) ===

For simplified installation, one can use standalone mhttpd with built-in https and password protection. This configuration
is only as secure as the underlying mongoose and OpenSSL web server and encryption packages. Use the odbedit tool to edit settings in the [[Webserver_ODB_tree|Webserver ODB tree]]:

* Set "Enable https port" to "y"
* Set "https port passwords" to "y"

Many current operating systems implement a built-in firewall which will block outside access to mhttpd. See instructions for [https://daq00.triumf.ca/DaqWiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29 CentOS 7].

Start [[mhttpd]] on the ''experiment host'' (localhost) like this:
[mhostpc] mhttpd

You will get the following messages:

[mhttpd,ERROR] [mhttpd.cxx:13880:mongoose,ERROR] mongoose web server cannot find password file "/home/johnfoo/online/htpasswd.txt"
[mhttpd,ERROR] [mhttpd.cxx:13881:mongoose,ERROR] please create password file: touch /home/johnfoo/online/htpasswd.txt
[mhttpd,ERROR] [mhttpd.cxx:13818:mongoose,ERROR] mongoose web server password file "/home/johnfoo/online/htpasswd.txt" has no passwords for realm "exptname"
[mhttpd,ERROR] [mhttpd.cxx:13819:mongoose,ERROR] please add passwords by running: htdigest /home/johnfoo/online/htpasswd.txt exptname midas
mongoose_init: Error: Cannot initialize authorization object!
Error: Could not start the mongoose web server, see messages and midas.log, bye!


Create the password file by following the instructions printed by mhttpd. The http digest domain name is the experiment name, suggested default user name is "midas". You will be asked to type in a password. (htdigest is part of the httpd-tools package)


[mhostpc] htdigest -c /home/johnfoo/online/htpasswd.txt exptname midas
Adding password for midas in realm exptname.
New password:
Re-type new password:

It is a good idea to set the password file {{Filepath|path=htpasswd.txt}} readable and writable by owner only.

Run {{Utility|name=mhttpd}} again.

If you get error messages like <code>https port "[::]:8443" requested, but mhttpd compiled without MG_ENABLE_SSL</code> then you need to re-compile MIDAS after [[Mhttpd#Build_mhttpd_with_SSL_support|adding SSL support]].

Otherwise, you will get the following messages:

HTTP Digest authentication with realm "exptname" and password file "/home/johnfoo/online/htpasswd.txt"
Hostlist off, connections from anywhere will be accepted
Listening on "http://localhost:8080", passwords OFF, hostlist OFF
Listening on "http://[::1]:8080", passwords OFF, hostlist OFF
[mhttpd,ERROR] [mhttpd.cxx:15739:mongoose_listen,ERROR] cannot find SSL certificate file "/home/johnfoo/online/ssl_cert.pem"
[mhttpd,ERROR] [mhttpd.cxx:15740:mongoose_listen,ERROR] please create SSL certificate file using openssl: cd $MIDASSYS; openssl req -new -nodes -newkey rsa:2048 -sha256 -out ssl_cert.csr -keyout ssl_cert.key -subj "/C=/ST=/L=/O=midas/OU=mhttpd/CN=localhost"; openssl x509 -req -days 365 -sha256 -in ssl_cert.csr -signkey ssl_cert.key -out ssl_cert.pem; cat ssl_cert.key >> ssl_cert.pem
[mhttpd,ERROR] [mhttpd.cxx:15741:mongoose_listen,ERROR] or using certbot (recommened): setup certbot per Let's Encrypt instructions, certificates are typically saved in /etc/letsencrypt/live/$HOSTNAME/, copy fullchain.pem and privkey.pem to $MIDASSYS; cd $MIDASSYS; cat fullchain.pem privkey.pem > ssl_cert.pem
[mhttpd,ERROR] [mhttpd.cxx:15918:mongoose_init,ERROR] Failed to listen on a TCP port enabled in ODB /WebServer
Error: Could not start the mongoose web server, see messages and midas.log, bye!


Create a self-signed certificate suitable for initial testing by executing the commands printed by mhttpd.

For production use, you should create a properly signed certificate, see [[Mhttpd#Create an SSL certificate|create your own SSL certificate]] or you should run mhttpd behind an SSL/https proxy (apache httpd).

Now restart {{Utility|name=mhttpd}}
[mhostpc] mhttpd
HTTP Digest authentication with realm "test_pol" and password file "/home/johnfoo/online/htpasswd.txt"
Hostlist off, connections from anywhere will be accepted
Listening on "http://localhost:8080", passwords OFF, hostlist OFF
Listening on "http://[::1]:8080", passwords OFF, hostlist OFF
Mongoose web server will use https certificate file "./ssl_cert.pem"
Listening on "https://[::]:8443", passwords enabled, hostlist OFF
Mongoose web server is using 1 threads
Mongoose web server is using 2 threads
Mongoose web server is using 3 threads
Mongoose web server is using 4 threads

Now point a web browser running on the same host computer (localhost) to https://localhost:8443.
If the web browser is running on a different computer, go to URL of the form

https://mhostpc.triumf.ca:8443 (substitute your host machine name and domain for "mhostpc.triumf.ca")

If you are using the default SSL certificate you will probably get a message: "This Connection is Untrusted". Click "I understand the risks" and add an exception. This is because the test certificate is self-signed. Then confirm an exception.

You should then see an authentication box asking you for the user name and password. The user name is "midas". Enter the password you just created. The Midas [[Status Page]] should appear with multiple buttons for run control as well as equipment listing (no equipments will be listed as yet) and application listings. Please refer to [[mhttpd]] (the MIDAS Web-based Run Control utility) for further information. You can start and stop runs from the main status page, and use the [[ODB Page]] to access the database (ODB).

; Note
: Default ports of 8080 and 8443 are used by [[mhttpd]]. To use some other ports, change the relevant values in the [[Webserver_ODB_tree|Webserver ODB tree]].

== Run the MIDAS logger ==
Start the MIDAS logger [[mlogger]] on the ''experiment host'' (localhost) :
[mhostpc] mlogger
This should start without error. It is usually run as a daemon, however in this case it is run in a terminal to check for errors. Starting the midas logger [[mlogger]] will automatically create more keys in the {{Odbpath|path=/Logger}} ODB tree.

From the MIDAS status page, go to the "Programs" page, then the "Logger" page and set ODB variables:
* Required -> set to "y"
* Start command -> set to "mlogger -D"
* Alarm Class -> set to "Alarm"

To test this, go to the "Programs" page, press "stop logger", then "start logger".

Start a run, wait 10 seconds, stop the run.

From the MIDAS "Status" page, go to the "logging channels -> channel #0" and set ODB variables:
* Output -> "FILE"

Start a run, go to the "Status" page, observe that the output file is written with lz4 compression and has the "mid.lz4" extension.

Stop the run.

== Run test frontend ==

Run <code>fetest</code>

On the MIDAS "Status" page there will be 5 new equipment entries:

{| class="wikitable"
|+ Equipment created by the fetest program
|-
! Equipment name !! Data it sends !! How often data is sent
|-
| test_random || Random data banks || 1Hz
|-
| test_slow || Example slow control data for testing the midas history plots || 1Hz, even when a run isn't in progress
|-
| test_rare || Data that look like a sine wave || 0.1Hz, even when a run isn't in progress
|-
| test_bulk || Large events || 1Hz
|-
| test_rpc || None. It shows an example of a "remote procedure call" (RPC) handler. Javascript/C++/Python code can connect to the program, and it will respond with the current time (in this toy example) || never
|}

Note the event counters for test_slow and test_rare are already incrementing.

Start a run.

Observe that the event counters for test_bulk and test_random are also incrementing now, and that the output data file size is growing.

Click on the "Event dump" page and then the "Run" button. You'll see the raw data that is being written to disk. (The data from most of the equipment is double-precision floating point numbers; the data from the "bulk" equipment is just a bunch of zeros).

Return the status page and stop the run.

== Create a history plot ==

Go to the MIDAS "History" page.

Say "new", panel name "test". In the history editor select event "test_slow", and click the "save" button (you can ignore the rest of the options for now). If the "test_slow" event isn't listed, make sure that the mlogger program is running, as described above.

History plot "test" should open with a sine-wave plot. Data on the plot should update as long as "fetest" is running (there is no need to start a run).

== Clients run on Localhost only ==
;NOTE:
* If creating a MIDAS experiment with a '''REMOTE''' frontend, '''continue by following the instructions''' [[#Running with one or more REMOTE frontends]].
* If all clients are running on the Experiment Host (i.e. '''localhost'''), continue with the following instructions:

=== Frontend and Analyzer (localhost) ===

THIS SECTION IS VERY OBSOLETE

There are several examples of [[Frontend user code]] ({{File|name=frontend.c}}) in the MIDAS package available under {{Filepath|path=$MIDASSYS/examples/}}. Choose a suitable example from this directory that you can later modify for your own particular setup (e.g. for a slow control, choose the {{Filepath|path=../slowcont/}} example).

A '''frontend''' is a program that usually reads data from the hardware and sends it to a buffer to be logged and analyzed. You can find documentation about the frontend structure under [[Frontend Application]], [[Frontend Operation]] and [[Frontend user code]].

The example chosen here is from {{Filepath|path=$MIDASSYS/examples/experiment/}}. It does '''not need any hardware''' and produces simulated events.

On the MIDAS ''experiment host'' (localhost), copy the example to the experiment directory and build it.

[mhostpc] cd $HOME/online
[mhostpc] cp $MIDASSYS/examples/experiment/* .
[mhostpc] make
The analyzer will only build if ROOT has previously been installed.
At this point the frontend and the analyzer should be ready if no errors were generated during the build. Try the frontend and analyzer by starting them in xterms.

[mhostpc] frontend
[mhostpc] analyzer

=== Equipment ODB tree ===
Starting the frontend will automatically create the {{Odbpath|path=/Equipment}} ODB tree and one or more equipments. In this case, the equipments "trigger" and "scaler" have been defined by code in {{File|name=frontend.c}} and created the first time frontend runs.

[mhostpc] odbedit
[local:e777:S]/>ls /equipment
Trigger
Scaler 

View the MIDAS main status web page (by pointing your browser to https://localhost:8443/ or appropriate port if not using the default).
Observe on the [[Status Page]] that the two Equipments (Trigger, Scaler) have appeared in the Equipment section (Figure 1) and are coloured green.
 
'''Click thumbnail to enlarge'''
[[File:quickstart_status_page.png|thumb|left|Figure 1: web server status page showing example frontend equipments]]
<div style="clear: both"></div> 

Rather than using odbedit, you can view the contents of these equipments by clicking on the ODB button, then on {{Odbpath|path=/Equipment}}, then {{Odbpath|path=Trigger}} or {{Odbpath|path=Scaler}}. See [[ODB Page]] for more information.

=== Start a run ===
Start a run by pressing the Start button on the [[Status Page]].
The Equipment display on the Status Page will now show some event statistics (Figure 2)
'''Click thumbnail to enlarge'''
[[File:sample_frontend_status.png|thumb|left|Figure 2: Equipments shown on Status Page showing event statistics]]
<div style="clear: both"></div> 

The frontend will show an event display that will update when a run is started, e.g.

Sample Frontend connected to <local>. Press "!" to exit 18:48:2546:08
==========================================================================
Run status: Running Run number 2 |/ /
===========================================================================
Equipment Status Events Events/sec Rate[B/s] ODB->FE FE->OD
---------------------------------------------------------------------------
Trigger OK 13948 99.0 5413.0 0 140
Scaler OK 15 0.3 15.4 0 15

The [[Status Page]] will also show events being generated (click the refresh button if necessary).

=== Event Dump ===
While a run is in progress, the midas application [[mdump]] will provide you an event dump of the collected data from the running frontend, e.g.

mhostpc> mdump -l 50
blank
- MIDAS revision: Mon Nov 2 11:50:51 2015 -0800 - 3b66779 -- Enter <!> to Exit ------- Midas Dump ---
------------------------ Event# 1 ------------------------
Evid:0001- Mask:0000- Serial:0- Time:0x567a0c5d- Dsize:40/0x28
#banks:2 - Bank list:-ADC0TDC0-
blank
Bank:ADC0 Length: 8(I*1)/2(I*4)/4(Type) Type:Unsigned Integer*2
1-> 0x0167 0x03c6 0x0069 0x0073
blank
Bank:TDC0 Length: 8(I*1)/2(I*4)/4(Type) Type:Unsigned Integer*2
1-> 0x0051 0x00ff 0x004a 0x00ec

For further data processing/analysis, either the midas analyzer or the rootana can used for data display as well.

=== Create a startup script (localhost) ===
It is useful to create a script that will automatically start all the required clients for the experiment. This is called a startup script. Here, the startup script will be {{File|name=start_daq.sh}} (any name can be used). Create {{Filepath|path=$HOME/online/bin/start_daq.sh}} on the ''experiment host'' as follows (supply the [[mhttpd]] ports if default is not used):

#!/bin/sh
# start_daq.sh
cd $HOME/online
odbedit -c clean
# start mhttpd on default port. (Mongoose https version)
mhttpd -D # you can optionally restrict access to localhost and other specified hosts - see [[mhttpd]]
xterm -e ./frontend &
xterm -e ./analyzer &
mlogger -D
#end file

Before running this script, you will need to shutdown any running clients:
mhostpc> odbedit -c "sh all"
or you can use the {{Utility|name=mhttpd}} [[Programs Page]] to shut them down.

The script will start them as daemons.
By running the startup script start_daq.sh, several MIDAS applications will be started in sequence:
# Cleanup previous MIDAS application (if any).
# Start the MIDAS web server [mhttpd]
# Start the frontend application in its own xterm (for debugging purpose).
# Start the analyzer application in its own xterm (for debugging purpose).
# Start the MIDAS Data logger [mlogger]
[mhostpc] sh ./start_daq

You may wish to modify the script to restart missing clients only.

Next [[#Customizing your experiment|customize your experiment]].

== Running with one or more REMOTE frontends ==
These instructions assume you have already followed the instructions to setup an experiment running on the MIDAS experiment host (localhost), i.e. you have setup the MIDAS Environment variables, downloaded MIDAS packages, setup the [[mhttpd]] password and have {{Utility|name=mhttpd}} running successfully.

In the case of '''remote''' frontend(s), the remote cpu(s) typically have access to some or all of the hardware. They might be VMIC cpus running in VME crates for example, connected to the MIDAS Host via Ethernet. They read data from the hardware and send it back to the MIDAS ''experiment host'' to be logged and analyzed.

=== Experimental setup (Remote) ===
These instructions assume that the MIDAS ''experiment host'' computer is 64-bit (if not see [[#Build 32-bit MIDAS libraries|note below]]), and there is one ''remote host'', a 32-bit machine named "rlxhost".

The ''remote host'' mounts the {{Filepath|path=/home}} disk of the ''experiment host'', so that it '''has access to the MIDAS packages''', the {{Filepath|path=$HOME/online}} directory and shares the .cshrc (or .bashrc) script. It is convenient to set up ssh keys (use ssh-keygen) so that the ''remote host'' can be accessed without supplying the password from the ''experiment host''.

If your setup is different, you will have to make changes as appropriate.

=== Build 32-bit MIDAS libraries ===
NOTE

If the MIDAS ''experiment host'' computer is 32-bit (i.e. the same as the ''remote host''), which is a simpler setup, you will have built the 32-bit MIDAS libraries already, and can skip the rest of this section. Note however, that in the .cshrc file in the [[#Modify .cshrc (.bashrc) for Remote host|following section]] you will not need the lines starting "# select 64-bit or 32-bit MIDAS and ROOT". The PATH should contain $MIDASSYS/bin on both experiment host and remote host. This will point to the 32-bit MIDAS libraries.
------------------------------

 
Assuming the MIDAS host computer is 64-bit, you will have already built the 64-bit MIDAS libraries.
In this case, if the remote frontend is 32-bit, you will need to build the 32-bit MIDAS libraries on the 64-bit machine, i.e.

[mhostpc] cd /home/packages/midas
[mhostpc] make linux32 ### build the 32-bit MIDAS libraries on 64-bit machine
[mhostpc] ls -l linux-m32/bin/odbedit ### check that the 32-bit odbedit has been created 
Do not run {{Utility|name=odbedit}} yet on the ''remote host''!

If the "make linux32" command fails, you may need to install 32bit support for your compiler (e.g. "apt install g++-multilib").

=== Modify .cshrc (.bashrc) for Remote host ===
Add the following to the {{File|name=.cshrc}} (modified appropriately if using {{File|name=.bashrc}}). When running with a REMOTE Host, the application [[mserver]] will be started (using a particular port) on the MIDAS ''experiment host'' machine only. Note that the environment variable MIDAS_SERVER_HOST is defined on the remote machine(s) but NOT on the ''experiment host''.

If ROOT has been installed properly and the script <ROOT installation directory>/bin/thisroot.sh is called from .cshrc or .bashrc, the ROOTSYS environment variable should be defined and the ROOT bin directory should be in the PATH environment variable. Make sure that this is the case.


# setenv MIDAS_HOST mhostpc # substitute your experiment host name
# setenv REMOTE_HOST rlxhost # substitute your remote host name
# setup the MIDAS mserver host
#
switch (`hostname`)
case $MIDAS_HOST*:
unsetenv MIDAS_SERVER_HOST ## MIDAS_SERVER_HOST not defined
breaksw
default:
setenv MIDAS_SERVER_HOST $MIDAS_HOST.triumf.ca:1175 ## for remote host, define MIDAS_SERVER_HOST, use your domain here
## as hostname + domain name + default mserver port
## port must match that of [[mserver]] used in start_daq.sh
endsw
#
# select 64-bit or 32-bit MIDAS
#
switch (`uname -i`)
case i386:
setenv PATH .:$MIDASSYS/linux-m32/bin:$PATH
breaksw
default:
setenv PATH .:$MIDASSYS/linux/bin:$PATH
endsw
#
setenv PATH .:$HOME/online/bin:$HOME/packages/roody/bin:$PATH
#

After executing {{File|name=.cshrc}} (or {{File|name=.bashrc}}) or logging out, on the MIDAS host (mhostpc) make sure environment variable MIDAS_SERVER_HOST is NOT defined
[mhostpc] echo $MIDAS_SERVER_HOST
[mhostpc] MIDAS_SERVER_HOST: Undefined variable.

and on the '''''remote host''''', make sure MIDAS_SERVER_HOST IS defined:
[rlxhost] echo $MIDAS_SERVER_HOST
[rlxhost] MIDAS_SERVER_HOST

Also on the ''remote host'', make sure that the correct (32-bit) {{Utility|name=odbedit}} will be used (do not run it yet!)
[rlxhost] which odbedit
/home/johnfoo/packages/midas/linux-m32/bin/odbedit

=== Grant Remote host access permission ===
Give permission for the remote host(s) to access the experiment by following the instructions to [[Security#MIDAS programs on remote machines|allow MIDAS programs on remote machines]]. If the MIDAS web server [[mhttpd]] is running, use [[ODB Page]] to edit the ODB keys listed, otherwise use [[odbedit]] on the ''experiment host'' machine.

=== Start mserver on Experiment Host ===
The application [[mserver]] is not required when running an experiment on the ''experiment host'' (localhost) only, but it '''IS required''' for access from a '''REMOTE''' frontend.
Start [[mserver]] on MIDAS host (default port 1175) or use the "-p" option for a different port.
The port must match the port defined for environment variable MIDAS_SERVER_HOST in {{File|name=.cshrc}} (or {{File|name=.bashrc}}) [[#Modify .cshrc (.bashrc) for Remote host|above]].
[mhostpc] mserver -D

The environment variables on the REMOTE host should be:

[rlxhost] printenv | grep MIDAS
MIDASSYS=/home/e777/packages/midas
MIDAS_EXPT_NAME=e777
MIDAS_SERVER_HOST=mhostpc.triumf.ca:1175
(a different port may be used - see above)

=== Check access to ODB from Remote host ===
Run {{Utility|name=odbedit}} on the remote host to check access to ODB.
[rlxhost] odbedit
[mhostpc.triumf.ca:1175:e777:Stopped]/>ls 
If it does not work correctly, check that
* MIDAS_SERVER_HOST is defined on REMOTE host with same port as [[mserver]]
* mserver is running on MIDAS host with the correct port
* MIDAS_SERVER_HOST is NOT defined on MIDAS host
* remote access permission to REMOTE host has been granted as described above.

=== Build the analyzer on Experiment Host ===
The analyzer will be built and run on the MIDAS ''experiment host'' machine.
Select an example from $MIDASSYS/examples/ (see [[#Frontend and Analyzer (localhost)]]) and copy it to $HOME/online on the MIDAS host machine. Since we are assuming a 64-bit MIDAS host and a 32-bit remote host, edit the example Makefile so only the analyzer will be built.

=== Build the frontend (32bit) for Remote host ===
The frontend runs on a 32-bit remote host in this example. It therefore must either
* be built 32-bit on the 64-bit MIDAS host with the "-m32" flag
* or built on the REMOTE host
and in either case the [[#Build 32-bit MIDAS libraries|32-bit MIDAS libraries]] used for linking.

To avoid confusion, create a different subdirectory for the frontend, e.g.
[rlxhost] mkdir $HOME/online/fe_32
Copy the Makefile and frontend code from the same example to this directory.
Modify the Makefile as needed, and only build the frontend.

Once built, start the frontend in an xterm on the REMOTE host.
[rlxhost] $HOME/online/fe_32/frontend
Check that it connects successfully to the experiment and the Equipments appear on the [[Status Page]] as described under [[#Frontend and Analyzer (localhost)]]. The Equipment Status (Figure 2) should show that the Equipments are now running on the remote cpu, i.e. "SampleFrontend@rlxhost.triumf.ca" rather than "Sample Frontend@localhost".

=== Run the Analyzer on Experiment host ===
Start the analyzer in an xterm on the MIDAS ''experiment host''
[johnfoo@mhostpc e777] $HOME/online/analyzer

Continue by starting a run and dumping the data as described under [[#Frontend and Analyzer (localhost)]].

=== Create experiment startup script ===
It is useful to create a script that will automatically start all the required clients for the experiment. This is called a startup script (any name can be used). Create {{Filepath|path=$HOME/online/bin/start_daq.sh}} on the ''experiment host'' as follows, (supply the [[mhttpd]] ports if default is not used) :

#!/bin/sh
# start_daq.sh
cd $HOME/online
#
switch (`hostname`)
case $MIDAS_HOST*: ## MIDAS_HOST is assigned in .cshrc (.bashrc)
echo "Good, we are on $MIDAS_HOST"
breaksw
case $REMOTE_HOST*: ## REMOTE_HOST is assigned in .cshrc (.bashrc)
echo "startup script should be executed on $MIDAS_HOST"
endsw
#
odbedit -c clean
# start mhttpd on default port (Mongoose https version)
mhttpd -D # you can optionally restrict access to localhost and/or other specified hosts - see [[mhttpd]]
# start mserver on default port (use argument -p to use a different port)
mserver -D # start mserver on default port (1175) or a different port using -p option - see [[mserver]]
# the port must match that of MIDAS_SERVER_HOST in .cshrc (.bashrc) - see above
# access must be specifically allowed - see [[#Grant Remote host access permission||above]]
xterm -e ./analyzer &
## start frontend on remote host
#ssh $REMOTE_HOST $HOME/online/bin/start_frontend -O >& $HOME/online/bin/start_frontend.log &
mlogger -D
#end file

Note that command to start the frontend on the remote host is commented out. It can be uncommented once the start_frontend script is written and tested.

Shut down any clients that are running already by running odbedit on the MIDAS ''experiment host'' and execute the new startup script {{File|name=start_daq.sh}} (make it executable if desired)

[mhostpc] odbedit
[local:e777:S]/>sh all
[mhostpc] ./start_daq.sh 
and check that {{Utility|name=odbedit}} still works on the ''REMOTE host''

[rlxhost] odbedit
[mhostpc.triumf.ca:1175:e777:Stopped]/>ls 

You may wish to modify the script to restart missing clients only.

=== Create script to start frontend ===
The startup file {{File|name=start_daq.sh}} invoked a script {{File|name=start_frontend}} to start the remote frontend. This line was commented out (see [[#Create experiment startup script|above]]). Once the frontend is working, create a script to start the frontend (e.g. file {{Filepath|path=$HOME/online/bin/start_frontend}}).

#!/bin/tcsh
# Script to start frontend running on $REMOTE_HOST
#
# This script runs on the REMOTE HOST
#
echo "starting frontend for experiment $MIDAS_EXPT_NAME "
xterm -geometry 150x50+800+0 -fg white -bg blue -title "Frontend" -e "$HOME/online/fe_32/frontend" &
echo "Frontend task has been started"
exit

Check that this script works by running it on the REMOTE host. If successful,
run the commented command in an xterm on the ''experiment host''
[mhostpc] ssh $REMOTE_HOST $HOME/online/bin/start_frontend -O >& $HOME/online/bin/start_frontend.log & 
This command relies on password-less access set up with SSH keys (see [[#Experimental setup (Remote)]]).
If successful, uncomment line in {{File|name=start_daq.sh}}, and run the script to check that it can start the remote client(s) from the MIDAS ''experiment host'' (first shutting down all running clients) :

[mhostpc] odbedit -c "sh all"
[mhostpc] ./start_daq.sh 

Next [[#Customizing your experiment|customize your experiment]].

== Customizing your experiment ==

Here are some of the more common options provided for users to customize their experiments according to their own preferences:

; Write frontend(s) to read out and control your hardware
: Modify the example frontend and Makefile (from $MIDASSYS/examples/) you have already copied and built so it accesses your hardware.

: The example Frontend.c is an example of [[Frontend user code]]. You can find documentation about the frontend structure under [[Frontend Application]], [[Frontend Operation]] and [[Frontend user code]]. You may need to set up some [[Event Notification (Hot-Link)|Hot-Links]] in the frontend. The MIDAS package includes device drivers for some of the more common hardware. See [[MIDAS Driver Library]] for more information.

; Modify analyzer(s) for your data
: Modify the example analyzer code as necessary to analyze the data from your frontends.

; Slow Controls
: Control your slow-controls equipment with slow-control frontend(s) optionally using the [[#MIDAS Package Installation|MSCB]] system.

; Data logger
: Set up the data logger [[mlogger]] to log data to a storage device. Many options are available - see [[Data Logger]], [[mlogger]] and [[/Logger ODB tree]] for instructions.

; Allow programs to be restarted from the [[Programs Page]]
: See mhttpd [[Programs Page]] and [[/Programs ODB tree]]

; Write scripts which run at start and end of run
: See [[/Programs ODB tree#Execute on start run]] and [[/Programs ODB tree#Execute on stop run]]

; Create a shutdown script
: Create a script similar to the startup script to shutdown the experiment

; Lazy logger
: Set up the [[lazylogger]] for archiving data files for storage.

; Create edit-on-start parameters
: See [[Edit-on-start Parameters]]

; Set up the alarm system
: See [[Alarm System]] and [[Alarms Page]]

; Set up history system
: See [[History System]] and [[History Page]]

; Set up the Electronic logbook (Elog)
: See [[Elog]] for details

; Set up the Sequencer
: The [[Sequencer Page#Sequencer]] can be used to run a sequence of runs, changing parameters as needed.

; Write a webserver Custom page
: To better control and monitor your experiment, you may want to write a [[Custom Page]].

== Start MIDAS on boot ==

To start MIDAS mhttpd, mlogger and elogd at boot time, add following cron entries:
<pre>
[iris@iris00 ~]$ crontab -l
@reboot /home/iris/packages/midas/bin/mhttpd -D
@reboot /home/iris/packages/midas/bin/mlogger -D
@reboot /home/iris/packages/elog/elogd -c /home/iris/packages/elog/elogd.cfgiris -D
[iris@iris00 ~]$
</pre>

[[Category:Installation]]

Python

2026-09-29T16:14:53Z

Dfujimoto:

Since December 2019, midas has shipped with python wrappers allowing midas clients to be written in python.

The main features are:
* A file reader, which works with python 2.7 and python 3.x
* A client library and frontend framework which work with python 3.x only

To use the python libraries, you must have installed midas, and compiled midas using cmake. The python code is found at <code>$MIDASSYS/python</code>. You can view [https://gitlab.triumf.ca/midas/midas/-/tree/develop/python/examples example scripts and frontends on GitLab].

== Midas file reader ==

<code>midas/file_reader.py</code> is a pure-python tool that lets you read midas files.
It presents a very pythonic interface, allowing you to do things like

import midas.file_reader

my_file = midas.file_reader.MidasFile("/path/to/file.mid.lz4")

for event in my_file:
event.dump()

To access information in the ODB dump, you can do something like

import midas.file_reader

my_file = midas.file_reader.MidasFile("/path/to/file.mid.lz4")
odb = my_file.get_bor_odb_dump().data

run_number = odb["Runinfo"]["Run number"]

For more details, see the documentation in <code>$MIDASSYS/python/midas/file_reader.py</code> and example code in <code>$MIDASSYS/python/examples/file_reader.py</code>.

* It works with python 2.7 and python 3.
* It does not require the rest of midas to be compiled.

== Client and frontend libraries ==

* <code>midas/client.py</code> provides pythonic access to a midas experiment, by wrapping midas' C library. It provides nice ODB access through functions like <code>odb_set</code> and <code>odb_get</code>, and can also handle event buffers, callback functions and more.
* <code>midas/frontend.py</code> builds upon the client and provides a framework for writing midas frontends in python. It supports periodic and polled equipment, and can be controlled just like a C frontend.

Many users write scripts in python or bash that call the <code>odbedit</code> command-line tool or the mhttpd web server. We hope that the pythonic midas client tools will simplify and robustify such scripts. The python frontend framework may prove particularly beneficial for controlling devices that natively talk in JSON or other text-based protocols that are
tedious to deal with in C. It may also reduce the development time required to write frontends that do not have strict performance requirements.

* They work with python 3 only.
* They require the rest of midas to be compiled, as they use the midas C library. For the correct libraries to be built, you must compile using CMake
* They are designed for ease of use, rather than optimal performance. If performance is critical for your application, you should write your code in C.

The python <code>midas.client</code> library provides support for:
* ODB access - creating/deleting/renaming keys, getting/setting/watching values, creating links
* run transitions - starting/stopping/pausing/resuming runs, and registering callback functions for each transition
* event buffers - connecting, sending events, receiving events
* remote procedure calls - connecting and talking to other midas clients
* alarms - creating and triggering alarms
* messages - sending and getting recent messages
* history system - reading data (since July 2023)

== Documentation ==

Documentation of the tools is written as docstrings in the source code (in <code>$MIDASSYS/python/midas</code>). We do not currently have a web version of the documentation.

The documentation assumes a reasonable familiarity with core midas concepts. See the rest of this wiki if you're unfamiliar with a term or concept in the python documentation.

== Installation ==

We highly recommend using [https://pypi.org/project/pip/ pip] and/or [https://virtualenv.pypa.io/en/latest/ virtualenv] to manage your python packages, but you can also just edit your paths if desired.

To install this package with pip:

pip install -e $MIDASSYS/python --user
# The -e flag makes this an "editable" install. If you upgrade midas in future,
# python will automatically see the new code without you having to re-install.
# The --user flag may or may not be needed, depending on your pip setup.

The python code has some optional dependencies that enable extra functionality:

* <code>numpy</code> can be used to improve performance in midas python frontends that are running at high data rates
* <code>lz4</code> allows the midas file reader to parse .mid.lz4 files (.mid and .mid.gz files do not need any extra pacakges)
* <code>pyflakes</code> allows the PySequencer to validate the syntax of your script before it gets executed, catching many errors before they are encountered

As of September 2026, the optional dependencies will be installed automatically when you "pip install" the midas package.

If you don't want to use pip, a hacky way to "install" is by editing environment variables. This will not install any of the optional dependencies.

export PYTHONPATH=$PYTHONPATH:$MIDASSYS/python

== Examples ==

You can now use <code>import midas</code> and <code>import midas.client</code> etc in your python scripts.

[https://gitlab.triumf.ca/midas/midas/-/tree/develop/python/examples $MIDASSYS/python/examples] contains several examples of complete clients and frontends written in python.

=== midas.client ODB access ===

Here we show a very simple python script for interacting with the ODB. Note that we do not have any error-checking. If any call to the midas library fails, an exception will be raised.

<nowiki>import midas.client

"""
A simple example program that connects to a midas experiment,
reads an ODB value, then sets an ODB value.

Expected output is:

```
The experiment is currently stopped
The new value of /pyexample/eg_float is 5.670000
```
"""

if __name__ == "__main__":
client = midas.client.MidasClient("pytest")

# Read a value from the ODB. The return value is a normal python
# type (an int in this case, but could also be a float, string, bool,
# list or dict).
state = client.odb_get("/Runinfo/State")

if state == midas.STATE_RUNNING:
print("The experiment is currently running")
elif state == midas.STATE_PAUSED:
print("The experiment is currently paused")
elif state == midas.STATE_STOPPED:
print("The experiment is currently stopped")
else:
print("The experiment is in an unexpected run state")

# Update or create a directory in the ODB by passing a dict to `odb_set`
client.odb_set("/pyexample", {"an_int": 1, "eg_float": 4.56})

# Update a single value in the ODB
client.odb_set("/pyexample/eg_float", 5.67)

# Read the value back
readback = client.odb_get("/pyexample/eg_float")

print("The new value of /pyexample/eg_float is %f" % readback)

# Delete the temporary directory we created
client.odb_delete("/pyexample")
</nowiki>

=== midas.frontend ===

Example of a basic midas frontend that has one periodic equipment. See <code>$MIDASSYS/examples/multi_frontend.py</code> for an example that uses more features (frontend index, polled equipment, ODB settings etc).

<nowiki>import midas
import midas.frontend
import midas.event

class MyPeriodicEquipment(midas.frontend.EquipmentBase):
"""
We define an "equipment" for each logically distinct task that this frontend
performs. For example, you may have one equipment for reading data from a
device and sending it to a midas buffer, and another equipment that updates
summary statistics every 10s.

Each equipment class you define should inherit from
`midas.frontend.EquipmentBase`, and should define a `readout_func` function.
If you're creating a "polled" equipment (rather than a periodic one), you
should also define a `poll_func` function in addition to `readout_func`.
"""
def __init__(self, client):
# The name of our equipment. This name will be used on the midas status
# page, and our info will appear in /Equipment/MyPeriodicEquipment in
# the ODB.
equip_name = "MyPeriodicEquipment"

# Define the "common" settings of a frontend. These will appear in
# /Equipment/MyPeriodicEquipment/Common. The values you set here are
# only used the very first time this frontend/equipment runs; after
# that the ODB settings are used.
default_common = midas.frontend.InitialEquipmentCommon()
default_common.equip_type = midas.EQ_PERIODIC
default_common.buffer_name = "SYSTEM"
default_common.trigger_mask = 0
default_common.event_id = 1
default_common.period_ms = 100
default_common.read_when = midas.RO_RUNNING
default_common.log_history = 1

# You MUST call midas.frontend.EquipmentBase.__init__ in your equipment's __init__ method!
midas.frontend.EquipmentBase.__init__(self, client, equip_name, default_common)

# You can set the status of the equipment (appears in the midas status page)
self.set_status("Initialized")

def readout_func(self):
"""
For a periodic equipment, this function will be called periodically
(every 100ms in this case). It should return either a `midas.event.Event`
or None (if we shouldn't write an event).
"""

# In this example, we just make a simple event with one bank.
event = midas.event.Event()

# Create a bank (called "MYBK") which in this case will store 8 ints.
# data can be a list, a tuple or a numpy array.
data = [1,2,3,4,5,6,7,8]
event.create_bank("MYBK", midas.TID_INT, data)

return event

class MyFrontend(midas.frontend.FrontendBase):
"""
A frontend contains a collection of equipment.
You can access self.client to access the ODB etc (see `midas.client.MidasClient`).
"""
def __init__(self):
# You must call __init__ from the base class.
midas.frontend.FrontendBase.__init__(self, "myfe_name")

# You can add equipment at any time before you call `run()`, but doing
# it in __init__() seems logical.
self.add_equipment(MyPeriodicEquipment(self.client))

def begin_of_run(self, run_number):
"""
This function will be called at the beginning of the run.
You don't have to define it, but you probably should.
You can access individual equipment classes through the `self.equipment`
dict if needed.
"""
self.set_all_equipment_status("Running", "greenLight")
self.client.msg("Frontend has seen start of run number %d" % run_number)
return midas.status_codes["SUCCESS"]

def end_of_run(self, run_number):
self.set_all_equipment_status("Finished", "greenLight")
self.client.msg("Frontend has seen end of run number %d" % run_number)
return midas.status_codes["SUCCESS"]

def frontend_exit(self):
"""
Most people won't need to define this function, but you can use
it for final cleanup if needed.
"""
print("Goodbye from user code!")

if __name__ == "__main__":
# The main executable is very simple - just create the frontend object,
# and call run() on it.
with MyFrontend() as my_fe:
my_fe.run()
</nowiki>

=== Doing work when the program exits ===

Since April 2023, the python frontend framework has included the ability to ensure that work gets performed when the program exits (more specifically, just before we disconnect from midas). This may be useful, for example, to reset the state of some hardware before the program exits.

In the frontend framework, this is done by implementing the <code>frontend_exit()</code> function in your frontend class (see the example code above). If you're using a bare <code>MidasClient</code> you can instead call <code>register_disconnect_callback()</code>.

To be certain that your function gets called, you should use the "context manager" style to create your client/frontend object.

# Context manager style - guaranteed that `frontend_exit()` will be called, even
# if there's an uncaught exception or exit(0) is called etc.
with MyFrontend() as my_fe:
my_fe.run()

# Not using a context manager - `frontend_exit()` may NOT be called!
my_fe = MyFrontend()
my_fe.run()

The reason for this is that the best we can do in the latter approach is to hook up the midas disconnection logic to the destructor of the <code>MidasClient</code> object. But python does not guarantee that an object's <code>__del__()</code> function will be called in a timely manner (or even at all!). It depends on your python version, python implementation, and if you do any weird things in your class (e.g. circular references).

In the context manager approach, we can hook up the midas disconnection logic to the <code>__exit__()</code> function, which python DOES guarantee will be called.

== Tests ==

<code>$MIDASSYS/python/tests</code> contains a variety of test scripts. If you wish, you can run them using

cd $MIDASSYS/python/tests
python -munittest

However, note that running these tests will interact with whichever midas experiment is currently active. They will edit the ODB and start and stop runs. If you are running these tests, we suggest doing so in a "dummy"/test experiment.

Setup MIDAS experiment at TRIUMF

2026-09-29T16:13:18Z

Dfujimoto: /* Install standalone elog */

{{Pagelinks}}
== Introduction ==
This page describes setting up a MIDAS experiment at TRIUMF. This information can be adapted for other sites.

== Environment variables ==
* '''MIDASSYS''' Base directory of the MIDAS package, midas and mxml should be at the same level.
* '''MIDAS_EXPTAB''' Experiment definition file
* '''MIDAS_SERVER_HOST''' MIDAS host server name for remote midas connections.
* '''MIDAS_EXPT_NAME''' Experiment name

== Standard layout of MIDAS experiment ==
The following shows the directory layout of a standard MIDAS experiment:

/home/exptuser/
packages/
root <---- ROOT
mxml
mscb
midas/ <---- MIDAS
linux/{lib,bin} <---- binaries matching the selected 64-bit/32-bit flavour of ROOT
linux-m32/{lib,bin} <---- limited function 32-bit binaries for 32-bit frontend machines, build by "make linux32"
linux-m64/{lib,bin} <---- limited function 64-bit binaries (only needed if ROOT and linux/bin are 32-bit)
linux-arm/{lib,bin} <---- full function ARM cross-compiled using "make linuxarm"
linux-crosscompile/{lib,bin} <---- cross-compiled limited function binaries for PPC and ARM frontends (see Makefile)
rootana <---- ROOT analyzer
roody <---- graphical online histogram viewer for MIDAS and ROOTANA
online/
exptab <---- experiment definition
{.ODB,.SYSTEM,.SYSMSG,etc}.SHM <---- MIDAS shared memory save files
src <---- experiment frontend sources
bin,scripts
elog <---- MIDAS elog
history <---- MIDAS history
data -> /data/exptname/current <---- symlink to the data directory
/data/exptname/current <---- experiment data directory with ODB save files and MIDAS .mid/.mid.gz data files

== Prepare computers ==

On some operating systems, several MIDAS functions require administrator access:

* on el7 linux (SL7/CC7/CentOS7/RHEL7) - access to mhttpd port 8443 requires special firewall rules, see here: https://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29
* on el7 linux - access to mserver to run frontends and other programs on some other computer requires special firewall rules, see here: https://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29
* on el7 linux - on the frontend machines (and other machines that will connect to the mserver, the same firewall rule needs to be created (use the IP address of the machine running the mserver)

== Prepare the user account ==
<div id="NOTES"></div>

* Setup the user account for running this instance of midas. For machines part of the LADD cluster, follow these [http://daq-plone.triumf.ca/SM/docs/local/NewLaddUser] instructions.
* check that the account is using the /bin/bash shell
* make $HOME/.profile look like this:

<pre>
#!/bin/echo You must source

export SVN_EDITOR="emacs -nw"
export GIT_EDITOR="emacs -nw"
export MIDASSYS=$HOME/packages/midas
export ROOTANASYS=$HOME/packages/rootana
export MIDAS_EXPTAB=$HOME/online/exptab
#
# setup the MIDAS mserver
#
case `hostname` in
daq07*)
unset MIDAS_SERVER_HOST
;;
*)
export MIDAS_SERVER_HOST=daq07.triumf.ca:7070
;;
esac
#
# select 64-bit or 32-bit MIDAS and ROOT
#
case `uname -i` in
i386)
source /daq/daqshare/olchansk/root/root_v5.34.01_SL62_32/bin/thisroot.sh
export PATH=.:$MIDASSYS/linux-m32/bin:$PATH
;;
*)
#source /daq/daqshare/olchansk/root/root_v5.34.34_SL67_64/bin/thisroot.sh
source $HOME/packages/root/bin/thisroot.sh
export PATH=.:$MIDASSYS/linux/bin:$PATH
;;
esac
#
export PATH=.:$HOME/online/bin:$HOME/packages/roody/bin:$PATH
#
#end
</pre>

* mkdir $HOME/packages
* Logout and login again, for .cshrc changes to take effect

== Install ROOT ==

* Identify the Linux version: RH9 (Red Hat Linux 9), FC3 (Fedora Core 3), RHEL4/SL4 (Red Hat Enterprise LInux 4/Scientific Linux 4), SL5x, SL6x, (CentOS/CC/SL) el7x: more /etc/redhat-release
* Decide to use 32-bit or 64-bit ROOT ('uname -a')
* cd $HOME/packages
* ls -l /daq/daqshare/olchansk/root/ ### to see all available ROOT packages
* ln -s /daq/daqshare/olchansk/root/root_vNNN_VVV_BB root, where NNN is the latest available version of ROOT ("ls -l /daq/daqshare/olchansk/root"), VVV is the Linux version code (RH9, FC3, SL4, etc) and BB is "_32" or "_64" for 32-bit or 64-bit ROOT. For example: /daq/daqshare/olchansk/root/root_v5.10.00_SL40
* for example: ln -s /daq/daqshare/olchansk/root/root_v5.34.34_el72_64 $HOME/packages/root
* Check that ROOT works: "source $HOME/packages/root/bin/thisroot.sh; root". Type ".q" to exit root.

== Install MIDAS ==

* cd $HOME/packages
* (OBSOLETE) svn co svn+ssh://svn@savannah.psi.ch/repos/meg/midas/trunk midas, password "svn". (password has to be entered twice)
* (OBSOLETE) svn co svn+ssh://svn@savannah.psi.ch/repos/meg/mxml/trunk mxml
* git clone https://gitlab.triumf.ca/midas/midas.git --recursive
* (OBSOLETE) git clone https://gitlab.triumf.ca/midas/mxml.git
* (OBSOLETE) git clone https://gitlab.triumf.ca/midas/mscb.git
* (IF GITLAB IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/mxml.git
* (IF GITLAB IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/mscb.git
* (IF GITLAB IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/midas.git
* cd midas
* make
* (only if needed) make linux32 ### build the 32-bit MIDAS libraries
* ls -l linux/bin/odbedit ### check that odbedit has been created (do not run it yet)

You can see a list of other installation problems at [[Common problems & Debugging recipes]].

;NOTE 1
: Optional features in MIDAS can be explicitly disabled if desired when making MIDAS using the NO_xxx feature (NO_ROOT,NO_MYSQL,NO_ODBC,NO_SQLITE,NO_MSCB), e.g. "make NO_ROOT=1" to disable ROOT. These NO_xxx Makefile variables are only used to control autodetection.

;NOTE 2
: Since June 2019 the mxml and mscb packages are submodules of the midas package, so no need to clone them separately. If you have an existing clone of midas but not yet the submodules, you need

$ git submodule update --init --recursive

: To update both midas and the submodules, you need

$ git pull --recurse-submodules

== Install ROOTANA ==

* cd $HOME/packages
* git clone https://gitlab.triumf.ca/midas/rootana.git
* cd rootana
* make

== Install ROODY ==

* cd $HOME/packages
* git clone https://gitlab.triumf.ca/midas/roody.git
* cd roody
* make
* $HOME/packages/roody/bin/roody, run the program

== Install additional additional packages ==

* cd $HOME/packages
* git clone https://bitbucket.org/ttriumfdaq/vme
* git clone https://bitbucket.org/ttriumfdaq/frontends

== Build special versions of MIDAS ==

Build special versions of MIDAS for the case when some MIDAS programs, such as VME frontends, will run on a different computer that may have a different flavour of operating system, i.e. 32-bit linux or an older version of Scientific Linux.

* login to the computer where the frontends will run and:
* if it is a 32-bit linux: cd $HOME/packages/midas; make linux32
* if it is a 64-bit linux: cd $HOME/packages/midas; make linux64

* login to the host computer to cross-compile ARM code:
* if it is an ARM linux: cd $HOME/packages/midas; make linuxarm # may need to install ARM cross compilers

== Prepare VME hardware ==

Hardware check list:
* VME crate
* VME processor (supported are V77xx, V7805, V7865)
* On all VME modules, set the VME address jumpers as described here: http://daq-plone.triumf.ca/SM/docs/local/vme_jumpers
* run vmescan to confirm correct VME addresses
** cd $HOME/packages
** svn checkout https://ladd00.triumf.ca/svn/daqsvn/trunk/vme
** cd vme
** make
** ./vmescan.exe (or _gef.exe, depending on the VME driver in use)

== Install Universe-II VME driver (V7648, V7750, V7805, V7851) ==

* login as root (ssh root@localhost)
* follow instructions: https://www.triumf.info/wiki/DAQwiki/index.php/VME-CPU#V7648.2C_V7750.2C_V7805.2C_V7851_:_Setup_vme_universe_VME_drivers
* cd ~/packages/vme; vmescan.exe

== Install Tsi-148 VME driver (V7865) ==

* login as root (ssh root@localhost)
* follow instructions: https://www.triumf.info/wiki/DAQwiki/index.php/VME-CPU#V7865_and_XVB-602_:_Setup_gefvme.2Ftsi148_VME_drivers
* cd ~/packages/vme; vmescan_gef.exe

== Setup the experiment environment ==

* Decide which computer will host MIDAS (where MIDAS shared memory buffers will reside).
: This computer will run the [[mserver]], [[mlogger]] and [[mhttpd]] applications. (It is usually the machine where the MIDAS,ROOT etc. packages have been downloaded). It will be referred to as the host machine (localhost).

The environment is slightly different depending on whether all programs run on the host machine, or whether some programs run on remote host(s) :
=== ALL programs run on localhost ===
:If all programs run on the host machine (localhost), it is not necessary to run [[mserver]]. [[Environment Variables#MIDAS_SERVER_HOST|MIDAS_SERVER_HOST]] will not be assigned (see example .cshrc [[#Prepare the user account|above]]).

=== Some programs run on REMOTE host(s) ===
: '''IMPORTANT:'''
# Since August 2015 '''you must explicitly allow access for clients running on remote machines'''. To do this, follow the '''[[Security#MIDAS programs on remote machines|instructions here]]'''.
# The example code .cshrc ([[#Prepare the user account|see above]]) should be present on both host and remote machine(s). This will ensure that [[Environment Variables#MIDAS_SERVER_HOST|MIDAS_SERVER_HOST]] will NOT be set for the host machine (localhost), but on a remote machine, MIDAS_SERVER_HOST will be set to the MIDAS host machine.
# The client [[mserver]] must be started on the MIDAS host machine. Note that multiple experiments can run on the same host machine by starting several instances of [[mserver]] (one for each experiment) running with different ports (and .cshrc would be edited so that MIDAS_SERVER_HOST is set to the appropriate port for the experiment).

 
On the host machine:
* mkdir $HOME/online
* cd $HOME/online
* create directories for local programs, sources, elog and history: mkdir bin src elog history
* create data directory: mkdir -p /ladd/data1/t2kvme5/data; ln -s /ladd/data1/t2kvme5/data $HOME/online
* create the exptab file "$HOME/online/exptab" following the example below. The first entry (exptname) is the name if the DAQ system (MIDAS experiment name), the second entry (/home/USER/online) is the location of MIDAS shared memory buffers (by convention, $HOME/online), the third entry (kopio03) is your username.
<pre>exptname /home/kopio03/online kopio03</pre>
* logout and login again for all changes to take effect

== Setup experiment startup scripts ==
* login to the experiment host computer
* echo $MIDAS_SERVER_HOST ### to check correct value - should be blank
* create $HOME/online/bin/start_daq.sh, replacing XXX with the hostname of the machine running the experiment (and changing the mserver and mhttpd ports, as needed).

#!/bin/sh
# start_daq.sh
cd $HOME/online
#
case `hostname` in XXX*)
echo "Good, we are on XXX!"
;;
*)
echo "The start_daq script should be executed on XXX"
exit 1
;;
esac
#
odbedit -c clean
# start [[mhttpd]] on default port. (Mongoose https version - see [[mhttpd]] for other options)
mhttpd -D -a localhost -a XXX.triumf.ca # optionally restrict access to specified hosts
#
# start [[mserver]] on default port (use argument -p to use a different port)
mserver -D # access must now be specifically allowed - see [[#Setup the experiment environment|above]]

# OR ([[#NOTES|older MIDAS versions]])
# mhttpd -p 8081 -D -a localhost -a XXX.triumf.ca # optionally restrict access to specified hosts
# mserver -p 7071 -D -a localhost -a lxdragon01.triumf.ca -a lxdragon02.triumf.ca -a XXX.triumf.ca # optionally restrict access to specified hosts

#
mlogger -D
#end file

== Run the MIDAS Web Server ==
Let's start the MIDAS webserver for the first time:

Start [[mhttpd]] on the ''experiment host'' (localhost) like this:
[mhostpc] mhttpd
You will get the following messages:
[mhttpd,INFO] ODB subtree /Runinfo corrected successfully
Mongoose web server will listen on ports "8080r,8443s"
[mhttpd,ERROR] [mhttpd.cxx:17892:mongoose,ERROR] cannot find SSL certificate file "/home/agdaq/online/ssl_cert.pem"
[mhttpd,ERROR] [mhttpd.cxx:17893:mongoose,ERROR] please create SSL certificate file: openssl req -new -nodes -newkey rsa:2048 -sha256 -out ssl_cert.csr -keyout ssl_cert.key; openssl x509 -req -days 365 -sha256 -in ssl_cert.csr -signkey ssl_cert.key -out ssl_cert.pem; cat ssl_cert.key >> ssl_cert.pem
could not start the mongoose web server, see messages and midas.log, bye!

Create a self-signed certificate suitable for initial testing by executing the command printed by mhttpd:


[mhostpc] openssl req -new -nodes -newkey rsa:2048 -sha256 -out ssl_cert.csr -keyout ssl_cert.key; openssl x509 -req -days 365 -sha256 -in ssl_cert.csr -signkey ssl_cert.key -out ssl_cert.pem; cat ssl_cert.key >> ssl_cert.pem

For production use, you should create a properly signed certificate, see [[Mhttpd#Create an SSL certificate|create your own SSL certificate]] or you should run mhttpd behind an SSL proxy.

Run mhttpd again.

You will get the following messages:
[mhttpd,INFO] ODB subtree /Runinfo corrected successfully
Mongoose web server will listen on ports "8080r,8443s"
Mongoose web server will use SSL certificate file "/home/johnfoo/packages/midas/ssl_cert.pem"
[mhttpd,ERROR] [mhttpd.cxx:17633:mongoose,ERROR] mongoose web server cannot find password file "/home/johnfoo/online/htpasswd.txt"
[mhttpd,ERROR] [mhttpd.cxx:17634:mongoose,ERROR] please create password file: htdigest -c /home/johnfoo/online/htpasswd.txt Default midas
could not start the mongoose web server, see messages and midas.log, bye!

Create the password file by following the instructions printed by mhttpd. The http digest domain name is the experiment name, suggested default user name is "midas". You will be asked to type in a password


[mhostpc] htdigest -c /home/johnfoo/online/htpasswd.txt exptname midas
Adding password for midas in realm exptname.
New password:
Re-type new password:

It is a good idea to set the password file {{Filepath|path=htpasswd.txt}} readable and writable by owner only.

Now restart {{Utility|name=mhttpd}}
[mhostpc] mhttpd
Mongoose web server will listen on ports "8080r,8443s" **see note
Mongoose web server will use SSL certificate file "/home/suz/packages/midas/ssl_cert.pem"
Mongoose web server will use authentication realm "Default", password file "./htpasswd.txt"

Now point a web browser running on the same host computer (localhost) to https://localhost:8443
If the web browser is running on a different computer, go to URL of the form

https://mhostpc.triumf.ca:8443 (substitute your host machine name and domain for "mhostpc.triumf.ca")

If you are using the default SSL certificate you will probably get a message: "This Connection is Untrusted". Click "I understand the risks" and add an exception. This is because the test certificate is self-signed. Then confirm an exception.

If instead you get a "connection refused" error, the midas host pc may have the firewall enabled. To make a firewall exception for MIDAS, follow instructions here http://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29

You should then see an authentication box asking you for the user name and password. The user name is "midas". Enter the password you just created. The Midas [[Status Page]] should appear with multiple buttons for run control as well as equipment listing (no equipments will be listed as yet) and application listings. Please refer to [[mhttpd]] (the MIDAS Web-based Run Control utility) for further information. You can start and stop runs from the main status page, and use the [[ODB Page]] to access the database (ODB).

; Note
: Default ports of 8080 and 8443 are used by [[mhttpd]]. If these ports are in use on your machine, start mhttpd with alternative ports, e.g.
[mhostpc] mhttpd --https 8448 --http 8089
: or see [[Mhttpd#Usage]] to change the default ports.

== Setup experiment database (ODB) ==

* run $HOME/online/bin/start_daq.sh

* odbedit, run these commands: (replace user names and directory names)
<pre>
# set "/Logger/Message file" "/home/kopio03/online/midas.log" # obsolete
set "/Logger/Data Dir" "/home/kopio03/online/data"
set "/Logger/History dir" "/home/kopio03/online/history"
create STRING "/Logger/Elog dir"
set "/Logger/Elog dir" "/home/kopio03/online/elog"
set "/Logger/ODB dump file" "/home/kopio03/online/history/run%05d.xml"
set "/Logger/ODB dump" "y"
set "/Logger/Channels/0/Settings/Filename" "run%05dsub%03d.mid.gz"
set "/Logger/Channels/0/Settings/Subrun byte limit" "1000000000"
set "/Logger/Channels/0/Settings/Compression" 1
set "/Logger/Channels/0/Settings/ODB Dump" "y"
set "/Programs/Logger/Required" y
set "/Programs/Logger/Start command" "mlogger -D"
set "/Programs/fevme/Required" "y"
set "/Programs/fevme/Start command" "ssh -n lxdaq09 $HOME/online/src/fevme_gef.exe -O"
exit
</pre>
* open web browser e.g. firefox.
* go to the midas status page at https://localhost:8443 (default port).
** if running [[mhttpd]] with Mongoose HTTPS/OpenSSL (the default) for the first time, you will need to create a password file. Follow the instructions (see [[mhttpd#HTTPS/SSL server (Mongoose)]] for details).
** For other options (i.e. HTTPS/SSL proxy) see [[#Secure MIDAS and ELOG Web access]]
*OR open the midas status page at http://localhost:8081 ([[#NOTES|older MIDAS versions]])
* midas status page will show most stuff "red" as nothing is running yet
* DON'T DO THIS YET run ./fevme.exe (on the computer with the VME interface, could be different from computer hosting the experiment), observe that corresponding equipments have been created
* save the url bookmark to the "personal toolbar"
* go to the Programs page, stop mlogger, stop fevme, start mlogger, start fevme
* go to the Status page, start run, stop run
* go back to the Status page, everything should be green
* start a run
* send signals to the ADC gate
* you should be getting events
* to look at data, proceed with setting up the [[ROOTANA|ROOT Analyzer]].

== Start DAQ programs at boot time ==

* add this to /etc/rc.local (replace username and location of the start_daq script)
<pre>
su - alpha -c /home/alpha/online/bin/start_daq.sh
</pre>

== Setup local software version control ==

Version control for experiment source code is setup using "git" (http://git-scm.com/)

* cd $HOME/online
* git init
* git add exptab
* git add bin/start_daq.sh
* git add .gitignore ### contents can be
<pre>
*~
*.o
*.exe
</pre>
* git add src/Makefile src/*.cxx ...
* git commit -a

== Adjust MIDAS buffer sizes ==

Default MIDAS SYSTEM buffer size is 8 Mbytes, fairly small for high-data-rate experiments. The rule of thumb is to have at least a few seconds worth of buffer space available. For example, if event size is 10 Kbytes and the event rate is 1 kHz, data rate is 10*10^3*1*10^3 = 10 Mbytes/sec. To buffer 10 seconds of data we need 100 Mbytes of buffer space.

To resize the MIDAS event buffers (SYSTEM, etc) do this:
* stop all frontends, stop mlogger
* start odbedit:
** cd "/Experiment/Buffer sizes"
** set SYSTEM 100000000
* run "mdump -z SYSTEM"
* if mdump complains about the size of .SYSTEM.SHM, remove it, try again.
* ls -l /dev/shm ### to observe that the size of shared memory is correct

== Secure MIDAS and ELOG Web access ==
In versions prior to May 2015, the default web access to MIDAS and ELOG uses the "http:" protocol which is insecure. In this case, all information is transmitted as clear text meaning that secret, confidential and sensitive information (such as the MIDAS and ELOG passwords and usernames) can be stolen "easily". This means that even "password protected" MIDAS and ELOG pages are not really protected if accessed using the "http" method.

Better security for HTTP is gained by using a password protected '''SSL (https) proxy'''. (It does not provide absolute security because of remaining problems with the security of SSL certificates, security of passwords, etc). Setting up an SSL (https) proxy is described [[#Setting up an HTTP proxy|below]].

Since May 2015, an ''alternative secure option'' to setting up an HTTP proxy is available to users of MIDAS. Recent versions of elogd (ELOG) do support SSL https:// connections, and [[#mhttpd with HTTPS/SSL server (Mongoose)]] is now available. This option is the default, and provides a similar level of security to an HTTP proxy.

See [[Security#Web Access]] for a comparison of these two secure options.

=== mhttpd with HTTPS/SSL server (Mongoose) ===

Since May 2015 the MIDAS web server [[mhttpd]] is explicitly linked with OpenSSL to provide secure HTTPS connections via the [https://bitbucket.org/tmidas/midas/src/ecb9a8537448a8a43f7f9a2bfdb82e578208cde3/doc/mongoose/?at=develop Mongoose] web server (see [[mhttpd]]). With this version, default web access to MIDAS uses the "https" protocol. Web access to {{Utility|name=mhttpd}} can be restricted by using the "-a hostname" switch of [[mhttpd]]. The first time {{Utility|name=mhttpd}} is run, a password file must be created. An SSL certificate is also required. See [[mhttpd#HTTPS/SSL server (Mongoose)|HTTPS/SSL server (Mongoose)]] for instructions.

=== mhttpd using an HTTPS/SSL proxy ===

THESE INSTRUCTIONS ARE WRONG, DO NOT DO THIS. See instead [https://daq00.triumf.ca/MidasWiki/index.php/Setup_MIDAS_experiment_at_TRIUMF#Install_https_proxy this section].

An [[#Setting up an HTTP proxy|HTTP proxy]] must be set up. This is the only way of securing older version of [[mhttpd]] (pre August 2015). Older versions of mhttpd are started using the -p port option e.g.
* mhttpd -D -p 8080 

To run a new version of mhttpd using an HTTP proxy, use the options provided to run the old (non-Mongoose) webserver on a given port, i.e.
* mhttpd --oldserver 8080 --nomg -D 

When using an SSL proxy, only access from the SSL proxy (and maybe some special trusted machines) should be permitted.
This is done using the "-a hostname" switch of [[mhttpd]]. Normally there will be only "-a localhost" switch, enabling access only for the local machine (where the SSL proxy is running). Additional "-a hostname" switches enable access from listed local machines. No "-a xxx" enables access from everywhere (defeating the purpose of the SSL proxy, unless access controls are enforced elsewhere, i.e. by a site firewall or by local firewall rules).

== Setting up an HTTP proxy ==
In this example, we use APACHE HTTPD to password-protect a typical midas/mhttpd and elog installation.

In this configuration, one uses the Linux stock httpd that accepts encrypted https:// connections and forwards them to mhttpd and elogd. Instead of (or in addition to) using mhttpd and elogd passwords, one configures password protection in httpd via the regular apache httpd password mechanisms (htpasswd, etc).

Recent versions of elogd do support SSL https:// connections, but if one is running an SSL proxy for anyway, it is simpler to run both through the same SSL proxy using the same SSL host certificate and the same httpd password file.

=== Restricting http: access to elogd ===
;Note
:Recent versions of elogd do support SSL https:// connections. The following information is for those using an HTTP proxy (see above).

For elogd, this is done using the "-n localhost" switch with enables only access from the same machine if present, or access from anywhere is absent (defeating the purpose of the SSL proxy, unless access controls are enforced elsewhere).

(It is recommended to run elogd from the same user as the main daq user and to keep elogd.cfg and all logbooks in the home directory of this user, where they are captured by the normal site backup system)

== Install standalone elog ==

* login into the user account that will run the elog
* cd $HOME/packages
* git clone https://gitea.psi.ch/ELOG/elog.git
* cd elog
* make
* create new file start_elogd with this contents:
<pre>
#!/bin/sh

killall elogd
sleep 1
killall -KILL elogd
sleep 1
$HOME/packages/elog/elogd -n localhost -x -c $HOME/packages/elog/elogd.cfg -p 8082 -D

#end
</pre>
* chmod a+x start_elogd
* edit elogd.cfg to read:
<pre>
[global]
port = 8082
SMTP host = smtp.triumf.ca
URL = https://titan00.triumf.ca/elog/

Reverse sort = 1
Display Mode = full

#List Menu commands = New, Find, Admin, Help
#Menu commands = New, Edit, Reply, Find, Duplicate, Help

Entries Per Page = 30
Supress Email on edit = 1
Default encoding = 1
Page title = TITAN ELOG
Resolve host names = 1

Logfile = /home/titan/packages/elog/elogd.log
#Logging level = 3

[midas]

List page Title = T2K M11 MIDAS ELOG
Comment = T2K M11 MIDAS ELOG
Page Title = T2K M11 MIDAS ELOG
RSS Title = [$logbook - $type - $system] $subject, posted by $author

Attributes = Author, Subject, Run, Type, System
Show Attributes Edit = Run, Author, Subject, Type, System
Required Attributes = Author, Type, System, Subject

Options Type = Routine, Reply, Shift Summary, Modification, Question, Info, Problem
Options System = General, DAQ, Beamline

Preset Run = $shell(MIDASSYS=. /home/t2km11/packages/midas/linux/bin/odbedit -d Runinfo -c 'ls -v \"run number\"')

Preset On Reply Type = Reply
Preset On Reply Run = $shell(MIDASSYS=. /home/t2km11/packages/midas/linux/bin/odbedit -d Runinfo -c 'ls -v \"run number\"')

List Display = Date, Subject, Type, System, Author, ID
Quick Filter = Date, Type, ID

Remove on reply = Author
Quote on reply = 1

Use lock = 1

************* Email Functionality ****************

Use Email Subject = [T2KM11 - $System] $Subject
Omit Email To = 1

Email System General = xxx
</pre>
* ./start_elogd &
* firefox http://localhost:8082 # hould show the elog message index

To start elogd automatically when the machine is rebooted, login as root and
* add this text to /etc/rc.local:
<pre>
su - titan -c "/home/titan/packages/elog/start_elogd"
</pre>
* chmod a+x /etc/rc.local
* systemctl start rc-local

To import elog entries from the mhttpd elog, do this:

* cd ~/packages/elog/logbooks
* ln -s /home/t2km11/online/elog midas
* cd midas
* ~/packages/elog/elconv)

== Install https proxy ==

THESE INSTRUCTIONS ARE OBSOLETE, INSTEAD,
* GO HERE: https://daq00.triumf.ca/DaqWiki/index.php/Ubuntu#Install_apache_httpd_proxy_for_midas_and_elog
* AND GO HERE: https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux#Run_the_MIDAS_Web_Server

FOLLOWING INSTRUCTIONS ARE OBSOLETE...

* login as root to the https proxy machine
* cd ~root
* yum install mod_ssl
* yum install crypto-utils # see http://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_monitoring_of_HTTPS_certificates
* create a certificate request (replace ladd09 with your hostname): openssl req -new -nodes -newkey rsa:2048 -sha256 -out ladd09.csr -keyout ladd09.key (answer: CA, BC, Vancouver, TRIUMF, DAQ, ladd09.triumf.ca, email@email.com
* sign it by TRIUMF:
** mail -s "Certificate request" yourself@email.com < ladd09.csr
** forward this request to Andrew Daviel
** he will email the signed crt file, copy it to this system as ladd09.crt
* sign it yourself: openssl x509 -req -days 365 -sha256 -in ladd09.csr -signkey ladd09.key -out ladd09.crt
* (if the certificate expires, renew it by signing it again)
* Additional commands for working with certificates:
** explore the private key: openssl pkey -in ladd09.key -text -noout
** explore the certificate request: openssl req -in ladd00.csr -text -noout
** explore the certificate: openssl x509 -in ladd09.crt -noout -text
* move certificate files to proper system locations:
* mv ladd09.key /etc/pki/tls/private/
* mv ladd09.crt /etc/pki/tls/certs/
* if selinux is enabled, do this:
** restorecon -Rv /etc/pki/tls/certs/
** restorecon -Rv /etc/pki/tls/private/
** /usr/sbin/setsebool -P httpd_can_network_connect 1
* open /etc/httpd/conf.d/ssl.conf in a text editor, go to the very bottom and right before the "</VirtualHost>" entry, add following text:
<pre>
...
SSLCertificateFile /etc/pki/tls/certs/ladd09.crt
SSLCertificateKeyFile /etc/pki/tls/private/ladd09.key

ProxyPass /elog/ http://localhost:8082/ retry=1
ProxyPass / http://localhost:8080/ retry=1

<Location />

SSLRequireSSL
AuthType Basic
AuthName "password protected site"
Require valid-user

# create password file: touch /etc/httpd/htpasswd
# to add new user or change password: htpasswd /etc/httpd/htpasswd username
AuthUserFile /etc/httpd/htpasswd

</Location>
</VirtualHost>
...
</pre>
* comment out duplicate "SSLCertificateFile" and "SSLCertificateKeyFile" elsewhere in the file
* (optionally) If you got a certificate that is signed by DigiCert or RapidSSL then you'll need to add a line specifying the certificate chain file:
<pre>
...
SSLCertificateChainFile /etc/pki/tls/certs/DigiCertCA.crt
...
</pre>
* touch /etc/httpd/htpasswd
* htpasswd /etc/httpd/htpasswd midas # enter password midas
* chkconfig httpd on
* service httpd restart
* firewall-cmd --add-port=443/tcp --permanent
* firewall-cmd --reload
* firewall-cmd --list-all
* test it
** test the SSL proxy: https://host/ should yield the midas status page, https://host/elog/ should yield the elog message index
* in ODB, set "/Elog/URL" to "https://host/elog/"
* now from the midas status page, the "Elog" button should take us to the https Elog URL

In needed, enable user directories: https://blah/~user in ~user/public_html

* edit /etc/httpd/conf.d/userdir.conf, replace "UserDir disabled" with "UserDir enabled"
* setsebool -P httpd_enable_homedirs true
* systemctl restart httpd

== Setup the history mhttpd for faster access to history plots ==
When running an SSL proxy,
* start the main mhttpd (orange command for old mhttpd, green for new mhttpd with Mongoose(post August2015):
** "mhttpd -p 8071 -D" or
** "mhttpd -D --oldserver 8071 --nomg" 

* start the history mhttpd
** "mhttpd -p 8072 -D -H" or
** "mhttpd -D -H --oldserver 8072 --nomg" 
* set ODB /History/URL to "http://alphacpc09.cern.ch:8072/HS/"
* open the MIDAS status page
* go to the history section, try to open any history plot, observe that the history plot gif image loads correctly, inspect it's URL (use "copy image URL" or "view source", etc), it should point to port 8072 causing connection to the history mhttpd.
* continue with these instructions to setup history mhttpd access through an SSL proxy:
* setup SSL proxy access (required mod_proxy_html)
** login as root to the SSL proxy machine
** on SL5, install the missing mod_proxy_html httpd module:
** yum install httpd-devel libxml2-devel
** wget http://apache.webthing.com/mod_proxy_html/mod_proxy_html.tar.bz2
** tar xjvf mod_proxy_html.tar.bz2
** cd mod_proxy_html
** apxs -c -I. -I/usr/include/libxml2 -i mod_proxy_html.c
** apxs -c -I. -I/usr/include/libxml2 -i mod_xml2enc.c
** cd /etc/httpd/conf.d, add this to ssl.conf:

;before the ProxyPass statements:
<pre>
# proxy the MIDAS web servers
LoadModule xml2enc_module modules/mod_xml2enc.so
LoadModule proxy_html_module modules/mod_proxy_html.so
ProxyHTMLLinks a href
ProxyHTMLLinks link href
ProxyHTMLLinks img src
#ProxyHTMLEnable On
ProxyRequests off
</pre>
;after the ProxyPass statements:
<pre>
# ALPHA1 history access
ProxyPass /alpha1/history/ http://alphacpc09.cern.ch:8072/HS/ retry=1
ProxyPass /alpha1/ http://alphacpc09.cern.ch:8071/ retry=1

ProxyHTMLEnable On
ProxyHTMLURLMap http://alphacpc09.cern.ch:8072/HS/ /alpha1/history/
</pre>
;adjust:
*"alpha1" is the experiment name
*"alphacpc09.cern.ch" is the machine running mhttpd
*"8071" is the port number of the main mhttpd
** "mhttpd -p 8071 -D" or
** "mhttpd -D --oldserver 8071 --nomg" 
* "8072" is the port number of the history mhttpd
** "mhttpd -p 8072 -D -H" or
** "mhttpd -D -H --oldserver 8072 --nomg" 

[[Category:Installation]] [[Category:Buffer]] [[Category:Driver]]

Setup MIDAS experiment at TRIUMF

2026-09-29T16:11:35Z

Dfujimoto: /* Install ROODY */

{{Pagelinks}}
== Introduction ==
This page describes setting up a MIDAS experiment at TRIUMF. This information can be adapted for other sites.

== Environment variables ==
* '''MIDASSYS''' Base directory of the MIDAS package, midas and mxml should be at the same level.
* '''MIDAS_EXPTAB''' Experiment definition file
* '''MIDAS_SERVER_HOST''' MIDAS host server name for remote midas connections.
* '''MIDAS_EXPT_NAME''' Experiment name

== Standard layout of MIDAS experiment ==
The following shows the directory layout of a standard MIDAS experiment:

/home/exptuser/
packages/
root <---- ROOT
mxml
mscb
midas/ <---- MIDAS
linux/{lib,bin} <---- binaries matching the selected 64-bit/32-bit flavour of ROOT
linux-m32/{lib,bin} <---- limited function 32-bit binaries for 32-bit frontend machines, build by "make linux32"
linux-m64/{lib,bin} <---- limited function 64-bit binaries (only needed if ROOT and linux/bin are 32-bit)
linux-arm/{lib,bin} <---- full function ARM cross-compiled using "make linuxarm"
linux-crosscompile/{lib,bin} <---- cross-compiled limited function binaries for PPC and ARM frontends (see Makefile)
rootana <---- ROOT analyzer
roody <---- graphical online histogram viewer for MIDAS and ROOTANA
online/
exptab <---- experiment definition
{.ODB,.SYSTEM,.SYSMSG,etc}.SHM <---- MIDAS shared memory save files
src <---- experiment frontend sources
bin,scripts
elog <---- MIDAS elog
history <---- MIDAS history
data -> /data/exptname/current <---- symlink to the data directory
/data/exptname/current <---- experiment data directory with ODB save files and MIDAS .mid/.mid.gz data files

== Prepare computers ==

On some operating systems, several MIDAS functions require administrator access:

* on el7 linux (SL7/CC7/CentOS7/RHEL7) - access to mhttpd port 8443 requires special firewall rules, see here: https://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29
* on el7 linux - access to mserver to run frontends and other programs on some other computer requires special firewall rules, see here: https://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29
* on el7 linux - on the frontend machines (and other machines that will connect to the mserver, the same firewall rule needs to be created (use the IP address of the machine running the mserver)

== Prepare the user account ==
<div id="NOTES"></div>

* Setup the user account for running this instance of midas. For machines part of the LADD cluster, follow these [http://daq-plone.triumf.ca/SM/docs/local/NewLaddUser] instructions.
* check that the account is using the /bin/bash shell
* make $HOME/.profile look like this:

<pre>
#!/bin/echo You must source

export SVN_EDITOR="emacs -nw"
export GIT_EDITOR="emacs -nw"
export MIDASSYS=$HOME/packages/midas
export ROOTANASYS=$HOME/packages/rootana
export MIDAS_EXPTAB=$HOME/online/exptab
#
# setup the MIDAS mserver
#
case `hostname` in
daq07*)
unset MIDAS_SERVER_HOST
;;
*)
export MIDAS_SERVER_HOST=daq07.triumf.ca:7070
;;
esac
#
# select 64-bit or 32-bit MIDAS and ROOT
#
case `uname -i` in
i386)
source /daq/daqshare/olchansk/root/root_v5.34.01_SL62_32/bin/thisroot.sh
export PATH=.:$MIDASSYS/linux-m32/bin:$PATH
;;
*)
#source /daq/daqshare/olchansk/root/root_v5.34.34_SL67_64/bin/thisroot.sh
source $HOME/packages/root/bin/thisroot.sh
export PATH=.:$MIDASSYS/linux/bin:$PATH
;;
esac
#
export PATH=.:$HOME/online/bin:$HOME/packages/roody/bin:$PATH
#
#end
</pre>

* mkdir $HOME/packages
* Logout and login again, for .cshrc changes to take effect

== Install ROOT ==

* Identify the Linux version: RH9 (Red Hat Linux 9), FC3 (Fedora Core 3), RHEL4/SL4 (Red Hat Enterprise LInux 4/Scientific Linux 4), SL5x, SL6x, (CentOS/CC/SL) el7x: more /etc/redhat-release
* Decide to use 32-bit or 64-bit ROOT ('uname -a')
* cd $HOME/packages
* ls -l /daq/daqshare/olchansk/root/ ### to see all available ROOT packages
* ln -s /daq/daqshare/olchansk/root/root_vNNN_VVV_BB root, where NNN is the latest available version of ROOT ("ls -l /daq/daqshare/olchansk/root"), VVV is the Linux version code (RH9, FC3, SL4, etc) and BB is "_32" or "_64" for 32-bit or 64-bit ROOT. For example: /daq/daqshare/olchansk/root/root_v5.10.00_SL40
* for example: ln -s /daq/daqshare/olchansk/root/root_v5.34.34_el72_64 $HOME/packages/root
* Check that ROOT works: "source $HOME/packages/root/bin/thisroot.sh; root". Type ".q" to exit root.

== Install MIDAS ==

* cd $HOME/packages
* (OBSOLETE) svn co svn+ssh://svn@savannah.psi.ch/repos/meg/midas/trunk midas, password "svn". (password has to be entered twice)
* (OBSOLETE) svn co svn+ssh://svn@savannah.psi.ch/repos/meg/mxml/trunk mxml
* git clone https://gitlab.triumf.ca/midas/midas.git --recursive
* (OBSOLETE) git clone https://gitlab.triumf.ca/midas/mxml.git
* (OBSOLETE) git clone https://gitlab.triumf.ca/midas/mscb.git
* (IF GITLAB IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/mxml.git
* (IF GITLAB IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/mscb.git
* (IF GITLAB IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/midas.git
* cd midas
* make
* (only if needed) make linux32 ### build the 32-bit MIDAS libraries
* ls -l linux/bin/odbedit ### check that odbedit has been created (do not run it yet)

You can see a list of other installation problems at [[Common problems & Debugging recipes]].

;NOTE 1
: Optional features in MIDAS can be explicitly disabled if desired when making MIDAS using the NO_xxx feature (NO_ROOT,NO_MYSQL,NO_ODBC,NO_SQLITE,NO_MSCB), e.g. "make NO_ROOT=1" to disable ROOT. These NO_xxx Makefile variables are only used to control autodetection.

;NOTE 2
: Since June 2019 the mxml and mscb packages are submodules of the midas package, so no need to clone them separately. If you have an existing clone of midas but not yet the submodules, you need

$ git submodule update --init --recursive

: To update both midas and the submodules, you need

$ git pull --recurse-submodules

== Install ROOTANA ==

* cd $HOME/packages
* git clone https://gitlab.triumf.ca/midas/rootana.git
* cd rootana
* make

== Install ROODY ==

* cd $HOME/packages
* git clone https://gitlab.triumf.ca/midas/roody.git
* cd roody
* make
* $HOME/packages/roody/bin/roody, run the program

== Install additional additional packages ==

* cd $HOME/packages
* git clone https://bitbucket.org/ttriumfdaq/vme
* git clone https://bitbucket.org/ttriumfdaq/frontends

== Build special versions of MIDAS ==

Build special versions of MIDAS for the case when some MIDAS programs, such as VME frontends, will run on a different computer that may have a different flavour of operating system, i.e. 32-bit linux or an older version of Scientific Linux.

* login to the computer where the frontends will run and:
* if it is a 32-bit linux: cd $HOME/packages/midas; make linux32
* if it is a 64-bit linux: cd $HOME/packages/midas; make linux64

* login to the host computer to cross-compile ARM code:
* if it is an ARM linux: cd $HOME/packages/midas; make linuxarm # may need to install ARM cross compilers

== Prepare VME hardware ==

Hardware check list:
* VME crate
* VME processor (supported are V77xx, V7805, V7865)
* On all VME modules, set the VME address jumpers as described here: http://daq-plone.triumf.ca/SM/docs/local/vme_jumpers
* run vmescan to confirm correct VME addresses
** cd $HOME/packages
** svn checkout https://ladd00.triumf.ca/svn/daqsvn/trunk/vme
** cd vme
** make
** ./vmescan.exe (or _gef.exe, depending on the VME driver in use)

== Install Universe-II VME driver (V7648, V7750, V7805, V7851) ==

* login as root (ssh root@localhost)
* follow instructions: https://www.triumf.info/wiki/DAQwiki/index.php/VME-CPU#V7648.2C_V7750.2C_V7805.2C_V7851_:_Setup_vme_universe_VME_drivers
* cd ~/packages/vme; vmescan.exe

== Install Tsi-148 VME driver (V7865) ==

* login as root (ssh root@localhost)
* follow instructions: https://www.triumf.info/wiki/DAQwiki/index.php/VME-CPU#V7865_and_XVB-602_:_Setup_gefvme.2Ftsi148_VME_drivers
* cd ~/packages/vme; vmescan_gef.exe

== Setup the experiment environment ==

* Decide which computer will host MIDAS (where MIDAS shared memory buffers will reside).
: This computer will run the [[mserver]], [[mlogger]] and [[mhttpd]] applications. (It is usually the machine where the MIDAS,ROOT etc. packages have been downloaded). It will be referred to as the host machine (localhost).

The environment is slightly different depending on whether all programs run on the host machine, or whether some programs run on remote host(s) :
=== ALL programs run on localhost ===
:If all programs run on the host machine (localhost), it is not necessary to run [[mserver]]. [[Environment Variables#MIDAS_SERVER_HOST|MIDAS_SERVER_HOST]] will not be assigned (see example .cshrc [[#Prepare the user account|above]]).

=== Some programs run on REMOTE host(s) ===
: '''IMPORTANT:'''
# Since August 2015 '''you must explicitly allow access for clients running on remote machines'''. To do this, follow the '''[[Security#MIDAS programs on remote machines|instructions here]]'''.
# The example code .cshrc ([[#Prepare the user account|see above]]) should be present on both host and remote machine(s). This will ensure that [[Environment Variables#MIDAS_SERVER_HOST|MIDAS_SERVER_HOST]] will NOT be set for the host machine (localhost), but on a remote machine, MIDAS_SERVER_HOST will be set to the MIDAS host machine.
# The client [[mserver]] must be started on the MIDAS host machine. Note that multiple experiments can run on the same host machine by starting several instances of [[mserver]] (one for each experiment) running with different ports (and .cshrc would be edited so that MIDAS_SERVER_HOST is set to the appropriate port for the experiment).

 
On the host machine:
* mkdir $HOME/online
* cd $HOME/online
* create directories for local programs, sources, elog and history: mkdir bin src elog history
* create data directory: mkdir -p /ladd/data1/t2kvme5/data; ln -s /ladd/data1/t2kvme5/data $HOME/online
* create the exptab file "$HOME/online/exptab" following the example below. The first entry (exptname) is the name if the DAQ system (MIDAS experiment name), the second entry (/home/USER/online) is the location of MIDAS shared memory buffers (by convention, $HOME/online), the third entry (kopio03) is your username.
<pre>exptname /home/kopio03/online kopio03</pre>
* logout and login again for all changes to take effect

== Setup experiment startup scripts ==
* login to the experiment host computer
* echo $MIDAS_SERVER_HOST ### to check correct value - should be blank
* create $HOME/online/bin/start_daq.sh, replacing XXX with the hostname of the machine running the experiment (and changing the mserver and mhttpd ports, as needed).

#!/bin/sh
# start_daq.sh
cd $HOME/online
#
case `hostname` in XXX*)
echo "Good, we are on XXX!"
;;
*)
echo "The start_daq script should be executed on XXX"
exit 1
;;
esac
#
odbedit -c clean
# start [[mhttpd]] on default port. (Mongoose https version - see [[mhttpd]] for other options)
mhttpd -D -a localhost -a XXX.triumf.ca # optionally restrict access to specified hosts
#
# start [[mserver]] on default port (use argument -p to use a different port)
mserver -D # access must now be specifically allowed - see [[#Setup the experiment environment|above]]

# OR ([[#NOTES|older MIDAS versions]])
# mhttpd -p 8081 -D -a localhost -a XXX.triumf.ca # optionally restrict access to specified hosts
# mserver -p 7071 -D -a localhost -a lxdragon01.triumf.ca -a lxdragon02.triumf.ca -a XXX.triumf.ca # optionally restrict access to specified hosts

#
mlogger -D
#end file

== Run the MIDAS Web Server ==
Let's start the MIDAS webserver for the first time:

Start [[mhttpd]] on the ''experiment host'' (localhost) like this:
[mhostpc] mhttpd
You will get the following messages:
[mhttpd,INFO] ODB subtree /Runinfo corrected successfully
Mongoose web server will listen on ports "8080r,8443s"
[mhttpd,ERROR] [mhttpd.cxx:17892:mongoose,ERROR] cannot find SSL certificate file "/home/agdaq/online/ssl_cert.pem"
[mhttpd,ERROR] [mhttpd.cxx:17893:mongoose,ERROR] please create SSL certificate file: openssl req -new -nodes -newkey rsa:2048 -sha256 -out ssl_cert.csr -keyout ssl_cert.key; openssl x509 -req -days 365 -sha256 -in ssl_cert.csr -signkey ssl_cert.key -out ssl_cert.pem; cat ssl_cert.key >> ssl_cert.pem
could not start the mongoose web server, see messages and midas.log, bye!

Create a self-signed certificate suitable for initial testing by executing the command printed by mhttpd:


[mhostpc] openssl req -new -nodes -newkey rsa:2048 -sha256 -out ssl_cert.csr -keyout ssl_cert.key; openssl x509 -req -days 365 -sha256 -in ssl_cert.csr -signkey ssl_cert.key -out ssl_cert.pem; cat ssl_cert.key >> ssl_cert.pem

For production use, you should create a properly signed certificate, see [[Mhttpd#Create an SSL certificate|create your own SSL certificate]] or you should run mhttpd behind an SSL proxy.

Run mhttpd again.

You will get the following messages:
[mhttpd,INFO] ODB subtree /Runinfo corrected successfully
Mongoose web server will listen on ports "8080r,8443s"
Mongoose web server will use SSL certificate file "/home/johnfoo/packages/midas/ssl_cert.pem"
[mhttpd,ERROR] [mhttpd.cxx:17633:mongoose,ERROR] mongoose web server cannot find password file "/home/johnfoo/online/htpasswd.txt"
[mhttpd,ERROR] [mhttpd.cxx:17634:mongoose,ERROR] please create password file: htdigest -c /home/johnfoo/online/htpasswd.txt Default midas
could not start the mongoose web server, see messages and midas.log, bye!

Create the password file by following the instructions printed by mhttpd. The http digest domain name is the experiment name, suggested default user name is "midas". You will be asked to type in a password


[mhostpc] htdigest -c /home/johnfoo/online/htpasswd.txt exptname midas
Adding password for midas in realm exptname.
New password:
Re-type new password:

It is a good idea to set the password file {{Filepath|path=htpasswd.txt}} readable and writable by owner only.

Now restart {{Utility|name=mhttpd}}
[mhostpc] mhttpd
Mongoose web server will listen on ports "8080r,8443s" **see note
Mongoose web server will use SSL certificate file "/home/suz/packages/midas/ssl_cert.pem"
Mongoose web server will use authentication realm "Default", password file "./htpasswd.txt"

Now point a web browser running on the same host computer (localhost) to https://localhost:8443
If the web browser is running on a different computer, go to URL of the form

https://mhostpc.triumf.ca:8443 (substitute your host machine name and domain for "mhostpc.triumf.ca")

If you are using the default SSL certificate you will probably get a message: "This Connection is Untrusted". Click "I understand the risks" and add an exception. This is because the test certificate is self-signed. Then confirm an exception.

If instead you get a "connection refused" error, the midas host pc may have the firewall enabled. To make a firewall exception for MIDAS, follow instructions here http://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29

You should then see an authentication box asking you for the user name and password. The user name is "midas". Enter the password you just created. The Midas [[Status Page]] should appear with multiple buttons for run control as well as equipment listing (no equipments will be listed as yet) and application listings. Please refer to [[mhttpd]] (the MIDAS Web-based Run Control utility) for further information. You can start and stop runs from the main status page, and use the [[ODB Page]] to access the database (ODB).

; Note
: Default ports of 8080 and 8443 are used by [[mhttpd]]. If these ports are in use on your machine, start mhttpd with alternative ports, e.g.
[mhostpc] mhttpd --https 8448 --http 8089
: or see [[Mhttpd#Usage]] to change the default ports.

== Setup experiment database (ODB) ==

* run $HOME/online/bin/start_daq.sh

* odbedit, run these commands: (replace user names and directory names)
<pre>
# set "/Logger/Message file" "/home/kopio03/online/midas.log" # obsolete
set "/Logger/Data Dir" "/home/kopio03/online/data"
set "/Logger/History dir" "/home/kopio03/online/history"
create STRING "/Logger/Elog dir"
set "/Logger/Elog dir" "/home/kopio03/online/elog"
set "/Logger/ODB dump file" "/home/kopio03/online/history/run%05d.xml"
set "/Logger/ODB dump" "y"
set "/Logger/Channels/0/Settings/Filename" "run%05dsub%03d.mid.gz"
set "/Logger/Channels/0/Settings/Subrun byte limit" "1000000000"
set "/Logger/Channels/0/Settings/Compression" 1
set "/Logger/Channels/0/Settings/ODB Dump" "y"
set "/Programs/Logger/Required" y
set "/Programs/Logger/Start command" "mlogger -D"
set "/Programs/fevme/Required" "y"
set "/Programs/fevme/Start command" "ssh -n lxdaq09 $HOME/online/src/fevme_gef.exe -O"
exit
</pre>
* open web browser e.g. firefox.
* go to the midas status page at https://localhost:8443 (default port).
** if running [[mhttpd]] with Mongoose HTTPS/OpenSSL (the default) for the first time, you will need to create a password file. Follow the instructions (see [[mhttpd#HTTPS/SSL server (Mongoose)]] for details).
** For other options (i.e. HTTPS/SSL proxy) see [[#Secure MIDAS and ELOG Web access]]
*OR open the midas status page at http://localhost:8081 ([[#NOTES|older MIDAS versions]])
* midas status page will show most stuff "red" as nothing is running yet
* DON'T DO THIS YET run ./fevme.exe (on the computer with the VME interface, could be different from computer hosting the experiment), observe that corresponding equipments have been created
* save the url bookmark to the "personal toolbar"
* go to the Programs page, stop mlogger, stop fevme, start mlogger, start fevme
* go to the Status page, start run, stop run
* go back to the Status page, everything should be green
* start a run
* send signals to the ADC gate
* you should be getting events
* to look at data, proceed with setting up the [[ROOTANA|ROOT Analyzer]].

== Start DAQ programs at boot time ==

* add this to /etc/rc.local (replace username and location of the start_daq script)
<pre>
su - alpha -c /home/alpha/online/bin/start_daq.sh
</pre>

== Setup local software version control ==

Version control for experiment source code is setup using "git" (http://git-scm.com/)

* cd $HOME/online
* git init
* git add exptab
* git add bin/start_daq.sh
* git add .gitignore ### contents can be
<pre>
*~
*.o
*.exe
</pre>
* git add src/Makefile src/*.cxx ...
* git commit -a

== Adjust MIDAS buffer sizes ==

Default MIDAS SYSTEM buffer size is 8 Mbytes, fairly small for high-data-rate experiments. The rule of thumb is to have at least a few seconds worth of buffer space available. For example, if event size is 10 Kbytes and the event rate is 1 kHz, data rate is 10*10^3*1*10^3 = 10 Mbytes/sec. To buffer 10 seconds of data we need 100 Mbytes of buffer space.

To resize the MIDAS event buffers (SYSTEM, etc) do this:
* stop all frontends, stop mlogger
* start odbedit:
** cd "/Experiment/Buffer sizes"
** set SYSTEM 100000000
* run "mdump -z SYSTEM"
* if mdump complains about the size of .SYSTEM.SHM, remove it, try again.
* ls -l /dev/shm ### to observe that the size of shared memory is correct

== Secure MIDAS and ELOG Web access ==
In versions prior to May 2015, the default web access to MIDAS and ELOG uses the "http:" protocol which is insecure. In this case, all information is transmitted as clear text meaning that secret, confidential and sensitive information (such as the MIDAS and ELOG passwords and usernames) can be stolen "easily". This means that even "password protected" MIDAS and ELOG pages are not really protected if accessed using the "http" method.

Better security for HTTP is gained by using a password protected '''SSL (https) proxy'''. (It does not provide absolute security because of remaining problems with the security of SSL certificates, security of passwords, etc). Setting up an SSL (https) proxy is described [[#Setting up an HTTP proxy|below]].

Since May 2015, an ''alternative secure option'' to setting up an HTTP proxy is available to users of MIDAS. Recent versions of elogd (ELOG) do support SSL https:// connections, and [[#mhttpd with HTTPS/SSL server (Mongoose)]] is now available. This option is the default, and provides a similar level of security to an HTTP proxy.

See [[Security#Web Access]] for a comparison of these two secure options.

=== mhttpd with HTTPS/SSL server (Mongoose) ===

Since May 2015 the MIDAS web server [[mhttpd]] is explicitly linked with OpenSSL to provide secure HTTPS connections via the [https://bitbucket.org/tmidas/midas/src/ecb9a8537448a8a43f7f9a2bfdb82e578208cde3/doc/mongoose/?at=develop Mongoose] web server (see [[mhttpd]]). With this version, default web access to MIDAS uses the "https" protocol. Web access to {{Utility|name=mhttpd}} can be restricted by using the "-a hostname" switch of [[mhttpd]]. The first time {{Utility|name=mhttpd}} is run, a password file must be created. An SSL certificate is also required. See [[mhttpd#HTTPS/SSL server (Mongoose)|HTTPS/SSL server (Mongoose)]] for instructions.

=== mhttpd using an HTTPS/SSL proxy ===

THESE INSTRUCTIONS ARE WRONG, DO NOT DO THIS. See instead [https://daq00.triumf.ca/MidasWiki/index.php/Setup_MIDAS_experiment_at_TRIUMF#Install_https_proxy this section].

An [[#Setting up an HTTP proxy|HTTP proxy]] must be set up. This is the only way of securing older version of [[mhttpd]] (pre August 2015). Older versions of mhttpd are started using the -p port option e.g.
* mhttpd -D -p 8080 

To run a new version of mhttpd using an HTTP proxy, use the options provided to run the old (non-Mongoose) webserver on a given port, i.e.
* mhttpd --oldserver 8080 --nomg -D 

When using an SSL proxy, only access from the SSL proxy (and maybe some special trusted machines) should be permitted.
This is done using the "-a hostname" switch of [[mhttpd]]. Normally there will be only "-a localhost" switch, enabling access only for the local machine (where the SSL proxy is running). Additional "-a hostname" switches enable access from listed local machines. No "-a xxx" enables access from everywhere (defeating the purpose of the SSL proxy, unless access controls are enforced elsewhere, i.e. by a site firewall or by local firewall rules).

== Setting up an HTTP proxy ==
In this example, we use APACHE HTTPD to password-protect a typical midas/mhttpd and elog installation.

In this configuration, one uses the Linux stock httpd that accepts encrypted https:// connections and forwards them to mhttpd and elogd. Instead of (or in addition to) using mhttpd and elogd passwords, one configures password protection in httpd via the regular apache httpd password mechanisms (htpasswd, etc).

Recent versions of elogd do support SSL https:// connections, but if one is running an SSL proxy for anyway, it is simpler to run both through the same SSL proxy using the same SSL host certificate and the same httpd password file.

=== Restricting http: access to elogd ===
;Note
:Recent versions of elogd do support SSL https:// connections. The following information is for those using an HTTP proxy (see above).

For elogd, this is done using the "-n localhost" switch with enables only access from the same machine if present, or access from anywhere is absent (defeating the purpose of the SSL proxy, unless access controls are enforced elsewhere).

(It is recommended to run elogd from the same user as the main daq user and to keep elogd.cfg and all logbooks in the home directory of this user, where they are captured by the normal site backup system)

== Install standalone elog ==

* login into the user account that will run the elog
* cd $HOME/packages
* git clone https://bitbucket.org/ritt/elog
* cd elog
* make
* create new file start_elogd with this contents:
<pre>
#!/bin/sh

killall elogd
sleep 1
killall -KILL elogd
sleep 1
$HOME/packages/elog/elogd -n localhost -x -c $HOME/packages/elog/elogd.cfg -p 8082 -D

#end
</pre>
* chmod a+x start_elogd
* edit elogd.cfg to read:
<pre>
[global]
port = 8082
SMTP host = smtp.triumf.ca
URL = https://titan00.triumf.ca/elog/

Reverse sort = 1
Display Mode = full

#List Menu commands = New, Find, Admin, Help
#Menu commands = New, Edit, Reply, Find, Duplicate, Help

Entries Per Page = 30
Supress Email on edit = 1
Default encoding = 1
Page title = TITAN ELOG
Resolve host names = 1

Logfile = /home/titan/packages/elog/elogd.log
#Logging level = 3

[midas]

List page Title = T2K M11 MIDAS ELOG
Comment = T2K M11 MIDAS ELOG
Page Title = T2K M11 MIDAS ELOG
RSS Title = [$logbook - $type - $system] $subject, posted by $author

Attributes = Author, Subject, Run, Type, System
Show Attributes Edit = Run, Author, Subject, Type, System
Required Attributes = Author, Type, System, Subject

Options Type = Routine, Reply, Shift Summary, Modification, Question, Info, Problem
Options System = General, DAQ, Beamline

Preset Run = $shell(MIDASSYS=. /home/t2km11/packages/midas/linux/bin/odbedit -d Runinfo -c 'ls -v \"run number\"')

Preset On Reply Type = Reply
Preset On Reply Run = $shell(MIDASSYS=. /home/t2km11/packages/midas/linux/bin/odbedit -d Runinfo -c 'ls -v \"run number\"')

List Display = Date, Subject, Type, System, Author, ID
Quick Filter = Date, Type, ID

Remove on reply = Author
Quote on reply = 1

Use lock = 1

************* Email Functionality ****************

Use Email Subject = [T2KM11 - $System] $Subject
Omit Email To = 1

Email System General = xxx
</pre>
* ./start_elogd &
* firefox http://localhost:8082 # hould show the elog message index

To start elogd automatically when the machine is rebooted, login as root and
* add this text to /etc/rc.local:
<pre>
su - titan -c "/home/titan/packages/elog/start_elogd"
</pre>
* chmod a+x /etc/rc.local
* systemctl start rc-local

To import elog entries from the mhttpd elog, do this:

* cd ~/packages/elog/logbooks
* ln -s /home/t2km11/online/elog midas
* cd midas
* ~/packages/elog/elconv)

== Install https proxy ==

THESE INSTRUCTIONS ARE OBSOLETE, INSTEAD,
* GO HERE: https://daq00.triumf.ca/DaqWiki/index.php/Ubuntu#Install_apache_httpd_proxy_for_midas_and_elog
* AND GO HERE: https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux#Run_the_MIDAS_Web_Server

FOLLOWING INSTRUCTIONS ARE OBSOLETE...

* login as root to the https proxy machine
* cd ~root
* yum install mod_ssl
* yum install crypto-utils # see http://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_monitoring_of_HTTPS_certificates
* create a certificate request (replace ladd09 with your hostname): openssl req -new -nodes -newkey rsa:2048 -sha256 -out ladd09.csr -keyout ladd09.key (answer: CA, BC, Vancouver, TRIUMF, DAQ, ladd09.triumf.ca, email@email.com
* sign it by TRIUMF:
** mail -s "Certificate request" yourself@email.com < ladd09.csr
** forward this request to Andrew Daviel
** he will email the signed crt file, copy it to this system as ladd09.crt
* sign it yourself: openssl x509 -req -days 365 -sha256 -in ladd09.csr -signkey ladd09.key -out ladd09.crt
* (if the certificate expires, renew it by signing it again)
* Additional commands for working with certificates:
** explore the private key: openssl pkey -in ladd09.key -text -noout
** explore the certificate request: openssl req -in ladd00.csr -text -noout
** explore the certificate: openssl x509 -in ladd09.crt -noout -text
* move certificate files to proper system locations:
* mv ladd09.key /etc/pki/tls/private/
* mv ladd09.crt /etc/pki/tls/certs/
* if selinux is enabled, do this:
** restorecon -Rv /etc/pki/tls/certs/
** restorecon -Rv /etc/pki/tls/private/
** /usr/sbin/setsebool -P httpd_can_network_connect 1
* open /etc/httpd/conf.d/ssl.conf in a text editor, go to the very bottom and right before the "</VirtualHost>" entry, add following text:
<pre>
...
SSLCertificateFile /etc/pki/tls/certs/ladd09.crt
SSLCertificateKeyFile /etc/pki/tls/private/ladd09.key

ProxyPass /elog/ http://localhost:8082/ retry=1
ProxyPass / http://localhost:8080/ retry=1

<Location />

SSLRequireSSL
AuthType Basic
AuthName "password protected site"
Require valid-user

# create password file: touch /etc/httpd/htpasswd
# to add new user or change password: htpasswd /etc/httpd/htpasswd username
AuthUserFile /etc/httpd/htpasswd

</Location>
</VirtualHost>
...
</pre>
* comment out duplicate "SSLCertificateFile" and "SSLCertificateKeyFile" elsewhere in the file
* (optionally) If you got a certificate that is signed by DigiCert or RapidSSL then you'll need to add a line specifying the certificate chain file:
<pre>
...
SSLCertificateChainFile /etc/pki/tls/certs/DigiCertCA.crt
...
</pre>
* touch /etc/httpd/htpasswd
* htpasswd /etc/httpd/htpasswd midas # enter password midas
* chkconfig httpd on
* service httpd restart
* firewall-cmd --add-port=443/tcp --permanent
* firewall-cmd --reload
* firewall-cmd --list-all
* test it
** test the SSL proxy: https://host/ should yield the midas status page, https://host/elog/ should yield the elog message index
* in ODB, set "/Elog/URL" to "https://host/elog/"
* now from the midas status page, the "Elog" button should take us to the https Elog URL

In needed, enable user directories: https://blah/~user in ~user/public_html

* edit /etc/httpd/conf.d/userdir.conf, replace "UserDir disabled" with "UserDir enabled"
* setsebool -P httpd_enable_homedirs true
* systemctl restart httpd

== Setup the history mhttpd for faster access to history plots ==
When running an SSL proxy,
* start the main mhttpd (orange command for old mhttpd, green for new mhttpd with Mongoose(post August2015):
** "mhttpd -p 8071 -D" or
** "mhttpd -D --oldserver 8071 --nomg" 

* start the history mhttpd
** "mhttpd -p 8072 -D -H" or
** "mhttpd -D -H --oldserver 8072 --nomg" 
* set ODB /History/URL to "http://alphacpc09.cern.ch:8072/HS/"
* open the MIDAS status page
* go to the history section, try to open any history plot, observe that the history plot gif image loads correctly, inspect it's URL (use "copy image URL" or "view source", etc), it should point to port 8072 causing connection to the history mhttpd.
* continue with these instructions to setup history mhttpd access through an SSL proxy:
* setup SSL proxy access (required mod_proxy_html)
** login as root to the SSL proxy machine
** on SL5, install the missing mod_proxy_html httpd module:
** yum install httpd-devel libxml2-devel
** wget http://apache.webthing.com/mod_proxy_html/mod_proxy_html.tar.bz2
** tar xjvf mod_proxy_html.tar.bz2
** cd mod_proxy_html
** apxs -c -I. -I/usr/include/libxml2 -i mod_proxy_html.c
** apxs -c -I. -I/usr/include/libxml2 -i mod_xml2enc.c
** cd /etc/httpd/conf.d, add this to ssl.conf:

;before the ProxyPass statements:
<pre>
# proxy the MIDAS web servers
LoadModule xml2enc_module modules/mod_xml2enc.so
LoadModule proxy_html_module modules/mod_proxy_html.so
ProxyHTMLLinks a href
ProxyHTMLLinks link href
ProxyHTMLLinks img src
#ProxyHTMLEnable On
ProxyRequests off
</pre>
;after the ProxyPass statements:
<pre>
# ALPHA1 history access
ProxyPass /alpha1/history/ http://alphacpc09.cern.ch:8072/HS/ retry=1
ProxyPass /alpha1/ http://alphacpc09.cern.ch:8071/ retry=1

ProxyHTMLEnable On
ProxyHTMLURLMap http://alphacpc09.cern.ch:8072/HS/ /alpha1/history/
</pre>
;adjust:
*"alpha1" is the experiment name
*"alphacpc09.cern.ch" is the machine running mhttpd
*"8071" is the port number of the main mhttpd
** "mhttpd -p 8071 -D" or
** "mhttpd -D --oldserver 8071 --nomg" 
* "8072" is the port number of the history mhttpd
** "mhttpd -p 8072 -D -H" or
** "mhttpd -D -H --oldserver 8072 --nomg" 

[[Category:Installation]] [[Category:Buffer]] [[Category:Driver]]

Setup MIDAS experiment at TRIUMF

2026-09-29T16:11:13Z

Dfujimoto: /* Install ROOTANA */

{{Pagelinks}}
== Introduction ==
This page describes setting up a MIDAS experiment at TRIUMF. This information can be adapted for other sites.

== Environment variables ==
* '''MIDASSYS''' Base directory of the MIDAS package, midas and mxml should be at the same level.
* '''MIDAS_EXPTAB''' Experiment definition file
* '''MIDAS_SERVER_HOST''' MIDAS host server name for remote midas connections.
* '''MIDAS_EXPT_NAME''' Experiment name

== Standard layout of MIDAS experiment ==
The following shows the directory layout of a standard MIDAS experiment:

/home/exptuser/
packages/
root <---- ROOT
mxml
mscb
midas/ <---- MIDAS
linux/{lib,bin} <---- binaries matching the selected 64-bit/32-bit flavour of ROOT
linux-m32/{lib,bin} <---- limited function 32-bit binaries for 32-bit frontend machines, build by "make linux32"
linux-m64/{lib,bin} <---- limited function 64-bit binaries (only needed if ROOT and linux/bin are 32-bit)
linux-arm/{lib,bin} <---- full function ARM cross-compiled using "make linuxarm"
linux-crosscompile/{lib,bin} <---- cross-compiled limited function binaries for PPC and ARM frontends (see Makefile)
rootana <---- ROOT analyzer
roody <---- graphical online histogram viewer for MIDAS and ROOTANA
online/
exptab <---- experiment definition
{.ODB,.SYSTEM,.SYSMSG,etc}.SHM <---- MIDAS shared memory save files
src <---- experiment frontend sources
bin,scripts
elog <---- MIDAS elog
history <---- MIDAS history
data -> /data/exptname/current <---- symlink to the data directory
/data/exptname/current <---- experiment data directory with ODB save files and MIDAS .mid/.mid.gz data files

== Prepare computers ==

On some operating systems, several MIDAS functions require administrator access:

* on el7 linux (SL7/CC7/CentOS7/RHEL7) - access to mhttpd port 8443 requires special firewall rules, see here: https://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29
* on el7 linux - access to mserver to run frontends and other programs on some other computer requires special firewall rules, see here: https://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29
* on el7 linux - on the frontend machines (and other machines that will connect to the mserver, the same firewall rule needs to be created (use the IP address of the machine running the mserver)

== Prepare the user account ==
<div id="NOTES"></div>

* Setup the user account for running this instance of midas. For machines part of the LADD cluster, follow these [http://daq-plone.triumf.ca/SM/docs/local/NewLaddUser] instructions.
* check that the account is using the /bin/bash shell
* make $HOME/.profile look like this:

<pre>
#!/bin/echo You must source

export SVN_EDITOR="emacs -nw"
export GIT_EDITOR="emacs -nw"
export MIDASSYS=$HOME/packages/midas
export ROOTANASYS=$HOME/packages/rootana
export MIDAS_EXPTAB=$HOME/online/exptab
#
# setup the MIDAS mserver
#
case `hostname` in
daq07*)
unset MIDAS_SERVER_HOST
;;
*)
export MIDAS_SERVER_HOST=daq07.triumf.ca:7070
;;
esac
#
# select 64-bit or 32-bit MIDAS and ROOT
#
case `uname -i` in
i386)
source /daq/daqshare/olchansk/root/root_v5.34.01_SL62_32/bin/thisroot.sh
export PATH=.:$MIDASSYS/linux-m32/bin:$PATH
;;
*)
#source /daq/daqshare/olchansk/root/root_v5.34.34_SL67_64/bin/thisroot.sh
source $HOME/packages/root/bin/thisroot.sh
export PATH=.:$MIDASSYS/linux/bin:$PATH
;;
esac
#
export PATH=.:$HOME/online/bin:$HOME/packages/roody/bin:$PATH
#
#end
</pre>

* mkdir $HOME/packages
* Logout and login again, for .cshrc changes to take effect

== Install ROOT ==

* Identify the Linux version: RH9 (Red Hat Linux 9), FC3 (Fedora Core 3), RHEL4/SL4 (Red Hat Enterprise LInux 4/Scientific Linux 4), SL5x, SL6x, (CentOS/CC/SL) el7x: more /etc/redhat-release
* Decide to use 32-bit or 64-bit ROOT ('uname -a')
* cd $HOME/packages
* ls -l /daq/daqshare/olchansk/root/ ### to see all available ROOT packages
* ln -s /daq/daqshare/olchansk/root/root_vNNN_VVV_BB root, where NNN is the latest available version of ROOT ("ls -l /daq/daqshare/olchansk/root"), VVV is the Linux version code (RH9, FC3, SL4, etc) and BB is "_32" or "_64" for 32-bit or 64-bit ROOT. For example: /daq/daqshare/olchansk/root/root_v5.10.00_SL40
* for example: ln -s /daq/daqshare/olchansk/root/root_v5.34.34_el72_64 $HOME/packages/root
* Check that ROOT works: "source $HOME/packages/root/bin/thisroot.sh; root". Type ".q" to exit root.

== Install MIDAS ==

* cd $HOME/packages
* (OBSOLETE) svn co svn+ssh://svn@savannah.psi.ch/repos/meg/midas/trunk midas, password "svn". (password has to be entered twice)
* (OBSOLETE) svn co svn+ssh://svn@savannah.psi.ch/repos/meg/mxml/trunk mxml
* git clone https://gitlab.triumf.ca/midas/midas.git --recursive
* (OBSOLETE) git clone https://gitlab.triumf.ca/midas/mxml.git
* (OBSOLETE) git clone https://gitlab.triumf.ca/midas/mscb.git
* (IF GITLAB IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/mxml.git
* (IF GITLAB IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/mscb.git
* (IF GITLAB IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/midas.git
* cd midas
* make
* (only if needed) make linux32 ### build the 32-bit MIDAS libraries
* ls -l linux/bin/odbedit ### check that odbedit has been created (do not run it yet)

You can see a list of other installation problems at [[Common problems & Debugging recipes]].

;NOTE 1
: Optional features in MIDAS can be explicitly disabled if desired when making MIDAS using the NO_xxx feature (NO_ROOT,NO_MYSQL,NO_ODBC,NO_SQLITE,NO_MSCB), e.g. "make NO_ROOT=1" to disable ROOT. These NO_xxx Makefile variables are only used to control autodetection.

;NOTE 2
: Since June 2019 the mxml and mscb packages are submodules of the midas package, so no need to clone them separately. If you have an existing clone of midas but not yet the submodules, you need

$ git submodule update --init --recursive

: To update both midas and the submodules, you need

$ git pull --recurse-submodules

== Install ROOTANA ==

* cd $HOME/packages
* git clone https://gitlab.triumf.ca/midas/rootana.git
* cd rootana
* make

== Install ROODY ==

* cd $HOME/packages
* git clone https://bitbucket.org/tmidas/roody
* cd roody
* make
* $HOME/packages/roody/bin/roody, run the program

== Install additional additional packages ==

* cd $HOME/packages
* git clone https://bitbucket.org/ttriumfdaq/vme
* git clone https://bitbucket.org/ttriumfdaq/frontends

== Build special versions of MIDAS ==

Build special versions of MIDAS for the case when some MIDAS programs, such as VME frontends, will run on a different computer that may have a different flavour of operating system, i.e. 32-bit linux or an older version of Scientific Linux.

* login to the computer where the frontends will run and:
* if it is a 32-bit linux: cd $HOME/packages/midas; make linux32
* if it is a 64-bit linux: cd $HOME/packages/midas; make linux64

* login to the host computer to cross-compile ARM code:
* if it is an ARM linux: cd $HOME/packages/midas; make linuxarm # may need to install ARM cross compilers

== Prepare VME hardware ==

Hardware check list:
* VME crate
* VME processor (supported are V77xx, V7805, V7865)
* On all VME modules, set the VME address jumpers as described here: http://daq-plone.triumf.ca/SM/docs/local/vme_jumpers
* run vmescan to confirm correct VME addresses
** cd $HOME/packages
** svn checkout https://ladd00.triumf.ca/svn/daqsvn/trunk/vme
** cd vme
** make
** ./vmescan.exe (or _gef.exe, depending on the VME driver in use)

== Install Universe-II VME driver (V7648, V7750, V7805, V7851) ==

* login as root (ssh root@localhost)
* follow instructions: https://www.triumf.info/wiki/DAQwiki/index.php/VME-CPU#V7648.2C_V7750.2C_V7805.2C_V7851_:_Setup_vme_universe_VME_drivers
* cd ~/packages/vme; vmescan.exe

== Install Tsi-148 VME driver (V7865) ==

* login as root (ssh root@localhost)
* follow instructions: https://www.triumf.info/wiki/DAQwiki/index.php/VME-CPU#V7865_and_XVB-602_:_Setup_gefvme.2Ftsi148_VME_drivers
* cd ~/packages/vme; vmescan_gef.exe

== Setup the experiment environment ==

* Decide which computer will host MIDAS (where MIDAS shared memory buffers will reside).
: This computer will run the [[mserver]], [[mlogger]] and [[mhttpd]] applications. (It is usually the machine where the MIDAS,ROOT etc. packages have been downloaded). It will be referred to as the host machine (localhost).

The environment is slightly different depending on whether all programs run on the host machine, or whether some programs run on remote host(s) :
=== ALL programs run on localhost ===
:If all programs run on the host machine (localhost), it is not necessary to run [[mserver]]. [[Environment Variables#MIDAS_SERVER_HOST|MIDAS_SERVER_HOST]] will not be assigned (see example .cshrc [[#Prepare the user account|above]]).

=== Some programs run on REMOTE host(s) ===
: '''IMPORTANT:'''
# Since August 2015 '''you must explicitly allow access for clients running on remote machines'''. To do this, follow the '''[[Security#MIDAS programs on remote machines|instructions here]]'''.
# The example code .cshrc ([[#Prepare the user account|see above]]) should be present on both host and remote machine(s). This will ensure that [[Environment Variables#MIDAS_SERVER_HOST|MIDAS_SERVER_HOST]] will NOT be set for the host machine (localhost), but on a remote machine, MIDAS_SERVER_HOST will be set to the MIDAS host machine.
# The client [[mserver]] must be started on the MIDAS host machine. Note that multiple experiments can run on the same host machine by starting several instances of [[mserver]] (one for each experiment) running with different ports (and .cshrc would be edited so that MIDAS_SERVER_HOST is set to the appropriate port for the experiment).

 
On the host machine:
* mkdir $HOME/online
* cd $HOME/online
* create directories for local programs, sources, elog and history: mkdir bin src elog history
* create data directory: mkdir -p /ladd/data1/t2kvme5/data; ln -s /ladd/data1/t2kvme5/data $HOME/online
* create the exptab file "$HOME/online/exptab" following the example below. The first entry (exptname) is the name if the DAQ system (MIDAS experiment name), the second entry (/home/USER/online) is the location of MIDAS shared memory buffers (by convention, $HOME/online), the third entry (kopio03) is your username.
<pre>exptname /home/kopio03/online kopio03</pre>
* logout and login again for all changes to take effect

== Setup experiment startup scripts ==
* login to the experiment host computer
* echo $MIDAS_SERVER_HOST ### to check correct value - should be blank
* create $HOME/online/bin/start_daq.sh, replacing XXX with the hostname of the machine running the experiment (and changing the mserver and mhttpd ports, as needed).

#!/bin/sh
# start_daq.sh
cd $HOME/online
#
case `hostname` in XXX*)
echo "Good, we are on XXX!"
;;
*)
echo "The start_daq script should be executed on XXX"
exit 1
;;
esac
#
odbedit -c clean
# start [[mhttpd]] on default port. (Mongoose https version - see [[mhttpd]] for other options)
mhttpd -D -a localhost -a XXX.triumf.ca # optionally restrict access to specified hosts
#
# start [[mserver]] on default port (use argument -p to use a different port)
mserver -D # access must now be specifically allowed - see [[#Setup the experiment environment|above]]

# OR ([[#NOTES|older MIDAS versions]])
# mhttpd -p 8081 -D -a localhost -a XXX.triumf.ca # optionally restrict access to specified hosts
# mserver -p 7071 -D -a localhost -a lxdragon01.triumf.ca -a lxdragon02.triumf.ca -a XXX.triumf.ca # optionally restrict access to specified hosts

#
mlogger -D
#end file

== Run the MIDAS Web Server ==
Let's start the MIDAS webserver for the first time:

Start [[mhttpd]] on the ''experiment host'' (localhost) like this:
[mhostpc] mhttpd
You will get the following messages:
[mhttpd,INFO] ODB subtree /Runinfo corrected successfully
Mongoose web server will listen on ports "8080r,8443s"
[mhttpd,ERROR] [mhttpd.cxx:17892:mongoose,ERROR] cannot find SSL certificate file "/home/agdaq/online/ssl_cert.pem"
[mhttpd,ERROR] [mhttpd.cxx:17893:mongoose,ERROR] please create SSL certificate file: openssl req -new -nodes -newkey rsa:2048 -sha256 -out ssl_cert.csr -keyout ssl_cert.key; openssl x509 -req -days 365 -sha256 -in ssl_cert.csr -signkey ssl_cert.key -out ssl_cert.pem; cat ssl_cert.key >> ssl_cert.pem
could not start the mongoose web server, see messages and midas.log, bye!

Create a self-signed certificate suitable for initial testing by executing the command printed by mhttpd:


[mhostpc] openssl req -new -nodes -newkey rsa:2048 -sha256 -out ssl_cert.csr -keyout ssl_cert.key; openssl x509 -req -days 365 -sha256 -in ssl_cert.csr -signkey ssl_cert.key -out ssl_cert.pem; cat ssl_cert.key >> ssl_cert.pem

For production use, you should create a properly signed certificate, see [[Mhttpd#Create an SSL certificate|create your own SSL certificate]] or you should run mhttpd behind an SSL proxy.

Run mhttpd again.

You will get the following messages:
[mhttpd,INFO] ODB subtree /Runinfo corrected successfully
Mongoose web server will listen on ports "8080r,8443s"
Mongoose web server will use SSL certificate file "/home/johnfoo/packages/midas/ssl_cert.pem"
[mhttpd,ERROR] [mhttpd.cxx:17633:mongoose,ERROR] mongoose web server cannot find password file "/home/johnfoo/online/htpasswd.txt"
[mhttpd,ERROR] [mhttpd.cxx:17634:mongoose,ERROR] please create password file: htdigest -c /home/johnfoo/online/htpasswd.txt Default midas
could not start the mongoose web server, see messages and midas.log, bye!

Create the password file by following the instructions printed by mhttpd. The http digest domain name is the experiment name, suggested default user name is "midas". You will be asked to type in a password


[mhostpc] htdigest -c /home/johnfoo/online/htpasswd.txt exptname midas
Adding password for midas in realm exptname.
New password:
Re-type new password:

It is a good idea to set the password file {{Filepath|path=htpasswd.txt}} readable and writable by owner only.

Now restart {{Utility|name=mhttpd}}
[mhostpc] mhttpd
Mongoose web server will listen on ports "8080r,8443s" **see note
Mongoose web server will use SSL certificate file "/home/suz/packages/midas/ssl_cert.pem"
Mongoose web server will use authentication realm "Default", password file "./htpasswd.txt"

Now point a web browser running on the same host computer (localhost) to https://localhost:8443
If the web browser is running on a different computer, go to URL of the form

https://mhostpc.triumf.ca:8443 (substitute your host machine name and domain for "mhostpc.triumf.ca")

If you are using the default SSL certificate you will probably get a message: "This Connection is Untrusted". Click "I understand the risks" and add an exception. This is because the test certificate is self-signed. Then confirm an exception.

If instead you get a "connection refused" error, the midas host pc may have the firewall enabled. To make a firewall exception for MIDAS, follow instructions here http://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29

You should then see an authentication box asking you for the user name and password. The user name is "midas". Enter the password you just created. The Midas [[Status Page]] should appear with multiple buttons for run control as well as equipment listing (no equipments will be listed as yet) and application listings. Please refer to [[mhttpd]] (the MIDAS Web-based Run Control utility) for further information. You can start and stop runs from the main status page, and use the [[ODB Page]] to access the database (ODB).

; Note
: Default ports of 8080 and 8443 are used by [[mhttpd]]. If these ports are in use on your machine, start mhttpd with alternative ports, e.g.
[mhostpc] mhttpd --https 8448 --http 8089
: or see [[Mhttpd#Usage]] to change the default ports.

== Setup experiment database (ODB) ==

* run $HOME/online/bin/start_daq.sh

* odbedit, run these commands: (replace user names and directory names)
<pre>
# set "/Logger/Message file" "/home/kopio03/online/midas.log" # obsolete
set "/Logger/Data Dir" "/home/kopio03/online/data"
set "/Logger/History dir" "/home/kopio03/online/history"
create STRING "/Logger/Elog dir"
set "/Logger/Elog dir" "/home/kopio03/online/elog"
set "/Logger/ODB dump file" "/home/kopio03/online/history/run%05d.xml"
set "/Logger/ODB dump" "y"
set "/Logger/Channels/0/Settings/Filename" "run%05dsub%03d.mid.gz"
set "/Logger/Channels/0/Settings/Subrun byte limit" "1000000000"
set "/Logger/Channels/0/Settings/Compression" 1
set "/Logger/Channels/0/Settings/ODB Dump" "y"
set "/Programs/Logger/Required" y
set "/Programs/Logger/Start command" "mlogger -D"
set "/Programs/fevme/Required" "y"
set "/Programs/fevme/Start command" "ssh -n lxdaq09 $HOME/online/src/fevme_gef.exe -O"
exit
</pre>
* open web browser e.g. firefox.
* go to the midas status page at https://localhost:8443 (default port).
** if running [[mhttpd]] with Mongoose HTTPS/OpenSSL (the default) for the first time, you will need to create a password file. Follow the instructions (see [[mhttpd#HTTPS/SSL server (Mongoose)]] for details).
** For other options (i.e. HTTPS/SSL proxy) see [[#Secure MIDAS and ELOG Web access]]
*OR open the midas status page at http://localhost:8081 ([[#NOTES|older MIDAS versions]])
* midas status page will show most stuff "red" as nothing is running yet
* DON'T DO THIS YET run ./fevme.exe (on the computer with the VME interface, could be different from computer hosting the experiment), observe that corresponding equipments have been created
* save the url bookmark to the "personal toolbar"
* go to the Programs page, stop mlogger, stop fevme, start mlogger, start fevme
* go to the Status page, start run, stop run
* go back to the Status page, everything should be green
* start a run
* send signals to the ADC gate
* you should be getting events
* to look at data, proceed with setting up the [[ROOTANA|ROOT Analyzer]].

== Start DAQ programs at boot time ==

* add this to /etc/rc.local (replace username and location of the start_daq script)
<pre>
su - alpha -c /home/alpha/online/bin/start_daq.sh
</pre>

== Setup local software version control ==

Version control for experiment source code is setup using "git" (http://git-scm.com/)

* cd $HOME/online
* git init
* git add exptab
* git add bin/start_daq.sh
* git add .gitignore ### contents can be
<pre>
*~
*.o
*.exe
</pre>
* git add src/Makefile src/*.cxx ...
* git commit -a

== Adjust MIDAS buffer sizes ==

Default MIDAS SYSTEM buffer size is 8 Mbytes, fairly small for high-data-rate experiments. The rule of thumb is to have at least a few seconds worth of buffer space available. For example, if event size is 10 Kbytes and the event rate is 1 kHz, data rate is 10*10^3*1*10^3 = 10 Mbytes/sec. To buffer 10 seconds of data we need 100 Mbytes of buffer space.

To resize the MIDAS event buffers (SYSTEM, etc) do this:
* stop all frontends, stop mlogger
* start odbedit:
** cd "/Experiment/Buffer sizes"
** set SYSTEM 100000000
* run "mdump -z SYSTEM"
* if mdump complains about the size of .SYSTEM.SHM, remove it, try again.
* ls -l /dev/shm ### to observe that the size of shared memory is correct

== Secure MIDAS and ELOG Web access ==
In versions prior to May 2015, the default web access to MIDAS and ELOG uses the "http:" protocol which is insecure. In this case, all information is transmitted as clear text meaning that secret, confidential and sensitive information (such as the MIDAS and ELOG passwords and usernames) can be stolen "easily". This means that even "password protected" MIDAS and ELOG pages are not really protected if accessed using the "http" method.

Better security for HTTP is gained by using a password protected '''SSL (https) proxy'''. (It does not provide absolute security because of remaining problems with the security of SSL certificates, security of passwords, etc). Setting up an SSL (https) proxy is described [[#Setting up an HTTP proxy|below]].

Since May 2015, an ''alternative secure option'' to setting up an HTTP proxy is available to users of MIDAS. Recent versions of elogd (ELOG) do support SSL https:// connections, and [[#mhttpd with HTTPS/SSL server (Mongoose)]] is now available. This option is the default, and provides a similar level of security to an HTTP proxy.

See [[Security#Web Access]] for a comparison of these two secure options.

=== mhttpd with HTTPS/SSL server (Mongoose) ===

Since May 2015 the MIDAS web server [[mhttpd]] is explicitly linked with OpenSSL to provide secure HTTPS connections via the [https://bitbucket.org/tmidas/midas/src/ecb9a8537448a8a43f7f9a2bfdb82e578208cde3/doc/mongoose/?at=develop Mongoose] web server (see [[mhttpd]]). With this version, default web access to MIDAS uses the "https" protocol. Web access to {{Utility|name=mhttpd}} can be restricted by using the "-a hostname" switch of [[mhttpd]]. The first time {{Utility|name=mhttpd}} is run, a password file must be created. An SSL certificate is also required. See [[mhttpd#HTTPS/SSL server (Mongoose)|HTTPS/SSL server (Mongoose)]] for instructions.

=== mhttpd using an HTTPS/SSL proxy ===

THESE INSTRUCTIONS ARE WRONG, DO NOT DO THIS. See instead [https://daq00.triumf.ca/MidasWiki/index.php/Setup_MIDAS_experiment_at_TRIUMF#Install_https_proxy this section].

An [[#Setting up an HTTP proxy|HTTP proxy]] must be set up. This is the only way of securing older version of [[mhttpd]] (pre August 2015). Older versions of mhttpd are started using the -p port option e.g.
* mhttpd -D -p 8080 

To run a new version of mhttpd using an HTTP proxy, use the options provided to run the old (non-Mongoose) webserver on a given port, i.e.
* mhttpd --oldserver 8080 --nomg -D 

When using an SSL proxy, only access from the SSL proxy (and maybe some special trusted machines) should be permitted.
This is done using the "-a hostname" switch of [[mhttpd]]. Normally there will be only "-a localhost" switch, enabling access only for the local machine (where the SSL proxy is running). Additional "-a hostname" switches enable access from listed local machines. No "-a xxx" enables access from everywhere (defeating the purpose of the SSL proxy, unless access controls are enforced elsewhere, i.e. by a site firewall or by local firewall rules).

== Setting up an HTTP proxy ==
In this example, we use APACHE HTTPD to password-protect a typical midas/mhttpd and elog installation.

In this configuration, one uses the Linux stock httpd that accepts encrypted https:// connections and forwards them to mhttpd and elogd. Instead of (or in addition to) using mhttpd and elogd passwords, one configures password protection in httpd via the regular apache httpd password mechanisms (htpasswd, etc).

Recent versions of elogd do support SSL https:// connections, but if one is running an SSL proxy for anyway, it is simpler to run both through the same SSL proxy using the same SSL host certificate and the same httpd password file.

=== Restricting http: access to elogd ===
;Note
:Recent versions of elogd do support SSL https:// connections. The following information is for those using an HTTP proxy (see above).

For elogd, this is done using the "-n localhost" switch with enables only access from the same machine if present, or access from anywhere is absent (defeating the purpose of the SSL proxy, unless access controls are enforced elsewhere).

(It is recommended to run elogd from the same user as the main daq user and to keep elogd.cfg and all logbooks in the home directory of this user, where they are captured by the normal site backup system)

== Install standalone elog ==

* login into the user account that will run the elog
* cd $HOME/packages
* git clone https://bitbucket.org/ritt/elog
* cd elog
* make
* create new file start_elogd with this contents:
<pre>
#!/bin/sh

killall elogd
sleep 1
killall -KILL elogd
sleep 1
$HOME/packages/elog/elogd -n localhost -x -c $HOME/packages/elog/elogd.cfg -p 8082 -D

#end
</pre>
* chmod a+x start_elogd
* edit elogd.cfg to read:
<pre>
[global]
port = 8082
SMTP host = smtp.triumf.ca
URL = https://titan00.triumf.ca/elog/

Reverse sort = 1
Display Mode = full

#List Menu commands = New, Find, Admin, Help
#Menu commands = New, Edit, Reply, Find, Duplicate, Help

Entries Per Page = 30
Supress Email on edit = 1
Default encoding = 1
Page title = TITAN ELOG
Resolve host names = 1

Logfile = /home/titan/packages/elog/elogd.log
#Logging level = 3

[midas]

List page Title = T2K M11 MIDAS ELOG
Comment = T2K M11 MIDAS ELOG
Page Title = T2K M11 MIDAS ELOG
RSS Title = [$logbook - $type - $system] $subject, posted by $author

Attributes = Author, Subject, Run, Type, System
Show Attributes Edit = Run, Author, Subject, Type, System
Required Attributes = Author, Type, System, Subject

Options Type = Routine, Reply, Shift Summary, Modification, Question, Info, Problem
Options System = General, DAQ, Beamline

Preset Run = $shell(MIDASSYS=. /home/t2km11/packages/midas/linux/bin/odbedit -d Runinfo -c 'ls -v \"run number\"')

Preset On Reply Type = Reply
Preset On Reply Run = $shell(MIDASSYS=. /home/t2km11/packages/midas/linux/bin/odbedit -d Runinfo -c 'ls -v \"run number\"')

List Display = Date, Subject, Type, System, Author, ID
Quick Filter = Date, Type, ID

Remove on reply = Author
Quote on reply = 1

Use lock = 1

************* Email Functionality ****************

Use Email Subject = [T2KM11 - $System] $Subject
Omit Email To = 1

Email System General = xxx
</pre>
* ./start_elogd &
* firefox http://localhost:8082 # hould show the elog message index

To start elogd automatically when the machine is rebooted, login as root and
* add this text to /etc/rc.local:
<pre>
su - titan -c "/home/titan/packages/elog/start_elogd"
</pre>
* chmod a+x /etc/rc.local
* systemctl start rc-local

To import elog entries from the mhttpd elog, do this:

* cd ~/packages/elog/logbooks
* ln -s /home/t2km11/online/elog midas
* cd midas
* ~/packages/elog/elconv)

== Install https proxy ==

THESE INSTRUCTIONS ARE OBSOLETE, INSTEAD,
* GO HERE: https://daq00.triumf.ca/DaqWiki/index.php/Ubuntu#Install_apache_httpd_proxy_for_midas_and_elog
* AND GO HERE: https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux#Run_the_MIDAS_Web_Server

FOLLOWING INSTRUCTIONS ARE OBSOLETE...

* login as root to the https proxy machine
* cd ~root
* yum install mod_ssl
* yum install crypto-utils # see http://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_monitoring_of_HTTPS_certificates
* create a certificate request (replace ladd09 with your hostname): openssl req -new -nodes -newkey rsa:2048 -sha256 -out ladd09.csr -keyout ladd09.key (answer: CA, BC, Vancouver, TRIUMF, DAQ, ladd09.triumf.ca, email@email.com
* sign it by TRIUMF:
** mail -s "Certificate request" yourself@email.com < ladd09.csr
** forward this request to Andrew Daviel
** he will email the signed crt file, copy it to this system as ladd09.crt
* sign it yourself: openssl x509 -req -days 365 -sha256 -in ladd09.csr -signkey ladd09.key -out ladd09.crt
* (if the certificate expires, renew it by signing it again)
* Additional commands for working with certificates:
** explore the private key: openssl pkey -in ladd09.key -text -noout
** explore the certificate request: openssl req -in ladd00.csr -text -noout
** explore the certificate: openssl x509 -in ladd09.crt -noout -text
* move certificate files to proper system locations:
* mv ladd09.key /etc/pki/tls/private/
* mv ladd09.crt /etc/pki/tls/certs/
* if selinux is enabled, do this:
** restorecon -Rv /etc/pki/tls/certs/
** restorecon -Rv /etc/pki/tls/private/
** /usr/sbin/setsebool -P httpd_can_network_connect 1
* open /etc/httpd/conf.d/ssl.conf in a text editor, go to the very bottom and right before the "</VirtualHost>" entry, add following text:
<pre>
...
SSLCertificateFile /etc/pki/tls/certs/ladd09.crt
SSLCertificateKeyFile /etc/pki/tls/private/ladd09.key

ProxyPass /elog/ http://localhost:8082/ retry=1
ProxyPass / http://localhost:8080/ retry=1

<Location />

SSLRequireSSL
AuthType Basic
AuthName "password protected site"
Require valid-user

# create password file: touch /etc/httpd/htpasswd
# to add new user or change password: htpasswd /etc/httpd/htpasswd username
AuthUserFile /etc/httpd/htpasswd

</Location>
</VirtualHost>
...
</pre>
* comment out duplicate "SSLCertificateFile" and "SSLCertificateKeyFile" elsewhere in the file
* (optionally) If you got a certificate that is signed by DigiCert or RapidSSL then you'll need to add a line specifying the certificate chain file:
<pre>
...
SSLCertificateChainFile /etc/pki/tls/certs/DigiCertCA.crt
...
</pre>
* touch /etc/httpd/htpasswd
* htpasswd /etc/httpd/htpasswd midas # enter password midas
* chkconfig httpd on
* service httpd restart
* firewall-cmd --add-port=443/tcp --permanent
* firewall-cmd --reload
* firewall-cmd --list-all
* test it
** test the SSL proxy: https://host/ should yield the midas status page, https://host/elog/ should yield the elog message index
* in ODB, set "/Elog/URL" to "https://host/elog/"
* now from the midas status page, the "Elog" button should take us to the https Elog URL

In needed, enable user directories: https://blah/~user in ~user/public_html

* edit /etc/httpd/conf.d/userdir.conf, replace "UserDir disabled" with "UserDir enabled"
* setsebool -P httpd_enable_homedirs true
* systemctl restart httpd

== Setup the history mhttpd for faster access to history plots ==
When running an SSL proxy,
* start the main mhttpd (orange command for old mhttpd, green for new mhttpd with Mongoose(post August2015):
** "mhttpd -p 8071 -D" or
** "mhttpd -D --oldserver 8071 --nomg" 

* start the history mhttpd
** "mhttpd -p 8072 -D -H" or
** "mhttpd -D -H --oldserver 8072 --nomg" 
* set ODB /History/URL to "http://alphacpc09.cern.ch:8072/HS/"
* open the MIDAS status page
* go to the history section, try to open any history plot, observe that the history plot gif image loads correctly, inspect it's URL (use "copy image URL" or "view source", etc), it should point to port 8072 causing connection to the history mhttpd.
* continue with these instructions to setup history mhttpd access through an SSL proxy:
* setup SSL proxy access (required mod_proxy_html)
** login as root to the SSL proxy machine
** on SL5, install the missing mod_proxy_html httpd module:
** yum install httpd-devel libxml2-devel
** wget http://apache.webthing.com/mod_proxy_html/mod_proxy_html.tar.bz2
** tar xjvf mod_proxy_html.tar.bz2
** cd mod_proxy_html
** apxs -c -I. -I/usr/include/libxml2 -i mod_proxy_html.c
** apxs -c -I. -I/usr/include/libxml2 -i mod_xml2enc.c
** cd /etc/httpd/conf.d, add this to ssl.conf:

;before the ProxyPass statements:
<pre>
# proxy the MIDAS web servers
LoadModule xml2enc_module modules/mod_xml2enc.so
LoadModule proxy_html_module modules/mod_proxy_html.so
ProxyHTMLLinks a href
ProxyHTMLLinks link href
ProxyHTMLLinks img src
#ProxyHTMLEnable On
ProxyRequests off
</pre>
;after the ProxyPass statements:
<pre>
# ALPHA1 history access
ProxyPass /alpha1/history/ http://alphacpc09.cern.ch:8072/HS/ retry=1
ProxyPass /alpha1/ http://alphacpc09.cern.ch:8071/ retry=1

ProxyHTMLEnable On
ProxyHTMLURLMap http://alphacpc09.cern.ch:8072/HS/ /alpha1/history/
</pre>
;adjust:
*"alpha1" is the experiment name
*"alphacpc09.cern.ch" is the machine running mhttpd
*"8071" is the port number of the main mhttpd
** "mhttpd -p 8071 -D" or
** "mhttpd -D --oldserver 8071 --nomg" 
* "8072" is the port number of the history mhttpd
** "mhttpd -p 8072 -D -H" or
** "mhttpd -D -H --oldserver 8072 --nomg" 

[[Category:Installation]] [[Category:Buffer]] [[Category:Driver]]

Setup MIDAS experiment at TRIUMF

2026-09-29T16:10:49Z

Dfujimoto: /* Install MIDAS */

{{Pagelinks}}
== Introduction ==
This page describes setting up a MIDAS experiment at TRIUMF. This information can be adapted for other sites.

== Environment variables ==
* '''MIDASSYS''' Base directory of the MIDAS package, midas and mxml should be at the same level.
* '''MIDAS_EXPTAB''' Experiment definition file
* '''MIDAS_SERVER_HOST''' MIDAS host server name for remote midas connections.
* '''MIDAS_EXPT_NAME''' Experiment name

== Standard layout of MIDAS experiment ==
The following shows the directory layout of a standard MIDAS experiment:

/home/exptuser/
packages/
root <---- ROOT
mxml
mscb
midas/ <---- MIDAS
linux/{lib,bin} <---- binaries matching the selected 64-bit/32-bit flavour of ROOT
linux-m32/{lib,bin} <---- limited function 32-bit binaries for 32-bit frontend machines, build by "make linux32"
linux-m64/{lib,bin} <---- limited function 64-bit binaries (only needed if ROOT and linux/bin are 32-bit)
linux-arm/{lib,bin} <---- full function ARM cross-compiled using "make linuxarm"
linux-crosscompile/{lib,bin} <---- cross-compiled limited function binaries for PPC and ARM frontends (see Makefile)
rootana <---- ROOT analyzer
roody <---- graphical online histogram viewer for MIDAS and ROOTANA
online/
exptab <---- experiment definition
{.ODB,.SYSTEM,.SYSMSG,etc}.SHM <---- MIDAS shared memory save files
src <---- experiment frontend sources
bin,scripts
elog <---- MIDAS elog
history <---- MIDAS history
data -> /data/exptname/current <---- symlink to the data directory
/data/exptname/current <---- experiment data directory with ODB save files and MIDAS .mid/.mid.gz data files

== Prepare computers ==

On some operating systems, several MIDAS functions require administrator access:

* on el7 linux (SL7/CC7/CentOS7/RHEL7) - access to mhttpd port 8443 requires special firewall rules, see here: https://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29
* on el7 linux - access to mserver to run frontends and other programs on some other computer requires special firewall rules, see here: https://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29
* on el7 linux - on the frontend machines (and other machines that will connect to the mserver, the same firewall rule needs to be created (use the IP address of the machine running the mserver)

== Prepare the user account ==
<div id="NOTES"></div>

* Setup the user account for running this instance of midas. For machines part of the LADD cluster, follow these [http://daq-plone.triumf.ca/SM/docs/local/NewLaddUser] instructions.
* check that the account is using the /bin/bash shell
* make $HOME/.profile look like this:

<pre>
#!/bin/echo You must source

export SVN_EDITOR="emacs -nw"
export GIT_EDITOR="emacs -nw"
export MIDASSYS=$HOME/packages/midas
export ROOTANASYS=$HOME/packages/rootana
export MIDAS_EXPTAB=$HOME/online/exptab
#
# setup the MIDAS mserver
#
case `hostname` in
daq07*)
unset MIDAS_SERVER_HOST
;;
*)
export MIDAS_SERVER_HOST=daq07.triumf.ca:7070
;;
esac
#
# select 64-bit or 32-bit MIDAS and ROOT
#
case `uname -i` in
i386)
source /daq/daqshare/olchansk/root/root_v5.34.01_SL62_32/bin/thisroot.sh
export PATH=.:$MIDASSYS/linux-m32/bin:$PATH
;;
*)
#source /daq/daqshare/olchansk/root/root_v5.34.34_SL67_64/bin/thisroot.sh
source $HOME/packages/root/bin/thisroot.sh
export PATH=.:$MIDASSYS/linux/bin:$PATH
;;
esac
#
export PATH=.:$HOME/online/bin:$HOME/packages/roody/bin:$PATH
#
#end
</pre>

* mkdir $HOME/packages
* Logout and login again, for .cshrc changes to take effect

== Install ROOT ==

* Identify the Linux version: RH9 (Red Hat Linux 9), FC3 (Fedora Core 3), RHEL4/SL4 (Red Hat Enterprise LInux 4/Scientific Linux 4), SL5x, SL6x, (CentOS/CC/SL) el7x: more /etc/redhat-release
* Decide to use 32-bit or 64-bit ROOT ('uname -a')
* cd $HOME/packages
* ls -l /daq/daqshare/olchansk/root/ ### to see all available ROOT packages
* ln -s /daq/daqshare/olchansk/root/root_vNNN_VVV_BB root, where NNN is the latest available version of ROOT ("ls -l /daq/daqshare/olchansk/root"), VVV is the Linux version code (RH9, FC3, SL4, etc) and BB is "_32" or "_64" for 32-bit or 64-bit ROOT. For example: /daq/daqshare/olchansk/root/root_v5.10.00_SL40
* for example: ln -s /daq/daqshare/olchansk/root/root_v5.34.34_el72_64 $HOME/packages/root
* Check that ROOT works: "source $HOME/packages/root/bin/thisroot.sh; root". Type ".q" to exit root.

== Install MIDAS ==

* cd $HOME/packages
* (OBSOLETE) svn co svn+ssh://svn@savannah.psi.ch/repos/meg/midas/trunk midas, password "svn". (password has to be entered twice)
* (OBSOLETE) svn co svn+ssh://svn@savannah.psi.ch/repos/meg/mxml/trunk mxml
* git clone https://gitlab.triumf.ca/midas/midas.git --recursive
* (OBSOLETE) git clone https://gitlab.triumf.ca/midas/mxml.git
* (OBSOLETE) git clone https://gitlab.triumf.ca/midas/mscb.git
* (IF GITLAB IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/mxml.git
* (IF GITLAB IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/mscb.git
* (IF GITLAB IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/midas.git
* cd midas
* make
* (only if needed) make linux32 ### build the 32-bit MIDAS libraries
* ls -l linux/bin/odbedit ### check that odbedit has been created (do not run it yet)

You can see a list of other installation problems at [[Common problems & Debugging recipes]].

;NOTE 1
: Optional features in MIDAS can be explicitly disabled if desired when making MIDAS using the NO_xxx feature (NO_ROOT,NO_MYSQL,NO_ODBC,NO_SQLITE,NO_MSCB), e.g. "make NO_ROOT=1" to disable ROOT. These NO_xxx Makefile variables are only used to control autodetection.

;NOTE 2
: Since June 2019 the mxml and mscb packages are submodules of the midas package, so no need to clone them separately. If you have an existing clone of midas but not yet the submodules, you need

$ git submodule update --init --recursive

: To update both midas and the submodules, you need

$ git pull --recurse-submodules

== Install ROOTANA ==

* cd $HOME/packages
* git clone https://bitbucket.org/tmidas/rootana
* cd rootana
* make

== Install ROODY ==

* cd $HOME/packages
* git clone https://bitbucket.org/tmidas/roody
* cd roody
* make
* $HOME/packages/roody/bin/roody, run the program

== Install additional additional packages ==

* cd $HOME/packages
* git clone https://bitbucket.org/ttriumfdaq/vme
* git clone https://bitbucket.org/ttriumfdaq/frontends

== Build special versions of MIDAS ==

Build special versions of MIDAS for the case when some MIDAS programs, such as VME frontends, will run on a different computer that may have a different flavour of operating system, i.e. 32-bit linux or an older version of Scientific Linux.

* login to the computer where the frontends will run and:
* if it is a 32-bit linux: cd $HOME/packages/midas; make linux32
* if it is a 64-bit linux: cd $HOME/packages/midas; make linux64

* login to the host computer to cross-compile ARM code:
* if it is an ARM linux: cd $HOME/packages/midas; make linuxarm # may need to install ARM cross compilers

== Prepare VME hardware ==

Hardware check list:
* VME crate
* VME processor (supported are V77xx, V7805, V7865)
* On all VME modules, set the VME address jumpers as described here: http://daq-plone.triumf.ca/SM/docs/local/vme_jumpers
* run vmescan to confirm correct VME addresses
** cd $HOME/packages
** svn checkout https://ladd00.triumf.ca/svn/daqsvn/trunk/vme
** cd vme
** make
** ./vmescan.exe (or _gef.exe, depending on the VME driver in use)

== Install Universe-II VME driver (V7648, V7750, V7805, V7851) ==

* login as root (ssh root@localhost)
* follow instructions: https://www.triumf.info/wiki/DAQwiki/index.php/VME-CPU#V7648.2C_V7750.2C_V7805.2C_V7851_:_Setup_vme_universe_VME_drivers
* cd ~/packages/vme; vmescan.exe

== Install Tsi-148 VME driver (V7865) ==

* login as root (ssh root@localhost)
* follow instructions: https://www.triumf.info/wiki/DAQwiki/index.php/VME-CPU#V7865_and_XVB-602_:_Setup_gefvme.2Ftsi148_VME_drivers
* cd ~/packages/vme; vmescan_gef.exe

== Setup the experiment environment ==

* Decide which computer will host MIDAS (where MIDAS shared memory buffers will reside).
: This computer will run the [[mserver]], [[mlogger]] and [[mhttpd]] applications. (It is usually the machine where the MIDAS,ROOT etc. packages have been downloaded). It will be referred to as the host machine (localhost).

The environment is slightly different depending on whether all programs run on the host machine, or whether some programs run on remote host(s) :
=== ALL programs run on localhost ===
:If all programs run on the host machine (localhost), it is not necessary to run [[mserver]]. [[Environment Variables#MIDAS_SERVER_HOST|MIDAS_SERVER_HOST]] will not be assigned (see example .cshrc [[#Prepare the user account|above]]).

=== Some programs run on REMOTE host(s) ===
: '''IMPORTANT:'''
# Since August 2015 '''you must explicitly allow access for clients running on remote machines'''. To do this, follow the '''[[Security#MIDAS programs on remote machines|instructions here]]'''.
# The example code .cshrc ([[#Prepare the user account|see above]]) should be present on both host and remote machine(s). This will ensure that [[Environment Variables#MIDAS_SERVER_HOST|MIDAS_SERVER_HOST]] will NOT be set for the host machine (localhost), but on a remote machine, MIDAS_SERVER_HOST will be set to the MIDAS host machine.
# The client [[mserver]] must be started on the MIDAS host machine. Note that multiple experiments can run on the same host machine by starting several instances of [[mserver]] (one for each experiment) running with different ports (and .cshrc would be edited so that MIDAS_SERVER_HOST is set to the appropriate port for the experiment).

 
On the host machine:
* mkdir $HOME/online
* cd $HOME/online
* create directories for local programs, sources, elog and history: mkdir bin src elog history
* create data directory: mkdir -p /ladd/data1/t2kvme5/data; ln -s /ladd/data1/t2kvme5/data $HOME/online
* create the exptab file "$HOME/online/exptab" following the example below. The first entry (exptname) is the name if the DAQ system (MIDAS experiment name), the second entry (/home/USER/online) is the location of MIDAS shared memory buffers (by convention, $HOME/online), the third entry (kopio03) is your username.
<pre>exptname /home/kopio03/online kopio03</pre>
* logout and login again for all changes to take effect

== Setup experiment startup scripts ==
* login to the experiment host computer
* echo $MIDAS_SERVER_HOST ### to check correct value - should be blank
* create $HOME/online/bin/start_daq.sh, replacing XXX with the hostname of the machine running the experiment (and changing the mserver and mhttpd ports, as needed).

#!/bin/sh
# start_daq.sh
cd $HOME/online
#
case `hostname` in XXX*)
echo "Good, we are on XXX!"
;;
*)
echo "The start_daq script should be executed on XXX"
exit 1
;;
esac
#
odbedit -c clean
# start [[mhttpd]] on default port. (Mongoose https version - see [[mhttpd]] for other options)
mhttpd -D -a localhost -a XXX.triumf.ca # optionally restrict access to specified hosts
#
# start [[mserver]] on default port (use argument -p to use a different port)
mserver -D # access must now be specifically allowed - see [[#Setup the experiment environment|above]]

# OR ([[#NOTES|older MIDAS versions]])
# mhttpd -p 8081 -D -a localhost -a XXX.triumf.ca # optionally restrict access to specified hosts
# mserver -p 7071 -D -a localhost -a lxdragon01.triumf.ca -a lxdragon02.triumf.ca -a XXX.triumf.ca # optionally restrict access to specified hosts

#
mlogger -D
#end file

== Run the MIDAS Web Server ==
Let's start the MIDAS webserver for the first time:

Start [[mhttpd]] on the ''experiment host'' (localhost) like this:
[mhostpc] mhttpd
You will get the following messages:
[mhttpd,INFO] ODB subtree /Runinfo corrected successfully
Mongoose web server will listen on ports "8080r,8443s"
[mhttpd,ERROR] [mhttpd.cxx:17892:mongoose,ERROR] cannot find SSL certificate file "/home/agdaq/online/ssl_cert.pem"
[mhttpd,ERROR] [mhttpd.cxx:17893:mongoose,ERROR] please create SSL certificate file: openssl req -new -nodes -newkey rsa:2048 -sha256 -out ssl_cert.csr -keyout ssl_cert.key; openssl x509 -req -days 365 -sha256 -in ssl_cert.csr -signkey ssl_cert.key -out ssl_cert.pem; cat ssl_cert.key >> ssl_cert.pem
could not start the mongoose web server, see messages and midas.log, bye!

Create a self-signed certificate suitable for initial testing by executing the command printed by mhttpd:


[mhostpc] openssl req -new -nodes -newkey rsa:2048 -sha256 -out ssl_cert.csr -keyout ssl_cert.key; openssl x509 -req -days 365 -sha256 -in ssl_cert.csr -signkey ssl_cert.key -out ssl_cert.pem; cat ssl_cert.key >> ssl_cert.pem

For production use, you should create a properly signed certificate, see [[Mhttpd#Create an SSL certificate|create your own SSL certificate]] or you should run mhttpd behind an SSL proxy.

Run mhttpd again.

You will get the following messages:
[mhttpd,INFO] ODB subtree /Runinfo corrected successfully
Mongoose web server will listen on ports "8080r,8443s"
Mongoose web server will use SSL certificate file "/home/johnfoo/packages/midas/ssl_cert.pem"
[mhttpd,ERROR] [mhttpd.cxx:17633:mongoose,ERROR] mongoose web server cannot find password file "/home/johnfoo/online/htpasswd.txt"
[mhttpd,ERROR] [mhttpd.cxx:17634:mongoose,ERROR] please create password file: htdigest -c /home/johnfoo/online/htpasswd.txt Default midas
could not start the mongoose web server, see messages and midas.log, bye!

Create the password file by following the instructions printed by mhttpd. The http digest domain name is the experiment name, suggested default user name is "midas". You will be asked to type in a password


[mhostpc] htdigest -c /home/johnfoo/online/htpasswd.txt exptname midas
Adding password for midas in realm exptname.
New password:
Re-type new password:

It is a good idea to set the password file {{Filepath|path=htpasswd.txt}} readable and writable by owner only.

Now restart {{Utility|name=mhttpd}}
[mhostpc] mhttpd
Mongoose web server will listen on ports "8080r,8443s" **see note
Mongoose web server will use SSL certificate file "/home/suz/packages/midas/ssl_cert.pem"
Mongoose web server will use authentication realm "Default", password file "./htpasswd.txt"

Now point a web browser running on the same host computer (localhost) to https://localhost:8443
If the web browser is running on a different computer, go to URL of the form

https://mhostpc.triumf.ca:8443 (substitute your host machine name and domain for "mhostpc.triumf.ca")

If you are using the default SSL certificate you will probably get a message: "This Connection is Untrusted". Click "I understand the risks" and add an exception. This is because the test certificate is self-signed. Then confirm an exception.

If instead you get a "connection refused" error, the midas host pc may have the firewall enabled. To make a firewall exception for MIDAS, follow instructions here http://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29

You should then see an authentication box asking you for the user name and password. The user name is "midas". Enter the password you just created. The Midas [[Status Page]] should appear with multiple buttons for run control as well as equipment listing (no equipments will be listed as yet) and application listings. Please refer to [[mhttpd]] (the MIDAS Web-based Run Control utility) for further information. You can start and stop runs from the main status page, and use the [[ODB Page]] to access the database (ODB).

; Note
: Default ports of 8080 and 8443 are used by [[mhttpd]]. If these ports are in use on your machine, start mhttpd with alternative ports, e.g.
[mhostpc] mhttpd --https 8448 --http 8089
: or see [[Mhttpd#Usage]] to change the default ports.

== Setup experiment database (ODB) ==

* run $HOME/online/bin/start_daq.sh

* odbedit, run these commands: (replace user names and directory names)
<pre>
# set "/Logger/Message file" "/home/kopio03/online/midas.log" # obsolete
set "/Logger/Data Dir" "/home/kopio03/online/data"
set "/Logger/History dir" "/home/kopio03/online/history"
create STRING "/Logger/Elog dir"
set "/Logger/Elog dir" "/home/kopio03/online/elog"
set "/Logger/ODB dump file" "/home/kopio03/online/history/run%05d.xml"
set "/Logger/ODB dump" "y"
set "/Logger/Channels/0/Settings/Filename" "run%05dsub%03d.mid.gz"
set "/Logger/Channels/0/Settings/Subrun byte limit" "1000000000"
set "/Logger/Channels/0/Settings/Compression" 1
set "/Logger/Channels/0/Settings/ODB Dump" "y"
set "/Programs/Logger/Required" y
set "/Programs/Logger/Start command" "mlogger -D"
set "/Programs/fevme/Required" "y"
set "/Programs/fevme/Start command" "ssh -n lxdaq09 $HOME/online/src/fevme_gef.exe -O"
exit
</pre>
* open web browser e.g. firefox.
* go to the midas status page at https://localhost:8443 (default port).
** if running [[mhttpd]] with Mongoose HTTPS/OpenSSL (the default) for the first time, you will need to create a password file. Follow the instructions (see [[mhttpd#HTTPS/SSL server (Mongoose)]] for details).
** For other options (i.e. HTTPS/SSL proxy) see [[#Secure MIDAS and ELOG Web access]]
*OR open the midas status page at http://localhost:8081 ([[#NOTES|older MIDAS versions]])
* midas status page will show most stuff "red" as nothing is running yet
* DON'T DO THIS YET run ./fevme.exe (on the computer with the VME interface, could be different from computer hosting the experiment), observe that corresponding equipments have been created
* save the url bookmark to the "personal toolbar"
* go to the Programs page, stop mlogger, stop fevme, start mlogger, start fevme
* go to the Status page, start run, stop run
* go back to the Status page, everything should be green
* start a run
* send signals to the ADC gate
* you should be getting events
* to look at data, proceed with setting up the [[ROOTANA|ROOT Analyzer]].

== Start DAQ programs at boot time ==

* add this to /etc/rc.local (replace username and location of the start_daq script)
<pre>
su - alpha -c /home/alpha/online/bin/start_daq.sh
</pre>

== Setup local software version control ==

Version control for experiment source code is setup using "git" (http://git-scm.com/)

* cd $HOME/online
* git init
* git add exptab
* git add bin/start_daq.sh
* git add .gitignore ### contents can be
<pre>
*~
*.o
*.exe
</pre>
* git add src/Makefile src/*.cxx ...
* git commit -a

== Adjust MIDAS buffer sizes ==

Default MIDAS SYSTEM buffer size is 8 Mbytes, fairly small for high-data-rate experiments. The rule of thumb is to have at least a few seconds worth of buffer space available. For example, if event size is 10 Kbytes and the event rate is 1 kHz, data rate is 10*10^3*1*10^3 = 10 Mbytes/sec. To buffer 10 seconds of data we need 100 Mbytes of buffer space.

To resize the MIDAS event buffers (SYSTEM, etc) do this:
* stop all frontends, stop mlogger
* start odbedit:
** cd "/Experiment/Buffer sizes"
** set SYSTEM 100000000
* run "mdump -z SYSTEM"
* if mdump complains about the size of .SYSTEM.SHM, remove it, try again.
* ls -l /dev/shm ### to observe that the size of shared memory is correct

== Secure MIDAS and ELOG Web access ==
In versions prior to May 2015, the default web access to MIDAS and ELOG uses the "http:" protocol which is insecure. In this case, all information is transmitted as clear text meaning that secret, confidential and sensitive information (such as the MIDAS and ELOG passwords and usernames) can be stolen "easily". This means that even "password protected" MIDAS and ELOG pages are not really protected if accessed using the "http" method.

Better security for HTTP is gained by using a password protected '''SSL (https) proxy'''. (It does not provide absolute security because of remaining problems with the security of SSL certificates, security of passwords, etc). Setting up an SSL (https) proxy is described [[#Setting up an HTTP proxy|below]].

Since May 2015, an ''alternative secure option'' to setting up an HTTP proxy is available to users of MIDAS. Recent versions of elogd (ELOG) do support SSL https:// connections, and [[#mhttpd with HTTPS/SSL server (Mongoose)]] is now available. This option is the default, and provides a similar level of security to an HTTP proxy.

See [[Security#Web Access]] for a comparison of these two secure options.

=== mhttpd with HTTPS/SSL server (Mongoose) ===

Since May 2015 the MIDAS web server [[mhttpd]] is explicitly linked with OpenSSL to provide secure HTTPS connections via the [https://bitbucket.org/tmidas/midas/src/ecb9a8537448a8a43f7f9a2bfdb82e578208cde3/doc/mongoose/?at=develop Mongoose] web server (see [[mhttpd]]). With this version, default web access to MIDAS uses the "https" protocol. Web access to {{Utility|name=mhttpd}} can be restricted by using the "-a hostname" switch of [[mhttpd]]. The first time {{Utility|name=mhttpd}} is run, a password file must be created. An SSL certificate is also required. See [[mhttpd#HTTPS/SSL server (Mongoose)|HTTPS/SSL server (Mongoose)]] for instructions.

=== mhttpd using an HTTPS/SSL proxy ===

THESE INSTRUCTIONS ARE WRONG, DO NOT DO THIS. See instead [https://daq00.triumf.ca/MidasWiki/index.php/Setup_MIDAS_experiment_at_TRIUMF#Install_https_proxy this section].

An [[#Setting up an HTTP proxy|HTTP proxy]] must be set up. This is the only way of securing older version of [[mhttpd]] (pre August 2015). Older versions of mhttpd are started using the -p port option e.g.
* mhttpd -D -p 8080 

To run a new version of mhttpd using an HTTP proxy, use the options provided to run the old (non-Mongoose) webserver on a given port, i.e.
* mhttpd --oldserver 8080 --nomg -D 

When using an SSL proxy, only access from the SSL proxy (and maybe some special trusted machines) should be permitted.
This is done using the "-a hostname" switch of [[mhttpd]]. Normally there will be only "-a localhost" switch, enabling access only for the local machine (where the SSL proxy is running). Additional "-a hostname" switches enable access from listed local machines. No "-a xxx" enables access from everywhere (defeating the purpose of the SSL proxy, unless access controls are enforced elsewhere, i.e. by a site firewall or by local firewall rules).

== Setting up an HTTP proxy ==
In this example, we use APACHE HTTPD to password-protect a typical midas/mhttpd and elog installation.

In this configuration, one uses the Linux stock httpd that accepts encrypted https:// connections and forwards them to mhttpd and elogd. Instead of (or in addition to) using mhttpd and elogd passwords, one configures password protection in httpd via the regular apache httpd password mechanisms (htpasswd, etc).

Recent versions of elogd do support SSL https:// connections, but if one is running an SSL proxy for anyway, it is simpler to run both through the same SSL proxy using the same SSL host certificate and the same httpd password file.

=== Restricting http: access to elogd ===
;Note
:Recent versions of elogd do support SSL https:// connections. The following information is for those using an HTTP proxy (see above).

For elogd, this is done using the "-n localhost" switch with enables only access from the same machine if present, or access from anywhere is absent (defeating the purpose of the SSL proxy, unless access controls are enforced elsewhere).

(It is recommended to run elogd from the same user as the main daq user and to keep elogd.cfg and all logbooks in the home directory of this user, where they are captured by the normal site backup system)

== Install standalone elog ==

* login into the user account that will run the elog
* cd $HOME/packages
* git clone https://bitbucket.org/ritt/elog
* cd elog
* make
* create new file start_elogd with this contents:
<pre>
#!/bin/sh

killall elogd
sleep 1
killall -KILL elogd
sleep 1
$HOME/packages/elog/elogd -n localhost -x -c $HOME/packages/elog/elogd.cfg -p 8082 -D

#end
</pre>
* chmod a+x start_elogd
* edit elogd.cfg to read:
<pre>
[global]
port = 8082
SMTP host = smtp.triumf.ca
URL = https://titan00.triumf.ca/elog/

Reverse sort = 1
Display Mode = full

#List Menu commands = New, Find, Admin, Help
#Menu commands = New, Edit, Reply, Find, Duplicate, Help

Entries Per Page = 30
Supress Email on edit = 1
Default encoding = 1
Page title = TITAN ELOG
Resolve host names = 1

Logfile = /home/titan/packages/elog/elogd.log
#Logging level = 3

[midas]

List page Title = T2K M11 MIDAS ELOG
Comment = T2K M11 MIDAS ELOG
Page Title = T2K M11 MIDAS ELOG
RSS Title = [$logbook - $type - $system] $subject, posted by $author

Attributes = Author, Subject, Run, Type, System
Show Attributes Edit = Run, Author, Subject, Type, System
Required Attributes = Author, Type, System, Subject

Options Type = Routine, Reply, Shift Summary, Modification, Question, Info, Problem
Options System = General, DAQ, Beamline

Preset Run = $shell(MIDASSYS=. /home/t2km11/packages/midas/linux/bin/odbedit -d Runinfo -c 'ls -v \"run number\"')

Preset On Reply Type = Reply
Preset On Reply Run = $shell(MIDASSYS=. /home/t2km11/packages/midas/linux/bin/odbedit -d Runinfo -c 'ls -v \"run number\"')

List Display = Date, Subject, Type, System, Author, ID
Quick Filter = Date, Type, ID

Remove on reply = Author
Quote on reply = 1

Use lock = 1

************* Email Functionality ****************

Use Email Subject = [T2KM11 - $System] $Subject
Omit Email To = 1

Email System General = xxx
</pre>
* ./start_elogd &
* firefox http://localhost:8082 # hould show the elog message index

To start elogd automatically when the machine is rebooted, login as root and
* add this text to /etc/rc.local:
<pre>
su - titan -c "/home/titan/packages/elog/start_elogd"
</pre>
* chmod a+x /etc/rc.local
* systemctl start rc-local

To import elog entries from the mhttpd elog, do this:

* cd ~/packages/elog/logbooks
* ln -s /home/t2km11/online/elog midas
* cd midas
* ~/packages/elog/elconv)

== Install https proxy ==

THESE INSTRUCTIONS ARE OBSOLETE, INSTEAD,
* GO HERE: https://daq00.triumf.ca/DaqWiki/index.php/Ubuntu#Install_apache_httpd_proxy_for_midas_and_elog
* AND GO HERE: https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux#Run_the_MIDAS_Web_Server

FOLLOWING INSTRUCTIONS ARE OBSOLETE...

* login as root to the https proxy machine
* cd ~root
* yum install mod_ssl
* yum install crypto-utils # see http://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_monitoring_of_HTTPS_certificates
* create a certificate request (replace ladd09 with your hostname): openssl req -new -nodes -newkey rsa:2048 -sha256 -out ladd09.csr -keyout ladd09.key (answer: CA, BC, Vancouver, TRIUMF, DAQ, ladd09.triumf.ca, email@email.com
* sign it by TRIUMF:
** mail -s "Certificate request" yourself@email.com < ladd09.csr
** forward this request to Andrew Daviel
** he will email the signed crt file, copy it to this system as ladd09.crt
* sign it yourself: openssl x509 -req -days 365 -sha256 -in ladd09.csr -signkey ladd09.key -out ladd09.crt
* (if the certificate expires, renew it by signing it again)
* Additional commands for working with certificates:
** explore the private key: openssl pkey -in ladd09.key -text -noout
** explore the certificate request: openssl req -in ladd00.csr -text -noout
** explore the certificate: openssl x509 -in ladd09.crt -noout -text
* move certificate files to proper system locations:
* mv ladd09.key /etc/pki/tls/private/
* mv ladd09.crt /etc/pki/tls/certs/
* if selinux is enabled, do this:
** restorecon -Rv /etc/pki/tls/certs/
** restorecon -Rv /etc/pki/tls/private/
** /usr/sbin/setsebool -P httpd_can_network_connect 1
* open /etc/httpd/conf.d/ssl.conf in a text editor, go to the very bottom and right before the "</VirtualHost>" entry, add following text:
<pre>
...
SSLCertificateFile /etc/pki/tls/certs/ladd09.crt
SSLCertificateKeyFile /etc/pki/tls/private/ladd09.key

ProxyPass /elog/ http://localhost:8082/ retry=1
ProxyPass / http://localhost:8080/ retry=1

<Location />

SSLRequireSSL
AuthType Basic
AuthName "password protected site"
Require valid-user

# create password file: touch /etc/httpd/htpasswd
# to add new user or change password: htpasswd /etc/httpd/htpasswd username
AuthUserFile /etc/httpd/htpasswd

</Location>
</VirtualHost>
...
</pre>
* comment out duplicate "SSLCertificateFile" and "SSLCertificateKeyFile" elsewhere in the file
* (optionally) If you got a certificate that is signed by DigiCert or RapidSSL then you'll need to add a line specifying the certificate chain file:
<pre>
...
SSLCertificateChainFile /etc/pki/tls/certs/DigiCertCA.crt
...
</pre>
* touch /etc/httpd/htpasswd
* htpasswd /etc/httpd/htpasswd midas # enter password midas
* chkconfig httpd on
* service httpd restart
* firewall-cmd --add-port=443/tcp --permanent
* firewall-cmd --reload
* firewall-cmd --list-all
* test it
** test the SSL proxy: https://host/ should yield the midas status page, https://host/elog/ should yield the elog message index
* in ODB, set "/Elog/URL" to "https://host/elog/"
* now from the midas status page, the "Elog" button should take us to the https Elog URL

In needed, enable user directories: https://blah/~user in ~user/public_html

* edit /etc/httpd/conf.d/userdir.conf, replace "UserDir disabled" with "UserDir enabled"
* setsebool -P httpd_enable_homedirs true
* systemctl restart httpd

== Setup the history mhttpd for faster access to history plots ==
When running an SSL proxy,
* start the main mhttpd (orange command for old mhttpd, green for new mhttpd with Mongoose(post August2015):
** "mhttpd -p 8071 -D" or
** "mhttpd -D --oldserver 8071 --nomg" 

* start the history mhttpd
** "mhttpd -p 8072 -D -H" or
** "mhttpd -D -H --oldserver 8072 --nomg" 
* set ODB /History/URL to "http://alphacpc09.cern.ch:8072/HS/"
* open the MIDAS status page
* go to the history section, try to open any history plot, observe that the history plot gif image loads correctly, inspect it's URL (use "copy image URL" or "view source", etc), it should point to port 8072 causing connection to the history mhttpd.
* continue with these instructions to setup history mhttpd access through an SSL proxy:
* setup SSL proxy access (required mod_proxy_html)
** login as root to the SSL proxy machine
** on SL5, install the missing mod_proxy_html httpd module:
** yum install httpd-devel libxml2-devel
** wget http://apache.webthing.com/mod_proxy_html/mod_proxy_html.tar.bz2
** tar xjvf mod_proxy_html.tar.bz2
** cd mod_proxy_html
** apxs -c -I. -I/usr/include/libxml2 -i mod_proxy_html.c
** apxs -c -I. -I/usr/include/libxml2 -i mod_xml2enc.c
** cd /etc/httpd/conf.d, add this to ssl.conf:

;before the ProxyPass statements:
<pre>
# proxy the MIDAS web servers
LoadModule xml2enc_module modules/mod_xml2enc.so
LoadModule proxy_html_module modules/mod_proxy_html.so
ProxyHTMLLinks a href
ProxyHTMLLinks link href
ProxyHTMLLinks img src
#ProxyHTMLEnable On
ProxyRequests off
</pre>
;after the ProxyPass statements:
<pre>
# ALPHA1 history access
ProxyPass /alpha1/history/ http://alphacpc09.cern.ch:8072/HS/ retry=1
ProxyPass /alpha1/ http://alphacpc09.cern.ch:8071/ retry=1

ProxyHTMLEnable On
ProxyHTMLURLMap http://alphacpc09.cern.ch:8072/HS/ /alpha1/history/
</pre>
;adjust:
*"alpha1" is the experiment name
*"alphacpc09.cern.ch" is the machine running mhttpd
*"8071" is the port number of the main mhttpd
** "mhttpd -p 8071 -D" or
** "mhttpd -D --oldserver 8071 --nomg" 
* "8072" is the port number of the history mhttpd
** "mhttpd -p 8072 -D -H" or
** "mhttpd -D -H --oldserver 8072 --nomg" 

[[Category:Installation]] [[Category:Buffer]] [[Category:Driver]]

Introduction

2026-09-29T16:09:15Z

Dfujimoto: /* History */

== What is Midas ==
"MIDAS" is an acronym for Maximum Integrated Data Acquisition System.

MIDAS is a general-purpose system for event-based data acquisition in small and medium scale Physics experiments. It is an on-going development at the Paul Scherrer Institute (Switzerland) and at TRIUMF (Canada), since 1993.

''' It is not to be confused with MIDAS (Multi Instance Data Acquisition System) from UK, or any of the MIDAS (Mobile Instrumentation Data Acquisition System), or MIDAS digital and analogue consoles and finally with MIDAS Brakes and Mufflers! '''

MIDAS is based on a modular networking capability and a central database system. MIDAS consists of a C library and several applications, which can run on many different platforms (i.e. operating systems) mainly UNIX-like based (we use to support Windows, VMS...). While the system is already in use in many laboratories, the development continues with addition of new features and tools. Recent developments involve multi-threading, FGPA/Linux support, MSCB extension.

==What Midas can do for you ==
In a few words, Midas can:
* '''Collect data from local and/or remote clients.'''
* Provides a mean to configure the hardware managed by any of the client ([[Online_Database| Online database]])
* Manages the [[Run_States_and_Transitions | run transitions]] functions during run operation (start, stop, etc.)
* Provides a set of essential [[Application_listing| Applications]] to control and monitor a data acquisition sequence.
* Implements [[Feature_listing | features ]]) specific to Data Aquisition for physics experiment.
* Records the collected data to various [[mlogger|storage media]].
* Provides programming interfaces of the data stream to different analysis packages ([[ROOTANA]], [http://bitbucket.org/tmidas/manalyzer/src/master/ manalyzer]).
* Provides a web interface and tools to build custom display for run monitoring and control ([[mhttpd]], [[Status_Page]]).

==History==
In the early '90s, based on a previous Data Acquisition system running under MS-DOS with network capability (HIX), Dr. Stefan Ritt at Paul Sherrer Institute PSI (Switzerland) started coding a new set of applications which would be OS independent. At that time, OS such as VMS, ULTRIX, VxWorks and Windows were available.
The first deployment of Midas was for the "Canadian High Acceptance Orbital Spectrometer" (CHAOS) experimental facility at TRIUMF (Canada). Network based, the data were collected from a VME processor running VxWorks (collecting CAMAC, FastBus & VME data) and sending them to a backend computer running VMS and later ULTRIX.

Since then, Midas has been deployed on all major experiments at TRIUMF and PSI. It is also used around the world in over 80 locations. From simple workbench test setups to world class experiments such as PiBeta - muon decay [http://meg.icepp.s.u-tokyo.ac.jp/ '''MEG'''] (PSI, Switzerland) - trapping antihydrogen atoms [http://alpha.web.cern.ch/ '''ALPHA''', '''ALPHA-g'''] (CERN, Switzerland) - neutrino oscillation [http://t2k-experiment.org/ '''T2K'''](J-Parc, Japan) - Decay study Pienu - precision muon measurement Twist (TRIUMF, Canada) - Dark Matter search [http://deap.phy.queensu.ca/ '''DEAP'''] (Sudbury, Canada) - Neutron Capture DANCE (Los Alamos, USA), - Dark Matter search [https://surfacetreatments.infn.it/projects/darkside-20k-project/ '''DARKIDE-20K'''] (LNGS, Italy), - P-ONE neutrino telescope [https://www.pacific-neutrino.org/ '''P-ONE'''] (Cascadia Basin, Canada), - P-ONE neutrino telescope [https://nexo.llnl.gov/ '''nEXO'''] (Subury, Canada)

Midas has demonstrated its versatile capabilities and proven to be a mature and modern Data Acquisition Software package.
A non-exhaustive list of experiments can be found [https://midas.triumf.ca/MidasWiki/index.php/MIDAS_around_the_world here].

=== Recent Developments ===https://www.pacific-neutrino.org/p-one
* July 2013 : Stefan Ritt visit at Triumf
** Midas code from local SVN repository to Bitbucket cloud based GIT repository [https://bitbucket.org/tmidas/midas Tmidas]
** Implementation of multi-threading transition
** New Midas web page
** JSON, JSON-P for custom Midas web page support
*Jan 2014 : New History Logging scheme "FILE"
* July 2015 : Stefan Ritt visit at Triumf
* August 2015 : Important Network Security upgrades:
** Default MIDAS is now secure (clients must run on localhost only)
** Network connections must be explicity allowed (see [[Security]])
** [[mhttpd]] uses secure HTTPS connections
* January 2016 : mjson-rpc functions added to Javascript libary [[mhttpd.js]] - see [[mjsonrpc]]
* July 2017
** New web page layout with menu buttons on left side and alarm and message display on every page
** New custom pages based on the mjson-rpc layer - see [[Custom_Page|Custom Pages 2017]]
* September 2026
** Migrated to [https://gitlab.triumf.ca/midas/midas TRIUMF-hosted gitlab]

==[[Overall Midas system diagram]]==
A graphical view of the generic acquisition components forming the MIDAS DAQ

==[[Midas Core]]==
A general description of the different components composing the Midas Data Acquisition package

ROOTANA

2026-09-29T16:08:18Z

Dfujimoto: /* Contacts */

{{Pagelinks}}

= Links =

* [[ROOT]]
* https://gitlab.triumf.ca/midas/rootana
* git clone https://gitlab.triumf.ca/midas/rootana.git
* git clone http://ladd00.triumf.ca/~daqweb/git/rootana.git
* (obsolete) [https://ladd00.triumf.ca/viewvc/rootana/trunk ROOTANA SVN reporitory] (username svn, password svn)
* [http://ladd00.triumf.ca/~daqweb/doc/rootana/html ROOTANA Doxygen documentation]

= Download and Quick start =

<pre>
echo $ROOTSYS # check correct installation of ROOT
root-config # check correct installation of ROOT

git clone https://gitlab.triumf.ca/midas/rootana.git
cd rootana
make
make dox

more analyzer.cxx
more event_dump.cxx
</pre>

= Introduction =

Welcome to ROOTANA, the ROOT (https://root.cern.ch) based data analysis package for the MIDAS Data Acquisition system (https://midas.triumf.ca).

The ROOTANA package includes 5 major components:
* a standalone library for reading and writing data files in the MIDAS .mid format, and for decoding the XML dump of MIDAS ODB typically embedded in every MIDAS data file. (libMidasInterface)
* a C++ interface class for connecting to an active MIDAS experiment, accessing ODB (read and write) and getting event data. (libMidasInterface)
* a set of C++ classes for exporting ROOT histogram and other objects to an external viewer for interactive visualization of live data, typically using the ROODY histogram viewer or using a standard web browser (experimental feature). (libMidasServer, libNetDirectory, libXmlServer)
* simple examples of using these components (a graphical analyzer, an event dump and an event skim programs) (analyzer.cxx, event_dump.cxx, event_skim.cxx)
* a full featured framework for graphical data analysis including code to unpack typical VME and CAMAC modules ( see [[rootana Analyzer Framework]] and [[rootana Display Framework]])

The ROOTANA package can be used without installing ROOT and MIDAS:
* without both ROOT and MIDAS one can only read (and write) existing .mid files. Only event_dump and event_skim will be built.
* if only the MIDAS package is installed, access to live data and access to live ODB becomes possible. Only event_dump and event_skim will be built.
* decoding of XML ODB dumps embedded in MIDAS files requires ROOT TXML and TDOMParser components (a version of XML decoder using libxml2 used to exist in the past).
* if the ROOT package is installed but MIDAS is absent, full function of ROOTANA is available, except for access to live data. This mode is suitable for offline data analysis, i.e. the user has a copy of MIDAS data files on their laptop and wants to analyze them.

The sources for ROOTANA are managed at https://gitlab.triumf.ca/midas/rootana.

= Components =

== MIDAS Interface ==

These classes provide the actual access to MIDAS data

* TMidasEvent: a class encapulating a MIDAS event.
* TMidasFile: a class encapsulating a MIDAS file
* TMidasOnline: a class that allows access to online MIDAS events; this class requires that the MIDAS libraries be installed
* VirtualODB: a class that provides access to ODB variables, either through ODB dumps at the start of files or by directly connecting to the online ODB.

== XML Server ==

xmlServer & co

== example programs ==

* event_dump
* event_skim
* standalone graphical analyzer.cxx

== ROOT Analyzer framework ==

rootana provides low-level access to the MIDAS data files and online MIDAS data. Using these low-level functionality users can design whatever sort of programs they want for accessing and analyzing the data.

rootana also provides a more structured, higher-level framework for organizing analysis programs. The framework includes a standard event loop class and a standard display program. Details on this higher-level framework available in [[rootana Analyzer Framework]] and [[rootana Display Framework]].

= Contacts =

* Before conversion from svn to git, ROOTANA was managed by members of the TRIUMF DAQ group
* The git version hosted on the TRIUMF gitlab is managed by the members of the MIDAS developers group.
* to report bugs, request improvements, contribute bug fixes - please go to the ROOTANA issue tracker https://gitlab.triumf.ca/midas/rootana/-/work_items
* for questions and discussion - please go to the MIDAS discussion forum https://midas.triumf.ca/forum

ROOTANA

2026-09-29T16:07:46Z

Dfujimoto: /* Introduction */

{{Pagelinks}}

= Links =

* [[ROOT]]
* https://gitlab.triumf.ca/midas/rootana
* git clone https://gitlab.triumf.ca/midas/rootana.git
* git clone http://ladd00.triumf.ca/~daqweb/git/rootana.git
* (obsolete) [https://ladd00.triumf.ca/viewvc/rootana/trunk ROOTANA SVN reporitory] (username svn, password svn)
* [http://ladd00.triumf.ca/~daqweb/doc/rootana/html ROOTANA Doxygen documentation]

= Download and Quick start =

<pre>
echo $ROOTSYS # check correct installation of ROOT
root-config # check correct installation of ROOT

git clone https://gitlab.triumf.ca/midas/rootana.git
cd rootana
make
make dox

more analyzer.cxx
more event_dump.cxx
</pre>

= Introduction =

Welcome to ROOTANA, the ROOT (https://root.cern.ch) based data analysis package for the MIDAS Data Acquisition system (https://midas.triumf.ca).

The ROOTANA package includes 5 major components:
* a standalone library for reading and writing data files in the MIDAS .mid format, and for decoding the XML dump of MIDAS ODB typically embedded in every MIDAS data file. (libMidasInterface)
* a C++ interface class for connecting to an active MIDAS experiment, accessing ODB (read and write) and getting event data. (libMidasInterface)
* a set of C++ classes for exporting ROOT histogram and other objects to an external viewer for interactive visualization of live data, typically using the ROODY histogram viewer or using a standard web browser (experimental feature). (libMidasServer, libNetDirectory, libXmlServer)
* simple examples of using these components (a graphical analyzer, an event dump and an event skim programs) (analyzer.cxx, event_dump.cxx, event_skim.cxx)
* a full featured framework for graphical data analysis including code to unpack typical VME and CAMAC modules ( see [[rootana Analyzer Framework]] and [[rootana Display Framework]])

The ROOTANA package can be used without installing ROOT and MIDAS:
* without both ROOT and MIDAS one can only read (and write) existing .mid files. Only event_dump and event_skim will be built.
* if only the MIDAS package is installed, access to live data and access to live ODB becomes possible. Only event_dump and event_skim will be built.
* decoding of XML ODB dumps embedded in MIDAS files requires ROOT TXML and TDOMParser components (a version of XML decoder using libxml2 used to exist in the past).
* if the ROOT package is installed but MIDAS is absent, full function of ROOTANA is available, except for access to live data. This mode is suitable for offline data analysis, i.e. the user has a copy of MIDAS data files on their laptop and wants to analyze them.

The sources for ROOTANA are managed at https://gitlab.triumf.ca/midas/rootana.

= Components =

== MIDAS Interface ==

These classes provide the actual access to MIDAS data

* TMidasEvent: a class encapulating a MIDAS event.
* TMidasFile: a class encapsulating a MIDAS file
* TMidasOnline: a class that allows access to online MIDAS events; this class requires that the MIDAS libraries be installed
* VirtualODB: a class that provides access to ODB variables, either through ODB dumps at the start of files or by directly connecting to the online ODB.

== XML Server ==

xmlServer & co

== example programs ==

* event_dump
* event_skim
* standalone graphical analyzer.cxx

== ROOT Analyzer framework ==

rootana provides low-level access to the MIDAS data files and online MIDAS data. Using these low-level functionality users can design whatever sort of programs they want for accessing and analyzing the data.

rootana also provides a more structured, higher-level framework for organizing analysis programs. The framework includes a standard event loop class and a standard display program. Details on this higher-level framework available in [[rootana Analyzer Framework]] and [[rootana Display Framework]].

= Contacts =

* Before conversion from svn to git, ROOTANA was managed by members of the TRIUMF DAQ group
* The git version hosted on bitbucket is managed by the members of the MIDAS developers group.
* to report bugs, request improvements, contribute bug fixes - please go to the ROOTANA issue tracker https://bitbucket.org/tmidas/rootana/issues
* for questions and discussion - please go to the MIDAS discussion forum https://midas.triumf.ca/forum

ROOTANA

2026-09-29T16:07:25Z

Dfujimoto: /* Download and Quick start */

{{Pagelinks}}

= Links =

* [[ROOT]]
* https://gitlab.triumf.ca/midas/rootana
* git clone https://gitlab.triumf.ca/midas/rootana.git
* git clone http://ladd00.triumf.ca/~daqweb/git/rootana.git
* (obsolete) [https://ladd00.triumf.ca/viewvc/rootana/trunk ROOTANA SVN reporitory] (username svn, password svn)
* [http://ladd00.triumf.ca/~daqweb/doc/rootana/html ROOTANA Doxygen documentation]

= Download and Quick start =

<pre>
echo $ROOTSYS # check correct installation of ROOT
root-config # check correct installation of ROOT

git clone https://gitlab.triumf.ca/midas/rootana.git
cd rootana
make
make dox

more analyzer.cxx
more event_dump.cxx
</pre>

= Introduction =

Welcome to ROOTANA, the ROOT (https://root.cern.ch) based data analysis package for the MIDAS Data Acquisition system (https://midas.triumf.ca).

The ROOTANA package includes 5 major components:
* a standalone library for reading and writing data files in the MIDAS .mid format, and for decoding the XML dump of MIDAS ODB typically embedded in every MIDAS data file. (libMidasInterface)
* a C++ interface class for connecting to an active MIDAS experiment, accessing ODB (read and write) and getting event data. (libMidasInterface)
* a set of C++ classes for exporting ROOT histogram and other objects to an external viewer for interactive visualization of live data, typically using the ROODY histogram viewer or using a standard web browser (experimental feature). (libMidasServer, libNetDirectory, libXmlServer)
* simple examples of using these components (a graphical analyzer, an event dump and an event skim programs) (analyzer.cxx, event_dump.cxx, event_skim.cxx)
* a full featured framework for graphical data analysis including code to unpack typical VME and CAMAC modules ( see [[rootana Analyzer Framework]] and [[rootana Display Framework]])

The ROOTANA package can be used without installing ROOT and MIDAS:
* without both ROOT and MIDAS one can only read (and write) existing .mid files. Only event_dump and event_skim will be built.
* if only the MIDAS package is installed, access to live data and access to live ODB becomes possible. Only event_dump and event_skim will be built.
* decoding of XML ODB dumps embedded in MIDAS files requires ROOT TXML and TDOMParser components (a version of XML decoder using libxml2 used to exist in the past).
* if the ROOT package is installed but MIDAS is absent, full function of ROOTANA is available, except for access to live data. This mode is suitable for offline data analysis, i.e. the user has a copy of MIDAS data files on their laptop and wants to analyze them.

The sources for ROOTANA are managed at https://bitbucket.org/tmidas/rootana.

= Components =

== MIDAS Interface ==

These classes provide the actual access to MIDAS data

* TMidasEvent: a class encapulating a MIDAS event.
* TMidasFile: a class encapsulating a MIDAS file
* TMidasOnline: a class that allows access to online MIDAS events; this class requires that the MIDAS libraries be installed
* VirtualODB: a class that provides access to ODB variables, either through ODB dumps at the start of files or by directly connecting to the online ODB.

== XML Server ==

xmlServer & co

== example programs ==

* event_dump
* event_skim
* standalone graphical analyzer.cxx

== ROOT Analyzer framework ==

rootana provides low-level access to the MIDAS data files and online MIDAS data. Using these low-level functionality users can design whatever sort of programs they want for accessing and analyzing the data.

rootana also provides a more structured, higher-level framework for organizing analysis programs. The framework includes a standard event loop class and a standard display program. Details on this higher-level framework available in [[rootana Analyzer Framework]] and [[rootana Display Framework]].

= Contacts =

* Before conversion from svn to git, ROOTANA was managed by members of the TRIUMF DAQ group
* The git version hosted on bitbucket is managed by the members of the MIDAS developers group.
* to report bugs, request improvements, contribute bug fixes - please go to the ROOTANA issue tracker https://bitbucket.org/tmidas/rootana/issues
* for questions and discussion - please go to the MIDAS discussion forum https://midas.triumf.ca/forum

ROOTANA

2026-09-29T16:07:09Z

Dfujimoto: /* Links */

{{Pagelinks}}

= Links =

* [[ROOT]]
* https://gitlab.triumf.ca/midas/rootana
* git clone https://gitlab.triumf.ca/midas/rootana.git
* git clone http://ladd00.triumf.ca/~daqweb/git/rootana.git
* (obsolete) [https://ladd00.triumf.ca/viewvc/rootana/trunk ROOTANA SVN reporitory] (username svn, password svn)
* [http://ladd00.triumf.ca/~daqweb/doc/rootana/html ROOTANA Doxygen documentation]

= Download and Quick start =

<pre>
echo $ROOTSYS # check correct installation of ROOT
root-config # check correct installation of ROOT

git clone https://bitbucket.org/tmidas/rootana.git
cd rootana
make
make dox

more analyzer.cxx
more event_dump.cxx
</pre>

= Introduction =

Welcome to ROOTANA, the ROOT (https://root.cern.ch) based data analysis package for the MIDAS Data Acquisition system (https://midas.triumf.ca).

The ROOTANA package includes 5 major components:
* a standalone library for reading and writing data files in the MIDAS .mid format, and for decoding the XML dump of MIDAS ODB typically embedded in every MIDAS data file. (libMidasInterface)
* a C++ interface class for connecting to an active MIDAS experiment, accessing ODB (read and write) and getting event data. (libMidasInterface)
* a set of C++ classes for exporting ROOT histogram and other objects to an external viewer for interactive visualization of live data, typically using the ROODY histogram viewer or using a standard web browser (experimental feature). (libMidasServer, libNetDirectory, libXmlServer)
* simple examples of using these components (a graphical analyzer, an event dump and an event skim programs) (analyzer.cxx, event_dump.cxx, event_skim.cxx)
* a full featured framework for graphical data analysis including code to unpack typical VME and CAMAC modules ( see [[rootana Analyzer Framework]] and [[rootana Display Framework]])

The ROOTANA package can be used without installing ROOT and MIDAS:
* without both ROOT and MIDAS one can only read (and write) existing .mid files. Only event_dump and event_skim will be built.
* if only the MIDAS package is installed, access to live data and access to live ODB becomes possible. Only event_dump and event_skim will be built.
* decoding of XML ODB dumps embedded in MIDAS files requires ROOT TXML and TDOMParser components (a version of XML decoder using libxml2 used to exist in the past).
* if the ROOT package is installed but MIDAS is absent, full function of ROOTANA is available, except for access to live data. This mode is suitable for offline data analysis, i.e. the user has a copy of MIDAS data files on their laptop and wants to analyze them.

The sources for ROOTANA are managed at https://bitbucket.org/tmidas/rootana.

= Components =

== MIDAS Interface ==

These classes provide the actual access to MIDAS data

* TMidasEvent: a class encapulating a MIDAS event.
* TMidasFile: a class encapsulating a MIDAS file
* TMidasOnline: a class that allows access to online MIDAS events; this class requires that the MIDAS libraries be installed
* VirtualODB: a class that provides access to ODB variables, either through ODB dumps at the start of files or by directly connecting to the online ODB.

== XML Server ==

xmlServer & co

== example programs ==

* event_dump
* event_skim
* standalone graphical analyzer.cxx

== ROOT Analyzer framework ==

rootana provides low-level access to the MIDAS data files and online MIDAS data. Using these low-level functionality users can design whatever sort of programs they want for accessing and analyzing the data.

rootana also provides a more structured, higher-level framework for organizing analysis programs. The framework includes a standard event loop class and a standard display program. Details on this higher-level framework available in [[rootana Analyzer Framework]] and [[rootana Display Framework]].

= Contacts =

* Before conversion from svn to git, ROOTANA was managed by members of the TRIUMF DAQ group
* The git version hosted on bitbucket is managed by the members of the MIDAS developers group.
* to report bugs, request improvements, contribute bug fixes - please go to the ROOTANA issue tracker https://bitbucket.org/tmidas/rootana/issues
* for questions and discussion - please go to the MIDAS discussion forum https://midas.triumf.ca/forum

Main Page

2026-09-29T16:06:05Z

Dfujimoto: /* Documentation */

= Introduction =

MIDAS is a modern data acquisition system developed at [http://www.psi.ch PSI] and [http://www.triumf.ca TRIUMF]. Supported hardware includes VME, Fastbus, CAMAC, RS232, GPIB, USB, ethernet, fiber optic and MSCB attached data acquisition devices. MIDAS is written in C/C++ and runs on Linux, MacOS and MS Windows. It is licensed under the [http://www.gnu.org/copyleft/gpl.html GNU Public License].
[[Introduction|'''more...''']]

= Documentation =

* [[Midas documentation| Main Documentation]]
* [[Installation | Installation Instructions]]
* [[Changelog | Changelog / Upgrade Instructions]]
* [https://daq.triumf.ca/~daqweb/doc/midas-develop/html/modules.html Doxygen] Code structure, functions, etc...
* [https://gitlab.triumf.ca/midas/midas git repository on gitlab] Source code, bug tracking, etc
* [https://midas.triumf.ca/forum MIDAS forum] Discussion, announcements, problem reports, requests for improvements
* [https://dsdaqgw.triumf.ca/zensical MIDAS documentation] pre-production early access to new documentation

= News and Support=

For news about bug fixes and new releases, go to the
[https://midas.triumf.ca/forum MIDAS forum] hosted by TRIUMF using
the [http://elog.psi.ch/elog ELOG] system. Users can register in this system to be notified automatically via
E-mail when new entries are submitted.

Another source of information is the [https://gitlab.triumf.ca/midas/midas Gitlab Repository] of MIDAS, where one can see the latest changes to the software.

Finally, you can read presentations from
* [https://indico.psi.ch/conferenceDisplay.py?ovw=True&confId=3793 2015 MIDAS Workshop]
* [https://meetings.triumf.ca/event/428/ 2017 MIDAS Workshop]
* [https://meetings.triumf.ca/event/429/ 2019 MIDAS Workshop]
* [[2023 MIDAS Workshop]]
* [[2025 MIDAS Workshop]]

= Related packages =

* [http://elog.psi.ch/elog ELOG] - Electronics Logbook from PSI
* [https://bitbucket.org/tmidas/manalyzer/src/master/ manalyzer] - data analysis from TRIUMF
* [[ROOT]] - data analysis package from CERN
* [[ROOTANA]] - ROOT-based analyzer for MIDAS
* [[ROODY]] - viewer for online histograms
* http://elog.psi.ch/rome/ - MEG/PSI data analysis package

History

2026-09-29T16:05:35Z

Dfujimoto: /* Current state of development */

MIDAS has been created by Stefan Ritt at Paul Sherrer Institute in Aargau, Switzerland in the early 1990s in collaboration with Pierre-Andre Amaudruz from Triumf in Vancouver, Canada.
Since then a collective participate to the improvment of this DAQ software package in particular Konstantin Olchanski, Suzannah Daviel from Triumf and ... PSI

== Where this system is used ==
Midas is used across all the continents for a simple setup on the lab workbench to a full blown intermediate physics experiments such as muon decay [http://meg.icepp.s.u-tokyo.ac.jp/ '''MEG'''], neutrino oscillation [http://t2k-experiment.org/ '''T2K'''], trapping antihydrogen atoms [http://alpha.web.cern.ch/ '''ALPHA'''] or dark matter search [http://deap.phy.queensu.ca/ '''DEAP'''].

A none exhaustive list of experiments can be found [https://midas.triumf.ca/MidasWiki/index.php/MIDAS_around_the_world here].

== Current state of development ==
This section list the latest news about Midas code management and development.

* September 2026
** Midas code migrated to gitlab server hosted by TRIUMF [https://gitlab.triumf.ca/midas/midas]
* July 2013 : Stefan visit at Triumf
** We moved the Midas code from a local SVN repository to BitBucket cloud based GIT repository [https://bitbucket.org/tmidas Midas]
** Moving towards JSON, JSON-P for custom Midas web page development (more later).
** mhttpd: added hidding equipments option
**

Automated tests

2026-09-29T16:04:30Z

Dfujimoto: /* BitBucket pipeline */

== make test ==

After compiling midas with cmake, you can run an automated test script that will:

* create a new midas experiment
* run a few programs and commands
* check that the output of the script matches what is expected.

From the main midas directory (not your build directory!) run:

unset MIDAS_EXPT_NAME
make test

At the end of the tests you should see

MIDAS self test completed with no errors

== Code coverage ==

As of summer 2022, midas supports generating code coverage reports for the C++ code.

=== Pre-requisites ===

Install the lcov package from apt/yum/brew/macports depending on your OS preference.

=== Compiling with support for code coverage ===

Build midas with the "Coverage" build type, e.g.

mkdir build_cov
cd build_cov
cmake .. -DCMAKE_BUILD_TYPE=Coverage
make install

=== Generating coverage report ===

From the main midas directory (not your build directory!) run:

unset MIDAS_EXPT_NAME
make coverage

This will generate HTML files with annotated source code coverage based on the same tests as <code>make test</code>. It will also print an overall summary to screen.

The starting point is ~14% code coverage in summer 2022 before any dedicated effort to increase the coverage.

=== Reverting to the non-coverage build ===

Note that the default build type for midas is <code>RelWithDebInfo</code>. The coverage build disables optimizations and enables code coverage instrumentation, so is only suitable for running the code coverage tests!

To revert back to your normal build, do one of the following:

If you compiled the coverage version in a ''separate build directory'' (e.g. build_cov), run:

cd build
make install

If you compiled the coverage version in your ''regular build directory'', run:

cd build
cmake .. -DCMAKE_BUILD_TYPE=RelWithDebInfo
make install

== Gitlab pipeline ==

Midas source code is hosted on Gitlab, and a CI/CD pipeline is configured to build on Ubuntu and run <code>make test</code>.

The pipeline is scheduled to run against the develop branch weekly (not on every commit), but can also be triggered manually by developers using the [https://gitlab.triumf.ca/midas/midas/-/pipelines Gitlab pipelines page].

Main Page

2026-09-29T16:02:28Z

Dfujimoto: /* News and Support */

= Introduction =

MIDAS is a modern data acquisition system developed at [http://www.psi.ch PSI] and [http://www.triumf.ca TRIUMF]. Supported hardware includes VME, Fastbus, CAMAC, RS232, GPIB, USB, ethernet, fiber optic and MSCB attached data acquisition devices. MIDAS is written in C/C++ and runs on Linux, MacOS and MS Windows. It is licensed under the [http://www.gnu.org/copyleft/gpl.html GNU Public License].
[[Introduction|'''more...''']]

= Documentation =

* [[Midas documentation| Main Documentation]]
* [[Installation | Installation Instructions]]
* [[Changelog | Changelog / Upgrade Instructions]]
* [https://daq.triumf.ca/~daqweb/doc/midas-develop/html/modules.html Doxygen] Code structure, functions, etc...
* [https://bitbucket.com/tmidas/midas git repository on bitbucket] Source code, bug tracking, etc
* [https://midas.triumf.ca/forum MIDAS forum] Discussion, announcements, problem reports, requests for improvements
* [https://dsdaqgw.triumf.ca/zensical MIDAS documentation] pre-production early access to new documentation

= News and Support=

For news about bug fixes and new releases, go to the
[https://midas.triumf.ca/forum MIDAS forum] hosted by TRIUMF using
the [http://elog.psi.ch/elog ELOG] system. Users can register in this system to be notified automatically via
E-mail when new entries are submitted.

Another source of information is the [https://gitlab.triumf.ca/midas/midas Gitlab Repository] of MIDAS, where one can see the latest changes to the software.

Finally, you can read presentations from
* [https://indico.psi.ch/conferenceDisplay.py?ovw=True&confId=3793 2015 MIDAS Workshop]
* [https://meetings.triumf.ca/event/428/ 2017 MIDAS Workshop]
* [https://meetings.triumf.ca/event/429/ 2019 MIDAS Workshop]
* [[2023 MIDAS Workshop]]
* [[2025 MIDAS Workshop]]

= Related packages =

* [http://elog.psi.ch/elog ELOG] - Electronics Logbook from PSI
* [https://bitbucket.org/tmidas/manalyzer/src/master/ manalyzer] - data analysis from TRIUMF
* [[ROOT]] - data analysis package from CERN
* [[ROOTANA]] - ROOT-based analyzer for MIDAS
* [[ROODY]] - viewer for online histograms
* http://elog.psi.ch/rome/ - MEG/PSI data analysis package

Setup MIDAS experiment at TRIUMF

2026-03-12T20:50:34Z

Dfujimoto: /* Setup experiment database (ODB) */

{{Pagelinks}}
== Introduction ==
This page describes setting up a MIDAS experiment at TRIUMF. This information can be adapted for other sites.

== Environment variables ==
* '''MIDASSYS''' Base directory of the MIDAS package, midas and mxml should be at the same level.
* '''MIDAS_EXPTAB''' Experiment definition file
* '''MIDAS_SERVER_HOST''' MIDAS host server name for remote midas connections.
* '''MIDAS_EXPT_NAME''' Experiment name

== Standard layout of MIDAS experiment ==
The following shows the directory layout of a standard MIDAS experiment:

/home/exptuser/
packages/
root <---- ROOT
mxml
mscb
midas/ <---- MIDAS
linux/{lib,bin} <---- binaries matching the selected 64-bit/32-bit flavour of ROOT
linux-m32/{lib,bin} <---- limited function 32-bit binaries for 32-bit frontend machines, build by "make linux32"
linux-m64/{lib,bin} <---- limited function 64-bit binaries (only needed if ROOT and linux/bin are 32-bit)
linux-arm/{lib,bin} <---- full function ARM cross-compiled using "make linuxarm"
linux-crosscompile/{lib,bin} <---- cross-compiled limited function binaries for PPC and ARM frontends (see Makefile)
rootana <---- ROOT analyzer
roody <---- graphical online histogram viewer for MIDAS and ROOTANA
online/
exptab <---- experiment definition
{.ODB,.SYSTEM,.SYSMSG,etc}.SHM <---- MIDAS shared memory save files
src <---- experiment frontend sources
bin,scripts
elog <---- MIDAS elog
history <---- MIDAS history
data -> /data/exptname/current <---- symlink to the data directory
/data/exptname/current <---- experiment data directory with ODB save files and MIDAS .mid/.mid.gz data files

== Prepare computers ==

On some operating systems, several MIDAS functions require administrator access:

* on el7 linux (SL7/CC7/CentOS7/RHEL7) - access to mhttpd port 8443 requires special firewall rules, see here: https://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29
* on el7 linux - access to mserver to run frontends and other programs on some other computer requires special firewall rules, see here: https://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29
* on el7 linux - on the frontend machines (and other machines that will connect to the mserver, the same firewall rule needs to be created (use the IP address of the machine running the mserver)

== Prepare the user account ==
<div id="NOTES"></div>

* Setup the user account for running this instance of midas. For machines part of the LADD cluster, follow these [http://daq-plone.triumf.ca/SM/docs/local/NewLaddUser] instructions.
* check that the account is using the /bin/bash shell
* make $HOME/.profile look like this:

<pre>
#!/bin/echo You must source

export SVN_EDITOR="emacs -nw"
export GIT_EDITOR="emacs -nw"
export MIDASSYS=$HOME/packages/midas
export ROOTANASYS=$HOME/packages/rootana
export MIDAS_EXPTAB=$HOME/online/exptab
#
# setup the MIDAS mserver
#
case `hostname` in
daq07*)
unset MIDAS_SERVER_HOST
;;
*)
export MIDAS_SERVER_HOST=daq07.triumf.ca:7070
;;
esac
#
# select 64-bit or 32-bit MIDAS and ROOT
#
case `uname -i` in
i386)
source /daq/daqshare/olchansk/root/root_v5.34.01_SL62_32/bin/thisroot.sh
export PATH=.:$MIDASSYS/linux-m32/bin:$PATH
;;
*)
#source /daq/daqshare/olchansk/root/root_v5.34.34_SL67_64/bin/thisroot.sh
source $HOME/packages/root/bin/thisroot.sh
export PATH=.:$MIDASSYS/linux/bin:$PATH
;;
esac
#
export PATH=.:$HOME/online/bin:$HOME/packages/roody/bin:$PATH
#
#end
</pre>

* mkdir $HOME/packages
* Logout and login again, for .cshrc changes to take effect

== Install ROOT ==

* Identify the Linux version: RH9 (Red Hat Linux 9), FC3 (Fedora Core 3), RHEL4/SL4 (Red Hat Enterprise LInux 4/Scientific Linux 4), SL5x, SL6x, (CentOS/CC/SL) el7x: more /etc/redhat-release
* Decide to use 32-bit or 64-bit ROOT ('uname -a')
* cd $HOME/packages
* ls -l /daq/daqshare/olchansk/root/ ### to see all available ROOT packages
* ln -s /daq/daqshare/olchansk/root/root_vNNN_VVV_BB root, where NNN is the latest available version of ROOT ("ls -l /daq/daqshare/olchansk/root"), VVV is the Linux version code (RH9, FC3, SL4, etc) and BB is "_32" or "_64" for 32-bit or 64-bit ROOT. For example: /daq/daqshare/olchansk/root/root_v5.10.00_SL40
* for example: ln -s /daq/daqshare/olchansk/root/root_v5.34.34_el72_64 $HOME/packages/root
* Check that ROOT works: "source $HOME/packages/root/bin/thisroot.sh; root". Type ".q" to exit root.

== Install MIDAS ==

* cd $HOME/packages
* (OBSOLETE) svn co svn+ssh://svn@savannah.psi.ch/repos/meg/midas/trunk midas, password "svn". (password has to be entered twice)
* (OBSOLETE) svn co svn+ssh://svn@savannah.psi.ch/repos/meg/mxml/trunk mxml
* git clone https://bitbucket.org/tmidas/midas --recursive
* (OBSOLETE) git clone https://bitbucket.org/tmidas/mxml
* (OBSOLETE) git clone https://bitbucket.org/tmidas/mscb
* (IF BITBUCKET IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/mxml.git
* (IF BITBUCKET IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/mscb.git
* (IF BITBUCKET IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/midas.git
* cd midas
* make
* (only if needed) make linux32 ### build the 32-bit MIDAS libraries
* ls -l linux/bin/odbedit ### check that odbedit has been created (do not run it yet)

You can see a list of other installation problems at [[Common problems & Debugging recipes]].

;NOTE 1
: Optional features in MIDAS can be explicitly disabled if desired when making MIDAS using the NO_xxx feature (NO_ROOT,NO_MYSQL,NO_ODBC,NO_SQLITE,NO_MSCB), e.g. "make NO_ROOT=1" to disable ROOT. These NO_xxx Makefile variables are only used to control autodetection.

;NOTE 2
: Since June 2019 the mxml and mscb packages are submodules of the midas package, so no need to clone them separately. If you have an existing clone of midas but not yet the submodules, you need

$ git submodule update --init --recursive

: To update both midas and the submodules, you need

$ git pull --recurse-submodules

== Install ROOTANA ==

* cd $HOME/packages
* git clone https://bitbucket.org/tmidas/rootana
* cd rootana
* make

== Install ROODY ==

* cd $HOME/packages
* git clone https://bitbucket.org/tmidas/roody
* cd roody
* make
* $HOME/packages/roody/bin/roody, run the program

== Install additional additional packages ==

* cd $HOME/packages
* git clone https://bitbucket.org/ttriumfdaq/vme
* git clone https://bitbucket.org/ttriumfdaq/frontends

== Build special versions of MIDAS ==

Build special versions of MIDAS for the case when some MIDAS programs, such as VME frontends, will run on a different computer that may have a different flavour of operating system, i.e. 32-bit linux or an older version of Scientific Linux.

* login to the computer where the frontends will run and:
* if it is a 32-bit linux: cd $HOME/packages/midas; make linux32
* if it is a 64-bit linux: cd $HOME/packages/midas; make linux64

* login to the host computer to cross-compile ARM code:
* if it is an ARM linux: cd $HOME/packages/midas; make linuxarm # may need to install ARM cross compilers

== Prepare VME hardware ==

Hardware check list:
* VME crate
* VME processor (supported are V77xx, V7805, V7865)
* On all VME modules, set the VME address jumpers as described here: http://daq-plone.triumf.ca/SM/docs/local/vme_jumpers
* run vmescan to confirm correct VME addresses
** cd $HOME/packages
** svn checkout https://ladd00.triumf.ca/svn/daqsvn/trunk/vme
** cd vme
** make
** ./vmescan.exe (or _gef.exe, depending on the VME driver in use)

== Install Universe-II VME driver (V7648, V7750, V7805, V7851) ==

* login as root (ssh root@localhost)
* follow instructions: https://www.triumf.info/wiki/DAQwiki/index.php/VME-CPU#V7648.2C_V7750.2C_V7805.2C_V7851_:_Setup_vme_universe_VME_drivers
* cd ~/packages/vme; vmescan.exe

== Install Tsi-148 VME driver (V7865) ==

* login as root (ssh root@localhost)
* follow instructions: https://www.triumf.info/wiki/DAQwiki/index.php/VME-CPU#V7865_and_XVB-602_:_Setup_gefvme.2Ftsi148_VME_drivers
* cd ~/packages/vme; vmescan_gef.exe

== Setup the experiment environment ==

* Decide which computer will host MIDAS (where MIDAS shared memory buffers will reside).
: This computer will run the [[mserver]], [[mlogger]] and [[mhttpd]] applications. (It is usually the machine where the MIDAS,ROOT etc. packages have been downloaded). It will be referred to as the host machine (localhost).

The environment is slightly different depending on whether all programs run on the host machine, or whether some programs run on remote host(s) :
=== ALL programs run on localhost ===
:If all programs run on the host machine (localhost), it is not necessary to run [[mserver]]. [[Environment Variables#MIDAS_SERVER_HOST|MIDAS_SERVER_HOST]] will not be assigned (see example .cshrc [[#Prepare the user account|above]]).

=== Some programs run on REMOTE host(s) ===
: '''IMPORTANT:'''
# Since August 2015 '''you must explicitly allow access for clients running on remote machines'''. To do this, follow the '''[[Security#MIDAS programs on remote machines|instructions here]]'''.
# The example code .cshrc ([[#Prepare the user account|see above]]) should be present on both host and remote machine(s). This will ensure that [[Environment Variables#MIDAS_SERVER_HOST|MIDAS_SERVER_HOST]] will NOT be set for the host machine (localhost), but on a remote machine, MIDAS_SERVER_HOST will be set to the MIDAS host machine.
# The client [[mserver]] must be started on the MIDAS host machine. Note that multiple experiments can run on the same host machine by starting several instances of [[mserver]] (one for each experiment) running with different ports (and .cshrc would be edited so that MIDAS_SERVER_HOST is set to the appropriate port for the experiment).

 
On the host machine:
* mkdir $HOME/online
* cd $HOME/online
* create directories for local programs, sources, elog and history: mkdir bin src elog history
* create data directory: mkdir -p /ladd/data1/t2kvme5/data; ln -s /ladd/data1/t2kvme5/data $HOME/online
* create the exptab file "$HOME/online/exptab" following the example below. The first entry (exptname) is the name if the DAQ system (MIDAS experiment name), the second entry (/home/USER/online) is the location of MIDAS shared memory buffers (by convention, $HOME/online), the third entry (kopio03) is your username.
<pre>exptname /home/kopio03/online kopio03</pre>
* logout and login again for all changes to take effect

== Setup experiment startup scripts ==
* login to the experiment host computer
* echo $MIDAS_SERVER_HOST ### to check correct value - should be blank
* create $HOME/online/bin/start_daq.sh, replacing XXX with the hostname of the machine running the experiment (and changing the mserver and mhttpd ports, as needed).

#!/bin/sh
# start_daq.sh
cd $HOME/online
#
case `hostname` in XXX*)
echo "Good, we are on XXX!"
;;
*)
echo "The start_daq script should be executed on XXX"
exit 1
;;
esac
#
odbedit -c clean
# start [[mhttpd]] on default port. (Mongoose https version - see [[mhttpd]] for other options)
mhttpd -D -a localhost -a XXX.triumf.ca # optionally restrict access to specified hosts
#
# start [[mserver]] on default port (use argument -p to use a different port)
mserver -D # access must now be specifically allowed - see [[#Setup the experiment environment|above]]

# OR ([[#NOTES|older MIDAS versions]])
# mhttpd -p 8081 -D -a localhost -a XXX.triumf.ca # optionally restrict access to specified hosts
# mserver -p 7071 -D -a localhost -a lxdragon01.triumf.ca -a lxdragon02.triumf.ca -a XXX.triumf.ca # optionally restrict access to specified hosts

#
mlogger -D
#end file

== Run the MIDAS Web Server ==
Let's start the MIDAS webserver for the first time:

Start [[mhttpd]] on the ''experiment host'' (localhost) like this:
[mhostpc] mhttpd
You will get the following messages:
[mhttpd,INFO] ODB subtree /Runinfo corrected successfully
Mongoose web server will listen on ports "8080r,8443s"
[mhttpd,ERROR] [mhttpd.cxx:17892:mongoose,ERROR] cannot find SSL certificate file "/home/agdaq/online/ssl_cert.pem"
[mhttpd,ERROR] [mhttpd.cxx:17893:mongoose,ERROR] please create SSL certificate file: openssl req -new -nodes -newkey rsa:2048 -sha256 -out ssl_cert.csr -keyout ssl_cert.key; openssl x509 -req -days 365 -sha256 -in ssl_cert.csr -signkey ssl_cert.key -out ssl_cert.pem; cat ssl_cert.key >> ssl_cert.pem
could not start the mongoose web server, see messages and midas.log, bye!

Create a self-signed certificate suitable for initial testing by executing the command printed by mhttpd:


[mhostpc] openssl req -new -nodes -newkey rsa:2048 -sha256 -out ssl_cert.csr -keyout ssl_cert.key; openssl x509 -req -days 365 -sha256 -in ssl_cert.csr -signkey ssl_cert.key -out ssl_cert.pem; cat ssl_cert.key >> ssl_cert.pem

For production use, you should create a properly signed certificate, see [[Mhttpd#Create an SSL certificate|create your own SSL certificate]] or you should run mhttpd behind an SSL proxy.

Run mhttpd again.

You will get the following messages:
[mhttpd,INFO] ODB subtree /Runinfo corrected successfully
Mongoose web server will listen on ports "8080r,8443s"
Mongoose web server will use SSL certificate file "/home/johnfoo/packages/midas/ssl_cert.pem"
[mhttpd,ERROR] [mhttpd.cxx:17633:mongoose,ERROR] mongoose web server cannot find password file "/home/johnfoo/online/htpasswd.txt"
[mhttpd,ERROR] [mhttpd.cxx:17634:mongoose,ERROR] please create password file: htdigest -c /home/johnfoo/online/htpasswd.txt Default midas
could not start the mongoose web server, see messages and midas.log, bye!

Create the password file by following the instructions printed by mhttpd. The http digest domain name is the experiment name, suggested default user name is "midas". You will be asked to type in a password


[mhostpc] htdigest -c /home/johnfoo/online/htpasswd.txt exptname midas
Adding password for midas in realm exptname.
New password:
Re-type new password:

It is a good idea to set the password file {{Filepath|path=htpasswd.txt}} readable and writable by owner only.

Now restart {{Utility|name=mhttpd}}
[mhostpc] mhttpd
Mongoose web server will listen on ports "8080r,8443s" **see note
Mongoose web server will use SSL certificate file "/home/suz/packages/midas/ssl_cert.pem"
Mongoose web server will use authentication realm "Default", password file "./htpasswd.txt"

Now point a web browser running on the same host computer (localhost) to https://localhost:8443
If the web browser is running on a different computer, go to URL of the form

https://mhostpc.triumf.ca:8443 (substitute your host machine name and domain for "mhostpc.triumf.ca")

If you are using the default SSL certificate you will probably get a message: "This Connection is Untrusted". Click "I understand the risks" and add an exception. This is because the test certificate is self-signed. Then confirm an exception.

If instead you get a "connection refused" error, the midas host pc may have the firewall enabled. To make a firewall exception for MIDAS, follow instructions here http://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29

You should then see an authentication box asking you for the user name and password. The user name is "midas". Enter the password you just created. The Midas [[Status Page]] should appear with multiple buttons for run control as well as equipment listing (no equipments will be listed as yet) and application listings. Please refer to [[mhttpd]] (the MIDAS Web-based Run Control utility) for further information. You can start and stop runs from the main status page, and use the [[ODB Page]] to access the database (ODB).

; Note
: Default ports of 8080 and 8443 are used by [[mhttpd]]. If these ports are in use on your machine, start mhttpd with alternative ports, e.g.
[mhostpc] mhttpd --https 8448 --http 8089
: or see [[Mhttpd#Usage]] to change the default ports.

== Setup experiment database (ODB) ==

* run $HOME/online/bin/start_daq.sh

* odbedit, run these commands: (replace user names and directory names)
<pre>
# set "/Logger/Message file" "/home/kopio03/online/midas.log" # obsolete
set "/Logger/Data Dir" "/home/kopio03/online/data"
set "/Logger/History dir" "/home/kopio03/online/history"
create STRING "/Logger/Elog dir"
set "/Logger/Elog dir" "/home/kopio03/online/elog"
set "/Logger/ODB dump file" "/home/kopio03/online/history/run%05d.xml"
set "/Logger/ODB dump" "y"
set "/Logger/Channels/0/Settings/Filename" "run%05dsub%03d.mid.gz"
set "/Logger/Channels/0/Settings/Subrun byte limit" "1000000000"
set "/Logger/Channels/0/Settings/Compression" 1
set "/Logger/Channels/0/Settings/ODB Dump" "y"
set "/Programs/Logger/Required" y
set "/Programs/Logger/Start command" "mlogger -D"
set "/Programs/fevme/Required" "y"
set "/Programs/fevme/Start command" "ssh -n lxdaq09 $HOME/online/src/fevme_gef.exe -O"
exit
</pre>
* open web browser e.g. firefox.
* go to the midas status page at https://localhost:8443 (default port).
** if running [[mhttpd]] with Mongoose HTTPS/OpenSSL (the default) for the first time, you will need to create a password file. Follow the instructions (see [[mhttpd#HTTPS/SSL server (Mongoose)]] for details).
** For other options (i.e. HTTPS/SSL proxy) see [[#Secure MIDAS and ELOG Web access]]
*OR open the midas status page at http://localhost:8081 ([[#NOTES|older MIDAS versions]])
* midas status page will show most stuff "red" as nothing is running yet
* DON'T DO THIS YET run ./fevme.exe (on the computer with the VME interface, could be different from computer hosting the experiment), observe that corresponding equipments have been created
* save the url bookmark to the "personal toolbar"
* go to the Programs page, stop mlogger, stop fevme, start mlogger, start fevme
* go to the Status page, start run, stop run
* go back to the Status page, everything should be green
* start a run
* send signals to the ADC gate
* you should be getting events
* to look at data, proceed with setting up the [[ROOTANA|ROOT Analyzer]].

== Start DAQ programs at boot time ==

* add this to /etc/rc.local (replace username and location of the start_daq script)
<pre>
su - alpha -c /home/alpha/online/bin/start_daq.sh
</pre>

== Setup local software version control ==

Version control for experiment source code is setup using "git" (http://git-scm.com/)

* cd $HOME/online
* git init
* git add exptab
* git add bin/start_daq.sh
* git add .gitignore ### contents can be
<pre>
*~
*.o
*.exe
</pre>
* git add src/Makefile src/*.cxx ...
* git commit -a

== Adjust MIDAS buffer sizes ==

Default MIDAS SYSTEM buffer size is 8 Mbytes, fairly small for high-data-rate experiments. The rule of thumb is to have at least a few seconds worth of buffer space available. For example, if event size is 10 Kbytes and the event rate is 1 kHz, data rate is 10*10^3*1*10^3 = 10 Mbytes/sec. To buffer 10 seconds of data we need 100 Mbytes of buffer space.

To resize the MIDAS event buffers (SYSTEM, etc) do this:
* stop all frontends, stop mlogger
* start odbedit:
** cd "/Experiment/Buffer sizes"
** set SYSTEM 100000000
* run "mdump -z SYSTEM"
* if mdump complains about the size of .SYSTEM.SHM, remove it, try again.
* ls -l /dev/shm ### to observe that the size of shared memory is correct

== Secure MIDAS and ELOG Web access ==
In versions prior to May 2015, the default web access to MIDAS and ELOG uses the "http:" protocol which is insecure. In this case, all information is transmitted as clear text meaning that secret, confidential and sensitive information (such as the MIDAS and ELOG passwords and usernames) can be stolen "easily". This means that even "password protected" MIDAS and ELOG pages are not really protected if accessed using the "http" method.

Better security for HTTP is gained by using a password protected '''SSL (https) proxy'''. (It does not provide absolute security because of remaining problems with the security of SSL certificates, security of passwords, etc). Setting up an SSL (https) proxy is described [[#Setting up an HTTP proxy|below]].

Since May 2015, an ''alternative secure option'' to setting up an HTTP proxy is available to users of MIDAS. Recent versions of elogd (ELOG) do support SSL https:// connections, and [[#mhttpd with HTTPS/SSL server (Mongoose)]] is now available. This option is the default, and provides a similar level of security to an HTTP proxy.

See [[Security#Web Access]] for a comparison of these two secure options.

=== mhttpd with HTTPS/SSL server (Mongoose) ===

Since May 2015 the MIDAS web server [[mhttpd]] is explicitly linked with OpenSSL to provide secure HTTPS connections via the [https://bitbucket.org/tmidas/midas/src/ecb9a8537448a8a43f7f9a2bfdb82e578208cde3/doc/mongoose/?at=develop Mongoose] web server (see [[mhttpd]]). With this version, default web access to MIDAS uses the "https" protocol. Web access to {{Utility|name=mhttpd}} can be restricted by using the "-a hostname" switch of [[mhttpd]]. The first time {{Utility|name=mhttpd}} is run, a password file must be created. An SSL certificate is also required. See [[mhttpd#HTTPS/SSL server (Mongoose)|HTTPS/SSL server (Mongoose)]] for instructions.

=== mhttpd using an HTTPS/SSL proxy ===

THESE INSTRUCTIONS ARE WRONG, DO NOT DO THIS. See instead [https://daq00.triumf.ca/MidasWiki/index.php/Setup_MIDAS_experiment_at_TRIUMF#Install_https_proxy this section].

An [[#Setting up an HTTP proxy|HTTP proxy]] must be set up. This is the only way of securing older version of [[mhttpd]] (pre August 2015). Older versions of mhttpd are started using the -p port option e.g.
* mhttpd -D -p 8080 

To run a new version of mhttpd using an HTTP proxy, use the options provided to run the old (non-Mongoose) webserver on a given port, i.e.
* mhttpd --oldserver 8080 --nomg -D 

When using an SSL proxy, only access from the SSL proxy (and maybe some special trusted machines) should be permitted.
This is done using the "-a hostname" switch of [[mhttpd]]. Normally there will be only "-a localhost" switch, enabling access only for the local machine (where the SSL proxy is running). Additional "-a hostname" switches enable access from listed local machines. No "-a xxx" enables access from everywhere (defeating the purpose of the SSL proxy, unless access controls are enforced elsewhere, i.e. by a site firewall or by local firewall rules).

== Setting up an HTTP proxy ==
In this example, we use APACHE HTTPD to password-protect a typical midas/mhttpd and elog installation.

In this configuration, one uses the Linux stock httpd that accepts encrypted https:// connections and forwards them to mhttpd and elogd. Instead of (or in addition to) using mhttpd and elogd passwords, one configures password protection in httpd via the regular apache httpd password mechanisms (htpasswd, etc).

Recent versions of elogd do support SSL https:// connections, but if one is running an SSL proxy for anyway, it is simpler to run both through the same SSL proxy using the same SSL host certificate and the same httpd password file.

=== Restricting http: access to elogd ===
;Note
:Recent versions of elogd do support SSL https:// connections. The following information is for those using an HTTP proxy (see above).

For elogd, this is done using the "-n localhost" switch with enables only access from the same machine if present, or access from anywhere is absent (defeating the purpose of the SSL proxy, unless access controls are enforced elsewhere).

(It is recommended to run elogd from the same user as the main daq user and to keep elogd.cfg and all logbooks in the home directory of this user, where they are captured by the normal site backup system)

== Install standalone elog ==

* login into the user account that will run the elog
* cd $HOME/packages
* git clone https://bitbucket.org/ritt/elog
* cd elog
* make
* create new file start_elogd with this contents:
<pre>
#!/bin/sh

killall elogd
sleep 1
killall -KILL elogd
sleep 1
$HOME/packages/elog/elogd -n localhost -x -c $HOME/packages/elog/elogd.cfg -p 8082 -D

#end
</pre>
* chmod a+x start_elogd
* edit elogd.cfg to read:
<pre>
[global]
port = 8082
SMTP host = smtp.triumf.ca
URL = https://titan00.triumf.ca/elog/

Reverse sort = 1
Display Mode = full

#List Menu commands = New, Find, Admin, Help
#Menu commands = New, Edit, Reply, Find, Duplicate, Help

Entries Per Page = 30
Supress Email on edit = 1
Default encoding = 1
Page title = TITAN ELOG
Resolve host names = 1

Logfile = /home/titan/packages/elog/elogd.log
#Logging level = 3

[midas]

List page Title = T2K M11 MIDAS ELOG
Comment = T2K M11 MIDAS ELOG
Page Title = T2K M11 MIDAS ELOG
RSS Title = [$logbook - $type - $system] $subject, posted by $author

Attributes = Author, Subject, Run, Type, System
Show Attributes Edit = Run, Author, Subject, Type, System
Required Attributes = Author, Type, System, Subject

Options Type = Routine, Reply, Shift Summary, Modification, Question, Info, Problem
Options System = General, DAQ, Beamline

Preset Run = $shell(MIDASSYS=. /home/t2km11/packages/midas/linux/bin/odbedit -d Runinfo -c 'ls -v \"run number\"')

Preset On Reply Type = Reply
Preset On Reply Run = $shell(MIDASSYS=. /home/t2km11/packages/midas/linux/bin/odbedit -d Runinfo -c 'ls -v \"run number\"')

List Display = Date, Subject, Type, System, Author, ID
Quick Filter = Date, Type, ID

Remove on reply = Author
Quote on reply = 1

Use lock = 1

************* Email Functionality ****************

Use Email Subject = [T2KM11 - $System] $Subject
Omit Email To = 1

Email System General = xxx
</pre>
* ./start_elogd &
* firefox http://localhost:8082 # hould show the elog message index

To start elogd automatically when the machine is rebooted, login as root and
* add this text to /etc/rc.local:
<pre>
su - titan -c "/home/titan/packages/elog/start_elogd"
</pre>
* chmod a+x /etc/rc.local
* systemctl start rc-local

To import elog entries from the mhttpd elog, do this:

* cd ~/packages/elog/logbooks
* ln -s /home/t2km11/online/elog midas
* cd midas
* ~/packages/elog/elconv)

== Install https proxy ==

THESE INSTRUCTIONS ARE OBSOLETE, INSTEAD,
* GO HERE: https://daq00.triumf.ca/DaqWiki/index.php/Ubuntu#Install_apache_httpd_proxy_for_midas_and_elog
* AND GO HERE: https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux#Run_the_MIDAS_Web_Server

FOLLOWING INSTRUCTIONS ARE OBSOLETE...

* login as root to the https proxy machine
* cd ~root
* yum install mod_ssl
* yum install crypto-utils # see http://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_monitoring_of_HTTPS_certificates
* create a certificate request (replace ladd09 with your hostname): openssl req -new -nodes -newkey rsa:2048 -sha256 -out ladd09.csr -keyout ladd09.key (answer: CA, BC, Vancouver, TRIUMF, DAQ, ladd09.triumf.ca, email@email.com
* sign it by TRIUMF:
** mail -s "Certificate request" yourself@email.com < ladd09.csr
** forward this request to Andrew Daviel
** he will email the signed crt file, copy it to this system as ladd09.crt
* sign it yourself: openssl x509 -req -days 365 -sha256 -in ladd09.csr -signkey ladd09.key -out ladd09.crt
* (if the certificate expires, renew it by signing it again)
* Additional commands for working with certificates:
** explore the private key: openssl pkey -in ladd09.key -text -noout
** explore the certificate request: openssl req -in ladd00.csr -text -noout
** explore the certificate: openssl x509 -in ladd09.crt -noout -text
* move certificate files to proper system locations:
* mv ladd09.key /etc/pki/tls/private/
* mv ladd09.crt /etc/pki/tls/certs/
* if selinux is enabled, do this:
** restorecon -Rv /etc/pki/tls/certs/
** restorecon -Rv /etc/pki/tls/private/
** /usr/sbin/setsebool -P httpd_can_network_connect 1
* open /etc/httpd/conf.d/ssl.conf in a text editor, go to the very bottom and right before the "</VirtualHost>" entry, add following text:
<pre>
...
SSLCertificateFile /etc/pki/tls/certs/ladd09.crt
SSLCertificateKeyFile /etc/pki/tls/private/ladd09.key

ProxyPass /elog/ http://localhost:8082/ retry=1
ProxyPass / http://localhost:8080/ retry=1

<Location />

SSLRequireSSL
AuthType Basic
AuthName "password protected site"
Require valid-user

# create password file: touch /etc/httpd/htpasswd
# to add new user or change password: htpasswd /etc/httpd/htpasswd username
AuthUserFile /etc/httpd/htpasswd

</Location>
</VirtualHost>
...
</pre>
* comment out duplicate "SSLCertificateFile" and "SSLCertificateKeyFile" elsewhere in the file
* (optionally) If you got a certificate that is signed by DigiCert or RapidSSL then you'll need to add a line specifying the certificate chain file:
<pre>
...
SSLCertificateChainFile /etc/pki/tls/certs/DigiCertCA.crt
...
</pre>
* touch /etc/httpd/htpasswd
* htpasswd /etc/httpd/htpasswd midas # enter password midas
* chkconfig httpd on
* service httpd restart
* firewall-cmd --add-port=443/tcp --permanent
* firewall-cmd --reload
* firewall-cmd --list-all
* test it
** test the SSL proxy: https://host/ should yield the midas status page, https://host/elog/ should yield the elog message index
* in ODB, set "/Elog/URL" to "https://host/elog/"
* now from the midas status page, the "Elog" button should take us to the https Elog URL

In needed, enable user directories: https://blah/~user in ~user/public_html

* edit /etc/httpd/conf.d/userdir.conf, replace "UserDir disabled" with "UserDir enabled"
* setsebool -P httpd_enable_homedirs true
* systemctl restart httpd

== Setup the history mhttpd for faster access to history plots ==
When running an SSL proxy,
* start the main mhttpd (orange command for old mhttpd, green for new mhttpd with Mongoose(post August2015):
** "mhttpd -p 8071 -D" or
** "mhttpd -D --oldserver 8071 --nomg" 

* start the history mhttpd
** "mhttpd -p 8072 -D -H" or
** "mhttpd -D -H --oldserver 8072 --nomg" 
* set ODB /History/URL to "http://alphacpc09.cern.ch:8072/HS/"
* open the MIDAS status page
* go to the history section, try to open any history plot, observe that the history plot gif image loads correctly, inspect it's URL (use "copy image URL" or "view source", etc), it should point to port 8072 causing connection to the history mhttpd.
* continue with these instructions to setup history mhttpd access through an SSL proxy:
* setup SSL proxy access (required mod_proxy_html)
** login as root to the SSL proxy machine
** on SL5, install the missing mod_proxy_html httpd module:
** yum install httpd-devel libxml2-devel
** wget http://apache.webthing.com/mod_proxy_html/mod_proxy_html.tar.bz2
** tar xjvf mod_proxy_html.tar.bz2
** cd mod_proxy_html
** apxs -c -I. -I/usr/include/libxml2 -i mod_proxy_html.c
** apxs -c -I. -I/usr/include/libxml2 -i mod_xml2enc.c
** cd /etc/httpd/conf.d, add this to ssl.conf:

;before the ProxyPass statements:
<pre>
# proxy the MIDAS web servers
LoadModule xml2enc_module modules/mod_xml2enc.so
LoadModule proxy_html_module modules/mod_proxy_html.so
ProxyHTMLLinks a href
ProxyHTMLLinks link href
ProxyHTMLLinks img src
#ProxyHTMLEnable On
ProxyRequests off
</pre>
;after the ProxyPass statements:
<pre>
# ALPHA1 history access
ProxyPass /alpha1/history/ http://alphacpc09.cern.ch:8072/HS/ retry=1
ProxyPass /alpha1/ http://alphacpc09.cern.ch:8071/ retry=1

ProxyHTMLEnable On
ProxyHTMLURLMap http://alphacpc09.cern.ch:8072/HS/ /alpha1/history/
</pre>
;adjust:
*"alpha1" is the experiment name
*"alphacpc09.cern.ch" is the machine running mhttpd
*"8071" is the port number of the main mhttpd
** "mhttpd -p 8071 -D" or
** "mhttpd -D --oldserver 8071 --nomg" 
* "8072" is the port number of the history mhttpd
** "mhttpd -p 8072 -D -H" or
** "mhttpd -D -H --oldserver 8072 --nomg" 

[[Category:Installation]] [[Category:Buffer]] [[Category:Driver]]

Setup MIDAS experiment at TRIUMF

2026-03-12T20:46:59Z

Dfujimoto: /* Setup experiment database (ODB) */

{{Pagelinks}}
== Introduction ==
This page describes setting up a MIDAS experiment at TRIUMF. This information can be adapted for other sites.

== Environment variables ==
* '''MIDASSYS''' Base directory of the MIDAS package, midas and mxml should be at the same level.
* '''MIDAS_EXPTAB''' Experiment definition file
* '''MIDAS_SERVER_HOST''' MIDAS host server name for remote midas connections.
* '''MIDAS_EXPT_NAME''' Experiment name

== Standard layout of MIDAS experiment ==
The following shows the directory layout of a standard MIDAS experiment:

/home/exptuser/
packages/
root <---- ROOT
mxml
mscb
midas/ <---- MIDAS
linux/{lib,bin} <---- binaries matching the selected 64-bit/32-bit flavour of ROOT
linux-m32/{lib,bin} <---- limited function 32-bit binaries for 32-bit frontend machines, build by "make linux32"
linux-m64/{lib,bin} <---- limited function 64-bit binaries (only needed if ROOT and linux/bin are 32-bit)
linux-arm/{lib,bin} <---- full function ARM cross-compiled using "make linuxarm"
linux-crosscompile/{lib,bin} <---- cross-compiled limited function binaries for PPC and ARM frontends (see Makefile)
rootana <---- ROOT analyzer
roody <---- graphical online histogram viewer for MIDAS and ROOTANA
online/
exptab <---- experiment definition
{.ODB,.SYSTEM,.SYSMSG,etc}.SHM <---- MIDAS shared memory save files
src <---- experiment frontend sources
bin,scripts
elog <---- MIDAS elog
history <---- MIDAS history
data -> /data/exptname/current <---- symlink to the data directory
/data/exptname/current <---- experiment data directory with ODB save files and MIDAS .mid/.mid.gz data files

== Prepare computers ==

On some operating systems, several MIDAS functions require administrator access:

* on el7 linux (SL7/CC7/CentOS7/RHEL7) - access to mhttpd port 8443 requires special firewall rules, see here: https://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29
* on el7 linux - access to mserver to run frontends and other programs on some other computer requires special firewall rules, see here: https://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29
* on el7 linux - on the frontend machines (and other machines that will connect to the mserver, the same firewall rule needs to be created (use the IP address of the machine running the mserver)

== Prepare the user account ==
<div id="NOTES"></div>

* Setup the user account for running this instance of midas. For machines part of the LADD cluster, follow these [http://daq-plone.triumf.ca/SM/docs/local/NewLaddUser] instructions.
* check that the account is using the /bin/bash shell
* make $HOME/.profile look like this:

<pre>
#!/bin/echo You must source

export SVN_EDITOR="emacs -nw"
export GIT_EDITOR="emacs -nw"
export MIDASSYS=$HOME/packages/midas
export ROOTANASYS=$HOME/packages/rootana
export MIDAS_EXPTAB=$HOME/online/exptab
#
# setup the MIDAS mserver
#
case `hostname` in
daq07*)
unset MIDAS_SERVER_HOST
;;
*)
export MIDAS_SERVER_HOST=daq07.triumf.ca:7070
;;
esac
#
# select 64-bit or 32-bit MIDAS and ROOT
#
case `uname -i` in
i386)
source /daq/daqshare/olchansk/root/root_v5.34.01_SL62_32/bin/thisroot.sh
export PATH=.:$MIDASSYS/linux-m32/bin:$PATH
;;
*)
#source /daq/daqshare/olchansk/root/root_v5.34.34_SL67_64/bin/thisroot.sh
source $HOME/packages/root/bin/thisroot.sh
export PATH=.:$MIDASSYS/linux/bin:$PATH
;;
esac
#
export PATH=.:$HOME/online/bin:$HOME/packages/roody/bin:$PATH
#
#end
</pre>

* mkdir $HOME/packages
* Logout and login again, for .cshrc changes to take effect

== Install ROOT ==

* Identify the Linux version: RH9 (Red Hat Linux 9), FC3 (Fedora Core 3), RHEL4/SL4 (Red Hat Enterprise LInux 4/Scientific Linux 4), SL5x, SL6x, (CentOS/CC/SL) el7x: more /etc/redhat-release
* Decide to use 32-bit or 64-bit ROOT ('uname -a')
* cd $HOME/packages
* ls -l /daq/daqshare/olchansk/root/ ### to see all available ROOT packages
* ln -s /daq/daqshare/olchansk/root/root_vNNN_VVV_BB root, where NNN is the latest available version of ROOT ("ls -l /daq/daqshare/olchansk/root"), VVV is the Linux version code (RH9, FC3, SL4, etc) and BB is "_32" or "_64" for 32-bit or 64-bit ROOT. For example: /daq/daqshare/olchansk/root/root_v5.10.00_SL40
* for example: ln -s /daq/daqshare/olchansk/root/root_v5.34.34_el72_64 $HOME/packages/root
* Check that ROOT works: "source $HOME/packages/root/bin/thisroot.sh; root". Type ".q" to exit root.

== Install MIDAS ==

* cd $HOME/packages
* (OBSOLETE) svn co svn+ssh://svn@savannah.psi.ch/repos/meg/midas/trunk midas, password "svn". (password has to be entered twice)
* (OBSOLETE) svn co svn+ssh://svn@savannah.psi.ch/repos/meg/mxml/trunk mxml
* git clone https://bitbucket.org/tmidas/midas --recursive
* (OBSOLETE) git clone https://bitbucket.org/tmidas/mxml
* (OBSOLETE) git clone https://bitbucket.org/tmidas/mscb
* (IF BITBUCKET IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/mxml.git
* (IF BITBUCKET IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/mscb.git
* (IF BITBUCKET IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/midas.git
* cd midas
* make
* (only if needed) make linux32 ### build the 32-bit MIDAS libraries
* ls -l linux/bin/odbedit ### check that odbedit has been created (do not run it yet)

You can see a list of other installation problems at [[Common problems & Debugging recipes]].

;NOTE 1
: Optional features in MIDAS can be explicitly disabled if desired when making MIDAS using the NO_xxx feature (NO_ROOT,NO_MYSQL,NO_ODBC,NO_SQLITE,NO_MSCB), e.g. "make NO_ROOT=1" to disable ROOT. These NO_xxx Makefile variables are only used to control autodetection.

;NOTE 2
: Since June 2019 the mxml and mscb packages are submodules of the midas package, so no need to clone them separately. If you have an existing clone of midas but not yet the submodules, you need

$ git submodule update --init --recursive

: To update both midas and the submodules, you need

$ git pull --recurse-submodules

== Install ROOTANA ==

* cd $HOME/packages
* git clone https://bitbucket.org/tmidas/rootana
* cd rootana
* make

== Install ROODY ==

* cd $HOME/packages
* git clone https://bitbucket.org/tmidas/roody
* cd roody
* make
* $HOME/packages/roody/bin/roody, run the program

== Install additional additional packages ==

* cd $HOME/packages
* git clone https://bitbucket.org/ttriumfdaq/vme
* git clone https://bitbucket.org/ttriumfdaq/frontends

== Build special versions of MIDAS ==

Build special versions of MIDAS for the case when some MIDAS programs, such as VME frontends, will run on a different computer that may have a different flavour of operating system, i.e. 32-bit linux or an older version of Scientific Linux.

* login to the computer where the frontends will run and:
* if it is a 32-bit linux: cd $HOME/packages/midas; make linux32
* if it is a 64-bit linux: cd $HOME/packages/midas; make linux64

* login to the host computer to cross-compile ARM code:
* if it is an ARM linux: cd $HOME/packages/midas; make linuxarm # may need to install ARM cross compilers

== Prepare VME hardware ==

Hardware check list:
* VME crate
* VME processor (supported are V77xx, V7805, V7865)
* On all VME modules, set the VME address jumpers as described here: http://daq-plone.triumf.ca/SM/docs/local/vme_jumpers
* run vmescan to confirm correct VME addresses
** cd $HOME/packages
** svn checkout https://ladd00.triumf.ca/svn/daqsvn/trunk/vme
** cd vme
** make
** ./vmescan.exe (or _gef.exe, depending on the VME driver in use)

== Install Universe-II VME driver (V7648, V7750, V7805, V7851) ==

* login as root (ssh root@localhost)
* follow instructions: https://www.triumf.info/wiki/DAQwiki/index.php/VME-CPU#V7648.2C_V7750.2C_V7805.2C_V7851_:_Setup_vme_universe_VME_drivers
* cd ~/packages/vme; vmescan.exe

== Install Tsi-148 VME driver (V7865) ==

* login as root (ssh root@localhost)
* follow instructions: https://www.triumf.info/wiki/DAQwiki/index.php/VME-CPU#V7865_and_XVB-602_:_Setup_gefvme.2Ftsi148_VME_drivers
* cd ~/packages/vme; vmescan_gef.exe

== Setup the experiment environment ==

* Decide which computer will host MIDAS (where MIDAS shared memory buffers will reside).
: This computer will run the [[mserver]], [[mlogger]] and [[mhttpd]] applications. (It is usually the machine where the MIDAS,ROOT etc. packages have been downloaded). It will be referred to as the host machine (localhost).

The environment is slightly different depending on whether all programs run on the host machine, or whether some programs run on remote host(s) :
=== ALL programs run on localhost ===
:If all programs run on the host machine (localhost), it is not necessary to run [[mserver]]. [[Environment Variables#MIDAS_SERVER_HOST|MIDAS_SERVER_HOST]] will not be assigned (see example .cshrc [[#Prepare the user account|above]]).

=== Some programs run on REMOTE host(s) ===
: '''IMPORTANT:'''
# Since August 2015 '''you must explicitly allow access for clients running on remote machines'''. To do this, follow the '''[[Security#MIDAS programs on remote machines|instructions here]]'''.
# The example code .cshrc ([[#Prepare the user account|see above]]) should be present on both host and remote machine(s). This will ensure that [[Environment Variables#MIDAS_SERVER_HOST|MIDAS_SERVER_HOST]] will NOT be set for the host machine (localhost), but on a remote machine, MIDAS_SERVER_HOST will be set to the MIDAS host machine.
# The client [[mserver]] must be started on the MIDAS host machine. Note that multiple experiments can run on the same host machine by starting several instances of [[mserver]] (one for each experiment) running with different ports (and .cshrc would be edited so that MIDAS_SERVER_HOST is set to the appropriate port for the experiment).

 
On the host machine:
* mkdir $HOME/online
* cd $HOME/online
* create directories for local programs, sources, elog and history: mkdir bin src elog history
* create data directory: mkdir -p /ladd/data1/t2kvme5/data; ln -s /ladd/data1/t2kvme5/data $HOME/online
* create the exptab file "$HOME/online/exptab" following the example below. The first entry (exptname) is the name if the DAQ system (MIDAS experiment name), the second entry (/home/USER/online) is the location of MIDAS shared memory buffers (by convention, $HOME/online), the third entry (kopio03) is your username.
<pre>exptname /home/kopio03/online kopio03</pre>
* logout and login again for all changes to take effect

== Setup experiment startup scripts ==
* login to the experiment host computer
* echo $MIDAS_SERVER_HOST ### to check correct value - should be blank
* create $HOME/online/bin/start_daq.sh, replacing XXX with the hostname of the machine running the experiment (and changing the mserver and mhttpd ports, as needed).

#!/bin/sh
# start_daq.sh
cd $HOME/online
#
case `hostname` in XXX*)
echo "Good, we are on XXX!"
;;
*)
echo "The start_daq script should be executed on XXX"
exit 1
;;
esac
#
odbedit -c clean
# start [[mhttpd]] on default port. (Mongoose https version - see [[mhttpd]] for other options)
mhttpd -D -a localhost -a XXX.triumf.ca # optionally restrict access to specified hosts
#
# start [[mserver]] on default port (use argument -p to use a different port)
mserver -D # access must now be specifically allowed - see [[#Setup the experiment environment|above]]

# OR ([[#NOTES|older MIDAS versions]])
# mhttpd -p 8081 -D -a localhost -a XXX.triumf.ca # optionally restrict access to specified hosts
# mserver -p 7071 -D -a localhost -a lxdragon01.triumf.ca -a lxdragon02.triumf.ca -a XXX.triumf.ca # optionally restrict access to specified hosts

#
mlogger -D
#end file

== Run the MIDAS Web Server ==
Let's start the MIDAS webserver for the first time:

Start [[mhttpd]] on the ''experiment host'' (localhost) like this:
[mhostpc] mhttpd
You will get the following messages:
[mhttpd,INFO] ODB subtree /Runinfo corrected successfully
Mongoose web server will listen on ports "8080r,8443s"
[mhttpd,ERROR] [mhttpd.cxx:17892:mongoose,ERROR] cannot find SSL certificate file "/home/agdaq/online/ssl_cert.pem"
[mhttpd,ERROR] [mhttpd.cxx:17893:mongoose,ERROR] please create SSL certificate file: openssl req -new -nodes -newkey rsa:2048 -sha256 -out ssl_cert.csr -keyout ssl_cert.key; openssl x509 -req -days 365 -sha256 -in ssl_cert.csr -signkey ssl_cert.key -out ssl_cert.pem; cat ssl_cert.key >> ssl_cert.pem
could not start the mongoose web server, see messages and midas.log, bye!

Create a self-signed certificate suitable for initial testing by executing the command printed by mhttpd:


[mhostpc] openssl req -new -nodes -newkey rsa:2048 -sha256 -out ssl_cert.csr -keyout ssl_cert.key; openssl x509 -req -days 365 -sha256 -in ssl_cert.csr -signkey ssl_cert.key -out ssl_cert.pem; cat ssl_cert.key >> ssl_cert.pem

For production use, you should create a properly signed certificate, see [[Mhttpd#Create an SSL certificate|create your own SSL certificate]] or you should run mhttpd behind an SSL proxy.

Run mhttpd again.

You will get the following messages:
[mhttpd,INFO] ODB subtree /Runinfo corrected successfully
Mongoose web server will listen on ports "8080r,8443s"
Mongoose web server will use SSL certificate file "/home/johnfoo/packages/midas/ssl_cert.pem"
[mhttpd,ERROR] [mhttpd.cxx:17633:mongoose,ERROR] mongoose web server cannot find password file "/home/johnfoo/online/htpasswd.txt"
[mhttpd,ERROR] [mhttpd.cxx:17634:mongoose,ERROR] please create password file: htdigest -c /home/johnfoo/online/htpasswd.txt Default midas
could not start the mongoose web server, see messages and midas.log, bye!

Create the password file by following the instructions printed by mhttpd. The http digest domain name is the experiment name, suggested default user name is "midas". You will be asked to type in a password


[mhostpc] htdigest -c /home/johnfoo/online/htpasswd.txt exptname midas
Adding password for midas in realm exptname.
New password:
Re-type new password:

It is a good idea to set the password file {{Filepath|path=htpasswd.txt}} readable and writable by owner only.

Now restart {{Utility|name=mhttpd}}
[mhostpc] mhttpd
Mongoose web server will listen on ports "8080r,8443s" **see note
Mongoose web server will use SSL certificate file "/home/suz/packages/midas/ssl_cert.pem"
Mongoose web server will use authentication realm "Default", password file "./htpasswd.txt"

Now point a web browser running on the same host computer (localhost) to https://localhost:8443
If the web browser is running on a different computer, go to URL of the form

https://mhostpc.triumf.ca:8443 (substitute your host machine name and domain for "mhostpc.triumf.ca")

If you are using the default SSL certificate you will probably get a message: "This Connection is Untrusted". Click "I understand the risks" and add an exception. This is because the test certificate is self-signed. Then confirm an exception.

If instead you get a "connection refused" error, the midas host pc may have the firewall enabled. To make a firewall exception for MIDAS, follow instructions here http://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29

You should then see an authentication box asking you for the user name and password. The user name is "midas". Enter the password you just created. The Midas [[Status Page]] should appear with multiple buttons for run control as well as equipment listing (no equipments will be listed as yet) and application listings. Please refer to [[mhttpd]] (the MIDAS Web-based Run Control utility) for further information. You can start and stop runs from the main status page, and use the [[ODB Page]] to access the database (ODB).

; Note
: Default ports of 8080 and 8443 are used by [[mhttpd]]. If these ports are in use on your machine, start mhttpd with alternative ports, e.g.
[mhostpc] mhttpd --https 8448 --http 8089
: or see [[Mhttpd#Usage]] to change the default ports.

== Setup experiment database (ODB) ==

* run $HOME/online/bin/start_daq.sh

* odbedit, run these commands: (replace user names and directory names)
<pre>
# set "/Logger/Message file" "/home/kopio03/online/midas.log" # obsolete
set "/Logger/Data Dir" "/home/kopio03/online/data"
create STRING "/Logger/History dir"
set "/Logger/History dir" "/home/kopio03/online/history"
create STRING "/Logger/Elog dir"
set "/Logger/Elog dir" "/home/kopio03/online/elog"
set "/Logger/ODB dump file" "/home/kopio03/online/history/run%05d.xml"
set "/Logger/ODB dump" "y"
set "/Logger/Channels/0/Settings/Filename" "run%05dsub%03d.mid.gz"
set "/Logger/Channels/0/Settings/Subrun byte limit" "1000000000"
set "/Logger/Channels/0/Settings/Compression" 1
set "/Logger/Channels/0/Settings/ODB Dump" "y"
set "/Programs/Logger/Required" y
set "/Programs/Logger/Start command" "mlogger -D"
set "/Programs/fevme/Required" "y"
set "/Programs/fevme/Start command" "ssh -n lxdaq09 $HOME/online/src/fevme_gef.exe -O"
exit
</pre>
* open web browser e.g. firefox.
* go to the midas status page at https://localhost:8443 (default port).
** if running [[mhttpd]] with Mongoose HTTPS/OpenSSL (the default) for the first time, you will need to create a password file. Follow the instructions (see [[mhttpd#HTTPS/SSL server (Mongoose)]] for details).
** For other options (i.e. HTTPS/SSL proxy) see [[#Secure MIDAS and ELOG Web access]]
*OR open the midas status page at http://localhost:8081 ([[#NOTES|older MIDAS versions]])
* midas status page will show most stuff "red" as nothing is running yet
* DON'T DO THIS YET run ./fevme.exe (on the computer with the VME interface, could be different from computer hosting the experiment), observe that corresponding equipments have been created
* save the url bookmark to the "personal toolbar"
* go to the Programs page, stop mlogger, stop fevme, start mlogger, start fevme
* go to the Status page, start run, stop run
* go back to the Status page, everything should be green
* start a run
* send signals to the ADC gate
* you should be getting events
* to look at data, proceed with setting up the [[ROOTANA|ROOT Analyzer]].

== Start DAQ programs at boot time ==

* add this to /etc/rc.local (replace username and location of the start_daq script)
<pre>
su - alpha -c /home/alpha/online/bin/start_daq.sh
</pre>

== Setup local software version control ==

Version control for experiment source code is setup using "git" (http://git-scm.com/)

* cd $HOME/online
* git init
* git add exptab
* git add bin/start_daq.sh
* git add .gitignore ### contents can be
<pre>
*~
*.o
*.exe
</pre>
* git add src/Makefile src/*.cxx ...
* git commit -a

== Adjust MIDAS buffer sizes ==

Default MIDAS SYSTEM buffer size is 8 Mbytes, fairly small for high-data-rate experiments. The rule of thumb is to have at least a few seconds worth of buffer space available. For example, if event size is 10 Kbytes and the event rate is 1 kHz, data rate is 10*10^3*1*10^3 = 10 Mbytes/sec. To buffer 10 seconds of data we need 100 Mbytes of buffer space.

To resize the MIDAS event buffers (SYSTEM, etc) do this:
* stop all frontends, stop mlogger
* start odbedit:
** cd "/Experiment/Buffer sizes"
** set SYSTEM 100000000
* run "mdump -z SYSTEM"
* if mdump complains about the size of .SYSTEM.SHM, remove it, try again.
* ls -l /dev/shm ### to observe that the size of shared memory is correct

== Secure MIDAS and ELOG Web access ==
In versions prior to May 2015, the default web access to MIDAS and ELOG uses the "http:" protocol which is insecure. In this case, all information is transmitted as clear text meaning that secret, confidential and sensitive information (such as the MIDAS and ELOG passwords and usernames) can be stolen "easily". This means that even "password protected" MIDAS and ELOG pages are not really protected if accessed using the "http" method.

Better security for HTTP is gained by using a password protected '''SSL (https) proxy'''. (It does not provide absolute security because of remaining problems with the security of SSL certificates, security of passwords, etc). Setting up an SSL (https) proxy is described [[#Setting up an HTTP proxy|below]].

Since May 2015, an ''alternative secure option'' to setting up an HTTP proxy is available to users of MIDAS. Recent versions of elogd (ELOG) do support SSL https:// connections, and [[#mhttpd with HTTPS/SSL server (Mongoose)]] is now available. This option is the default, and provides a similar level of security to an HTTP proxy.

See [[Security#Web Access]] for a comparison of these two secure options.

=== mhttpd with HTTPS/SSL server (Mongoose) ===

Since May 2015 the MIDAS web server [[mhttpd]] is explicitly linked with OpenSSL to provide secure HTTPS connections via the [https://bitbucket.org/tmidas/midas/src/ecb9a8537448a8a43f7f9a2bfdb82e578208cde3/doc/mongoose/?at=develop Mongoose] web server (see [[mhttpd]]). With this version, default web access to MIDAS uses the "https" protocol. Web access to {{Utility|name=mhttpd}} can be restricted by using the "-a hostname" switch of [[mhttpd]]. The first time {{Utility|name=mhttpd}} is run, a password file must be created. An SSL certificate is also required. See [[mhttpd#HTTPS/SSL server (Mongoose)|HTTPS/SSL server (Mongoose)]] for instructions.

=== mhttpd using an HTTPS/SSL proxy ===

THESE INSTRUCTIONS ARE WRONG, DO NOT DO THIS. See instead [https://daq00.triumf.ca/MidasWiki/index.php/Setup_MIDAS_experiment_at_TRIUMF#Install_https_proxy this section].

An [[#Setting up an HTTP proxy|HTTP proxy]] must be set up. This is the only way of securing older version of [[mhttpd]] (pre August 2015). Older versions of mhttpd are started using the -p port option e.g.
* mhttpd -D -p 8080 

To run a new version of mhttpd using an HTTP proxy, use the options provided to run the old (non-Mongoose) webserver on a given port, i.e.
* mhttpd --oldserver 8080 --nomg -D 

When using an SSL proxy, only access from the SSL proxy (and maybe some special trusted machines) should be permitted.
This is done using the "-a hostname" switch of [[mhttpd]]. Normally there will be only "-a localhost" switch, enabling access only for the local machine (where the SSL proxy is running). Additional "-a hostname" switches enable access from listed local machines. No "-a xxx" enables access from everywhere (defeating the purpose of the SSL proxy, unless access controls are enforced elsewhere, i.e. by a site firewall or by local firewall rules).

== Setting up an HTTP proxy ==
In this example, we use APACHE HTTPD to password-protect a typical midas/mhttpd and elog installation.

In this configuration, one uses the Linux stock httpd that accepts encrypted https:// connections and forwards them to mhttpd and elogd. Instead of (or in addition to) using mhttpd and elogd passwords, one configures password protection in httpd via the regular apache httpd password mechanisms (htpasswd, etc).

Recent versions of elogd do support SSL https:// connections, but if one is running an SSL proxy for anyway, it is simpler to run both through the same SSL proxy using the same SSL host certificate and the same httpd password file.

=== Restricting http: access to elogd ===
;Note
:Recent versions of elogd do support SSL https:// connections. The following information is for those using an HTTP proxy (see above).

For elogd, this is done using the "-n localhost" switch with enables only access from the same machine if present, or access from anywhere is absent (defeating the purpose of the SSL proxy, unless access controls are enforced elsewhere).

(It is recommended to run elogd from the same user as the main daq user and to keep elogd.cfg and all logbooks in the home directory of this user, where they are captured by the normal site backup system)

== Install standalone elog ==

* login into the user account that will run the elog
* cd $HOME/packages
* git clone https://bitbucket.org/ritt/elog
* cd elog
* make
* create new file start_elogd with this contents:
<pre>
#!/bin/sh

killall elogd
sleep 1
killall -KILL elogd
sleep 1
$HOME/packages/elog/elogd -n localhost -x -c $HOME/packages/elog/elogd.cfg -p 8082 -D

#end
</pre>
* chmod a+x start_elogd
* edit elogd.cfg to read:
<pre>
[global]
port = 8082
SMTP host = smtp.triumf.ca
URL = https://titan00.triumf.ca/elog/

Reverse sort = 1
Display Mode = full

#List Menu commands = New, Find, Admin, Help
#Menu commands = New, Edit, Reply, Find, Duplicate, Help

Entries Per Page = 30
Supress Email on edit = 1
Default encoding = 1
Page title = TITAN ELOG
Resolve host names = 1

Logfile = /home/titan/packages/elog/elogd.log
#Logging level = 3

[midas]

List page Title = T2K M11 MIDAS ELOG
Comment = T2K M11 MIDAS ELOG
Page Title = T2K M11 MIDAS ELOG
RSS Title = [$logbook - $type - $system] $subject, posted by $author

Attributes = Author, Subject, Run, Type, System
Show Attributes Edit = Run, Author, Subject, Type, System
Required Attributes = Author, Type, System, Subject

Options Type = Routine, Reply, Shift Summary, Modification, Question, Info, Problem
Options System = General, DAQ, Beamline

Preset Run = $shell(MIDASSYS=. /home/t2km11/packages/midas/linux/bin/odbedit -d Runinfo -c 'ls -v \"run number\"')

Preset On Reply Type = Reply
Preset On Reply Run = $shell(MIDASSYS=. /home/t2km11/packages/midas/linux/bin/odbedit -d Runinfo -c 'ls -v \"run number\"')

List Display = Date, Subject, Type, System, Author, ID
Quick Filter = Date, Type, ID

Remove on reply = Author
Quote on reply = 1

Use lock = 1

************* Email Functionality ****************

Use Email Subject = [T2KM11 - $System] $Subject
Omit Email To = 1

Email System General = xxx
</pre>
* ./start_elogd &
* firefox http://localhost:8082 # hould show the elog message index

To start elogd automatically when the machine is rebooted, login as root and
* add this text to /etc/rc.local:
<pre>
su - titan -c "/home/titan/packages/elog/start_elogd"
</pre>
* chmod a+x /etc/rc.local
* systemctl start rc-local

To import elog entries from the mhttpd elog, do this:

* cd ~/packages/elog/logbooks
* ln -s /home/t2km11/online/elog midas
* cd midas
* ~/packages/elog/elconv)

== Install https proxy ==

THESE INSTRUCTIONS ARE OBSOLETE, INSTEAD,
* GO HERE: https://daq00.triumf.ca/DaqWiki/index.php/Ubuntu#Install_apache_httpd_proxy_for_midas_and_elog
* AND GO HERE: https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux#Run_the_MIDAS_Web_Server

FOLLOWING INSTRUCTIONS ARE OBSOLETE...

* login as root to the https proxy machine
* cd ~root
* yum install mod_ssl
* yum install crypto-utils # see http://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_monitoring_of_HTTPS_certificates
* create a certificate request (replace ladd09 with your hostname): openssl req -new -nodes -newkey rsa:2048 -sha256 -out ladd09.csr -keyout ladd09.key (answer: CA, BC, Vancouver, TRIUMF, DAQ, ladd09.triumf.ca, email@email.com
* sign it by TRIUMF:
** mail -s "Certificate request" yourself@email.com < ladd09.csr
** forward this request to Andrew Daviel
** he will email the signed crt file, copy it to this system as ladd09.crt
* sign it yourself: openssl x509 -req -days 365 -sha256 -in ladd09.csr -signkey ladd09.key -out ladd09.crt
* (if the certificate expires, renew it by signing it again)
* Additional commands for working with certificates:
** explore the private key: openssl pkey -in ladd09.key -text -noout
** explore the certificate request: openssl req -in ladd00.csr -text -noout
** explore the certificate: openssl x509 -in ladd09.crt -noout -text
* move certificate files to proper system locations:
* mv ladd09.key /etc/pki/tls/private/
* mv ladd09.crt /etc/pki/tls/certs/
* if selinux is enabled, do this:
** restorecon -Rv /etc/pki/tls/certs/
** restorecon -Rv /etc/pki/tls/private/
** /usr/sbin/setsebool -P httpd_can_network_connect 1
* open /etc/httpd/conf.d/ssl.conf in a text editor, go to the very bottom and right before the "</VirtualHost>" entry, add following text:
<pre>
...
SSLCertificateFile /etc/pki/tls/certs/ladd09.crt
SSLCertificateKeyFile /etc/pki/tls/private/ladd09.key

ProxyPass /elog/ http://localhost:8082/ retry=1
ProxyPass / http://localhost:8080/ retry=1

<Location />

SSLRequireSSL
AuthType Basic
AuthName "password protected site"
Require valid-user

# create password file: touch /etc/httpd/htpasswd
# to add new user or change password: htpasswd /etc/httpd/htpasswd username
AuthUserFile /etc/httpd/htpasswd

</Location>
</VirtualHost>
...
</pre>
* comment out duplicate "SSLCertificateFile" and "SSLCertificateKeyFile" elsewhere in the file
* (optionally) If you got a certificate that is signed by DigiCert or RapidSSL then you'll need to add a line specifying the certificate chain file:
<pre>
...
SSLCertificateChainFile /etc/pki/tls/certs/DigiCertCA.crt
...
</pre>
* touch /etc/httpd/htpasswd
* htpasswd /etc/httpd/htpasswd midas # enter password midas
* chkconfig httpd on
* service httpd restart
* firewall-cmd --add-port=443/tcp --permanent
* firewall-cmd --reload
* firewall-cmd --list-all
* test it
** test the SSL proxy: https://host/ should yield the midas status page, https://host/elog/ should yield the elog message index
* in ODB, set "/Elog/URL" to "https://host/elog/"
* now from the midas status page, the "Elog" button should take us to the https Elog URL

In needed, enable user directories: https://blah/~user in ~user/public_html

* edit /etc/httpd/conf.d/userdir.conf, replace "UserDir disabled" with "UserDir enabled"
* setsebool -P httpd_enable_homedirs true
* systemctl restart httpd

== Setup the history mhttpd for faster access to history plots ==
When running an SSL proxy,
* start the main mhttpd (orange command for old mhttpd, green for new mhttpd with Mongoose(post August2015):
** "mhttpd -p 8071 -D" or
** "mhttpd -D --oldserver 8071 --nomg" 

* start the history mhttpd
** "mhttpd -p 8072 -D -H" or
** "mhttpd -D -H --oldserver 8072 --nomg" 
* set ODB /History/URL to "http://alphacpc09.cern.ch:8072/HS/"
* open the MIDAS status page
* go to the history section, try to open any history plot, observe that the history plot gif image loads correctly, inspect it's URL (use "copy image URL" or "view source", etc), it should point to port 8072 causing connection to the history mhttpd.
* continue with these instructions to setup history mhttpd access through an SSL proxy:
* setup SSL proxy access (required mod_proxy_html)
** login as root to the SSL proxy machine
** on SL5, install the missing mod_proxy_html httpd module:
** yum install httpd-devel libxml2-devel
** wget http://apache.webthing.com/mod_proxy_html/mod_proxy_html.tar.bz2
** tar xjvf mod_proxy_html.tar.bz2
** cd mod_proxy_html
** apxs -c -I. -I/usr/include/libxml2 -i mod_proxy_html.c
** apxs -c -I. -I/usr/include/libxml2 -i mod_xml2enc.c
** cd /etc/httpd/conf.d, add this to ssl.conf:

;before the ProxyPass statements:
<pre>
# proxy the MIDAS web servers
LoadModule xml2enc_module modules/mod_xml2enc.so
LoadModule proxy_html_module modules/mod_proxy_html.so
ProxyHTMLLinks a href
ProxyHTMLLinks link href
ProxyHTMLLinks img src
#ProxyHTMLEnable On
ProxyRequests off
</pre>
;after the ProxyPass statements:
<pre>
# ALPHA1 history access
ProxyPass /alpha1/history/ http://alphacpc09.cern.ch:8072/HS/ retry=1
ProxyPass /alpha1/ http://alphacpc09.cern.ch:8071/ retry=1

ProxyHTMLEnable On
ProxyHTMLURLMap http://alphacpc09.cern.ch:8072/HS/ /alpha1/history/
</pre>
;adjust:
*"alpha1" is the experiment name
*"alphacpc09.cern.ch" is the machine running mhttpd
*"8071" is the port number of the main mhttpd
** "mhttpd -p 8071 -D" or
** "mhttpd -D --oldserver 8071 --nomg" 
* "8072" is the port number of the history mhttpd
** "mhttpd -p 8072 -D -H" or
** "mhttpd -D -H --oldserver 8072 --nomg" 

[[Category:Installation]] [[Category:Buffer]] [[Category:Driver]]

Setup MIDAS experiment at TRIUMF

2026-02-03T17:32:31Z

Dfujimoto: /* mhttpd using an HTTPS/SSL proxy */

{{Pagelinks}}
== Introduction ==
This page describes setting up a MIDAS experiment at TRIUMF. This information can be adapted for other sites.

== Environment variables ==
* '''MIDASSYS''' Base directory of the MIDAS package, midas and mxml should be at the same level.
* '''MIDAS_EXPTAB''' Experiment definition file
* '''MIDAS_SERVER_HOST''' MIDAS host server name for remote midas connections.
* '''MIDAS_EXPT_NAME''' Experiment name

== Standard layout of MIDAS experiment ==
The following shows the directory layout of a standard MIDAS experiment:

/home/exptuser/
packages/
root <---- ROOT
mxml
mscb
midas/ <---- MIDAS
linux/{lib,bin} <---- binaries matching the selected 64-bit/32-bit flavour of ROOT
linux-m32/{lib,bin} <---- limited function 32-bit binaries for 32-bit frontend machines, build by "make linux32"
linux-m64/{lib,bin} <---- limited function 64-bit binaries (only needed if ROOT and linux/bin are 32-bit)
linux-arm/{lib,bin} <---- full function ARM cross-compiled using "make linuxarm"
linux-crosscompile/{lib,bin} <---- cross-compiled limited function binaries for PPC and ARM frontends (see Makefile)
rootana <---- ROOT analyzer
roody <---- graphical online histogram viewer for MIDAS and ROOTANA
online/
exptab <---- experiment definition
{.ODB,.SYSTEM,.SYSMSG,etc}.SHM <---- MIDAS shared memory save files
src <---- experiment frontend sources
bin,scripts
elog <---- MIDAS elog
history <---- MIDAS history
data -> /data/exptname/current <---- symlink to the data directory
/data/exptname/current <---- experiment data directory with ODB save files and MIDAS .mid/.mid.gz data files

== Prepare computers ==

On some operating systems, several MIDAS functions require administrator access:

* on el7 linux (SL7/CC7/CentOS7/RHEL7) - access to mhttpd port 8443 requires special firewall rules, see here: https://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29
* on el7 linux - access to mserver to run frontends and other programs on some other computer requires special firewall rules, see here: https://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29
* on el7 linux - on the frontend machines (and other machines that will connect to the mserver, the same firewall rule needs to be created (use the IP address of the machine running the mserver)

== Prepare the user account ==
<div id="NOTES"></div>

* Setup the user account for running this instance of midas. For machines part of the LADD cluster, follow these [http://daq-plone.triumf.ca/SM/docs/local/NewLaddUser] instructions.
* check that the account is using the /bin/bash shell
* make $HOME/.profile look like this:

<pre>
#!/bin/echo You must source

export SVN_EDITOR="emacs -nw"
export GIT_EDITOR="emacs -nw"
export MIDASSYS=$HOME/packages/midas
export ROOTANASYS=$HOME/packages/rootana
export MIDAS_EXPTAB=$HOME/online/exptab
#
# setup the MIDAS mserver
#
case `hostname` in
daq07*)
unset MIDAS_SERVER_HOST
;;
*)
export MIDAS_SERVER_HOST=daq07.triumf.ca:7070
;;
esac
#
# select 64-bit or 32-bit MIDAS and ROOT
#
case `uname -i` in
i386)
source /daq/daqshare/olchansk/root/root_v5.34.01_SL62_32/bin/thisroot.sh
export PATH=.:$MIDASSYS/linux-m32/bin:$PATH
;;
*)
#source /daq/daqshare/olchansk/root/root_v5.34.34_SL67_64/bin/thisroot.sh
source $HOME/packages/root/bin/thisroot.sh
export PATH=.:$MIDASSYS/linux/bin:$PATH
;;
esac
#
export PATH=.:$HOME/online/bin:$HOME/packages/roody/bin:$PATH
#
#end
</pre>

* mkdir $HOME/packages
* Logout and login again, for .cshrc changes to take effect

== Install ROOT ==

* Identify the Linux version: RH9 (Red Hat Linux 9), FC3 (Fedora Core 3), RHEL4/SL4 (Red Hat Enterprise LInux 4/Scientific Linux 4), SL5x, SL6x, (CentOS/CC/SL) el7x: more /etc/redhat-release
* Decide to use 32-bit or 64-bit ROOT ('uname -a')
* cd $HOME/packages
* ls -l /daq/daqshare/olchansk/root/ ### to see all available ROOT packages
* ln -s /daq/daqshare/olchansk/root/root_vNNN_VVV_BB root, where NNN is the latest available version of ROOT ("ls -l /daq/daqshare/olchansk/root"), VVV is the Linux version code (RH9, FC3, SL4, etc) and BB is "_32" or "_64" for 32-bit or 64-bit ROOT. For example: /daq/daqshare/olchansk/root/root_v5.10.00_SL40
* for example: ln -s /daq/daqshare/olchansk/root/root_v5.34.34_el72_64 $HOME/packages/root
* Check that ROOT works: "source $HOME/packages/root/bin/thisroot.sh; root". Type ".q" to exit root.

== Install MIDAS ==

* cd $HOME/packages
* (OBSOLETE) svn co svn+ssh://svn@savannah.psi.ch/repos/meg/midas/trunk midas, password "svn". (password has to be entered twice)
* (OBSOLETE) svn co svn+ssh://svn@savannah.psi.ch/repos/meg/mxml/trunk mxml
* git clone https://bitbucket.org/tmidas/midas --recursive
* (OBSOLETE) git clone https://bitbucket.org/tmidas/mxml
* (OBSOLETE) git clone https://bitbucket.org/tmidas/mscb
* (IF BITBUCKET IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/mxml.git
* (IF BITBUCKET IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/mscb.git
* (IF BITBUCKET IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/midas.git
* cd midas
* make
* (only if needed) make linux32 ### build the 32-bit MIDAS libraries
* ls -l linux/bin/odbedit ### check that odbedit has been created (do not run it yet)

You can see a list of other installation problems at [[Common problems & Debugging recipes]].

;NOTE 1
: Optional features in MIDAS can be explicitly disabled if desired when making MIDAS using the NO_xxx feature (NO_ROOT,NO_MYSQL,NO_ODBC,NO_SQLITE,NO_MSCB), e.g. "make NO_ROOT=1" to disable ROOT. These NO_xxx Makefile variables are only used to control autodetection.

;NOTE 2
: Since June 2019 the mxml and mscb packages are submodules of the midas package, so no need to clone them separately. If you have an existing clone of midas but not yet the submodules, you need

$ git submodule update --init --recursive

: To update both midas and the submodules, you need

$ git pull --recurse-submodules

== Install ROOTANA ==

* cd $HOME/packages
* git clone https://bitbucket.org/tmidas/rootana
* cd rootana
* make

== Install ROODY ==

* cd $HOME/packages
* git clone https://bitbucket.org/tmidas/roody
* cd roody
* make
* $HOME/packages/roody/bin/roody, run the program

== Install additional additional packages ==

* cd $HOME/packages
* git clone https://bitbucket.org/ttriumfdaq/vme
* git clone https://bitbucket.org/ttriumfdaq/frontends

== Build special versions of MIDAS ==

Build special versions of MIDAS for the case when some MIDAS programs, such as VME frontends, will run on a different computer that may have a different flavour of operating system, i.e. 32-bit linux or an older version of Scientific Linux.

* login to the computer where the frontends will run and:
* if it is a 32-bit linux: cd $HOME/packages/midas; make linux32
* if it is a 64-bit linux: cd $HOME/packages/midas; make linux64

* login to the host computer to cross-compile ARM code:
* if it is an ARM linux: cd $HOME/packages/midas; make linuxarm # may need to install ARM cross compilers

== Prepare VME hardware ==

Hardware check list:
* VME crate
* VME processor (supported are V77xx, V7805, V7865)
* On all VME modules, set the VME address jumpers as described here: http://daq-plone.triumf.ca/SM/docs/local/vme_jumpers
* run vmescan to confirm correct VME addresses
** cd $HOME/packages
** svn checkout https://ladd00.triumf.ca/svn/daqsvn/trunk/vme
** cd vme
** make
** ./vmescan.exe (or _gef.exe, depending on the VME driver in use)

== Install Universe-II VME driver (V7648, V7750, V7805, V7851) ==

* login as root (ssh root@localhost)
* follow instructions: https://www.triumf.info/wiki/DAQwiki/index.php/VME-CPU#V7648.2C_V7750.2C_V7805.2C_V7851_:_Setup_vme_universe_VME_drivers
* cd ~/packages/vme; vmescan.exe

== Install Tsi-148 VME driver (V7865) ==

* login as root (ssh root@localhost)
* follow instructions: https://www.triumf.info/wiki/DAQwiki/index.php/VME-CPU#V7865_and_XVB-602_:_Setup_gefvme.2Ftsi148_VME_drivers
* cd ~/packages/vme; vmescan_gef.exe

== Setup the experiment environment ==

* Decide which computer will host MIDAS (where MIDAS shared memory buffers will reside).
: This computer will run the [[mserver]], [[mlogger]] and [[mhttpd]] applications. (It is usually the machine where the MIDAS,ROOT etc. packages have been downloaded). It will be referred to as the host machine (localhost).

The environment is slightly different depending on whether all programs run on the host machine, or whether some programs run on remote host(s) :
=== ALL programs run on localhost ===
:If all programs run on the host machine (localhost), it is not necessary to run [[mserver]]. [[Environment Variables#MIDAS_SERVER_HOST|MIDAS_SERVER_HOST]] will not be assigned (see example .cshrc [[#Prepare the user account|above]]).

=== Some programs run on REMOTE host(s) ===
: '''IMPORTANT:'''
# Since August 2015 '''you must explicitly allow access for clients running on remote machines'''. To do this, follow the '''[[Security#MIDAS programs on remote machines|instructions here]]'''.
# The example code .cshrc ([[#Prepare the user account|see above]]) should be present on both host and remote machine(s). This will ensure that [[Environment Variables#MIDAS_SERVER_HOST|MIDAS_SERVER_HOST]] will NOT be set for the host machine (localhost), but on a remote machine, MIDAS_SERVER_HOST will be set to the MIDAS host machine.
# The client [[mserver]] must be started on the MIDAS host machine. Note that multiple experiments can run on the same host machine by starting several instances of [[mserver]] (one for each experiment) running with different ports (and .cshrc would be edited so that MIDAS_SERVER_HOST is set to the appropriate port for the experiment).

 
On the host machine:
* mkdir $HOME/online
* cd $HOME/online
* create directories for local programs, sources, elog and history: mkdir bin src elog history
* create data directory: mkdir -p /ladd/data1/t2kvme5/data; ln -s /ladd/data1/t2kvme5/data $HOME/online
* create the exptab file "$HOME/online/exptab" following the example below. The first entry (exptname) is the name if the DAQ system (MIDAS experiment name), the second entry (/home/USER/online) is the location of MIDAS shared memory buffers (by convention, $HOME/online), the third entry (kopio03) is your username.
<pre>exptname /home/kopio03/online kopio03</pre>
* logout and login again for all changes to take effect

== Setup experiment startup scripts ==
* login to the experiment host computer
* echo $MIDAS_SERVER_HOST ### to check correct value - should be blank
* create $HOME/online/bin/start_daq.sh, replacing XXX with the hostname of the machine running the experiment (and changing the mserver and mhttpd ports, as needed).

#!/bin/sh
# start_daq.sh
cd $HOME/online
#
case `hostname` in XXX*)
echo "Good, we are on XXX!"
;;
*)
echo "The start_daq script should be executed on XXX"
exit 1
;;
esac
#
odbedit -c clean
# start [[mhttpd]] on default port. (Mongoose https version - see [[mhttpd]] for other options)
mhttpd -D -a localhost -a XXX.triumf.ca # optionally restrict access to specified hosts
#
# start [[mserver]] on default port (use argument -p to use a different port)
mserver -D # access must now be specifically allowed - see [[#Setup the experiment environment|above]]

# OR ([[#NOTES|older MIDAS versions]])
# mhttpd -p 8081 -D -a localhost -a XXX.triumf.ca # optionally restrict access to specified hosts
# mserver -p 7071 -D -a localhost -a lxdragon01.triumf.ca -a lxdragon02.triumf.ca -a XXX.triumf.ca # optionally restrict access to specified hosts

#
mlogger -D
#end file

== Run the MIDAS Web Server ==
Let's start the MIDAS webserver for the first time:

Start [[mhttpd]] on the ''experiment host'' (localhost) like this:
[mhostpc] mhttpd
You will get the following messages:
[mhttpd,INFO] ODB subtree /Runinfo corrected successfully
Mongoose web server will listen on ports "8080r,8443s"
[mhttpd,ERROR] [mhttpd.cxx:17892:mongoose,ERROR] cannot find SSL certificate file "/home/agdaq/online/ssl_cert.pem"
[mhttpd,ERROR] [mhttpd.cxx:17893:mongoose,ERROR] please create SSL certificate file: openssl req -new -nodes -newkey rsa:2048 -sha256 -out ssl_cert.csr -keyout ssl_cert.key; openssl x509 -req -days 365 -sha256 -in ssl_cert.csr -signkey ssl_cert.key -out ssl_cert.pem; cat ssl_cert.key >> ssl_cert.pem
could not start the mongoose web server, see messages and midas.log, bye!

Create a self-signed certificate suitable for initial testing by executing the command printed by mhttpd:


[mhostpc] openssl req -new -nodes -newkey rsa:2048 -sha256 -out ssl_cert.csr -keyout ssl_cert.key; openssl x509 -req -days 365 -sha256 -in ssl_cert.csr -signkey ssl_cert.key -out ssl_cert.pem; cat ssl_cert.key >> ssl_cert.pem

For production use, you should create a properly signed certificate, see [[Mhttpd#Create an SSL certificate|create your own SSL certificate]] or you should run mhttpd behind an SSL proxy.

Run mhttpd again.

You will get the following messages:
[mhttpd,INFO] ODB subtree /Runinfo corrected successfully
Mongoose web server will listen on ports "8080r,8443s"
Mongoose web server will use SSL certificate file "/home/johnfoo/packages/midas/ssl_cert.pem"
[mhttpd,ERROR] [mhttpd.cxx:17633:mongoose,ERROR] mongoose web server cannot find password file "/home/johnfoo/online/htpasswd.txt"
[mhttpd,ERROR] [mhttpd.cxx:17634:mongoose,ERROR] please create password file: htdigest -c /home/johnfoo/online/htpasswd.txt Default midas
could not start the mongoose web server, see messages and midas.log, bye!

Create the password file by following the instructions printed by mhttpd. The http digest domain name is the experiment name, suggested default user name is "midas". You will be asked to type in a password


[mhostpc] htdigest -c /home/johnfoo/online/htpasswd.txt exptname midas
Adding password for midas in realm exptname.
New password:
Re-type new password:

It is a good idea to set the password file {{Filepath|path=htpasswd.txt}} readable and writable by owner only.

Now restart {{Utility|name=mhttpd}}
[mhostpc] mhttpd
Mongoose web server will listen on ports "8080r,8443s" **see note
Mongoose web server will use SSL certificate file "/home/suz/packages/midas/ssl_cert.pem"
Mongoose web server will use authentication realm "Default", password file "./htpasswd.txt"

Now point a web browser running on the same host computer (localhost) to https://localhost:8443
If the web browser is running on a different computer, go to URL of the form

https://mhostpc.triumf.ca:8443 (substitute your host machine name and domain for "mhostpc.triumf.ca")

If you are using the default SSL certificate you will probably get a message: "This Connection is Untrusted". Click "I understand the risks" and add an exception. This is because the test certificate is self-signed. Then confirm an exception.

If instead you get a "connection refused" error, the midas host pc may have the firewall enabled. To make a firewall exception for MIDAS, follow instructions here http://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29

You should then see an authentication box asking you for the user name and password. The user name is "midas". Enter the password you just created. The Midas [[Status Page]] should appear with multiple buttons for run control as well as equipment listing (no equipments will be listed as yet) and application listings. Please refer to [[mhttpd]] (the MIDAS Web-based Run Control utility) for further information. You can start and stop runs from the main status page, and use the [[ODB Page]] to access the database (ODB).

; Note
: Default ports of 8080 and 8443 are used by [[mhttpd]]. If these ports are in use on your machine, start mhttpd with alternative ports, e.g.
[mhostpc] mhttpd --https 8448 --http 8089
: or see [[Mhttpd#Usage]] to change the default ports.

== Setup experiment database (ODB) ==

* run $HOME/online/bin/start_daq.sh

* odbedit, run these commands: (replace user names and directory names)
<pre>
set "/Logger/Message file" "/home/kopio03/online/midas.log"
set "/Logger/Data Dir" "/home/kopio03/online/data"
create STRING "/Logger/History dir"
set "/Logger/History dir" "/home/kopio03/online/history"
create STRING "/Logger/Elog dir"
set "/Logger/Elog dir" "/home/kopio03/online/elog"
set "/Logger/ODB dump file" "/home/kopio03/online/history/run%05d.xml"
set "/Logger/ODB dump" "y"
set "/Logger/Channels/0/Settings/Filename" "run%05dsub%03d.mid.gz"
set "/Logger/Channels/0/Settings/Subrun byte limit" "1000000000"
set "/Logger/Channels/0/Settings/Compression" 1
set "/Logger/Channels/0/Settings/ODB Dump" "y"
set "/Programs/Logger/Required" y
set "/Programs/Logger/Start command" "mlogger -D"
set "/Programs/fevme/Required" "y"
set "/Programs/fevme/Start command" "ssh -n lxdaq09 $HOME/online/src/fevme_gef.exe -O"
exit
</pre>
* open web browser e.g. firefox.
* go to the midas status page at https://localhost:8443 (default port).
** if running [[mhttpd]] with Mongoose HTTPS/OpenSSL (the default) for the first time, you will need to create a password file. Follow the instructions (see [[mhttpd#HTTPS/SSL server (Mongoose)]] for details).
** For other options (i.e. HTTPS/SSL proxy) see [[#Secure MIDAS and ELOG Web access]]
*OR open the midas status page at http://localhost:8081 ([[#NOTES|older MIDAS versions]])
* midas status page will show most stuff "red" as nothing is running yet
* DON'T DO THIS YET run ./fevme.exe (on the computer with the VME interface, could be different from computer hosting the experiment), observe that corresponding equipments have been created
* save the url bookmark to the "personal toolbar"
* go to the Programs page, stop mlogger, stop fevme, start mlogger, start fevme
* go to the Status page, start run, stop run
* go back to the Status page, everything should be green
* start a run
* send signals to the ADC gate
* you should be getting events
* to look at data, proceed with setting up the [[ROOTANA|ROOT Analyzer]].

== Start DAQ programs at boot time ==

* add this to /etc/rc.local (replace username and location of the start_daq script)
<pre>
su - alpha -c /home/alpha/online/bin/start_daq.sh
</pre>

== Setup local software version control ==

Version control for experiment source code is setup using "git" (http://git-scm.com/)

* cd $HOME/online
* git init
* git add exptab
* git add bin/start_daq.sh
* git add .gitignore ### contents can be
<pre>
*~
*.o
*.exe
</pre>
* git add src/Makefile src/*.cxx ...
* git commit -a

== Adjust MIDAS buffer sizes ==

Default MIDAS SYSTEM buffer size is 8 Mbytes, fairly small for high-data-rate experiments. The rule of thumb is to have at least a few seconds worth of buffer space available. For example, if event size is 10 Kbytes and the event rate is 1 kHz, data rate is 10*10^3*1*10^3 = 10 Mbytes/sec. To buffer 10 seconds of data we need 100 Mbytes of buffer space.

To resize the MIDAS event buffers (SYSTEM, etc) do this:
* stop all frontends, stop mlogger
* start odbedit:
** cd "/Experiment/Buffer sizes"
** set SYSTEM 100000000
* run "mdump -z SYSTEM"
* if mdump complains about the size of .SYSTEM.SHM, remove it, try again.
* ls -l /dev/shm ### to observe that the size of shared memory is correct

== Secure MIDAS and ELOG Web access ==
In versions prior to May 2015, the default web access to MIDAS and ELOG uses the "http:" protocol which is insecure. In this case, all information is transmitted as clear text meaning that secret, confidential and sensitive information (such as the MIDAS and ELOG passwords and usernames) can be stolen "easily". This means that even "password protected" MIDAS and ELOG pages are not really protected if accessed using the "http" method.

Better security for HTTP is gained by using a password protected '''SSL (https) proxy'''. (It does not provide absolute security because of remaining problems with the security of SSL certificates, security of passwords, etc). Setting up an SSL (https) proxy is described [[#Setting up an HTTP proxy|below]].

Since May 2015, an ''alternative secure option'' to setting up an HTTP proxy is available to users of MIDAS. Recent versions of elogd (ELOG) do support SSL https:// connections, and [[#mhttpd with HTTPS/SSL server (Mongoose)]] is now available. This option is the default, and provides a similar level of security to an HTTP proxy.

See [[Security#Web Access]] for a comparison of these two secure options.

=== mhttpd with HTTPS/SSL server (Mongoose) ===

Since May 2015 the MIDAS web server [[mhttpd]] is explicitly linked with OpenSSL to provide secure HTTPS connections via the [https://bitbucket.org/tmidas/midas/src/ecb9a8537448a8a43f7f9a2bfdb82e578208cde3/doc/mongoose/?at=develop Mongoose] web server (see [[mhttpd]]). With this version, default web access to MIDAS uses the "https" protocol. Web access to {{Utility|name=mhttpd}} can be restricted by using the "-a hostname" switch of [[mhttpd]]. The first time {{Utility|name=mhttpd}} is run, a password file must be created. An SSL certificate is also required. See [[mhttpd#HTTPS/SSL server (Mongoose)|HTTPS/SSL server (Mongoose)]] for instructions.

=== mhttpd using an HTTPS/SSL proxy ===

THESE INSTRUCTIONS ARE WRONG, DO NOT DO THIS. See instead [https://daq00.triumf.ca/MidasWiki/index.php/Setup_MIDAS_experiment_at_TRIUMF#Install_https_proxy this section].

An [[#Setting up an HTTP proxy|HTTP proxy]] must be set up. This is the only way of securing older version of [[mhttpd]] (pre August 2015). Older versions of mhttpd are started using the -p port option e.g.
* mhttpd -D -p 8080 

To run a new version of mhttpd using an HTTP proxy, use the options provided to run the old (non-Mongoose) webserver on a given port, i.e.
* mhttpd --oldserver 8080 --nomg -D 

When using an SSL proxy, only access from the SSL proxy (and maybe some special trusted machines) should be permitted.
This is done using the "-a hostname" switch of [[mhttpd]]. Normally there will be only "-a localhost" switch, enabling access only for the local machine (where the SSL proxy is running). Additional "-a hostname" switches enable access from listed local machines. No "-a xxx" enables access from everywhere (defeating the purpose of the SSL proxy, unless access controls are enforced elsewhere, i.e. by a site firewall or by local firewall rules).

== Setting up an HTTP proxy ==
In this example, we use APACHE HTTPD to password-protect a typical midas/mhttpd and elog installation.

In this configuration, one uses the Linux stock httpd that accepts encrypted https:// connections and forwards them to mhttpd and elogd. Instead of (or in addition to) using mhttpd and elogd passwords, one configures password protection in httpd via the regular apache httpd password mechanisms (htpasswd, etc).

Recent versions of elogd do support SSL https:// connections, but if one is running an SSL proxy for anyway, it is simpler to run both through the same SSL proxy using the same SSL host certificate and the same httpd password file.

=== Restricting http: access to elogd ===
;Note
:Recent versions of elogd do support SSL https:// connections. The following information is for those using an HTTP proxy (see above).

For elogd, this is done using the "-n localhost" switch with enables only access from the same machine if present, or access from anywhere is absent (defeating the purpose of the SSL proxy, unless access controls are enforced elsewhere).

(It is recommended to run elogd from the same user as the main daq user and to keep elogd.cfg and all logbooks in the home directory of this user, where they are captured by the normal site backup system)

== Install standalone elog ==

* login into the user account that will run the elog
* cd $HOME/packages
* git clone https://bitbucket.org/ritt/elog
* cd elog
* make
* create new file start_elogd with this contents:
<pre>
#!/bin/sh

killall elogd
sleep 1
killall -KILL elogd
sleep 1
$HOME/packages/elog/elogd -n localhost -x -c $HOME/packages/elog/elogd.cfg -p 8082 -D

#end
</pre>
* chmod a+x start_elogd
* edit elogd.cfg to read:
<pre>
[global]
port = 8082
SMTP host = smtp.triumf.ca
URL = https://titan00.triumf.ca/elog/

Reverse sort = 1
Display Mode = full

#List Menu commands = New, Find, Admin, Help
#Menu commands = New, Edit, Reply, Find, Duplicate, Help

Entries Per Page = 30
Supress Email on edit = 1
Default encoding = 1
Page title = TITAN ELOG
Resolve host names = 1

Logfile = /home/titan/packages/elog/elogd.log
#Logging level = 3

[midas]

List page Title = T2K M11 MIDAS ELOG
Comment = T2K M11 MIDAS ELOG
Page Title = T2K M11 MIDAS ELOG
RSS Title = [$logbook - $type - $system] $subject, posted by $author

Attributes = Author, Subject, Run, Type, System
Show Attributes Edit = Run, Author, Subject, Type, System
Required Attributes = Author, Type, System, Subject

Options Type = Routine, Reply, Shift Summary, Modification, Question, Info, Problem
Options System = General, DAQ, Beamline

Preset Run = $shell(MIDASSYS=. /home/t2km11/packages/midas/linux/bin/odbedit -d Runinfo -c 'ls -v \"run number\"')

Preset On Reply Type = Reply
Preset On Reply Run = $shell(MIDASSYS=. /home/t2km11/packages/midas/linux/bin/odbedit -d Runinfo -c 'ls -v \"run number\"')

List Display = Date, Subject, Type, System, Author, ID
Quick Filter = Date, Type, ID

Remove on reply = Author
Quote on reply = 1

Use lock = 1

************* Email Functionality ****************

Use Email Subject = [T2KM11 - $System] $Subject
Omit Email To = 1

Email System General = xxx
</pre>
* ./start_elogd &
* firefox http://localhost:8082 # hould show the elog message index

To start elogd automatically when the machine is rebooted, login as root and
* add this text to /etc/rc.local:
<pre>
su - titan -c "/home/titan/packages/elog/start_elogd"
</pre>
* chmod a+x /etc/rc.local
* systemctl start rc-local

To import elog entries from the mhttpd elog, do this:

* cd ~/packages/elog/logbooks
* ln -s /home/t2km11/online/elog midas
* cd midas
* ~/packages/elog/elconv)

== Install https proxy ==

THESE INSTRUCTIONS ARE OBSOLETE, INSTEAD,
* GO HERE: https://daq00.triumf.ca/DaqWiki/index.php/Ubuntu#Install_apache_httpd_proxy_for_midas_and_elog
* AND GO HERE: https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux#Run_the_MIDAS_Web_Server

FOLLOWING INSTRUCTIONS ARE OBSOLETE...

* login as root to the https proxy machine
* cd ~root
* yum install mod_ssl
* yum install crypto-utils # see http://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_monitoring_of_HTTPS_certificates
* create a certificate request (replace ladd09 with your hostname): openssl req -new -nodes -newkey rsa:2048 -sha256 -out ladd09.csr -keyout ladd09.key (answer: CA, BC, Vancouver, TRIUMF, DAQ, ladd09.triumf.ca, email@email.com
* sign it by TRIUMF:
** mail -s "Certificate request" yourself@email.com < ladd09.csr
** forward this request to Andrew Daviel
** he will email the signed crt file, copy it to this system as ladd09.crt
* sign it yourself: openssl x509 -req -days 365 -sha256 -in ladd09.csr -signkey ladd09.key -out ladd09.crt
* (if the certificate expires, renew it by signing it again)
* Additional commands for working with certificates:
** explore the private key: openssl pkey -in ladd09.key -text -noout
** explore the certificate request: openssl req -in ladd00.csr -text -noout
** explore the certificate: openssl x509 -in ladd09.crt -noout -text
* move certificate files to proper system locations:
* mv ladd09.key /etc/pki/tls/private/
* mv ladd09.crt /etc/pki/tls/certs/
* if selinux is enabled, do this:
** restorecon -Rv /etc/pki/tls/certs/
** restorecon -Rv /etc/pki/tls/private/
** /usr/sbin/setsebool -P httpd_can_network_connect 1
* open /etc/httpd/conf.d/ssl.conf in a text editor, go to the very bottom and right before the "</VirtualHost>" entry, add following text:
<pre>
...
SSLCertificateFile /etc/pki/tls/certs/ladd09.crt
SSLCertificateKeyFile /etc/pki/tls/private/ladd09.key

ProxyPass /elog/ http://localhost:8082/ retry=1
ProxyPass / http://localhost:8080/ retry=1

<Location />

SSLRequireSSL
AuthType Basic
AuthName "password protected site"
Require valid-user

# create password file: touch /etc/httpd/htpasswd
# to add new user or change password: htpasswd /etc/httpd/htpasswd username
AuthUserFile /etc/httpd/htpasswd

</Location>
</VirtualHost>
...
</pre>
* comment out duplicate "SSLCertificateFile" and "SSLCertificateKeyFile" elsewhere in the file
* (optionally) If you got a certificate that is signed by DigiCert or RapidSSL then you'll need to add a line specifying the certificate chain file:
<pre>
...
SSLCertificateChainFile /etc/pki/tls/certs/DigiCertCA.crt
...
</pre>
* touch /etc/httpd/htpasswd
* htpasswd /etc/httpd/htpasswd midas # enter password midas
* chkconfig httpd on
* service httpd restart
* firewall-cmd --add-port=443/tcp --permanent
* firewall-cmd --reload
* firewall-cmd --list-all
* test it
** test the SSL proxy: https://host/ should yield the midas status page, https://host/elog/ should yield the elog message index
* in ODB, set "/Elog/URL" to "https://host/elog/"
* now from the midas status page, the "Elog" button should take us to the https Elog URL

In needed, enable user directories: https://blah/~user in ~user/public_html

* edit /etc/httpd/conf.d/userdir.conf, replace "UserDir disabled" with "UserDir enabled"
* setsebool -P httpd_enable_homedirs true
* systemctl restart httpd

== Setup the history mhttpd for faster access to history plots ==
When running an SSL proxy,
* start the main mhttpd (orange command for old mhttpd, green for new mhttpd with Mongoose(post August2015):
** "mhttpd -p 8071 -D" or
** "mhttpd -D --oldserver 8071 --nomg" 

* start the history mhttpd
** "mhttpd -p 8072 -D -H" or
** "mhttpd -D -H --oldserver 8072 --nomg" 
* set ODB /History/URL to "http://alphacpc09.cern.ch:8072/HS/"
* open the MIDAS status page
* go to the history section, try to open any history plot, observe that the history plot gif image loads correctly, inspect it's URL (use "copy image URL" or "view source", etc), it should point to port 8072 causing connection to the history mhttpd.
* continue with these instructions to setup history mhttpd access through an SSL proxy:
* setup SSL proxy access (required mod_proxy_html)
** login as root to the SSL proxy machine
** on SL5, install the missing mod_proxy_html httpd module:
** yum install httpd-devel libxml2-devel
** wget http://apache.webthing.com/mod_proxy_html/mod_proxy_html.tar.bz2
** tar xjvf mod_proxy_html.tar.bz2
** cd mod_proxy_html
** apxs -c -I. -I/usr/include/libxml2 -i mod_proxy_html.c
** apxs -c -I. -I/usr/include/libxml2 -i mod_xml2enc.c
** cd /etc/httpd/conf.d, add this to ssl.conf:

;before the ProxyPass statements:
<pre>
# proxy the MIDAS web servers
LoadModule xml2enc_module modules/mod_xml2enc.so
LoadModule proxy_html_module modules/mod_proxy_html.so
ProxyHTMLLinks a href
ProxyHTMLLinks link href
ProxyHTMLLinks img src
#ProxyHTMLEnable On
ProxyRequests off
</pre>
;after the ProxyPass statements:
<pre>
# ALPHA1 history access
ProxyPass /alpha1/history/ http://alphacpc09.cern.ch:8072/HS/ retry=1
ProxyPass /alpha1/ http://alphacpc09.cern.ch:8071/ retry=1

ProxyHTMLEnable On
ProxyHTMLURLMap http://alphacpc09.cern.ch:8072/HS/ /alpha1/history/
</pre>
;adjust:
*"alpha1" is the experiment name
*"alphacpc09.cern.ch" is the machine running mhttpd
*"8071" is the port number of the main mhttpd
** "mhttpd -p 8071 -D" or
** "mhttpd -D --oldserver 8071 --nomg" 
* "8072" is the port number of the history mhttpd
** "mhttpd -p 8072 -D -H" or
** "mhttpd -D -H --oldserver 8072 --nomg" 

[[Category:Installation]] [[Category:Buffer]] [[Category:Driver]]

Setup MIDAS experiment at TRIUMF

2026-02-03T17:30:46Z

Dfujimoto: /* Install https proxy */

{{Pagelinks}}
== Introduction ==
This page describes setting up a MIDAS experiment at TRIUMF. This information can be adapted for other sites.

== Environment variables ==
* '''MIDASSYS''' Base directory of the MIDAS package, midas and mxml should be at the same level.
* '''MIDAS_EXPTAB''' Experiment definition file
* '''MIDAS_SERVER_HOST''' MIDAS host server name for remote midas connections.
* '''MIDAS_EXPT_NAME''' Experiment name

== Standard layout of MIDAS experiment ==
The following shows the directory layout of a standard MIDAS experiment:

/home/exptuser/
packages/
root <---- ROOT
mxml
mscb
midas/ <---- MIDAS
linux/{lib,bin} <---- binaries matching the selected 64-bit/32-bit flavour of ROOT
linux-m32/{lib,bin} <---- limited function 32-bit binaries for 32-bit frontend machines, build by "make linux32"
linux-m64/{lib,bin} <---- limited function 64-bit binaries (only needed if ROOT and linux/bin are 32-bit)
linux-arm/{lib,bin} <---- full function ARM cross-compiled using "make linuxarm"
linux-crosscompile/{lib,bin} <---- cross-compiled limited function binaries for PPC and ARM frontends (see Makefile)
rootana <---- ROOT analyzer
roody <---- graphical online histogram viewer for MIDAS and ROOTANA
online/
exptab <---- experiment definition
{.ODB,.SYSTEM,.SYSMSG,etc}.SHM <---- MIDAS shared memory save files
src <---- experiment frontend sources
bin,scripts
elog <---- MIDAS elog
history <---- MIDAS history
data -> /data/exptname/current <---- symlink to the data directory
/data/exptname/current <---- experiment data directory with ODB save files and MIDAS .mid/.mid.gz data files

== Prepare computers ==

On some operating systems, several MIDAS functions require administrator access:

* on el7 linux (SL7/CC7/CentOS7/RHEL7) - access to mhttpd port 8443 requires special firewall rules, see here: https://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29
* on el7 linux - access to mserver to run frontends and other programs on some other computer requires special firewall rules, see here: https://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29
* on el7 linux - on the frontend machines (and other machines that will connect to the mserver, the same firewall rule needs to be created (use the IP address of the machine running the mserver)

== Prepare the user account ==
<div id="NOTES"></div>

* Setup the user account for running this instance of midas. For machines part of the LADD cluster, follow these [http://daq-plone.triumf.ca/SM/docs/local/NewLaddUser] instructions.
* check that the account is using the /bin/bash shell
* make $HOME/.profile look like this:

<pre>
#!/bin/echo You must source

export SVN_EDITOR="emacs -nw"
export GIT_EDITOR="emacs -nw"
export MIDASSYS=$HOME/packages/midas
export ROOTANASYS=$HOME/packages/rootana
export MIDAS_EXPTAB=$HOME/online/exptab
#
# setup the MIDAS mserver
#
case `hostname` in
daq07*)
unset MIDAS_SERVER_HOST
;;
*)
export MIDAS_SERVER_HOST=daq07.triumf.ca:7070
;;
esac
#
# select 64-bit or 32-bit MIDAS and ROOT
#
case `uname -i` in
i386)
source /daq/daqshare/olchansk/root/root_v5.34.01_SL62_32/bin/thisroot.sh
export PATH=.:$MIDASSYS/linux-m32/bin:$PATH
;;
*)
#source /daq/daqshare/olchansk/root/root_v5.34.34_SL67_64/bin/thisroot.sh
source $HOME/packages/root/bin/thisroot.sh
export PATH=.:$MIDASSYS/linux/bin:$PATH
;;
esac
#
export PATH=.:$HOME/online/bin:$HOME/packages/roody/bin:$PATH
#
#end
</pre>

* mkdir $HOME/packages
* Logout and login again, for .cshrc changes to take effect

== Install ROOT ==

* Identify the Linux version: RH9 (Red Hat Linux 9), FC3 (Fedora Core 3), RHEL4/SL4 (Red Hat Enterprise LInux 4/Scientific Linux 4), SL5x, SL6x, (CentOS/CC/SL) el7x: more /etc/redhat-release
* Decide to use 32-bit or 64-bit ROOT ('uname -a')
* cd $HOME/packages
* ls -l /daq/daqshare/olchansk/root/ ### to see all available ROOT packages
* ln -s /daq/daqshare/olchansk/root/root_vNNN_VVV_BB root, where NNN is the latest available version of ROOT ("ls -l /daq/daqshare/olchansk/root"), VVV is the Linux version code (RH9, FC3, SL4, etc) and BB is "_32" or "_64" for 32-bit or 64-bit ROOT. For example: /daq/daqshare/olchansk/root/root_v5.10.00_SL40
* for example: ln -s /daq/daqshare/olchansk/root/root_v5.34.34_el72_64 $HOME/packages/root
* Check that ROOT works: "source $HOME/packages/root/bin/thisroot.sh; root". Type ".q" to exit root.

== Install MIDAS ==

* cd $HOME/packages
* (OBSOLETE) svn co svn+ssh://svn@savannah.psi.ch/repos/meg/midas/trunk midas, password "svn". (password has to be entered twice)
* (OBSOLETE) svn co svn+ssh://svn@savannah.psi.ch/repos/meg/mxml/trunk mxml
* git clone https://bitbucket.org/tmidas/midas --recursive
* (OBSOLETE) git clone https://bitbucket.org/tmidas/mxml
* (OBSOLETE) git clone https://bitbucket.org/tmidas/mscb
* (IF BITBUCKET IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/mxml.git
* (IF BITBUCKET IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/mscb.git
* (IF BITBUCKET IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/midas.git
* cd midas
* make
* (only if needed) make linux32 ### build the 32-bit MIDAS libraries
* ls -l linux/bin/odbedit ### check that odbedit has been created (do not run it yet)

You can see a list of other installation problems at [[Common problems & Debugging recipes]].

;NOTE 1
: Optional features in MIDAS can be explicitly disabled if desired when making MIDAS using the NO_xxx feature (NO_ROOT,NO_MYSQL,NO_ODBC,NO_SQLITE,NO_MSCB), e.g. "make NO_ROOT=1" to disable ROOT. These NO_xxx Makefile variables are only used to control autodetection.

;NOTE 2
: Since June 2019 the mxml and mscb packages are submodules of the midas package, so no need to clone them separately. If you have an existing clone of midas but not yet the submodules, you need

$ git submodule update --init --recursive

: To update both midas and the submodules, you need

$ git pull --recurse-submodules

== Install ROOTANA ==

* cd $HOME/packages
* git clone https://bitbucket.org/tmidas/rootana
* cd rootana
* make

== Install ROODY ==

* cd $HOME/packages
* git clone https://bitbucket.org/tmidas/roody
* cd roody
* make
* $HOME/packages/roody/bin/roody, run the program

== Install additional additional packages ==

* cd $HOME/packages
* git clone https://bitbucket.org/ttriumfdaq/vme
* git clone https://bitbucket.org/ttriumfdaq/frontends

== Build special versions of MIDAS ==

Build special versions of MIDAS for the case when some MIDAS programs, such as VME frontends, will run on a different computer that may have a different flavour of operating system, i.e. 32-bit linux or an older version of Scientific Linux.

* login to the computer where the frontends will run and:
* if it is a 32-bit linux: cd $HOME/packages/midas; make linux32
* if it is a 64-bit linux: cd $HOME/packages/midas; make linux64

* login to the host computer to cross-compile ARM code:
* if it is an ARM linux: cd $HOME/packages/midas; make linuxarm # may need to install ARM cross compilers

== Prepare VME hardware ==

Hardware check list:
* VME crate
* VME processor (supported are V77xx, V7805, V7865)
* On all VME modules, set the VME address jumpers as described here: http://daq-plone.triumf.ca/SM/docs/local/vme_jumpers
* run vmescan to confirm correct VME addresses
** cd $HOME/packages
** svn checkout https://ladd00.triumf.ca/svn/daqsvn/trunk/vme
** cd vme
** make
** ./vmescan.exe (or _gef.exe, depending on the VME driver in use)

== Install Universe-II VME driver (V7648, V7750, V7805, V7851) ==

* login as root (ssh root@localhost)
* follow instructions: https://www.triumf.info/wiki/DAQwiki/index.php/VME-CPU#V7648.2C_V7750.2C_V7805.2C_V7851_:_Setup_vme_universe_VME_drivers
* cd ~/packages/vme; vmescan.exe

== Install Tsi-148 VME driver (V7865) ==

* login as root (ssh root@localhost)
* follow instructions: https://www.triumf.info/wiki/DAQwiki/index.php/VME-CPU#V7865_and_XVB-602_:_Setup_gefvme.2Ftsi148_VME_drivers
* cd ~/packages/vme; vmescan_gef.exe

== Setup the experiment environment ==

* Decide which computer will host MIDAS (where MIDAS shared memory buffers will reside).
: This computer will run the [[mserver]], [[mlogger]] and [[mhttpd]] applications. (It is usually the machine where the MIDAS,ROOT etc. packages have been downloaded). It will be referred to as the host machine (localhost).

The environment is slightly different depending on whether all programs run on the host machine, or whether some programs run on remote host(s) :
=== ALL programs run on localhost ===
:If all programs run on the host machine (localhost), it is not necessary to run [[mserver]]. [[Environment Variables#MIDAS_SERVER_HOST|MIDAS_SERVER_HOST]] will not be assigned (see example .cshrc [[#Prepare the user account|above]]).

=== Some programs run on REMOTE host(s) ===
: '''IMPORTANT:'''
# Since August 2015 '''you must explicitly allow access for clients running on remote machines'''. To do this, follow the '''[[Security#MIDAS programs on remote machines|instructions here]]'''.
# The example code .cshrc ([[#Prepare the user account|see above]]) should be present on both host and remote machine(s). This will ensure that [[Environment Variables#MIDAS_SERVER_HOST|MIDAS_SERVER_HOST]] will NOT be set for the host machine (localhost), but on a remote machine, MIDAS_SERVER_HOST will be set to the MIDAS host machine.
# The client [[mserver]] must be started on the MIDAS host machine. Note that multiple experiments can run on the same host machine by starting several instances of [[mserver]] (one for each experiment) running with different ports (and .cshrc would be edited so that MIDAS_SERVER_HOST is set to the appropriate port for the experiment).

 
On the host machine:
* mkdir $HOME/online
* cd $HOME/online
* create directories for local programs, sources, elog and history: mkdir bin src elog history
* create data directory: mkdir -p /ladd/data1/t2kvme5/data; ln -s /ladd/data1/t2kvme5/data $HOME/online
* create the exptab file "$HOME/online/exptab" following the example below. The first entry (exptname) is the name if the DAQ system (MIDAS experiment name), the second entry (/home/USER/online) is the location of MIDAS shared memory buffers (by convention, $HOME/online), the third entry (kopio03) is your username.
<pre>exptname /home/kopio03/online kopio03</pre>
* logout and login again for all changes to take effect

== Setup experiment startup scripts ==
* login to the experiment host computer
* echo $MIDAS_SERVER_HOST ### to check correct value - should be blank
* create $HOME/online/bin/start_daq.sh, replacing XXX with the hostname of the machine running the experiment (and changing the mserver and mhttpd ports, as needed).

#!/bin/sh
# start_daq.sh
cd $HOME/online
#
case `hostname` in XXX*)
echo "Good, we are on XXX!"
;;
*)
echo "The start_daq script should be executed on XXX"
exit 1
;;
esac
#
odbedit -c clean
# start [[mhttpd]] on default port. (Mongoose https version - see [[mhttpd]] for other options)
mhttpd -D -a localhost -a XXX.triumf.ca # optionally restrict access to specified hosts
#
# start [[mserver]] on default port (use argument -p to use a different port)
mserver -D # access must now be specifically allowed - see [[#Setup the experiment environment|above]]

# OR ([[#NOTES|older MIDAS versions]])
# mhttpd -p 8081 -D -a localhost -a XXX.triumf.ca # optionally restrict access to specified hosts
# mserver -p 7071 -D -a localhost -a lxdragon01.triumf.ca -a lxdragon02.triumf.ca -a XXX.triumf.ca # optionally restrict access to specified hosts

#
mlogger -D
#end file

== Run the MIDAS Web Server ==
Let's start the MIDAS webserver for the first time:

Start [[mhttpd]] on the ''experiment host'' (localhost) like this:
[mhostpc] mhttpd
You will get the following messages:
[mhttpd,INFO] ODB subtree /Runinfo corrected successfully
Mongoose web server will listen on ports "8080r,8443s"
[mhttpd,ERROR] [mhttpd.cxx:17892:mongoose,ERROR] cannot find SSL certificate file "/home/agdaq/online/ssl_cert.pem"
[mhttpd,ERROR] [mhttpd.cxx:17893:mongoose,ERROR] please create SSL certificate file: openssl req -new -nodes -newkey rsa:2048 -sha256 -out ssl_cert.csr -keyout ssl_cert.key; openssl x509 -req -days 365 -sha256 -in ssl_cert.csr -signkey ssl_cert.key -out ssl_cert.pem; cat ssl_cert.key >> ssl_cert.pem
could not start the mongoose web server, see messages and midas.log, bye!

Create a self-signed certificate suitable for initial testing by executing the command printed by mhttpd:


[mhostpc] openssl req -new -nodes -newkey rsa:2048 -sha256 -out ssl_cert.csr -keyout ssl_cert.key; openssl x509 -req -days 365 -sha256 -in ssl_cert.csr -signkey ssl_cert.key -out ssl_cert.pem; cat ssl_cert.key >> ssl_cert.pem

For production use, you should create a properly signed certificate, see [[Mhttpd#Create an SSL certificate|create your own SSL certificate]] or you should run mhttpd behind an SSL proxy.

Run mhttpd again.

You will get the following messages:
[mhttpd,INFO] ODB subtree /Runinfo corrected successfully
Mongoose web server will listen on ports "8080r,8443s"
Mongoose web server will use SSL certificate file "/home/johnfoo/packages/midas/ssl_cert.pem"
[mhttpd,ERROR] [mhttpd.cxx:17633:mongoose,ERROR] mongoose web server cannot find password file "/home/johnfoo/online/htpasswd.txt"
[mhttpd,ERROR] [mhttpd.cxx:17634:mongoose,ERROR] please create password file: htdigest -c /home/johnfoo/online/htpasswd.txt Default midas
could not start the mongoose web server, see messages and midas.log, bye!

Create the password file by following the instructions printed by mhttpd. The http digest domain name is the experiment name, suggested default user name is "midas". You will be asked to type in a password


[mhostpc] htdigest -c /home/johnfoo/online/htpasswd.txt exptname midas
Adding password for midas in realm exptname.
New password:
Re-type new password:

It is a good idea to set the password file {{Filepath|path=htpasswd.txt}} readable and writable by owner only.

Now restart {{Utility|name=mhttpd}}
[mhostpc] mhttpd
Mongoose web server will listen on ports "8080r,8443s" **see note
Mongoose web server will use SSL certificate file "/home/suz/packages/midas/ssl_cert.pem"
Mongoose web server will use authentication realm "Default", password file "./htpasswd.txt"

Now point a web browser running on the same host computer (localhost) to https://localhost:8443
If the web browser is running on a different computer, go to URL of the form

https://mhostpc.triumf.ca:8443 (substitute your host machine name and domain for "mhostpc.triumf.ca")

If you are using the default SSL certificate you will probably get a message: "This Connection is Untrusted". Click "I understand the risks" and add an exception. This is because the test certificate is self-signed. Then confirm an exception.

If instead you get a "connection refused" error, the midas host pc may have the firewall enabled. To make a firewall exception for MIDAS, follow instructions here http://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29

You should then see an authentication box asking you for the user name and password. The user name is "midas". Enter the password you just created. The Midas [[Status Page]] should appear with multiple buttons for run control as well as equipment listing (no equipments will be listed as yet) and application listings. Please refer to [[mhttpd]] (the MIDAS Web-based Run Control utility) for further information. You can start and stop runs from the main status page, and use the [[ODB Page]] to access the database (ODB).

; Note
: Default ports of 8080 and 8443 are used by [[mhttpd]]. If these ports are in use on your machine, start mhttpd with alternative ports, e.g.
[mhostpc] mhttpd --https 8448 --http 8089
: or see [[Mhttpd#Usage]] to change the default ports.

== Setup experiment database (ODB) ==

* run $HOME/online/bin/start_daq.sh

* odbedit, run these commands: (replace user names and directory names)
<pre>
set "/Logger/Message file" "/home/kopio03/online/midas.log"
set "/Logger/Data Dir" "/home/kopio03/online/data"
create STRING "/Logger/History dir"
set "/Logger/History dir" "/home/kopio03/online/history"
create STRING "/Logger/Elog dir"
set "/Logger/Elog dir" "/home/kopio03/online/elog"
set "/Logger/ODB dump file" "/home/kopio03/online/history/run%05d.xml"
set "/Logger/ODB dump" "y"
set "/Logger/Channels/0/Settings/Filename" "run%05dsub%03d.mid.gz"
set "/Logger/Channels/0/Settings/Subrun byte limit" "1000000000"
set "/Logger/Channels/0/Settings/Compression" 1
set "/Logger/Channels/0/Settings/ODB Dump" "y"
set "/Programs/Logger/Required" y
set "/Programs/Logger/Start command" "mlogger -D"
set "/Programs/fevme/Required" "y"
set "/Programs/fevme/Start command" "ssh -n lxdaq09 $HOME/online/src/fevme_gef.exe -O"
exit
</pre>
* open web browser e.g. firefox.
* go to the midas status page at https://localhost:8443 (default port).
** if running [[mhttpd]] with Mongoose HTTPS/OpenSSL (the default) for the first time, you will need to create a password file. Follow the instructions (see [[mhttpd#HTTPS/SSL server (Mongoose)]] for details).
** For other options (i.e. HTTPS/SSL proxy) see [[#Secure MIDAS and ELOG Web access]]
*OR open the midas status page at http://localhost:8081 ([[#NOTES|older MIDAS versions]])
* midas status page will show most stuff "red" as nothing is running yet
* DON'T DO THIS YET run ./fevme.exe (on the computer with the VME interface, could be different from computer hosting the experiment), observe that corresponding equipments have been created
* save the url bookmark to the "personal toolbar"
* go to the Programs page, stop mlogger, stop fevme, start mlogger, start fevme
* go to the Status page, start run, stop run
* go back to the Status page, everything should be green
* start a run
* send signals to the ADC gate
* you should be getting events
* to look at data, proceed with setting up the [[ROOTANA|ROOT Analyzer]].

== Start DAQ programs at boot time ==

* add this to /etc/rc.local (replace username and location of the start_daq script)
<pre>
su - alpha -c /home/alpha/online/bin/start_daq.sh
</pre>

== Setup local software version control ==

Version control for experiment source code is setup using "git" (http://git-scm.com/)

* cd $HOME/online
* git init
* git add exptab
* git add bin/start_daq.sh
* git add .gitignore ### contents can be
<pre>
*~
*.o
*.exe
</pre>
* git add src/Makefile src/*.cxx ...
* git commit -a

== Adjust MIDAS buffer sizes ==

Default MIDAS SYSTEM buffer size is 8 Mbytes, fairly small for high-data-rate experiments. The rule of thumb is to have at least a few seconds worth of buffer space available. For example, if event size is 10 Kbytes and the event rate is 1 kHz, data rate is 10*10^3*1*10^3 = 10 Mbytes/sec. To buffer 10 seconds of data we need 100 Mbytes of buffer space.

To resize the MIDAS event buffers (SYSTEM, etc) do this:
* stop all frontends, stop mlogger
* start odbedit:
** cd "/Experiment/Buffer sizes"
** set SYSTEM 100000000
* run "mdump -z SYSTEM"
* if mdump complains about the size of .SYSTEM.SHM, remove it, try again.
* ls -l /dev/shm ### to observe that the size of shared memory is correct

== Secure MIDAS and ELOG Web access ==
In versions prior to May 2015, the default web access to MIDAS and ELOG uses the "http:" protocol which is insecure. In this case, all information is transmitted as clear text meaning that secret, confidential and sensitive information (such as the MIDAS and ELOG passwords and usernames) can be stolen "easily". This means that even "password protected" MIDAS and ELOG pages are not really protected if accessed using the "http" method.

Better security for HTTP is gained by using a password protected '''SSL (https) proxy'''. (It does not provide absolute security because of remaining problems with the security of SSL certificates, security of passwords, etc). Setting up an SSL (https) proxy is described [[#Setting up an HTTP proxy|below]].

Since May 2015, an ''alternative secure option'' to setting up an HTTP proxy is available to users of MIDAS. Recent versions of elogd (ELOG) do support SSL https:// connections, and [[#mhttpd with HTTPS/SSL server (Mongoose)]] is now available. This option is the default, and provides a similar level of security to an HTTP proxy.

See [[Security#Web Access]] for a comparison of these two secure options.

=== mhttpd with HTTPS/SSL server (Mongoose) ===

Since May 2015 the MIDAS web server [[mhttpd]] is explicitly linked with OpenSSL to provide secure HTTPS connections via the [https://bitbucket.org/tmidas/midas/src/ecb9a8537448a8a43f7f9a2bfdb82e578208cde3/doc/mongoose/?at=develop Mongoose] web server (see [[mhttpd]]). With this version, default web access to MIDAS uses the "https" protocol. Web access to {{Utility|name=mhttpd}} can be restricted by using the "-a hostname" switch of [[mhttpd]]. The first time {{Utility|name=mhttpd}} is run, a password file must be created. An SSL certificate is also required. See [[mhttpd#HTTPS/SSL server (Mongoose)|HTTPS/SSL server (Mongoose)]] for instructions.

=== mhttpd using an HTTPS/SSL proxy ===

THESE INSTRUCTIONS ARE WRONG, DO NOT DO THIS. See instead the [https://daq00.triumf.ca/DaqWiki/index.php/Ubuntu#Install_apache_httpd_proxy_for_midas_and_elog TRIUMF DAQ site].

An [[#Setting up an HTTP proxy|HTTP proxy]] must be set up. This is the only way of securing older version of [[mhttpd]] (pre August 2015). Older versions of mhttpd are started using the -p port option e.g.
* mhttpd -D -p 8080 

To run a new version of mhttpd using an HTTP proxy, use the options provided to run the old (non-Mongoose) webserver on a given port, i.e.
* mhttpd --oldserver 8080 --nomg -D 

When using an SSL proxy, only access from the SSL proxy (and maybe some special trusted machines) should be permitted.
This is done using the "-a hostname" switch of [[mhttpd]]. Normally there will be only "-a localhost" switch, enabling access only for the local machine (where the SSL proxy is running). Additional "-a hostname" switches enable access from listed local machines. No "-a xxx" enables access from everywhere (defeating the purpose of the SSL proxy, unless access controls are enforced elsewhere, i.e. by a site firewall or by local firewall rules).

== Setting up an HTTP proxy ==
In this example, we use APACHE HTTPD to password-protect a typical midas/mhttpd and elog installation.

In this configuration, one uses the Linux stock httpd that accepts encrypted https:// connections and forwards them to mhttpd and elogd. Instead of (or in addition to) using mhttpd and elogd passwords, one configures password protection in httpd via the regular apache httpd password mechanisms (htpasswd, etc).

Recent versions of elogd do support SSL https:// connections, but if one is running an SSL proxy for anyway, it is simpler to run both through the same SSL proxy using the same SSL host certificate and the same httpd password file.

=== Restricting http: access to elogd ===
;Note
:Recent versions of elogd do support SSL https:// connections. The following information is for those using an HTTP proxy (see above).

For elogd, this is done using the "-n localhost" switch with enables only access from the same machine if present, or access from anywhere is absent (defeating the purpose of the SSL proxy, unless access controls are enforced elsewhere).

(It is recommended to run elogd from the same user as the main daq user and to keep elogd.cfg and all logbooks in the home directory of this user, where they are captured by the normal site backup system)

== Install standalone elog ==

* login into the user account that will run the elog
* cd $HOME/packages
* git clone https://bitbucket.org/ritt/elog
* cd elog
* make
* create new file start_elogd with this contents:
<pre>
#!/bin/sh

killall elogd
sleep 1
killall -KILL elogd
sleep 1
$HOME/packages/elog/elogd -n localhost -x -c $HOME/packages/elog/elogd.cfg -p 8082 -D

#end
</pre>
* chmod a+x start_elogd
* edit elogd.cfg to read:
<pre>
[global]
port = 8082
SMTP host = smtp.triumf.ca
URL = https://titan00.triumf.ca/elog/

Reverse sort = 1
Display Mode = full

#List Menu commands = New, Find, Admin, Help
#Menu commands = New, Edit, Reply, Find, Duplicate, Help

Entries Per Page = 30
Supress Email on edit = 1
Default encoding = 1
Page title = TITAN ELOG
Resolve host names = 1

Logfile = /home/titan/packages/elog/elogd.log
#Logging level = 3

[midas]

List page Title = T2K M11 MIDAS ELOG
Comment = T2K M11 MIDAS ELOG
Page Title = T2K M11 MIDAS ELOG
RSS Title = [$logbook - $type - $system] $subject, posted by $author

Attributes = Author, Subject, Run, Type, System
Show Attributes Edit = Run, Author, Subject, Type, System
Required Attributes = Author, Type, System, Subject

Options Type = Routine, Reply, Shift Summary, Modification, Question, Info, Problem
Options System = General, DAQ, Beamline

Preset Run = $shell(MIDASSYS=. /home/t2km11/packages/midas/linux/bin/odbedit -d Runinfo -c 'ls -v \"run number\"')

Preset On Reply Type = Reply
Preset On Reply Run = $shell(MIDASSYS=. /home/t2km11/packages/midas/linux/bin/odbedit -d Runinfo -c 'ls -v \"run number\"')

List Display = Date, Subject, Type, System, Author, ID
Quick Filter = Date, Type, ID

Remove on reply = Author
Quote on reply = 1

Use lock = 1

************* Email Functionality ****************

Use Email Subject = [T2KM11 - $System] $Subject
Omit Email To = 1

Email System General = xxx
</pre>
* ./start_elogd &
* firefox http://localhost:8082 # hould show the elog message index

To start elogd automatically when the machine is rebooted, login as root and
* add this text to /etc/rc.local:
<pre>
su - titan -c "/home/titan/packages/elog/start_elogd"
</pre>
* chmod a+x /etc/rc.local
* systemctl start rc-local

To import elog entries from the mhttpd elog, do this:

* cd ~/packages/elog/logbooks
* ln -s /home/t2km11/online/elog midas
* cd midas
* ~/packages/elog/elconv)

== Install https proxy ==

THESE INSTRUCTIONS ARE OBSOLETE, INSTEAD,
* GO HERE: https://daq00.triumf.ca/DaqWiki/index.php/Ubuntu#Install_apache_httpd_proxy_for_midas_and_elog
* AND GO HERE: https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux#Run_the_MIDAS_Web_Server

FOLLOWING INSTRUCTIONS ARE OBSOLETE...

* login as root to the https proxy machine
* cd ~root
* yum install mod_ssl
* yum install crypto-utils # see http://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_monitoring_of_HTTPS_certificates
* create a certificate request (replace ladd09 with your hostname): openssl req -new -nodes -newkey rsa:2048 -sha256 -out ladd09.csr -keyout ladd09.key (answer: CA, BC, Vancouver, TRIUMF, DAQ, ladd09.triumf.ca, email@email.com
* sign it by TRIUMF:
** mail -s "Certificate request" yourself@email.com < ladd09.csr
** forward this request to Andrew Daviel
** he will email the signed crt file, copy it to this system as ladd09.crt
* sign it yourself: openssl x509 -req -days 365 -sha256 -in ladd09.csr -signkey ladd09.key -out ladd09.crt
* (if the certificate expires, renew it by signing it again)
* Additional commands for working with certificates:
** explore the private key: openssl pkey -in ladd09.key -text -noout
** explore the certificate request: openssl req -in ladd00.csr -text -noout
** explore the certificate: openssl x509 -in ladd09.crt -noout -text
* move certificate files to proper system locations:
* mv ladd09.key /etc/pki/tls/private/
* mv ladd09.crt /etc/pki/tls/certs/
* if selinux is enabled, do this:
** restorecon -Rv /etc/pki/tls/certs/
** restorecon -Rv /etc/pki/tls/private/
** /usr/sbin/setsebool -P httpd_can_network_connect 1
* open /etc/httpd/conf.d/ssl.conf in a text editor, go to the very bottom and right before the "</VirtualHost>" entry, add following text:
<pre>
...
SSLCertificateFile /etc/pki/tls/certs/ladd09.crt
SSLCertificateKeyFile /etc/pki/tls/private/ladd09.key

ProxyPass /elog/ http://localhost:8082/ retry=1
ProxyPass / http://localhost:8080/ retry=1

<Location />

SSLRequireSSL
AuthType Basic
AuthName "password protected site"
Require valid-user

# create password file: touch /etc/httpd/htpasswd
# to add new user or change password: htpasswd /etc/httpd/htpasswd username
AuthUserFile /etc/httpd/htpasswd

</Location>
</VirtualHost>
...
</pre>
* comment out duplicate "SSLCertificateFile" and "SSLCertificateKeyFile" elsewhere in the file
* (optionally) If you got a certificate that is signed by DigiCert or RapidSSL then you'll need to add a line specifying the certificate chain file:
<pre>
...
SSLCertificateChainFile /etc/pki/tls/certs/DigiCertCA.crt
...
</pre>
* touch /etc/httpd/htpasswd
* htpasswd /etc/httpd/htpasswd midas # enter password midas
* chkconfig httpd on
* service httpd restart
* firewall-cmd --add-port=443/tcp --permanent
* firewall-cmd --reload
* firewall-cmd --list-all
* test it
** test the SSL proxy: https://host/ should yield the midas status page, https://host/elog/ should yield the elog message index
* in ODB, set "/Elog/URL" to "https://host/elog/"
* now from the midas status page, the "Elog" button should take us to the https Elog URL

In needed, enable user directories: https://blah/~user in ~user/public_html

* edit /etc/httpd/conf.d/userdir.conf, replace "UserDir disabled" with "UserDir enabled"
* setsebool -P httpd_enable_homedirs true
* systemctl restart httpd

== Setup the history mhttpd for faster access to history plots ==
When running an SSL proxy,
* start the main mhttpd (orange command for old mhttpd, green for new mhttpd with Mongoose(post August2015):
** "mhttpd -p 8071 -D" or
** "mhttpd -D --oldserver 8071 --nomg" 

* start the history mhttpd
** "mhttpd -p 8072 -D -H" or
** "mhttpd -D -H --oldserver 8072 --nomg" 
* set ODB /History/URL to "http://alphacpc09.cern.ch:8072/HS/"
* open the MIDAS status page
* go to the history section, try to open any history plot, observe that the history plot gif image loads correctly, inspect it's URL (use "copy image URL" or "view source", etc), it should point to port 8072 causing connection to the history mhttpd.
* continue with these instructions to setup history mhttpd access through an SSL proxy:
* setup SSL proxy access (required mod_proxy_html)
** login as root to the SSL proxy machine
** on SL5, install the missing mod_proxy_html httpd module:
** yum install httpd-devel libxml2-devel
** wget http://apache.webthing.com/mod_proxy_html/mod_proxy_html.tar.bz2
** tar xjvf mod_proxy_html.tar.bz2
** cd mod_proxy_html
** apxs -c -I. -I/usr/include/libxml2 -i mod_proxy_html.c
** apxs -c -I. -I/usr/include/libxml2 -i mod_xml2enc.c
** cd /etc/httpd/conf.d, add this to ssl.conf:

;before the ProxyPass statements:
<pre>
# proxy the MIDAS web servers
LoadModule xml2enc_module modules/mod_xml2enc.so
LoadModule proxy_html_module modules/mod_proxy_html.so
ProxyHTMLLinks a href
ProxyHTMLLinks link href
ProxyHTMLLinks img src
#ProxyHTMLEnable On
ProxyRequests off
</pre>
;after the ProxyPass statements:
<pre>
# ALPHA1 history access
ProxyPass /alpha1/history/ http://alphacpc09.cern.ch:8072/HS/ retry=1
ProxyPass /alpha1/ http://alphacpc09.cern.ch:8071/ retry=1

ProxyHTMLEnable On
ProxyHTMLURLMap http://alphacpc09.cern.ch:8072/HS/ /alpha1/history/
</pre>
;adjust:
*"alpha1" is the experiment name
*"alphacpc09.cern.ch" is the machine running mhttpd
*"8071" is the port number of the main mhttpd
** "mhttpd -p 8071 -D" or
** "mhttpd -D --oldserver 8071 --nomg" 
* "8072" is the port number of the history mhttpd
** "mhttpd -p 8072 -D -H" or
** "mhttpd -D -H --oldserver 8072 --nomg" 

[[Category:Installation]] [[Category:Buffer]] [[Category:Driver]]

Setup MIDAS experiment at TRIUMF

2026-02-03T17:30:09Z

Dfujimoto: /* Secure MIDAS and ELOG Web access */

{{Pagelinks}}
== Introduction ==
This page describes setting up a MIDAS experiment at TRIUMF. This information can be adapted for other sites.

== Environment variables ==
* '''MIDASSYS''' Base directory of the MIDAS package, midas and mxml should be at the same level.
* '''MIDAS_EXPTAB''' Experiment definition file
* '''MIDAS_SERVER_HOST''' MIDAS host server name for remote midas connections.
* '''MIDAS_EXPT_NAME''' Experiment name

== Standard layout of MIDAS experiment ==
The following shows the directory layout of a standard MIDAS experiment:

/home/exptuser/
packages/
root <---- ROOT
mxml
mscb
midas/ <---- MIDAS
linux/{lib,bin} <---- binaries matching the selected 64-bit/32-bit flavour of ROOT
linux-m32/{lib,bin} <---- limited function 32-bit binaries for 32-bit frontend machines, build by "make linux32"
linux-m64/{lib,bin} <---- limited function 64-bit binaries (only needed if ROOT and linux/bin are 32-bit)
linux-arm/{lib,bin} <---- full function ARM cross-compiled using "make linuxarm"
linux-crosscompile/{lib,bin} <---- cross-compiled limited function binaries for PPC and ARM frontends (see Makefile)
rootana <---- ROOT analyzer
roody <---- graphical online histogram viewer for MIDAS and ROOTANA
online/
exptab <---- experiment definition
{.ODB,.SYSTEM,.SYSMSG,etc}.SHM <---- MIDAS shared memory save files
src <---- experiment frontend sources
bin,scripts
elog <---- MIDAS elog
history <---- MIDAS history
data -> /data/exptname/current <---- symlink to the data directory
/data/exptname/current <---- experiment data directory with ODB save files and MIDAS .mid/.mid.gz data files

== Prepare computers ==

On some operating systems, several MIDAS functions require administrator access:

* on el7 linux (SL7/CC7/CentOS7/RHEL7) - access to mhttpd port 8443 requires special firewall rules, see here: https://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29
* on el7 linux - access to mserver to run frontends and other programs on some other computer requires special firewall rules, see here: https://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29
* on el7 linux - on the frontend machines (and other machines that will connect to the mserver, the same firewall rule needs to be created (use the IP address of the machine running the mserver)

== Prepare the user account ==
<div id="NOTES"></div>

* Setup the user account for running this instance of midas. For machines part of the LADD cluster, follow these [http://daq-plone.triumf.ca/SM/docs/local/NewLaddUser] instructions.
* check that the account is using the /bin/bash shell
* make $HOME/.profile look like this:

<pre>
#!/bin/echo You must source

export SVN_EDITOR="emacs -nw"
export GIT_EDITOR="emacs -nw"
export MIDASSYS=$HOME/packages/midas
export ROOTANASYS=$HOME/packages/rootana
export MIDAS_EXPTAB=$HOME/online/exptab
#
# setup the MIDAS mserver
#
case `hostname` in
daq07*)
unset MIDAS_SERVER_HOST
;;
*)
export MIDAS_SERVER_HOST=daq07.triumf.ca:7070
;;
esac
#
# select 64-bit or 32-bit MIDAS and ROOT
#
case `uname -i` in
i386)
source /daq/daqshare/olchansk/root/root_v5.34.01_SL62_32/bin/thisroot.sh
export PATH=.:$MIDASSYS/linux-m32/bin:$PATH
;;
*)
#source /daq/daqshare/olchansk/root/root_v5.34.34_SL67_64/bin/thisroot.sh
source $HOME/packages/root/bin/thisroot.sh
export PATH=.:$MIDASSYS/linux/bin:$PATH
;;
esac
#
export PATH=.:$HOME/online/bin:$HOME/packages/roody/bin:$PATH
#
#end
</pre>

* mkdir $HOME/packages
* Logout and login again, for .cshrc changes to take effect

== Install ROOT ==

* Identify the Linux version: RH9 (Red Hat Linux 9), FC3 (Fedora Core 3), RHEL4/SL4 (Red Hat Enterprise LInux 4/Scientific Linux 4), SL5x, SL6x, (CentOS/CC/SL) el7x: more /etc/redhat-release
* Decide to use 32-bit or 64-bit ROOT ('uname -a')
* cd $HOME/packages
* ls -l /daq/daqshare/olchansk/root/ ### to see all available ROOT packages
* ln -s /daq/daqshare/olchansk/root/root_vNNN_VVV_BB root, where NNN is the latest available version of ROOT ("ls -l /daq/daqshare/olchansk/root"), VVV is the Linux version code (RH9, FC3, SL4, etc) and BB is "_32" or "_64" for 32-bit or 64-bit ROOT. For example: /daq/daqshare/olchansk/root/root_v5.10.00_SL40
* for example: ln -s /daq/daqshare/olchansk/root/root_v5.34.34_el72_64 $HOME/packages/root
* Check that ROOT works: "source $HOME/packages/root/bin/thisroot.sh; root". Type ".q" to exit root.

== Install MIDAS ==

* cd $HOME/packages
* (OBSOLETE) svn co svn+ssh://svn@savannah.psi.ch/repos/meg/midas/trunk midas, password "svn". (password has to be entered twice)
* (OBSOLETE) svn co svn+ssh://svn@savannah.psi.ch/repos/meg/mxml/trunk mxml
* git clone https://bitbucket.org/tmidas/midas --recursive
* (OBSOLETE) git clone https://bitbucket.org/tmidas/mxml
* (OBSOLETE) git clone https://bitbucket.org/tmidas/mscb
* (IF BITBUCKET IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/mxml.git
* (IF BITBUCKET IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/mscb.git
* (IF BITBUCKET IS DOWN) git clone -v --progress https://daq.triumf.ca/~daqweb/git/midas.git
* cd midas
* make
* (only if needed) make linux32 ### build the 32-bit MIDAS libraries
* ls -l linux/bin/odbedit ### check that odbedit has been created (do not run it yet)

You can see a list of other installation problems at [[Common problems & Debugging recipes]].

;NOTE 1
: Optional features in MIDAS can be explicitly disabled if desired when making MIDAS using the NO_xxx feature (NO_ROOT,NO_MYSQL,NO_ODBC,NO_SQLITE,NO_MSCB), e.g. "make NO_ROOT=1" to disable ROOT. These NO_xxx Makefile variables are only used to control autodetection.

;NOTE 2
: Since June 2019 the mxml and mscb packages are submodules of the midas package, so no need to clone them separately. If you have an existing clone of midas but not yet the submodules, you need

$ git submodule update --init --recursive

: To update both midas and the submodules, you need

$ git pull --recurse-submodules

== Install ROOTANA ==

* cd $HOME/packages
* git clone https://bitbucket.org/tmidas/rootana
* cd rootana
* make

== Install ROODY ==

* cd $HOME/packages
* git clone https://bitbucket.org/tmidas/roody
* cd roody
* make
* $HOME/packages/roody/bin/roody, run the program

== Install additional additional packages ==

* cd $HOME/packages
* git clone https://bitbucket.org/ttriumfdaq/vme
* git clone https://bitbucket.org/ttriumfdaq/frontends

== Build special versions of MIDAS ==

Build special versions of MIDAS for the case when some MIDAS programs, such as VME frontends, will run on a different computer that may have a different flavour of operating system, i.e. 32-bit linux or an older version of Scientific Linux.

* login to the computer where the frontends will run and:
* if it is a 32-bit linux: cd $HOME/packages/midas; make linux32
* if it is a 64-bit linux: cd $HOME/packages/midas; make linux64

* login to the host computer to cross-compile ARM code:
* if it is an ARM linux: cd $HOME/packages/midas; make linuxarm # may need to install ARM cross compilers

== Prepare VME hardware ==

Hardware check list:
* VME crate
* VME processor (supported are V77xx, V7805, V7865)
* On all VME modules, set the VME address jumpers as described here: http://daq-plone.triumf.ca/SM/docs/local/vme_jumpers
* run vmescan to confirm correct VME addresses
** cd $HOME/packages
** svn checkout https://ladd00.triumf.ca/svn/daqsvn/trunk/vme
** cd vme
** make
** ./vmescan.exe (or _gef.exe, depending on the VME driver in use)

== Install Universe-II VME driver (V7648, V7750, V7805, V7851) ==

* login as root (ssh root@localhost)
* follow instructions: https://www.triumf.info/wiki/DAQwiki/index.php/VME-CPU#V7648.2C_V7750.2C_V7805.2C_V7851_:_Setup_vme_universe_VME_drivers
* cd ~/packages/vme; vmescan.exe

== Install Tsi-148 VME driver (V7865) ==

* login as root (ssh root@localhost)
* follow instructions: https://www.triumf.info/wiki/DAQwiki/index.php/VME-CPU#V7865_and_XVB-602_:_Setup_gefvme.2Ftsi148_VME_drivers
* cd ~/packages/vme; vmescan_gef.exe

== Setup the experiment environment ==

* Decide which computer will host MIDAS (where MIDAS shared memory buffers will reside).
: This computer will run the [[mserver]], [[mlogger]] and [[mhttpd]] applications. (It is usually the machine where the MIDAS,ROOT etc. packages have been downloaded). It will be referred to as the host machine (localhost).

The environment is slightly different depending on whether all programs run on the host machine, or whether some programs run on remote host(s) :
=== ALL programs run on localhost ===
:If all programs run on the host machine (localhost), it is not necessary to run [[mserver]]. [[Environment Variables#MIDAS_SERVER_HOST|MIDAS_SERVER_HOST]] will not be assigned (see example .cshrc [[#Prepare the user account|above]]).

=== Some programs run on REMOTE host(s) ===
: '''IMPORTANT:'''
# Since August 2015 '''you must explicitly allow access for clients running on remote machines'''. To do this, follow the '''[[Security#MIDAS programs on remote machines|instructions here]]'''.
# The example code .cshrc ([[#Prepare the user account|see above]]) should be present on both host and remote machine(s). This will ensure that [[Environment Variables#MIDAS_SERVER_HOST|MIDAS_SERVER_HOST]] will NOT be set for the host machine (localhost), but on a remote machine, MIDAS_SERVER_HOST will be set to the MIDAS host machine.
# The client [[mserver]] must be started on the MIDAS host machine. Note that multiple experiments can run on the same host machine by starting several instances of [[mserver]] (one for each experiment) running with different ports (and .cshrc would be edited so that MIDAS_SERVER_HOST is set to the appropriate port for the experiment).

 
On the host machine:
* mkdir $HOME/online
* cd $HOME/online
* create directories for local programs, sources, elog and history: mkdir bin src elog history
* create data directory: mkdir -p /ladd/data1/t2kvme5/data; ln -s /ladd/data1/t2kvme5/data $HOME/online
* create the exptab file "$HOME/online/exptab" following the example below. The first entry (exptname) is the name if the DAQ system (MIDAS experiment name), the second entry (/home/USER/online) is the location of MIDAS shared memory buffers (by convention, $HOME/online), the third entry (kopio03) is your username.
<pre>exptname /home/kopio03/online kopio03</pre>
* logout and login again for all changes to take effect

== Setup experiment startup scripts ==
* login to the experiment host computer
* echo $MIDAS_SERVER_HOST ### to check correct value - should be blank
* create $HOME/online/bin/start_daq.sh, replacing XXX with the hostname of the machine running the experiment (and changing the mserver and mhttpd ports, as needed).

#!/bin/sh
# start_daq.sh
cd $HOME/online
#
case `hostname` in XXX*)
echo "Good, we are on XXX!"
;;
*)
echo "The start_daq script should be executed on XXX"
exit 1
;;
esac
#
odbedit -c clean
# start [[mhttpd]] on default port. (Mongoose https version - see [[mhttpd]] for other options)
mhttpd -D -a localhost -a XXX.triumf.ca # optionally restrict access to specified hosts
#
# start [[mserver]] on default port (use argument -p to use a different port)
mserver -D # access must now be specifically allowed - see [[#Setup the experiment environment|above]]

# OR ([[#NOTES|older MIDAS versions]])
# mhttpd -p 8081 -D -a localhost -a XXX.triumf.ca # optionally restrict access to specified hosts
# mserver -p 7071 -D -a localhost -a lxdragon01.triumf.ca -a lxdragon02.triumf.ca -a XXX.triumf.ca # optionally restrict access to specified hosts

#
mlogger -D
#end file

== Run the MIDAS Web Server ==
Let's start the MIDAS webserver for the first time:

Start [[mhttpd]] on the ''experiment host'' (localhost) like this:
[mhostpc] mhttpd
You will get the following messages:
[mhttpd,INFO] ODB subtree /Runinfo corrected successfully
Mongoose web server will listen on ports "8080r,8443s"
[mhttpd,ERROR] [mhttpd.cxx:17892:mongoose,ERROR] cannot find SSL certificate file "/home/agdaq/online/ssl_cert.pem"
[mhttpd,ERROR] [mhttpd.cxx:17893:mongoose,ERROR] please create SSL certificate file: openssl req -new -nodes -newkey rsa:2048 -sha256 -out ssl_cert.csr -keyout ssl_cert.key; openssl x509 -req -days 365 -sha256 -in ssl_cert.csr -signkey ssl_cert.key -out ssl_cert.pem; cat ssl_cert.key >> ssl_cert.pem
could not start the mongoose web server, see messages and midas.log, bye!

Create a self-signed certificate suitable for initial testing by executing the command printed by mhttpd:


[mhostpc] openssl req -new -nodes -newkey rsa:2048 -sha256 -out ssl_cert.csr -keyout ssl_cert.key; openssl x509 -req -days 365 -sha256 -in ssl_cert.csr -signkey ssl_cert.key -out ssl_cert.pem; cat ssl_cert.key >> ssl_cert.pem

For production use, you should create a properly signed certificate, see [[Mhttpd#Create an SSL certificate|create your own SSL certificate]] or you should run mhttpd behind an SSL proxy.

Run mhttpd again.

You will get the following messages:
[mhttpd,INFO] ODB subtree /Runinfo corrected successfully
Mongoose web server will listen on ports "8080r,8443s"
Mongoose web server will use SSL certificate file "/home/johnfoo/packages/midas/ssl_cert.pem"
[mhttpd,ERROR] [mhttpd.cxx:17633:mongoose,ERROR] mongoose web server cannot find password file "/home/johnfoo/online/htpasswd.txt"
[mhttpd,ERROR] [mhttpd.cxx:17634:mongoose,ERROR] please create password file: htdigest -c /home/johnfoo/online/htpasswd.txt Default midas
could not start the mongoose web server, see messages and midas.log, bye!

Create the password file by following the instructions printed by mhttpd. The http digest domain name is the experiment name, suggested default user name is "midas". You will be asked to type in a password


[mhostpc] htdigest -c /home/johnfoo/online/htpasswd.txt exptname midas
Adding password for midas in realm exptname.
New password:
Re-type new password:

It is a good idea to set the password file {{Filepath|path=htpasswd.txt}} readable and writable by owner only.

Now restart {{Utility|name=mhttpd}}
[mhostpc] mhttpd
Mongoose web server will listen on ports "8080r,8443s" **see note
Mongoose web server will use SSL certificate file "/home/suz/packages/midas/ssl_cert.pem"
Mongoose web server will use authentication realm "Default", password file "./htpasswd.txt"

Now point a web browser running on the same host computer (localhost) to https://localhost:8443
If the web browser is running on a different computer, go to URL of the form

https://mhostpc.triumf.ca:8443 (substitute your host machine name and domain for "mhostpc.triumf.ca")

If you are using the default SSL certificate you will probably get a message: "This Connection is Untrusted". Click "I understand the risks" and add an exception. This is because the test certificate is self-signed. Then confirm an exception.

If instead you get a "connection refused" error, the midas host pc may have the firewall enabled. To make a firewall exception for MIDAS, follow instructions here http://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_firewall_for_MIDAS_.28CentOS7.29

You should then see an authentication box asking you for the user name and password. The user name is "midas". Enter the password you just created. The Midas [[Status Page]] should appear with multiple buttons for run control as well as equipment listing (no equipments will be listed as yet) and application listings. Please refer to [[mhttpd]] (the MIDAS Web-based Run Control utility) for further information. You can start and stop runs from the main status page, and use the [[ODB Page]] to access the database (ODB).

; Note
: Default ports of 8080 and 8443 are used by [[mhttpd]]. If these ports are in use on your machine, start mhttpd with alternative ports, e.g.
[mhostpc] mhttpd --https 8448 --http 8089
: or see [[Mhttpd#Usage]] to change the default ports.

== Setup experiment database (ODB) ==

* run $HOME/online/bin/start_daq.sh

* odbedit, run these commands: (replace user names and directory names)
<pre>
set "/Logger/Message file" "/home/kopio03/online/midas.log"
set "/Logger/Data Dir" "/home/kopio03/online/data"
create STRING "/Logger/History dir"
set "/Logger/History dir" "/home/kopio03/online/history"
create STRING "/Logger/Elog dir"
set "/Logger/Elog dir" "/home/kopio03/online/elog"
set "/Logger/ODB dump file" "/home/kopio03/online/history/run%05d.xml"
set "/Logger/ODB dump" "y"
set "/Logger/Channels/0/Settings/Filename" "run%05dsub%03d.mid.gz"
set "/Logger/Channels/0/Settings/Subrun byte limit" "1000000000"
set "/Logger/Channels/0/Settings/Compression" 1
set "/Logger/Channels/0/Settings/ODB Dump" "y"
set "/Programs/Logger/Required" y
set "/Programs/Logger/Start command" "mlogger -D"
set "/Programs/fevme/Required" "y"
set "/Programs/fevme/Start command" "ssh -n lxdaq09 $HOME/online/src/fevme_gef.exe -O"
exit
</pre>
* open web browser e.g. firefox.
* go to the midas status page at https://localhost:8443 (default port).
** if running [[mhttpd]] with Mongoose HTTPS/OpenSSL (the default) for the first time, you will need to create a password file. Follow the instructions (see [[mhttpd#HTTPS/SSL server (Mongoose)]] for details).
** For other options (i.e. HTTPS/SSL proxy) see [[#Secure MIDAS and ELOG Web access]]
*OR open the midas status page at http://localhost:8081 ([[#NOTES|older MIDAS versions]])
* midas status page will show most stuff "red" as nothing is running yet
* DON'T DO THIS YET run ./fevme.exe (on the computer with the VME interface, could be different from computer hosting the experiment), observe that corresponding equipments have been created
* save the url bookmark to the "personal toolbar"
* go to the Programs page, stop mlogger, stop fevme, start mlogger, start fevme
* go to the Status page, start run, stop run
* go back to the Status page, everything should be green
* start a run
* send signals to the ADC gate
* you should be getting events
* to look at data, proceed with setting up the [[ROOTANA|ROOT Analyzer]].

== Start DAQ programs at boot time ==

* add this to /etc/rc.local (replace username and location of the start_daq script)
<pre>
su - alpha -c /home/alpha/online/bin/start_daq.sh
</pre>

== Setup local software version control ==

Version control for experiment source code is setup using "git" (http://git-scm.com/)

* cd $HOME/online
* git init
* git add exptab
* git add bin/start_daq.sh
* git add .gitignore ### contents can be
<pre>
*~
*.o
*.exe
</pre>
* git add src/Makefile src/*.cxx ...
* git commit -a

== Adjust MIDAS buffer sizes ==

Default MIDAS SYSTEM buffer size is 8 Mbytes, fairly small for high-data-rate experiments. The rule of thumb is to have at least a few seconds worth of buffer space available. For example, if event size is 10 Kbytes and the event rate is 1 kHz, data rate is 10*10^3*1*10^3 = 10 Mbytes/sec. To buffer 10 seconds of data we need 100 Mbytes of buffer space.

To resize the MIDAS event buffers (SYSTEM, etc) do this:
* stop all frontends, stop mlogger
* start odbedit:
** cd "/Experiment/Buffer sizes"
** set SYSTEM 100000000
* run "mdump -z SYSTEM"
* if mdump complains about the size of .SYSTEM.SHM, remove it, try again.
* ls -l /dev/shm ### to observe that the size of shared memory is correct

== Secure MIDAS and ELOG Web access ==
In versions prior to May 2015, the default web access to MIDAS and ELOG uses the "http:" protocol which is insecure. In this case, all information is transmitted as clear text meaning that secret, confidential and sensitive information (such as the MIDAS and ELOG passwords and usernames) can be stolen "easily". This means that even "password protected" MIDAS and ELOG pages are not really protected if accessed using the "http" method.

Better security for HTTP is gained by using a password protected '''SSL (https) proxy'''. (It does not provide absolute security because of remaining problems with the security of SSL certificates, security of passwords, etc). Setting up an SSL (https) proxy is described [[#Setting up an HTTP proxy|below]].

Since May 2015, an ''alternative secure option'' to setting up an HTTP proxy is available to users of MIDAS. Recent versions of elogd (ELOG) do support SSL https:// connections, and [[#mhttpd with HTTPS/SSL server (Mongoose)]] is now available. This option is the default, and provides a similar level of security to an HTTP proxy.

See [[Security#Web Access]] for a comparison of these two secure options.

=== mhttpd with HTTPS/SSL server (Mongoose) ===

Since May 2015 the MIDAS web server [[mhttpd]] is explicitly linked with OpenSSL to provide secure HTTPS connections via the [https://bitbucket.org/tmidas/midas/src/ecb9a8537448a8a43f7f9a2bfdb82e578208cde3/doc/mongoose/?at=develop Mongoose] web server (see [[mhttpd]]). With this version, default web access to MIDAS uses the "https" protocol. Web access to {{Utility|name=mhttpd}} can be restricted by using the "-a hostname" switch of [[mhttpd]]. The first time {{Utility|name=mhttpd}} is run, a password file must be created. An SSL certificate is also required. See [[mhttpd#HTTPS/SSL server (Mongoose)|HTTPS/SSL server (Mongoose)]] for instructions.

=== mhttpd using an HTTPS/SSL proxy ===

THESE INSTRUCTIONS ARE WRONG, DO NOT DO THIS. See instead the [https://daq00.triumf.ca/DaqWiki/index.php/Ubuntu#Install_apache_httpd_proxy_for_midas_and_elog TRIUMF DAQ site].

An [[#Setting up an HTTP proxy|HTTP proxy]] must be set up. This is the only way of securing older version of [[mhttpd]] (pre August 2015). Older versions of mhttpd are started using the -p port option e.g.
* mhttpd -D -p 8080 

To run a new version of mhttpd using an HTTP proxy, use the options provided to run the old (non-Mongoose) webserver on a given port, i.e.
* mhttpd --oldserver 8080 --nomg -D 

When using an SSL proxy, only access from the SSL proxy (and maybe some special trusted machines) should be permitted.
This is done using the "-a hostname" switch of [[mhttpd]]. Normally there will be only "-a localhost" switch, enabling access only for the local machine (where the SSL proxy is running). Additional "-a hostname" switches enable access from listed local machines. No "-a xxx" enables access from everywhere (defeating the purpose of the SSL proxy, unless access controls are enforced elsewhere, i.e. by a site firewall or by local firewall rules).

== Setting up an HTTP proxy ==
In this example, we use APACHE HTTPD to password-protect a typical midas/mhttpd and elog installation.

In this configuration, one uses the Linux stock httpd that accepts encrypted https:// connections and forwards them to mhttpd and elogd. Instead of (or in addition to) using mhttpd and elogd passwords, one configures password protection in httpd via the regular apache httpd password mechanisms (htpasswd, etc).

Recent versions of elogd do support SSL https:// connections, but if one is running an SSL proxy for anyway, it is simpler to run both through the same SSL proxy using the same SSL host certificate and the same httpd password file.

=== Restricting http: access to elogd ===
;Note
:Recent versions of elogd do support SSL https:// connections. The following information is for those using an HTTP proxy (see above).

For elogd, this is done using the "-n localhost" switch with enables only access from the same machine if present, or access from anywhere is absent (defeating the purpose of the SSL proxy, unless access controls are enforced elsewhere).

(It is recommended to run elogd from the same user as the main daq user and to keep elogd.cfg and all logbooks in the home directory of this user, where they are captured by the normal site backup system)

== Install standalone elog ==

* login into the user account that will run the elog
* cd $HOME/packages
* git clone https://bitbucket.org/ritt/elog
* cd elog
* make
* create new file start_elogd with this contents:
<pre>
#!/bin/sh

killall elogd
sleep 1
killall -KILL elogd
sleep 1
$HOME/packages/elog/elogd -n localhost -x -c $HOME/packages/elog/elogd.cfg -p 8082 -D

#end
</pre>
* chmod a+x start_elogd
* edit elogd.cfg to read:
<pre>
[global]
port = 8082
SMTP host = smtp.triumf.ca
URL = https://titan00.triumf.ca/elog/

Reverse sort = 1
Display Mode = full

#List Menu commands = New, Find, Admin, Help
#Menu commands = New, Edit, Reply, Find, Duplicate, Help

Entries Per Page = 30
Supress Email on edit = 1
Default encoding = 1
Page title = TITAN ELOG
Resolve host names = 1

Logfile = /home/titan/packages/elog/elogd.log
#Logging level = 3

[midas]

List page Title = T2K M11 MIDAS ELOG
Comment = T2K M11 MIDAS ELOG
Page Title = T2K M11 MIDAS ELOG
RSS Title = [$logbook - $type - $system] $subject, posted by $author

Attributes = Author, Subject, Run, Type, System
Show Attributes Edit = Run, Author, Subject, Type, System
Required Attributes = Author, Type, System, Subject

Options Type = Routine, Reply, Shift Summary, Modification, Question, Info, Problem
Options System = General, DAQ, Beamline

Preset Run = $shell(MIDASSYS=. /home/t2km11/packages/midas/linux/bin/odbedit -d Runinfo -c 'ls -v \"run number\"')

Preset On Reply Type = Reply
Preset On Reply Run = $shell(MIDASSYS=. /home/t2km11/packages/midas/linux/bin/odbedit -d Runinfo -c 'ls -v \"run number\"')

List Display = Date, Subject, Type, System, Author, ID
Quick Filter = Date, Type, ID

Remove on reply = Author
Quote on reply = 1

Use lock = 1

************* Email Functionality ****************

Use Email Subject = [T2KM11 - $System] $Subject
Omit Email To = 1

Email System General = xxx
</pre>
* ./start_elogd &
* firefox http://localhost:8082 # hould show the elog message index

To start elogd automatically when the machine is rebooted, login as root and
* add this text to /etc/rc.local:
<pre>
su - titan -c "/home/titan/packages/elog/start_elogd"
</pre>
* chmod a+x /etc/rc.local
* systemctl start rc-local

To import elog entries from the mhttpd elog, do this:

* cd ~/packages/elog/logbooks
* ln -s /home/t2km11/online/elog midas
* cd midas
* ~/packages/elog/elconv)

== Install https proxy ==

THESE INSTRUCTIONS ARE OBSOLETE, INSTEAD,
* GO HERE: https://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Configure_HTTPS_server_.28CentOS7.29
* AND GO HERE: https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux#Run_the_MIDAS_Web_Server

FOLLOWING INSTRUCTIONS ARE OBSOLETE...

* login as root to the https proxy machine
* cd ~root
* yum install mod_ssl
* yum install crypto-utils # see http://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_monitoring_of_HTTPS_certificates
* create a certificate request (replace ladd09 with your hostname): openssl req -new -nodes -newkey rsa:2048 -sha256 -out ladd09.csr -keyout ladd09.key (answer: CA, BC, Vancouver, TRIUMF, DAQ, ladd09.triumf.ca, email@email.com
* sign it by TRIUMF:
** mail -s "Certificate request" yourself@email.com < ladd09.csr
** forward this request to Andrew Daviel
** he will email the signed crt file, copy it to this system as ladd09.crt
* sign it yourself: openssl x509 -req -days 365 -sha256 -in ladd09.csr -signkey ladd09.key -out ladd09.crt
* (if the certificate expires, renew it by signing it again)
* Additional commands for working with certificates:
** explore the private key: openssl pkey -in ladd09.key -text -noout
** explore the certificate request: openssl req -in ladd00.csr -text -noout
** explore the certificate: openssl x509 -in ladd09.crt -noout -text
* move certificate files to proper system locations:
* mv ladd09.key /etc/pki/tls/private/
* mv ladd09.crt /etc/pki/tls/certs/
* if selinux is enabled, do this:
** restorecon -Rv /etc/pki/tls/certs/
** restorecon -Rv /etc/pki/tls/private/
** /usr/sbin/setsebool -P httpd_can_network_connect 1
* open /etc/httpd/conf.d/ssl.conf in a text editor, go to the very bottom and right before the "</VirtualHost>" entry, add following text:
<pre>
...
SSLCertificateFile /etc/pki/tls/certs/ladd09.crt
SSLCertificateKeyFile /etc/pki/tls/private/ladd09.key

ProxyPass /elog/ http://localhost:8082/ retry=1
ProxyPass / http://localhost:8080/ retry=1

<Location />

SSLRequireSSL
AuthType Basic
AuthName "password protected site"
Require valid-user

# create password file: touch /etc/httpd/htpasswd
# to add new user or change password: htpasswd /etc/httpd/htpasswd username
AuthUserFile /etc/httpd/htpasswd

</Location>
</VirtualHost>
...
</pre>
* comment out duplicate "SSLCertificateFile" and "SSLCertificateKeyFile" elsewhere in the file
* (optionally) If you got a certificate that is signed by DigiCert or RapidSSL then you'll need to add a line specifying the certificate chain file:
<pre>
...
SSLCertificateChainFile /etc/pki/tls/certs/DigiCertCA.crt
...
</pre>
* touch /etc/httpd/htpasswd
* htpasswd /etc/httpd/htpasswd midas # enter password midas
* chkconfig httpd on
* service httpd restart
* firewall-cmd --add-port=443/tcp --permanent
* firewall-cmd --reload
* firewall-cmd --list-all
* test it
** test the SSL proxy: https://host/ should yield the midas status page, https://host/elog/ should yield the elog message index
* in ODB, set "/Elog/URL" to "https://host/elog/"
* now from the midas status page, the "Elog" button should take us to the https Elog URL

In needed, enable user directories: https://blah/~user in ~user/public_html

* edit /etc/httpd/conf.d/userdir.conf, replace "UserDir disabled" with "UserDir enabled"
* setsebool -P httpd_enable_homedirs true
* systemctl restart httpd

== Setup the history mhttpd for faster access to history plots ==
When running an SSL proxy,
* start the main mhttpd (orange command for old mhttpd, green for new mhttpd with Mongoose(post August2015):
** "mhttpd -p 8071 -D" or
** "mhttpd -D --oldserver 8071 --nomg" 

* start the history mhttpd
** "mhttpd -p 8072 -D -H" or
** "mhttpd -D -H --oldserver 8072 --nomg" 
* set ODB /History/URL to "http://alphacpc09.cern.ch:8072/HS/"
* open the MIDAS status page
* go to the history section, try to open any history plot, observe that the history plot gif image loads correctly, inspect it's URL (use "copy image URL" or "view source", etc), it should point to port 8072 causing connection to the history mhttpd.
* continue with these instructions to setup history mhttpd access through an SSL proxy:
* setup SSL proxy access (required mod_proxy_html)
** login as root to the SSL proxy machine
** on SL5, install the missing mod_proxy_html httpd module:
** yum install httpd-devel libxml2-devel
** wget http://apache.webthing.com/mod_proxy_html/mod_proxy_html.tar.bz2
** tar xjvf mod_proxy_html.tar.bz2
** cd mod_proxy_html
** apxs -c -I. -I/usr/include/libxml2 -i mod_proxy_html.c
** apxs -c -I. -I/usr/include/libxml2 -i mod_xml2enc.c
** cd /etc/httpd/conf.d, add this to ssl.conf:

;before the ProxyPass statements:
<pre>
# proxy the MIDAS web servers
LoadModule xml2enc_module modules/mod_xml2enc.so
LoadModule proxy_html_module modules/mod_proxy_html.so
ProxyHTMLLinks a href
ProxyHTMLLinks link href
ProxyHTMLLinks img src
#ProxyHTMLEnable On
ProxyRequests off
</pre>
;after the ProxyPass statements:
<pre>
# ALPHA1 history access
ProxyPass /alpha1/history/ http://alphacpc09.cern.ch:8072/HS/ retry=1
ProxyPass /alpha1/ http://alphacpc09.cern.ch:8071/ retry=1

ProxyHTMLEnable On
ProxyHTMLURLMap http://alphacpc09.cern.ch:8072/HS/ /alpha1/history/
</pre>
;adjust:
*"alpha1" is the experiment name
*"alphacpc09.cern.ch" is the machine running mhttpd
*"8071" is the port number of the main mhttpd
** "mhttpd -p 8071 -D" or
** "mhttpd -D --oldserver 8071 --nomg" 
* "8072" is the port number of the history mhttpd
** "mhttpd -p 8072 -D -H" or
** "mhttpd -D -H --oldserver 8072 --nomg" 

[[Category:Installation]] [[Category:Buffer]] [[Category:Driver]]

Setup MySQL database for MIDAS

2026-02-03T01:37:26Z

Dfujimoto: /* Setting up MySQL on Ubuntu 24.04 LTS */

This page provides instructions for setting up a MySQL database to use with the MIDAS history logging.

Setting up the MySQL database is highly dependent on what OS and OS version you are using. Most of the setup work is just MySQL related. For this reason we provide below instructions for the MySQL database setup on a couple different OS. You will probably need to modify these instructions for your OS. Here's the some installation instructions:

* [[#Setting up MySQL on Ubuntu 24.04 LTS | Setting up MySQL on Ubuntu 24.04 LTS ]]
* [[#Setting up MySQL using mariadb on macbook | Setting up MySQL using mariadb on macbook]]

== Setting up MySQL on Ubuntu 24.04 LTS ==

This instructions assume you have root privileges and install MySQL as ROOT.

=== Do these steps as root user ===

Install the MySQL server software.

<pre>
apt install mysql-server
</pre>

You will find mysql has started after installation

<pre>
root@mpmt-daq03:/home1/wheel# service mysql status
mysql.service - MySQL Community Server
Loaded: loaded (/usr/lib/systemd/system/mysql.service; enabled; preset: enabled)
Active: active (running) since Fri 2025-04-25 15:12:58 PDT; 4s ago
Process: 153129 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=0>
Main PID: 153137 (mysqld)
Status: "Server is operational"
</pre>

Default configuration only allows MySQL connections to localhost. Good
<pre>
root@mpmt-daq03:/home1/wheel# grep bind-add /etc/mysql/mysql.conf.d/mysqld.cnf
bind-address = 127.0.0.1
mysqlx-bind-address = 127.0.0.1
</pre>

Login to MySQL as root and create the MySQL accounts

<pre>
mysql -h localhost
...
mysql> CREATE DATABASE IF NOT EXISTS history;
mysql> CREATE USER history_reader@'localhost' IDENTIFIED BY 'CHANGE_THIS_PASSWORD!';
mysql> GRANT SELECT ON history.* TO history_reader@'localhost';
mysql> CREATE USER 'history_writer'@'localhost' IDENTIFIED BY 'CHANGE_THIS_PASSWORD!';
mysql> GRANT SELECT,INSERT,CREATE,ALTER,INDEX ON history.* TO 'history_writer'@'localhost';
mysql> flush privileges;
</pre>

ATTENTION: change the account password listed above to something different!

Check the socket to connect to:

<pre>
root@mpmt-daq03:/home1/wheel# grep sock /etc/mysql/my.cnf
# Port or socket location where to connect
socket = /run/mysqld/mysqld.sock
</pre>

=== Do these steps as the experiment user ===

Rebuild MIDAS with support for MySQL. If this works correctly the build will notice MySQL is installed. For instance, with cmake we will see

<pre>
-- MIDAS: Found MySQL version 8.0.41
</pre>

Go to online directory and create files so MIDAS knows accounts to talk to ODB

<pre>

testmysql@mpmt-daq03:~/online$ cd ~/online
testmysql@mpmt-daq03:~/online$ emacs -nw mysql_reader.txt
testmysql@mpmt-daq03:~/online$ emacs -nw mysql_writer.txt
testmysql@mpmt-daq03:~/online$ more mysql_*.txt
::::::::::::::
mysql_reader.txt
::::::::::::::
database=history
socket=/run/mysqld/mysqld.sock
user=history_reader
password=CHANGE_THIS_PASSWORD!
::::::::::::::
mysql_writer.txt
::::::::::::::
database=history
socket=/run/mysqld/mysqld.sock
user=history_writer
password=CHANGE_THIS_PASSWORD!
</pre>

Again, set the correct password in these files.

Change the mlogger settings so we use the MySQL for the history logging:
* disable the previous history logger
* enable the MySQL history logging:
<pre>
set "/Logger/History/MYSQL/Active" y
</pre>

Restart the mlogger on the command line and see that some MySQL tables are created and columns added:

<pre>
testmysql@mpmt-daq03:~/online$ mlogger
[Logger,INFO] Per-variable history is enabled
[Logger,ERROR] [history_schema.cxx:1258:Mysql::Prepare,ERROR] mysql_query(SELECT event_name, table_name, itimestamp FROM _history_index WHERE table_name!='';) error 1146 (Table 'history._history_index' doesn't exist)
[Logger,INFO] Adding SQL table "myperiodicequipment_pydt"
[Logger,INFO] Adding SQL table "_history_index"
[Logger,INFO] Adding column "event_name" to SQL table "_history_index", status 1
[Logger,INFO] Adding column "table_name" to SQL table "_history_index", status 1
...
[Logger,INFO] Adding column "fluxgate0" to SQL table "myperiodicequipment_pydt", status 1
[Logger,INFO] Adding column "fluxgate1" to SQL table "myperiodicequipment_pydt", status 1
[Logger,INFO] Adding column "valvex" to SQL table "myperiodicequipment_pydt", status 1
[Logger,INFO] Adding column "valvey" to SQL table "myperiodicequipment_pydt", status 1
[Logger,INFO] Adding column "t5" to SQL table "myperiodicequipment_pydt", status 1
...
</pre>

Login directly to the MySQL database and make sure the data is actually being logged

<pre>
mysql> select _i_time,fluxgate0 from myperiodicequipment_pydt where _i_time <1745620795;
+------------+-----------+
| _i_time | fluxgate0 |
+------------+-----------+
| 1745620782 | 6.20209 |
| 1745620783 | 6.3519 |
| 1745620784 | 6.5677 |
| 1745620785 | 7.81413 |
| 1745620786 | 5.41127 |
| 1745620787 | 7.31112 |
| 1745620788 | 6.14285 |
| 1745620789 | 7.20425 |
| 1745620790 | 6.9303 |
| 1745620791 | 7.08178 |
| 1745620792 | 7.44422 |
| 1745620794 | 5.32677 |
+------------+-----------+
12 rows in set (0.00 sec)
</pre>

Restart mhttpd and you will see the history plots starting to get data from MySQL database.

=== Change database location ===

To move the database to a different location on file do the following as root ([https://tecadmin.net/change-default-mysql-data-directory-in-linux/ source]):
<pre>
systemctl stop mysql
cp -a /var/lib/mysql /new/data/directory
</pre>
Then update the MySQL configuration file /etc/mysql/my.cnf
<pre>
[mysqld]
...
datadir=/new/data/directory
...
</pre>
You will need to either (a) adjust your apparmor settings, or (b) fully [https://daq00.triumf.ca/DaqWiki/index.php/Ubuntu#disable_apparmor disable apparmor].
Finally, restart the server:
<pre>
systemctl start mysql
</pre>
You can now delete the original database directory
<pre>
rm -rf /var/lib/mysql
</pre>

== Setting up MySQL using mariadb on macbook ==

* install mariadb-10.4
<pre>
macos:
sudo port install mariadb-10.4
ln -s ln -fs /opt/local/lib/mariadb-10.4/bin/mysql_config ~/bin/
mysql_config --version
10.4.13
</pre>
* rebuild midas, check the cmake finds the correct mysql: MIDAS: Found MySQL version 10.4.13
* create $HOME/.my.cnf
<pre>
[mysqld]
#port=3306
socket=/Users/olchansk/mysql/mysql.sock
datadir=/Users/olchansk/mysql/db
pid-file=/Users/olchansk/mysql/mysqld.pid

[client]
#port=3306
socket=/Users/olchansk/mysql/mysql.sock
</pre>
* mkdir $HOME/mysql
* start mysql:
<pre>
macos: /opt/local/lib/mariadb-10.4/bin/mysqld
</pre>
* configure the database: start the "mysql" utility (/opt/local/lib/mariadb-10.4/bin/mysql)
<pre>
NOTE: instructions thanks to Ben S.
NOTE: use some other passwords instead of the example "reader_password" and "writer_password"

MariaDB [(none)]> CREATE DATABASE IF NOT EXISTS history;
Query OK, 1 row affected (0.003 sec)

MariaDB [(none)]> CREATE USER history_reader@'localhost' IDENTIFIED BY 'reader_password';
Query OK, 0 rows affected (0.027 sec)

MariaDB [(none)]> GRANT SELECT ON history.* TO history_reader@'localhost';
Query OK, 0 rows affected (0.016 sec)

MariaDB [(none)]> CREATE USER 'history_writer'@'localhost' IDENTIFIED BY 'writer_password';
Query OK, 0 rows affected (0.014 sec)

MariaDB [(none)]> GRANT SELECT,INSERT,CREATE,ALTER,INDEX ON history.* TO 'history_writer'@'localhost';
Query OK, 0 rows affected (0.017 sec)

MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.001 sec)
</pre>
* try connecting to both users:
<pre>
/opt/local/lib/mariadb-10.4/bin/mysql -u history_reader -p ### should ask for reader password
/opt/local/lib/mariadb-10.4/bin/mysql -u history_writer -p ### should ask for writer password
</pre>
* create midas config files:
** mysql_writer.txt
<pre>
#server=localhost
#port=xxx
database=history
socket=/Users/olchansk/mysql/mysql.sock
user=history_writer
password=writer_password ### change this!!!
#buffer=1000
</pre>
** mysql_reader.txt
<pre>
#server=localhost
#port=xxx
database=history
socket=/Users/olchansk/mysql/mysql.sock
user=history_reader
password=reader_password ### change this!!!
</pre>
* test MIDAS connection
<pre>
mh2sql --mysql mysql_writer.txt
(there should be no errors printed)
</pre>
* if desired, import midas history from .hst files:
<pre>
mh2sql --mysql mysql_writer.txt *.hst
Reading 130813.hst
[mh2sql,ERROR] [history_schema.cxx:1165:Mysql::Prepare,ERROR] mysql_query(SELECT event_name, table_name, itimestamp FROM _history_index WHERE table_name!='';) error 1146 (Table 'history._history_index' doesn't exist)
[mh2sql,INFO] Adding SQL table "rpcexample", status 311
[mh2sql,INFO] Adding SQL table "rpcexample_1596127680", status 1
[mh2sql,INFO] Adding SQL table "_history_index", status 1
[mh2sql,INFO] Adding column "event_name" to SQL table "_history_index", status 1
...
[mh2sql,INFO] Adding SQL table "run_transitions", status 311
[mh2sql,INFO] Adding SQL table "run_transitions_1596127681", status 1
[mh2sql,INFO] Adding column "state" to SQL table "run_transitions_1596127681", status 1
[mh2sql,INFO] Adding column "run_number" to SQL table "run_transitions_1596127681", status 1
Reading 130815.hst
</pre>

** change ODB /Logger/History/MYSQL/Active to "y", "mysql writer" and "mysql reader" to location of these files (leave the default values if these files are in the experiment directory).
* enable MYSQL history in mhttpd:
** change ODB /History/LoggerHistoryChannel to "MYSQL"

Message System

2026-01-15T20:32:58Z

Dfujimoto:

{{Pagelinks}}
<div id="midas.log"></div>
== MIDAS Log file ==
MIDAS provides a general MIDAS log file for recording system and user messages across the different components of the data acquisition clients. The default name of this file is '''''midas.log'''''. It is not necessary for [[mlogger]] to be running for the MIDAS message logging system to work. As soon as any MIDAS application is started, the MIDAS log file is produced.
The [[#Example MIDAS Log file|MIDAS log file]] contains system and user messages generated by any application connected to the given experiment.

Note that MIDAS messages may alternatively be sent into the data stream for logging.

== Viewing MIDAS messages ==
All the MIDAS messages can be found in '''''midas.log'''''.
More convenient is the [[mhttpd]] [[Message Page]] where the latest MIDAS messages are displayed. The very latest midas message can also be found on the [[mhttpd]] [[Status Page]]. Midas messages are also sent to any clients running in an xterm (e.g. [[odbedit]]).

<div id="midasloglocation"></div>
== Location of the MIDAS log file midas.log ==
The location of this file is dependent on how the experiment has been setup, i.e.

; with the [[/Logger ODB tree]] :
If the ''/Logger'' ODB tree exists (i.e. has been created by [[mlogger]])
* The message file will be located in the defined directory specified by the key [[Keys in the ODB /Logger tree#Data dir|/Logger/Data dir]].
* The name of the message file may be changed using the ODB key [[Keys in the ODB /Logger tree#Message File|/Logger/Message File]].

;without the [[/Logger ODB tree]] :
If the ''/Logger'' ODB tree does NOT exist , i.e. the MIDAS standard logger [[mlogger]] is not being used,
* the location of the log file will be in the experiment-specific directory defined by either
** the Environment Variable named MIDAS_DIR , or if this not defined,
** the exptab file .

----
== Generating messages for the MIDAS log ==
Any client can produce status or error messages to be sent to the [[#MIDAS Log file]] with a single call to
[http://ladd00.triumf.ca/~daqweb/doc/midas-devel/doc/html/group__msgfunctionc.html#gaac032ca2438c47466bfc9722de6746ea cm_msg()] using the MIDAS library, e.g.

status = db_find_key(hDB, 0, "/Equipment/Cycle_scalers/Settings/",&hKey);
if(status != DB_SUCCESS && status != DB_NO_KEY)
{
cm_msg(MERROR, "begin_of_run", "error accessing \"/Equipment/Cycle_scalers/Settings/\" (%d)",status);
return status;
}

Depending on the [[#message types|message type]], these messages are forwarded to any other clients who may be available to receive these messages, as well as to the central MIDAS Log file
midas.log.
The message system is based on the [[Midas Core#The Buffer Manager|buffer manager]] scheme, but with a dedicated header to identify the type of message. A dedicated buffer (i.e. shared
memory) .SYSMSG.SHM is used to receive and distribute messages.

<div id="Message Macros"></div>

== Message Types ==

The predefined '''MIDAS Message macros''' contained in the MIDAS library provide a list of defined messages types.
These Macros compact the 3 first arguments of the cm_msg() call.
The Macro replaces the type of message, the routine name and the line number in the C-code. See
[http://ladd00.triumf.ca/~daqweb/doc/midas-devel/doc/html/group__msgfunctionc.html#gaac032ca2438c47466bfc9722de6746ea example] in cm_msg().

The available message types ( defined in midas.h ) for use with cm_msg() are:

;MT_ERROR
:Error message, to be displayed in red
:<code>(MT_ERROR, __FILE__, __LINE__)</code>
;MT_INFO
:Info or status message
:<code>(MT_INFO, __FILE__, __LINE__)</code>
;MT_DEBUG
:Only sent to SYSMSG buffer, not to midas.log file. Handy if you produce lots of message and don't want to flood the message file. Plus it does not change the timing of your app, since the SYSMSG buffer is much faster than writing to a file.
:<code>(MT_DEBUG, __FILE__, __LINE__)</code>
;MT_USER
:Message generated interactively by a user, like in the chat window or via the odbedit "msg" command
:<code>(MT_USER, __FILE__, __LINE__)</code>
;MT_LOG
:Messages with are only logged but not put into the SYSMSG buffer
:<code>(MT_LOG, __FILE__, __LINE__)</code>
;MT_TALK
:Messages which should go through the speech synthesis in the browser and are "spoken"
:<code>(MT_TALK, __FILE__, __LINE__)</code>
;MT_CALL
:Message which would be forwarded to the user via a messaging app (historically this was an actual analog telephone call via a modem)
:<code>(MT_CALL, __FILE__, __LINE__)</code>

== Example MIDAS Log file ==

Fri Mar 24 10:48:40 2000 [CHAOS] Run 8362 started
Fri Mar 24 10:48:40 2000 [Logger] Run #8362 started
Fri Mar 24 10:55:04 2000 [Lazy_Tape] cni-043[10] (cp:383.6s) /dev/nst0/run08360.ybs 849.896MB file NEW
Fri Mar 24 11:24:03 2000 [MStatus] Program MStatus on host umelba started
Fri Mar 24 11:24:03 2000 [MStatus] Program MStatus on host umelba stopped
Fri Mar 24 11:27:02 2000 [Logger] stopping run after having received 1200000 events
Fri Mar 24 11:27:03 2000 [CHAOS] Run 8362 stopped
Fri Mar 24 11:27:03 2000 [SUSI] saving info in run log
Fri Mar 24 11:27:03 2000 [Logger] Run #8362 stopped
Fri Mar 24 11:27:13 2000 [Logger] starting new run
Fri Mar 24 11:27:14 2000 [CHAOS] Run 8363 started
Fri Mar 24 11:27:14 2000 [CHAOS] odb_access_file -I- /Equipment/kos_trigger/Dump not found
Fri Mar 24 11:27:14 2000 [Logger] Run #8363 started
Fri Mar 24 11:33:47 2000 [Lazy_Tape] cni-043[11] (cp:391.8s) /dev/nst0/run08361.ybs 850.209MB file NEW
Fri Mar 24 11:42:35 2000 [CHAOS] Run 8363 stopped
Fri Mar 24 11:42:40 2000 [SUSI] saving info in run log
Fri Mar 24 11:42:41 2000 [ODBEdit] Run #8363 stopped
Fri Mar 24 12:19:57 2000 [MChart] client [umelba.Triumf.CA]MChart failed watchdog test after 10 sec
Fri Mar 24 12:19:57 2000 [MChart] Program MChart on host koslx0 stopped

[[Category:Messages]] [[Category:Macros]]

Alarm System

2025-10-31T17:16:50Z

Dfujimoto: Added messagebird voice calling instructions

{{Pagelinks}}

= Links =
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* [[/Alarms ODB tree]]
* [[Alarms Page]]
* [[Mhttpd|mhttpd MIDAS web server]]
</div>

= Introduction =
MIDAS provides an alarm system, which by default is turned off. When the alarm system is [[/Alarms ODB tree#Alarm system active|activated]] and an alarm condition is detected, alarm messages are sent by the system which appear as an alarm banner on the [[Status Page|mhttpd status page]], and as a message on any windows running [[odbedit]] clients. The alarm system is flexible and can be extensively customized for each experiment
using the [[Alarms Page|mhttpd Alarms Page]] or [[odbedit]].

The alarm system is built-in and part of the main experiment scheduler. This means no separate task is necessary to benefit from the alarm system. Its setup and activation is done through the '''[[/Alarms ODB tree]]'''. The alarm system includes several other features such as sequencing and control of the experiment. The alarm capabilities are:
* Alarm setting on any ODB variable against a threshold parameter.
* Alarm triggered by evaluated condition
* Selection of Alarm check frequency
* Selection of Alarm trigger frequency
* Customization alarm scheme; under this scheme multiple choices of alarm type can be selected
* Selection of alarm message destination (to system message log or to elog)
* email or SMS alerts can be sent
* Alarm triggered when a Program is not running

= Implementation of the MIDAS Alarm System =

The alarm system source code is [http://ladd00.triumf.ca/~daqweb/doc/midas-devel/doc/html/alarm_8c_source.html alarm.c].
Alarms are checked inside [http://ladd00.triumf.ca/~daqweb/doc/midas-devel/doc/html/group__alfunctioncode.html alarm.c::al_check()]. This function is called by cm_yield() every 10 seconds and by rpc_server_thread(), also every 10 seconds. For remote MIDAS clients, their al_check() issues an RPC_AL_CHECK RPC call into the MIDAS server utility [[mserver]], where rpc_server_dispatch() calls the local al_check(). As result, all alarm checks run inside a process directly attached to the local MIDAS shared memory (inside a local client or inside an mserver process for a remote client). Each and every MIDAS client runs the alarm checks. To prevent race conditions between different MIDAS clients, access to al_check() is serialized using the ALARM semaphore. Inside al_check(), alarms are triggered using
[http://ladd00.triumf.ca/~daqweb/doc/midas-devel/doc/html/group__alfunctioncode.html al_trigger_alarm()], which in turn calls al_trigger_class(). Inside al_trigger_class(), the alarm is recorded into an elog or into [[Message System|midas.log]] using cm_msg(MTALK).

Special note should be made of the ODB setting [[/Alarms ODB tree#System message interval|system message interval]], which has a surprising effect - after an alarm is recorded into system messages (using cm_msg(MTALK)), no record is made of any subsequent alarms until the time interval set by this variable elapses. With default value of 60 seconds, after one alarm, no more alarms are recorded for 60 seconds. Also, because all the alarms are checked at the same time, '''only the first''' triggered alarm will be recorded.

As of alarm.c rev 4683, {{Odbpath|path=/Alarms/System message interval}} is set to 0 ensures that every alarm is recorded into the [[Message System#MIDAS Log file|MIDAS log file]]. (In previous revisions, this setting may still miss some alarms).

<div id="Alarm class"></div>
= Alarms structure =
The [[/Alarms ODB tree]] structure is split into 2 sections:
*"Alarms" which define the condition to be tested. The user can create as many [[/Alarms ODB tree#Alarms subtree|Alarms]] as desired, but each must be one of the four defined [[#Alarm Types|Alarm types]] .
*"Classes" which define the action to be taken when the alarm occurs. Two Classes (Alarm and Warning) are defined by default. The user can add more [[/Alarms ODB tree#Classes subtree|Classes]] as desired.
 

In order to make the system flexible, each alarm class may perform different actions when an alarm is given. For example, it may
* write a system message (see [[/Alarms ODB tree#Write System Message|Write System Message]])
* write to the elog (see [[/Alarms ODB tree#Write Elog Message|Write Elog Message]])
* stop the run (see [[/Alarms ODB tree#Stop run|Stop run]])
* spawn a detached script listed in the ODB variable [[/Alarms ODB tree#Execute command|Execute command]]. This feature is used when an Alarm triggers Email or SMS alerts (see [[#Alarm triggering Email or SMS alerts|example]]).
 

= Alarm Types =

The four available Alarm Types are shown in Table 1. They are defined in [http://ladd00.triumf.ca/~daqweb/doc/midas-devel/html/midas_8h.html&nbsp midas.h]. The alarm type is entered into the [[/Alarms ODB tree#Type|Type]] key.

{| style="text-align: left; width: 100%; background-color: white;" border="3" cellpadding="2" cellspacing="2"
|+ Table 1 : Defined Alarm Types
|-
| colspan="2" rowspan="1" style="vertical-align: top; background-color: lavender; font-weight: bold;" | Alarm Type
| colspan="1" rowspan="1" style="vertical-align: top; background-color: lavender; font-weight: bold;" | INT value
| colspan="1" rowspan="1" style="vertical-align: top; background-color: lavender; font-weight: bold;" | Explanation

|-
| colspan="1" rowspan="1" style="vertical-align: top; background-color: white; font-weight: bold;" |Internal alarms
| colspan="1" rowspan="1" style="vertical-align: top; background-color: white; font-weight: normal;" | AT_INTERNAL
|1
|Trigger on internal (program) alarm setting through the use of the al_...() functions.

|-
| style="vertical-align: top; background-color: white; font-weight: bold;" |Program alarms
|AT_PROGRAM
|2
|Triggered on condition of the state of the defined task (i.e. program not running)

|-
| style="vertical-align: top; background-color: white; font-weight: bold;" |Evaluated alarms
|AT_EVALUATED
|3
|Triggered by ODB value on given arithmetical condition.

|-
| style="vertical-align: top; background-color: white; font-weight: bold;" |Periodic alarms
|AT_PERIODIC
|4
|Triggered by timeout condition defined in the alarm setting.

|}

== Program Alarm ==

Program (or rather "Program not running") alarms, when enabled, warn the user when a program is not running.

Program alarms are enabled by setting the ODB key [[/Programs ODB tree#Alarm class|/Programs/<client-name>/Alarm class]] to a valid Alarm class specified in the [[/Alarms ODB tree]]. The first time the alarm is triggered, an /Alarms/Alarms/<client-name> subtree will be created automatically. The program alarm will not be visible in the [[Alarms Page]] until the alarm has triggered, and the subtree created.

The alarm system periodically calls [http://ladd00.triumf.ca/~daqweb/doc/midas-devel/doc/html/group__alfunctioncode.html al_check()]. This causes every client listed in the {{Odbpath|path=/Programs}} ODB tree to be tested using [http://ladd00.triumf.ca/~daqweb/doc/midas-devel/doc/html/group__cmfunctionc.html cm_exist()] to see if it is running. If the client is not running, the time of first failure is recorded in the ODB key [[/Programs ODB tree#First failed|/Programs/<client-name>/First failed]].

If the client has not been running for longer than the time set in ODB key [[/Programs ODB tree#Check interval|/Programs/<client-name>/Check interval]], a "Program not running" alarm is triggered (if enabled by [[/Programs ODB tree#Alarm class|Alarm class]]) and the program is restarted (if enabled by [[/Programs ODB tree#Auto restart|/Programs/<client-name>/Auto restart]] and a valid [[/Programs ODB tree#Start command|Start command]] is supplied).

The "not running" condition is tested every 10 seconds (each time al_check() is called), but the frequency of ''Program not running'' alarms can be reduced by increasing the value of the ODB key
[[/Programs ODB tree#Check interval|/Programs/<client-name>/Check interval]]
(default value 60 seconds). This can be useful if [[/Alarms ODB tree#System message interval|System message interval]] in the specified alarm class subtree is set to zero.

== Periodic Alarm ==
The periodic alarm is activated periodically according to the time in [[/Programs ODB tree#Check interval|/Programs/<client-name>/Check interval]]. An example of a periodic alarm is "Demo Periodic" in the [[/Alarms ODB tree#Example|example]].

== Evaluated Alarm ==

Evaluated alarms require an ''alarm condition'' which is entered into the ODB key [[/Alarms ODB tree#Condition|Condition]] in the [[/Alarms ODB tree#<alarm_name> subtree|<alarm_name> subtree]].
The condition may be simply a '''comparison''' between any ODB variable and a threshold parameter, e.g.

/Runinfo/Run number > 100
or it may be an '''evaluated condition'''. One can write conditions like

/Equipment/HV/Variables/Input[*] > 100
or

/Equipment/HV/Variables/Input[2-3] > 100
or

/Equipment/HV/Variables/Input[1,4,5-8,10] > 100

to check all or certain values from an array. A dash means a range including both indices. If one array element fulfills the alarm condition, the alarm is triggered. In addition, bit-wise alarm conditions are possible, e.g.

/Equipment/Environment/Variables/Input[0] & 8
The alarm is triggered if bit #3 is set in Input[0].

The value of an evaluated alarm is computed using al_evaluate_condition() in [http://ladd00.triumf.ca/~daqweb/doc/midas-devel/html/alarm_8c_source.html alarm.c].

Sometimes alarm trigger kind of accidentally on some analog input voltage if they are noise. In order to avoid this, the "Trigger count required" can be used. If this value contains a non-zero value of N, then the alarm system requires the alarm condition to be met N consecutive times until the alarm to be triggered.

== Internal Alarm ==
These are triggered in a program using a call to
[https://daq.triumf.ca/~daqweb/doc/midas-devel/html/group__alfunctioncode.html al_trigger_alarm()]. See also description of al_trigger_alarm() sequence [[#Implementation of the MIDAS Alarm System|above]].

There is nothing surprising in these alarms. Each alarm is checked with a time period set by ODB key [[ /Alarms ODB tree#Check interval|Check interval]] in the [[/Alarms ODB tree]].

= Alarm triggering Email or SMS alerts =

It is possible to have the MIDAS alarm system send email or SMS alerts to cell phones when alarms are triggered. This can be configured by defining an ODB alarm on a critical ODB parameter, e.g.

/Alarms/Alarms/Liquid Level
Active y
Triggered 0 (0x0)
Type 3 (0x3)
Check interval 60 (0x3C)
Checked last 1227690148 (0x492D10A4)
Trigger count 0 (0x0)
Trigger count required 0 (0x0)
Time triggered first (empty)
Time triggered last (empty)
Condition /Equipment/Environment/Variables/Input[0] < 10
Alarm Class Level Alarm
Alarm Message Liquid Level is only %s

In this example, the alarm triggers an alarm of class "Level Alarm". This alarm class is defined as follows:

/Alarms/Classes/Level Alarm
Write system message y
Write Elog message n
System message interval 600 (0x258)
System message last 0 (0x0)
Execute command /home/midas/level_alarm '%s'
Execute interval 1800 (0x708)
Execute last 0 (0x0)
Stop run n
Display BGColor red
Display FGColor black

The key here is to call a script "level_alarm", which can send emails. Use something like:

#/bin/csh
echo $1 | mail -s \"Level Alarm\" your.name@domain.edu
odbedit -c 'msg 2 level_alarm \"Alarm was sent to your.name@domain.edu\"'

Ensure the shell script is executable (e.g., <code>chmod +x script.sh</code>).

The second command just generates a MIDAS system message for confirmation. Most cell phones (depends on the provider) have an email address. If you send an email there, it will be translated into a SMS message.

The script file above can of course be more complicated. A perl script could be used that parses an address list, so other interested parties can register by adding his/her email address to that list. The script may also collects some other slow control variables (like pressure, temperature) and combine them into the SMS message.

For very sensitive systems, having an alarm via SMS may not be sufficient, since the alarm system could be down (e.g. computer crash, network failure). In this case 'negative alarms' can be used. For example, every 30 minutes the system may send an SMS with the current parameter values. If the expected message is not received, it may indicate that something in the MIDAS system is wrong.

= Alarm triggering Slack notifications =

A more modern way for notification is to use messengers apps such as Slack. To send alarms to Slack, do the following:

* Go to https://api.slack.com/apps
* Create an App "MIDAS alarms", select "From scratch"
* Name it "MIDAS alarm" or similar, select your workspace
* Click on "Add features and functionality"
* Select "Incoming Webhooks"
* Activate Incoming Webhooks
* Click on "Add New Webhook to Workspace"
* Select channel where alarms get posted, allow access
* Copy sample curl request and replace "Hello World" by "$1", such as:

curl -X POST -H 'Content-type: application/json' --data "{\"text\":\"$1\"}' https://hooks.slack.com/services/[xxx]/[yyy]/[zzz]

(leave they keys xxx, yyy, zzz as shown on the web page).

* Call the curl command from the alarm system by putting the above command under <code>/Alarms/Classes/All/Execute command</code> into the ODB.
* Ensure the shell script is executable (e.g., <code>chmod +x script.sh</code>)

= Alarm triggering Telegram notifications =

A telegram bot needs to be created to post Midas alarms into telegram channels.

* Go to https://web.telegram.org and login by scanning the QR-code with the telegram app on your phone
* Go to https://t.me/botfather
* Click on "open in web"
* A chat window will open.
* Send "/newbot" into the chat and assign name and username to your bot
* At the end of the bot creation a access token will be displayed. Save that access token.
* Post "/mybots" into the chat
* Select the bot that was just created
* Click on "Bot settings"
* Click on "Allow Groups"
* Enable groups for the bot
* Click on "back to settings"
* Click on Channel Admin Rights
* Click on "Manage channel" (a checkmark should appear next to the text)
* Close this chat.

* Add the created bot to a telegram group chat via searching for the bot name in "add members" (same as adding new People to the group)
* Now you need the chat_id of the same group chat. When you open the group chat in a webbrowser the URL should look like this https://web.telegram.org/a/#-1700000000 (where 1700000000 will be a different number for you)
* The group chat_id is then -1001700000000 (add -100 in front of the number in your URL)
* With the chat_id from above and the access token from the bot creation process you can post to the telegram group like this: (replace <chat_id> and <accessToken> with your numbers)

curl -s -X POST "https://api.telegram.org/bot<accessToken>/sendMessage" -d chat_id=<chat_id> -d text="🚨 Midas alarm triggered with message = $1"

* Create a shell script with this command
* Ensure the shell script is executable (e.g., <code>chmod +x script.sh</code>)
* Call the shell script from the alarm system by putting <code>path/to/script.sh '%1$s'</code> under <code>/Alarms/Classes/All/Execute command</code> into the ODB.
* Should you want to post the same alarm to multiple systems you can just separate scripts in <code>/Alarms/Classes/All/Execute command</code> with a semicolon <code>path/to/script.sh '%1$s';path/to/script2.sh '%1$s'</code>

= Alarm triggering Mattermost notifications =

First a mattermost bot needs to be created. To do this you need to be an Admin of the Team that the bot should post to.

* In the drop down menu on the top left corner of the mattermost page click on "Integrations" (note: this field will be missing if you are not an Admin of the selected mattermost team)
* Select "Incoming Webhooks"
* Click on "Add Incoming Webhook"
* Give the bot a title and username (e.g. midasalarms) and select the channel to post to, click "save"
* Now a URL for the incoming webhook should be displayed (https://mattermost.gitlab.rlp.net/hooks/<token>). Copy that URL into the curl command in the following script:

#!/bin/bash
alarm_message=$1
generate_post_data()
{
cat <<EOF
{
"text": "@all Midas alarm triggered with message = $alarm_message",
"icon_emoji":":rotating_light:",
"username":"Midas"}
EOF
}
curl -i -X POST -H 'Content-Type: application/jsoni' --data "$(generate_post_data)" https://mattermost.gitlab.rlp.net/hooks/<token>

- Call the shell script from the alarm system by putting <code>path/to/script.sh '%1$s'</code> under <code>/Alarms/Classes/All/Execute command</code> into the ODB.
Ensure the shell script is executable (e.g., <code>chmod +x script.sh</code>).

= Alarm triggering MessageBird Voice Call =

We will use a text-to-voice to read out MIDAS alarms in a phone call to shifters. The service is called bird (formerly messagebird). Note that bird hosts two platforms: "engagement" and "connectivity". Each has its own API, documentation, and web interface. We will use the engagement platform. Be sure to look at the [https://docs.bird.com correct documentation].

== MessageBird Setup ==

# Make an account: https://app.bird.com/
# Add some funds to your main wallet: https://app.bird.com/settings/billing/overview/wallets/
# Buy a number. This number will be the where the call originates from, so you should pick something local to your area.
## In the left-hand navigation side panel: "Developer" -> "Numbers" -> "Numbers & Senders"
## Top-right: "Browse Inventory"
## Make sure your number is capable of voice calling (it seems like most of them are)
## Follow the on-screen prompts to finish your purchase and activate the number. You will have to create a UseCase. Either "Mixed" or "Security Alert" should do.
# Ensure compliance with local regulations
## Go back to the number summary: "Developer" -> "Numbers" -> "Numbers & Senders". You should now have you number listed. Click on it.
## At the top there is a menu, select "Voice"
## You should see some links to set up brand identification and compliance (what will you be using the number for, the company associated with the number, etc). Follow the on-screen instructions. Once compliance has been requested it may take a few days to be approved.
## While you wait, you also need to create a Voice Channel. In this same menu, click on "+ Create channel". You can see your channels from the side bar navigation: "Developer" -> "Voice" -> "Voice Setup".

== Information Needed to Make Calls ==

You need a few pieces of info:

* API Access Key
** In the side bar navigation: "Developer" -> "API Access Keys" (at the bottom)
** Make a new key. Keep it somewhere safe.
* Workspace ID
** In the side bar navigation: "Settings" (at the bottom) -> "Workspaces" -> Click on your workspace
* Channel ID
** In the side bar navigation: "Developer" -> "Voice" -> "Voice Setup" -> Click on your voice channel -> Settings (top right gear)
* Call ID
** Only available once the call is placed. See the script below.
* Outgoing number
** In the side bar navigation: "Developer" -> "Numbers" -> "Numbers & Senders"

== Placing a Call ==

You can follow the documentation here: https://docs.bird.com/api/voice-api/voice-calls-api. Here is a python example script:
<pre>
import json, requests, time

# settings
workspaceId = 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
channelId = 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
APIKEY = 'YourAPIKey'
outgoing = '16042221047' # outgoing number with country code
targetnumber = 'YourPurcahsedPhoneNumber' # who to call
message = "Hello, world! This is my first message."

# place a call
# https://docs.bird.com/api/voice-api/voice-calls-api/initiate-an-outbound-call
def place_call(number):

# place the call
response = requests.post(
f"https://api.bird.com/workspaces/{workspaceId}/channels/{channelId}/calls",
headers={"Authorization":f"AccessKey {APIKEY}","Content-Type":"application/json"},
data=json.dumps({
"from": outgoing,
"to": number,
"ringTimeout": 60,
"maxDuration": 120,
"record": False,
})
)

# call id is needed for the next steps
data = response.json()
callId = data['id']
return callId

# get the call status
# https://docs.bird.com/api/voice-api/voice-calls-api/get-a-call-insights
def get_call_status(callId):
response = requests.get(
f"https://api.bird.com/workspaces/{workspaceId}/channels/{channelId}/calls/{callId}/insights",
headers={"Authorization":f"AccessKey {APIKEY}","Accept":"*/*"},
)
data = response.json()
return data['status']

# text-to-voice: play message in the call
# https://docs.bird.com/api/voice-api/voice-calls-api/say-text-to-speech-tts
def play_message(callId, message):
response = requests.post(
f"https://api.bird.com/workspaces/{workspaceId}/channels/{channelId}/calls/{callId}/say",
headers={"Authorization":f"AccessKey {APIKEY}","Content-Type":"application/json"},
data=json.dumps({
"text": message,
"locale": "en-US",
"voice": "text",
"loop": 10, # repeat the message this many times
"timeout": 30
})
)
return response.json()

# RUN =========================

# place call
callId = place_call(targetnumber)

# wait until call is answered, or until 30s has passed
t0 = time.time()
while time.time() - t0 < 30:
status = get_call_status(callId)
print(f'Call status: "{status}"')

# call was answered
if status == 'ongoing':
break

# check again later
time.sleep(1)

# play the message
time.sleep(0.5)
play_message(callId, message+',') # comma after message introduces a pause between loops
</pre>

[[Category:Alarms]]