> I recently upgraded my macbook to MacOS 10.11.  The compilation of MIDAS failed after the upgrade, 
> complaining about  
> 
> gcc  -c -g -O2 -Wall <snip> src/mongoose.c
> src/mongoose.c:322:10: fatal error: 'openssl/ssl.h' file not found
> 
> It seems that MacOS has now fully removed openssl header files (they were deprecated for a while).  There 
> seems to be some notes on that here
> 
> http://lists.apple.com/archives/macnetworkprog/2015/Jun/msg00025.html
> 
> Konstantin suggested installing open-source builds of openssl using MacPorts.  I did that and MIDAS 
> compiled fine.  I documented the procedure here:
> 
> https://midas.triumf.ca/MidasWiki/index.php/Installation/Compilation_problems#MacOS_10.11_.28El_Capitan.2
> 9_openssl_errors

The MIDAS Wiki page points to https://guide.macports.org/  which covers OSX up to 10.9. Installers for 10.10 and the current 10.11 
(El Captain) can be found here: https://www.macports.org/install.php

Stefan

Dear all,
      I'm trying to use the logging on a mysql DB. Following the instructions on 
the Wiki, I recompiled MIDAS after installing mysql, and cmake with NEED_MYSQL=1 
can find it:

-- MIDAS: Found MySQL version 8.0.23

Then, I compiled my frontend (cmake with no options + make) and run it, but in the 
ODB I cannot find the tree for mySQL. I have only:

Logger/Runlog/ASCII

while I would expect also:

Logger/Runlog/SQL

What could be missing? Maybe should I add something in the CMakeList file or run 
cmake with some option?

Thank you,
      Francesco

I solved this, it was a failed "make clean" before recompiling. Now it works.

Sorry for the noise.

Francesco

> Dear all,
>       I'm trying to use the logging on a mysql DB. Following the instructions on 
> the Wiki, I recompiled MIDAS after installing mysql, and cmake with NEED_MYSQL=1 
> can find it:
> 
> -- MIDAS: Found MySQL version 8.0.23
> 
> Then, I compiled my frontend (cmake with no options + make) and run it, but in the 
> ODB I cannot find the tree for mySQL. I have only:
> 
> Logger/Runlog/ASCII
> 
> while I would expect also:
> 
> Logger/Runlog/SQL
> 
> What could be missing? Maybe should I add something in the CMakeList file or run 
> cmake with some option?
> 
> Thank you,
>       Francesco

Hi,
 has anyone tried runnning midas frontend on a Linux running 
on a Motorola MVME167 motorola embedded CPU?
I have seen people running Linux on a MV167 
(http://www.sleepie.demon.co.uk/linuxvme/)
so in principle this can be done.

The reason I am asking is that we have a lot of them in house 
and we would like to avoid paying for VxWorks
(I have succesfully run Midas on a mvme167/VxWorks node)

Or maybe one has come up with a much better solution 
[short of dumping mv167 into a sewer :)]

Piotr

After much discussion, and following the MIDAS workshop at TRIUMF, we made the decision to use C++11 in MIDAS.

There are many benefits, and only one drawback - no c++11 compilers in the default OS install on older computers (i.e. 
RHEL/SL/CentOS before el7). (the same applies to our use of cmake).

Specifically for el6, the solution is to use c++11 compatible gcc-8 from devtoolset-8, see 
https://midas.triumf.ca/elog/Midas/1649

The c++11 features we most welcome - initialization of class members at declaration time (no more forgetting to add initialization to 
each and every constructor), c++ threads and mutexes, lambdas and "auto".

K.O.

> After much discussion, and following the MIDAS workshop at TRIUMF, we made the decision to use C++11 in MIDAS.
> 
> There are many benefits, and only one drawback - no c++11 compilers in the default OS install on older computers (i.e. 
> RHEL/SL/CentOS before el7). (the same applies to our use of cmake).
>

It turns out that support for the c++11 "regex" feature is missing on el7 (CentOS-7, our most common platform at TRIUMF).

According to https://stackoverflow.com/questions/12530406/is-gcc-4-8-or-earlier-buggy-about-regular-expressions
gcc 4.9.0 is the first one to implement c++11 regular expressions. el7 comes with gcc-4.8.5 and I confirm
that examples of using std::regex_replace() do not compile. I was looking to use std::regex_replace to implement URL rewriting
in the reverse proxy code in mhttpd.

I do not need this feature immediately, but I am surprised that such a thing can happen, thought others should know.

K.O.

About the boost library, that is exactly 
what I did for a project of mine (the 
calibration software for the WAGASCI 
experiment). It turned out not so easy to 
mantain because different Linux distros 
package different versions of boost.

The reason I went down the "c++11 plus 
boost" road is that the official T2K OS 
is CentOS7 as well.

Looking back I think that using c++17 and 
requiring a more recent version of the 
compiler is much easier to maintain than 
the combo c++11 + boost. In CentOS is 
just a matter of installing a recent 
devtool package ...

Another solution might be too repackage 
boost into MIDAS so you have full control 
of the environment.

> > After much discussion, and following 
the MIDAS workshop at TRIUMF, we made the 
decision to use C++11 in MIDAS.
> > 
> > There are many benefits, and only one 
drawback - no c++11 compilers in the 
default OS install on older computers 
(i.e. 
> > RHEL/SL/CentOS before el7). (the same 
applies to our use of cmake).
> >
> 
> It turns out that support for the c++11 
"regex" feature is missing on el7 
(CentOS-7, our most common platform at 
TRIUMF).
> 
> According to 
https://stackoverflow.com/questions/12530
406/is-gcc-4-8-or-earlier-buggy-about-
regular-expressions
> gcc 4.9.0 is the first one to implement 
c++11 regular expressions. el7 comes with 
gcc-4.8.5 and I confirm
> that examples of using 
std::regex_replace() do not compile. I 
was looking to use std::regex_replace to 
implement URL rewriting
> in the reverse proxy code in mhttpd.
> 
> I do not need this feature immediately, 
but I am surprised that such a thing can 
happen, thought others should know.
> 
> K.O.

Just FYI, I am looking at the ROOT web programming component JSROOT and I notice that the RPC mechanism quite different from the JSON-
RPC I implemented for MIDAS.

https://github.com/root-project/jsroot/blob/master/docs/HttpServer.md (explanation of JSROOT RPC and server side machinery)
https://github.com/root-project/jsroot/blob/master/docs/JSROOT.md (explanation of JSROOT javascript library)

Then I looked at the dates:
MIDAS mjsonrpc was done at the end of 2013
JSROOT main development started at the end of 2014.

The web server component in both projects is (almost) the same - vanilla mongoose in mhttpd
and civetweb, a fork of an older version of mongoose, in ROOT/JSROOT.

The web server in both projects is partially multithreaded:
- ROOT THttpServer/TCivetWeb uses multiple threads to handle the network connections and some file access,
but interaction with ROOT is done in the main thread of ROOT. (The main thread must periodically call ProcessRequests()).
- mhttpd uses a single thread to multiplex the network connections (it is a change from old mongoose/civetweb to current mongoose 6.16),
but all requests are farmed to a pool of threads and execute in parallel (unless not thread-safe, i.e. accessing history files).

Both implementations suffer from "head of queue" blocking, a "slow" request i.e. a slow file read, will
delay subsequent quick requests, see https://en.wikipedia.org/wiki/Head-of-line_blocking#In_HTTP
Solution for this problem is to use HTTP/2 when it becomes supported in mongoose/civetweb/apache httpd (in el7).

It will be interesting to see which on of the two systems works better for building "user facing" web pages... especially
hybrid pages that have to pull data both from midas (using mjsonrpc) and from online ROOT analyzers (using jsroot).

K.O.

Hi All,

I was wondering if anyone has attempted to install MIDAS under Scientific Linux 6?  I am planning to install 
Scientific Linux on one of the PCs in our lab to run MIDAS. I would like to know if anyone has been 
successful in getting MIDAS to run under SL6.  Thanks.

Cheng-Ju

Hi Cheng-Ju,

Midas will install and run under SL6. We're presently running SL6.2.
Cheers, PAA

> Hi All,
> 
> I was wondering if anyone has attempted to install MIDAS under Scientific Linux 6?  I am planning to install 
> Scientific Linux on one of the PCs in our lab to run MIDAS. I would like to know if anyone has been 
> successful in getting MIDAS to run under SL6.  Thanks.
> 
> Cheng-Ju

We're using the History Logger MIDAS feature and writing to mySQL tables, but 
in some cases have run into issues installing mySQL on centos7 systems.

Has anyone ever tried running this MIDAS feature with MariaDB rather than 
mySQL?

> We're using the History Logger MIDAS feature and writing to mySQL tables, but 
> in some cases have run into issues installing mySQL on centos7 systems.
> 
> Has anyone ever tried running this MIDAS feature with MariaDB rather than 
> mySQL?

Hi

I've used MIDAS Logger feature with MariaDB and had no issue thus far .

> We're using the History Logger MIDAS feature and writing to mySQL tables, but 
> in some cases have run into issues installing mySQL on centos7 systems.
> 
> Has anyone ever tried running this MIDAS feature with MariaDB rather than 
> mySQL?

The best I can tell, MariaDB is *the* mysql. "the other thing" is an abandonware fork.

I personally at this moment do not run any daq stations with mysql logging, and I accidentally
removed all the 25 versions and flavours of mysql from my home laptop, so if there
is any problems with mysql, please holler, and I will reinstall mysql (ahem, mariadb) and restore
my ability to test midas against it.

As a connected question, is there any need to have postgres support in midas as well?

K.O.

Great news! I got convinced by some colleagues to switch midas to Cmake. After spending about one day, I wrote some initial CMakeLists.txt file and am so excited about the advantages that I regret 
not having done this step much earlier. Here is some information:

- The Cmake and old Makefile systems can co-exist. So the old "make" in the midas root still works as previously.

- To use Cmake, do

midas$ mkdir build
midas$ cd build
midas/build$ cmake ..
midas/build$ make

Depending on your installation, it might be necessary to call "cmake3" instead of "cmake". The configuration requires Cmake 3.0 or later.

- After successful compilation, all programs and libraries are in the "build" directory. We kind of concluded that a system-wide midas installation (like under /usr/local/bin) is not necessary these days, 
as long as you have your MIDASSYS and PATH environment variables defined correctly. Some examples move all files from "build" to "bin"/"lib" under midas, but I'm not sure if we need that.

- Interestingly enough, in my iMac(Late 2015), the old Makefile build takes 19.5s, which the new one take 12s. So apparently some clever dependency checking is done in Cmake.

- The compile options are now handled in the Cmake cache file which is important to remember. Changing option(USE_SSL ON) in CMakeLists.txt just modifies the default value on a fresh install. To 
change the flags between compilations, use the "ccmake .." interface instead. This lets you also switch from Debug to Release mode easily.

- I love how the library handling is done. The code

  find_package(OpenSSL REQUIRED)
  include_directories(${OPENSSL_INCLUDE_DIR})
  target_link_libraries(mhttpd midas ${OPENSSL_LIBRARIES})

is so much simpler than our clumsy conditional compiling we needed in the old Makefile. 

- Cmake is the basis of the CLion IDE which is my favourite development environment now (https://www.jetbrains.com/clion/). So I can work inside the IDE and see the full project, I can do interactive 
debugging etc. and still do a simple 'make' on systems where CLion is not installed. I can only recommend everybody to have a look at CLion. It is free for university teachers and open source 
developers (like I got my free license because of ELOG).

- The CMakeLists.txt is not yet complete. It does not contain cross compilation, since I don't have access to these compilers. 

- The next step will be to add a CMakeLists.txt into each "example" directory and build everything hierarchically. 

- I'm a novice in cmake. If someone of your has more experience (and I'm sure that there are plenty of people out there!), please have a look at my CMakeLists.txt and check if things can be made 
simpler or more elegantly.

- Any comment are as usual welcome.

Have fun,
Stefan

> Great news!

Some additional information.

1) cmake3 is available on all currently supported systems:

- SL6 (el6), CentOS7 (el7): yum install cmake3 (from EPEL) (invoke as "cmake3")
- Ubuntu 18.04 LTS: apt-get install cmake (invoke as "cmake").
- MacOS: install "mac ports", then "port install cmake"
- Windows - we hope to revive windows10 support this summer

> - To use Cmake, do
> 
> midas$ mkdir build
> midas$ cd build
> midas/build$ cmake ..
> midas/build$ make
>
> - After successful compilation, all programs and libraries are in the "build" directory
> 

The old "linux", "darwin", etc subdirectories go away. Makefiles for frontends and analyzers become simplified
and can refer to MIDAS in a standard way:

header files: -I$(MIDASSYS)/include
libraries and object files: -L$(MIDASSYS)/build/lib -lmidas
executables: PATH += $(MIDASSYS)/build/bin

>
> ... cross compilation ...
>

We will review the situation with cross-compilation once the dust settles a little bit on changes
with cmake and with the switch to C++.

Since cross-compilation environments are rarely standardized, I do not expect cmake to be of much help and most
likely we will have a simplified Makefile for cross-building feature-reduced versions of MIDAS - probably only
the pieces needed for running remotely-connected frontends (see "ifdef LOCAL_ROUTINES").

K.O.

> > - After successful compilation, all programs and libraries are in the "build" directory
> > 
> 
> The old "linux", "darwin", etc subdirectories go away. Makefiles for frontends and analyzers become simplified
> and can refer to MIDAS in a standard way:
> 
> header files: -I$(MIDASSYS)/include
> libraries and object files: -L$(MIDASSYS)/build/lib -lmidas
> executables: PATH += $(MIDASSYS)/build/bin

Actually the library and executables go directly into the build directory (without "lib" and "bin"), so we need

header files: -I$(MIDASSYS)/include
libraries and object files: -L$(MIDASSYS)/build -lmidas
executables: PATH += $(MIDASSYS)/build

Or course that can be changed in the Cmake file, but not sure if that would be necessary/useful.

> 
> > > - After successful compilation, all programs and libraries are in the "build" directory
> > > 
> > 
> > The old "linux", "darwin", etc subdirectories go away. Makefiles for frontends and analyzers become simplified
> > and can refer to MIDAS in a standard way:
> > 
> > header files: -I$(MIDASSYS)/include
> > libraries and object files: -L$(MIDASSYS)/build/lib -lmidas
> > executables: PATH += $(MIDASSYS)/build/bin
> 
> Actually the library and executables go directly into the build directory (without "lib" and "bin"), so we need
> 
> header files: -I$(MIDASSYS)/include
> libraries and object files: -L$(MIDASSYS)/build -lmidas
> executables: PATH += $(MIDASSYS)/build
> 
> Or course that can be changed in the Cmake file, but not sure if that would be necessary/useful.

Actually I like the proposed separation between the library and the binaries, so I reworked it again. Now we have

header files: -I$(MIDASSYS)/include
libraries and object files: -L$(MIDASSYS)/lib -lmidas
executables: PATH += $(MIDASSYS)/bin

When issuing a "cmake .." followed by a "make" in the build directory, everything ends up in the build directory. To 
move things to the lib and bin directories, do a "make install". Seems to me like this is the standard way for
many packages so we should follow it.

Furthermore, I followed a proposal from KO to separate the code in the "src" directory between library source code
and programs. I moved all programs now to a separate "progs" directory, and left only code for the midas library in
the "src" directory. New CMakeLists.txt have been written for the "progs" and "utils" directories.

Care has been taken so that even when source files were moved around, their revision history is kept to "git annotate"
still works.

This is quite a change so sorry if this breaks some existing installations, but it will make things much easier in the future.

Stefan

> 1) cmake3 is available on all currently supported systems:
> 
> - SL6 (el6), CentOS7 (el7): yum install cmake3 (from EPEL) (invoke as "cmake3")
> - Ubuntu 18.04 LTS: apt-get install cmake (invoke as "cmake").
> - MacOS: install "mac ports", then "port install cmake"
> - Windows - we hope to revive windows10 support this summer

- el8 (RHEL8): cmake 3.11.something is part of the base system (latest cmake), (invoke as "cmake", the best I can tell).

K.O.

Status update on the cmake conversion:

- we have cmake builds working on all supported systems (el6, el7, ubuntu 18.04 LTS, macos 10.13, 10.14)
- I am happy with the result - for example, include file dependancies work much better now
- we are still fixing a few problems where the cmake build is different from the old make build (mfe.o, mlogger/rmlogger, etc)
- until all of these problems are straightened out, we cannot finalize the instructions for writing experiment makefiles (do we have to use -lmfe or 
we can keep the old mfe.o)

After everything is finalized, I hope to post a short guide for converting experiment makefiles to the new system.

The next release of MIDAS (midas-2019-06 series) will be the first C++ midas and cmake will be the primary build system.

K.O.