> The following code found in mhttpd does not work on MacOS (BSD UNIX). ...

Because of this problem, on MacOS, run transitions can get stuck forever - most timeouts do not work. (Specifically, recv_string() never times out)

K.O.

It turns out that because of differences between select() syscall implementation between UNIX (MacOS, 
maybe BSD) and Linux,  network socket timeouts do not work.

This affects timeouts during run transitions (transition calls to dead clients do not timeout), maybe other 
places.

I am looking into fixing this. The main difficulty is with UNIX select() not updating the timeout parameter 
when it is interrupted by the MIDAS watchdog alarm signal. Linux select() subtracts the elapsed time from 
the timeout value and this code from system.c works correctly: while (1) { status = select(..., &timeout); if 
(status==0) break; } (value of timeout becomes smaller each time), while on MacOS it loops forever (value 
of timeout does not change).
K.O.

I recently upgraded my macbook to MacOS 10.11.  The compilation of MIDAS failed after the upgrade, 
complaining about  

gcc  -c -g -O2 -Wall <snip> src/mongoose.c
src/mongoose.c:322:10: fatal error: 'openssl/ssl.h' file not found

It seems that MacOS has now fully removed openssl header files (they were deprecated for a while).  There 
seems to be some notes on that here

http://lists.apple.com/archives/macnetworkprog/2015/Jun/msg00025.html

Konstantin suggested installing open-source builds of openssl using MacPorts.  I did that and MIDAS 
compiled fine.  I documented the procedure here:

https://midas.triumf.ca/MidasWiki/index.php/Installation/Compilation_problems#MacOS_10.11_.28El_Capitan.2
9_openssl_errors

> I recently upgraded my macbook to MacOS 10.11.  The compilation of MIDAS failed after the upgrade, 
> complaining about  
> 
> gcc  -c -g -O2 -Wall <snip> src/mongoose.c
> src/mongoose.c:322:10: fatal error: 'openssl/ssl.h' file not found
> 
> It seems that MacOS has now fully removed openssl header files (they were deprecated for a while).  There 
> seems to be some notes on that here
> 
> http://lists.apple.com/archives/macnetworkprog/2015/Jun/msg00025.html
> 
> Konstantin suggested installing open-source builds of openssl using MacPorts.  I did that and MIDAS 
> compiled fine.  I documented the procedure here:
> 
> https://midas.triumf.ca/MidasWiki/index.php/Installation/Compilation_problems#MacOS_10.11_.28El_Capitan.2
> 9_openssl_errors

The MIDAS Wiki page points to https://guide.macports.org/  which covers OSX up to 10.9. Installers for 10.10 and the current 10.11 
(El Captain) can be found here: https://www.macports.org/install.php

Stefan

Dear all,
      I'm trying to use the logging on a mysql DB. Following the instructions on 
the Wiki, I recompiled MIDAS after installing mysql, and cmake with NEED_MYSQL=1 
can find it:

-- MIDAS: Found MySQL version 8.0.23

Then, I compiled my frontend (cmake with no options + make) and run it, but in the 
ODB I cannot find the tree for mySQL. I have only:

Logger/Runlog/ASCII

while I would expect also:

Logger/Runlog/SQL

What could be missing? Maybe should I add something in the CMakeList file or run 
cmake with some option?

Thank you,
      Francesco

I solved this, it was a failed "make clean" before recompiling. Now it works.

Sorry for the noise.

Francesco

> Dear all,
>       I'm trying to use the logging on a mysql DB. Following the instructions on 
> the Wiki, I recompiled MIDAS after installing mysql, and cmake with NEED_MYSQL=1 
> can find it:
> 
> -- MIDAS: Found MySQL version 8.0.23
> 
> Then, I compiled my frontend (cmake with no options + make) and run it, but in the 
> ODB I cannot find the tree for mySQL. I have only:
> 
> Logger/Runlog/ASCII
> 
> while I would expect also:
> 
> Logger/Runlog/SQL
> 
> What could be missing? Maybe should I add something in the CMakeList file or run 
> cmake with some option?
> 
> Thank you,
>       Francesco

Hello everyone, 

I just wanted to let you know, that I published a MSL Syntax Highlighting Extension for VSCode.
It is still in a quite early stage, so there might be some missing keywords and edge cases which are not fully handled. So in case you find any issues or have suggestions for improvements, I am happy to implement them. Also I only tested it with a custom theme (One Monokai), so it might look very different with the default theme and other ones.

The extension is called "MSL Syntax Highlighter" and can be found in the extension marketplace in VSCode.  (vscode marketplace: https://marketplace.visualstudio.com/items?itemName=LukasMandok.msl-syntax-highlighter, github repo: https://github.com/LukasMandok/msl-syntax-highlighter)

One additional remark:
- To keep a consitent style with existing themes, one is a bit limited in regard to colors. For this reason a distinction betrween LOOP and IF Blocks is not really possible without writing a custom theme. A workaround would be to add the theming in the custom user settings (explained in the readme).

Hi,
 has anyone tried runnning midas frontend on a Linux running 
on a Motorola MVME167 motorola embedded CPU?
I have seen people running Linux on a MV167 
(http://www.sleepie.demon.co.uk/linuxvme/)
so in principle this can be done.

The reason I am asking is that we have a lot of them in house 
and we would like to avoid paying for VxWorks
(I have succesfully run Midas on a mvme167/VxWorks node)

Or maybe one has come up with a much better solution 
[short of dumping mv167 into a sewer :)]

Piotr

Dear MIDAS enthusiasts,

We are planning a fifth MIDAS workshop, following on from previous successful 
workshops in 2015, 2017, 2019 and 2023.  The goals of the workshop include:

- Getting updates from MIDAS developers on new features, bug fixes and planned 
changes.
- Getting reports from MIDAS users on how they are using MIDAS and what problems 
they are facing.
- Making plans for future MIDAS changes and improvements

We are planning to have an online workshop on Sept 22-23, 2025 (it will coincide 
with a visit of Stefan to TRIUMF).  We are tentatively planning to have a four 
hour session on each day, with the sessions timed for morning in Vancouver and 
afternoon/evening in Europe.  Sorry, the sessions are likely to again not be well 
timed for our colleagues in Asia.  

We will provide exact times and more details closer to the date.  But I hope 
people can mark the dates in their calendars; we are keen to hear from as much of 
the MIDAS community as possible.  

Best Regards,
Thomas Lindner

Dear all,

We have setup an indico page for the MIDAS workshop on Sept 22-23.  The page is here

https://indico.psi.ch/event/17580/overview

As I mentioned, we are keen to hear reports from any users or developers; we want to hear  
how MIDAS is working for you and what improvements you would like to see.  If you or your 
experiment would like to give a talk about your MIDAS experiences then please submit an 
abstract through the indico page.  

Also, feel free to also register for the workshop (no fees).  Registration is not 
mandatory, but it would be useful for us to have an idea how many people will connect.

Thanks,
Thomas


> Dear MIDAS enthusiasts,
> 
> We are planning a fifth MIDAS workshop, following on from previous successful 
> workshops in 2015, 2017, 2019 and 2023.  The goals of the workshop include:
> 
> - Getting updates from MIDAS developers on new features, bug fixes and planned 
> changes.
> - Getting reports from MIDAS users on how they are using MIDAS and what problems 
> they are facing.
> - Making plans for future MIDAS changes and improvements
> 
> We are planning to have an online workshop on Sept 22-23, 2025 (it will coincide 
> with a visit of Stefan to TRIUMF).  We are tentatively planning to have a four 
> hour session on each day, with the sessions timed for morning in Vancouver and 
> afternoon/evening in Europe.  Sorry, the sessions are likely to again not be well 
> timed for our colleagues in Asia.  
> 
> We will provide exact times and more details closer to the date.  But I hope 
> people can mark the dates in their calendars; we are keen to hear from as much of 
> the MIDAS community as possible.  
> 
> Best Regards,
> Thomas Lindner

Dear all,

A reminder we will have our MIDAS workshop starting two weeks from today (Sept 22-23).  The 
meeting will be in the morning in Vancouver, evening in Europe.  A detailed schedule is available 
here

https://indico.psi.ch/event/17580/timetable/#20250922.detailed

The zoom link is available from the indico overview page. The schedule will allow for a fair bit 
of discussion time, so it is unlikely that talks will start exactly on time.

Looking forward to seeing people there.

Thomas

> Dear all,
> 
> We have setup an indico page for the MIDAS workshop on Sept 22-23.  The page is here
> 
> https://indico.psi.ch/event/17580/overview
> 
> As I mentioned, we are keen to hear reports from any users or developers; we want to hear  
> how MIDAS is working for you and what improvements you would like to see.  If you or your 
> experiment would like to give a talk about your MIDAS experiences then please submit an 
> abstract through the indico page.  
> 
> Also, feel free to also register for the workshop (no fees).  Registration is not 
> mandatory, but it would be useful for us to have an idea how many people will connect.
> 
> Thanks,
> Thomas
> 
> 
> > Dear MIDAS enthusiasts,
> > 
> > We are planning a fifth MIDAS workshop, following on from previous successful 
> > workshops in 2015, 2017, 2019 and 2023.  The goals of the workshop include:
> > 
> > - Getting updates from MIDAS developers on new features, bug fixes and planned 
> > changes.
> > - Getting reports from MIDAS users on how they are using MIDAS and what problems 
> > they are facing.
> > - Making plans for future MIDAS changes and improvements
> > 
> > We are planning to have an online workshop on Sept 22-23, 2025 (it will coincide 
> > with a visit of Stefan to TRIUMF).  We are tentatively planning to have a four 
> > hour session on each day, with the sessions timed for morning in Vancouver and 
> > afternoon/evening in Europe.  Sorry, the sessions are likely to again not be well 
> > timed for our colleagues in Asia.  
> > 
> > We will provide exact times and more details closer to the date.  But I hope 
> > people can mark the dates in their calendars; we are keen to hear from as much of 
> > the MIDAS community as possible.  
> > 
> > Best Regards,
> > Thomas Lindner

Dear all,

A final reminder that the MIDAS workshop will be occurring next Monday and Tuesday (at least in Europe 
and the Americas).  See the full schedule is posted here

https://indico.psi.ch/event/17580/timetable/#20250922.detailed

The zoom link is on the indico overview page.

Note that for those wanting to attend the workshop in person we will be meeting in ISAC2 room 223 at 
TRIUMF.

See you there.
Thomas

> Dear all,
> 
> A reminder we will have our MIDAS workshop starting two weeks from today (Sept 22-23).  The 
> meeting will be in the morning in Vancouver, evening in Europe.  A detailed schedule is available 
> here
> 
> https://indico.psi.ch/event/17580/timetable/#20250922.detailed
> 
> The zoom link is available from the indico overview page. The schedule will allow for a fair bit 
> of discussion time, so it is unlikely that talks will start exactly on time.
> 
> Looking forward to seeing people there.
> 
> Thomas
> 
> > Dear all,
> > 
> > We have setup an indico page for the MIDAS workshop on Sept 22-23.  The page is here
> > 
> > https://indico.psi.ch/event/17580/overview
> > 
> > As I mentioned, we are keen to hear reports from any users or developers; we want to hear  
> > how MIDAS is working for you and what improvements you would like to see.  If you or your 
> > experiment would like to give a talk about your MIDAS experiences then please submit an 
> > abstract through the indico page.  
> > 
> > Also, feel free to also register for the workshop (no fees).  Registration is not 
> > mandatory, but it would be useful for us to have an idea how many people will connect.
> > 
> > Thanks,
> > Thomas
> > 
> > 
> > > Dear MIDAS enthusiasts,
> > > 
> > > We are planning a fifth MIDAS workshop, following on from previous successful 
> > > workshops in 2015, 2017, 2019 and 2023.  The goals of the workshop include:
> > > 
> > > - Getting updates from MIDAS developers on new features, bug fixes and planned 
> > > changes.
> > > - Getting reports from MIDAS users on how they are using MIDAS and what problems 
> > > they are facing.
> > > - Making plans for future MIDAS changes and improvements
> > > 
> > > We are planning to have an online workshop on Sept 22-23, 2025 (it will coincide 
> > > with a visit of Stefan to TRIUMF).  We are tentatively planning to have a four 
> > > hour session on each day, with the sessions timed for morning in Vancouver and 
> > > afternoon/evening in Europe.  Sorry, the sessions are likely to again not be well 
> > > timed for our colleagues in Asia.  
> > > 
> > > We will provide exact times and more details closer to the date.  But I hope 
> > > people can mark the dates in their calendars; we are keen to hear from as much of 
> > > the MIDAS community as possible.  
> > > 
> > > Best Regards,
> > > Thomas Lindner

Dear all,

The original zoom link on the indico page was not correct.  We have created a new zoom link and updated the 
Indico overview page.  Please refresh the Indico overview page to get the correct link.

Thomas

> Dear all,
> 
> A final reminder that the MIDAS workshop will be occurring next Monday and Tuesday (at least in Europe 
> and the Americas).  See the full schedule is posted here
> 
> https://indico.psi.ch/event/17580/timetable/#20250922.detailed
> 
> The zoom link is on the indico overview page.
> 
> Note that for those wanting to attend the workshop in person we will be meeting in ISAC2 room 223 at 
> TRIUMF.
> 
> See you there.
> Thomas
> 
> > Dear all,
> > 
> > A reminder we will have our MIDAS workshop starting two weeks from today (Sept 22-23).  The 
> > meeting will be in the morning in Vancouver, evening in Europe.  A detailed schedule is available 
> > here
> > 
> > https://indico.psi.ch/event/17580/timetable/#20250922.detailed
> > 
> > The zoom link is available from the indico overview page. The schedule will allow for a fair bit 
> > of discussion time, so it is unlikely that talks will start exactly on time.
> > 
> > Looking forward to seeing people there.
> > 
> > Thomas
> > 
> > > Dear all,
> > > 
> > > We have setup an indico page for the MIDAS workshop on Sept 22-23.  The page is here
> > > 
> > > https://indico.psi.ch/event/17580/overview
> > > 
> > > As I mentioned, we are keen to hear reports from any users or developers; we want to hear  
> > > how MIDAS is working for you and what improvements you would like to see.  If you or your 
> > > experiment would like to give a talk about your MIDAS experiences then please submit an 
> > > abstract through the indico page.  
> > > 
> > > Also, feel free to also register for the workshop (no fees).  Registration is not 
> > > mandatory, but it would be useful for us to have an idea how many people will connect.
> > > 
> > > Thanks,
> > > Thomas
> > > 
> > > 
> > > > Dear MIDAS enthusiasts,
> > > > 
> > > > We are planning a fifth MIDAS workshop, following on from previous successful 
> > > > workshops in 2015, 2017, 2019 and 2023.  The goals of the workshop include:
> > > > 
> > > > - Getting updates from MIDAS developers on new features, bug fixes and planned 
> > > > changes.
> > > > - Getting reports from MIDAS users on how they are using MIDAS and what problems 
> > > > they are facing.
> > > > - Making plans for future MIDAS changes and improvements
> > > > 
> > > > We are planning to have an online workshop on Sept 22-23, 2025 (it will coincide 
> > > > with a visit of Stefan to TRIUMF).  We are tentatively planning to have a four 
> > > > hour session on each day, with the sessions timed for morning in Vancouver and 
> > > > afternoon/evening in Europe.  Sorry, the sessions are likely to again not be well 
> > > > timed for our colleagues in Asia.  
> > > > 
> > > > We will provide exact times and more details closer to the date.  But I hope 
> > > > people can mark the dates in their calendars; we are keen to hear from as much of 
> > > > the MIDAS community as possible.  
> > > > 
> > > > Best Regards,
> > > > Thomas Lindner

After much discussion, and following the MIDAS workshop at TRIUMF, we made the decision to use C++11 in MIDAS.

There are many benefits, and only one drawback - no c++11 compilers in the default OS install on older computers (i.e. 
RHEL/SL/CentOS before el7). (the same applies to our use of cmake).

Specifically for el6, the solution is to use c++11 compatible gcc-8 from devtoolset-8, see 
https://midas.triumf.ca/elog/Midas/1649

The c++11 features we most welcome - initialization of class members at declaration time (no more forgetting to add initialization to 
each and every constructor), c++ threads and mutexes, lambdas and "auto".

K.O.

> After much discussion, and following the MIDAS workshop at TRIUMF, we made the decision to use C++11 in MIDAS.
> 
> There are many benefits, and only one drawback - no c++11 compilers in the default OS install on older computers (i.e. 
> RHEL/SL/CentOS before el7). (the same applies to our use of cmake).
>

It turns out that support for the c++11 "regex" feature is missing on el7 (CentOS-7, our most common platform at TRIUMF).

According to https://stackoverflow.com/questions/12530406/is-gcc-4-8-or-earlier-buggy-about-regular-expressions
gcc 4.9.0 is the first one to implement c++11 regular expressions. el7 comes with gcc-4.8.5 and I confirm
that examples of using std::regex_replace() do not compile. I was looking to use std::regex_replace to implement URL rewriting
in the reverse proxy code in mhttpd.

I do not need this feature immediately, but I am surprised that such a thing can happen, thought others should know.

K.O.

About the boost library, that is exactly 
what I did for a project of mine (the 
calibration software for the WAGASCI 
experiment). It turned out not so easy to 
mantain because different Linux distros 
package different versions of boost.

The reason I went down the "c++11 plus 
boost" road is that the official T2K OS 
is CentOS7 as well.

Looking back I think that using c++17 and 
requiring a more recent version of the 
compiler is much easier to maintain than 
the combo c++11 + boost. In CentOS is 
just a matter of installing a recent 
devtool package ...

Another solution might be too repackage 
boost into MIDAS so you have full control 
of the environment.

> > After much discussion, and following 
the MIDAS workshop at TRIUMF, we made the 
decision to use C++11 in MIDAS.
> > 
> > There are many benefits, and only one 
drawback - no c++11 compilers in the 
default OS install on older computers 
(i.e. 
> > RHEL/SL/CentOS before el7). (the same 
applies to our use of cmake).
> >
> 
> It turns out that support for the c++11 
"regex" feature is missing on el7 
(CentOS-7, our most common platform at 
TRIUMF).
> 
> According to 
https://stackoverflow.com/questions/12530
406/is-gcc-4-8-or-earlier-buggy-about-
regular-expressions
> gcc 4.9.0 is the first one to implement 
c++11 regular expressions. el7 comes with 
gcc-4.8.5 and I confirm
> that examples of using 
std::regex_replace() do not compile. I 
was looking to use std::regex_replace to 
implement URL rewriting
> in the reverse proxy code in mhttpd.
> 
> I do not need this feature immediately, 
but I am surprised that such a thing can 
happen, thought others should know.
> 
> K.O.

Just FYI, I am looking at the ROOT web programming component JSROOT and I notice that the RPC mechanism quite different from the JSON-
RPC I implemented for MIDAS.

https://github.com/root-project/jsroot/blob/master/docs/HttpServer.md (explanation of JSROOT RPC and server side machinery)
https://github.com/root-project/jsroot/blob/master/docs/JSROOT.md (explanation of JSROOT javascript library)

Then I looked at the dates:
MIDAS mjsonrpc was done at the end of 2013
JSROOT main development started at the end of 2014.

The web server component in both projects is (almost) the same - vanilla mongoose in mhttpd
and civetweb, a fork of an older version of mongoose, in ROOT/JSROOT.

The web server in both projects is partially multithreaded:
- ROOT THttpServer/TCivetWeb uses multiple threads to handle the network connections and some file access,
but interaction with ROOT is done in the main thread of ROOT. (The main thread must periodically call ProcessRequests()).
- mhttpd uses a single thread to multiplex the network connections (it is a change from old mongoose/civetweb to current mongoose 6.16),
but all requests are farmed to a pool of threads and execute in parallel (unless not thread-safe, i.e. accessing history files).

Both implementations suffer from "head of queue" blocking, a "slow" request i.e. a slow file read, will
delay subsequent quick requests, see https://en.wikipedia.org/wiki/Head-of-line_blocking#In_HTTP
Solution for this problem is to use HTTP/2 when it becomes supported in mongoose/civetweb/apache httpd (in el7).

It will be interesting to see which on of the two systems works better for building "user facing" web pages... especially
hybrid pages that have to pull data both from midas (using mjsonrpc) and from online ROOT analyzers (using jsroot).

K.O.

Hi All,

I was wondering if anyone has attempted to install MIDAS under Scientific Linux 6?  I am planning to install 
Scientific Linux on one of the PCs in our lab to run MIDAS. I would like to know if anyone has been 
successful in getting MIDAS to run under SL6.  Thanks.

Cheng-Ju

Hi Cheng-Ju,

Midas will install and run under SL6. We're presently running SL6.2.
Cheers, PAA

> Hi All,
> 
> I was wondering if anyone has attempted to install MIDAS under Scientific Linux 6?  I am planning to install 
> Scientific Linux on one of the PCs in our lab to run MIDAS. I would like to know if anyone has been 
> successful in getting MIDAS to run under SL6.  Thanks.
> 
> Cheng-Ju

We're using the History Logger MIDAS feature and writing to mySQL tables, but 
in some cases have run into issues installing mySQL on centos7 systems.

Has anyone ever tried running this MIDAS feature with MariaDB rather than 
mySQL?