Some time ago Ben Smith added test coverage reports using GCC -fprofile-arcs -
ftest-coverage and lcov.

It reports code coverage after running "make test". Reported code coverage is 
surprisingly high for the very little code executed by "make test" - create ODB, 
start a frontend,  start run, stop run, check that mlogger created data files 
and history files.

(Of course coverage can never reach 100%, some obscure branches are unreachable 
without using an automated fuzzer, and some error paths are unreachable without 
also using a system call fault injector).

Simplest way to generate a coverage report:

make clean
make cmake YES_COVERAGE=1
make coverage
open cov_html/index.html

K.O.

> time to choose the next c++!

Ununtu-24.04, MIDAS builds with -std=c++23 without errors or any additional warnings. (but does it work? "make 
test" is ok).

K.O.

I need to look at histograms inside a ROOT file, but all the old ways for doing this no longer work. (in theory I can scp the ROOT file to 
the computer I am sitting in front of, but this assumes I have a working ROOT there. anyhow it is pointless to fight this, all modern 
packages are written to only work on the developer's laptop).

- root new TBrowser starts a web server, tries to open firefox (and fails)
- root --web=off new TBrowser using ssh X11 tunnel no longer works, ROOT X11 graphics refresh is broken
- macos root binary kit is built without X11 support, root --web=off does not work at all
- root7 recommended "rootssh" prints an error message (and fails)

What does work well is JSROOT which we use to look at manalyzer live histograms (through apache and mhttpd web proxies).

So I wrote mjsroot.exe. It opens a ROOT file and starts JSROOT to look at it (plus a bit of dancing around to make it actually work):

mjsroot.exe -R8082 root_output_files/output00371.root

To actually see the histograms:

a) if you sitting in front of the same computer, open http://localhost:8082
b) if you are somewhere else, start an ssh tunnel: ssh daq13 -L8082:localhost:8082, open http://localhost:8082
c) if daq13 is running mhttpd, setup http proxy:
set ODB /webserver/proxy/mjsroot to http://localhost:8082
open https://daq13.triumf.ca/proxy/mjsroot/
also
set ODB /alias/mjsroot to "/proxy/mjsroot/"
reload MIDAS status page, observe "mjsroot" in listed in the left-hand side, open it.

K.O.
 

Hello everyone, 

I just wanted to let you know, that I published a MSL Syntax Highlighting Extension for VSCode.
It is still in a quite early stage, so there might be some missing keywords and edge cases which are not fully handled. So in case you find any issues or have suggestions for improvements, I am happy to implement them. Also I only tested it with a custom theme (One Monokai), so it might look very different with the default theme and other ones.

The extension is called "MSL Syntax Highlighter" and can be found in the extension marketplace in VSCode.  (vscode marketplace: https://marketplace.visualstudio.com/items?itemName=LukasMandok.msl-syntax-highlighter, github repo: https://github.com/LukasMandok/msl-syntax-highlighter)

One additional remark:
- To keep a consitent style with existing themes, one is a bit limited in regard to colors. For this reason a distinction betrween LOOP and IF Blocks is not really possible without writing a custom theme. A workaround would be to add the theming in the custom user settings (explained in the readme).

Dear MIDAS enthusiasts,

We are planning a fifth MIDAS workshop, following on from previous successful 
workshops in 2015, 2017, 2019 and 2023.  The goals of the workshop include:

- Getting updates from MIDAS developers on new features, bug fixes and planned 
changes.
- Getting reports from MIDAS users on how they are using MIDAS and what problems 
they are facing.
- Making plans for future MIDAS changes and improvements

We are planning to have an online workshop on Sept 22-23, 2025 (it will coincide 
with a visit of Stefan to TRIUMF).  We are tentatively planning to have a four 
hour session on each day, with the sessions timed for morning in Vancouver and 
afternoon/evening in Europe.  Sorry, the sessions are likely to again not be well 
timed for our colleagues in Asia.  

We will provide exact times and more details closer to the date.  But I hope 
people can mark the dates in their calendars; we are keen to hear from as much of 
the MIDAS community as possible.  

Best Regards,
Thomas Lindner

I had to change the ODBXX API: https://bitbucket.org/tmidas/midas/commits/273c4e434795453c0c6bceb46bac9a0d2d27db18

The old C API is case-insensitive, meaning db_find_key("name") returns a key "name" or "Name" or "NAME". We can discuss if this is good or bad, but that's how it is since 30 years.

I now realized the the ODBXX API keys is case sensitive, so a o["NAME"] does not return any key "name". Rather, it tries to create a new key which of course fails. I changed therefore
the ODBXX to become case-insensitive like the old C API.

Stefan

it is a very convenient interface! Does it support the ODB links in the path names ? -- thanks, regards, Pasha

I was working on an odb corruption crash inside db_delete_key() and I noticed 
that I did not test db_delete_key() with follow_links set to TRUE. Then I noticed 
that nobody nowhere seems to use db_delete_key() with follow_links set to TRUE. 

Instead of testing it, can I just remove it?

This feature existed since day 1 (1st commit) and it does something unexpected 
compared to filesystem "/bin/rm": the best I can tell, it is removes the link 
*and* whatever the link points to. For people familiar with "/bin/rm", this is 
somewhat unexpected and by my thinking, if nobody ever added such a feature to 
"/bin/rm", it is probably not considered generally useful or desirable. (I would 
think it dangerous, it removes not 1 but 2 files, the 2nd file would be in some 
other directory far away from where we are).

By this thinking, I should remove "follow_links" (actually just make it do thing 
, to reduce the disturbance to other source code). db_delete_key() should work 
similar to /bin/rm aka the unlink() syscall.

K.O.

I would handle this actually like symbolic links are handled under linux. If you delete a symbolic link, the link gets 
detected and NOT the file the link is pointing to.

So I conclude that the "follow links" is a misconception and should be removed.

Stefan

Dear All, 

I'm pretty sure many know this already, however I found this feature by a an accident 
and want to share with those who don't know about it yet - seems very useful. 

- it looks that one can use ROOT scripting with rootcling and call from the 
  interactive ROOT prompt any function defined in midas.h and access ODB seemingly 
  WITHOUT DOING anything special 

- more surprisingly, that also works for odbxx, with one minor exception in handling 
  the 64-bit types - the proof is in attachment. The script test_odbxx.C loaded 
  interactively is Stefan's

 https://bitbucket.org/tmidas/midas/src/develop/examples/odbxx/odbxx_test.cxx 

with one minor change - the line 
 
   o[Int64 Key] = -1LL;

is replaced with

   int64_t x = -1LL;
   o["Int64 Key"] = x;

- apparently the interpeter has its limitations. 

My rootlogon.C file doesn't load any libraries, it only defines the appropriate 
include paths. So it seems that everything works pretty much out of the box. 

One issue has surfaced however. All that worked despite my experiment 
had its name="test_025", while the example specifies experiment="test". 
Is it possible that that only first 4 characters are being tested ? 

-- regards, Pasha

Dear all,

We have setup an indico page for the MIDAS workshop on Sept 22-23.  The page is here

https://indico.psi.ch/event/17580/overview

As I mentioned, we are keen to hear reports from any users or developers; we want to hear  
how MIDAS is working for you and what improvements you would like to see.  If you or your 
experiment would like to give a talk about your MIDAS experiences then please submit an 
abstract through the indico page.  

Also, feel free to also register for the workshop (no fees).  Registration is not 
mandatory, but it would be useful for us to have an idea how many people will connect.

Thanks,
Thomas


> Dear MIDAS enthusiasts,
> 
> We are planning a fifth MIDAS workshop, following on from previous successful 
> workshops in 2015, 2017, 2019 and 2023.  The goals of the workshop include:
> 
> - Getting updates from MIDAS developers on new features, bug fixes and planned 
> changes.
> - Getting reports from MIDAS users on how they are using MIDAS and what problems 
> they are facing.
> - Making plans for future MIDAS changes and improvements
> 
> We are planning to have an online workshop on Sept 22-23, 2025 (it will coincide 
> with a visit of Stefan to TRIUMF).  We are tentatively planning to have a four 
> hour session on each day, with the sessions timed for morning in Vancouver and 
> afternoon/evening in Europe.  Sorry, the sessions are likely to again not be well 
> timed for our colleagues in Asia.  
> 
> We will provide exact times and more details closer to the date.  But I hope 
> people can mark the dates in their calendars; we are keen to hear from as much of 
> the MIDAS community as possible.  
> 
> Best Regards,
> Thomas Lindner

Hello, I'm currently writing a paper about making a dataset publicly available 
from the University of Minnesota and it's a MIDAS dataset.

I'd like to use an image that shows the MIDAS data format (that's been slightly 
modified to fit in the paper) and am wondering (1) if I could get permission to do 
so and (2) what the preferred attribution would be. 

> Hello, I'm currently writing a paper about making a dataset publicly available 
> from the University of Minnesota and it's a MIDAS dataset.
> 
> I'd like to use an image that shows the MIDAS data format (that's been slightly 
> modified to fit in the paper) and am wondering (1) if I could get permission to do 
> so and (2) what the preferred attribution would be. 

Feel free to use whatever you like, the documentation is on the same open source license as midas itself.

Stefan

Today the way the history system is configured has changed. Whenever one adds new equipment 
variables, the logger has to add that variable to the history system. Previously, this happened during 
startup of the logger and at run start. We have now a case in the Mu3e experiment where we have many 
variables and the history configuration takes about 15 seconds, which delays data taking considerably.

After discussion with KO we decided to remove the history re-configuration at run start. This speeds up 
the run start considerably, also for other experiments with many history variables. It means however that 
once you add/remove/rename any equipment variable going into the history system, you have to restart the 
logger for this to become active.

https://bitbucket.org/tmidas/midas/commits/c0a14c2d0166feb6b38c645947f2c5e0bef013d5

Stefan

We had issues with the frontend write cache size and the way it was implemented in the frontend 
framework mfe. Specifically, we had two equipments like in the experiment/examples/frontend.cxx, one 
trigger event and one periodic event. While the trigger event sets the cache size correctly in the 
equipment table, the periodic event (defined afterwards) overwrites the cache size to zero, causing slow 
data taking. 

The underlying problem is that one event buffer (usually "SYSTEM") can only have one cache size for all 
events in one frontend. To avoid mishaps like that, I remove the write cache size from the equipment table 
and instead defined a function which explicitly sets the cache size for a buffer. In your frontend_init() you 
can now call

  set_cache_size("SYSTEM", 10000000);

to set the cache size of the SYSTEM buffer. Note that 10 MiB is the minimum cache size. Especially for 
smaller events, this dramatically can speed up your maximal DAQ rate.

Full commit is here:

  https://bitbucket.org/tmidas/midas/commits/24fbf2c02037ae5f7db74d0cadab23cd4bfe3b13

Note that this only affects frontends using the mfe framework, NOT for those using the tmfe framework.

Stefan

Dear all,

A reminder we will have our MIDAS workshop starting two weeks from today (Sept 22-23).  The 
meeting will be in the morning in Vancouver, evening in Europe.  A detailed schedule is available 
here

https://indico.psi.ch/event/17580/timetable/#20250922.detailed

The zoom link is available from the indico overview page. The schedule will allow for a fair bit 
of discussion time, so it is unlikely that talks will start exactly on time.

Looking forward to seeing people there.

Thomas

> Dear all,
> 
> We have setup an indico page for the MIDAS workshop on Sept 22-23.  The page is here
> 
> https://indico.psi.ch/event/17580/overview
> 
> As I mentioned, we are keen to hear reports from any users or developers; we want to hear  
> how MIDAS is working for you and what improvements you would like to see.  If you or your 
> experiment would like to give a talk about your MIDAS experiences then please submit an 
> abstract through the indico page.  
> 
> Also, feel free to also register for the workshop (no fees).  Registration is not 
> mandatory, but it would be useful for us to have an idea how many people will connect.
> 
> Thanks,
> Thomas
> 
> 
> > Dear MIDAS enthusiasts,
> > 
> > We are planning a fifth MIDAS workshop, following on from previous successful 
> > workshops in 2015, 2017, 2019 and 2023.  The goals of the workshop include:
> > 
> > - Getting updates from MIDAS developers on new features, bug fixes and planned 
> > changes.
> > - Getting reports from MIDAS users on how they are using MIDAS and what problems 
> > they are facing.
> > - Making plans for future MIDAS changes and improvements
> > 
> > We are planning to have an online workshop on Sept 22-23, 2025 (it will coincide 
> > with a visit of Stefan to TRIUMF).  We are tentatively planning to have a four 
> > hour session on each day, with the sessions timed for morning in Vancouver and 
> > afternoon/evening in Europe.  Sorry, the sessions are likely to again not be well 
> > timed for our colleagues in Asia.  
> > 
> > We will provide exact times and more details closer to the date.  But I hope 
> > people can mark the dates in their calendars; we are keen to hear from as much of 
> > the MIDAS community as possible.  
> > 
> > Best Regards,
> > Thomas Lindner

Dear all,

A final reminder that the MIDAS workshop will be occurring next Monday and Tuesday (at least in Europe 
and the Americas).  See the full schedule is posted here

https://indico.psi.ch/event/17580/timetable/#20250922.detailed

The zoom link is on the indico overview page.

Note that for those wanting to attend the workshop in person we will be meeting in ISAC2 room 223 at 
TRIUMF.

See you there.
Thomas

> Dear all,
> 
> A reminder we will have our MIDAS workshop starting two weeks from today (Sept 22-23).  The 
> meeting will be in the morning in Vancouver, evening in Europe.  A detailed schedule is available 
> here
> 
> https://indico.psi.ch/event/17580/timetable/#20250922.detailed
> 
> The zoom link is available from the indico overview page. The schedule will allow for a fair bit 
> of discussion time, so it is unlikely that talks will start exactly on time.
> 
> Looking forward to seeing people there.
> 
> Thomas
> 
> > Dear all,
> > 
> > We have setup an indico page for the MIDAS workshop on Sept 22-23.  The page is here
> > 
> > https://indico.psi.ch/event/17580/overview
> > 
> > As I mentioned, we are keen to hear reports from any users or developers; we want to hear  
> > how MIDAS is working for you and what improvements you would like to see.  If you or your 
> > experiment would like to give a talk about your MIDAS experiences then please submit an 
> > abstract through the indico page.  
> > 
> > Also, feel free to also register for the workshop (no fees).  Registration is not 
> > mandatory, but it would be useful for us to have an idea how many people will connect.
> > 
> > Thanks,
> > Thomas
> > 
> > 
> > > Dear MIDAS enthusiasts,
> > > 
> > > We are planning a fifth MIDAS workshop, following on from previous successful 
> > > workshops in 2015, 2017, 2019 and 2023.  The goals of the workshop include:
> > > 
> > > - Getting updates from MIDAS developers on new features, bug fixes and planned 
> > > changes.
> > > - Getting reports from MIDAS users on how they are using MIDAS and what problems 
> > > they are facing.
> > > - Making plans for future MIDAS changes and improvements
> > > 
> > > We are planning to have an online workshop on Sept 22-23, 2025 (it will coincide 
> > > with a visit of Stefan to TRIUMF).  We are tentatively planning to have a four 
> > > hour session on each day, with the sessions timed for morning in Vancouver and 
> > > afternoon/evening in Europe.  Sorry, the sessions are likely to again not be well 
> > > timed for our colleagues in Asia.  
> > > 
> > > We will provide exact times and more details closer to the date.  But I hope 
> > > people can mark the dates in their calendars; we are keen to hear from as much of 
> > > the MIDAS community as possible.  
> > > 
> > > Best Regards,
> > > Thomas Lindner

Dear all,

The original zoom link on the indico page was not correct.  We have created a new zoom link and updated the 
Indico overview page.  Please refresh the Indico overview page to get the correct link.

Thomas

> Dear all,
> 
> A final reminder that the MIDAS workshop will be occurring next Monday and Tuesday (at least in Europe 
> and the Americas).  See the full schedule is posted here
> 
> https://indico.psi.ch/event/17580/timetable/#20250922.detailed
> 
> The zoom link is on the indico overview page.
> 
> Note that for those wanting to attend the workshop in person we will be meeting in ISAC2 room 223 at 
> TRIUMF.
> 
> See you there.
> Thomas
> 
> > Dear all,
> > 
> > A reminder we will have our MIDAS workshop starting two weeks from today (Sept 22-23).  The 
> > meeting will be in the morning in Vancouver, evening in Europe.  A detailed schedule is available 
> > here
> > 
> > https://indico.psi.ch/event/17580/timetable/#20250922.detailed
> > 
> > The zoom link is available from the indico overview page. The schedule will allow for a fair bit 
> > of discussion time, so it is unlikely that talks will start exactly on time.
> > 
> > Looking forward to seeing people there.
> > 
> > Thomas
> > 
> > > Dear all,
> > > 
> > > We have setup an indico page for the MIDAS workshop on Sept 22-23.  The page is here
> > > 
> > > https://indico.psi.ch/event/17580/overview
> > > 
> > > As I mentioned, we are keen to hear reports from any users or developers; we want to hear  
> > > how MIDAS is working for you and what improvements you would like to see.  If you or your 
> > > experiment would like to give a talk about your MIDAS experiences then please submit an 
> > > abstract through the indico page.  
> > > 
> > > Also, feel free to also register for the workshop (no fees).  Registration is not 
> > > mandatory, but it would be useful for us to have an idea how many people will connect.
> > > 
> > > Thanks,
> > > Thomas
> > > 
> > > 
> > > > Dear MIDAS enthusiasts,
> > > > 
> > > > We are planning a fifth MIDAS workshop, following on from previous successful 
> > > > workshops in 2015, 2017, 2019 and 2023.  The goals of the workshop include:
> > > > 
> > > > - Getting updates from MIDAS developers on new features, bug fixes and planned 
> > > > changes.
> > > > - Getting reports from MIDAS users on how they are using MIDAS and what problems 
> > > > they are facing.
> > > > - Making plans for future MIDAS changes and improvements
> > > > 
> > > > We are planning to have an online workshop on Sept 22-23, 2025 (it will coincide 
> > > > with a visit of Stefan to TRIUMF).  We are tentatively planning to have a four 
> > > > hour session on each day, with the sessions timed for morning in Vancouver and 
> > > > afternoon/evening in Europe.  Sorry, the sessions are likely to again not be well 
> > > > timed for our colleagues in Asia.  
> > > > 
> > > > We will provide exact times and more details closer to the date.  But I hope 
> > > > people can mark the dates in their calendars; we are keen to hear from as much of 
> > > > the MIDAS community as possible.  
> > > > 
> > > > Best Regards,
> > > > Thomas Lindner

As part of discussions with Stefan during the MIDAS workshop,
an update to supported versions of c++ on different platforms.

- el6/SL6 is gone, gcc 4.4.7, no c++11
- el7/CentOS-7 is out the door, gcc 4.8.5, full c++11, no c++14, no c++17
- el8 was a still born child of RedHat
- el9  - gcc 11.5 with 12, 13, and 14 available, incomplete c++26, c++23, experimental c++20, default c++17
- el10 - gcc 14.2, incomplete c++26, c++23, experimental c++20, default c++17

- U-18 - gcc  7.5, experimental, incomplete c++17
- U-20 - gcc  9.4 default, 10.5 available, experimental, incomplete c++17
- U-22 - gcc 11.4 default, 12.3 available, default c++17
- U-24 - gcc 13.3 default, 14.2 available, default c++17

- D-11 - gcc 10.2, experimental c++17
- D-12 - gcc 12.2, default c++17

- MacOS 15.2 - llvm/clang 16, default c++17
- MacOS 15.7 - llvm/clang 17, default c++17

Taken from C++ level support:

GCC C++ support: https://gcc.gnu.org/projects/cxx-status.html
LLVM clang c++ support: https://clang.llvm.org/cxx_status.html
GLIBC c++ support: https://gcc.gnu.org/onlinedocs/libstdc++/manual/status.html

This suggests that c++17 is available on all current platforms.

K.O.

P.S. ROOT moved to C++17 as of release 6.30 on November 6, 2023.