Hello, 

This is the error message I got:

[RP Streaming Frontend,ERROR] [midas.cxx:17806:cm_write_event_to_odb,ERROR] 
cannot create key for bank "DATA" with tid 24 in ODB, db_create_key() status 309
read_periodic_event: No data in ring buffer or error occurred

[RP Streaming Frontend,ERROR] [odb.cxx:3373:db_create_key,ERROR] invalid key type 
24 to create 'DATA' in '/Equipment/Periodic/Variables'

[RP Streaming Frontend,ERROR] [midas.cxx:17806:cm_write_event_to_odb,ERROR] 
cannot create key for bank "DATA" with tid 24 in ODB, db_create_key() status 309



I just need more information on what the error message means. Which data type 
refers to tid 24 and what does status 309 indicate?? 

There is definitely data in the ring buffer but I keep on getting this error.

Thank you!

M.G 

> [RP Streaming Frontend,ERROR] [midas.cxx:17806:cm_write_event_to_odb,ERROR] 
> cannot create key for bank "DATA" with tid 24 in ODB, db_create_key() status 309
> 
> 
> 
> I just need more information on what the error message means. Which data type 
> refers to tid 24 and what does status 309 indicate?? 

A tid (type identification) of 24 does actually not exist. See midas.h:327, so this tells
me that your bank header got corrupted. Somewhere you write over your data.

Stefan

A new slow control mini-framework has been developed for MIDAS and been successfully tested in the Mu3e experiment. It 
might be suited for other experiments as well.

Background

Since the late 90’s we have the three-tier bases slow control framework in MIDAS with class drivers, device drivers and bus 
drivers. While it was used successfully since many years, it is complicated to understand and limited in its flexibility. If we 
have a HV device with a demand value, a measured voltage and a current it’s fine, but if we want to control more things like 
trip voltage, temperature and status readout etc. it soon hits its limits. With the development of the new odbxx API 
(https://daq00.triumf.ca/MidasWiki/index.php/Odbxx) there is now an opportunity to make everything much simpler.

Design principles

Instead of a three-tier system, the new “mdev” framework (“m”idas “dev”ices) uses a simple base class which is attached to 
a certain MIDAS equipment. It implements five simple functions:

- odb_setup() to setup /Equipment/<name>/Settings and /Equipment/<name>/Variables to its desired structure

- init() to initialize the slow control device

- exit() to close the connection to the device

- loop() which is called periodically to read the device

- read_event() which returns a MIDAS event going to the data stream

A device driver inherits from this base class and implements the functions. A simple example can be found in 

  midas/drivers/mdev/mdev_mscb.[h,cxx]

for the MSCB field bus system used at TRIUMF and PSI. It basically boils down to two calls:

Init:
   m_variables.connect(“/Equipment/<name>/Variables”);
   m_variables[“Output”].watch(midas::odb &o) {
      m_mscb[“HV”] = o[0]; // transfer value from ODB to MSCB device
   }

Reading a value in the loop function:
   m_variables[“Input”][0] = m_mscb[“HVMeas”];

The member variable m_variables is a midas::odb variable attached to the “Input” and “Output” variables in the ODB. The 
watch() functions executes the lambda function whenever the “Output” in the ODB changes. It then simply transfers the new 
value to the device. The reading of measured values just work in the other direction from the device to the ODB.

If you look at the mdev_mscb.cxx code, you see of course some more things like connecting to the MSCB device with proper 
error handling, looping over several devices and variables, setting up the “Setting” directory in the ODB to define labels for 
all variables. In addition we have a mirror for output variables, so that new values are only sent to the device if they differ 
from the previous variable (needed to reduce some communication traffic). 

The midas/drivers/mdev directory contains also an example frontend in the mfe.cxx framework, but this is no a requirement. 
The mdev framework can also be used in the tmfe framework and others as well. Please note how compact the frontend 
code now looks.

User interface

Since the beginning, MIDAS allows access to the the slow control devices through the “equipment” page (on the main status 
page, click on one equipment). A few more options can control now the behavior of this page, allowing quite some flexibility 
without having to write a dedicated custom page (which of course can still be done). Attached is an example from Mu3e where 
the details of the equipment display are controlled through some options in the setting subdirectory as described in 
https://daq00.triumf.ca/MidasWiki/index.php//Equipment_ODB_tree (especially the “grid display”, “Editable” and “Format” 
flags).

Conclusions

The new “mdev” framework offers a compact and effective way to communicate from MIDAS to slow control devices. Since 
all interface code is now not “hidden” any more in system class and device drivers, the user has much higher flexibility in 
controlling different devices. If a device has a new parameter, the user can add a single line of code to connect this 
parameter to an ODB entry.

The framework is very simple and misses some features of the old system. Ramping of HV voltages and current trips are not 
available in the framework (like with the old HV class driver), but modern devices usually implement this in hardware which 
is much better. The new framework is not multi-threaded, but modern devices are these day much faster than in the ‘90s. 
Since the ODB is thread save, nothing prevents us from putting a device readout into its own thread in the frontend.

We will use the new system for all devices in Mu3e, with probably some new features being added soon, so stay tuned.

/Stefan

Dear all,

a new feature was implemented which allows to search the log messages in MIDAS. Attached one can find a more detailed explanation of how to use the feature.

If you see any issues / bugs feel don't hesitate to report them. For now the code was tested on Linux / Mac OS using Chrome, Firefox and Safari.

Best,
Marius 

to continue where we left off in 2019,
https://daq00.triumf.ca/elog-midas/Midas/1520

time to choose the next c++!

snapshot of 2019:

- Linux RHEL/SL/CentOS6 - gcc 4.4.7, no C++11.
- Linux RHEL/SL/CentOS7 - gcc 4.8.5, full C++11, no C++14, no C++17
- Ubuntu 18.04.2 LTS - gcc 7.3.0, full C++11, full C++14, "experimental" C++17.
- MacOS 10.13 - llvm 10.0.0 (clang-1000.11.45.5), full C++11, full C++14, full C++17

the world moved on:

- el6/SL6 is gone
- el7/CentOS-7 is out the door, only two experiments on my plate (EMMA and ALPHA-g)
- el8 was a still born child of RedHat
- el9 - gcc 11.5 with 12, 13, and 14 available.
- el10 - gcc 14.2

- U-18 - gcc  7.5
- U-20 - gcc  9.4 default, 10.5 available
- U-22 - gcc 11.4 default, 12.3 available
- U-24 - gcc 13.3 default, 14.2 available

- MacOS 15.2 - llvm/clang 16

Next we read C++ level support:

(see here for GCC C++ support: https://gcc.gnu.org/projects/cxx-status.html)
(see here for LLVM clang c++ support: https://clang.llvm.org/cxx_status.html)
(see here for GLIBC c++ support: https://gcc.gnu.org/onlinedocs/libstdc++/manual/status.html)

gcc:
4.4.7 - no C++11
4.8.5 - full C++11, no C++14, no C++17
7.3.0 - full C++11, full C++14, "experimental" C++17.
7.5.0 - c++17 and older
9.4.0 - c++17 and older
10.5 - no c++26, no c++23, mostly c++20, c++17 and older
11.4 - no c++26, no c++23, full c++20 and older
12.3 - no c++26, mostly c++23, full c++20 and older
13.3 - no c++26, mostly c++23, full c++20 and older
14.2 - limited c++26, mostly c++23, full c++20 and older

clang:
16 - no c++26, mostly c++23, mostly c++20, full c++17 and older

I think our preference is c++23, the number of useful improvements is quite big.

This choice will limit us to:
- el9 or older
- U-22 or older
- current MacOS 15.2 with Xcode 16.

It looks like gcc and llvm support for c++23 is only partial, so obviously we will use a subset of c++23 
supported by both.

Next step is to try to build midas with c++23 on el9 and U-22,24 and see what happens.

K.O.

Hello,

I've implemented a very basic syntax validation in the sequencer GUI. Click the validation button to check the syntax in the current tab.

Please note that this does only a simple syntax validation, the correctness of the logic is still on you :)

Some time ago Ben Smith added test coverage reports using GCC -fprofile-arcs -
ftest-coverage and lcov.

It reports code coverage after running "make test". Reported code coverage is 
surprisingly high for the very little code executed by "make test" - create ODB, 
start a frontend,  start run, stop run, check that mlogger created data files 
and history files.

(Of course coverage can never reach 100%, some obscure branches are unreachable 
without using an automated fuzzer, and some error paths are unreachable without 
also using a system call fault injector).

Simplest way to generate a coverage report:

make clean
make cmake YES_COVERAGE=1
make coverage
open cov_html/index.html

K.O.

> time to choose the next c++!

Ununtu-24.04, MIDAS builds with -std=c++23 without errors or any additional warnings. (but does it work? "make 
test" is ok).

K.O.

I need to look at histograms inside a ROOT file, but all the old ways for doing this no longer work. (in theory I can scp the ROOT file to 
the computer I am sitting in front of, but this assumes I have a working ROOT there. anyhow it is pointless to fight this, all modern 
packages are written to only work on the developer's laptop).

- root new TBrowser starts a web server, tries to open firefox (and fails)
- root --web=off new TBrowser using ssh X11 tunnel no longer works, ROOT X11 graphics refresh is broken
- macos root binary kit is built without X11 support, root --web=off does not work at all
- root7 recommended "rootssh" prints an error message (and fails)

What does work well is JSROOT which we use to look at manalyzer live histograms (through apache and mhttpd web proxies).

So I wrote mjsroot.exe. It opens a ROOT file and starts JSROOT to look at it (plus a bit of dancing around to make it actually work):

mjsroot.exe -R8082 root_output_files/output00371.root

To actually see the histograms:

a) if you sitting in front of the same computer, open http://localhost:8082
b) if you are somewhere else, start an ssh tunnel: ssh daq13 -L8082:localhost:8082, open http://localhost:8082
c) if daq13 is running mhttpd, setup http proxy:
set ODB /webserver/proxy/mjsroot to http://localhost:8082
open https://daq13.triumf.ca/proxy/mjsroot/
also
set ODB /alias/mjsroot to "/proxy/mjsroot/"
reload MIDAS status page, observe "mjsroot" in listed in the left-hand side, open it.

K.O.
 

Hello everyone, 

I just wanted to let you know, that I published a MSL Syntax Highlighting Extension for VSCode.
It is still in a quite early stage, so there might be some missing keywords and edge cases which are not fully handled. So in case you find any issues or have suggestions for improvements, I am happy to implement them. Also I only tested it with a custom theme (One Monokai), so it might look very different with the default theme and other ones.

The extension is called "MSL Syntax Highlighter" and can be found in the extension marketplace in VSCode.  (vscode marketplace: https://marketplace.visualstudio.com/items?itemName=LukasMandok.msl-syntax-highlighter, github repo: https://github.com/LukasMandok/msl-syntax-highlighter)

One additional remark:
- To keep a consitent style with existing themes, one is a bit limited in regard to colors. For this reason a distinction betrween LOOP and IF Blocks is not really possible without writing a custom theme. A workaround would be to add the theming in the custom user settings (explained in the readme).

Dear MIDAS enthusiasts,

We are planning a fifth MIDAS workshop, following on from previous successful 
workshops in 2015, 2017, 2019 and 2023.  The goals of the workshop include:

- Getting updates from MIDAS developers on new features, bug fixes and planned 
changes.
- Getting reports from MIDAS users on how they are using MIDAS and what problems 
they are facing.
- Making plans for future MIDAS changes and improvements

We are planning to have an online workshop on Sept 22-23, 2025 (it will coincide 
with a visit of Stefan to TRIUMF).  We are tentatively planning to have a four 
hour session on each day, with the sessions timed for morning in Vancouver and 
afternoon/evening in Europe.  Sorry, the sessions are likely to again not be well 
timed for our colleagues in Asia.  

We will provide exact times and more details closer to the date.  But I hope 
people can mark the dates in their calendars; we are keen to hear from as much of 
the MIDAS community as possible.  

Best Regards,
Thomas Lindner

I had to change the ODBXX API: https://bitbucket.org/tmidas/midas/commits/273c4e434795453c0c6bceb46bac9a0d2d27db18

The old C API is case-insensitive, meaning db_find_key("name") returns a key "name" or "Name" or "NAME". We can discuss if this is good or bad, but that's how it is since 30 years.

I now realized the the ODBXX API keys is case sensitive, so a o["NAME"] does not return any key "name". Rather, it tries to create a new key which of course fails. I changed therefore
the ODBXX to become case-insensitive like the old C API.

Stefan

it is a very convenient interface! Does it support the ODB links in the path names ? -- thanks, regards, Pasha

I was working on an odb corruption crash inside db_delete_key() and I noticed 
that I did not test db_delete_key() with follow_links set to TRUE. Then I noticed 
that nobody nowhere seems to use db_delete_key() with follow_links set to TRUE. 

Instead of testing it, can I just remove it?

This feature existed since day 1 (1st commit) and it does something unexpected 
compared to filesystem "/bin/rm": the best I can tell, it is removes the link 
*and* whatever the link points to. For people familiar with "/bin/rm", this is 
somewhat unexpected and by my thinking, if nobody ever added such a feature to 
"/bin/rm", it is probably not considered generally useful or desirable. (I would 
think it dangerous, it removes not 1 but 2 files, the 2nd file would be in some 
other directory far away from where we are).

By this thinking, I should remove "follow_links" (actually just make it do thing 
, to reduce the disturbance to other source code). db_delete_key() should work 
similar to /bin/rm aka the unlink() syscall.

K.O.

I would handle this actually like symbolic links are handled under linux. If you delete a symbolic link, the link gets 
detected and NOT the file the link is pointing to.

So I conclude that the "follow links" is a misconception and should be removed.

Stefan

Dear All, 

I'm pretty sure many know this already, however I found this feature by a an accident 
and want to share with those who don't know about it yet - seems very useful. 

- it looks that one can use ROOT scripting with rootcling and call from the 
  interactive ROOT prompt any function defined in midas.h and access ODB seemingly 
  WITHOUT DOING anything special 

- more surprisingly, that also works for odbxx, with one minor exception in handling 
  the 64-bit types - the proof is in attachment. The script test_odbxx.C loaded 
  interactively is Stefan's

 https://bitbucket.org/tmidas/midas/src/develop/examples/odbxx/odbxx_test.cxx 

with one minor change - the line 
 
   o[Int64 Key] = -1LL;

is replaced with

   int64_t x = -1LL;
   o["Int64 Key"] = x;

- apparently the interpeter has its limitations. 

My rootlogon.C file doesn't load any libraries, it only defines the appropriate 
include paths. So it seems that everything works pretty much out of the box. 

One issue has surfaced however. All that worked despite my experiment 
had its name="test_025", while the example specifies experiment="test". 
Is it possible that that only first 4 characters are being tested ? 

-- regards, Pasha

Dear all,

We have setup an indico page for the MIDAS workshop on Sept 22-23.  The page is here

https://indico.psi.ch/event/17580/overview

As I mentioned, we are keen to hear reports from any users or developers; we want to hear  
how MIDAS is working for you and what improvements you would like to see.  If you or your 
experiment would like to give a talk about your MIDAS experiences then please submit an 
abstract through the indico page.  

Also, feel free to also register for the workshop (no fees).  Registration is not 
mandatory, but it would be useful for us to have an idea how many people will connect.

Thanks,
Thomas


> Dear MIDAS enthusiasts,
> 
> We are planning a fifth MIDAS workshop, following on from previous successful 
> workshops in 2015, 2017, 2019 and 2023.  The goals of the workshop include:
> 
> - Getting updates from MIDAS developers on new features, bug fixes and planned 
> changes.
> - Getting reports from MIDAS users on how they are using MIDAS and what problems 
> they are facing.
> - Making plans for future MIDAS changes and improvements
> 
> We are planning to have an online workshop on Sept 22-23, 2025 (it will coincide 
> with a visit of Stefan to TRIUMF).  We are tentatively planning to have a four 
> hour session on each day, with the sessions timed for morning in Vancouver and 
> afternoon/evening in Europe.  Sorry, the sessions are likely to again not be well 
> timed for our colleagues in Asia.  
> 
> We will provide exact times and more details closer to the date.  But I hope 
> people can mark the dates in their calendars; we are keen to hear from as much of 
> the MIDAS community as possible.  
> 
> Best Regards,
> Thomas Lindner

Hello, I'm currently writing a paper about making a dataset publicly available 
from the University of Minnesota and it's a MIDAS dataset.

I'd like to use an image that shows the MIDAS data format (that's been slightly 
modified to fit in the paper) and am wondering (1) if I could get permission to do 
so and (2) what the preferred attribution would be. 

> Hello, I'm currently writing a paper about making a dataset publicly available 
> from the University of Minnesota and it's a MIDAS dataset.
> 
> I'd like to use an image that shows the MIDAS data format (that's been slightly 
> modified to fit in the paper) and am wondering (1) if I could get permission to do 
> so and (2) what the preferred attribution would be. 

Feel free to use whatever you like, the documentation is on the same open source license as midas itself.

Stefan

Today the way the history system is configured has changed. Whenever one adds new equipment 
variables, the logger has to add that variable to the history system. Previously, this happened during 
startup of the logger and at run start. We have now a case in the Mu3e experiment where we have many 
variables and the history configuration takes about 15 seconds, which delays data taking considerably.

After discussion with KO we decided to remove the history re-configuration at run start. This speeds up 
the run start considerably, also for other experiments with many history variables. It means however that 
once you add/remove/rename any equipment variable going into the history system, you have to restart the 
logger for this to become active.

https://bitbucket.org/tmidas/midas/commits/c0a14c2d0166feb6b38c645947f2c5e0bef013d5

Stefan