I added Makefile rules for running MIDAS through the clang-tidy static code 
analyzer. (rules for running cppcheck have gone missing, I hope I find them).

Reports from clang-tidy are somewhat repetitive (complain about the same class of 
non-problem problems repeatedly), but several warnings and errors identified real 
bugs.

Specifically, detection of memory leaks in error paths and detection of NULL 
pointer dereference and use of uninitialized variables is pretty solid.

Fixes for the worst problems reported by cppcheck and clang-tidy are already 
committed to midas git.

K.O.

> I would handle this actually like symbolic links are handled under linux. If you delete a symbolic link, the link gets 
> detected and NOT the file the link is pointing to.
> 
> So I conclude that the "follow links" is a misconception and should be removed.

I am finally writing test code for the ODB API, partially to discover what db_delete_key(TRUE) actually does,
because when I went ahead to remove it, I found that it's use is inconsistent. And indeed
it does strange and unexpected things, I will proceed with removing it.

K.O.

> > I would handle this actually like symbolic links are handled under linux. If you delete a symbolic link, the link gets 
> > detected and NOT the file the link is pointing to.
> > 
> > So I conclude that the "follow links" is a misconception and should be removed.
> 
> I am finally writing test code for the ODB API, partially to discover what db_delete_key(TRUE) actually does,
> because when I went ahead to remove it, I found that it's use is inconsistent. And indeed
> it does strange and unexpected things, I will proceed with removing it.
> 

this is now merged into develop. there will be deprecation warnings from mvodb and from midas_c_compat, I and Ben will clean 
them up, just not today.

for more detail, see separate message.

K.O.

> > Following discussions at the MIDAS workshop, we propose to move MIDAS from c++11 to c++17.
> We shall move forward with this change.

It is done. Last c++11 MIDAS is midas-2025-11-a (plus the db_delete_key merge).

I notice the cmake does not actually pass "-std=c++17" to the c++ compiler, and on U-20, it is likely 
the default c++14 is used. cmake always does the wrong thing and this will need to be fixed later.

K.O.

> > we should finally bite the bullet and remove ROOT support from MIDAS ...

as discussed, HAVE_ROOT and OBSOLETE were removed from mlogger. rmana and ROOT support in manalyzed remain, 
untouched.

last rmlogger is in MIDAS tagged midas-2025-11-a.

K.O.

> I notice the cmake does not actually pass "-std=c++17" to the c++ compiler, and on U-20, it is likely 
> the default c++14 is used. cmake always does the wrong thing and this will need to be fixed later.

Does adding

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

Get it to do the right thing?

Cheers,

Nick..

> (rules for running cppcheck have gone missing, I hope I find them).

found them. I built cppcheck from sources.

  520  ~/git/cppcheck/build/bin/cppcheck src/midas.cxx
  523  ~/git/cppcheck/build/bin/cppcheck manalyzer/manalyzer.cxx manalyzer/mjsroot.cxx 
  524  ~/git/cppcheck/build/bin/cppcheck src/tmfe.cxx
  525  ~/git/cppcheck/build/bin/cppcheck midasio/*.cxx
  526  ~/git/cppcheck/build/bin/cppcheck mjson/*.cxx

K.O.

> > > Following discussions at the MIDAS workshop, we propose to move MIDAS from c++11 to c++17.
> > We shall move forward with this change.
> 
> It is done. Last c++11 MIDAS is midas-2025-11-a (plus the db_delete_key merge).
> 
> I notice the cmake does not actually pass "-std=c++17" to the c++ compiler, and on U-20, it is likely 
> the default c++14 is used. cmake always does the wrong thing and this will need to be fixed later.
> 
> K.O.

We should either use 

set(CMAKE_CSS_STANDARD 17)

or

target_compile_features(<target> PUBLIC cxx_std_17)

but not mix both. We have already the second one for the midas library, like 

target_compile_features(objlib PUBLIC cxx_std_17)

which correctly causes a

c++ -std=gnu++17 ...

(at leas in my case).

If the compiler flag is missing for a target, we should add the target_compile_feature above for that target.

Stefan

> 
> > I notice the cmake does not actually pass "-std=c++17" to the c++ compiler, and on U-20, it is likely 
> > the default c++14 is used. cmake always does the wrong thing and this will need to be fixed later.
> 
> set(CMAKE_CXX_STANDARD 17)
> set(CMAKE_CXX_STANDARD_REQUIRED ON)
>

We used to have this, it is not there now.

> 
> Get it to do the right thing?
> 

Unlikely, as Stefan reported, asking for C++17 yields -std=gnu++17 which is close enough, but not the same 
thing.

For now, it does not matter, U-22 and U-24 are c++17 by default, if somebody accidentally commits c++20 
code, builds will fail and somebody will catch it and complain, plus the weekly bitbucket build will bomb-
out.

On U-20, default is c++14 and builds will start bombing out as soon as we commit some c++17 code.

el7 builds have not worked for some time now (a bizarre mysterious error)

el8, el9, el10 likely same situation as Ubuntu.

macos, not sure.

K.O.

> target_compile_features(<target> PUBLIC cxx_std_17)
> which correctly causes a
> c++ -std=gnu++17 ...

I think this is set in a couple of places, yet I do not see any -std=xxx flag passed to the compiler.

(and I am not keen on spending a full day fighting cmake)

(btw, -std=c++17 and -std=gnu++17 are not the same thing, I am not sure how well GNU extensions are supported on 
macos)

K.O.

I switched from 

  target_compile_features(<target> PUBLIC css_std_17)

to

  set(CMAKE_CXX_STANDARD 17)
  set(CMAKE_CXX_STANDARD_REQUIRED ON)
  set(CMAKE_CXX_EXTENSIONS OFF)   # optional: disables GNU extensions

Which is now global in the CMakeLists.txt, so we only have to deal with one location if we want to change it. It also turns off the g++ options. On my
Mac I get now a clean

  -std=c++17

Please everybody test on your side. Change is committed.

Stefan

> 
>   set(CMAKE_CXX_STANDARD 17)
>   set(CMAKE_CXX_STANDARD_REQUIRED ON)
>   set(CMAKE_CXX_EXTENSIONS OFF)   # optional: disables GNU extensions
> 

Looks like it works, I see -std=c++17 everywhere. Added same to manalyzer and mscb (mscb was still c++11).

Build on U-20 works (g++ accepts -std=c++17), build on CentOS-7 bombs, cmake 3.17.5 does not know CXX17.

K.O.

> This is moving slowly. I now have RPC caller side support for std::string and 
> std::vector<char>. RPC server side is next. K.O.

The RPC_CXX code is now merged into MIDAS branch feature/rpc_call_cxx.

This code fully supports passing std::string and std::vector<char> through the MIDAS RPC is both directions.

For data passed from client to mserver, memory for string and vector data is allocated automatically as needed.

For data returned from mserver to client, memory to hold returned string and vector data is allocated automatically as 
need.

This means that RPC calls can return data of arbitrary size, the rpc caller does not need to know maximum data size.

Removing this limitation was the main motivation for this development.

I completed this code in June 2024, but could not merge it because I broke my git repository (oops). Now I am doing 
the merge manually. Changes are isolated to rpc_call_encode(), rpc_call_decode() and rpc_execute(). My intent right 
now is to use the new RPC code only for RPCs that pass std::string and std::vector<char>, existing RPCs will use the 
old code without any changes. This seems to be the safest way to move forward.

Included is test_rpc() which tests and probes most normal uses cases and some corner cases. When writing the test 
code, I found a few bugs in the old MIDAS RPC code. If I remember right, I committed fixes for those bugs to main 
MIDAS right then and there.

K.O.

Just to remember, MIDAS has cm_expand_env() to expand environment variables, in 
file paths, etc. It is used in several places in mhttpd, msequencer and mjsonrpc.

std::string my_secret_file = cm_expand_env("$HOME/.ssh/authorised_keys");

One could add it everywhere we open files, etc, except for the security 
consideration. We should not permit any/every web site to read any/every local 
file (directly by injecting malicious js code or by cross-site mjsonrpc call). 
Access should be limited to files in designated MIDAS experiment subdirectories. 
Places like $HOME/.ssh, $HOME/.cache/google-chrome, etc must be protected.

K.O.

> > This is moving slowly. I now have RPC caller side support for std::string and 
> > std::vector<char>. RPC server side is next. K.O.
> The RPC_CXX code is now merged into MIDAS branch feature/rpc_call_cxx.
> This code fully supports passing std::string and std::vector<char> through the MIDAS RPC is both directions.

The RPC_CXX in now merged into MIDAS develop. commit 34cd969fbbfecc82c290e6c2dfc7c6d53b6e0121.

There is a new RPC parameter encoder and decoder. To avoid unexpected breakage, it is only used for newly added RPC_CXX 
calls, but I expect to eventually switch all RPC calls to use the new encoder and decoder.

As examples of new code, see RPC_JRPC_CXX and RPC_BRPC_CXX, they return RPC data in an std::string and std::vector<char> 
respectively, amount of returned data is unlimited, mjsonrpc parameter "max_reply_length" is no longer needed/used.

Also included of RPC_BM_RECEIVE_EVENT_CXX, it receives event data as an std::vector<char> and maximum event size is no 
longer limited, ODB /Experiment/MAX_EVENT_SIZE is no longer needed/used. To avoid unexpected breakage, this new code is not 
enabled yet.

K.O.

I added cmake support for the thread sanitizer (address sanitizer was already 
there). Use:

make cmake -j YES_THREAD_SANITIZER=1 # (or YES_ADDRESS_SANITIZER=1)

However, thread sanitizer is broken on U-24, programs refuse to start ("FATAL: 
ThreadSanitizer: unexpected memory mapping") and report what looks like bogus 
complaints about mutexes ("unlock of an unlocked mutex (or by a wrong thread)").

On macos, thread sanitizer does not report any errors or warnings or ...

P.S.

The Undefined Behaviour Sanitizer (UBSAN) complained about a few places where 
functions could have been called with a NULL pointer arguments, I added some 
assert()s to make it happy.

K.O.

Hello everyone,

Stefan and I have make a few updates to mplot to clean up some of the code and make it more usable directly from Javascript. With one exception this does not change the html interface. The below describes changes up to commit cd9f85c. 

Breaking Changes:

• The idea is to have a "graph" be the overarching figure object, whereas the "plot" is the line or points associated with a single dataset.
  • Some internal variable names have been changed to reflect this while minimizing breaking changes
  • defaultParam renamed defaultGraphParam.
  • There is no longer an initialized defaultParam.plot[0], these defaults are now defaultPlotParam which is a separate global variable
  • MPlotGraph constructor signature MPlotGraph(divElement, param) changed to MPlotGraph(divElement, graphParam)
• HTML key data-bgcolor changed to data-zero-color as the former was misleading

New Features

• New addPlot() function. 
  • While the functionality of setData is preserved you can now use addPlot(plotParam) to add a new plot to the graph with minimal copying of the old defaultParam.plot[0]
  • Minimal example, from plot_example.html: given some div container with id "P6":

       let d = document.getElementById("P6"); // get div 
       d.mpg = new MPlotGraph(d, { title: { text: "Generated" }}); // make graph
       d.mpg.addPlot( { xData: [0, 1, 2, 3, 4], yData: [10, 12, 12, 14, 11] } ); // add plot to the graph

  • modifyPlot() and deletePlot() still to come
• New lines styles: none, solid, dashed, dotted
• Barplot-style category plots

After popular demand and during some quite holidays I implemented ad-hoc history plots. To enable this 
for a certain equipment, put 

  /Equipment/<name>/Settings/History buttons = true

into the ODB. You will then see a graph button after each variable. Pressing this button reveals the history 
for this variable, see attachment.

Stefan

Tabbed custom pages have been implemented in MIDAS. Below you see and example. The documentation 
is here:

  https://daq00.triumf.ca/MidasWiki/index.php/Custom_Page#Tabbed_Pages

Stefan

Dear all,

For my personal convenience, I started to add an homebrew formula
for
midas (*):
https://github.com/guiguem/homebrew-tap/blob/main/Formula/
midas.rb
It
is convenient in particular to deploy as it automatically gets all
the right
dependencies; for MacOS (**), there are bottles already available.

The
installation would then be
brew tap guiguem/tap
brew install midas

I
thought I
would share it here, if this is helpful to someone else (***).
This
was tested
rather extensively, including the development of manalyzer modules
using this
bottled version as backend.
A possible upgrade (if people are
interested) would
be to develop/deploy a "mainstream" midas version (and I would
rename mine
"midas-mod").

Cheers
Mathieu
-----
Notes:
(*) The version installed
by this
formula is a very slightly modified version of midas, designed to
support more
than 100 front-ends (needed for HK).
See commits here:
https://
gitlab.in2p3.fr/
hk/clocks/midas/-/
commit/060b77afb38e38f9a3155d2606860f12d680f4de
https://
gitlab.in2p3.fr/hk/
clocks/midas/-/
commit/1da438ad1946de7ba697e849de6a6675ac45ebb8
I have the
recollection this
version might not be compatible with the main midas one. 

(**) I also have some
stuff for Ubuntu, but Ubuntu seems to do additional
linkage to curl which needs
to be handled (easy).
That being said the
installation from sources works fine!
(***) Some oddities were unraveled such as
the fact that the build_interface
pointing to the source include directory are
still appearing in the
midasConfig.cmake files (leading to issues in brew). This
was fixed by replacing
the faulty path to the final installation location. Maybe
this should be fixed ?