> Sorry to intervene there, but the FEs are usually compiled against libmidas.a . Therefore you have to compile midas, usually do a "make install" to update the libmidas.a/so, then recompile the FEs. You probably forgot the "make install".
> 
> Stefan

OK, solved, closed. Turned out I messed up rebuilding some of the FEs.

More generally, and with the "Documentation" discussion of the workshop in mind, ODB mismatch error messages of all kind are a recurring phenomena confusing users. And MIDASGPT gave complete wrong suggestions.  

Sorry to intervene there, but the FEs are usually compiled against libmidas.a . Therefore you have to compile midas, usually do a "make install" to update the libmidas.a/so, then recompile the FEs. You probably forgot the "make install".

Stefan

Just to be clear, it seems that your "EPICS Frontend" was either not recompiled against the new midas yet or the old binary is being run, but "SC Frontend" is using the new midas.

> > So I started our DAQ with an updated midas, after ca. 6 months+.
> 
> Would be worthwhile mentioning the git commit hash or tag you are using.
> 
> > No issues except all FEs complaining about the Alarm ODB structure.
> > * I adapted to the new structure ( trigger count & trigger count required )
> > * restarted fe's
> > * recompiled
> > 
> > 18:17:40.015 2025/09/30 [EPICS Frontend,INFO] Fixing ODB "/Alarms/Alarms/logger" 
> > struct size mismatch (expected 452, odb size 460)
> > 
> > 18:17:40.009 2025/09/30 [SC Frontend,INFO] Fixing ODB "/Alarms/Alarms/logger" 
> > struct size mismatch (expected 460, odb size 452)
> 
> This seems to be https://daq00.triumf.ca/elog-midas/Midas/2980
> 
> > how do I get the FEs + ODB back in line here?
> 
> Recompile all frontends against new midas.
> 
> Nick.

> So I started our DAQ with an updated midas, after ca. 6 months+.

Would be worthwhile mentioning the git commit hash or tag you are using.

> No issues except all FEs complaining about the Alarm ODB structure.
> * I adapted to the new structure ( trigger count & trigger count required )
> * restarted fe's
> * recompiled  
> 
> 18:17:40.015 2025/09/30 [EPICS Frontend,INFO] Fixing ODB "/Alarms/Alarms/logger" 
> struct size mismatch (expected 452, odb size 460)
> 
> 18:17:40.009 2025/09/30 [SC Frontend,INFO] Fixing ODB "/Alarms/Alarms/logger" 
> struct size mismatch (expected 460, odb size 452)

This seems to be https://daq00.triumf.ca/elog-midas/Midas/2980

> how do I get the FEs + ODB back in line here?

Recompile all frontends against new midas.

Nick.

So I started our DAQ with an updated midas, after ca. 6 months+. 

No issues except all FEs complaining about the Alarm ODB structure.
* I adapted to the new structure ( trigger count & trigger count required )
* restarted fe's
* recompiled  

18:17:40.015 2025/09/30 [EPICS Frontend,INFO] Fixing ODB "/Alarms/Alarms/logger" 
struct size mismatch (expected 452, odb size 460)

18:17:40.009 2025/09/30 [SC Frontend,INFO] Fixing ODB "/Alarms/Alarms/logger" 
struct size mismatch (expected 460, odb size 452)

how do I get the FEs + ODB back in line here?

thanks

> > ...I talked to KO so he agreed that yo then commit your proposed change of manalyzer 
> Merged and pushed.

negative. as I already 

> ...I talked to KO so he agreed that yo then commit your proposed change of manalyzer 

Merged and pushed.

Thanks,

Mark.

Sorry, 

I have had now the time to dig deeper in our code and realized that we actually use rmana, i.e. WITH ROOT. If there is an easy way to incorporate the necessary parts temporarily to our side, we will do it. Without ROOT this would have been quite easy, with ROOT, I am not that sure. Anyhow, as said the timeline for this is only until end of 2026.

Andreas

> > Hi, at the LEM Experiment at PSI, we still use mana.c and would like to keep it until end of 2026, where we will enter a long shutdown.
> 
> Excellent, good to hear from you! Once we remove ROOT support rmana.o will be gone, only mana.o (no ROOT) will remain. Will this break your builds?
> 
> One solution could be to copy mana.c from MIDAS into your source tree and compile/link it from there (not from MIDAS).
> 
> Perhaps the way to proceed is create a test branch with ROOT and mana.c removed, you can try it,
> report success/fail and we go from there.
> 
> We should schedule this work for when both of us have a block of free time to work on it.
> 
> K.O.

Documenting a discussion I had with Konstantin a while ago.

One aspect of the MIDAS history plotting I find frustrating is the sequence for adding a new history 
variable and then plotting them.  At least for me ,the sequence for a new variable is:

1) Add a new named variable to a MIDAS bank; compile the frontend and restart it; check that the new 
variable is being displayed correctly in MIDAS equipment pages.

2) Stop and restart programs; usually I do
- stop and restart mlogger
- stop and restart mhttpd
- stop and restart mlogger
- stop and restart mhttpd

3) Start adding the new variable to the history plots.

My frustration is with step 2, where the logger and web server need to be restarted multiple times.  
I think that only one of those programs actually needs to be restart twice, but I can never remember 
which one, so I restart both programs twice just to be sure.

I don't entirely understand the sequence of what happens with these restarts so that the web server 
becomes aware of what new variables are in the history system; so I can't make a well-motivated 
suggestion.

Ideally it would be nice if step 2 only required restarting the mlogger and mhttpd automatically 
became aware of what new variables were in the system.

But even just having a single restart of mlogger, then mhttpd would be an improvement on the current 
situation and easier to explain to users.

Hopefully this would not be a huge amount of work.

> > perhaps c++20? - std::format is an immediately useful feature. --regards, Pasha
> 
> confirmed. std::format is an improvement over K&R C printf().
> 
> but seems unavailable on U-20 and older, requires --std=c++20 on U-24 and MacOS.

agreed! - availability is significantly more important. -- regards, Pasha

> perhaps c++20? - std::format is an immediately useful feature. --regards, Pasha

confirmed. std::format is an improvement over K&R C printf().

but seems unavailable on U-20 and older, requires --std=c++20 on U-24 and MacOS.

but also available as a standalone library: https://github.com/fmtlib/fmt

myself, I use printf() and msprintf(), I think std::format is in the "too little, too late to save C++" 
department.

K.O.

To record discussion with Stefan about long history variable names.

We have several requests to remove the 32-byte limit on history variable names.

Presently, history variable names are formed from two 32-byte strings: history event name and 
history tag name:

* history event name is usually same as the equipment name (also a 32-byte string)
* history tag name is composed from /eq/xxx/variables/yyy name (also a 32-byte string) or from 
names in /eq/xxx/variables/names and "names zzz", which can have arbitrary length (and tag name 
would have to be truncated).

This worked well for "per-equipment" history, history events corresponded to equipment/variables 
and all data from equipment/variables were written together in one go. (this very inefficient if 
values of only one variable is updated).

Then at some point we implemented "per-variable" history:

* history event name is a equipment name (32-byte string) plus /eq/xxx/variables/vvv variable name 
(also 32-byte string). (obviously truncation is quite possible)
* history tag name is unchanged (also can be truncated)

With "per-variable" history, history events correspond to individual variables (ODB entries) in 
/eq/xxx/variables. If value of one variable is updated, only that variable is written to ODB. This 
is much more efficient. (If variable is an array, the whole array is written, is variable is a 
subdirectory, the whole subdirectory is written).

We considered even finer granularity, writing to history file only the one value that changed, but 
decided against slicing the data too fine. (for arrays, MIDAS frontends usually update all values 
of an array, as in "array of 10 temperatures" or "array of 4000 high voltages").

Many years later, we have the SQL history and the FILE history which do not have the 32-byte limit 
on history event names and history tag names (no limit in the MIDAS C++ code. MySQL and PgSQL have 
limits on table name and column name lengths, 64 bytes and 31 bytes respectively, best I can 
tell).

But the API still uses the MIDAS "struct TAG" with the 32-byte tag name length limit.

It is pretty easy to change the API to use a new "struct TAG_CXX" with std::string unlimited-
length tag names, but the old MIDAS history will be unable to deal with long names. Hence
the discussion about removing the old MIDAS history and keeping only the FILE and SQL history 
(plus the mhdump and mh2sql tools to convert old history files to the new formats).

(some code in mhttpd may need to be corrected for long history names. javascript code should be 
okey, history plot code may need adjustment to display pathologically long names. use small font, 
truncate, etc).

K.O.

perhaps c++20? - std::format is an immediately useful feature. --regards, Pasha

> Following discussions at the MIDAS workshop, we propose to move MIDAS from c++11 to c++17. There is 
> many new features and we want to start using some of them.
> 
> Per my previous message https://daq00.triumf.ca/elog-midas/Midas/3084,
> c++17 is available on current MacOS, U-22 and newer, el9 and newer, D-12 and newer.
> 
> (ROOT moved to C++17 as of release 6.30 on November 6, 2023)
> 
> As I reported earlier, MIDAS already builds with c++23 on U-24, and this move does not require any 
> actual code changes other than a bump of c++ version in CMakeLists.txt and Makefile.
> 
> Please let us know if this change will cause problems or if you think that we should move to an older 
> c++ (c++14) or newer c++ (c++20 or c++23 or c++26).
> 
> If we do not hear anything, we will implement this change in about 2-3 weeks.
> 
> K.O.

To record discussion with Stefan regarding 64-bit time_t

To remember:

signed   32-bit time_t will overflow in 2038 (soon enough)
unsigned 32-bit time_t will overflow in 2106 ("not my problem")

https://en.wikipedia.org/wiki/Year_2038_problem
https://wiki.debian.org/ReleaseGoals/64bit-time

64-bit Linux uses 64-bit time_t since as far back as el6.

MIDAS uses unsigned 32-bit (DWORD) time-in-seconds in many places (ODB, event 
headers, event buffers, etc)

MIDAS also uses unsigned 32-bit (DWORD) time-in-milliseconds in many places.

All time arithmetic is done using unsigned 32-bit math, these time calculations
are good until year 2106, at which time they will wrap around
(but still work correctly).

So we do not need to do anything, but...

To reduce confusion between the different time types, we will probably
introduce a 32-bit-time-in-seconds data type (i.e. time32_t alias for uint32_t),
and a 32-bit-time-in-milliseconds data type (i.e. millitime32_t alias for 
uint32_t). Also rename ss_time() to ss_time32() and ss_millitime() to 
ss_millitime32().

This should help avoiding accidental mixing of MIDAS 32-bit time, system 64-bit 
time and MIDAS 32-bit-time-in-milliseconds.

(confusion between time-in-seconds and time-in-milliseconds happened several 
times in MIDAS code).

There will be additional discussion and announcements if we go ahead with these 
changes.

K.O.

> Hi, at the LEM Experiment at PSI, we still use mana.c and would like to keep it until end of 2026, where we will enter a long shutdown.

Excellent, good to hear from you! Once we remove ROOT support rmana.o will be gone, only mana.o (no ROOT) will remain. Will this break your builds?

One solution could be to copy mana.c from MIDAS into your source tree and compile/link it from there (not from MIDAS).

Perhaps the way to proceed is create a test branch with ROOT and mana.c removed, you can try it,
report success/fail and we go from there.

We should schedule this work for when both of us have a block of free time to work on it.

K.O.

Hi,
at the LEM Experiment at PSI, we still use mana.c and would like to keep it until end of 2026, where we will enter a long shutdown.
There we will switch to the manalyzer. Before that, there is simply no time for the change over. One thing I already noticed is the "lack" of documentation, since 
for a lot of items I found simply "TBW". I know that writing documentation is boring and hard, but I hope at the time we will switch there is a more complete 
documentation available.

Thanks a lot for the ongoing development and support

Andreas

> Following discussions at the MIDAS workshop and the proposed removal of support for ROOT, the very obsolete mana.c 
> analyzer framework has reached the end of the line.
> 
> Right now we cannot remember any experiment that uses a mana.c based analyzer. Most experiments use analyzers based 
> on the rootana package (developed for ALPHA-1 at CERN) and on the manalyzer package (developed for ALPHA-2 and ALPHA-
> g at CERN, with multithreading support contributed by Joseph McKenna).
> 
> If you know of any experiment that uses a mana.c based analyzer, please let us know. We can help with building it 
> using an outside-of-midas local copy of mana.c or help with migration to a newer framework (or migration to a 
> framework-free standalone analyzer).
> 
> If we do not hear from anybody, we will remove mana.c (and rmana) at the same time as we remove ROOT support from 
> mlogger (rmlogger).
> 
> K.O.

Following discussions at the MIDAS workshop and the proposed removal of support for ROOT, the very obsolete mana.c 
analyzer framework has reached the end of the line.

Right now we cannot remember any experiment that uses a mana.c based analyzer. Most experiments use analyzers based 
on the rootana package (developed for ALPHA-1 at CERN) and on the manalyzer package (developed for ALPHA-2 and ALPHA-
g at CERN, with multithreading support contributed by Joseph McKenna).

If you know of any experiment that uses a mana.c based analyzer, please let us know. We can help with building it 
using an outside-of-midas local copy of mana.c or help with migration to a newer framework (or migration to a 
framework-free standalone analyzer).

If we do not hear from anybody, we will remove mana.c (and rmana) at the same time as we remove ROOT support from 
mlogger (rmlogger).

K.O.

Historically, building MIDAS with ROOT caused us many problems - build failures because of c++ version 
mismatch, CFLAGS mismatch; run-time failures because of ROOT library mismatches; etc, etc.

Following discussions at the MIDAS Workshop, we think we should finally bite the bullet and remove ROOT 
support from MIDAS:

- remove support for writing data in ROOT TTree format in mlogger (rmlogger)
- remove support for ROOT in mana.c based analyzer (which itself we propose to remove)
- remove ROOT helper functions in rmidas.h
- remove ROOT support in cmake
- remove rmlogger and rmana

This change will not affect the rootana and manalyzer packages, they will continue to be built with 
ROOT support if ROOT is available.

Right now, we cannot remember any experiment that uses the ROOT TTree output function in mlogger. 

If you use this feature, we very much would like to hear from you. Please contact Stefan or myself or 
reply to this message.

As replacement for rmlogger, we could implement identical or similar functionality using an manalyzer-
based custom logger, but we need at least one user who can test it for us and confirm that it works 
correctly.

K.O.

Following discussions at the MIDAS workshop, we propose to move MIDAS from c++11 to c++17. There is 
many new features and we want to start using some of them.

Per my previous message https://daq00.triumf.ca/elog-midas/Midas/3084,
c++17 is available on current MacOS, U-22 and newer, el9 and newer, D-12 and newer.

(ROOT moved to C++17 as of release 6.30 on November 6, 2023)

As I reported earlier, MIDAS already builds with c++23 on U-24, and this move does not require any 
actual code changes other than a bump of c++ version in CMakeLists.txt and Makefile.

Please let us know if this change will cause problems or if you think that we should move to an older 
c++ (c++14) or newer c++ (c++20 or c++23 or c++26).

If we do not hear anything, we will implement this change in about 2-3 weeks.

K.O.

> > > Of course with this patch extending manalyzer to process two or more runs at the same time becomes impossible.
> > Yes, I hadn't realised that was an option.
> 
> It is an option I would like to keep open. Not too many use cases, but imagine a "split brain" experiment
> that has two MIDAS instances record data into two separate midas files. (if LIGO were to use MIDAS,
> consider LIGO Hanford and LIGO Livingston).
> 
> Assuming data in these two data sets have common precision timestamps,
> our task is to assemble data from two input files into single physics events. The analyzer will need
> to read two input files, each file with it's run number, it's own ODB dump, etc, process the midas
> events (unpack, calibrate, filter, etc), look at the timestamps, assemble the data into physics events.
> 
> This trivially generalizes into reading 2, 3, or more input files.
> 
> > For that to work I guess the aforementioned static members could be made thread local storage, and 
> > processing of each run kept to a specific thread. Although I could imagine user code making assumptions and breaking, like storing a midas::odb as a 
> > class member or something.
> 
> manalyzer is already multithreaded, if you will need to keep track of which thread should see which odbxx global object,
> seems like abuse of the thread-local storage idea and intent.

I made the global variables storing the file name of type "thread_local", so each thread gets it's own copy. This means however that each thread must then call midas::odb::set_odb_source() individually before 
creating any midas::odb objects. Interestingly enough I just learned that thread_local (at least under linux) is almost zero overhead, since these variable are placed by the linker into a separate memory space which is 
separate for each thread, so accessing them only means to add a memory offset.

Let's see how far we get with this...

Stefan