 24 Jul 2025, Konstantin Olchanski, Bug Fix, support for large history files
|
FILE history code (mhf_*.dat files) did not support reading history files bigger than about 2GB, this is now
fixed on branch "feature/history_off64_t" (in final testing, to be merged ASAP).
|
|
23 Jul 2025, Konstantin Olchanski, Suggestion, K.O.'s guide to new C/C++ data types
|
Over the last 10 years, the traditional C/C++ data types have been
displaced by a hodgepodge of new data types that promise portability
and generate useful (and not so useful) warnings, for example:
|
 24 Jul 2025, Konstantin Olchanski, Suggestion, K.O.'s guide to new C/C++ data types
|
> for (int i=0; i<array_of_10_elements.size(); i++)
becomes
|
|
19 Feb 2025, Lukas Gerritzen, Bug Report, Default write cache size for new equipments breaks compatibility with older equipments
|
We have a frontend for slow control with a lot of legacy code. I wanted to add a new equipment using the
mdev_mscb class. It seems like the default write cache size is 1000000B now, which produces error
messages like this:
|
|
24 Feb 2025, Stefan Ritt, Bug Report, Default write cache size for new equipments breaks compatibility with older equipments
|
The commit that introduced the write cache size check is https://bitbucket.org/tmidas/midas/commits/3619ecc6ba1d29d74c16aa6571e40920018184c0
Unfortunately K.O. added the write cache size to the equipment list, but there is currently no way to change this programmatically from the user frontend |
|
20 Mar 2025, Konstantin Olchanski, Bug Report, Default write cache size for new equipments breaks compatibility with older equipments
|
I think I added the cache size correctly:
{"Trigger", /* equipment name */
|
|
20 Mar 2025, Konstantin Olchanski, Bug Report, Default write cache size for new equipments breaks compatibility with older equipments
|
the main purpose of the event buffer write cache is to prevent high contention for the
event buffer shared memory semaphore in the pathological case of very high rate of very
small events.
|
|
20 Mar 2025, Konstantin Olchanski, Bug Report, Default write cache size for new equipments breaks compatibility with older equipments
|
> the main purpose of the event buffer write cache
how to control the write cache size:
|
|
20 Mar 2025, Konstantin Olchanski, Bug Report, Default write cache size for new equipments breaks compatibility with older equipments
|
> > the main purpose of the event buffer write cache
> how to control the write cache size:
|
|
21 Mar 2025, Stefan Ritt, Bug Report, Default write cache size for new equipments breaks compatibility with older equipments
|
> All this is kind of reasonable, as only two settings of write cache size are useful: 0 to
> disable it, and 10 Mbytes to limit semaphore locking rate to reasonable value for all event
> rate and size values practical on current computers.
|
|
25 Mar 2025, Konstantin Olchanski, Bug Report, Default write cache size for new equipments breaks compatibility with older equipments
|
> > All this is kind of reasonable, as only two settings of write cache size are useful: 0 to
> > disable it, and 10 Mbytes to limit semaphore locking rate to reasonable value for all event
> > rate and size values practical on current computers.
|
|
21 Jul 2025, Stefan Ritt, Bug Report, Default write cache size for new equipments breaks compatibility with older equipments
|
> Perhaps have:
>
> set_write_cache_size("SYSTEM", 0);
|
|
13 Jul 2025, Zaher Salman, Info, PySequencer
|
As many of you already know Ben introduced the new PySequencer that allows running python scripts from MIDAS. In the last couple of month we have been working
on integrating it into the MIDAS pages. We think that it is now ready for general testing.
|
|
04 Jul 2025, Mark Grimes, Bug Report, Memory leaks in mhttpd
|
Something changed in our system and we started seeing memory leaks in mhttpd again. I guess someone
updated some front end or custom page code that interacted with mhttpd differently.
I found a few memory leaks in some (presumably) rarely seen corner cases and we now see steady
|
|
04 Jun 2025, Mark Grimes, Bug Report, Memory leak in mhttpd binary RPC code
|
Hi,
During an evening of running we noticed that memory usage of mhttpd grew to close to 100Gb. We think we've traced this to the following issue when making
RPC calls.
|
|
04 Jun 2025, Konstantin Olchanski, Bug Report, Memory leak in mhttpd binary RPC code
|
Noted. I will look at this asap. K.O.
[quote="Mark Grimes"]Hi,
|
07 Jun 2025, Mark Grimes, Bug Report, Memory leak in mhttpd binary RPC code 
|
Hi,
We applied an intermediate fix for this locally and it seems to have fixed our issue. The attached plot shows the percentage memory use
on our machine with 128 Gb memory, as a rough proxy for mhttpd memory use. After applying our fix mhttpd seems to be happy using ~7% of the memory |
|
10 Jun 2025, Konstantin Olchanski, Bug Report, Memory leak in mhttpd binary RPC code
|
I confirm that MJSON_ARRAYBUFFER does not work correctly for zero-size buffers,
buffer is leaked in the destructor and copied as NULL in MJsonNode::Copy().
|
|
15 Jun 2025, Mark Grimes, Bug Report, Memory leak in mhttpd binary RPC code
|
Many thanks for the fix. We've applied and see better memory performance. We still have to kill and restart
mhttpd after a few days however. I think the official fix is missing this part:
|
|
23 Jun 2025, Stefan Ritt, Bug Report, Memory leak in mhttpd binary RPC code
|
Since this memory leak is quite obvious, I pushed the fix to develop.
Stefan |
|
10 Jun 2025, Nik Berger, Bug Report, History variables with leading spaces
|
By accident we had history variables with leading spaces. The history schema check then decides that this is a new variable (the leading space is not read
from the history file) and starts a new file. We found this because the run start became slow due to the many, many history files created. It would be
nice to just get an error if one has a malformed variable name like this.
|
|
19 Jun 2025, Stefan Ritt, Bug Report, History variables with leading spaces
|
I added now code to the logger so it properly complains if there would be a leading space in a variable name.
Stefan
|
|
19 Jun 2025, Frederik Wauters, Bug Report, add history variables
|
I have encounter this a few times
* Make a new history panel
* Use the web GUI to add history variables
|
|
11 Jun 2025, Stefan Ritt, Info, Frontend write cache size
|
We had issues with the frontend write cache size and the way it was implemented in the frontend
framework mfe. Specifically, we had two equipments like in the experiment/examples/frontend.cxx, one
trigger event and one periodic event. While the trigger event sets the cache size correctly in the
|
|
10 Jun 2025, Stefan Ritt, Info, History configuration changed
|
Today the way the history system is configured has changed. Whenever one adds new equipment
variables, the logger has to add that variable to the history system. Previously, this happened during
startup of the logger and at run start. We have now a case in the Mu3e experiment where we have many
|
|
10 Jun 2025, Amy Roberts, Info, use of modified image showing MIDAS data format?
|
Hello, I'm currently writing a paper about making a dataset publicly available
from the University of Minnesota and it's a MIDAS dataset.
|
|
10 Jun 2025, Stefan Ritt, Info, use of modified image showing MIDAS data format?
|
> Hello, I'm currently writing a paper about making a dataset publicly available
> from the University of Minnesota and it's a MIDAS dataset.
>
|
|
16 Apr 2025, Thomas Lindner, Info, MIDAS workshop (online) Sept 22-23, 2025
|
Dear MIDAS enthusiasts,
We are planning a fifth MIDAS workshop, following on from previous successful
|
|
03 Jun 2025, Thomas Lindner, Info, MIDAS workshop (online) Sept 22-23, 2025
|
Dear all,
We have setup an indico page for the MIDAS workshop on Sept 22-23. The page is here
|
|
27 May 2025, Pavel Murat, Suggestion, handling of 2+ like-long messages
|
Dear MIDAS experts,
currently, the MIDAS messaging system is optimized for one-line long messages,
|
|
26 May 2025, Francesco Renga, Forum, Reading two devices in parallel
|
Dear experts,
in the CYGNO experiment, we readout CMOS cameras for optical readout of GEM-TPCs. So far, we only developed the readout for a single camera. In the
future, we will have multiple cameras to read out (up to 18 in the next phase of the experiment), and we are investigating how to optimize the readout |
|
26 May 2025, Stefan Ritt, Forum, Reading two devices in parallel
|
> Dear experts,
> in the CYGNO experiment, we readout CMOS cameras for optical readout of GEM-TPCs. So far, we only developed the readout for a single camera. In
the future, we will have multiple cameras to read out (up to 18 in the next phase of the experiment), and we are investigating how to optimize the readout |
|
25 May 2025, Pavel Murat, Bug Report, subdirectory ordering in ODB browser ?
|
Dear MIDAS experts,
I'm running into a minor but annoying issue with the subdirectory name ordering by the ODB browser.
|
|
24 May 2025, Pavel Murat, Info, ROOT scripting for MIDAS seems to work pretty much out of the box
|
Dear All,
I'm pretty sure many know this already, however I found this feature by a an accident
|
|
19 May 2025, Jonas A. Krieger, Suggestion, manalyzer root output file with custom filename including run number
|
Hi all,
Would it be possible to extend manalyzer to support custom .root file names that include the run number?
|
|
16 May 2025, Marius Koeppel, Bug Report, history_schema.cxx fails to build
|
Hi all,
we have a CI setup which fails since 06.05.2025 to build the history_schema.cxx. There was a major change in this code in the commits fe7f6a6 and 159d8d3.
|
|
16 May 2025, Konstantin Olchanski, Bug Report, history_schema.cxx fails to build
|
> we have a CI setup which fails since 06.05.2025 to build the history_schema.cxx.
> There was a major change in this code in the commits fe7f6a6 and 159d8d3.
|
|
16 May 2025, Konstantin Olchanski, Bug Report, history_schema.cxx fails to build
|
> > we have a CI setup which fails since 06.05.2025 to build the history_schema.cxx.
> > There was a major change in this code in the commits fe7f6a6 and 159d8d3.
>
|
|
05 May 2025, Konstantin Olchanski, Info, db_delete_key(TRUE)
|
I was working on an odb corruption crash inside db_delete_key() and I noticed
that I did not test db_delete_key() with follow_links set to TRUE. Then I noticed
that nobody nowhere seems to use db_delete_key() with follow_links set to TRUE.
|
|
05 May 2025, Stefan Ritt, Info, db_delete_key(TRUE)
|
I would handle this actually like symbolic links are handled under linux. If you delete a symbolic link, the link gets
detected and NOT the file the link is pointing to.
|
|
05 May 2025, Konstantin Olchanski, Bug Report, abort and core dump in cm_disconnect_experiment()
|
I noticed that some programs like mhist, if they take too long, there is an abort and core dump at the very end. This is because they forgot to
set/disable the watchdog timeout, and they got remove from odb and from the SYSMSG event buffer.
|
|
05 May 2025, Stefan Ritt, Bug Report, abort and core dump in cm_disconnect_experiment()
|
I would be in favor of not curing the symptoms, but fixing the cause of the problem. I guess you put the watchdog disable into mhist, right? Usually mhist
is called locally, so no mserver should be
involved. If not, I would prefer to propagate the watchdog disable to the mserver side as well, if that's not been done already. Actually I never would |