ID |
Date |
Author |
Topic |
Subject |
3056
|
10 Jun 2025 |
Stefan Ritt | Info | History configuration changed |
Today the way the history system is configured has changed. Whenever one adds new equipment
variables, the logger has to add that variable to the history system. Previously, this happened during
startup of the logger and at run start. We have now a case in the Mu3e experiment where we have many
variables and the history configuration takes about 15 seconds, which delays data taking considerably.
After discussion with KO we decided to remove the history re-configuration at run start. This speeds up
the run start considerably, also for other experiments with many history variables. It means however that
once you add/remove/rename any equipment variable going into the history system, you have to restart the
logger for this to become active.
https://bitbucket.org/tmidas/midas/commits/c0a14c2d0166feb6b38c645947f2c5e0bef013d5
Stefan |
3057
|
11 Jun 2025 |
Stefan Ritt | Info | Frontend write cache size |
We had issues with the frontend write cache size and the way it was implemented in the frontend
framework mfe. Specifically, we had two equipments like in the experiment/examples/frontend.cxx, one
trigger event and one periodic event. While the trigger event sets the cache size correctly in the
equipment table, the periodic event (defined afterwards) overwrites the cache size to zero, causing slow
data taking.
The underlying problem is that one event buffer (usually "SYSTEM") can only have one cache size for all
events in one frontend. To avoid mishaps like that, I remove the write cache size from the equipment table
and instead defined a function which explicitly sets the cache size for a buffer. In your frontend_init() you
can now call
set_cache_size("SYSTEM", 10000000);
to set the cache size of the SYSTEM buffer. Note that 10 MiB is the minimum cache size. Especially for
smaller events, this dramatically can speed up your maximal DAQ rate.
Full commit is here:
https://bitbucket.org/tmidas/midas/commits/24fbf2c02037ae5f7db74d0cadab23cd4bfe3b13
Note that this only affects frontends using the mfe framework, NOT for those using the tmfe framework.
Stefan |
3058
|
15 Jun 2025 |
Mark Grimes | Bug Report | Memory leak in mhttpd binary RPC code |
Many thanks for the fix. We've applied and see better memory performance. We still have to kill and restart
mhttpd after a few days however. I think the official fix is missing this part:
diff --git a/src/mjsonrpc.cxx b/src/mjsonrpc.cxx
index 2201d228..38f0b99b 100644
--- a/src/mjsonrpc.cxx
+++ b/src/mjsonrpc.cxx
@@ -3454,6 +3454,7 @@ static MJsonNode* brpc(const MJsonNode* params)
status = cm_connect_client(name.c_str(), &hconn);
if (status != RPC_SUCCESS) {
+ free(buf);
return mjsonrpc_make_result("status", MJsonNode::MakeInt(status));
}
When the other process returns a failure the memory block is also currently leaked. I originally stated "...although I
don't think it contributed to the leak we were seeing" but it seems this was false.
Thanks,
Mark.
> I confirm that MJSON_ARRAYBUFFER does not work correctly for zero-size buffers,
> buffer is leaked in the destructor and copied as NULL in MJsonNode::Copy().
>
> I also confirm memory leak in mjsonrpc "brpc" error path (already fixed).
>
> Affected by the MJSON_ARRAYBUFFER memory leak are "brpc" (where user code returns
> a zero-size data buffer) and "js_read_binary_file" (if reading from an empty
> file, return of "new char[0]" is never freed).
>
> "receive_event" and "read_history" RPCs never use zero-size buffers and are not
> affected by this bug.
>
> mjson commit c798c1f0a835f6cea3e505a87bbb4a12b701196c
> midas commit 576f2216ba2575b8857070ce7397210555f864e5
> rootana commit a0d9bb4d8459f1528f0882bced9f2ab778580295
>
> Please post bug reports a plain-text so I can quote from them.
>
> K.O. |
3059
|
19 Jun 2025 |
Frederik Wauters | Bug Report | add history variables |
I have encounter this a few times
* Make a new history panel
* Use the web GUI to add history variables
* When I am at the "add history variables" panel, there is not scroll option. So
depending on the size and zoom of my screen, some variables further down the list
can not be selected
tried Chrome and Firefox |
3060
|
19 Jun 2025 |
Stefan Ritt | Bug Report | History variables with leading spaces |
I added now code to the logger so it properly complains if there would be a leading space in a variable name.
Stefan
> By accident we had history variables with leading spaces. The history schema check then decides that this is a new variable (the leading space is not read from the history file) and starts a new file. We found this because the run start became slow due to the many, many history files created. It would be nice to just get an error if one has a malformed variable name like this.
>
> How to reproduce: Try to put a variable with a leading space in the name into the history, repeatedly start runs.
> Sugested fix: Produce an error if a history variable has a leading space. |
3061
|
23 Jun 2025 |
Stefan Ritt | Bug Report | Memory leak in mhttpd binary RPC code |
Since this memory leak is quite obvious, I pushed the fix to develop.
Stefan |