21 Mar 2025, Stefan Ritt, Bug Report, please fix compiler warning
|
> Unnamed person who added this clever bit of c++ coding, please fix this compiler warning. Stock g++ on Ubuntu LTS 24.04. Thanks in advance!
>
> /home/olchansk/git/midas/src/system.cxx: In function ‘std::string ss_execs(const char*)’:
> /home/olchansk/git/midas/src/system.cxx:2256:43: warning: ignoring attributes on template argument ‘int (*)(FILE*)’ [-Wignored-attributes]
> 2256 | std::unique_ptr<FILE, decltype(&pclose)> pipe(popen(cmd, "r"), pclose);
>
> K.O.
Replace the code with:
auto pclose_deleter = [](FILE* f) { pclose(f); };
auto pipe = std::unique_ptr<FILE, decltype(pclose_deleter)>(
popen(cmd, "r"),
pclose_deleter
);
Hope this is now warning-free.
Stefan |
21 Mar 2025, Stefan Ritt, Bug Fix, bitbucket builds fixed
|
> bitbucket automatic builds were broken after mfe.cxx started printing some additional messages added in commit
> https://bitbucket.org/tmidas/midas/commits/0ae08cd3b96ebd8e4f57bfe00dd45527d82d7a38
>
> this is now fixed. to check if your changes will break automatic builds, before final push, please do:
>
> make clean
> make mini -j
> make cmake -j
> make test
Unfortunately we will break the automatic build each time a program outputs one different character, which even might happen if we add a line of code and
a cm_msg() gets produced with a different line number. Is there a standard way to update testexpt.example (like "make testexpt" or so). Should be trigger
the update of testexpt.example before each commit via a hook?
Stefan |
21 Mar 2025, Stefan Ritt, Bug Report, please fix mscb compiler warning
|
I hopefully fixed the waring (narrowing down from size_t to int). Please double check with your compiler.
Stefan |
21 Mar 2025, Stefan Ritt, Forum, LabView-Midas interface
|
> Hello,
>
> Does anyone have experience with writing a MIDAS frontends to communicate with a device that operates using LabView (e.g. superconducting magnets, cryostats etc.). Any information or experience regarding this would be highly appreciated.
>
> thanks,
> Zaher
We do have a superconducting magnet from Cryogenic, UK, which comes with a LabView control program on a Windows PC. I did the only reasonable with this: trash it in the waste basket. Do NOT use Labveiw for anything which should run more than 24h in a row. Too many bad experiences with LabView control programs
for separators at PSI and other devices. Instead of the Windows PC, we use MSCB devices and RasperryPis to communicate with the power supply directly, which has been proven to be much more stable (running for years without crashes). I'm happy to share our code with you.
Stefan |
01 Apr 2025, Stefan Ritt, Suggestion, Sequencer ODBSET feature requests
|
A new sequencer which understands Python is in the works. There you can use all features from that language.
Stefan |
01 Apr 2025, Stefan Ritt, Suggestion, Sequencer ODBSET feature requests
|
The extended ODBSET[x,y1-y2,z] could make sense to be implemented, since it then will match the alarm system which uses the same syntax.
The $GOODCHANNELS/$BADCHANNELS is however a very strange syntax which I haven't seen in any other computer language. It would take me probably several days to properly implement this, while it would take you much less time to explicitly use a few ODBSET statements to set the bad channels to zero.
For the file edit workflow, the author of the editor will have a look.
Stefan
Lukas Gerritzen wrote: | I would like to request the following sequencer features if you find the ideas as sensible as I do:
- A "Reload File" button
- Support for patterns in ODBSET, e.g.:
-
ODBSET "/Path/value[1,3,5]", 1 -
ODBSET "/Path/value[1-5,7-9]", 1 - Arbitrary combinations of the above
- Support for variable substitution:
-
SET GOODCHANNELS, "1-5,7,9"; ODBSET "/Path/value[$GOODCHANNELS]", 1 -
SET BADCHANNELS, "6,8"; ODBSET "/Path/value[!$BADCHANNELS]", 1 -
ODBSET "/Path/value[0-100, except $BADCHANNELS]", 1
|
|
02 Apr 2025, Stefan Ritt, Suggestion, Sequencer ODBSET feature requests
|
And there is one more argument:
We have a Python expert in our development team who wrote already the Python-to-C bindings. That means when running a Python
script, we can already start/stop runs, write/read to the ODB etc. We only have to get the single stepping going which seems feasible to
me, since there are some libraries like inspect.currentframe() and traceback.extract_stack(). For single-stepping there are debug APIs
like debugpy. With Lua we really would have to start from scratch.
Stefan |
07 Apr 2025, Stefan Ritt, Suggestion, Sequencer ODBSET feature requests
|
If people are simultaneously editing scripts this is indeed an issue, which probably never can be resolved by technical means. It need communication between the users.
For the main script some ODB locking might look like:
- First person clicks on "Edit", system checks that file is not locked and sequencer is not running, then goes into edit mode
- When entering edit mode, the editor puts a lock in to the ODB, like "Scrip lock = pc1234".
- When another person clicks on "Edit", the system replies "File current being edited on pc1234"
- When the first person saves the file or closes the web browser, the lock gets removed.
- Since a browser can crash without removing a lock, we need some automatic lock recovery, like if the lock is there, the next users gets a message "file currently locked. Click "override" to "steal" the lock and edit the file".
All that is not 100% perfect, but will probably cover 99% of the cases.
There is still the problem on all other scripts. In principle we would need a lock for each file which is not so simple to implement (would need arrays of files and host names).
Another issue will arise if a user opens a file twice for editing. The second attempt will fail, but I believe this is what we want.
A hostname for the lock is the easiest we can get. Would be better to also have a user name, but since the midas API does not require a log in, we won't have a user name handy. At it would be too tedious to ask "if you want to edit this file, enter your username".
Just some thoughts.
Stefan |
30 Apr 2025, Stefan Ritt, Info, New ODB++ API
|
I had to change the ODBXX API: https://bitbucket.org/tmidas/midas/commits/273c4e434795453c0c6bceb46bac9a0d2d27db18
The old C API is case-insensitive, meaning db_find_key("name") returns a key "name" or "Name" or "NAME". We can discuss if this is good or bad, but that's how it is since 30 years.
I now realized the the ODBXX API keys is case sensitive, so a o["NAME"] does not return any key "name". Rather, it tries to create a new key which of course fails. I changed therefore
the ODBXX to become case-insensitive like the old C API.
Stefan |
30 Apr 2025, Stefan Ritt, Bug Report, ODBXX : ODB links in the path names ?
|
Indeed this was missing from the very beginning. I added it, please report back if it's not working.
Stefan |
05 May 2025, Stefan Ritt, Bug Report, abort and core dump in cm_disconnect_experiment()
|
I would be in favor of not curing the symptoms, but fixing the cause of the problem. I guess you put the watchdog disable into mhist, right? Usually mhist is called locally, so no mserver should be
involved. If not, I would prefer to propagate the watchdog disable to the mserver side as well, if that's not been done already. Actually I never would disable the watchdog, but set it to a reasonable
maximal value, like a few minutes or so. In that case, the client gets still removed if it crashes for some reason.
My five cents,
Stefan |
05 May 2025, Stefan Ritt, Info, db_delete_key(TRUE)
|
I would handle this actually like symbolic links are handled under linux. If you delete a symbolic link, the link gets
detected and NOT the file the link is pointing to.
So I conclude that the "follow links" is a misconception and should be removed.
Stefan |
26 May 2025, Stefan Ritt, Forum, Reading two devices in parallel
|
> Dear experts,
> in the CYGNO experiment, we readout CMOS cameras for optical readout of GEM-TPCs. So far, we only developed the readout for a single camera. In the future, we will have multiple cameras to read out (up to 18 in the next phase of the experiment), and we are investigating how to optimize the readout by exploiting parallelization.
>
> One idea is to start parallel threads within a single equipment. Alternatively, one could associate different equipment with each camera and run an Event Builder. Perhaps other solutions did not come to mind. Which one would you regard as the most effective and elegant?
>
> Thank you very much,
> Francesco
In principle both will work. It's kind of matter of taste. In the multi-threaded approach one has a single frontend to start and stop, and in the second case you have to start 18 individual frontends and make sure that they are running.
For the multi-threaded frontend you have to ensure proper synchronization between the threads (like common run start/stop), and in the end you also have to do some event building, sending all 18 streams into a single buffer. As you know, multi-thread programming can be a bit of an art, using mutexes or semaphores, but it can be more flexible as the event builder which is a given piece of software.
Best,
Stefan |
10 Jun 2025, Stefan Ritt, Info, use of modified image showing MIDAS data format?
|
> Hello, I'm currently writing a paper about making a dataset publicly available
> from the University of Minnesota and it's a MIDAS dataset.
>
> I'd like to use an image that shows the MIDAS data format (that's been slightly
> modified to fit in the paper) and am wondering (1) if I could get permission to do
> so and (2) what the preferred attribution would be.
Feel free to use whatever you like, the documentation is on the same open source license as midas itself.
Stefan |
10 Jun 2025, Stefan Ritt, Info, History configuration changed
|
Today the way the history system is configured has changed. Whenever one adds new equipment
variables, the logger has to add that variable to the history system. Previously, this happened during
startup of the logger and at run start. We have now a case in the Mu3e experiment where we have many
variables and the history configuration takes about 15 seconds, which delays data taking considerably.
After discussion with KO we decided to remove the history re-configuration at run start. This speeds up
the run start considerably, also for other experiments with many history variables. It means however that
once you add/remove/rename any equipment variable going into the history system, you have to restart the
logger for this to become active.
https://bitbucket.org/tmidas/midas/commits/c0a14c2d0166feb6b38c645947f2c5e0bef013d5
Stefan |
11 Jun 2025, Stefan Ritt, Info, Frontend write cache size
|
We had issues with the frontend write cache size and the way it was implemented in the frontend
framework mfe. Specifically, we had two equipments like in the experiment/examples/frontend.cxx, one
trigger event and one periodic event. While the trigger event sets the cache size correctly in the
equipment table, the periodic event (defined afterwards) overwrites the cache size to zero, causing slow
data taking.
The underlying problem is that one event buffer (usually "SYSTEM") can only have one cache size for all
events in one frontend. To avoid mishaps like that, I remove the write cache size from the equipment table
and instead defined a function which explicitly sets the cache size for a buffer. In your frontend_init() you
can now call
set_cache_size("SYSTEM", 10000000);
to set the cache size of the SYSTEM buffer. Note that 10 MiB is the minimum cache size. Especially for
smaller events, this dramatically can speed up your maximal DAQ rate.
Full commit is here:
https://bitbucket.org/tmidas/midas/commits/24fbf2c02037ae5f7db74d0cadab23cd4bfe3b13
Note that this only affects frontends using the mfe framework, NOT for those using the tmfe framework.
Stefan |
19 Jun 2025, Stefan Ritt, Bug Report, History variables with leading spaces
|
I added now code to the logger so it properly complains if there would be a leading space in a variable name.
Stefan
> By accident we had history variables with leading spaces. The history schema check then decides that this is a new variable (the leading space is not read from the history file) and starts a new file. We found this because the run start became slow due to the many, many history files created. It would be nice to just get an error if one has a malformed variable name like this.
>
> How to reproduce: Try to put a variable with a leading space in the name into the history, repeatedly start runs.
> Sugested fix: Produce an error if a history variable has a leading space. |
23 Jun 2025, Stefan Ritt, Bug Report, Memory leak in mhttpd binary RPC code
|
Since this memory leak is quite obvious, I pushed the fix to develop.
Stefan |
21 Jul 2025, Stefan Ritt, Bug Report, Default write cache size for new equipments breaks compatibility with older equipments
|
> Perhaps have:
>
> set_write_cache_size("SYSTEM", 0);
> set_write_cache_size("BUF1", bigsize);
>
> with an internal std::map<std::string,size_t>; for write cache size for each named buffer
Ok, this is implemented now in mfed.cxx and called from examples/experiment/frontend.cxx
Stefan |
13 Oct 2020, Soichiro Kuribayashi, Info, About remote control of front end part of MIDAS on chip
|
Hello!
My name is Soichiro Kuribayashi and I am a Ph.D. student at Kyoto University.
I'm a T2K collaborator and working for Super FGD which is new detector in ND280.
I'm a beginner of MIDAS and I've just started to develop the DAQ software with
MIDAS for Super FGD.
For the DAQ of Super FGD, we will run remotely front end part of MIDAS on ZYNQ
which is system on chip.
For this remote control of front end part with mserver, we have to mount home
directory of DAQ PC(Cent OS8) on that of Linux on ZYNQ.
So I wonder if we should use NFS(Network file system) + NIS(Network information
service) + autofs for the mounting. Is it correct?
If you have any information or any suggestion for the remote control on chip,
please let me know.
Best regards,
Soichiro |
|