 16 Mar 2020, Konstantin Olchanski, Release, midas-2020-03-a
|
midas-2020-03-a is here.
Accumulated changes and bug fixes since last tag midas-2019-09-i.
|
 22 May 2020, Konstantin Olchanski, Release, midas-2020-03-a
|
> midas-2020-03-a is here.
> checkout the top of branch release/midas-2020-03 (recommended) or
> checkout the tag midas-2020-03-a.
|
12 May 2020, Stefan Ritt, Info, New ODB++ API 
|
Since the beginning of the lockdown I have been working hard on a new object-oriented interface to the online database ODB. I have the code now in an initial
state where it is ready for
testing and commenting. The basic idea is that there is an object midas::odb, which represents a value or a sub-tree in the ODB. Reading, writing and watching |
|
20 May 2020, Konstantin Olchanski, Info, New ODB++ API
|
> midas::odb o;
> o["foo"] = 1;
|
|
20 May 2020, Stefan Ritt, Info, New ODB++ API
|
In meanwhile, there have been minor changes and improvements to the API:
Previously, we had:
|
|
20 May 2020, Pintaudi Giorgio, Info, New ODB++ API
|
All this is very good news. I really wish this were available some months ago: it would have helped me immensely. The old C API was clunky at best.
I really like the idea and looking forward to using it (even if at the moment I do not have the need to) ... |
|
20 May 2020, Konstantin Olchanski, Info, New ODB++ API
|
> All this is very good news. I really wish this were available some months ago: it would have helped me immensely. The old C API was clunky at best.
> I really like the idea and looking forward to using it (even if at the moment I do not have the need to) ...
|
|
12 May 2020, Ruslan Podviianiuk, Forum, List of sequencer files
|
Hello,
We are going to implement a list of sequencer files to allow users to select one
|
|
13 May 2020, Stefan Ritt, Forum, List of sequencer files
|
If you load a file into the sequencer from the web interface, you get a list of all files in that directory.
This basically gives you a list of possible sequencer files. It's even more powerful, since you can
create subdirectories and thus group the sequencer files. Attached an example from our
|
|
18 May 2020, Ruslan Podviianiuk, Forum, List of sequencer files
|
> If you load a file into the sequencer from the web interface, you get a list of all files in that directory.
> This basically gives you a list of possible sequencer files. It's even more powerful, since you can
> create subdirectories and thus group the sequencer files. Attached an example from our
|
|
19 May 2020, Ruslan Podviianiuk, Forum, List of sequencer files
|
> If you load a file into the sequencer from the web interface, you get a list of all files in that directory.
> This basically gives you a list of possible sequencer files. It's even more powerful, since you can
> create subdirectories and thus group the sequencer files. Attached an example from our
|
|
20 May 2020, Konstantin Olchanski, Forum, List of sequencer files
|
>
> We have a custom webpage and trying to get list of files from the custom webpage and need jrpc command to show it
> in custom page. Is there a jrpc command to get this file list?
|
|
07 May 2020, Estelle, Bug Report, Conflic between Rootana and midas about the redefinition of TID_xxx data types
|
Dear Midas and Rootana people,
We have tried to update our midas DAQ with the new TID definitions describe in https://midas.triumf.ca/elog/Midas/1871
|
|
20 May 2020, Konstantin Olchanski, Bug Report, Conflic between Rootana and midas about the redefinition of TID_xxx data types
|
> Dear Midas and Rootana people,
>
> We have tried to update our midas DAQ with the new TID definitions describe in https://midas.triumf.ca/elog/Midas/1871
|
|
01 May 2020, Joseph McKenna, Forum, Taking MIDAS beyond 64 clients
|
Hi all,
|
|
01 May 2020, Stefan Ritt, Forum, Taking MIDAS beyond 64 clients
|
Hi Joseph,
here some thoughts from my side:
|
|
01 May 2020, Pierre Gorel, Forum, Taking MIDAS beyond 64 clients
|
> - On the other hand, if we have to break compatibility, now is maybe a good time since most accelerators worldwide are off. But before doing so, I would
like to get feedback from the main experiments
> around the world (MEG, T2K, g-2, DEAP besides ALPHA).
|
|
02 May 2020, Stefan Ritt, Forum, Taking MIDAS beyond 64 clients
|
TRIUMF stayed quiet, probably they have other things to do.
I allowed myself to move the maximum number of clients back to its original value, in order not to break running experiments.
|
|
02 May 2020, Joseph McKenna, Forum, Taking MIDAS beyond 64 clients
|
Thank you very much for feedback.
|
|
02 May 2020, Stefan Ritt, Forum, Taking MIDAS beyond 64 clients
|
> Perhaps a item for future discussion would be for the odbinit program to be able to 'upgrade' the ODB and enable some backwards
> compatibility.
|
|
02 May 2020, Konstantin Olchanski, Forum, Taking MIDAS beyond 64 clients
|
>
> Does the community here have strong opinions about increasing the
> MAX_CLIENTS and MAX_RPC_CONNECTION limits?
|
|
02 May 2020, Konstantin Olchanski, Forum, Taking MIDAS beyond 64 clients
|
> >
> > Does the community here have strong opinions about increasing the
> > MAX_CLIENTS and MAX_RPC_CONNECTION limits?
|
|
02 May 2020, Konstantin Olchanski, Forum, Taking MIDAS beyond 64 clients
|
> > >
> > > Does the community here have strong opinions about increasing the
> > > MAX_CLIENTS and MAX_RPC_CONNECTION limits?
|
|
03 Apr 2020, Francesco Renga, Info, CLOCK_REALTIME on MacOS
|
Dear all,
I'm trying to compile MIDAS on MacOS 10.10 and I get this error:
|
|
03 Apr 2020, Stefan Ritt, Info, CLOCK_REALTIME on MacOS
|
> Dear all,
> I'm trying to compile MIDAS on MacOS 10.10 and I get this error:
>
|
|
25 Apr 2020, Konstantin Olchanski, Info, CLOCK_REALTIME on MacOS
|
> > /Users/francesco/MIDAS/midas/src/system.cxx:3187:18: error: use of undeclared identifier
> > 'CLOCK_REALTIME'
> > clock_settime(CLOCK_REALTIME, <m);
|
|
26 Apr 2020, Stefan Ritt, Info, CLOCK_REALTIME on MacOS
|
> > > /Users/francesco/MIDAS/midas/src/system.cxx:3187:18: error: use of undeclared identifier
> > > 'CLOCK_REALTIME'
> > > clock_settime(CLOCK_REALTIME, <m);
|
|
26 Apr 2020, Yu Chen (SYSU), Forum, Questions and discussions on the Frontend ODB tree structure.
|
Dear MIDAS developers and colleagues,
This is Yu CHEN of School of Physics, Sun Yat-sen University, China, working in the PandaX-III collaboration, an experiment under development to search |
|
26 Apr 2020, Stefan Ritt, Forum, Questions and discussions on the Frontend ODB tree structure.
|
Dear Yu Chen,
in my opinion, you can follow two strategies:
|
|
25 Apr 2020, Konstantin Olchanski, Info, new mac!
|
I received my new 2020 mac book air, so between Stefan and myself, MacOS support for
MIDAS is assured for 5 more years at the least. K.O. |
|
07 Apr 2020, Ivo Schulthess, Suggestion, Sequencer loop break
|
I am using the Midas sequencer to run subsequent measurements in a loop, without
knowing how many iterations in advance. Therefore, I am using the "infinity"
option. Since I have other commands after the loop, it would be nice to have the
|
|
21 Apr 2020, Stefan Ritt, Suggestion, Sequencer loop break
|
> I am using the Midas sequencer to run subsequent measurements in a loop, without
> knowing how many iterations in advance. Therefore, I am using the "infinity"
> option. Since I have other commands after the loop, it would be nice to have the
|
|
23 Apr 2020, Ivo Schulthess, Suggestion, Sequencer loop break
|
> You can do that with the "GOTO" statement, jumping to the first line after the loop.
>
> Here is a working example:
|
|
23 Apr 2020, Stefan Ritt, Suggestion, Sequencer loop break
|
> > You can do that with the "GOTO" statement, jumping to the first line after the loop.
> >
> > Here is a working example:
|
|
25 Apr 2020, Konstantin Olchanski, Suggestion, Sequencer loop break
|
> LOOP runs, 5
> ...
> ENDLOOP
|
|
16 Mar 2020, Konstantin Olchanski, Info, mhttpd mongoose 6.16 update
|
the update of mhttpd to mongoose version 6.16 was committed to the develop branch of midas. If you do not want to use this
updated code or if it causes problems, please use the mhttpd6 executable or midas from the midas-2020-03 release branch.
|
|
16 Mar 2020, Konstantin Olchanski, Info, mhttpd mongoose 6.16 update
|
> the update of mhttpd to mongoose version 6.16 was committed to the develop branch of midas.
The new code implements 3 http ports:
|
|
16 Mar 2020, Konstantin Olchanski, Info, mhttpd mongoose 6.16 update
|
> > the update of mhttpd to mongoose version 6.16 was committed to the develop branch of midas.
Configuration is done by ODB /WebServer:
|
|
17 Mar 2020, Konstantin Olchanski, Info, mbedtls, mhttpd mongoose 6.16 update
|
> > > the update of mhttpd to mongoose version 6.16 was committed to the develop branch of midas.
current code looks for the mbedtls library in ../mbedtls (next to midas)
|
|
30 Mar 2020, Stefan Ritt, Info, mbedtls, mhttpd mongoose 6.16 update
|
I had some quick look at the new mongoose code and didn't find anything I dislike. Did a quick test of the proxy which worked and is nice to have.
Agree with all KO said about authentication.
|
|
25 Mar 2020, Andreas Suter, Forum, mlogger: misleading error messages for ROOT
|
Dear All,
At our experiment we write ROOT files. When starting/stopping runs we get the following error messages:
|
|
25 Mar 2020, Konstantin Olchanski, Forum, mlogger: misleading error messages for ROOT
|
> [Logger,ERROR] [mlogger.cxx:3358:root_write,ERROR] Cannot write system event into ROOT file, event_id 0xffff8000
Hi, Andreas, please open a bug report for this problem on bitbucket, there is now at least 2 bugs against
|
|
27 Mar 2020, Stefan Ritt, Forum, mlogger: misleading error messages for ROOT
|
Dear simplest solution seems to me to just remove the error message generation and silently ignore the BOE EOR events.
Committed that change.
|
|
27 Mar 2020, Andreas Suter, Forum, mlogger: misleading error messages for ROOT
|
Hi Stefan,
I think this only partially resolves the issue, in log_write:
|
|
27 Mar 2020, Stefan Ritt, Forum, mlogger: misleading error messages for ROOT
|
Ok, changed.
Stefan |
|
23 Mar 2020, Ivo Schulthess, Forum, Save data to FTP
|
Dear all
I try to save data to an FTP server but don't get any data on the server. Midas does not complain or message any error but also nothing gets saved. Does
somebody have experience with this? I use the following settings for the ODB mlogger channel settings: Type: FTP, Filename: server.com, 21, user, pw, ., |
|
23 Mar 2020, Konstantin Olchanski, Forum, Save data to FTP
|
> I try to save data to an FTP server but don't get any data on the server. Midas does not complain or message any error but also nothing gets saved. Does
somebody have experience with this? I use the following settings for the ODB mlogger channel settings: Type: FTP, Filename: server.com, 21, user, pw, .,
run%06d.mid, Format: MIDAS, Output: FILE. What would be the Output: FTP setting for? I tried this but it does not work at all.
|
|
24 Mar 2020, Ivo Schulthess, Forum, Save data to FTP
|
> > I try to save data to an FTP server but don't get any data on the server. Midas does not complain or message any error but also nothing gets saved.
Does somebody have experience with this? I use the following settings for the ODB mlogger channel settings: Type: FTP, Filename: server.com, 21, user,
pw, ., run%06d.mid, Format: MIDAS, Output: FILE. What would be the Output: FTP setting for? I tried this but it does not work at all.
|
|
24 Mar 2020, Stefan Ritt, Forum, Save data to FTP
|
Logging directly from the midas logger to FTP is a bit cumbersome. In case of delays during login etc. this can throttle the whole DAQ chain.
What we use in our lab is to write to local disk, then use the lazylogger (https://midas.triumf.ca/MidasWiki/index.php/Lazylogger) to copy the
local files to a remote FTP server. This way we de-couple data taking from backup, making the system much more swift.
|
|
24 Mar 2020, Ivo Schulthess, Forum, Save data to FTP
|
> Logging directly from the midas logger to FTP is a bit cumbersome. In case of delays during login etc. this can throttle the whole DAQ chain.
> What we use in our lab is to write to local disk, then use the lazylogger (https://midas.triumf.ca/MidasWiki/index.php/Lazylogger) to copy the
> local files to a remote FTP server. This way we de-couple data taking from backup, making the system much more swift.
|
|
24 Mar 2020, Konstantin Olchanski, Forum, Save data to FTP
|
>
> Since ILL only provides access via SFTP and everything else is not existent or blocked (not even ssh is possible),
> this is the only thing we can work with by now.
|
|
08 Aug 2019, Konstantin Olchanski, Info, MIDAS will use C++11
|
After much discussion, and following the MIDAS workshop at TRIUMF, we made the decision to use C++11 in MIDAS.
There are many benefits, and only one drawback - no c++11 compilers in the default OS install on older computers (i.e.
|
|
16 Mar 2020, Konstantin Olchanski, Info, MIDAS will use C++11
|
> After much discussion, and following the MIDAS workshop at TRIUMF, we made the decision to use C++11 in MIDAS.
>
> There are many benefits, and only one drawback - no c++11 compilers in the default OS install on older computers (i.e.
|
|
16 Mar 2020, Pintaudi Giorgio, Info, MIDAS will use C++11
|
About the boost library, that is exactly
what I did for a project of mine (the
calibration software for the WAGASCI
|
|
10 Mar 2020, Konstantin Olchanski, Info, MIDAS vs JSROOT web pages
|
Just FYI, I am looking at the ROOT web programming component JSROOT and I notice that the RPC mechanism quite different from the JSON-
RPC I implemented for MIDAS.
|
|
06 Mar 2020, Lars Martin, Forum, RPC error
|
I ported a bunch of frontends to C++ and now I'm occasionally getting this RPC
error message:
|
|
08 Mar 2020, Konstantin Olchanski, Forum, RPC error
|
I do not see this error, but there was one more report (they did not clearly say what http errors
they see) https://bitbucket.org/tmidas/midas/issues/209/get-rid-of-mjsonrpc-dialogs-put-it-to-
the
|
|
29 Jan 2020, Berta Beltran, Bug Report, Compiling Midas in OS 10.15 Catalina
|
Hi all,
I have updated our daq computer to the latest OS 10.15 with the idea that then I will get all our daq
|
|
02 Feb 2020, Konstantin Olchanski, Bug Report, Compiling Midas in OS 10.15 Catalina
|
> I have updated our daq computer to the latest OS 10.15 ...
FWIW, I do not have macos 10.15. I have 10.13 at home and 10.14 in the office. Maybe Stefan has it?
|
|
03 Feb 2020, Stefan Ritt, Bug Report, Compiling Midas in OS 10.15 Catalina
|
> > I have updated our daq computer to the latest OS 10.15 ...
>
> FWIW, I do not have macos 10.15. I have 10.13 at home and 10.14 in the office. Maybe Stefan has it?
|
|
04 Feb 2020, Konstantin Olchanski, Bug Report, Compiling Midas in OS 10.15 Catalina
|
> > > I have updated our daq computer to the latest OS 10.15 ...
> >
> > FWIW, I do not have macos 10.15. I have 10.13 at home and 10.14 in the office. Maybe Stefan has it?
|
|
06 Feb 2020, Berta Beltran, Bug Report, Compiling Midas in OS 10.15 Catalina
|
>
> Ok, in this case, I will update my office mac mini to 10.15.
|
|
10 Feb 2020, Konstantin Olchanski, Bug Report, Compiling Midas in OS 10.15 Catalina
|
> Any luck with Midas in OS 10.15?
Best I can tell, the problem is not in midas: pthread.h should be there, somewhere.
|
|
11 Feb 2020, Berta Beltran, Bug Report, Compiling Midas in OS 10.15 Catalina
|
> > Any luck with Midas in OS 10.15?
>
> Best I can tell, the problem is not in midas: pthread.h should be there, somewhere.
|
|
11 Feb 2020, Stefan Ritt, Bug Report, Compiling Midas in OS 10.15 Catalina
|
For your reference, here on my MacOSX 10.14.6 with XCode 11.3.1 the pthread.h file is present in locations listed below.
Did you execute "xcode-select --install" ?
|
|
12 Feb 2020, Berta Beltran, Bug Report, Compiling Midas in OS 10.15 Catalina
|
> For your reference, here on my MacOSX 10.14.6 with XCode 11.3.1 the pthread.h file is present in locations listed below.
>
> Did you execute "xcode-select --install" ?
|
|
12 Feb 2020, Stefan Ritt, Bug Report, Compiling Midas in OS 10.15 Catalina
|
Another thought: Can you delete the midas build directory and run cmake again? Like
$ cd midas/build
|
|
12 Feb 2020, Berta Beltran, Bug Report, Compiling Midas in OS 10.15 Catalina
|
> Another thought: Can you delete the midas build directory and run cmake again? Like
>
> $ cd midas/build
|
|
13 Feb 2020, Stefan Ritt, Bug Report, Compiling Midas in OS 10.15 Catalina
|
Now you are stuck with openssl, which is optional for mhttpd. If you only use mhttpd locally, you maybe don't need SSL support. In that case you can jus
do
|
|
13 Feb 2020, Berta Beltran, Bug Report, Compiling Midas in OS 10.15 Catalina
|
> Now you are stuck with openssl, which is optional for mhttpd. If you only use mhttpd locally, you maybe don't need SSL support. In that case you can jus
do
>
|
|
28 Feb 2020, Konstantin Olchanski, Bug Report, Compiling Midas in OS 10.15 Catalina
|
> > [midas/build] $ cmake -D NO_SSL=1 ..
> If I run the compilation with the flag NO_SSL it works just fine. ...
|
|
03 Mar 2020, Berta Beltran, Bug Report, Compiling Midas in OS 10.15 Catalina
|
Thanks Konstantin,
I will keep an eye for the next release so that I can update my Midas to include ssl libraries.
|
|
13 Feb 2020, Marius Koeppel, Forum, Writting Midas Events via FPGAs
|
Dear all,
we creating Midas events directly inside a FPGA and send them off via DMA into the PC RAM. For reading out this RAM via Midas the FPGA sends as a pointer |
|
13 Feb 2020, Stefan Ritt, Forum, Writting Midas Events via FPGAs
|
The rb_xxx function are (thoroughly tested!) robust against high data rate given that you use them as intended:
1) Once you create the ring buffer via rb_create(), specify the maximum event size (overall event size, not bank size!). Later there is no protection any |
|
14 Feb 2020, Konrad Briggl, Forum, Writting Midas Events via FPGAs
|
Hello Stefan,
is there a difference for the later data processing (after writing the ring buffer blocks)
if we write single events or multiple in one rb_get_wp - memcopy - rb_increment_wp cycle?
|
|
14 Feb 2020, Stefan Ritt, Forum, Writting Midas Events via FPGAs
|
rb_xxx functions are midas event agnostic. The receiving side in mfe.cxx (lines 1418 in receive_trigger_event) however pulls one event at a time. If you
have some inconsistency I would put some debugging code there.
|
|
20 Feb 2020, Konstantin Olchanski, Forum, Writting Midas Events via FPGAs
|
> rb_xxx functions are midas event agnostic. The receiving side in mfe.cxx (lines 1418 in receive_trigger_event) however pulls one event at a time. If you
> have some inconsistency I would put some debugging code there.
|
|
20 Feb 2020, Marius Koeppel, Forum, Writting Midas Events via FPGAs
|
We also agree and found the problem now. Since we build everything (MIDAS Event Header, Bank Header, Banks etc.) in the FPGA we had some struggle with
the MIDAS data format (http://lmu.web.psi.ch/docu/manuals/bulk_manuals/software/midas195/html/AppendixA.html). We thought that only the MIDAS Event needs |
|
20 Feb 2020, Stefan Ritt, Forum, Writting Midas Events via FPGAs
|
Actually the cause of all of the is a real bug in the midas functions. We want each bank 8-byte aligned, so there is code in bk_close like:
midas.cxx:14788:
|
|
21 Feb 2020, Konstantin Olchanski, Forum, Writting Midas Events via FPGAs
|
Hi, Stefan - is this our famous 64-bit misalignement? Where we have each alternating bank aligned and misaligned at 64 bits? Without changing the data
format, one can always store data in 64-bit aligned banks by inserting a dummy banks between real banks:
|
|
21 Feb 2020, Stefan Ritt, Forum, Writting Midas Events via FPGAs
|
> Hi, Stefan - is this our famous 64-bit misalignement? Where we have each alternating bank aligned and misaligned at 64 bits? Without changing the data
> format, one can always store data in 64-bit aligned banks by inserting a dummy banks between real banks:
>
|
|
21 Feb 2020, Konstantin Olchanski, Forum, Writting Midas Events via FPGAs
|
> We also agree and found the problem now.
Good. what was wrong?
|
|
20 Feb 2020, Marius Koeppel, ,
|
We also agree and found the problem now. Since we build everything (MIDAS Event Header, Bank Header, Banks etc.) in the FPGA we had some struggle with
the MIDAS data format (http://lmu.web.psi.ch/docu/manuals/bulk_manuals/software/midas195/html/AppendixA.html). We thought that only the MIDAS Event needs |
|
12 Feb 2020, Marius Koeppel, Forum, Difference between "Event Data Size" and "All Bank Size"
|
Dear all,
we are trying to build Midas events on FPGAs and send them directly to the midas
|
|
12 Feb 2020, Stefan Ritt, Forum, Difference between "Event Data Size" and "All Bank Size"
|
Thanks for pointing out this error. The "All Bank Size" contains the size of all banks including their
bank headers, but NOT the global bank header itself. I modified the documentation accordingly.
|
|
20 Feb 2020, Konstantin Olchanski, Forum, Difference between "Event Data Size" and "All Bank Size"
|
> Thanks for pointing out this error. The "All Bank Size" contains the size of all banks including their
> bank headers, but NOT the global bank header itself. I modified the documentation accordingly.
>
|
|
18 Feb 2020, Lukas Gerritzen, Bug Report, RPC Error: ACK or other control chars from "db_get_values"
|
Hi,
for some reason we occasionally get JSON errors in the browser when accessing MIDAS. It is then not possible to open a new window or tab, see attachment.
The unexpected token is \0x6, so the acknowledge symbol.
|
|
18 Feb 2020, Stefan Ritt, Bug Report, RPC Error: ACK or other control chars from "db_get_values"
|
You are the first one reporting this error, so it must be due to your values in the ODB. Can you track it down to specific ODB contents? If so, can you
post it so that I can reproduce your error?
|
|
20 Feb 2020, Konstantin Olchanski, Bug Report, RPC Error: ACK or other control chars from "db_get_values"
|
> The unexpected token is \0x6
> RPC Error json parser exception: SyntaxError: JSON.parse: bad control character in string literal at line 80 column 30 of the JSON data, method: "db_get_valus",
params: [object Object], id: 1582020074098.
|