ID |
Date |
Author |
Topic |
Subject |
1887
|
25 Apr 2020 |
Konstantin Olchanski | Info | new mac! | I received my new 2020 mac book air, so between Stefan and myself, MacOS support for
MIDAS is assured for 5 more years at the least. K.O. |
1886
|
25 Apr 2020 |
Konstantin Olchanski | Info | CLOCK_REALTIME on MacOS | > > /Users/francesco/MIDAS/midas/src/system.cxx:3187:18: error: use of undeclared identifier
> > 'CLOCK_REALTIME'
> > clock_settime(CLOCK_REALTIME, <m);
> >
> > Is it related to my (old) version of MacOS? Can I fix it somehow?
I think the "set clock" function is a holdover from embedded operating systems
that did not keep track of clock time, i.e. VxWorks, and similar. Here a midas program
will get the time from the mserver and set it on the local system. Poor man's ntp,
poor man's ntpd/chronyd.
We should check if this function is called by anything, and if nothing calls it, maybe remove it?
K.O. |
1885
|
25 Apr 2020 |
Konstantin Olchanski | Suggestion | Sequencer loop break | > LOOP runs, 5
> ...
> ENDLOOP
Classical loop constructs usually include "break" (exit the loop) and "continue" (loop again!)
constructs. I would say it is an unfortunate omission if they are not present in the midas sequencer.
But Stefan is right, of course, both commands are just funny names for "goto".
K.O. |
1884
|
25 Apr 2020 |
Konstantin Olchanski | Forum | API to read MIDAS format file | <p>[quote="Pintaudi Giorgio"]Dear MIDAS people, I need to borrow your
wisdom for a bit. I am developing a piece of software that should read the history data
stored in a [FONT=Times New Roman].midas[/FONT] file (MIDAS format) and integrate it
into the WAGASCI data quality output. In other words, I need to read some temperature
values stored in a [FONT=Times New Roman].midas[/FONT] file and compare them with
the MPPC gains and check for temperature/gain dependence. I see three possibilities:
[LIST] [*] write a custom parser in C++ using the instructions contained in the
[URL=https://midas.triumf.ca/MidasWiki/index.php/Mhformat]Mhformat page[/URL]; [*] call
the mhist program from within my application; [*] call the mhdump program from within my
application; [/LIST] [B]Which solution do you think is the best?[/B] Because there is no
need for raw performance, if possible, I would like to write my application in Python3 but
C++ is also an option. [/quote]</p>
(Please write messages in plain text format, thank you)
The format of .hst midas history files is pretty simple and mhdump.cxx is an easy to read
illustration on how to read it from basic principles (without going through the midas library,
which can be somewhat complicated). The newer "FILE" format for history is even simpler
to read because it is just fixed-record-size binary data prepended by a text header.
You can also use the mh2sql program to import history data into an sql database (mysql
and sqlite should work) or to convert .hst files to "FILE" format files. This works well
for "archiving" history data, because the "FILE" format works better for looking at old data,
and for looking at data in "months" or "years" timescale.
Back to your question, you can certainly use "mhdump" as is, using a pipe (popen()), or
you can package mhdump.cxx as a c++ class and use it in your application. If you go this
route, your contribution of such a c++ class back to midas would be very welcome.
You can also use mhist, but the mhist code cannot be trivially packaged as a c++ class
to use in your application.
You can also suggest that we write an easier to use history utility, we are always open to
suggested improvements.
Let us know how it works out for you. Good luck!
K.O. |
1883
|
24 Apr 2020 |
Stefan Ritt | Forum | API to read MIDAS format file |
Pintaudi Giorgio wrote: |
Hypothetically which one between the two lends itself the better to being "batched"? I mean to be read and controlled by a program/routine. For example, some programs give the option to have the output formatted in json, etc... |
Can't say on the top of my head. Both program are pretty old (written well before JSON has been invented, so there is no support for that in both). mhist was written by me mhdump was written by Konstantin. I would both give a try and see what you like more.
Stefan |
1882
|
24 Apr 2020 |
Pintaudi Giorgio | Forum | API to read MIDAS format file |
Stefan Ritt wrote: | I guess all three options would work. I just tried mhist and it still works with the "FILE" history
mhist -e <equipment name> -v <variable name> -h 10
for dumping a variable for the last 10 hours.
I could not get mhdump to work with current history files, maybe it only works with "MIDAS" history and not "FILE" history (see https://midas.triumf.ca/MidasWiki/index.php/History_System#History_drivers). Maybe Konstantin who wrote mhdump has some idea.
Writing your own parser is certainly possible (even in Python), but of course more work.
Stefan |
Dear Stefan,
thank you very much for the quick reply. Sorry if my message was not very clear, actually we are using the "MIDAS" history format and not the "FILE" one. So both mhist and mhdump should be ok (however I have only tested mhist).
Hypothetically which one between the two lends itself the better to being "batched"? I mean to be read and controlled by a program/routine. For example, some programs give the option to have the output formatted in json, etc... |
Draft
|
24 Apr 2020 |
Pintaudi Giorgio | Forum | API to read MIDAS format file |
Stefan Ritt wrote: | I guess all three options would work. I just tried mhist and it still works with the "FILE" historymhist -e <equipment name> -v <variable name> -h 10for dumping a variable for the last 10 hours.I could not get mhdump to work with current history files, maybe it only works with "MIDAS" history and not "FILE" history (see https://midas.triumf.ca/MidasWiki/index.php/History_System#History_drivers). Maybe Konstantin who wrote mhdump has some idea.Writing your own parser is certainly possible (even in Python), but of course more work.Stefan |
Thank you for the quick reply. Do notice that we have "MIDAS" history files and not "FILE", so both mhist and mhdump should be fine (however I have only tested mhist). Hipotetically, which one between mhist and mhdump do you think is better suited to be "batched"? I mean to be controlled and read by a routine? |
1880
|
24 Apr 2020 |
Stefan Ritt | Forum | API to read MIDAS format file | I guess all three options would work. I just tried mhist and it still works with the "FILE" history
mhist -e <equipment name> -v <variable name> -h 10
for dumping a variable for the last 10 hours.
I could not get mhdump to work with current history files, maybe it only works with "MIDAS" history and not "FILE" history (see https://midas.triumf.ca/MidasWiki/index.php/History_System#History_drivers). Maybe Konstantin who wrote mhdump has some idea.
Writing your own parser is certainly possible (even in Python), but of course more work.
Stefan |
Draft
|
24 Apr 2020 |
Stefan Ritt | Forum | API to read MIDAS format file | |
1878
|
24 Apr 2020 |
Pintaudi Giorgio | Forum | API to read MIDAS format file | Dear MIDAS people,
I need to borrow your wisdom for a bit.
I am developing a piece of software that should read the history data stored in a
.midas file (MIDAS format) and integrate it into the WAGASCI data quality output.
In other words, I need to read some temperature values stored in a .midas file and
compare them with the MPPC gains and check for temperature/gain dependence.
I see three possibilities:
- write a custom parser in C++ using the instructions contained in the Mhformat page;
- call the mhist program from within my application;
- call the mhdump program from within my application;
Which solution do you think is the best?
Because there is no need for raw performance, if possible, I would like to write my application in Python3 but C++ is also an option. |
1877
|
23 Apr 2020 |
Stefan Ritt | Suggestion | Sequencer loop break | > > You can do that with the "GOTO" statement, jumping to the first line after the loop.
> >
> > Here is a working example:
> >
> >
> > LOOP runs, 5
> > WAIT Seconds 3
> > IF $runs > 2
> > GOTO 7
> > ENDIF
> > ENDLOOP
> > MESSAGE "Finished", 1
> >
> > Best,
> > Stefan
>
> Hoi Stefan
>
> Thanks for your answer. As I understand it, this has to be in the sequence script before
> running. So, in the end, it is not different than just saying "LOOP runs, 2" and
> therefore the number of runs has do be known in advance as well. Or is there an option to
> change the script on runtime? What I would like, is to start a sequence with "LOOP runs,
> infinite" and when I come back to the experiment after falling asleep being able to break
> the loop after the next iteration, but still execute everything after ENDLOOP, i.e. the
> MESSAGE statement in your example. Because if I do a "Stop after current run", this seems
> not to happen.
>
> Best, Ivo
First, you have the sequencer button "Stop after current run", but that does of course ot
execute anything after ENDLOOP.
Second, you can put anything in the IF statement. Like create a variable on the ODB like
/Experiment/Run parameters/Stop loop and set this to zero. Then put in your script:
...
ODBGET /Experiment/Run parameters/Stop loop, flag
IF $flag == 1
GOTO 7
...
So once you want to stop the loop, set the flag in the ODB to one.
Best,
Stefan |
1876
|
23 Apr 2020 |
Ivo Schulthess | Suggestion | Sequencer loop break | > You can do that with the "GOTO" statement, jumping to the first line after the loop.
>
> Here is a working example:
>
>
> LOOP runs, 5
> WAIT Seconds 3
> IF $runs > 2
> GOTO 7
> ENDIF
> ENDLOOP
> MESSAGE "Finished", 1
>
> Best,
> Stefan
Hoi Stefan
Thanks for your answer. As I understand it, this has to be in the sequence script before
running. So, in the end, it is not different than just saying "LOOP runs, 2" and
therefore the number of runs has do be known in advance as well. Or is there an option to
change the script on runtime? What I would like, is to start a sequence with "LOOP runs,
infinite" and when I come back to the experiment after falling asleep being able to break
the loop after the next iteration, but still execute everything after ENDLOOP, i.e. the
MESSAGE statement in your example. Because if I do a "Stop after current run", this seems
not to happen.
Best, Ivo |
1875
|
21 Apr 2020 |
Stefan Ritt | Suggestion | Sequencer loop break | > I am using the Midas sequencer to run subsequent measurements in a loop, without
> knowing how many iterations in advance. Therefore, I am using the "infinity"
> option. Since I have other commands after the loop, it would be nice to have the
> possibility to break the loop, but let the sequencer then finish the rest of the
> commands.
> Cheers,
> Ivo
You can do that with the "GOTO" statement, jumping to the first line after the loop.
Here is a working example:
LOOP runs, 5
WAIT Seconds 3
IF $runs > 2
GOTO 7
ENDIF
ENDLOOP
MESSAGE "Finished", 1
Best,
Stefan |
1874
|
07 Apr 2020 |
Ivo Schulthess | Suggestion | Sequencer loop break | I am using the Midas sequencer to run subsequent measurements in a loop, without
knowing how many iterations in advance. Therefore, I am using the "infinity"
option. Since I have other commands after the loop, it would be nice to have the
possibility to break the loop, but let the sequencer then finish the rest of the
commands.
Cheers,
Ivo |
1873
|
03 Apr 2020 |
Stefan Ritt | Info | CLOCK_REALTIME on MacOS | > Dear all,
> I'm trying to compile MIDAS on MacOS 10.10 and I get this error:
>
> /Users/francesco/MIDAS/midas/src/system.cxx:3187:18: error: use of undeclared identifier
> 'CLOCK_REALTIME'
> clock_settime(CLOCK_REALTIME, <m);
>
> Is it related to my (old) version of MacOS? Can I fix it somehow?
>
> Thank you,
> Francesco
If I see this correctly, you need at least MacOSX 10.12. If you can't upgrade, you can just remove line 3187
from system.cxx. This function is only used in an online environment, where you would run a frontend on your
Mac, which you probably don't do. So removing it does not hurt you.
Stefan |
1872
|
03 Apr 2020 |
Francesco Renga | Info | CLOCK_REALTIME on MacOS | Dear all,
I'm trying to compile MIDAS on MacOS 10.10 and I get this error:
/Users/francesco/MIDAS/midas/src/system.cxx:3187:18: error: use of undeclared identifier
'CLOCK_REALTIME'
clock_settime(CLOCK_REALTIME, <m);
Is it related to my (old) version of MacOS? Can I fix it somehow?
Thank you,
Francesco |
1871
|
03 Apr 2020 |
Stefan Ritt | Info | Change of TID_xxx data types | We have to request of a 64-bit integer data type to be included in MIDAS banks.
Since 64-bit integers are on some systems "long" and on other systems "long long",
I decided to create the two new data types
TID_INT64
TID_UINT64
which follows more the standard C++ tradition:
https://en.cppreference.com/w/cpp/types/integer
To be consistent, I renamed the old types:
TID_BYTE -> TID_UINT8
TID_SBYTE -> TID_INT8
TID_WORD -> TID_UINT16
TID_SHORT -> TID_INT16
TID_DWORD -> TID_UINT32
TID_INT -> TID_INT32
I left the old definitions in midas.h, so old code will still compile fine and be binary
compatible. But if you write new code, the new types are recommended.
If you save the ODB in ASCII format, the new types are used as stings as well, like
[/Experiment]
ODB timeout = INT32 : 10000
but the old types are still understood when you load an old ODB file.
I hope I didn't break anything, please report if you have any issue.
Stefan |
1870
|
30 Mar 2020 |
Stefan Ritt | Info | mbedtls, mhttpd mongoose 6.16 update | I had some quick look at the new mongoose code and didn't find anything I dislike. Did a quick test of the proxy which worked and is nice to have.
Agree with all KO said about authentication.
So if there are no complaints, I would suggest that we move the summary of this thread into the official documentation.
Stefan |
1869
|
27 Mar 2020 |
Stefan Ritt | Forum | mlogger: misleading error messages for ROOT | Ok, changed.
Stefan |
1868
|
27 Mar 2020 |
Andreas Suter | Forum | mlogger: misleading error messages for ROOT | Hi Stefan,
I think this only partially resolves the issue, in log_write:
#ifdef HAVE_ROOT
} else if (log_chn->format == FORMAT_ROOT) {
status = root_write(log_chn, pevent, pevent->data_size + sizeof(EVENT_HEADER));
#endif
}
actual_time = ss_millitime();
if ((int) actual_time - (int) start_time > 3000)
cm_msg(MINFO, "log_write", "Write operation on \'%s\' took %d ms", log_chn->path.c_str(), actual_time - start_time);
if (status != SS_SUCCESS && !stop_requested) {
cm_msg(MTALK, "log_write", "Error writing output file, stopping run");
cm_msg(MERROR, "log_write", "Cannot write \'%s\', error %d, stopping run", log_chn->path.c_str(), status);
stop_the_run(0);
return status;
}
In your solution root_write returns quietly but status == SS_INVALID_FORMAT (not SS_SUCCESS) and hence I get another misleading error message "Error writing output file, stopping run".
In order to prevent this you also would need to change the return value to SS_SUCCESS. |
|