 13 Nov 2023, Ivo Schulthess, Forum, mlogger does not HAVE_ROOT
|
Good evening,
I am setting up Midas (v2.1) for a new experiment. We want to save the data in the ROOT format. We installed ROOT from source (v6.28/06), and ROOTSYS is |
 13 Nov 2023, Konstantin Olchanski, Forum, mlogger does not HAVE_ROOT
|
> I am setting up Midas (v2.1) for a new experiment. We want to save the data in the ROOT format. We installed ROOT from source (v6.28/06), and ROOTSYS
is set. When we compile Midas, it says that it found ROOT. We set up a second logger channel where we set the filename to run%05d.root, the format to ROOT,
and the output to ROOT. Nevertheless, when starting a run, the logger writes the error that "channel '1' requested ROOT output, but mlogger is built without |
|
13 Nov 2023, Stefan Ritt, Forum, mlogger does not HAVE_ROOT
|
When you do "cmake .." in the build directory, you will see
-- MIDAS: Found ROOT version xxx in yyy
|
|
14 Nov 2023, Konstantin Olchanski, Forum, mlogger does not HAVE_ROOT
|
> Finally, make sure you start "rmlogger" and not "mlogger". Only "rmlogger" contains the ROOT binding.
Stefan is right. I forgot this. As solution to our troubles, mlogger is built without root support. use rmlogger instead.
|
|
14 Nov 2023, Ivo Schulthess, Forum, mlogger does not HAVE_ROOT
|
> Stefan is right. I forgot this. As solution to our troubles, mlogger is built without root support. use rmlogger instead.
>
> K.O.
|
|
14 Nov 2023, Stefan Ritt, Forum, mlogger does not HAVE_ROOT
|
No, I'm not aware of this problem, but I suspect that your events somehow got corrupted. You can try the mdump utility
or the "Event Dump" web page to peek into your events, maybe you see an issue there. To give you more detailed information,
I would have to reproduce your problem, which is probably hard without your hardware.
|
|
15 Nov 2023, Ivo Schulthess, Forum, mlogger does not HAVE_ROOT
|
> No, I'm not aware of this problem, but I suspect that your events somehow got corrupted. You can try the mdump utility
> or the "Event Dump" web page to peek into your events, maybe you see an issue there. To give you more detailed information,
> I would have to reproduce your problem, which is probably hard without your hardware.
|
|
23 Oct 2023, Francesco Renga, Forum, Device with inputs and outputs
|
Dear all,
I'm writing a very simple device driver starting from the nulldev.cxx
example.
|
|
24 Oct 2023, Stefan Ritt, Forum, Device with inputs and outputs
|
The "multi" class driver takes care of that. It properly calls the SET and GET functions
with the correct index. The code for that is in multi.cxx:105:
|
|
03 Oct 2023, Gennaro Tortone, Bug Report, Python midas.file_reader get_eor_odb_dump()
|
Hi,
|
|
16 Oct 2023, Ben Smith, Bug Report, Python midas.file_reader get_eor_odb_dump()
|
Thanks for the bug report Gennaro!
I've fixed the code so that we'll now find the end-of-run ODB dump even if the user is already at the end of the file when they call get_eor_odb_dump().
|
|
06 Oct 2023, Stefan Ritt, Info, New equipment display
|
Since a long time we tried to convert all "static" mhttpd-generated pages to
dynamic JavaScript. With the new history panel editor we were almost there. Now I
committed the last missing piece - the equipment display. This is shown when you
|
09 Oct 2023, Stefan Ritt, Info, New equipment display 
|
An additional functionality has been implemented on the equipment table:
You can now select several elements by Ctrl/Shift-Click on their names, then change the
|
|
06 Oct 2023, Konstantin Olchanski, Info, new history panel editor
|
the new history panel editor has been activated. it is meant to work the same as
the old editor, with some improvements to the history variables selection page.
this new version is written in html+javascript and it will be easier to improve,
|
|
06 Oct 2023, Konstantin Olchanski, Info, default midas history switched to "FILE" and "PerVariable" history
|
We are very happy with the "FILE" implementation of MIDAS history and it is time
to make it the default for new experiments. This history driver works best if
"per variable" history is alos enabled. (SQL history already only works in "per-
|
|
02 Aug 2023, Stefan Ritt, Bug Report, Error accessing history files
|
We sporadically (like once per few hours) have an error message when we access the
history plots through mhttpd:
|
|
09 Aug 2023, Konstantin Olchanski, Bug Report, Error accessing history files
|
I confirm I see same on the agmini system. Two problems: (a) error message is wrong, it's a
short read, not a read error (clue: read() syscall does not return "no such file"). (b)
mlogger is supposed to write history in record-size blocks, read in the same record size
|
|
16 Aug 2023, Stefan Ritt, Bug Report, Error accessing history files
|
Tonight we got another error of that type after the update:
04:17 - [mhttpd,ERROR] [history_schema.cxx:2913:FileHistory::read_data,ERROR] Cannot read
|
|
17 Aug 2023, Konstantin Olchanski, Bug Report, Error accessing history files
|
Confirmed. The error message is wrong. It is printed after a short read(), but short read() does not
set errno, and errno reported by the error message is from some previous syscall. Corrected error
message is already committed. K.O.
|
|
19 Aug 2023, Stefan Ritt, Bug Report, Error accessing history files
|
Still get the same error with the latest version:
3:28 [mhttpd,ERROR] [history_schema.cxx:2913:FileHistory::read_data,ERROR] Cannot read
|
|
06 Oct 2023, Konstantin Olchanski, Bug Report, Error accessing history files
|
> Still get the same error with the latest version:
> 3:28 [mhttpd,ERROR] [history_schema.cxx:2913:FileHistory::read_data,ERROR] Cannot read
> '/data2/history/mhf_1692391703_20230818_hv_tc.dat', read() errno 2 (No such file or directory)
|
|
06 Oct 2023, Konstantin Olchanski, Bug Report, Error accessing history files
|
> two fixes forthcoming:
> a) check for short read in the 2nd place that I missed
> b) two write() are replaced by 2 memcpy() to a preallocated buffer and 1 write()
|
|
03 Oct 2023, Konstantin Olchanski, Bug Fix, wrong array size after loading xml or json file
|
both the xml and the json decoders have a bug (fix pending). loading saved odb
from xml and json file did not truncate arrays in odb to the size of arrays in
the file. for example, if /example/double_array has size 20 in odb, but size 5
|
|
30 Sep 2023, Gennaro Tortone, Bug Report, ODB page and hex values
|
Hi,
|
|
01 Oct 2023, Stefan Ritt, Bug Report, ODB page and hex values
|
Thanks for reporting this bug, I fixed it in the last commit.
Best,
|
|
26 Sep 2023, Stefan Ritt, Info, mjsonrpc_db_save / mjsonrpc_db_load have been dropped
|
The JavaScript function
mjsonrpc_db_save / mjonrpc_db_load
|
|
24 Sep 2023, Frederik Wauters, Suggestion, scroll when browsing for a link
|
Another small user experience request:
When making a link in the odb (web interface) a nice browser window pop's up. There is however not scrolling possible in the window. As a result, you can |
|
26 Sep 2023, Stefan Ritt, Suggestion, scroll when browsing for a link
|
> When making a link in the odb (web interface) a nice browser window pop's up. There is however not scrolling possible in the window. As a result, you
can not reach a odb key if it is nested to deeply.
>
|
|
19 Sep 2023, Frederik Wauters, Bug Report, epics fe "Start Command"
|
The epics frontend overwrites the "start command" odb after each start:
// set start command in ODB
|
|
20 Sep 2023, Stefan Ritt, Bug Report, epics fe "Start Command"
|
Thanks for reporting this problem. It has been fixed today, so the start command is only written if it's emtpy.
Stefan |
|
08 Sep 2023, Nick Hastings, Forum, Hide start and stop buttons
|
The wiki documents an odb variable to enable the hiding of the Start and Stop buttons on the mhttpd status page
https://daq00.triumf.ca/MidasWiki/index.php//Experiment_ODB_tree#Start-Stop_Buttons
|
|
08 Sep 2023, Nick Hastings, Forum, Hide start and stop buttons
|
> Is there now some other mechanism to hide the start and stop buttons?
> Note that this is for a pure slow control system that does not take runs.
|
|
13 Sep 2023, Stefan Ritt, Forum, Hide start and stop buttons
|
Indeed the ODB settings are obsolete. Now that the status page is fully dynamic
(JavaScript), it's much more powerful to modify the status.html page directly. You
can not only hide the buttons, but also remove the run numbers, the running time,
|
|
13 Sep 2023, Nick Hastings, Forum, Hide start and stop buttons
|
Hi Stefan,
> Indeed the ODB settings are obsolete.
|
|
13 Sep 2023, Stefan Ritt, Forum, Hide start and stop buttons
|
> Hi Stefan,
>
> > Indeed the ODB settings are obsolete.
|
|
14 Sep 2023, Konstantin Olchanski, Forum, Hide start and stop buttons
|
I believe the original "hide run start / stop" was added specifically for ND280 GSC MIDAS. I do not know
why it was removed. "hide pause / resume" is still there. I will restore them. Hiding logger channel
section should probably be automatic of there is no /logger/channels, I can check if it works and what
|
|
14 Sep 2023, Stefan Ritt, Forum, Hide start and stop buttons
|
> I believe the original "hide run start / stop" was added specifically for ND280 GSC MIDAS. I do not know
> why it was removed. "hide pause / resume" is still there. I will restore them. Hiding logger channel
> section should probably be automatic of there is no /logger/channels, I can check if it works and what
|
|
15 Sep 2023, Stefan Ritt, Forum, Hide start and stop buttons
|
> I believe the original "hide run start / stop" was added specifically for ND280 GSC MIDAS. I do not know
> why it was removed. "hide pause / resume" is still there. I will restore them. Hiding logger channel
> section should probably be automatic of there is no /logger/channels, I can check if it works and what
|
|
14 Sep 2023, Nick Hastings, Forum, Hide start and stop buttons
|
Hi
> > > Indeed the ODB settings are obsolete.
|
|
12 Sep 2023, Maia Henriksson-Ward, Suggestion, Syntax highlighting for sequencer scripts
|
Recently I was trying to read sequencer scripts written by a previous student, and realized it would be easier to
quickly read/skim sequencer code with some form of syntax highlighting. I've been using Visual Studio Code as my
editor, so I made myself an extension for VS Code that provides basic syntax highlighting (with help from
|
|
12 Sep 2023, Stefan Ritt, Suggestion, Syntax highlighting for sequencer scripts
|
I like the idea of syntax highlighting, but your solution is just for one editor which not everybody
is using. It would be better if the editor built into mhttpd for MSL files would have the possibility.
|
|
01 Jun 2023, Thomas Lindner, Info, MIDAS Workshop 2023
|
Dear MIDAS users,
We would like to arrange another MIDAS workshop, following on from previous successful workshops in 2015, 2017 and 2019. The
|
|
13 Jun 2023, Thomas Lindner, Info, MIDAS Workshop 2023 - Sept 13
|
Hi All,
Thanks to everyone who filled out the doodle poll.
|
|
17 Aug 2023, Thomas Lindner, Info, MIDAS Workshop 2023 - Sept 12-13
|
Dear All,
A quick update on the MIDAS workshop. Based on the number of planned talks we have made the decision to switch to a two day workshop on Sept 12 and 13 |
|
06 Sep 2023, Thomas Lindner, Info, MIDAS Workshop 2023 - Sept 12-13
|
Dear All,
A final reminder about the MIDAS workshop in 6 days. A (hopefully) finalized agenda is posted here:
|
|
16 May 2023, Konstantin Olchanski, Bug Report, excessive logging of http requests
|
Our default configuration of apache httpd logs every request. MIDAS custom web pages can easily make a huge number of RPC calls creating a
huge log file and filling system disk to 100% capacity
|
|
16 May 2023, Konstantin Olchanski, Bug Report, excessive logging of http requests
|
> Our default configuration of apache httpd logs every request. MIDAS custom web pages can easily make a huge number of RPC calls creating a
> huge log file and filling system disk to 100% capacity
|
|
16 May 2023, Stefan Ritt, Bug Report, excessive logging of http requests
|
Maybe you remember the problems we had with a custom page in Japan loading it from TRIUMF. It took almost one minute since each RPC request took
about 1s round-trip. This got fixed by the modb* scheme where the framework actually collects all ODB variables in a custom page and puts them
into ONE rpc request (making the path an actual array of paths). That reduced the requests from 100 to 1 in the above example. Maybe the same
|
|
02 Aug 2023, Konstantin Olchanski, Bug Report, excessive logging of http requests
|
> > Our default configuration of apache httpd logs every request. MIDAS custom web pages can easily make a huge number of RPC calls creating a
> > huge log file and filling system disk to 100% capacity
> perhaps use existing logrotate, add limit on file size (size) and limit of 2 old log files (rotate).
|
|
03 Aug 2023, Konstantin Olchanski, Bug Report, excessive logging of http requests
|
> > > Our default configuration of apache httpd logs every request. MIDAS custom web pages can easily make a huge number of RPC calls creating a
> > > huge log file and filling system disk to 100% capacity
> > perhaps use existing logrotate, add limit on file size (size) and limit of 2 old log files (rotate).
|
|
14 Aug 2023, Konstantin Olchanski, Bug Report, excessive logging of http requests
|
> Our default configuration of apache httpd logs every request.
> MIDAS custom web pages can easily make a huge number of RPC calls creating a
> huge log file and filling system disk to 100% capacity.
|
|
16 Aug 2023, Konstantin Olchanski, Bug Report, excessive logging of http requests
|
> > Our default configuration of apache httpd logs every request.
> > MIDAS custom web pages can easily make a huge number of RPC calls creating a
> > huge log file and filling system disk to 100% capacity.
|
|
17 Aug 2023, Konstantin Olchanski, Bug Report, excessive logging of http requests
|
> > > Our default configuration of apache httpd logs every request.
> > > MIDAS custom web pages can easily make a huge number of RPC calls creating a
> > > huge log file and filling system disk to 100% capacity.
|
|
16 Aug 2023, Konstantin Olchanski, Bug Report, midas wants to show notification?
|
I started to get web browser popups about "midas wants to show notifications,
block/allow/x". is this a glitch or a new unannounced/undocumented feature?
google chrome on macos. K.O. |
|
16 Aug 2023, Stefan Ritt, Bug Report, midas wants to show notification?
|
> I started to get web browser popups about "midas wants to show notifications,
> block/allow/x". is this a glitch or a new unannounced/undocumented feature?
> google chrome on macos. K.O.
|
|
16 Aug 2023, Stefan Ritt, Bug Report, midas wants to show notification?
|
> > I started to get web browser popups about "midas wants to show notifications,
> > block/allow/x". is this a glitch or a new unannounced/undocumented feature?
> > google chrome on macos. K.O.
|
|
16 Aug 2023, Konstantin Olchanski, Bug Report, midas wants to show notification?
|
> This feature was asked by some people ...
"show notifications" popups are strongly associated with disreputable web sites (presumably to
|
|
17 Aug 2023, Stefan Ritt, Bug Report, midas wants to show notification?
|
> > This feature was asked by some people ...
>
> "show notifications" popups are strongly associated with disreputable web sites (presumably to
|
|
15 Aug 2023, Konstantin Olchanski, Info, mlogger update
|
A bit of update to the mlogger. In preparation for more cleanup when Stefan is
here at TRIUMF.
|
|
09 Aug 2023, Konstantin Olchanski, Bug Fix, Stefan's improved ODB flush to disk
|
This is an important improvement, should have a post of it's own. K.O.
> > > RFE filed:
|
|
31 Mar 2022, Stefan Ritt, Suggestion, Maximum ODB size
|
Anybody some idea what the maximum ODB size can be? In the old days, the linux
kernels had a severe limit on shared memory of usually 8MB, but in the age of
64GB RAM being a standard, we should be able to grow bigger. Tried
|
|
04 Apr 2022, Konstantin Olchanski, Suggestion, Maximum ODB size
|
> Anybody some idea what the maximum ODB size can be?
It turns out ODB size limit is hardwired on db_open_database() at 100 Mbytes.
|
|
27 Apr 2023, Marius Koeppel, Suggestion, Maximum ODB size
|
Hi all,
> I agree, I think we can safely bump the limit from 100 Mbytes to 1 Gbyte, maybe 1.5 or
|
|
27 Apr 2023, Konstantin Olchanski, Suggestion, Maximum ODB size
|
> > I agree, I think we can safely bump the limit from 100 Mbytes to 1 Gbyte, maybe 1.5 or
> > 1.99 Gbytes. Above that we run into 32-bit/31-bit cleanliness problems.
>
|
|
27 Apr 2023, Marius Koeppel, Suggestion, Maximum ODB size
|
> This is change is wrong. As I wrote, ODB is not 64-bit clean and it is not 32-bit clean. We think is is 31-bit clean, so maximum size would be slightly
less than 2 Gbytes.
|
|
27 Apr 2023, Konstantin Olchanski, Suggestion, Maximum ODB size
|
> You said the writing into .odb is buggy. Do you mean it’s buggy in general or only in this specific case?
> We save the ODB most of the time in the .odb format.
|
|
27 Apr 2023, Konstantin Olchanski, Suggestion, Maximum ODB size
|
my vote is to bump the ODB size limit to 1999*1000*1000 (not quite 2GB). but this needs to be tested. especially save and restore from ODB, XML and JSON
files, including how long it takes to save and load a 1.9GB ODB. K.O. |
|
28 Apr 2023, Marius Koeppel, Suggestion, Maximum ODB size
|
> my vote is to bump the ODB size limit to 1999*1000*1000 (not quite 2GB). but this needs to be tested. especially save and restore from ODB, XML and JSON
files, including how long it takes to save and load a 1.9GB ODB. K.O.
|
|
28 Apr 2023, Stefan Ritt, Suggestion, Maximum ODB size
|
> Is this maybe related to what Stefan said about the run start - so that odbedit needs some time to load the bigger ODB?
At the run start mlogger writes the ODB to the .mid file. This needs conversion (binary ODB -> XML ASCII) which can take time.
|
|
28 Apr 2023, Marius Koeppel, Suggestion, Maximum ODB size
|
> At the run start mlogger writes the ODB to the .mid file. This needs conversion (binary ODB -> XML ASCII) which can take time.
> This does NOT depend on the ODB size, but on the ODB *content*. Every key in the ODB takes time to convert. So if your ODB as 1.5 GB
> but only a few keys, this is still fast. Only if you have 200 million keys int he ODB, then mlogger takes lots of time to convert
|
|
28 Apr 2023, Konstantin Olchanski, Suggestion, Maximum ODB size
|
> > Is this maybe related to what Stefan said about the run start - so that odbedit needs some time to load the bigger ODB?
>
> At the run start mlogger writes the ODB to the .mid file. This needs conversion (binary ODB -> XML ASCII) which can take time.
|
|
27 Apr 2023, Stefan Ritt, Suggestion, Maximum ODB size
|
> Congratulations. created != "it works".
Two other tings to consider:
|
|
28 Apr 2023, Konstantin Olchanski, Suggestion, Maximum ODB size
|
> > Congratulations. created != "it works".
>
> Two other tings to consider:
|
|
09 Jun 2023, Konstantin Olchanski, Suggestion, Maximum ODB size
|
> > 1) The ODB shared memory is dumped into a binary file (".ODB.SHM") after the last client finished ...
correction: ODB shared memory is saved to .ODB.SHM each time a client stops, this is db_close_database().
|
|
12 Jun 2023, Stefan Ritt, Suggestion, Maximum ODB size
|
> correction: ODB shared memory is saved to .ODB.SHM each time a client stops, this is db_close_database().
The original design of the midas shared memory (back in the 1990's) was that the ODB shared memory file gets
|
|
12 Jun 2023, Konstantin Olchanski, Suggestion, Maximum ODB size
|
> > correction: ODB shared memory is saved to .ODB.SHM each time a client stops, this is db_close_database().
>
> The original design of the midas shared memory (back in the 1990's) was that the ODB shared memory file gets
|
|
13 Jun 2023, Stefan Ritt, Suggestion, Maximum ODB size
|
> I remember the same, but I tracked it down in git to the very first commit, and there is no if() there,
> odb is saved to .ODB.SHM on every client shutdown, not just the last client. I guess we both misremebered.
|
|
13 Jun 2023, Marius Koeppel, Suggestion, Maximum ODB size
|
> BTW, how do I resize the ODB. I remember we discussed this some time ago, and concluded that odbedit needs a resize flag. Has this even been
> done? If not, what is the "official" way to resize the ODB. We had some documentation about that some time ago, but I can't find it anymore.
|
|
13 Jun 2023, Konstantin Olchanski, Suggestion, Maximum ODB size
|
> > BTW, how do I resize the ODB.
ODB cannot be resized "online". Everything has to stop, save content to odb.json, get rid of old ODB.SHM, ensure ODB shared memory is destroyed (SysV or |
|
13 Jun 2023, Konstantin Olchanski, Suggestion, Maximum ODB size
|
> > I remember the same, but I tracked it down in git to the very first commit, and there is no if() there,
> > odb is saved to .ODB.SHM on every client shutdown, not just the last client. I guess we both misremebered.
|
|
13 Jun 2023, Stefan Ritt, Suggestion, Maximum ODB size
|
> small problem. build an experiment, start taking data, observe how ODB is never saved to disk because the "last client" never stops. as bonus, crash
|
|
13 Jun 2023, Konstantin Olchanski, Suggestion, Maximum ODB size
|
>
> > small problem. build an experiment, start taking data, observe how ODB is never saved to disk because the "last client" never stops. as bonus, crash
|
|
13 Jun 2023, Stefan Ritt, Suggestion, Maximum ODB size
|
> are you sure? when/how often does "last midas program finishes" happen? it does not happen on a system crash, not on power loss, not on "shutdown -r
now"
|
|
15 Jun 2023, Konstantin Olchanski, Suggestion, Maximum ODB size
|
>
> > are you sure? when/how often does "last midas program finishes" happen? it does not happen on a system crash, not on power loss, not on "shutdown -r
now"
|
|
28 Jul 2023, Stefan Ritt, Suggestion, Maximum ODB size
|
> RFE filed:
> https://bitbucket.org/tmidas/midas/issues/367/odb-should-be-saved-to-disk-periodically
|
|
09 Aug 2023, Konstantin Olchanski, Suggestion, Maximum ODB size
|
> > RFE filed:
> > https://bitbucket.org/tmidas/midas/issues/367/odb-should-be-saved-to-disk-periodically
>
|