ID |
Date |
Author |
Topic |
Subject |
1865
|
25 Mar 2020 |
Andreas Suter | Forum | mlogger: misleading error messages for ROOT | Dear All,
At our experiment we write ROOT files. When starting/stopping runs we get the following error messages:
[Logger,ERROR] [mlogger.cxx:3358:root_write,ERROR] Cannot write system event into ROOT file, event_id 0xffff8000
[Logger,ERROR] [mlogger.cxx:3358:root_write,ERROR] Cannot write system event into ROOT file, event_id 0xffff8001
Looking into the source code I found that log_write (line 4248) sends these Midas System Events (BOR,EOR) to root_write without filtering them. root_write() checks in a first step if it gets such Midas System Events and if yes, moans.
Wouldn't it be better just to filter these events in log_write, before calling root_write, avoiding unnecessary error messages?
Is there something I miss?
Thanks,
Andreas |
1866
|
25 Mar 2020 |
Konstantin Olchanski | Forum | mlogger: misleading error messages for ROOT | > [Logger,ERROR] [mlogger.cxx:3358:root_write,ERROR] Cannot write system event into ROOT file, event_id 0xffff8000
Hi, Andreas, please open a bug report for this problem on bitbucket, there is now at least 2 bugs against
the ROOT writer (some events are written in duplicate sometimes), and I hope to fix this next time i review
the mlogger (RSN!). Biggest problem is that I do not use the ROOT output myself, so I have no way
to know if ROOT files produced by mlogger are correct or make sense. (without setting up some kind
of test environment with a ROOT file reader.
Thank you for reporting this problem here, so more people know about it.
If somebody has a patch to fix this, please send it in!
K.O. |
1867
|
27 Mar 2020 |
Stefan Ritt | Forum | mlogger: misleading error messages for ROOT | Dear simplest solution seems to me to just remove the error message generation and silently ignore the BOE EOR events.
Committed that change.
Stefan |
1868
|
27 Mar 2020 |
Andreas Suter | Forum | mlogger: misleading error messages for ROOT | Hi Stefan,
I think this only partially resolves the issue, in log_write:
#ifdef HAVE_ROOT
} else if (log_chn->format == FORMAT_ROOT) {
status = root_write(log_chn, pevent, pevent->data_size + sizeof(EVENT_HEADER));
#endif
}
actual_time = ss_millitime();
if ((int) actual_time - (int) start_time > 3000)
cm_msg(MINFO, "log_write", "Write operation on \'%s\' took %d ms", log_chn->path.c_str(), actual_time - start_time);
if (status != SS_SUCCESS && !stop_requested) {
cm_msg(MTALK, "log_write", "Error writing output file, stopping run");
cm_msg(MERROR, "log_write", "Cannot write \'%s\', error %d, stopping run", log_chn->path.c_str(), status);
stop_the_run(0);
return status;
}
In your solution root_write returns quietly but status == SS_INVALID_FORMAT (not SS_SUCCESS) and hence I get another misleading error message "Error writing output file, stopping run".
In order to prevent this you also would need to change the return value to SS_SUCCESS. |
1869
|
27 Mar 2020 |
Stefan Ritt | Forum | mlogger: misleading error messages for ROOT | Ok, changed.
Stefan |
1375
|
03 Jul 2018 |
Frederik Wauters | Forum | mlogger? jamming | We run as follows:
* sis3316 digitizers in a vme crate
* 1-2 midas events /s
* data rate at 20 MB/s
At a rate of 30 MB/s the daq crashed because the I think the mlogger can`t follow:
* it runs at 100% cpu
* memory usage of mlogger process goes from 2% to 15%
* All other processes < 50 % cpu and < 20% RAM
Both the vme frontend and the mlogger crash about 2.5 minutes into a run. Both
the logger and vme fe spit out:
bm_validate_client_pointers: Assertion `pclient->read_pointer >= 0 &&
pclient->read_pointer <= pheader->size' failed.
Aborted
I first thought that writing-to-disk could be a bottle neck. But when I write to
an SSD, same thing.
Is there another bottleneck which keeps the mlogger busy? |
2249
|
29 Jun 2021 |
Lukas Gerritzen | Bug Report | modbcheckbox behaves erroneous with UINT32 variables | For boolean and INT32 variables, modbcheckbox works as expected. You click, it
sets the variable to true or 1, the checkbox stays checked until you click again
and it's being set back to 0.
For UINT32 variables, you can turn the variable "on", but the checkbox visually
becomes unchecked immediately. Clicking again does not set the variable to
0/false and the tick visually appears for a fraction of a second, but vanishes
again. |
2250
|
30 Jun 2021 |
Stefan Ritt | Bug Report | modbcheckbox behaves erroneous with UINT32 variables | > For boolean and INT32 variables, modbcheckbox works as expected. You click, it
> sets the variable to true or 1, the checkbox stays checked until you click again
> and it's being set back to 0.
>
> For UINT32 variables, you can turn the variable "on", but the checkbox visually
> becomes unchecked immediately. Clicking again does not set the variable to
> 0/false and the tick visually appears for a fraction of a second, but vanishes
> again.
Thanks for reporting that bug. Fixed in
https://bitbucket.org/tmidas/midas/commits/4ef26bdc5a32716efe8e8f0e9ce328bafad6a7bf
Stefan |
2252
|
30 Jun 2021 |
Lukas Gerritzen | Bug Report | modbcheckbox behaves erroneous with UINT32 variables | Thanks for the quick fix. |
2161
|
07 May 2021 |
Zaher Salman | Bug Report | modbselect trigget hotlink | It seems that a modbselect triggers a "change" in an ODB which has a hot link. This happens onload (or whenever the custom page is reloaded) and otherwise it behaves as expected, i.e. no change unless the modbselect is actually changed. Is this the intended behaviour? can this be modified? |
2162
|
10 May 2021 |
Stefan Ritt | Bug Report | modbselect trigget hotlink | Thanks for reporting that bug, I fixed it in the last commit.
Stefan |
1294
|
31 May 2017 |
Konstantin Olchanski | Info | modified db_watch() arguments | for reasons unknown, db_watch() did not have an "info" parameter passed through to the callback
handler function, like it is done with db_open_record().
This omission makes it difficult to write db_watch handler functions that must watch multiple odb
trees - db_watch only delivers the hkey of the modified item inside the tree, leaving us with no
simple way to tell which tree it came from. An example of this is mfe.c watching the Common
structure for multiple equipments. There are other
uses for the "info" parameter, for example it is needed to implement c++ wrapper classes.
this omission is now corrected at the cost of changing the definition db_watch().
all uses of db_watch() in the midas tree have been corrected, but all out-of-tree programs
will not compile. For quick conversion, add a NULL parameter to db_watch() calls and add a
"void*info" parameter to your watch handler function.
sorry about this disturbance,
K.O. |
718
|
13 Sep 2010 |
Konstantin Olchanski | Info | modified mhttpd history panel editor | mhttpd.c svn rev 4823 implements a modified history planel editor. all previous functions should work
as before (minus new bugs).
New experimental functions added:
a) there is a new column "Order" containing numbers 10, 20, 30, etc. If you change "30" to (say) "15"
and press "refresh", the history variables will be reordered according to the new values. If you change
(say) "10" to "" (empty) or "0" and press "refresh", this variable will be deleted. (But there is a UI wart - if
you accidentally change the order value to something non-numeric (i.e. "aaa1" or " 1" (leading space)
and press "enter", the variable will be immediately deleted from odb - "enter" works as "refresh + save"
- should probably work as "refresh" requiring explicit press on the "save" button).
b) there is a new button "List all" to list all existing variables - next to each variable is a checkbox -
select any checkboxes and press "add selected" to add selected variables to the history plot. You may
find this function useful (or not), depending on how many variables you have in your history. For
T2K/ND280 this is still not good enough (there are still too many variables) and I want to change this to
a 3 level (equipment, history event, history tag) expandable/collapsable tree (or whatever is simplest to
implement) - to permit the user to quickly zoom on the interesting variables.
I may still tweak with the UI of these new functions, but the basic functionality (reorder+delete and
selection of multiple variables from a list) seems to be solid. Comments and suggestions on how to
make it work the best for your experiment are very welcome.
K.O. |
719
|
17 Sep 2010 |
Konstantin Olchanski | Info | modified mhttpd history panel editor | > mhttpd.c svn rev 4823 implements a modified history planel editor. all previous functions should work
> as before (minus new bugs). New experimental functions added:
>
> a) there is a new column "Order" containing numbers 10, 20, 30, etc. ...
While this seems to work well enough, it might remain a function for "advanced users". For novice
users, a simpler gui, i.e. with "move up" and "move down" buttons, would have been "better", or
at least more familiar. (However I have double plus negative experience using nice
looking "move up and down buttons" to rearrange something I actually need to rearrange,
so I have no interest in implementing something I do not want to use. Think about moving
an item all the way from the bottom of a 10 item list to the very top. No do this not as a mental
exercise, but on a slow loading mhttpd web page running somewhere in Japan).
> b) there is a new button "List all variables" to list all existing variables
Some improvement here (mhttpd.c svn 4823): variables are organized by equipment and by history event
into an expandable list. (I already know that this list expansion does not play well with web page
scrolling, same problem exists in the ODB inline editor).
Again, midas users who have a small number of history events may find this new function
not so useful, but the old way was pretty much unusable for T2K/ND280.
Also, for users with a large number of history events, there 2 new ODB variables
/History/MaxDisplayEvents and /History/MaxDisplayTags which limit the maximum
number of events and tags listed in the old scrollable "option" selector history editor.
For the T2K/ND280 case, this reduces the size of the web page and reduces the page load
time quite substantially. (I picked default values of 20 events and 200 tags quite arbitrary,
perhaps the default should have been "no limit", but then nobody would benefit from this
possibility to substantially reduce web page load times - unless they read documentation (yea, right!)
that is not yet written).
K.O. |
721
|
20 Sep 2010 |
Stefan Ritt | Info | modified mhttpd history panel editor | Just some idea:
The ultimate solution to that would be to do that completely JavaScript driven. You load ONCE the list of all
variables into a local array, then sort this into your history panel LOCALLY. When I did the original mhttpd
history config page, there was not much JavaScript around, but today this would be the ultimate option. It
even supports drag-and-drop. So let's keep that in mind for the future.
- Stefan |
1182
|
13 Jun 2016 |
Konstantin Olchanski | Info | mongoose v6.4 is ready for use | latest version of mongoose web server library (v6.4) is now implemented in midas. To try it out, edit
the Makefile, comment-out USE_MONGOOSE4, uncomment USE_MONGOOSE6, make clean,
make.
After some more testing mongoose v6 will be made the default. (if you see problems, please report
them here).
Main user-visible change is implementation of pipelined http requests, where the same socket
connection is reused for many requests (instead of opening a new connection for each request).
This is supposed to significantly speed up things like ajax requests over https (ssl handshake is
done only once). (As a buglet, some midas web pages do not generated the "ContentLength"
header, and force connection reset).
Special features: (implemented in mhttpd.cxx)
- https support (same as mongoose v4)
- https score A- at SSLlabs (if ignore whining about self-signed certificate)
- CORS support (same as v4) (cross-origin AJAX requests - web pages loaded from some other
web server can make requests into midas)
- password protection (same as v4, uses http digest authentication)
- http-to-https redirect (same as v4)
- setuid-root mode for binding to port 80 (special request from PSI).
K.O. |
1199
|
13 Sep 2016 |
Konstantin Olchanski | Info | mongoose v6.4 is ready for use | > latest version of mongoose web server library (v6.4) is now implemented in midas.
A number of bugs were found in the mongoose v6 implementation of HTTP digest authentication:
- unusual URL in the form "https://blah:8443/?" (notice trailing "?") were rejected. These URLs are sometimes generated by
MIDAS.
- URLs longer than 200 bytes were rejected
- a check for matching URIs between the HTTP request and in digest authentication was missing (required by specs)
If you are using mhttpd with mongoose v6 https, please update mhttpd.cxx to the latest version.
We continue to recommend that mhttpd be used behind a proper HTTPS proxy with password protection (i.e. apache httpd).
mongoose v4 does not seem to have the same bugs, old server does not support https so does not have these bugs.
K.O. |
1201
|
26 Sep 2016 |
Wes Gohn | Info | mongoose v6.4 is ready for use | Since updating to the most recent midas commit, we get the following error if we try running mhttpd without su privileges:
>mhttpd -e CR --http 8081
mhttpd is running in setuid-root mode.
mhttpd is listening on port 80
Mongoose version 4 cannot listen to port 80 in setuid mode. Please use mongoose version 6. Sorry, bye!
[mhttpd,ERROR] [midas.c:1960:,ERROR] cm_disconnect_experiment not called at end of program
It works if we run it as root, but that creates other problems. Is there a flag to turn off setuid-root mode? Or some other fix?
Thanks,
Wes |
1202
|
26 Sep 2016 |
Konstantin Olchanski | Info | mongoose v6.4 is ready for use | > Since updating to the most recent midas commit, we get the following error if we try running mhttpd without su privileges:
>
> >mhttpd -e CR --http 8081
> mhttpd is running in setuid-root mode.
> mhttpd is listening on port 80
> Mongoose version 4 cannot listen to port 80 in setuid mode. Please use mongoose version 6. Sorry, bye!
> [mhttpd,ERROR] [midas.c:1960:,ERROR] cm_disconnect_experiment not called at end of program
>
> It works if we run it as root, but that creates other problems. Is there a flag to turn off setuid-root mode? Or some other fix?
>
From these messages, it looks like you really are using the setuid-root mode. And indeed it is not usable with the mongoose version 4 implementation in MIDAS.
I can suggest several fixes:
1) the setuid-root mode was only ever intended for use at PSI because of peculiar network configuration of the PSI corporate firewall. It is not intended for general
use.
1a) I as an author of MIDAS recommend against using the setuid-root mode and against installing mhttpd as setuid-root because it is not secure. (normally you
would run mhttpd behind an apache https proxy providing https encryption and password protection).
1b) if you follow the midas installation instructions at https://midas.triumf.a you will see that we do not login as root and run "make install" to install mhttpd as
setuid-root.
1c) if you follow these instructions, or if you run mhttpd from the midas build directory ($MIDASSYS/linux/bin/mhttpd), the setuid-root mode will not activate and
everything will work ok.
2) you can run in the "old server" mode, but this more does not implement the JSON-RPC methods, so the "programs" and "alarms" pages will not work.
3) you can build mhttpd with the mongoose version 6 implementation, it will work even with the setuid-root mode. To do this, edit the Makefile, comment-out
"USE_MONGOOSE4=1" and uncomment "USE_MONGOOSE6=1", then make clean, make.
K.O. |
142
|
26 Jul 2003 |
Konstantin Olchanski | | more ODB checks in src/odb.c | Add more checks to db_validate_key() for pkey->total_size, item_size and
num_values. Automatically correct total_size to be item_size*num_values (we
saw this corruption and tested this fix).
K.O.
For your enjoyment, here is the diff:
RCS file: /usr/local/cvsroot/midas/src/odb.c,v
retrieving revision 1.64
diff -r1.64 odb.c
718a719,744
> /* check key sizes */
> if ((pkey->total_size < 0)||(pkey->total_size > pheader->key_size))
> {
> cm_msg(MERROR, "db_validate_key", "Warning: invalid key \"%s\"
total_size: %d", path, pkey->total_size);
> return 0;
> }
>
> if ((pkey->item_size < 0)||(pkey->item_size > pheader->key_size))
> {
> cm_msg(MERROR, "db_validate_key", "Warning: invalid key \"%s\"
item_size: %d", path, pkey->item_size);
> return 0;
> }
>
> if ((pkey->num_values < 0)||(pkey->num_values > pheader->key_size))
> {
> cm_msg(MERROR, "db_validate_key", "Warning: invalid key \"%s\"
num_values: %d", path, pkey->num_values);
> return 0;
> }
>
> /* check and correct key size */
> if (pkey->total_size != pkey->item_size*pkey->num_values)
> {
> cm_msg(MINFO, "db_validate_key", "Warning: corrected key \"%s\" size:
total_size=%d, should be %d*%d=%d", path, pkey->total_size, pkey->item_size,
pkey->num_values, pkey
->item_size*pkey->num_values);
> pkey->total_size = pkey->item_size*pkey->num_values;
> }
> |
|