ID |
Date |
Author |
Topic |
Subject |
1648
|
08 Aug 2019 |
Lauren Manton | Bug Report | ROOTANA bug? | Hi,
Thank you, commenting out the line worked and we can now compile the code. However, when we try to run ana.exe or anaDisplay.exe, we get the following errors:
Error in <TCling::RegisterModule>: cannot find dictionary module TMainDisplayWindowDict_rdict.pcm
Error in <TCling::RegisterModule>: cannot find dictionary module TRootanaDisplayDict_rdict.pcm
Error in <TCling::RegisterModule>: cannot find dictionary module TFancyHistogramCanvasDict_rdict.pcm
We see that the files are in /rootana/obj but we cannot find a way to point the compiler to them.
Could you please advise how to proceed,
Many thanks
> Hi Paolo,
>
> Sorry for the slow response. We were discussing this with Konstantin yesterday. He is aware of the problem now and will be working on a solution soon.
>
> In the short term I found that it works if you just comment out the offending line:
>
> indnerlt:rootana lindner$ git diff libMidasInterface/TMidasOnline.cxx
> diff --git a/libMidasInterface/TMidasOnline.cxx b/libMidasInterface/TMidasOnline.cxx
> index 92eb3e9..67da613 100644
> --- a/libMidasInterface/TMidasOnline.cxx
> +++ b/libMidasInterface/TMidasOnline.cxx
> @@ -191,7 +191,7 @@ bool TMidasOnline::sleep(int mdelay)
> #ifdef CH_IPC
> ss_suspend_set_dispatch(CH_IPC, 0, NULL);
> #else
> - ss_suspend_set_dispatch_ipc(NULL);
> + // ss_suspend_set_dispatch_ipc(NULL);
> #endif
> int status = ss_suspend(mdelay, 0);
> if (status == SS_SUCCESS)
>
> This compiles and at least runs for me; so maybe that is helpful for you. But Konstantin will provide a longer term solution.
>
>
>
> > Hi,
> >
> > I posted on the ROOTANA elog but there seems to be little activity there...
> >
> > Could someone confirm if this is a bug?
> > https://midas.triumf.ca/elog/Rootana/14
> >
> > Another user replied that they are encountering the same issue, so I think it is unlikely it is just our installation.
> >
> > While ROOTANA is unusable for us, I tried to use the example Frontend and Analyzer (under the Experiment source folder). The analyzer does not seem to do much though. A root file is produced but nothing is placed into it. Is that normal?
> >
> > Any help would be welcome. |
1654
|
08 Aug 2019 |
Konstantin Olchanski | Bug Report | ROOTANA bug? | > indnerlt:rootana lindner$ git diff libMidasInterface/TMidasOnline.cxx
> diff --git a/libMidasInterface/TMidasOnline.cxx b/libMidasInterface/TMidasOnline.cxx
> index 92eb3e9..67da613 100644
> --- a/libMidasInterface/TMidasOnline.cxx
> +++ b/libMidasInterface/TMidasOnline.cxx
> @@ -191,7 +191,7 @@ bool TMidasOnline::sleep(int mdelay)
> #ifdef CH_IPC
> ss_suspend_set_dispatch(CH_IPC, 0, NULL);
> #else
> - ss_suspend_set_dispatch_ipc(NULL);
> + // ss_suspend_set_dispatch_ipc(NULL);
> #endif
> int status = ss_suspend(mdelay, 0);
> if (status == SS_SUCCESS)
>
> This compiles and at least runs for me; so maybe that is helpful for you. But Konstantin will provide a longer term solution.
This is a problem with the latest development version of MIDAS. ss_suspend overrides have been removed from system.cxx
and there is no way for rootana to avoid the problem of recursive call to the event handler.
I recommend that instead of using the latest development version of MIDAS you use one of the recent released versions (use "git tag -l", midas-2019-06-b is the latest release).
All the released versions of midas have the ss_suspend overrides implemented and rootana will work correctly. For the next release of midas
I will restore the ss_suspend override and update the rootana code.
K.O. |
1657
|
09 Aug 2019 |
Konstantin Olchanski | Bug Report | Fetest History Plot | > Hi, our logger was running.
One more thing is to check that the history files are actually being written to. Be default
the history files are written into the same directory where you have ODB, etc and
the file names are "*.hst".
Second thing, you can run "mlogger -v" and it will report that it is writing history events
into the history.
If all these things are happening,
Third, you can run "mhist" to see the data directly from the data files.
If all of these work, but you still get nothing in mhttpd, that would be very strange. If I remember
right, to see the sine wave, the history variable to plot is equipment "slow", variable "slow".
K.O.
> I have tried restarting mlogger (even though we haven't
> changed variable names). We ran the following commands one after another and still no
> luck with history plot. Is there anything else that could be causing these problems?
>
> Kind regards,
> Hassan
>
> ==================================================================================
>
> [lm17773@it038146 ~]$ cd /opt/midas_software/midas/bin/
> [lm17773@it038146 bin]$ mhttpd
> [mhttpd,ERROR] [odb.cxx:1646:db_open_database,ERROR] Removed ODB client 'mhttpd',
> index 0 because process pid 20094 does not exists
> [mhttpd,ERROR] [odb.cxx:1646:db_open_database,ERROR] Removed ODB client 'Logger',
> index 1 because process pid 20214 does not exists
> [mhttpd,INFO] Removed open record flag from "/Experiment/Security/RPC hosts/Allowed hosts"
> [mhttpd,INFO] Removed exclusive access mode from "/Experiment/Security/RPC
> hosts/Allowed hosts"
> [mhttpd,INFO] Removed open record flag from "/Experiment/Security/mhttpd hosts/Allowed
> hosts"
> [mhttpd,INFO] Removed exclusive access mode from "/Experiment/Security/mhttpd
> hosts/Allowed hosts"
> [mhttpd,INFO] Removed open record flag from "/Logger/History"
> [mhttpd,INFO] Removed exclusive access mode from "/Logger/History"
> [mhttpd,INFO] Removed open record flag from "/Sequencer/State"
> [mhttpd,INFO] Removed exclusive access mode from "/Sequencer/State"
> [mhttpd,INFO] Removed open record flag from "/History/LoggerHistoryChannel"
> [mhttpd,INFO] Removed exclusive access mode from "/History/LoggerHistoryChannel"
> [mhttpd,INFO] Removed open record flag from "/Equipment/slow/Variables"
> [mhttpd,INFO] Removed exclusive access mode from "/Equipment/slow/Variables"
> [mhttpd,INFO] Removed open record flag from "/Equipment/Trigger/Statistics/Events per
> sec."
> [mhttpd,INFO] Removed exclusive access mode from "/Equipment/Trigger/Statistics/Events
> per sec."
> [mhttpd,INFO] Removed open record flag from "/Equipment/Trigger/Statistics/kBytes per
> sec."
> [mhttpd,INFO] Removed exclusive access mode from "/Equipment/Trigger/Statistics/kBytes
> per sec."
> [mhttpd,INFO] Removed open record flag from "/Equipment/Periodic/Variables"
> [mhttpd,INFO] Removed exclusive access mode from "/Equipment/Periodic/Variables"
> [mhttpd,INFO] Removed open record flag from "/Equipment/Scaler/Variables"
> [mhttpd,INFO] Removed exclusive access mode from "/Equipment/Scaler/Variables"
> [mhttpd,INFO] Corrected 10 ODB entries
> [mhttpd,INFO] Deleted entry '/System/Clients/20094' for client 'mhttpd' because it is
> not connected to ODB
> Mongoose web server will use SSL certificate file "/home/lm17773/online/ssl_cert.pem"
> Mongoose web server will use authentication realm "sampleexpt", password file
> "/home/lm17773/online/htpasswd.txt"
> mongoose web server is redirecting HTTP port 8080 to
> https://it038146.users.bris.ac.uk:8443
> mongoose web server is listening on the HTTP port 8080
> mongoose web server is listening on the HTTPS port 8443
>
====================================================================================
>
> [lm17773@it038146 bin]$ mlogger
> [Logger,INFO] Deleted entry '/System/Clients/20214' for client 'Logger' because it is
> not connected to ODB
> Log directory is /home/lm17773/online/
> Data directory is same as Log unless specified in /Logger/channels/
> History directory is same as Log unless specified in /Logger/history/
> ELog directory is same as Log
> SQL database is localhost/sampleexpt/Runlog
> MIDAS logger started. Stop with "!"
>
====================================================================================
> [lm17773@it038146 bin]$ fetest
> Frontend name : fetest
> Event buffer size : 10485760
> User max event size : 4194304
> User max frag. size : 4194304
> # of events per buffer : 2
>
> Connect to experiment sampleexpt...
> OK
> Init hardware...frontend_init!
> Event size set to 10240 bytes
> Ring buffer wait sleep 1 ms
> OK
> time 1564131394, data 97.814758
> time 1564131395, data 96.592583
> time 1564131396, data 95.105652
> time 1564131397, data 93.358040
> time 1564131398, data 91.354546
> time 1564131399, data 89.100655
> time 1564131400, data 86.602539
> time 1564131401, data 83.867058
> time 1564131402, data 80.901703
> time 1564131403, data 77.714592
> Warning: bank RND4 has zero size
> time 1564131404, data 74.314484
> time 1564131405, data 70.710678
> time 1564131406, data 66.913063
> time 1564131407, data 62.932041
>
====================================================================================
>
>
>
>
>
>
> > > Hi,
> > >
> > > We've been trying to run Fetest in the attempt of plotting the sine wave data on
> > > the history page on the web server. However each time we've tried running a new
> > > plot we have come across the error of 'no data' from the variables. In the
> > > status page we are clearly obtaining data from the frontend and it is updating
> > > the variable as expected in SLOW.
> > >
> > > When setting up MIDAS we managed to produce a history plot from Fetest but are
> > > unable to do so any longer. We did have a go at modifying the Fetest code but
> > > created a backup before doing so and are now running the original backup.
> > >
> > > What could be causing the Fetest data not to be showing in the history plot?
> >
> > Is the logger running? (this application is handling the history data).
> > If yes: Did you change the variable names? If yes: restart the logger. |
1658
|
09 Aug 2019 |
Konstantin Olchanski | Bug Report | Fetest History Plot | > Hi, our logger was running.
Please do these simple tests:
- run "mlogger -v", it should report that it is writing slow/slow data into the history with rate 1 Hz (fetest
should be running at this point, yes?)
- normally the history files are written into the experiment directory (where ODB is, etc) and have file names
"*.hst". Observe that the files are growing. Use "ls -ltr". (mlogger and fetest should be running at this point,
yes?)
- if all if this is happening, you can try to run "mhist" to see the history data
If all of the above works, but you still get nothing from the history plots in mhttpd, then we probably have a
bug in midas and we would like very much to fix it. For this we will need some more information from you. I
hope you have some time available to help us with this.
Hmm... the fetest history plots are not defined automatically, you have to create the history plot manually,
maybe this is where the problem happens. One thing to check here, the correct variable to plot
is "slow/slow", if I remember right.
K.O.
> I have tried restarting mlogger (even though we haven't
> changed variable names). We ran the following commands one after another and still no
> luck with history plot. Is there anything else that could be causing these problems?
>
> Kind regards,
> Hassan
>
>
================================================================================
==
>
> [lm17773@it038146 ~]$ cd /opt/midas_software/midas/bin/
> [lm17773@it038146 bin]$ mhttpd
> [mhttpd,ERROR] [odb.cxx:1646:db_open_database,ERROR] Removed ODB client 'mhttpd',
> index 0 because process pid 20094 does not exists
> [mhttpd,ERROR] [odb.cxx:1646:db_open_database,ERROR] Removed ODB client 'Logger',
> index 1 because process pid 20214 does not exists
> [mhttpd,INFO] Removed open record flag from "/Experiment/Security/RPC hosts/Allowed hosts"
> [mhttpd,INFO] Removed exclusive access mode from "/Experiment/Security/RPC
> hosts/Allowed hosts"
> [mhttpd,INFO] Removed open record flag from "/Experiment/Security/mhttpd hosts/Allowed
> hosts"
> [mhttpd,INFO] Removed exclusive access mode from "/Experiment/Security/mhttpd
> hosts/Allowed hosts"
> [mhttpd,INFO] Removed open record flag from "/Logger/History"
> [mhttpd,INFO] Removed exclusive access mode from "/Logger/History"
> [mhttpd,INFO] Removed open record flag from "/Sequencer/State"
> [mhttpd,INFO] Removed exclusive access mode from "/Sequencer/State"
> [mhttpd,INFO] Removed open record flag from "/History/LoggerHistoryChannel"
> [mhttpd,INFO] Removed exclusive access mode from "/History/LoggerHistoryChannel"
> [mhttpd,INFO] Removed open record flag from "/Equipment/slow/Variables"
> [mhttpd,INFO] Removed exclusive access mode from "/Equipment/slow/Variables"
> [mhttpd,INFO] Removed open record flag from "/Equipment/Trigger/Statistics/Events per
> sec."
> [mhttpd,INFO] Removed exclusive access mode from "/Equipment/Trigger/Statistics/Events
> per sec."
> [mhttpd,INFO] Removed open record flag from "/Equipment/Trigger/Statistics/kBytes per
> sec."
> [mhttpd,INFO] Removed exclusive access mode from "/Equipment/Trigger/Statistics/kBytes
> per sec."
> [mhttpd,INFO] Removed open record flag from "/Equipment/Periodic/Variables"
> [mhttpd,INFO] Removed exclusive access mode from "/Equipment/Periodic/Variables"
> [mhttpd,INFO] Removed open record flag from "/Equipment/Scaler/Variables"
> [mhttpd,INFO] Removed exclusive access mode from "/Equipment/Scaler/Variables"
> [mhttpd,INFO] Corrected 10 ODB entries
> [mhttpd,INFO] Deleted entry '/System/Clients/20094' for client 'mhttpd' because it is
> not connected to ODB
> Mongoose web server will use SSL certificate file "/home/lm17773/online/ssl_cert.pem"
> Mongoose web server will use authentication realm "sampleexpt", password file
> "/home/lm17773/online/htpasswd.txt"
> mongoose web server is redirecting HTTP port 8080 to
> https://it038146.users.bris.ac.uk:8443
> mongoose web server is listening on the HTTP port 8080
> mongoose web server is listening on the HTTPS port 8443
>
================================================================================
====
>
> [lm17773@it038146 bin]$ mlogger
> [Logger,INFO] Deleted entry '/System/Clients/20214' for client 'Logger' because it is
> not connected to ODB
> Log directory is /home/lm17773/online/
> Data directory is same as Log unless specified in /Logger/channels/
> History directory is same as Log unless specified in /Logger/history/
> ELog directory is same as Log
> SQL database is localhost/sampleexpt/Runlog
> MIDAS logger started. Stop with "!"
>
================================================================================
====
> [lm17773@it038146 bin]$ fetest
> Frontend name : fetest
> Event buffer size : 10485760
> User max event size : 4194304
> User max frag. size : 4194304
> # of events per buffer : 2
>
> Connect to experiment sampleexpt...
> OK
> Init hardware...frontend_init!
> Event size set to 10240 bytes
> Ring buffer wait sleep 1 ms
> OK
> time 1564131394, data 97.814758
> time 1564131395, data 96.592583
> time 1564131396, data 95.105652
> time 1564131397, data 93.358040
> time 1564131398, data 91.354546
> time 1564131399, data 89.100655
> time 1564131400, data 86.602539
> time 1564131401, data 83.867058
> time 1564131402, data 80.901703
> time 1564131403, data 77.714592
> Warning: bank RND4 has zero size
> time 1564131404, data 74.314484
> time 1564131405, data 70.710678
> time 1564131406, data 66.913063
> time 1564131407, data 62.932041
>
================================================================================
====
>
>
>
>
>
>
> > > Hi,
> > >
> > > We've been trying to run Fetest in the attempt of plotting the sine wave data on
> > > the history page on the web server. However each time we've tried running a new
> > > plot we have come across the error of 'no data' from the variables. In the
> > > status page we are clearly obtaining data from the frontend and it is updating
> > > the variable as expected in SLOW.
> > >
> > > When setting up MIDAS we managed to produce a history plot from Fetest but are
> > > unable to do so any longer. We did have a go at modifying the Fetest code but
> > > created a backup before doing so and are now running the original backup.
> > >
> > > What could be causing the Fetest data not to be showing in the history plot?
> >
> > Is the logger running? (this application is handling the history data).
> > If yes: Did you change the variable names? If yes: restart the logger. |
1664
|
14 Aug 2019 |
Konstantin Olchanski | Bug Report | ROOTANA bug? | > - ss_suspend_set_dispatch_ipc(NULL);
> + // ss_suspend_set_dispatch_ipc(NULL);
>
> This compiles and at least runs for me; so maybe that is helpful for you. But Konstantin will provide a longer term solution.
I now understand why this fix worked. Around December 2018 timeframe, I reworked the MIDAS event buffer code
and one improvement was to only send UDP buffer notifications if somebody is waiting for them. This probably
reduced to zero the probability of recursive calls to the user event handler - the problem originally fixed by the monkey
work against the midas ipc handler.
After looking at it, I now understand that the correct solution is to call ss_suspend(MSG_BM), but it turns out
inside MIDAS, handling of MSG_BM was incomplete and recursive calls to the user event handler were still
possible. (but most likely not actually happening anymore because of those changes to the event buffer code).
So.
a) ss_suspend(MSG_BM) inside midas now works correctly, recursive call to the user event handler will not happen.
b) TMidasOnline::sleep() now calls ss_suspend(MSG_BM), monkey business with ss_suspend_set_dispatch_ipc() is removed.
The problem of recursive call to the analyzer event handler is now fixed, both rootana and manalyzer (both use the same TMidasOnline code).
Read more about this here:
https://midas.triumf.ca/elog/Midas/1663
K.O. |
1680
|
08 Sep 2019 |
Vinzenz Bildstein | Bug Report | https redirect and ODB access | I'm not sure if these issues are related or not, but I'm getting an error
message when I want to access the root of the ODB via the webserver:
[mhttpd,ERROR] [mhttpd.cxx:563:rread,ERROR] Cannot read file '/root', read of
4096 returned -1, errno 21 (Is a directory)
I also tried turning the re-direct from http to https off, but this does not
seem to work. I also noticed that the redirect changes the localhost into a
hostname. Where does mongoose take this hostname from?
EDIT: Seems that the change of the hostname is due to a setting in /etc/hosts,
i.e. all my fault ...
EDIT: I think there was some issue with the mhttpd. When I checked the output (I
used screen to run it), it was full of these messages:
ss_semaphore_wait_for: semop/semtimedop(2588679) returned -1, errno 43
(Identifier removed)
al_check: Something is wrong with our semaphore, ss_semaphore_wait_for()
returned 408, aborting.
al_check: Cannot abort - this will lock you out of odb. From this point, MIDAS
will not work correctly. Please read the discussion at
https://midas.triumf.ca/elog/Midas/945
Restarted it and it stopped redirecting. So accessing the root of the ODB via
the webserver is the only issue now. |
1686
|
16 Sep 2019 |
Konstantin Olchanski | Bug Report | https redirect and ODB access | > I'm not sure if these issues are related or not, but I'm getting an error
> message when I want to access the root of the ODB via the webserver:
> [mhttpd,ERROR] [mhttpd.cxx:563:rread,ERROR] Cannot read file '/root', read of
> 4096 returned -1, errno 21 (Is a directory)
This is an old bug. It was part of the "custom path" confusion. Fixed (I think) in all midas-2019
releases.
To confirm, which version are you using (run "odbedit ver" or look on the mhttpd "help" page)?
If you have an older version, I recommend that you update to midas-2019-03 (cd midas; git pull;
git checkout midas-2019-03; make clean; make).
If you feel adventurous, you can also update to the head of the development version
and see all the new features (cmake, c++11, new history pages).
If you do not feel adventurous, wait until we have midas-2019-09 ready, use midas-2019-03
until then.
K.O. |
1700
|
20 Sep 2019 |
Frederik Wauters | Bug Report | lazylogger in cmake & max_event_size | compiling:
----------
The compile option -DHAVE_FTPLIB checked in mdsupport.cxx disappeared if you
compile with cmake.
I added:
add_compile_options(-DHAVE_FTPLIB)
to CMakeLists.txt to fix this. Can probably be done in a more elegant way
(checking if the right libraries exist?).
running:
-------
Our MAX_EVENT_SIZE is set in the odb to 805306368. This number is also used in
INT lazy_copy(char *outfile, char *infile, int max_event_size)
this is to big when copying with ftp, causing a crash. Reducing it here with a
factor 10 solves our problems. |
1709
|
27 Sep 2019 |
Konstantin Olchanski | Bug Report | lazylogger in cmake & max_event_size | > The compile option -DHAVE_FTPLIB checked in mdsupport.cxx disappeared if you
> compile with cmake.
Hi, Stefan - do we still need to support FTP in the logger? In the lazylogger, special support for
FTP is not needed, they can you the "script" method and do FTP without our help.
I move to remove FTP support from MIDAS. (second? other opinions?)
> Our MAX_EVENT_SIZE is set in the odb to 805306368. This number is also used in
> this is to big when copying with ftp, causing a crash. Reducing it here with a
> factor 10 solves our problems.
I am surprised that changing MAX_EVENT_SIZE (to a "too big" value) causes lazylogger to
crash. More usually MAX_EVENT_SIZE has no effect until you try to write an event that is
somehow "too big", then there is a crash. Perhaps there is a bug specifically in the FTP code.
Anyhow, I recommend the solution of using the "script" method. We have example lazylogger
scripts in midas/progs/lazy*.perl (the scripts do not have to be in perl, python is ok). We do
not have any example that uses FTP because we do not use FTP for data storage. But you can
easily adapt lazy_test.perl and lazy_copy.perl to use scp and sftp, the secure versions of FTP.
K.O. |
1719
|
06 Oct 2019 |
Nik Berger | Bug Report | History data size mismatch | Logging a list of variables to the history via links in the history ODB subtree,
we get messages as follows at every run start:
19:43:24.009 2019/10/06 [Logger,ERROR] [history_schema.cxx:2676:hs_write_event,ERROR] Event 'System' data size mismatch: expected 412 bytes, got 416 bytes
19:43:24.008 2019/10/06 [Logger,ERROR] [history_schema.cxx:2676:hs_write_event,ERROR] Event 'System' data size mismatch: expected 412 bytes, got 416 bytes
19:43:23.850 2019/10/06 [Logger,ERROR] [history_schema.cxx:455:hs_write_event,ERROR] Event 'System' data size mismatch count: 25, expected 412 bytes, hs_write_event() called with as much as 416 bytes
19:43:23.850 2019/10/06 [Logger,ERROR] [history_schema.cxx:455:hs_write_event,ERROR] Event 'System' data size mismatch count: 25, expected 412 bytes, hs_write_event() called with as much as 416 bytes
The history calculates the size of a record from the size of the individual variables, (history_schema.cxx, L2666 ff), whereas the ODB delivers the data aligned/padded to the size of the largest value in the record.
In our history, a long list of doubles (64 Bit) fas followed by three floats (32 bit), leading to a padded response from the ODB, 4 byte longer than the history expects.
Quick fix: Add another 32 bit dummy variable to the history. Gets rid of the error messages...
Should probably be fixed at a deeper level... |
1720
|
06 Oct 2019 |
Stefan Ritt | Bug Report | History data size mismatch | I wonder why do you this via ODB links. The "standard" way of writing to the history should be to create events for an equipment and flag this equipment as being written to the
history. All variables under /Equipment/<name>/Variables then automatically go into the history and you don't have to worry about ODB links. Only variables not fitting the
equipment/variables scheme should be dealt with via ODB links, like variables under equipment/statistics or parameters in another ODB tree. In a typical midas experiment, only
very few variables typically go into the 'System' event. This is however probably not a solution to your problem. If you have a similar structure (doubles plus an odd number of floats)
under 'variables', you might get the same error. I'n in contact with KO to fix this problem at the root level.
Stefan
> Logging a list of variables to the history via links in the history ODB subtree,
> we get messages as follows at every run start:
>
> 19:43:24.009 2019/10/06 [Logger,ERROR] [history_schema.cxx:2676:hs_write_event,ERROR] Event 'System' data size mismatch: expected 412 bytes, got 416 bytes
>
> 19:43:24.008 2019/10/06 [Logger,ERROR] [history_schema.cxx:2676:hs_write_event,ERROR] Event 'System' data size mismatch: expected 412 bytes, got 416 bytes
>
> 19:43:23.850 2019/10/06 [Logger,ERROR] [history_schema.cxx:455:hs_write_event,ERROR] Event 'System' data size mismatch count: 25, expected 412 bytes, hs_write_event() called with as much as 416 bytes
>
> 19:43:23.850 2019/10/06 [Logger,ERROR] [history_schema.cxx:455:hs_write_event,ERROR] Event 'System' data size mismatch count: 25, expected 412 bytes, hs_write_event() called with as much as 416 bytes
>
> The history calculates the size of a record from the size of the individual variables, (history_schema.cxx, L2666 ff), whereas the ODB delivers the data aligned/padded to the size of the largest value in the record.
> In our history, a long list of doubles (64 Bit) fas followed by three floats (32 bit), leading to a padded response from the ODB, 4 byte longer than the history expects.
> Quick fix: Add another 32 bit dummy variable to the history. Gets rid of the error messages...
> Should probably be fixed at a deeper level... |
1721
|
10 Oct 2019 |
Konstantin Olchanski | Bug Report | History data size mismatch | >
> In our history, a long list of doubles (64 Bit) fas followed by three floats (32 bit)
>
Padding trouble, mixing "double" and "float" trouble. Ouch.
Best wisdom I received on this: never use "float", always use "double".
I was burned by "float" with following code, which produced the same result from
analyzing 100 files as from analyzing 1000 files. (why did we take data for 10 weeks
instead of 1 week?). Hint: "float" overflows way too quickly, after overflow sum+=1 does not change
the value of "sum". The actual code used ROOT TH1F. Lesson: always use TH1D.
float sum = 0; // should always be "double" !!!
foreach data_file {
foreach data from current data file {
sum += data;
}
}
print sum;
K.O. |
1722
|
10 Oct 2019 |
Nik Berger | Bug Report | History data size mismatch | >I wonder why do you this via ODB links. The "standard" way of writing to the history should be to create events for an equipment and flag this equipment as being written to the
>history. All variables under /Equipment/<name>/Variables then automatically go into the history and you don't have to worry about ODB links. Only variables not fitting the
>equipment/variables scheme should be dealt with via ODB links, like variables under equipment/statistics or parameters in another ODB tree. In a typical midas experiment, only
>very few variables typically go into the 'System' event. This is however probably not a solution to your problem. If you have a similar structure (doubles plus an odd number of floats)
>under 'variables', you might get the same error.
>
> In our history, a long list of doubles (64 Bit) fas followed by three floats (32 bit)
>
We do this in the MuX DAQ and mix things that come directly from MIDAS (the MIDAS trigger rate) and things from the
analyzer (rates in the self-triggering detectors) and some temperatures from yet somewhere else. Yes, we could have
kept that apart, yes, in this case a double would also work (and not break things), but a bug is a bug...
I could think of senisble use cases where doubles and ints are mixed and I also know quite a few areas where it makes
sense to use floats...
Nik |
1723
|
10 Oct 2019 |
Stefan Ritt | Bug Report | History data size mismatch | > Yes, we could have
> kept that apart, yes, in this case a double would also work (and not break things), but a bug is a bug...
> I could think of senisble use cases where doubles and ints are mixed and I also know quite a few areas where it makes
> sense to use floats...
I agree with Nik that we should fix this on the midas level. Since it happens in history_schema.cxx which was written by KO, maybe he can have a look.
Stefan |
1724
|
14 Oct 2019 |
Stefan Ritt | Bug Report | lazylogger in cmake & max_event_size | > > The compile option -DHAVE_FTPLIB checked in mdsupport.cxx disappeared if you
> > compile with cmake.
>
> Hi, Stefan - do we still need to support FTP in the logger? In the lazylogger, special support for
> FTP is not needed, they can you the "script" method and do FTP without our help.
>
> I move to remove FTP support from MIDAS. (second? other opinions?)
I oppose to remove FTP support from lazylogger. We still use it heavily at PSI. In comparison to the "script" method, it
shows the current speed in MB/s which helps us to diagnose some network problem by writing this number into the
history. The "script" method only give you an integral transfer speed after a file has be completely written.
I'm however not sure who FTP is used in lazylogger. It goes into mdsupport.cxx and I seem to remember that Pierre
wrote the FTP code by hand, so no external library is necessary.
Stefan |
Attachment 1: Screenshot_2019-10-14_at_13.54.53_.png
|
|
1730
|
24 Oct 2019 |
Konstantin Olchanski | Bug Report | lazylogger in cmake & max_event_size | > > > The compile option -DHAVE_FTPLIB checked in mdsupport.cxx disappeared if you
> > > compile with cmake.
> >
> > Hi, Stefan - do we still need to support FTP in the logger? In the lazylogger, special support for
> > FTP is not needed, they can you the "script" method and do FTP without our help.
> >
> > I move to remove FTP support from MIDAS. (second? other opinions?)
>
> I oppose to remove FTP support from lazylogger.
Confirmed. FTP support in lazylogger stays.
K.O. |
1731
|
08 Nov 2019 |
Pierre Gorel | Bug Report | Newly installed MIDAS on OSX: mhttpd crahes | Context: out of the box MIDAS (using cmake) on OSX Mojave.
Running with mongoose/opensslm installation following instruction here:
https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux
mhttpd crashing when midas webpage opened with Safari (12.1.2). Usually when opening the "chat" tab but sometimes also with the "message" tab.
mhttpd(11109,0x70000827a000) malloc: *** error for object 0x7f8669501ef0: pointer being freed was not allocated
mhttpd(11109,0x70000827a000) malloc: *** set a breakpoint in malloc_error_break to debug
No crash if using firefox (70.0.1 (64-bit)) |
1732
|
12 Nov 2019 |
Konstantin Olchanski | Bug Report | Newly installed MIDAS on OSX: mhttpd crahes | > Context: out of the box MIDAS (using cmake) on OSX Mojave.
>
> Running with mongoose/opensslm installation following instruction here:
> https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux
>
> mhttpd crashing when midas webpage opened with Safari (12.1.2). Usually when opening the "chat" tab but sometimes also with the "message" tab.
> mhttpd(11109,0x70000827a000) malloc: *** error for object 0x7f8669501ef0: pointer being freed was not allocated
> mhttpd(11109,0x70000827a000) malloc: *** set a breakpoint in malloc_error_break to debug
>
> No crash if using firefox (70.0.1 (64-bit))
I think we also have reports of mhttpd crash on macos with safari from the Dragon experiment,
but cannot reproduce the problem.
If you can reproduce this, can you capture the crash stack trace?
One way to do this is to enable core dumps in odb "/expt/enable core dumps" set to "y", restart mhttpd,
wait for the crash. I think macos writes core dumps into /cores/... Or you can run mhttpd inside lldb
and wait for the crash. the lldb command to show the stack trace is "bt", but you may need
to switch to different threads to see which one actually crashed. I forget what the command
for that is.
BTW, the mhttpd networking code has not changed in a long time, but an update
of mongoose web server library is overdue (to fix a memory leak, at least).
K.O. |
1734
|
15 Nov 2019 |
Pierre Gorel | Bug Report | Newly installed MIDAS on OSX: mhttpd crahes | It is reproducible alright.
Here are the core dump and the backtrace (I think the former is more informative).
> > Context: out of the box MIDAS (using cmake) on OSX Mojave.
> >
> > Running with mongoose/opensslm installation following instruction here:
> > https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux
> >
> > mhttpd crashing when midas webpage opened with Safari (12.1.2). Usually when opening the "chat" tab but sometimes also with the "message" tab.
> > mhttpd(11109,0x70000827a000) malloc: *** error for object 0x7f8669501ef0: pointer being freed was not allocated
> > mhttpd(11109,0x70000827a000) malloc: *** set a breakpoint in malloc_error_break to debug
> >
> > No crash if using firefox (70.0.1 (64-bit))
>
> I think we also have reports of mhttpd crash on macos with safari from the Dragon experiment,
> but cannot reproduce the problem.
>
> If you can reproduce this, can you capture the crash stack trace?
>
> One way to do this is to enable core dumps in odb "/expt/enable core dumps" set to "y", restart mhttpd,
> wait for the crash. I think macos writes core dumps into /cores/... Or you can run mhttpd inside lldb
> and wait for the crash. the lldb command to show the stack trace is "bt", but you may need
> to switch to different threads to see which one actually crashed. I forget what the command
> for that is.
>
> BTW, the mhttpd networking code has not changed in a long time, but an update
> of mongoose web server library is overdue (to fix a memory leak, at least).
>
> K.O. |
Attachment 1: mhttpd_lldb_bt.txt
|
SnoGlobe:~/packages/midas/build> lldb mhttpd
(lldb) target create "mhttpd"
Current executable set to 'mhttpd' (x86_64).
(lldb) r
Process 15988 launched: '/Users/acquis/packages/midas/bin/mhttpd' (x86_64)
Mongoose web server will use SSL certificate file "/Users/acquis/online/ssl_cert.pem"
Mongoose web server will use authentication realm "snoglobe", password file "/Users/acquis/online/htpasswd.txt"
mongoose web server is redirecting HTTP port 8080 to https://SnoGlobe.local:8443
mongoose web server is listening on the HTTP port 8080
mongoose web server is listening on the HTTPS port 8443
mhttpd(15988,0x70000146e000) malloc: *** error for object 0x100608790: pointer being freed was not allocated
mhttpd(15988,0x70000146e000) malloc: *** set a breakpoint in malloc_error_break to debug
Process 15988 stopped
* thread #2, stop reason = signal SIGABRT
frame #0: 0x00007fff7456c2c6 libsystem_kernel.dylib`__pthread_kill + 10
libsystem_kernel.dylib`__pthread_kill:
-> 0x7fff7456c2c6 <+10>: jae 0x7fff7456c2d0 ; <+20>
0x7fff7456c2c8 <+12>: movq %rax, %rdi
0x7fff7456c2cb <+15>: jmp 0x7fff74566457 ; cerror_nocancel
0x7fff7456c2d0 <+20>: retq
Target 0: (mhttpd) stopped.
(lldb) bt
* thread #2, stop reason = signal SIGABRT
* frame #0: 0x00007fff7456c2c6 libsystem_kernel.dylib`__pthread_kill + 10
frame #1: 0x00007fff74627bf1 libsystem_pthread.dylib`pthread_kill + 284
frame #2: 0x00007fff744d66a6 libsystem_c.dylib`abort + 127
frame #3: 0x00007fff745e5077 libsystem_malloc.dylib`malloc_vreport + 545
frame #4: 0x00007fff745e4e38 libsystem_malloc.dylib`malloc_report + 151
frame #5: 0x000000010004fe53 mhttpd`AuthEntry::~AuthEntry(this=0x0000000101300220) at mhttpd.cxx:16983:8
frame #6: 0x000000010004fe25 mhttpd`AuthEntry::~AuthEntry(this=0x0000000101300220) at mhttpd.cxx:16983:8
frame #7: 0x000000010004fe09 mhttpd`std::__1::allocator<AuthEntry>::destroy(this=0x00000001001b0f10, __p=0x0000000101300220) at memory:1880:64
frame #8: 0x000000010004fddd mhttpd`void std::__1::allocator_traits<std::__1::allocator<AuthEntry> >::__destroy<AuthEntry>((null)=std::__1::true_type @ 0x000070000146cc28, __a=0x00000001001b0f10, __p=0x0000000101300220) at memory:1742:18
frame #9: 0x000000010004fd9d mhttpd`void std::__1::allocator_traits<std::__1::allocator<AuthEntry> >::destroy<AuthEntry>(__a=0x00000001001b0f10, __p=0x0000000101300220) at memory:1595:14
frame #10: 0x000000010004fd58 mhttpd`std::__1::__vector_base<AuthEntry, std::__1::allocator<AuthEntry> >::__destruct_at_end(this=0x00000001001b0f00, __new_last=0x0000000101300220) at vector:421:9
frame #11: 0x000000010004fc68 mhttpd`std::__1::__vector_base<AuthEntry, std::__1::allocator<AuthEntry> >::clear(this=0x00000001001b0f00) at vector:364:29
frame #12: 0x000000010004ff76 mhttpd`std::__1::vector<AuthEntry, std::__1::allocator<AuthEntry> >::clear(this=0x00000001001b0f00 size=1) at vector:766:17
frame #13: 0x00000001000494d3 mhttpd`read_passwords(auth=0x00000001001b0ec8) at mhttpd.cxx:17047:20
frame #14: 0x00000001000519d5 mhttpd`handle_http_message(nc=0x0000000100608860, msg=0x000070000146d5a0) at mhttpd.cxx:17719:20
frame #15: 0x00000001000499e0 mhttpd`handle_http_event_mg(nc=0x0000000100608860, ev=100, ev_data=0x000070000146d5a0) at mhttpd.cxx:17765:7
frame #16: 0x00000001000650db mhttpd`mg_call(nc=0x0000000100608860, ev_handler=(mhttpd`handle_http_event_mg(mg_connection*, int, void*) at mhttpd.cxx:17760), ev=100, ev_data=0x000070000146d5a0)(mg_connection*, int, void*), int, void*) at mongoose6.cxx:2120:5
frame #17: 0x000000010006c92b mhttpd`mg_http_call_endpoint_handler(nc=0x0000000100608860, ev=100, hm=0x000070000146d5a0) at mongoose6.cxx:4945:3
frame #18: 0x000000010006c7c5 mhttpd`mg_http_handler(nc=0x0000000100608860, ev=3, ev_data=0x000070000146dbcc) at mongoose6.cxx:5139:5
frame #19: 0x00000001000650db mhttpd`mg_call(nc=0x0000000100608860, ev_handler=(mhttpd`mg_http_handler(mg_connection*, int, void*) at mongoose6.cxx:4974), ev=3, ev_data=0x000070000146dbcc)(mg_connection*, int, void*), int, void*) at mongoose6.cxx:2120:5
frame #20: 0x00000001000670c8 mhttpd`mg_recv_common(nc=0x0000000100608860, buf=0x0000000100801000, len=401) at mongoose6.cxx:2676:3
frame #21: 0x0000000100066f83 mhttpd`mg_if_recv_tcp_cb(nc=0x0000000100608860, buf=0x0000000100801000, len=401) at mongoose6.cxx:2680:3
frame #22: 0x0000000100069d32 mhttpd`mg_read_from_socket(conn=0x0000000100608860) at mongoose6.cxx:3378:7
frame #23: 0x00000001000695f6 mhttpd`mg_mgr_handle_conn(nc=0x0000000100608860, fd_flags=1, now=1573832400.4958119) at mongoose6.cxx:3511:9
frame #24: 0x0000000100065b74 mhttpd`::mg_mgr_poll(mgr=0x000070000146ded8, timeout_ms=1000) at mongoose6.cxx:3687:5
frame #25: 0x0000000100076087 mhttpd`per_connection_thread_function(param=0x0000000100608860) at mongoose6.cxx:3805:5
frame #26: 0x00007fff746252eb libsystem_pthread.dylib`_pthread_body + 126
frame #27: 0x00007fff74628249 libsystem_pthread.dylib`_pthread_start + 66
frame #28: 0x00007fff7462440d libsystem_pthread.dylib`thread_start + 13
(lldb) .q
|
Attachment 2: mhttpd_2019-11-15-104252_SnoGlobe.crash
|
Process: mhttpd [16037]
Path: /Users/USER/*/mhttpd
Identifier: mhttpd
Version: 0
Code Type: X86-64 (Native)
Parent Process: tcsh [16008]
Responsible: mhttpd [16037]
User ID: 501
Date/Time: 2019-11-15 10:42:51.751 -0500
OS Version: Mac OS X 10.14.6 (18G95)
Report Version: 12
Anonymous UUID: 27D16641-6679-1AAE-440D-3CF11831B5B1
Sleep/Wake UUID: 6841695D-3153-4F9B-A340-F98C52E55F40
Time Awake Since Boot: 760000 seconds
Time Since Wake: 69000 seconds
System Integrity Protection: enabled
Crashed Thread: 4
Exception Type: EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note: EXC_CORPSE_NOTIFY
Application Specific Information:
abort() called
mhttpd(16037,0x70000256e000) malloc: *** error for object 0x7fe8ba000030: pointer being freed was not allocated
Thread 0:: Dispatch queue: com.apple.main-thread
0 libsystem_kernel.dylib 0x00007fff7456d61a __select + 10
1 mhttpd 0x00000001003719ae mg_mgr_poll + 846 (mongoose6.cxx:3651)
2 mhttpd 0x0000000100355ca5 loop_mg() + 133 (mhttpd.cxx:18013)
3 mhttpd 0x00000001003567c9 main + 2825 (mhttpd.cxx:18271)
4 libdyld.dylib 0x00007fff744313d5 start + 1
Thread 1:
0 libsystem_kernel.dylib 0x00007fff74569f32 __semwait_signal + 10
1 libsystem_c.dylib 0x00007fff744f5914 nanosleep + 199
2 mhttpd 0x00000001003efa3e ss_sleep(int) + 126 (system.cxx:3310)
3 mhttpd 0x00000001003b43da cm_watchdog_thread(void*) + 90 (midas.cxx:6453)
4 libsystem_pthread.dylib 0x00007fff746252eb _pthread_body + 126
5 libsystem_pthread.dylib 0x00007fff74628249 _pthread_start + 66
6 libsystem_pthread.dylib 0x00007fff7462440d thread_start + 13
Thread 2:
0 libsystem_kernel.dylib 0x00007fff7456d61a __select + 10
1 mhttpd 0x00000001003719ae mg_mgr_poll + 846 (mongoose6.cxx:3651)
2 mhttpd 0x0000000100382087 per_connection_thread_function(void*) + 71 (mongoose6.cxx:3804)
3 libsystem_pthread.dylib 0x00007fff746252eb _pthread_body + 126
4 libsystem_pthread.dylib 0x00007fff74628249 _pthread_start + 66
5 libsystem_pthread.dylib 0x00007fff7462440d thread_start + 13
Thread 3:
0 mhttpd 0x0000000100405477 std::__1::enable_if<(is_move_constructible<MJsonNode**>::value) && (is_move_assignable<MJsonNode**>::value), void>::type std::__1::swap<MJsonNode**>(MJsonNode**&, MJsonNode**&) + 55 (type_traits:4506)
1 mhttpd 0x0000000100404fbd std::__1::vector<MJsonNode*, std::__1::allocator<MJsonNode*> >::__swap_out_circular_buffer(std::__1::__split_buffer<MJsonNode*, std::__1::allocator<MJsonNode*>&>&) + 125 (vector:934)
2 mhttpd 0x0000000100404d8c void std::__1::vector<MJsonNode*, std::__1::allocator<MJsonNode*> >::__push_back_slow_path<MJsonNode* const&>(MJsonNode* const&&&) + 172 (vector:1615)
3 mhttpd 0x00000001003ff204 std::__1::vector<MJsonNode*, std::__1::allocator<MJsonNode*> >::push_back(MJsonNode* const&) + 164 (vector:1633)
4 mhttpd 0x00000001003ff781 MJsonNode::AddToObject(char const*, MJsonNode*) + 113 (mjson.cxx:899)
5 mhttpd 0x0000000100410112 js_db_get_values(MJsonNode const*) + 4258 (mjsonrpc.cxx:767)
6 mhttpd 0x000000010042208b mjsonrpc_handle_request(MJsonNode const*) + 2475 (mjsonrpc.cxx:4150)
7 mhttpd 0x00000001004213dd mjsonrpc_decode_post_data(char const*) + 1293 (mjsonrpc.cxx:4265)
8 mhttpd 0x000000010035fa4a handle_http_post(mg_connection*, http_message const*, char const*, RequestTrace*) + 858 (mhttpd.cxx:17545)
9 mhttpd 0x000000010035dd41 handle_http_message(mg_connection*, http_message*) + 1745 (mhttpd.cxx:17744)
10 mhttpd 0x00000001003559e0 handle_http_event_mg(mg_connection*, int, void*) + 96 (mhttpd.cxx:17766)
11 mhttpd 0x00000001003710db mg_call(mg_connection*, void (*)(mg_connection*, int, void*), int, void*) + 427 (mongoose6.cxx:2121)
12 mhttpd 0x000000010037892b mg_http_call_endpoint_handler(mg_connection*, int, http_message*) + 203 (mongoose6.cxx:4947)
13 mhttpd 0x00000001003787c5 mg_http_handler(mg_connection*, int, void*) + 1365 (mongoose6.cxx:5141)
14 mhttpd 0x00000001003710db mg_call(mg_connection*, void (*)(mg_connection*, int, void*), int, void*) + 427 (mongoose6.cxx:2121)
15 mhttpd 0x00000001003730c8 mg_recv_common(mg_connection*, void*, int) + 312 (mongoose6.cxx:2677)
16 mhttpd 0x0000000100372f83 mg_if_recv_tcp_cb(mg_connection*, void*, int) + 35 (mongoose6.cxx:2681)
17 mhttpd 0x0000000100375d32 mg_read_from_socket(mg_connection*) + 514 (mongoose6.cxx:3379)
18 mhttpd 0x00000001003755f6 mg_mgr_handle_conn(mg_connection*, int, double) + 486
19 mhttpd 0x0000000100371b74 mg_mgr_poll + 1300 (mongoose6.cxx:3663)
20 mhttpd 0x0000000100382087 per_connection_thread_function(void*) + 71 (mongoose6.cxx:3804)
21 libsystem_pthread.dylib 0x00007fff746252eb _pthread_body + 126
22 libsystem_pthread.dylib 0x00007fff74628249 _pthread_start + 66
23 libsystem_pthread.dylib 0x00007fff7462440d thread_start + 13
Thread 4 Crashed:
0 libsystem_kernel.dylib 0x00007fff7456c2c6 __pthread_kill + 10
1 libsystem_pthread.dylib 0x00007fff74627bf1 pthread_kill + 284
2 libsystem_c.dylib 0x00007fff744d66a6 abort + 127
3 libsystem_malloc.dylib 0x00007fff745e5077 malloc_vreport + 545
4 libsystem_malloc.dylib 0x00007fff745e4e38 malloc_report + 151
5 mhttpd 0x000000010035be53 HsSchemaEntry::~HsSchemaEntry() + 35 (mhttpd.cxx:16983)
6 mhttpd 0x000000010035be25 HsSchemaEntry::~HsSchemaEntry() + 21 (mhttpd.cxx:16983)
7 mhttpd 0x000000010035be09 std::__1::allocator<AuthEntry>::destroy(AuthEntry*) + 25 (memory:1880)
8 mhttpd 0x000000010035bddd void std::__1::allocator_traits<std::__1::allocator<AuthEntry> >::__destroy<AuthEntry>(std::__1::integral_constant<bool, true>, std::__1::allocator<AuthEntry>&, AuthEntry*) + 29 (memory:1742)
9 mhttpd 0x000000010035bd9d void std::__1::allocator_traits<std::__1::allocator<AuthEntry> >::destroy<AuthEntry>(std::__1::allocator<AuthEntry>&, AuthEntry*) + 29 (memory:1595)
10 mhttpd 0x000000010035bd58 std::__1::__vector_base<AuthEntry, std::__1::allocator<AuthEntry> >::__destruct_at_end(AuthEntry*) + 88 (vector:421)
11 mhttpd 0x000000010035bc68 std::__1::__vector_base<AuthEntry, std::__1::allocator<AuthEntry> >::clear() + 24 (vector:364)
12 mhttpd 0x000000010035bf76 std::__1::vector<AuthEntry, std::__1::allocator<AuthEntry> >::clear() + 38 (vector:767)
13 mhttpd 0x00000001003554d3 read_passwords(Auth*) + 195 (mhttpd.cxx:17049)
14 mhttpd 0x000000010035d9d5 handle_http_message(mg_connection*, http_message*) + 869 (mhttpd.cxx:17719)
15 mhttpd 0x00000001003559e0 handle_http_event_mg(mg_connection*, int, void*) + 96 (mhttpd.cxx:17766)
16 mhttpd 0x00000001003710db mg_call(mg_connection*, void (*)(mg_connection*, int, void*), int, void*) + 427 (mongoose6.cxx:2121)
17 mhttpd 0x000000010037892b mg_http_call_endpoint_handler(mg_connection*, int, http_message*) + 203 (mongoose6.cxx:4947)
18 mhttpd 0x00000001003787c5 mg_http_handler(mg_connection*, int, void*) + 1365 (mongoose6.cxx:5141)
19 mhttpd 0x00000001003710db mg_call(mg_connection*, void (*)(mg_connection*, int, void*), int, void*) + 427 (mongoose6.cxx:2121)
20 mhttpd 0x00000001003730c8 mg_recv_common(mg_connection*, void*, int) + 312 (mongoose6.cxx:2677)
21 mhttpd 0x0000000100372f83 mg_if_recv_tcp_cb(mg_connection*, void*, int) + 35 (mongoose6.cxx:2681)
22 mhttpd 0x0000000100375d32 mg_read_from_socket(mg_connection*) + 514 (mongoose6.cxx:3379)
23 mhttpd 0x00000001003755f6 mg_mgr_handle_conn(mg_connection*, int, double) + 486
24 mhttpd 0x0000000100371b74 mg_mgr_poll + 1300 (mongoose6.cxx:3663)
25 mhttpd 0x0000000100382087 per_connection_thread_function(void*) + 71 (mongoose6.cxx:3804)
26 libsystem_pthread.dylib 0x00007fff746252eb _pthread_body + 126
27 libsystem_pthread.dylib 0x00007fff74628249 _pthread_start + 66
28 libsystem_pthread.dylib 0x00007fff7462440d thread_start + 13
Thread 5:
0 libsystem_kernel.dylib 0x00007fff7456d61a __select + 10
1 mhttpd 0x00000001003719ae mg_mgr_poll + 846 (mongoose6.cxx:3651)
2 mhttpd 0x0000000100382087 per_connection_thread_function(void*) + 71 (mongoose6.cxx:3804)
3 libsystem_pthread.dylib 0x00007fff746252eb _pthread_body + 126
4 libsystem_pthread.dylib 0x00007fff74628249 _pthread_start + 66
5 libsystem_pthread.dylib 0x00007fff7462440d thread_start + 13
Thread 6:
0 libsystem_kernel.dylib 0x00007fff7456d61a __select + 10
1 mhttpd 0x00000001003719ae mg_mgr_poll + 846 (mongoose6.cxx:3651)
2 mhttpd 0x0000000100382087 per_connection_thread_function(void*) + 71 (mongoose6.cxx:3804)
3 libsystem_pthread.dylib 0x00007fff746252eb _pthread_body + 126
4 libsystem_pthread.dylib 0x00007fff74628249 _pthread_start + 66
5 libsystem_pthread.dylib 0x00007fff7462440d thread_start + 13
Thread 4 crashed with X86 Thread State (64-bit):
rax: 0x0000000000000000 rbx: 0x000070000256e000 rcx: 0x000070000256c9f8 rdx: 0x0000000000000000
rdi: 0x0000000000000d07 rsi: 0x0000000000000006 rbp: 0x000070000256ca30 rsp: 0x000070000256c9f8
r8: 0x0000000000000000 r9: 0x000070000256c950 r10: 0x0000000000000000 r11: 0x0000000000000206
r12: 0x0000000000000d07 r13: 0x0000000100ee1000 r14: 0x0000000000000006 r15: 0x000000000000002d
rip: 0x00007fff7456c2c6 rfl: 0x0000000000000206 cr2: 0x000000072c967008
Logical CPU: 0
Error Code: 0x0200005d
Trap Number: 133
Binary Images:
0x10030c000 - 0x1004afff7 +mhttpd (0) <7F574F87-D453-37BF-B8C0-97FD8E8A6C84> /Users/USER/*/mhttpd
0x104f06000 - 0x104f7070f dyld (655.1.1) <DFC3C4AF-6F97-3B34-B18D-7DCB23F2A83A> /usr/lib/dyld
0x7fff6f14e000 - 0x7fff6f14ffff com.apple.TrustEvaluationAgent (2.0 - 31.200.1) <15DF9C73-54E4-3C41-BCF4-378338C55FB4> /System/Library/PrivateFrameworks/TrustEvaluationAgent.framework/Versions/A/TrustEvaluationAgent
0x7fff71418000 - 0x7fff71419ffb libSystem.B.dylib (1252.250.1) <B1006948-7AD0-3CA9-81E0-833F4DD6BFB4> /usr/lib/libSystem.B.dylib
0x7fff7165b000 - 0x7fff716aeff7 libc++.1.dylib (400.9.4) <9A60A190-6C34-339F-BB3D-AACE942009A4> /usr/lib/libc++.1.dylib
0x7fff716af000 - 0x7fff716c4ff7 libc++abi.dylib (400.17) <38C09CED-9090-3719-90F3-04A2749F5428> /usr/lib/libc++abi.dylib
0x7fff71b1a000 - 0x7fff71c12ff7 libcrypto.35.dylib (22.260.1) <91C3D71A-4D1D-331D-89CC-67863DF10574> /usr/lib/libcrypto.35.dylib
0x7fff72c4f000 - 0x7fff733d4fdf libobjc.A.dylib (756.2) <7C312627-43CB-3234-9324-4DEA92D59F50> /usr/lib/libobjc.A.dylib
0x7fff737e0000 - 0x7fff73813ff3 libssl.35.dylib (22.260.1) <EEBCC1DE-A2C2-30CC-AEA3-1B2193B11DAA> /usr/lib/libssl.35.dylib
0x7fff73abe000 - 0x7fff73ad0ff7 libz.1.dylib (70.200.4) <B048FC1F-058F-3A08-A1FE-81D5308CB3E6> /usr/lib/libz.1.dylib
0x7fff742b4000 - 0x7fff742b8ff3 libcache.dylib (81) <1987D1E1-DB11-3291-B12A-EBD55848E02D> /usr/lib/system/libcache.dylib
0x7fff742b9000 - 0x7fff742c3ff3 libcommonCrypto.dylib (60118.250.2) <1765BB6E-6784-3653-B16B-CB839721DC9A> /usr/lib/system/libcommonCrypto.dylib
0x7fff742c4000 - 0x7fff742cbff7 libcompiler_rt.dylib (63.4) <5212BA7B-B7EA-37B4-AF6E-AC4F507EDFB8> /usr/lib/system/libcompiler_rt.dylib
0x7fff742cc000 - 0x7fff742d5ff7 libcopyfile.dylib (146.250.1) <98CD00CD-9B91-3B5C-A9DB-842638050FA8> /usr/lib/system/libcopyfile.dylib
0x7fff742d6000 - 0x7fff7435afc3 libcorecrypto.dylib (602.260.2) <01464D24-570C-3B83-9D18-467769E0FCDD> /usr/lib/system/libcorecrypto.dylib
0x7fff743e1000 - 0x7fff7441aff7 libdispatch.dylib (1008.270.1) <97273678-E94C-3C8C-89F6-2E2020F4B43B> /usr/lib/system/libdispatch.dylib
0x7fff7441b000 - 0x7fff74447ff7 libdyld.dylib (655.1.1) <002418CC-AD11-3D10-865B-015591D24E6C> /usr/lib/system/libdyld.dylib
0x7fff74448000 - 0x7fff74448ffb libkeymgr.dylib (30) <0D0F9CA2-8D5A-3273-8723-59987B5827F2> /usr/lib/system/libkeymgr.dylib
0x7fff74456000 - 0x7fff74456ff7 liblaunch.dylib (1336.261.2) <2B07E27E-D404-3E98-9D28-BCA641E5C479> /usr/lib/system/liblaunch.dylib
0x7fff74457000 - 0x7fff7445cfff libmacho.dylib (927.0.3) <A377D608-77AB-3F6E-90F0-B4F251A5C12F> /usr/lib/system/libmacho.dylib
0x7fff7445d000 - 0x7fff7445fffb libquarantine.dylib (86.220.1) <6D0BC770-7348-3608-9254-F7FFBD347634> /usr/lib/system/libquarantine.dylib
0x7fff74460000 - 0x7fff74461ff7 libremovefile.dylib (45.200.2) <9FBEB2FF-EEBE-31BC-BCFC-C71F8D0E99B6> /usr/lib/system/libremovefile.dylib
0x7fff74462000 - 0x7fff74479ff3 libsystem_asl.dylib (356.200.4) <A62A7249-38B8-33FA-9875-F1852590796C> /usr/lib/system/libsystem_asl.dylib
0x7fff7447a000 - 0x7fff7447aff7 libsystem_blocks.dylib (73) <A453E8EE-860D-3CED-B5DC-BE54E9DB4348> /usr/lib/system/libsystem_blocks.dylib
0x7fff7447b000 - 0x7fff74502fff libsystem_c.dylib (1272.250.1) <7EDACF78-2FA3-35B8-B051-D70475A35117> /usr/lib/system/libsystem_c.dylib
0x7fff74503000 - 0x7fff74506ffb libsystem_configuration.dylib (963.270.3) <2B4A836D-68A4-33E6-8D48-CD4486B03387> /usr/lib/system/libsystem_configuration.dylib
0x7fff74507000 - 0x7fff7450aff7 libsystem_coreservices.dylib (66) <719F75A4-74C5-3BA6-A09E-0C5A3E5889D7> /usr/lib/system/libsystem_coreservices.dylib
0x7fff7450b000 - 0x7fff74511fff libsystem_darwin.dylib (1272.250.1) <EC9B39A5-9592-3577-8997-7DC721D20D8C> /usr/lib/system/libsystem_darwin.dylib
0x7fff74512000 - 0x7fff74518ff7 libsystem_dnssd.dylib (878.270.2) <E9A5ACCF-E35F-3909-AF0A-2A37CD217276> /usr/lib/system/libsystem_dnssd.dylib
0x7fff74519000 - 0x7fff74564ffb libsystem_info.dylib (517.200.9) <D09D5AE0-2FDC-3A6D-93EC-729F931B1457> /usr/lib/system/libsystem_info.dylib
0x7fff74565000 - 0x7fff7458dff7 libsystem_kernel.dylib (4903.271.2) <EA204E3C-870B-30DD-B4AF-D1BB66420D14> /usr/lib/system/libsystem_kernel.dylib
0x7fff7458e000 - 0x7fff745d9ff7 libsystem_m.dylib (3158.200.7) <F19B6DB7-014F-3820-831F-389CCDA06EF6> /usr/lib/system/libsystem_m.dylib
0x7fff745da000 - 0x7fff74604fff libsystem_malloc.dylib (166.270.1) <011F3AD0-8E6A-3A89-AE64-6E5F6840F30A> /usr/lib/system/libsystem_malloc.dylib
0x7fff74605000 - 0x7fff7460fff7 libsystem_networkextension.dylib (767.250.2) <FF06F13A-AEFE-3A27-A073-910EF78AEA36> /usr/lib/system/libsystem_networkextension.dylib
0x7fff74610000 - 0x7fff74617fff libsystem_notify.dylib (172.200.21) <145B5CFC-CF73-33CE-BD3D-E8DDE268FFDE> /usr/lib/system/libsystem_notify.dylib
0x7fff74618000 - 0x7fff74621fef libsystem_platform.dylib (177.270.1) <9D1FE5E4-EB7D-3B3F-A8D1-A96D9CF1348C> /usr/lib/system/libsystem_platform.dylib
0x7fff74622000 - 0x7fff7462cff7 libsystem_pthread.dylib (330.250.2) <2D5C08FF-484F-3D59-9132-CE1DCB3F76D7> /usr/lib/system/libsystem_pthread.dylib
0x7fff7462d000 - 0x7fff74630ff7 libsystem_sandbox.dylib (851.270.1) <9494594B-5199-3186-82AB-5FF8BED6EE16> /usr/lib/system/libsystem_sandbox.dylib
0x7fff74631000 - 0x7fff74633ff3 libsystem_secinit.dylib (30.260.2) <EF1EA47B-7B22-35E8-BD9B-F7003DCB96AE> /usr/lib/system/libsystem_secinit.dylib
0x7fff74634000 - 0x7fff7463bff3 libsystem_symptoms.dylib (820.267.1) <03F1C2DD-0F5A-3D9D-88F6-B26C0F94EB52> /usr/lib/system/libsystem_symptoms.dylib
0x7fff7463c000 - 0x7fff74651fff libsystem_trace.dylib (906.260.1) <FC761C3B-5434-3A52-912D-F1B15FAA8EB2> /usr/lib/system/libsystem_trace.dylib
0x7fff74653000 - 0x7fff74658ffb libunwind.dylib (35.4) <24A97A67-F017-3CFC-B0D0-6BD0224B1336> /usr/lib/system/libunwind.dylib
0x7fff74659000 - 0x7fff74688fff libxpc.dylib (1336.261.2) <7DEE2300-6D8E-3C00-9C63-E3E80D56B0C4> /usr/lib/system/libxpc.dylib
External Modification Summary:
Calls made by other processes targeting this process:
task_for_pid: 1
thread_create: 0
thread_set_state: 0
Calls made by this process:
task_for_pid: 0
thread_create: 0
thread_set_state: 0
Calls made by all processes on this machine:
task_for_pid: 402898
thread_create: 0
thread_set_state: 9
VM Region Summary:
ReadOnly portion of Libraries: Total=238.2M resident=0K(0%) swapped_out_or_unallocated=238.2M(100%)
Writable regions: Total=174.0M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=174.0M(100%)
VIRTUAL REGION
REGION TYPE SIZE COUNT (non-coalesced)
=========== ======= =======
Kernel Alloc Once 8K 1
MALLOC 136.6M 15
Stack 67.1M 14
VM_ALLOCATE 12K 2
__DATA 3488K 44
__LINKEDIT 223.7M 3
__TEXT 14.5M 47
shared memory 1288K 5
=========== ======= =======
TOTAL 446.5M 131
|
1736
|
15 Nov 2019 |
Konstantin Olchanski | Bug Report | Newly installed MIDAS on OSX: mhttpd crahes | > It is reproducible alright.
Thanks. At first blush, a guess, read_passwords() is not thread-safe and is called from multiple threads, not protected by semaphore. Crash report shows 2 active threads
(one made it is far as processing the mjson rpc, the other one crashed in read_passwords()).
K.O.
> Here are the core dump and the backtrace (I think the former is more informative).
>
>
>
> > > Context: out of the box MIDAS (using cmake) on OSX Mojave.
> > >
> > > Running with mongoose/opensslm installation following instruction here:
> > > https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux
> > >
> > > mhttpd crashing when midas webpage opened with Safari (12.1.2). Usually when opening the "chat" tab but sometimes also with the "message" tab.
> > > mhttpd(11109,0x70000827a000) malloc: *** error for object 0x7f8669501ef0: pointer being freed was not allocated
> > > mhttpd(11109,0x70000827a000) malloc: *** set a breakpoint in malloc_error_break to debug
> > >
> > > No crash if using firefox (70.0.1 (64-bit))
> >
> > I think we also have reports of mhttpd crash on macos with safari from the Dragon experiment,
> > but cannot reproduce the problem.
> >
> > If you can reproduce this, can you capture the crash stack trace?
> >
> > One way to do this is to enable core dumps in odb "/expt/enable core dumps" set to "y", restart mhttpd,
> > wait for the crash. I think macos writes core dumps into /cores/... Or you can run mhttpd inside lldb
> > and wait for the crash. the lldb command to show the stack trace is "bt", but you may need
> > to switch to different threads to see which one actually crashed. I forget what the command
> > for that is.
> >
> > BTW, the mhttpd networking code has not changed in a long time, but an update
> > of mongoose web server library is overdue (to fix a memory leak, at least).
> >
> > K.O. |
|