Back Midas Rome Roody Rootana
  Midas DAQ System, Page 56 of 142  Not logged in ELOG logo
 Find |  Login |  Help 
 Full | Summary | Threaded | Hide attachments  Show only new entries   2828 Entries 
Goto page Previous  1, 2, 3 ... 55, 56, 57 ... 140, 141, 142   Next  
ID Date Author Topic Subject
  1742   28 Nov 2019 Konstantin OlchanskiBug Reportmidas alarm sound unreliable in google-chrome
I accidentally discovered a problem with the alarm sounds played by midas.

The javascript code is very simple: var audio=new Audio("alarm.mp3"); audio.play();

In the past, this reliably played a sound.

More recently, I started seeing javascript log messages about "unhandled exception" from audio.play(). (for me, they often
interrupt my javascript debugger sessions, in a very annoying way).

Adding an exception handler to audio.play() was not effective: it turns out, these days, audio.play() returns a Promise
and one must handle the Promise rejection case. (also it turns out that in the rejection handler
one *must* clear audio.src to avoid problems with excessive memory and cpu use).

But why does audio.play() throw a rejected promise?!?

This is the error message provided: "play() failed because the user didn't interact with the document first. https://goo.gl/xX8pDD"
name: "NotAllowedError".

The link takes us to https://developers.google.com/web/updates/2017/09/autoplay-policy-changes. This web page seems to explain
everything neatly, but most of the information turns out to be wrong and unhelpful:

a) "Muted autoplay is always allowed" - wrong - I see audio.play() rejection from muted tabs
b) "User has interacted with the domain (click, tap, etc.)." - wrong - I see rejection in the javascript debugger even as I debug the web page
c) "user's Media Engagement Index threshold has been crossed" - broken - playback count and MEI is always 0 (zero) for midas - per chrome://media-engagement/
d) "user has added the site to their home screen on mobile or installed the PWA on desktop" - what ?!?
e) chrome://flags/#autoplay-policy does not exist (was removed)

There may be a way to start google-chrome with special flags to globally enable autoplay, but it seems
to be aimed at "kiosk" type applications, not for general use browsers:
https://stackoverflow.com/questions/57455849/chrome-autoplay-policy-chrome-76

The bottom line:

there is no way to ensure that the alarm sound will always play

K.O.

P.S. I am tracking this problem here:
https://bitbucket.org/tmidas/midas/issues/191/exception-on-audioplay
  1741   18 Nov 2019 Konstantin OlchanskiSuggestionjavascript comunication
> > Hi, Stefan - I did not look at your code, if all midas tabs are inactive, will the alarm sound still play?
> I added some code to do ONLY alarm updates if in the background (once every 10 seconds)

Ok, good.

> I don't have however a solution to your problem "not enough interaction with page" which I have never seen so far.

When it happens, you should see my messages about it in the javascript console. (rejected promise from audio.play()).

How to make it happen so one can see it and be sure it is handled correctly (audio.src has to be set to blank, at the least), I have no idea.

What I have been looking at for the huge memory and cpu use problem, I would call "a bad interaction between two kludges",
the first kludge is the throttling javascript in inactive tabs, the second kludge is the "not interacted" rejection of audio.play() promise.
(audio.play() returning a promise, and the strange sequencing between this promise success/reject and
the audio events loaded/canplay/done/etc smells of a kludge, too).

> I wonder if we should try push notifications: https://developers.google.com/web/fundamentals/codelabs/push-notifications
> which seems a bit complicated to me. It might also be too subtle if someone is sleeping in front of the computer.

I did not read all the explanation, but if it requires use of 3rd party services, I think we cannot use it, we do not want
to miss "experiment is on fire" alarms just because google is down.

K.O.
  1740   18 Nov 2019 Stefan RittSuggestionjavascript comunication
> a) google chrome slows down the execution of javascript in inactive tabs, leading to trouble
> with memory management - midas pages poll at 1/sec, each poll allocates memory for processing RPC messages,
> and (until recently) allocates memory for new DOM objects to update the web page - but the garbage collector
> gets slowed down and does not keep up - leading to huge memory use (up to 200 Mbytes) for inactive midas pages
> that normally consume 50-ish Mbytes.

Try my latest change, which drops any update if hidden, except the alarm sound. At the moment this only works on the main "status" page.

> most midas web pages poll in two places - mhttpd_refresh() updates the current date timestamp, alarms, currently active midas.log message;
> and each page has it's own loop for updating it's own data (i.e. "alarms" page, "programs" page).

That's correct. I changed mhttpd_refresh() now and the main loop for the "status" page. If that works for everybody, we can do that also for "programs" and other pages. The code is here:

      if (document.hidden) {
         // don't update page if hidden
         setTimeout(update_page, 500);
         return;
      }

where "update_page" has to be replaced with the proper function.

Stefan
  1739   18 Nov 2019 Stefan RittSuggestionjavascript comunication
> Hi, Stefan - I did not look at your code, if all midas tabs are inactive, will the alarm sound still play?

Nope. All updates are done in mhhtpd_refresh(), and I changed it such that nothing is updated if hidden.

I agree however that this is bad. You want to hear alarms always. So I added some code to do ONLY alarm updates if in the background. 
No GUI changes, only audio playing. Checking is reduced from 1 Hz to 0.1 Hz (once every 10 seconds). I don't have however a solution 
to your problem "not enough interaction with page" which I have never seen so far.

I wonder if we should try push notifications: https://developers.google.com/web/fundamentals/codelabs/push-notifications
which seems a bit complicated to me. It might also be too subtle if someone is sleeping in front of the computer.

Stefan
  1738   17 Nov 2019 Konstantin OlchanskiSuggestionjavascript comunication
> Very good idea. And thanks for finding the document.hidden solution. I put it in, so give it a try.

Hi, Stefan - I did not look at your code, if all midas tabs are inactive, will the alarm sound still play?

K.O.


> 
> Best,
> Stefan
> 
> 
> > I am currently testing the new history system on the mhttpd side and stumbled over the following issue: typically our user open a lot of midas web-page tabs and keep them open. With the current version this leads after a night typically to a state where the browser is busy with itself and not reacting anymore.
> > 
> > One important reason seems to be that ALL tabs trying to communicate all the time which is totally unnecessary, since I think a hidden tab should stay in a sleeping mode. 
> > 
> > I was browsing if there is a way to find out if a tab is active or not, and found the following API which exactly does this:
> > 
> > https://developer.mozilla.org/en-US/docs/Web/API/Page_Visibility_API
> > 
> > Furthermore, the simple
> > 
> > document.hidden 
> > 
> > tag, could be used to find out if the page is currently active.
> > 
> > Wouldn't it a good idea to send all midas tabs which are not active into a sleep mode and only reactivate them if they come into focus?
> > 
> > I had a quick look at the JavaScript libs of midas, but I am not quite certain where to best inject this.
  1737   17 Nov 2019 Konstantin OlchanskiSuggestionjavascript comunication
> I am currently testing the new history system on the mhttpd side and stumbled over the following issue:
> typically our user open a lot of midas web-page tabs and keep them open. With the current version this leads after a night typically to a state where the browser is busy with itself and not reacting anymore.
> 
> One important reason seems to be that ALL tabs trying to communicate all the time which is totally unnecessary, since I think a hidden tab should stay in a sleeping mode. 

I am looking at two more problems with inactive tabs:

a) google chrome slows down the execution of javascript in inactive tabs, leading to trouble
with memory management - midas pages poll at 1/sec, each poll allocates memory for processing RPC messages,
and (until recently) allocates memory for new DOM objects to update the web page - but the garbage collector
gets slowed down and does not keep up - leading to huge memory use (up to 200 Mbytes) for inactive midas pages
that normally consume 50-ish Mbytes.

b) the playing of the alarm sound is throttled by "user has not interacted with the document" thing, but there is a bug -
instead of canceling the playing of the alarm sound, the sound file is still loaded, (but not played). (this is hard to debug
because I do not know how to manually trigger the "user has not interacted..." condition, I have to wait for many days for it.
Then, for inactive tabs, the loading of the sound files is slowed down, leading to many of them getting queued up,
and eventually they all try to load and play at the same time, again leading to huge memory and cpu use in inactive tabs.
(this sounds incredible because we play the alarm sound at most 1/minute, for sure the previous sound file must have
finished playing by then, but no, it is easy to see it happen - add a few console.log messages and wait for a few days).

>
> I was browsing if there is a way to find out if a tab is active or not, and found the following API which exactly does this:
> https://developer.mozilla.org/en-US/docs/Web/API/Page_Visibility_API
> 

From looking at the inactive tab business, I see that javascript in inactive tabs runs quite differently from javascript
in active tabs (i.e. timers do not work the same) and I see how the "visibility api" had to be invented to counter that.

> 
> Wouldn't it a good idea to send all midas tabs which are not active into a sleep mode and only reactivate them if they come into focus?
> I had a quick look at the JavaScript libs of midas, but I am not quite certain where to best inject this.
>

most midas web pages poll in two places - mhttpd_refresh() updates the current date timestamp, alarms, currently active midas.log message;
and each page has it's own loop for updating it's own data (i.e. "alarms" page, "programs" page).

we should be careful to not completely disable all polling as some experiments do use and do rely on the midas producing
loud alarm messages ("program logged is not running!!!", "program mhttpd aborted!!!"). Even if all midas tabs are inactive,
some javascript is some tab still has to run frequently enough to poll midas and to sound the alarm sounds (even though
I am not sure how to 100% reliably counteract the google-chrome not playing sound files because
of the "user did not interact with the site..." thing).

K.O.
  1736   15 Nov 2019 Konstantin OlchanskiBug ReportNewly installed MIDAS on OSX: mhttpd crahes
> It is reproducible alright.

Thanks. At first blush, a guess, read_passwords() is not thread-safe and is called from multiple threads, not protected by semaphore. Crash report shows 2 active threads 
(one made it is far as processing the mjson rpc, the other one crashed in read_passwords()).

K.O.


> Here are the core dump and the backtrace (I think  the former is more informative).
> 
> 
> 
> > > Context: out of the box  MIDAS (using cmake) on OSX Mojave. 
> > > 
> > > Running with mongoose/opensslm installation following instruction here:
> > > https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux
> > > 
> > > mhttpd crashing when midas webpage opened with Safari (12.1.2). Usually when opening the "chat" tab but sometimes also with the "message" tab.
> > > mhttpd(11109,0x70000827a000) malloc: *** error for object 0x7f8669501ef0: pointer being freed was not allocated
> > > mhttpd(11109,0x70000827a000) malloc: *** set a breakpoint in malloc_error_break to debug
> > > 
> > > No crash if using firefox (70.0.1 (64-bit))
> > 
> > I think we also have reports of mhttpd crash on macos with safari from the Dragon experiment,
> > but cannot reproduce the problem.
> > 
> > If you can reproduce this, can you capture the crash stack trace?
> > 
> > One way to do this is to enable core dumps in odb "/expt/enable core dumps" set to "y", restart mhttpd,
> > wait for the crash. I think macos writes core dumps into /cores/... Or you can run mhttpd inside lldb
> > and wait for the crash. the lldb command to show the stack trace is "bt", but you may need
> > to switch to different threads to see which one actually crashed. I forget what the command
> > for that is.
> > 
> > BTW, the mhttpd networking code has not changed in a long time, but an update
> > of mongoose web server library is overdue (to fix a memory leak, at least).
> > 
> > K.O.
  1735   15 Nov 2019 Stefan RittSuggestionjavascript comunication
Very good idea. And thanks for finding the document.hidden solution. I put it in, so give it a try.

Best,
Stefan


> I am currently testing the new history system on the mhttpd side and stumbled over the following issue: typically our user open a lot of midas web-page tabs and keep them open. With the current version this leads after a night typically to a state where the browser is busy with itself and not reacting anymore.
> 
> One important reason seems to be that ALL tabs trying to communicate all the time which is totally unnecessary, since I think a hidden tab should stay in a sleeping mode. 
> 
> I was browsing if there is a way to find out if a tab is active or not, and found the following API which exactly does this:
> 
> https://developer.mozilla.org/en-US/docs/Web/API/Page_Visibility_API
> 
> Furthermore, the simple
> 
> document.hidden 
> 
> tag, could be used to find out if the page is currently active.
> 
> Wouldn't it a good idea to send all midas tabs which are not active into a sleep mode and only reactivate them if they come into focus?
> 
> I had a quick look at the JavaScript libs of midas, but I am not quite certain where to best inject this.
  1734   15 Nov 2019 Pierre GorelBug ReportNewly installed MIDAS on OSX: mhttpd crahes
It is reproducible alright.
Here are the core dump and the backtrace (I think  the former is more informative).



> > Context: out of the box  MIDAS (using cmake) on OSX Mojave. 
> > 
> > Running with mongoose/opensslm installation following instruction here:
> > https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux
> > 
> > mhttpd crashing when midas webpage opened with Safari (12.1.2). Usually when opening the "chat" tab but sometimes also with the "message" tab.
> > mhttpd(11109,0x70000827a000) malloc: *** error for object 0x7f8669501ef0: pointer being freed was not allocated
> > mhttpd(11109,0x70000827a000) malloc: *** set a breakpoint in malloc_error_break to debug
> > 
> > No crash if using firefox (70.0.1 (64-bit))
> 
> I think we also have reports of mhttpd crash on macos with safari from the Dragon experiment,
> but cannot reproduce the problem.
> 
> If you can reproduce this, can you capture the crash stack trace?
> 
> One way to do this is to enable core dumps in odb "/expt/enable core dumps" set to "y", restart mhttpd,
> wait for the crash. I think macos writes core dumps into /cores/... Or you can run mhttpd inside lldb
> and wait for the crash. the lldb command to show the stack trace is "bt", but you may need
> to switch to different threads to see which one actually crashed. I forget what the command
> for that is.
> 
> BTW, the mhttpd networking code has not changed in a long time, but an update
> of mongoose web server library is overdue (to fix a memory leak, at least).
> 
> K.O.
Attachment 1: mhttpd_lldb_bt.txt 
SnoGlobe:~/packages/midas/build> lldb mhttpd
(lldb) target create "mhttpd"
Current executable set to 'mhttpd' (x86_64).
(lldb) r
Process 15988 launched: '/Users/acquis/packages/midas/bin/mhttpd' (x86_64)
Mongoose web server will use SSL certificate file "/Users/acquis/online/ssl_cert.pem"
Mongoose web server will use authentication realm "snoglobe", password file "/Users/acquis/online/htpasswd.txt"
mongoose web server is redirecting HTTP port 8080 to https://SnoGlobe.local:8443
mongoose web server is listening on the HTTP port 8080
mongoose web server is listening on the HTTPS port 8443
mhttpd(15988,0x70000146e000) malloc: *** error for object 0x100608790: pointer being freed was not allocated
mhttpd(15988,0x70000146e000) malloc: *** set a breakpoint in malloc_error_break to debug
Process 15988 stopped
* thread #2, stop reason = signal SIGABRT
    frame #0: 0x00007fff7456c2c6 libsystem_kernel.dylib`__pthread_kill + 10
libsystem_kernel.dylib`__pthread_kill:
->  0x7fff7456c2c6 <+10>: jae    0x7fff7456c2d0            ; <+20>
    0x7fff7456c2c8 <+12>: movq   %rax, %rdi
    0x7fff7456c2cb <+15>: jmp    0x7fff74566457            ; cerror_nocancel
    0x7fff7456c2d0 <+20>: retq   
Target 0: (mhttpd) stopped.
(lldb) bt
* thread #2, stop reason = signal SIGABRT
  * frame #0: 0x00007fff7456c2c6 libsystem_kernel.dylib`__pthread_kill + 10
    frame #1: 0x00007fff74627bf1 libsystem_pthread.dylib`pthread_kill + 284
    frame #2: 0x00007fff744d66a6 libsystem_c.dylib`abort + 127
    frame #3: 0x00007fff745e5077 libsystem_malloc.dylib`malloc_vreport + 545
    frame #4: 0x00007fff745e4e38 libsystem_malloc.dylib`malloc_report + 151
    frame #5: 0x000000010004fe53 mhttpd`AuthEntry::~AuthEntry(this=0x0000000101300220) at mhttpd.cxx:16983:8
    frame #6: 0x000000010004fe25 mhttpd`AuthEntry::~AuthEntry(this=0x0000000101300220) at mhttpd.cxx:16983:8
    frame #7: 0x000000010004fe09 mhttpd`std::__1::allocator<AuthEntry>::destroy(this=0x00000001001b0f10, __p=0x0000000101300220) at memory:1880:64
    frame #8: 0x000000010004fddd mhttpd`void std::__1::allocator_traits<std::__1::allocator<AuthEntry> >::__destroy<AuthEntry>((null)=std::__1::true_type @ 0x000070000146cc28, __a=0x00000001001b0f10, __p=0x0000000101300220) at memory:1742:18
    frame #9: 0x000000010004fd9d mhttpd`void std::__1::allocator_traits<std::__1::allocator<AuthEntry> >::destroy<AuthEntry>(__a=0x00000001001b0f10, __p=0x0000000101300220) at memory:1595:14
    frame #10: 0x000000010004fd58 mhttpd`std::__1::__vector_base<AuthEntry, std::__1::allocator<AuthEntry> >::__destruct_at_end(this=0x00000001001b0f00, __new_last=0x0000000101300220) at vector:421:9
    frame #11: 0x000000010004fc68 mhttpd`std::__1::__vector_base<AuthEntry, std::__1::allocator<AuthEntry> >::clear(this=0x00000001001b0f00) at vector:364:29
    frame #12: 0x000000010004ff76 mhttpd`std::__1::vector<AuthEntry, std::__1::allocator<AuthEntry> >::clear(this=0x00000001001b0f00 size=1) at vector:766:17
    frame #13: 0x00000001000494d3 mhttpd`read_passwords(auth=0x00000001001b0ec8) at mhttpd.cxx:17047:20
    frame #14: 0x00000001000519d5 mhttpd`handle_http_message(nc=0x0000000100608860, msg=0x000070000146d5a0) at mhttpd.cxx:17719:20
    frame #15: 0x00000001000499e0 mhttpd`handle_http_event_mg(nc=0x0000000100608860, ev=100, ev_data=0x000070000146d5a0) at mhttpd.cxx:17765:7
    frame #16: 0x00000001000650db mhttpd`mg_call(nc=0x0000000100608860, ev_handler=(mhttpd`handle_http_event_mg(mg_connection*, int, void*) at mhttpd.cxx:17760), ev=100, ev_data=0x000070000146d5a0)(mg_connection*, int, void*), int, void*) at mongoose6.cxx:2120:5
    frame #17: 0x000000010006c92b mhttpd`mg_http_call_endpoint_handler(nc=0x0000000100608860, ev=100, hm=0x000070000146d5a0) at mongoose6.cxx:4945:3
    frame #18: 0x000000010006c7c5 mhttpd`mg_http_handler(nc=0x0000000100608860, ev=3, ev_data=0x000070000146dbcc) at mongoose6.cxx:5139:5
    frame #19: 0x00000001000650db mhttpd`mg_call(nc=0x0000000100608860, ev_handler=(mhttpd`mg_http_handler(mg_connection*, int, void*) at mongoose6.cxx:4974), ev=3, ev_data=0x000070000146dbcc)(mg_connection*, int, void*), int, void*) at mongoose6.cxx:2120:5
    frame #20: 0x00000001000670c8 mhttpd`mg_recv_common(nc=0x0000000100608860, buf=0x0000000100801000, len=401) at mongoose6.cxx:2676:3
    frame #21: 0x0000000100066f83 mhttpd`mg_if_recv_tcp_cb(nc=0x0000000100608860, buf=0x0000000100801000, len=401) at mongoose6.cxx:2680:3
    frame #22: 0x0000000100069d32 mhttpd`mg_read_from_socket(conn=0x0000000100608860) at mongoose6.cxx:3378:7
    frame #23: 0x00000001000695f6 mhttpd`mg_mgr_handle_conn(nc=0x0000000100608860, fd_flags=1, now=1573832400.4958119) at mongoose6.cxx:3511:9
    frame #24: 0x0000000100065b74 mhttpd`::mg_mgr_poll(mgr=0x000070000146ded8, timeout_ms=1000) at mongoose6.cxx:3687:5
    frame #25: 0x0000000100076087 mhttpd`per_connection_thread_function(param=0x0000000100608860) at mongoose6.cxx:3805:5
    frame #26: 0x00007fff746252eb libsystem_pthread.dylib`_pthread_body + 126
    frame #27: 0x00007fff74628249 libsystem_pthread.dylib`_pthread_start + 66
    frame #28: 0x00007fff7462440d libsystem_pthread.dylib`thread_start + 13
(lldb) .q
Attachment 2: mhttpd_2019-11-15-104252_SnoGlobe.crash 
Process:               mhttpd [16037]
Path:                  /Users/USER/*/mhttpd
Identifier:            mhttpd
Version:               0
Code Type:             X86-64 (Native)
Parent Process:        tcsh [16008]
Responsible:           mhttpd [16037]
User ID:               501

Date/Time:             2019-11-15 10:42:51.751 -0500
OS Version:            Mac OS X 10.14.6 (18G95)
Report Version:        12
Anonymous UUID:        27D16641-6679-1AAE-440D-3CF11831B5B1

Sleep/Wake UUID:       6841695D-3153-4F9B-A340-F98C52E55F40

Time Awake Since Boot: 760000 seconds
Time Since Wake:       69000 seconds

System Integrity Protection: enabled

Crashed Thread:        4

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
abort() called
mhttpd(16037,0x70000256e000) malloc: *** error for object 0x7fe8ba000030: pointer being freed was not allocated
 

Thread 0:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	0x00007fff7456d61a __select + 10
1   mhttpd                        	0x00000001003719ae mg_mgr_poll + 846 (mongoose6.cxx:3651)
2   mhttpd                        	0x0000000100355ca5 loop_mg() + 133 (mhttpd.cxx:18013)
3   mhttpd                        	0x00000001003567c9 main + 2825 (mhttpd.cxx:18271)
4   libdyld.dylib                 	0x00007fff744313d5 start + 1

Thread 1:
0   libsystem_kernel.dylib        	0x00007fff74569f32 __semwait_signal + 10
1   libsystem_c.dylib             	0x00007fff744f5914 nanosleep + 199
2   mhttpd                        	0x00000001003efa3e ss_sleep(int) + 126 (system.cxx:3310)
3   mhttpd                        	0x00000001003b43da cm_watchdog_thread(void*) + 90 (midas.cxx:6453)
4   libsystem_pthread.dylib       	0x00007fff746252eb _pthread_body + 126
5   libsystem_pthread.dylib       	0x00007fff74628249 _pthread_start + 66
6   libsystem_pthread.dylib       	0x00007fff7462440d thread_start + 13

Thread 2:
0   libsystem_kernel.dylib        	0x00007fff7456d61a __select + 10
1   mhttpd                        	0x00000001003719ae mg_mgr_poll + 846 (mongoose6.cxx:3651)
2   mhttpd                        	0x0000000100382087 per_connection_thread_function(void*) + 71 (mongoose6.cxx:3804)
3   libsystem_pthread.dylib       	0x00007fff746252eb _pthread_body + 126
4   libsystem_pthread.dylib       	0x00007fff74628249 _pthread_start + 66
5   libsystem_pthread.dylib       	0x00007fff7462440d thread_start + 13

Thread 3:
0   mhttpd                        	0x0000000100405477 std::__1::enable_if<(is_move_constructible<MJsonNode**>::value) && (is_move_assignable<MJsonNode**>::value), void>::type std::__1::swap<MJsonNode**>(MJsonNode**&, MJsonNode**&) + 55 (type_traits:4506)
1   mhttpd                        	0x0000000100404fbd std::__1::vector<MJsonNode*, std::__1::allocator<MJsonNode*> >::__swap_out_circular_buffer(std::__1::__split_buffer<MJsonNode*, std::__1::allocator<MJsonNode*>&>&) + 125 (vector:934)
2   mhttpd                        	0x0000000100404d8c void std::__1::vector<MJsonNode*, std::__1::allocator<MJsonNode*> >::__push_back_slow_path<MJsonNode* const&>(MJsonNode* const&&&) + 172 (vector:1615)
3   mhttpd                        	0x00000001003ff204 std::__1::vector<MJsonNode*, std::__1::allocator<MJsonNode*> >::push_back(MJsonNode* const&) + 164 (vector:1633)
4   mhttpd                        	0x00000001003ff781 MJsonNode::AddToObject(char const*, MJsonNode*) + 113 (mjson.cxx:899)
5   mhttpd                        	0x0000000100410112 js_db_get_values(MJsonNode const*) + 4258 (mjsonrpc.cxx:767)
6   mhttpd                        	0x000000010042208b mjsonrpc_handle_request(MJsonNode const*) + 2475 (mjsonrpc.cxx:4150)
7   mhttpd                        	0x00000001004213dd mjsonrpc_decode_post_data(char const*) + 1293 (mjsonrpc.cxx:4265)
8   mhttpd                        	0x000000010035fa4a handle_http_post(mg_connection*, http_message const*, char const*, RequestTrace*) + 858 (mhttpd.cxx:17545)
9   mhttpd                        	0x000000010035dd41 handle_http_message(mg_connection*, http_message*) + 1745 (mhttpd.cxx:17744)
10  mhttpd                        	0x00000001003559e0 handle_http_event_mg(mg_connection*, int, void*) + 96 (mhttpd.cxx:17766)
11  mhttpd                        	0x00000001003710db mg_call(mg_connection*, void (*)(mg_connection*, int, void*), int, void*) + 427 (mongoose6.cxx:2121)
12  mhttpd                        	0x000000010037892b mg_http_call_endpoint_handler(mg_connection*, int, http_message*) + 203 (mongoose6.cxx:4947)
13  mhttpd                        	0x00000001003787c5 mg_http_handler(mg_connection*, int, void*) + 1365 (mongoose6.cxx:5141)
14  mhttpd                        	0x00000001003710db mg_call(mg_connection*, void (*)(mg_connection*, int, void*), int, void*) + 427 (mongoose6.cxx:2121)
15  mhttpd                        	0x00000001003730c8 mg_recv_common(mg_connection*, void*, int) + 312 (mongoose6.cxx:2677)
16  mhttpd                        	0x0000000100372f83 mg_if_recv_tcp_cb(mg_connection*, void*, int) + 35 (mongoose6.cxx:2681)
17  mhttpd                        	0x0000000100375d32 mg_read_from_socket(mg_connection*) + 514 (mongoose6.cxx:3379)
18  mhttpd                        	0x00000001003755f6 mg_mgr_handle_conn(mg_connection*, int, double) + 486
19  mhttpd                        	0x0000000100371b74 mg_mgr_poll + 1300 (mongoose6.cxx:3663)
20  mhttpd                        	0x0000000100382087 per_connection_thread_function(void*) + 71 (mongoose6.cxx:3804)
21  libsystem_pthread.dylib       	0x00007fff746252eb _pthread_body + 126
22  libsystem_pthread.dylib       	0x00007fff74628249 _pthread_start + 66
23  libsystem_pthread.dylib       	0x00007fff7462440d thread_start + 13

Thread 4 Crashed:
0   libsystem_kernel.dylib        	0x00007fff7456c2c6 __pthread_kill + 10
1   libsystem_pthread.dylib       	0x00007fff74627bf1 pthread_kill + 284
2   libsystem_c.dylib             	0x00007fff744d66a6 abort + 127
3   libsystem_malloc.dylib        	0x00007fff745e5077 malloc_vreport + 545
4   libsystem_malloc.dylib        	0x00007fff745e4e38 malloc_report + 151
5   mhttpd                        	0x000000010035be53 HsSchemaEntry::~HsSchemaEntry() + 35 (mhttpd.cxx:16983)
6   mhttpd                        	0x000000010035be25 HsSchemaEntry::~HsSchemaEntry() + 21 (mhttpd.cxx:16983)
7   mhttpd                        	0x000000010035be09 std::__1::allocator<AuthEntry>::destroy(AuthEntry*) + 25 (memory:1880)
8   mhttpd                        	0x000000010035bddd void std::__1::allocator_traits<std::__1::allocator<AuthEntry> >::__destroy<AuthEntry>(std::__1::integral_constant<bool, true>, std::__1::allocator<AuthEntry>&, AuthEntry*) + 29 (memory:1742)
9   mhttpd                        	0x000000010035bd9d void std::__1::allocator_traits<std::__1::allocator<AuthEntry> >::destroy<AuthEntry>(std::__1::allocator<AuthEntry>&, AuthEntry*) + 29 (memory:1595)
10  mhttpd                        	0x000000010035bd58 std::__1::__vector_base<AuthEntry, std::__1::allocator<AuthEntry> >::__destruct_at_end(AuthEntry*) + 88 (vector:421)
11  mhttpd                        	0x000000010035bc68 std::__1::__vector_base<AuthEntry, std::__1::allocator<AuthEntry> >::clear() + 24 (vector:364)
12  mhttpd                        	0x000000010035bf76 std::__1::vector<AuthEntry, std::__1::allocator<AuthEntry> >::clear() + 38 (vector:767)
13  mhttpd                        	0x00000001003554d3 read_passwords(Auth*) + 195 (mhttpd.cxx:17049)
14  mhttpd                        	0x000000010035d9d5 handle_http_message(mg_connection*, http_message*) + 869 (mhttpd.cxx:17719)
15  mhttpd                        	0x00000001003559e0 handle_http_event_mg(mg_connection*, int, void*) + 96 (mhttpd.cxx:17766)
16  mhttpd                        	0x00000001003710db mg_call(mg_connection*, void (*)(mg_connection*, int, void*), int, void*) + 427 (mongoose6.cxx:2121)
17  mhttpd                        	0x000000010037892b mg_http_call_endpoint_handler(mg_connection*, int, http_message*) + 203 (mongoose6.cxx:4947)
18  mhttpd                        	0x00000001003787c5 mg_http_handler(mg_connection*, int, void*) + 1365 (mongoose6.cxx:5141)
19  mhttpd                        	0x00000001003710db mg_call(mg_connection*, void (*)(mg_connection*, int, void*), int, void*) + 427 (mongoose6.cxx:2121)
20  mhttpd                        	0x00000001003730c8 mg_recv_common(mg_connection*, void*, int) + 312 (mongoose6.cxx:2677)
21  mhttpd                        	0x0000000100372f83 mg_if_recv_tcp_cb(mg_connection*, void*, int) + 35 (mongoose6.cxx:2681)
22  mhttpd                        	0x0000000100375d32 mg_read_from_socket(mg_connection*) + 514 (mongoose6.cxx:3379)
23  mhttpd                        	0x00000001003755f6 mg_mgr_handle_conn(mg_connection*, int, double) + 486
24  mhttpd                        	0x0000000100371b74 mg_mgr_poll + 1300 (mongoose6.cxx:3663)
25  mhttpd                        	0x0000000100382087 per_connection_thread_function(void*) + 71 (mongoose6.cxx:3804)
26  libsystem_pthread.dylib       	0x00007fff746252eb _pthread_body + 126
27  libsystem_pthread.dylib       	0x00007fff74628249 _pthread_start + 66
28  libsystem_pthread.dylib       	0x00007fff7462440d thread_start + 13

Thread 5:
0   libsystem_kernel.dylib        	0x00007fff7456d61a __select + 10
1   mhttpd                        	0x00000001003719ae mg_mgr_poll + 846 (mongoose6.cxx:3651)
2   mhttpd                        	0x0000000100382087 per_connection_thread_function(void*) + 71 (mongoose6.cxx:3804)
3   libsystem_pthread.dylib       	0x00007fff746252eb _pthread_body + 126
4   libsystem_pthread.dylib       	0x00007fff74628249 _pthread_start + 66
5   libsystem_pthread.dylib       	0x00007fff7462440d thread_start + 13

Thread 6:
0   libsystem_kernel.dylib        	0x00007fff7456d61a __select + 10
1   mhttpd                        	0x00000001003719ae mg_mgr_poll + 846 (mongoose6.cxx:3651)
2   mhttpd                        	0x0000000100382087 per_connection_thread_function(void*) + 71 (mongoose6.cxx:3804)
3   libsystem_pthread.dylib       	0x00007fff746252eb _pthread_body + 126
4   libsystem_pthread.dylib       	0x00007fff74628249 _pthread_start + 66
5   libsystem_pthread.dylib       	0x00007fff7462440d thread_start + 13

Thread 4 crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000000  rbx: 0x000070000256e000  rcx: 0x000070000256c9f8  rdx: 0x0000000000000000
  rdi: 0x0000000000000d07  rsi: 0x0000000000000006  rbp: 0x000070000256ca30  rsp: 0x000070000256c9f8
   r8: 0x0000000000000000   r9: 0x000070000256c950  r10: 0x0000000000000000  r11: 0x0000000000000206
  r12: 0x0000000000000d07  r13: 0x0000000100ee1000  r14: 0x0000000000000006  r15: 0x000000000000002d
  rip: 0x00007fff7456c2c6  rfl: 0x0000000000000206  cr2: 0x000000072c967008
  
Logical CPU:     0
Error Code:      0x0200005d
Trap Number:     133


Binary Images:
       0x10030c000 -        0x1004afff7 +mhttpd (0) <7F574F87-D453-37BF-B8C0-97FD8E8A6C84> /Users/USER/*/mhttpd
       0x104f06000 -        0x104f7070f  dyld (655.1.1) <DFC3C4AF-6F97-3B34-B18D-7DCB23F2A83A> /usr/lib/dyld
    0x7fff6f14e000 -     0x7fff6f14ffff  com.apple.TrustEvaluationAgent (2.0 - 31.200.1) <15DF9C73-54E4-3C41-BCF4-378338C55FB4> /System/Library/PrivateFrameworks/TrustEvaluationAgent.framework/Versions/A/TrustEvaluationAgent
    0x7fff71418000 -     0x7fff71419ffb  libSystem.B.dylib (1252.250.1) <B1006948-7AD0-3CA9-81E0-833F4DD6BFB4> /usr/lib/libSystem.B.dylib
    0x7fff7165b000 -     0x7fff716aeff7  libc++.1.dylib (400.9.4) <9A60A190-6C34-339F-BB3D-AACE942009A4> /usr/lib/libc++.1.dylib
    0x7fff716af000 -     0x7fff716c4ff7  libc++abi.dylib (400.17) <38C09CED-9090-3719-90F3-04A2749F5428> /usr/lib/libc++abi.dylib
    0x7fff71b1a000 -     0x7fff71c12ff7  libcrypto.35.dylib (22.260.1) <91C3D71A-4D1D-331D-89CC-67863DF10574> /usr/lib/libcrypto.35.dylib
    0x7fff72c4f000 -     0x7fff733d4fdf  libobjc.A.dylib (756.2) <7C312627-43CB-3234-9324-4DEA92D59F50> /usr/lib/libobjc.A.dylib
    0x7fff737e0000 -     0x7fff73813ff3  libssl.35.dylib (22.260.1) <EEBCC1DE-A2C2-30CC-AEA3-1B2193B11DAA> /usr/lib/libssl.35.dylib
    0x7fff73abe000 -     0x7fff73ad0ff7  libz.1.dylib (70.200.4) <B048FC1F-058F-3A08-A1FE-81D5308CB3E6> /usr/lib/libz.1.dylib
    0x7fff742b4000 -     0x7fff742b8ff3  libcache.dylib (81) <1987D1E1-DB11-3291-B12A-EBD55848E02D> /usr/lib/system/libcache.dylib
    0x7fff742b9000 -     0x7fff742c3ff3  libcommonCrypto.dylib (60118.250.2) <1765BB6E-6784-3653-B16B-CB839721DC9A> /usr/lib/system/libcommonCrypto.dylib
    0x7fff742c4000 -     0x7fff742cbff7  libcompiler_rt.dylib (63.4) <5212BA7B-B7EA-37B4-AF6E-AC4F507EDFB8> /usr/lib/system/libcompiler_rt.dylib
    0x7fff742cc000 -     0x7fff742d5ff7  libcopyfile.dylib (146.250.1) <98CD00CD-9B91-3B5C-A9DB-842638050FA8> /usr/lib/system/libcopyfile.dylib
    0x7fff742d6000 -     0x7fff7435afc3  libcorecrypto.dylib (602.260.2) <01464D24-570C-3B83-9D18-467769E0FCDD> /usr/lib/system/libcorecrypto.dylib
    0x7fff743e1000 -     0x7fff7441aff7  libdispatch.dylib (1008.270.1) <97273678-E94C-3C8C-89F6-2E2020F4B43B> /usr/lib/system/libdispatch.dylib
    0x7fff7441b000 -     0x7fff74447ff7  libdyld.dylib (655.1.1) <002418CC-AD11-3D10-865B-015591D24E6C> /usr/lib/system/libdyld.dylib
    0x7fff74448000 -     0x7fff74448ffb  libkeymgr.dylib (30) <0D0F9CA2-8D5A-3273-8723-59987B5827F2> /usr/lib/system/libkeymgr.dylib
    0x7fff74456000 -     0x7fff74456ff7  liblaunch.dylib (1336.261.2) <2B07E27E-D404-3E98-9D28-BCA641E5C479> /usr/lib/system/liblaunch.dylib
    0x7fff74457000 -     0x7fff7445cfff  libmacho.dylib (927.0.3) <A377D608-77AB-3F6E-90F0-B4F251A5C12F> /usr/lib/system/libmacho.dylib
    0x7fff7445d000 -     0x7fff7445fffb  libquarantine.dylib (86.220.1) <6D0BC770-7348-3608-9254-F7FFBD347634> /usr/lib/system/libquarantine.dylib
    0x7fff74460000 -     0x7fff74461ff7  libremovefile.dylib (45.200.2) <9FBEB2FF-EEBE-31BC-BCFC-C71F8D0E99B6> /usr/lib/system/libremovefile.dylib
    0x7fff74462000 -     0x7fff74479ff3  libsystem_asl.dylib (356.200.4) <A62A7249-38B8-33FA-9875-F1852590796C> /usr/lib/system/libsystem_asl.dylib
    0x7fff7447a000 -     0x7fff7447aff7  libsystem_blocks.dylib (73) <A453E8EE-860D-3CED-B5DC-BE54E9DB4348> /usr/lib/system/libsystem_blocks.dylib
    0x7fff7447b000 -     0x7fff74502fff  libsystem_c.dylib (1272.250.1) <7EDACF78-2FA3-35B8-B051-D70475A35117> /usr/lib/system/libsystem_c.dylib
    0x7fff74503000 -     0x7fff74506ffb  libsystem_configuration.dylib (963.270.3) <2B4A836D-68A4-33E6-8D48-CD4486B03387> /usr/lib/system/libsystem_configuration.dylib
    0x7fff74507000 -     0x7fff7450aff7  libsystem_coreservices.dylib (66) <719F75A4-74C5-3BA6-A09E-0C5A3E5889D7> /usr/lib/system/libsystem_coreservices.dylib
    0x7fff7450b000 -     0x7fff74511fff  libsystem_darwin.dylib (1272.250.1) <EC9B39A5-9592-3577-8997-7DC721D20D8C> /usr/lib/system/libsystem_darwin.dylib
    0x7fff74512000 -     0x7fff74518ff7  libsystem_dnssd.dylib (878.270.2) <E9A5ACCF-E35F-3909-AF0A-2A37CD217276> /usr/lib/system/libsystem_dnssd.dylib
    0x7fff74519000 -     0x7fff74564ffb  libsystem_info.dylib (517.200.9) <D09D5AE0-2FDC-3A6D-93EC-729F931B1457> /usr/lib/system/libsystem_info.dylib
    0x7fff74565000 -     0x7fff7458dff7  libsystem_kernel.dylib (4903.271.2) <EA204E3C-870B-30DD-B4AF-D1BB66420D14> /usr/lib/system/libsystem_kernel.dylib
    0x7fff7458e000 -     0x7fff745d9ff7  libsystem_m.dylib (3158.200.7) <F19B6DB7-014F-3820-831F-389CCDA06EF6> /usr/lib/system/libsystem_m.dylib
    0x7fff745da000 -     0x7fff74604fff  libsystem_malloc.dylib (166.270.1) <011F3AD0-8E6A-3A89-AE64-6E5F6840F30A> /usr/lib/system/libsystem_malloc.dylib
    0x7fff74605000 -     0x7fff7460fff7  libsystem_networkextension.dylib (767.250.2) <FF06F13A-AEFE-3A27-A073-910EF78AEA36> /usr/lib/system/libsystem_networkextension.dylib
    0x7fff74610000 -     0x7fff74617fff  libsystem_notify.dylib (172.200.21) <145B5CFC-CF73-33CE-BD3D-E8DDE268FFDE> /usr/lib/system/libsystem_notify.dylib
    0x7fff74618000 -     0x7fff74621fef  libsystem_platform.dylib (177.270.1) <9D1FE5E4-EB7D-3B3F-A8D1-A96D9CF1348C> /usr/lib/system/libsystem_platform.dylib
    0x7fff74622000 -     0x7fff7462cff7  libsystem_pthread.dylib (330.250.2) <2D5C08FF-484F-3D59-9132-CE1DCB3F76D7> /usr/lib/system/libsystem_pthread.dylib
    0x7fff7462d000 -     0x7fff74630ff7  libsystem_sandbox.dylib (851.270.1) <9494594B-5199-3186-82AB-5FF8BED6EE16> /usr/lib/system/libsystem_sandbox.dylib
    0x7fff74631000 -     0x7fff74633ff3  libsystem_secinit.dylib (30.260.2) <EF1EA47B-7B22-35E8-BD9B-F7003DCB96AE> /usr/lib/system/libsystem_secinit.dylib
    0x7fff74634000 -     0x7fff7463bff3  libsystem_symptoms.dylib (820.267.1) <03F1C2DD-0F5A-3D9D-88F6-B26C0F94EB52> /usr/lib/system/libsystem_symptoms.dylib
    0x7fff7463c000 -     0x7fff74651fff  libsystem_trace.dylib (906.260.1) <FC761C3B-5434-3A52-912D-F1B15FAA8EB2> /usr/lib/system/libsystem_trace.dylib
    0x7fff74653000 -     0x7fff74658ffb  libunwind.dylib (35.4) <24A97A67-F017-3CFC-B0D0-6BD0224B1336> /usr/lib/system/libunwind.dylib
    0x7fff74659000 -     0x7fff74688fff  libxpc.dylib (1336.261.2) <7DEE2300-6D8E-3C00-9C63-E3E80D56B0C4> /usr/lib/system/libxpc.dylib

External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 1
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 402898
    thread_create: 0
    thread_set_state: 9

VM Region Summary:
ReadOnly portion of Libraries: Total=238.2M resident=0K(0%) swapped_out_or_unallocated=238.2M(100%)
Writable regions: Total=174.0M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=174.0M(100%)
 
                                VIRTUAL   REGION 
REGION TYPE                        SIZE    COUNT (non-coalesced) 
===========                     =======  ======= 
Kernel Alloc Once                    8K        1 
MALLOC                           136.6M       15 
Stack                             67.1M       14 
VM_ALLOCATE                         12K        2 
__DATA                            3488K       44 
__LINKEDIT                       223.7M        3 
__TEXT                            14.5M       47 
shared memory                     1288K        5 
===========                     =======  ======= 
TOTAL                            446.5M      131
  1733   15 Nov 2019 Andreas SuterSuggestionjavascript comunication
I am currently testing the new history system on the mhttpd side and stumbled over the following issue: typically our user open a lot of midas web-page tabs and keep them open. With the current version this leads after a night typically to a state where the browser is busy with itself and not reacting anymore.

One important reason seems to be that ALL tabs trying to communicate all the time which is totally unnecessary, since I think a hidden tab should stay in a sleeping mode. 

I was browsing if there is a way to find out if a tab is active or not, and found the following API which exactly does this:

https://developer.mozilla.org/en-US/docs/Web/API/Page_Visibility_API

Furthermore, the simple

document.hidden 

tag, could be used to find out if the page is currently active.

Wouldn't it a good idea to send all midas tabs which are not active into a sleep mode and only reactivate them if they come into focus?

I had a quick look at the JavaScript libs of midas, but I am not quite certain where to best inject this.
  1732   12 Nov 2019 Konstantin OlchanskiBug ReportNewly installed MIDAS on OSX: mhttpd crahes
> Context: out of the box  MIDAS (using cmake) on OSX Mojave. 
> 
> Running with mongoose/opensslm installation following instruction here:
> https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux
> 
> mhttpd crashing when midas webpage opened with Safari (12.1.2). Usually when opening the "chat" tab but sometimes also with the "message" tab.
> mhttpd(11109,0x70000827a000) malloc: *** error for object 0x7f8669501ef0: pointer being freed was not allocated
> mhttpd(11109,0x70000827a000) malloc: *** set a breakpoint in malloc_error_break to debug
> 
> No crash if using firefox (70.0.1 (64-bit))

I think we also have reports of mhttpd crash on macos with safari from the Dragon experiment,
but cannot reproduce the problem.

If you can reproduce this, can you capture the crash stack trace?

One way to do this is to enable core dumps in odb "/expt/enable core dumps" set to "y", restart mhttpd,
wait for the crash. I think macos writes core dumps into /cores/... Or you can run mhttpd inside lldb
and wait for the crash. the lldb command to show the stack trace is "bt", but you may need
to switch to different threads to see which one actually crashed. I forget what the command
for that is.

BTW, the mhttpd networking code has not changed in a long time, but an update
of mongoose web server library is overdue (to fix a memory leak, at least).

K.O.
  1731   08 Nov 2019 Pierre GorelBug ReportNewly installed MIDAS on OSX: mhttpd crahes
Context: out of the box  MIDAS (using cmake) on OSX Mojave. 

Running with mongoose/opensslm installation following instruction here:
https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux

mhttpd crashing when midas webpage opened with Safari (12.1.2). Usually when opening the "chat" tab but sometimes also with the "message" tab.
mhttpd(11109,0x70000827a000) malloc: *** error for object 0x7f8669501ef0: pointer being freed was not allocated
mhttpd(11109,0x70000827a000) malloc: *** set a breakpoint in malloc_error_break to debug

No crash if using firefox (70.0.1 (64-bit))
  1730   24 Oct 2019 Konstantin OlchanskiBug Reportlazylogger in cmake & max_event_size
> > > The compile option -DHAVE_FTPLIB checked in mdsupport.cxx disappeared if you 
> > > compile with cmake.
> > 
> > Hi, Stefan - do we still need to support FTP in the logger? In the lazylogger, special support for 
> > FTP is not needed, they can you the "script" method and do FTP without our help.
> > 
> > I move to remove FTP support from MIDAS. (second? other opinions?)
> 
> I oppose to remove FTP support from lazylogger.

Confirmed. FTP support in lazylogger stays.

K.O.
  1729   23 Oct 2019 Konstantin OlchanskiForumData for key truncated
> I keep on getting messages like this:
> 16:25:35 [fecaen,ERROR] [odb.c:4567:db_get_data,ERROR] data for key
> "/DAQ/params/VX1730/custom/Board 0/Channel 0/Input range" truncated
>
>  [  bool fInputRange... ]
>          size = sizeof(fInputRange);
>          db_get_data(hDb, hSubKey, &fInputRange, &size, TID_BOOL);
>

The error is correct. size of TID_BOOL is 4 byte (uint32_t) and you give is sizeof(bool) instead which is probably not 4.

Note that sizeof(bool) is not well defined, sometimes it is 1 (you need 4), sometimes something else, see
https://stackoverflow.com/questions/4897844/is-sizeofbool-defined-in-the-c-language-standard

A good fix would be to change fInputRange from bool to uint32_t (which is always 4 byte size).

#include <stdint.h>
...
uint32_t fInputRange;

K.O.
  1728   21 Oct 2019 Vinzenz BildsteinForumData for key truncated
I keep on getting messages like this:

16:25:35 [fecaen,ERROR] [odb.c:4567:db_get_data,ERROR] data for key
"/DAQ/params/VX1730/custom/Board 0/Channel 0/Input range" truncated

whenever I start my frontend. Input range is defined to be a BOOL and using
odbedit to read it shows:

Key name                        Type    #Val  Size  Last Opn Mode Value
---------------------------------------------------------------------------
Input range                     BOOL    1     4     75h  0   RWD  y

without any error message. The entry is read using

         size = sizeof(fInputRange);
         db_get_data(hDb, hSubKey, &fInputRange, &size, TID_BOOL);

where fInputRange is a bool.

Where does this message come from and how can I resolve this?
  1727   18 Oct 2019 Joseph McKennaInfosysmon: New system monitor and performance logging frontend added to MIDAS

I have written a system monitor tool for MIDAS, that has been merged in the develop branch today: sysmon

https://bitbucket.org/tmidas/midas/pull-requests/8/system-monitoring-a-new-frontend-to-log/diff

To use it, simply run the new program
sysmon
on any host that you want to monitor, no configuring required.




The program is a frontend for MIDAS, there is no need for configuration, as upon initialisation it builds a history display for you. Simply run one instance per machine you want to monitor. By default, it only logs once per 10 seconds.

The equipment name is derived from the hostname, so multiple instances can be run across multiple machines without conflict. A new history display will be created for each host.

sysmon uses the /proc pseudo-filesystem, so unfortunately only linux is supported. It does however work with multiple architectures, so x86 and ARM processors are supported.

If the build machine has NVIDIA drivers installed, there is an additional version of sysmon that gets built: sysmon-nvidia. This will log the GPU temperature and usage, as well as CPU, memory and swap. A host should only run either sysmon or sysmon-nvidia

elog:1727/1 shows the History Display generated by sysmon-nvidia. sysmon would only generate the first two displays (sysmon/localhost and sysmon/localhost-CPU)
Attachment 1: sysmon-gpu.png
sysmon-gpu.png
  1726   15 Oct 2019 Stefan RittSuggestionrecover daq and hardware safety.
There is a not-so-well-known function in the ODB to write protect some keys. You can do

odbedit> chmod 1 /Equipment/HV/Demand

which will write protect your Demand values. You see that by doing

odbedit> ls -ls

where you only see a "R" at the right end instead a "RWD". I haven't tried it yet (so better do a dry run yourself), but that should prevent an odb load to overwrite your demand values. To change the values, put some logic on a custom page to unprotect the 
values, change them, and then protect them again.

Stefan
  Draft   14 Oct 2019 Joseph McKennaForumtmfe.cxx - Future frontend design
Hi,

I have been looking at the 2019 workshop slides, I am interested in the C++ future of MIDAS. 

I am quite interested in using the object oriented 


ALPHA will start data taking in 2021
  1724   14 Oct 2019 Stefan RittBug Reportlazylogger in cmake & max_event_size
> > The compile option -DHAVE_FTPLIB checked in mdsupport.cxx disappeared if you 
> > compile with cmake.
> 
> Hi, Stefan - do we still need to support FTP in the logger? In the lazylogger, special support for 
> FTP is not needed, they can you the "script" method and do FTP without our help.
> 
> I move to remove FTP support from MIDAS. (second? other opinions?)

I oppose to remove FTP support from lazylogger. We still use it heavily at PSI. In comparison to the "script" method, it 
shows the current speed in MB/s which helps us to diagnose some network problem by writing this number into the 
history. The "script" method only give you an integral transfer speed after a file has be completely written.

I'm however not sure who FTP is used in lazylogger. It goes into mdsupport.cxx and I seem to remember that Pierre 
wrote the FTP code by hand, so no external library is necessary.

Stefan
Attachment 1: Screenshot_2019-10-14_at_13.54.53_.png
Screenshot_2019-10-14_at_13.54.53_.png
  1723   10 Oct 2019 Stefan RittBug ReportHistory data size mismatch
> Yes, we could have
> kept that apart, yes, in this case a double would also work (and not break things), but a bug is a bug...
> I could think of senisble use cases where doubles and ints are mixed and I also know quite a few areas where it makes
> sense to use floats...

I agree with Nik that we should fix this on the midas level. Since it happens in history_schema.cxx which was written by KO, maybe he can have a look.

Stefan
Goto page Previous  1, 2, 3 ... 55, 56, 57 ... 140, 141, 142   Next  
ELOG V3.1.4-2e1708b5