ID |
Date |
Author |
Topic |
Subject |
1706
|
27 Sep 2019 |
Konstantin Olchanski | Release | midas-2019-09 | I created the release branch for midas-2019-09 and tag midas-2019-09-a.
Since the previous release midas-2019-06, some news:
- new history graphics (Stefan)
- c++ frontend framework mvodb.h and tmfe.h merged from ALPHA-g (K.O.)
- we think we have all the fallout from switching to cmake and to c++11 sorted out
There is a number of known problems with the current code, see the bitbucket bug tracker:
https://bitbucket.org/tmidas/midas/issues?status=new&status=open
Hopefully we can use this release as a baseline for more testing and with luck we will
fix all the pending bugs and add all the pending missing code (the new sequencer web pages,
the "m" analyzer, etc) quickly and our next release midas-2019-10 will be the best midas ever.
To obtain this release, either checkout the top of branch feature/midas-2019-09 (recommended)
or checkout the tag midas-2019-09-a.
If you are using the last pre-cmake/c++ release midas-2019-03, I recommend that you stay with it
until our next release midas-2019-10.
K.O. |
1707
|
27 Sep 2019 |
Konstantin Olchanski | Forum | Open a hotlink to a single element in an ODB array | > I will try to use the db_watch function in the future.
Note that db_watch() and db_open_record() work exactly the same way, both only allow
watching "whole" odb entries, you cannot watch individual array elements.
The db_watch() callback function gives you the array index of the array element that was
changed and that fired the notification.
*but*
If you change many array elements quickly you will not necessary receive notifications for
all and each of of them (underlying transport is UDP allows notification packet loss).
If you are watching 1 array element change at a slow rate (1/sec), db_watch() will work well.
Otherwise, you can watch the whole array, in the db_watch() callback, read the new array
contents, compare it with your saved copy of pervious array contents, identify which array
elements have changed and dance from here. (this method does not work if you do not
actually change the array element values: change from "1" to "1", this is an old weakness in
the midas hot link mechanism).
If you are not sure how to use db_watch(), look inside midas/progs/odbedit.cxx search for
db_watch() and search for the db_watch() callback function.
K.O. |
1708
|
27 Sep 2019 |
Konstantin Olchanski | Suggestion | recover daq and hardware safety. | > We have encountered a safety issue with our HPGe HV and it's midas frontend.
At TRIUMF and other labs the words "safety issue" have very specific meaning and
we tend to follow this guidance: MIDAS is not certified for and is not intended for use with
safety critical applications as defined here:
https://en.wikipedia.org/wiki/Safety-critical_system
> A safety-critical system ... malfunction may result in ... following outcomes:
> death or serious injury to people
> loss or severe damage to equipment/property
> environmental harm
If this is your case, you should use properly certified software *and hardware*. Safety
officers at most institutions require certified hardware interlocks and other protections to
prevent such undesirable outcomes. Use of certified PLCs is sometimes permitted.
But I suspect in your case, there is no "safety issue", you only want to protect some
valuable but not critical equipment against accidental damage.
In this case, you can probably use midas, but if midas malfunction may result in destroying
your experiment (i.e. accidentally set wrong voltage on 3000 phototubes), you should also
have hardware based protections (hardware limits on max/min high voltage). Most HV
power supplies implement such protections (screw-driver actuated max voltage limits).
If there is danger of destroying your experiment you should also have an independent
review of your control system to avoid avoidable mistakes and obvious problems.
> Turning off or changing HV unknowingly has to be avoided at all costs
The function of changing high-voltage is implemented in your frontend program. Right in
the place in this program where you transmit the voltage setting from ODB to the hardware
is where you implement your protections (validate the voltage range, check that changing
the voltage is permitted, etc). This protects you against unexpected/incorrect/erroneous
changes in ODB (wrong ODB is loaded, wrong values in ODB, ODB is corrupted, etc).
In addition, it is wise to set software based limits in the HV power supply (software
controlled max high voltage, software controlled max current, etc). Most HV power supplies
implement such functions.
To ensure high voltage cannot be changed at the wrong times, you can also implement
procedural and hardware protections, such as unplug the power supply control connection
(usually ethernet or serial or usb cable). This will prevent you from monitoring the high
voltage currents and the only solution is to use a power supply with a hardware "write
protect" function (a key needs to be inserted and turned to allow changing anything).
All of this is generic and applies to any controls software, not just MIDAS.
Without at least some of these protections (especially protections in your frontend
program), the questions you asked about loading ODB are insufficient.
K.O. |
1709
|
27 Sep 2019 |
Konstantin Olchanski | Bug Report | lazylogger in cmake & max_event_size | > The compile option -DHAVE_FTPLIB checked in mdsupport.cxx disappeared if you
> compile with cmake.
Hi, Stefan - do we still need to support FTP in the logger? In the lazylogger, special support for
FTP is not needed, they can you the "script" method and do FTP without our help.
I move to remove FTP support from MIDAS. (second? other opinions?)
> Our MAX_EVENT_SIZE is set in the odb to 805306368. This number is also used in
> this is to big when copying with ftp, causing a crash. Reducing it here with a
> factor 10 solves our problems.
I am surprised that changing MAX_EVENT_SIZE (to a "too big" value) causes lazylogger to
crash. More usually MAX_EVENT_SIZE has no effect until you try to write an event that is
somehow "too big", then there is a crash. Perhaps there is a bug specifically in the FTP code.
Anyhow, I recommend the solution of using the "script" method. We have example lazylogger
scripts in midas/progs/lazy*.perl (the scripts do not have to be in perl, python is ok). We do
not have any example that uses FTP because we do not use FTP for data storage. But you can
easily adapt lazy_test.perl and lazy_copy.perl to use scp and sftp, the secure versions of FTP.
K.O. |
1710
|
27 Sep 2019 |
Konstantin Olchanski | Forum | History plot problems for frontend with multiple indicies | We should fix this for midas-2019-10.
https://bitbucket.org/tmidas/midas/issues/193/confusion-in-history-event-ids
K.O.
> Hi Konstantin,
>
> > > [local:e666:S]History>ls -l /History/Events
> > > Key name Type #Val Size Last Opn Mode Value
> > > ---------------------------------------------------------------------------
> > > 1 STRING 1 10 2m 0 RWD FeDummy02
> > > 0 STRING 1 16 2m 0 RWD Run transitions
> >
> > Something is very broken. There should be more entries here, at least
> > there should be entries for "FeDummy01" and usually there is also an entry
> > for "FeDummy" because one invariably runs fedummy without "-i" at least once.
>
> This is a fresh experiment that I started just to test this this issue, that is why there are not many
> entries in /History/Events. I agree though that we should expect to see a FeDummy01 entry.
>
> > The fact that changing from "midas" storage to "file" storage makes no difference
> > also indicates that something is very broken.
> >
> > I want to debug this.
> >
> > Since you tried the "file" storage, can you send me the output of "ls -l mhf*.dat" in the directory
> > with the history files? (it should have the "*.hst" files from the "midas" storage and "mhf*.dat"
> files
> > from the "file" storage.
>
> When I started this experiment yesterday(?) I disabled the Midas history when I enbled the file
> history. Jsut now I reenabled the Midas history, so they are currently both active.
>
> % ls -l work/online/{*.hst,mhf*.dat}
> -rw-r--r-- 1 hastings hastings 14996 Sep 17 10:21 work/online/190917.hst
> -rw-r--r-- 1 hastings hastings 3292 Sep 18 16:29 work/online/190918.hst
> -rw-r--r-- 1 hastings hastings 867288 Sep 18 16:29 work/online/mhf_1568683062_20190917_fedummy01.dat
> -rw-r--r-- 1 hastings hastings 867288 Sep 18 16:29 work/online/mhf_1568683062_20190917_fedummy02.dat
> -rw-r--r-- 1 hastings hastings 166 Sep 17 10:17
> work/online/mhf_1568683062_20190917_run_transitions.dat
>
> And again, just as a sanity check:
>
> % odbedit -c 'ls -l /History/Events'
> Key name Type #Val Size Last Opn Mode Value
> ---------------------------------------------------------------------------
> 1 STRING 1 10 1m 0 RWD FeDummy02
> 0 STRING 1 16 1m 0 RWD Run transitions
>
> Regards,
>
> Nick. |
1711
|
27 Sep 2019 |
Konstantin Olchanski | Forum | mhttpd start and stop redirect to Transition page | > I recently upgraded to MIDAS version midas-2019-06-b. I had to make a few changes
> to get our custom page running again, but am a little confused on starting and
> stopping runs.
So far so good.
> When I click on my "Start" button, it now redirects to a
> Transition page rather than reloading the status page.
Are you sure? The "start" button redirects to the "start" page (start.html) which redirects
to the "transition" page (transition.html), which does not redirect anywhere so you can see
the result of the transition.
> Could someone explain the reasoning for the current behavior?
It's been like this for years now. Stefan suggest that we implement the "start" page
and the "transition" page as overlays on top of the status page, but it did not happen yet.
> Furthermore my "Stop" button is now broken with the following error:
> Error: Invalid URL "CS/EngeRun&" or query "cmd=Stop&redir=EngeRun%26" or command "Stop"
I grep for "EngeRun" and I do not see it anywhere in the midas sources. Can you grep for it
to see if it is coming from one of your pages?
If you want to start/stop runs from your custom page, look at start.html and transition.html - you will
need to make the run transition RPC calls (cut-and-paste the code to your page) and (obviously)
you will not have any redirects to some strange pages.
> For example, start calls:
> location.search = "cmd=Transition";
> whereas stop does:
> mhttpd_goto_page("Transition"); // DOES NOT RETURN
It's the same thing, look at mhttpd_goto_page().
> Can anyone offer any insights or advice? I can change the former to "cmd=Status", but
> the latter doesn't allow it.
I am not sure what you are trying to do. If you need the "start" button on the status page
to do something different from what it does now, just hack status.html until it does so.
If you need some specific help with that, I am happy to help. I think I answered all questions
you asked so far.
K.O. |
1715
|
29 Sep 2019 |
Konstantin Olchanski | Suggestion | recover daq and hardware safety. | >
> The issue occurs when e.g. one channel can not be turned on and ramp for some temp/specific
> reason, and someone else is working on the daq and reloads the odb for e.g. 1h ago.
>
So you want to ensure that some HV channels are turned off and stay turned off. Yes?
Most effective solution will depend on the consequences of unwanted turning-on of your channels:
- if hardware is destroyed if turned on - I think you should have a hardware lock-out. (unplug the HV cable)
- if hardware malfunctions and will degrade if left turned on for long time (i.e. a hot phototube or sparking wire chamber) - your data
monitoring software should detect the anomaly (it will show up as a hot channel, dead channel, etc) and the people running the
experiment will realize the mistake and turn the channel back off. also hardware monitoring (HV currents, etc) should detect this, with
same effect.
- if collected data becomes useless (the turned-off channel make big noise in all other channels), then same thing, your data
monitoring should catch it.
The next consideration is what are you protecting against:
a) one person flags channel defective, turns it off, next person knows nothing, turns it back on - you need to work on documentation,
shift hand-off and other human-level procedures
b) people running experiment load random odb files - same thing, from human-level procedures and documentation it should be made
clear which odb files are correct and which should not be used
c) software malfunction (not human person) causes data change in odb causes turned-off channel to turn back on
d) hardware malfunction causes turned-off channel to turn back on (HV power supply hardware or firmware malfunctions and decides
that all channels should be turned on at maximum high voltage)
In the experiments I am most familiar with, problem (b) is avoided by never loading/reloading odb files directly, most/all interaction
with the experiment is done through web pages, and these web pages are carefully coded to be safe against most user mistakes.
Cases (a), (b) and (c) you can protect against by changing the frontend code to refuse to turn on some channels:
int set_hv(int channel, int voltage) {
if (channel == 35) return COMMAND_REFUSED;
write_to_hardware(channel, voltage);
return COMMAND_SUCCESS;
}
But in reality this solution only creates problem (e):
e) people running the experiment start random versions of the frontend program, make random changes to the frontend source code,
multiple people working on the frontend have their own personal versions/copies of the source code, etc.
This is the worst-case scenario, meaning the experiment lost control of software configuration, and even basic software version
control tools (like svn or git) are not being used. If your experiment gets that chaotic, all protections are likely to be ineffective -
documentation will not work (people will ignore post-it notes "do not turn on!"), hardware protections will not work (unplugged cable
labeled "do not plug in!" will be plugged back in and powered), etc. good luck, then.
K.O. |
1716
|
29 Sep 2019 |
Konstantin Olchanski | Forum | MIDAS interface for WAGASCI online monitor | > online monitor would show (almost in real-time) the
> gain, the dark noise, and the pedestal for all the channels, the 2D tracks
> inside the detectors for each spill and so on.
Hmm... I now realize that the midas distribution does not include an example web page that
integrates all these elements into one easy to understand html file.
I think an example page that would answer your questions and the questions from the other
thread about starting/stopping runs, should include following elements:
- the general midas web page framework (the midas left-hand side menu, the top side
status display, Stefan's new odb tags)
- buttons to start and stop runs (javascript code to call the run transition RPCs)
- embedded images for history display (old style gif and new style canvas)
- embedded images for ROOT histograms (via the ROOT http server and jsroot)
- code to live-update all these elements independantly from each other (to allow history
plots and ROOT histograms to update at different frequencies).
As for the web page code for showing a mini-event-display, I think we do not know yet how
to do - the event data lives inside the analyzer as C++ data structures, so somehow it
needs to be encoded as json (this code is missing - but one can use the ROOT C++ to json
encoder/streamer), needs to be transported to the web browser (we know how to do this)
and at the end, plotting the json data on a canvas is the easy part.
I know some experiments have done all of this, and I think we should have such a pipeline
available as part of the ROOTANA package. Maybe some day...
K.O. |
1718
|
30 Sep 2019 |
Konstantin Olchanski | Forum | MIDAS interface for WAGASCI online monitor | >
> As Thomas said, maybe the simplest thing would be to use the ROOT THttpServer. Honestly, I do
> not think that ROOT was ever meant to act as an online monitor due to its wacky memory
> management and abysmal multithread support. In other words, I think that by using ROOT we would
> inevitably lose some performance.
>
Yes. The previous data analysis frameworks - PAW/PAW++/CERNlib (CERN), NOVA (TRIUMF) - certainly had
support for online monitoring. In CERNlib/PAW the histograms were stored in shared memory,
the analyzer running in the background was filling them, the PAW/PAW++ display was displaying
them "live". I was very surprised to find this function removed/not implemented in ROOT, given
that the same people were behind both projects (Rene Brun & co).
We tried to roll our own implementation of this in ROOTANA/ROODY, with mixed success.
I am glad the JSROOT project finally gained traction and web based "I can see the data" is now
available in ROOT.
>
> Perhaps there are better ways of achieving the same goal. For example, I was leaning towards a
> plotly.js based approach where
>
There are many web/javascript graphics libraries out there, all have the weak spot - how do you
get your data into them?
Going forward, I see us standardizing on JSROOT: https://root.cern.ch/js/
>
> ... send them to the client through the MJSONRPC mechanism ...
>
I am not sure JSROOT have any support for interacting from the web page to the back-end analyzer. Perhaps
we can use the MIDAS MJSONRPC library for this. Hmm... (Note that the ROOT HTTP server is a derivative
of the mongoose web server library, which we use in mhttpd, so I already know how to work it)
>
> I would encode a series of vectors in base64 strings (for better transmission performance)
>
We looked into this when deciding on the data encoding for the midas history data. There is a tradeoff
between network use and cpu use - to save on the network, you try to reduce the data size by using
compressed binary data - to save on the CPU you try to minimize data encoding.
For history data, we gave up on binary json (extra decoding needed), gave up on text json (extra decoding
needed), gave up on compression (extra cpu use for decompression) and use javascript native binary processing
("arraybuffer").
Our thinking is that network bandwidth is usually quite big and is getting bigger, but cpu resource is limited
and is expensive. (mobile devices seems to be stuck with ~2 GHz CPUs; cpu use means battery use and
battery capacity is limited, not improving quickly)
>
> So ROOT it is. I will use the ROOTANA javascript display as a reference. Do you happen to know
> who wrote that part?
>
Yes. See "Contact" at https://root.cern.ch/js/
>
> In that example, you have some "static" histograms that you keep always in memory, while in our
> case the number of channels is so big that we have to dynamically generate the histograms only
> when needed (when the user select a single channel).
>
This requires interaction with the analyzer, requires some kind of RPC mechanism. I am now curious what jsroot
have, also it would not be too hard to add the mjsonrpc library to rootana. Cooperation from ROOT multithreading
is not required: I can queue the RPC requests in a separate (thread safe, non-ROOT) buffer, then process
them in the ROOT main event loop (this is how the ROOTANA histogram server worked in the days when
ROOT had no multithread support at all).
K.O. |
1721
|
10 Oct 2019 |
Konstantin Olchanski | Bug Report | History data size mismatch | >
> In our history, a long list of doubles (64 Bit) fas followed by three floats (32 bit)
>
Padding trouble, mixing "double" and "float" trouble. Ouch.
Best wisdom I received on this: never use "float", always use "double".
I was burned by "float" with following code, which produced the same result from
analyzing 100 files as from analyzing 1000 files. (why did we take data for 10 weeks
instead of 1 week?). Hint: "float" overflows way too quickly, after overflow sum+=1 does not change
the value of "sum". The actual code used ROOT TH1F. Lesson: always use TH1D.
float sum = 0; // should always be "double" !!!
foreach data_file {
foreach data from current data file {
sum += data;
}
}
print sum;
K.O. |
1729
|
23 Oct 2019 |
Konstantin Olchanski | Forum | Data for key truncated | > I keep on getting messages like this:
> 16:25:35 [fecaen,ERROR] [odb.c:4567:db_get_data,ERROR] data for key
> "/DAQ/params/VX1730/custom/Board 0/Channel 0/Input range" truncated
>
> [ bool fInputRange... ]
> size = sizeof(fInputRange);
> db_get_data(hDb, hSubKey, &fInputRange, &size, TID_BOOL);
>
The error is correct. size of TID_BOOL is 4 byte (uint32_t) and you give is sizeof(bool) instead which is probably not 4.
Note that sizeof(bool) is not well defined, sometimes it is 1 (you need 4), sometimes something else, see
https://stackoverflow.com/questions/4897844/is-sizeofbool-defined-in-the-c-language-standard
A good fix would be to change fInputRange from bool to uint32_t (which is always 4 byte size).
#include <stdint.h>
...
uint32_t fInputRange;
K.O. |
1730
|
24 Oct 2019 |
Konstantin Olchanski | Bug Report | lazylogger in cmake & max_event_size | > > > The compile option -DHAVE_FTPLIB checked in mdsupport.cxx disappeared if you
> > > compile with cmake.
> >
> > Hi, Stefan - do we still need to support FTP in the logger? In the lazylogger, special support for
> > FTP is not needed, they can you the "script" method and do FTP without our help.
> >
> > I move to remove FTP support from MIDAS. (second? other opinions?)
>
> I oppose to remove FTP support from lazylogger.
Confirmed. FTP support in lazylogger stays.
K.O. |
1732
|
12 Nov 2019 |
Konstantin Olchanski | Bug Report | Newly installed MIDAS on OSX: mhttpd crahes | > Context: out of the box MIDAS (using cmake) on OSX Mojave.
>
> Running with mongoose/opensslm installation following instruction here:
> https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux
>
> mhttpd crashing when midas webpage opened with Safari (12.1.2). Usually when opening the "chat" tab but sometimes also with the "message" tab.
> mhttpd(11109,0x70000827a000) malloc: *** error for object 0x7f8669501ef0: pointer being freed was not allocated
> mhttpd(11109,0x70000827a000) malloc: *** set a breakpoint in malloc_error_break to debug
>
> No crash if using firefox (70.0.1 (64-bit))
I think we also have reports of mhttpd crash on macos with safari from the Dragon experiment,
but cannot reproduce the problem.
If you can reproduce this, can you capture the crash stack trace?
One way to do this is to enable core dumps in odb "/expt/enable core dumps" set to "y", restart mhttpd,
wait for the crash. I think macos writes core dumps into /cores/... Or you can run mhttpd inside lldb
and wait for the crash. the lldb command to show the stack trace is "bt", but you may need
to switch to different threads to see which one actually crashed. I forget what the command
for that is.
BTW, the mhttpd networking code has not changed in a long time, but an update
of mongoose web server library is overdue (to fix a memory leak, at least).
K.O. |
1736
|
15 Nov 2019 |
Konstantin Olchanski | Bug Report | Newly installed MIDAS on OSX: mhttpd crahes | > It is reproducible alright.
Thanks. At first blush, a guess, read_passwords() is not thread-safe and is called from multiple threads, not protected by semaphore. Crash report shows 2 active threads
(one made it is far as processing the mjson rpc, the other one crashed in read_passwords()).
K.O.
> Here are the core dump and the backtrace (I think the former is more informative).
>
>
>
> > > Context: out of the box MIDAS (using cmake) on OSX Mojave.
> > >
> > > Running with mongoose/opensslm installation following instruction here:
> > > https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux
> > >
> > > mhttpd crashing when midas webpage opened with Safari (12.1.2). Usually when opening the "chat" tab but sometimes also with the "message" tab.
> > > mhttpd(11109,0x70000827a000) malloc: *** error for object 0x7f8669501ef0: pointer being freed was not allocated
> > > mhttpd(11109,0x70000827a000) malloc: *** set a breakpoint in malloc_error_break to debug
> > >
> > > No crash if using firefox (70.0.1 (64-bit))
> >
> > I think we also have reports of mhttpd crash on macos with safari from the Dragon experiment,
> > but cannot reproduce the problem.
> >
> > If you can reproduce this, can you capture the crash stack trace?
> >
> > One way to do this is to enable core dumps in odb "/expt/enable core dumps" set to "y", restart mhttpd,
> > wait for the crash. I think macos writes core dumps into /cores/... Or you can run mhttpd inside lldb
> > and wait for the crash. the lldb command to show the stack trace is "bt", but you may need
> > to switch to different threads to see which one actually crashed. I forget what the command
> > for that is.
> >
> > BTW, the mhttpd networking code has not changed in a long time, but an update
> > of mongoose web server library is overdue (to fix a memory leak, at least).
> >
> > K.O. |
1737
|
17 Nov 2019 |
Konstantin Olchanski | Suggestion | javascript comunication | > I am currently testing the new history system on the mhttpd side and stumbled over the following issue:
> typically our user open a lot of midas web-page tabs and keep them open. With the current version this leads after a night typically to a state where the browser is busy with itself and not reacting anymore.
>
> One important reason seems to be that ALL tabs trying to communicate all the time which is totally unnecessary, since I think a hidden tab should stay in a sleeping mode.
I am looking at two more problems with inactive tabs:
a) google chrome slows down the execution of javascript in inactive tabs, leading to trouble
with memory management - midas pages poll at 1/sec, each poll allocates memory for processing RPC messages,
and (until recently) allocates memory for new DOM objects to update the web page - but the garbage collector
gets slowed down and does not keep up - leading to huge memory use (up to 200 Mbytes) for inactive midas pages
that normally consume 50-ish Mbytes.
b) the playing of the alarm sound is throttled by "user has not interacted with the document" thing, but there is a bug -
instead of canceling the playing of the alarm sound, the sound file is still loaded, (but not played). (this is hard to debug
because I do not know how to manually trigger the "user has not interacted..." condition, I have to wait for many days for it.
Then, for inactive tabs, the loading of the sound files is slowed down, leading to many of them getting queued up,
and eventually they all try to load and play at the same time, again leading to huge memory and cpu use in inactive tabs.
(this sounds incredible because we play the alarm sound at most 1/minute, for sure the previous sound file must have
finished playing by then, but no, it is easy to see it happen - add a few console.log messages and wait for a few days).
>
> I was browsing if there is a way to find out if a tab is active or not, and found the following API which exactly does this:
> https://developer.mozilla.org/en-US/docs/Web/API/Page_Visibility_API
>
From looking at the inactive tab business, I see that javascript in inactive tabs runs quite differently from javascript
in active tabs (i.e. timers do not work the same) and I see how the "visibility api" had to be invented to counter that.
>
> Wouldn't it a good idea to send all midas tabs which are not active into a sleep mode and only reactivate them if they come into focus?
> I had a quick look at the JavaScript libs of midas, but I am not quite certain where to best inject this.
>
most midas web pages poll in two places - mhttpd_refresh() updates the current date timestamp, alarms, currently active midas.log message;
and each page has it's own loop for updating it's own data (i.e. "alarms" page, "programs" page).
we should be careful to not completely disable all polling as some experiments do use and do rely on the midas producing
loud alarm messages ("program logged is not running!!!", "program mhttpd aborted!!!"). Even if all midas tabs are inactive,
some javascript is some tab still has to run frequently enough to poll midas and to sound the alarm sounds (even though
I am not sure how to 100% reliably counteract the google-chrome not playing sound files because
of the "user did not interact with the site..." thing).
K.O. |
1738
|
17 Nov 2019 |
Konstantin Olchanski | Suggestion | javascript comunication | > Very good idea. And thanks for finding the document.hidden solution. I put it in, so give it a try.
Hi, Stefan - I did not look at your code, if all midas tabs are inactive, will the alarm sound still play?
K.O.
>
> Best,
> Stefan
>
>
> > I am currently testing the new history system on the mhttpd side and stumbled over the following issue: typically our user open a lot of midas web-page tabs and keep them open. With the current version this leads after a night typically to a state where the browser is busy with itself and not reacting anymore.
> >
> > One important reason seems to be that ALL tabs trying to communicate all the time which is totally unnecessary, since I think a hidden tab should stay in a sleeping mode.
> >
> > I was browsing if there is a way to find out if a tab is active or not, and found the following API which exactly does this:
> >
> > https://developer.mozilla.org/en-US/docs/Web/API/Page_Visibility_API
> >
> > Furthermore, the simple
> >
> > document.hidden
> >
> > tag, could be used to find out if the page is currently active.
> >
> > Wouldn't it a good idea to send all midas tabs which are not active into a sleep mode and only reactivate them if they come into focus?
> >
> > I had a quick look at the JavaScript libs of midas, but I am not quite certain where to best inject this. |
1741
|
18 Nov 2019 |
Konstantin Olchanski | Suggestion | javascript comunication | > > Hi, Stefan - I did not look at your code, if all midas tabs are inactive, will the alarm sound still play?
> I added some code to do ONLY alarm updates if in the background (once every 10 seconds)
Ok, good.
> I don't have however a solution to your problem "not enough interaction with page" which I have never seen so far.
When it happens, you should see my messages about it in the javascript console. (rejected promise from audio.play()).
How to make it happen so one can see it and be sure it is handled correctly (audio.src has to be set to blank, at the least), I have no idea.
What I have been looking at for the huge memory and cpu use problem, I would call "a bad interaction between two kludges",
the first kludge is the throttling javascript in inactive tabs, the second kludge is the "not interacted" rejection of audio.play() promise.
(audio.play() returning a promise, and the strange sequencing between this promise success/reject and
the audio events loaded/canplay/done/etc smells of a kludge, too).
> I wonder if we should try push notifications: https://developers.google.com/web/fundamentals/codelabs/push-notifications
> which seems a bit complicated to me. It might also be too subtle if someone is sleeping in front of the computer.
I did not read all the explanation, but if it requires use of 3rd party services, I think we cannot use it, we do not want
to miss "experiment is on fire" alarms just because google is down.
K.O. |
1742
|
28 Nov 2019 |
Konstantin Olchanski | Bug Report | midas alarm sound unreliable in google-chrome | I accidentally discovered a problem with the alarm sounds played by midas.
The javascript code is very simple: var audio=new Audio("alarm.mp3"); audio.play();
In the past, this reliably played a sound.
More recently, I started seeing javascript log messages about "unhandled exception" from audio.play(). (for me, they often
interrupt my javascript debugger sessions, in a very annoying way).
Adding an exception handler to audio.play() was not effective: it turns out, these days, audio.play() returns a Promise
and one must handle the Promise rejection case. (also it turns out that in the rejection handler
one *must* clear audio.src to avoid problems with excessive memory and cpu use).
But why does audio.play() throw a rejected promise?!?
This is the error message provided: "play() failed because the user didn't interact with the document first. https://goo.gl/xX8pDD"
name: "NotAllowedError".
The link takes us to https://developers.google.com/web/updates/2017/09/autoplay-policy-changes. This web page seems to explain
everything neatly, but most of the information turns out to be wrong and unhelpful:
a) "Muted autoplay is always allowed" - wrong - I see audio.play() rejection from muted tabs
b) "User has interacted with the domain (click, tap, etc.)." - wrong - I see rejection in the javascript debugger even as I debug the web page
c) "user's Media Engagement Index threshold has been crossed" - broken - playback count and MEI is always 0 (zero) for midas - per chrome://media-engagement/
d) "user has added the site to their home screen on mobile or installed the PWA on desktop" - what ?!?
e) chrome://flags/#autoplay-policy does not exist (was removed)
There may be a way to start google-chrome with special flags to globally enable autoplay, but it seems
to be aimed at "kiosk" type applications, not for general use browsers:
https://stackoverflow.com/questions/57455849/chrome-autoplay-policy-chrome-76
The bottom line:
there is no way to ensure that the alarm sound will always play
K.O.
P.S. I am tracking this problem here:
https://bitbucket.org/tmidas/midas/issues/191/exception-on-audioplay |
1744
|
28 Nov 2019 |
Konstantin Olchanski | Bug Fix | improvement for midas web page resource use (alarm sound and fit_message) | > > I noticed that midas web pages consume unexpectedly large amount of resources, as observed by the chrome browser
> > "task manager" and by other tools.
The final fix is in. (plus a fix from Stefan).
When the audio.play() promise is rejected, one must clear audio.src, otherwise the browser will continue
with loading the audio file (but will not play it at the end).
Normally, this should not be a problem, but in inactive tabs, all activity is throttled down, and it so happens
that these audio objects accumulate (they are in the state of "we are trying to load the sound file, but
browser slows us down so much!"), consume huge amounts of memory (page memory use goes from ~50 Mbytes
to ~100-200 Mbytes) and consume huge amounts of CPU (not clear how, probably it's the firing of "loading", "canplay", etc
event handlers).
It does not help that mhttpd_fit_message() had a performance bug and consumed large amounts of CPU causing even
more slowing down by the be browser.
After adding audio.src="", all this is gone. I see no special CPU use, and I do not see any strange large memory use.
I still sometimes see inactive tabs grow from ~50 Mbytes to amount ~100 Mbytes. After I open them (activate them),
they quickly shrink back to ~50 Mbytes. I conclude that the browser is slowing down the garbage collector in inactive
tabs so much that it does not keep up with our 1/sec data polling.
So Stefan's fix to reduce polling from 1/sec to 1/10sec should help with this, too. (plus reduction of CPU use by fit_message() should
leave more time for the garbage collector to run).
P.S. General rules for browser slow down of inactive tabs seem to be written here:
https://developer.mozilla.org/en-US/docs/Web/API/Page_Visibility_API
Slowdown of timers is written here:
https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/setTimeout#Notes
K.O. |
1745
|
28 Nov 2019 |
Konstantin Olchanski | Bug Fix | improvement for midas web page resource use (alarm sound and fit_message) | > > > I noticed that midas web pages consume unexpectedly large amount of resources, as observed by the chrome browser
> > > "task manager" and by other tools.
The work on this problem has been blogged in the bitbucket issue tracker:
https://bitbucket.org/tmidas/midas/issues/158/midas-status-page-memory-leak
K.O.
Below is a dump of the issue for posterity ---
Team Midas MIDAS related packages midas Issues
midas status page memory leak
Create issue
Issue #158 RESOLVEDOpen Workflow More Edit
dd1 created an issue 2018-12-25
I have the midas status page (https://daq16.triumf.ca/) open in macos google chrome 71.0.3578.98 and I watch in the "task manager" how the memory use is
246 Mbytes and growing at around 1 Mbyte every 2-3 seconds. CPU use is around 3-5%, network use is 47 kBytes/sec. The slowly growing memory use
indicates that we have a memory leak. (Note that javascript uses "automatic garbage collection" memory management, which does not eliminate memory
leaks. Only capability to explicitly free unused memory is eliminated). K.O.
Comments (35)
dd1 REPORTER
Actual memory use goes up to around 250-something MBytes, then drops down to 240-something, them slowly grows back up, drops down, rinse, repeat.
This is the javascript garbage collection in action. So there is no memory leak on the status page, but still why do we generate around 1 Mbyte/sec of
javascript memory allocations? As comparison, the NYTimes front page consumes 270 Mbytes. One would expect the midas front page to be much more light
weight... K.O.
Edit Pin to top Mark as spam Delete 2018-12-27
dd1 REPORTER
Then there is a question of memory use by the "message" page. This page does grow infinitely large by design - as new messages are added to midas.log -
as as the user keeps scrolling the messages back in time. Perhaps we should somehow limit the total memory use there... K.O.
Edit Pin to top Mark as spam Delete 2018-12-27
Stefan Ritt
changed status to closed
I see the same behaviour. The relatively large memory allocation by Chrome probably comes from some bitmap caching. The browser prints the page
contents into some temporary bitmap and then flushes it to the screen. That can easily take a few MB. I monitor such behaviour since several years now (for
other processes) and concluded that I don't need to worry about JavaScript memory consumption.
Concerning the messages page: One line takes about 100 Bytes. If you scroll really fast, you can do maybe 30 lines per second, thus 3kB. If we allow the
browser to consume another 100 MB (should be easily possible these days), you have to continuously scroll for 100000kb/3kb=30000 seconds or eight
hours. Good luck!
Closing this topic if no complaints.
Pin to top Mark as spam Delete 2019-01-08
dd1 REPORTER
changed status to open
still see high memory use by midas pages. K.O.
Edit Pin to top Mark as spam Delete 2019-09-15
dd1 REPORTER
See high memory use from long running (days-weeks) web pages:
status page of my test experiment - 953 MB - 155 MB after reload
odb editor - 661 MB - 80 MB after reload
programs page - 602 MB - 64 MB after reload
sequencer - 253 MB - 151 MB after reload
sequencer - ??? (very big) - reloaded before I wrote it down
I think we are leaking memory somewhere. Or causing unnecessary allocations that the javascript garbage collector does not keep up with or does not
cleanup correctly. K.O.
‌
‌
Edit Pin to top Mark as spam Delete 2019-09-15
dd1 REPORTER
I am suspicious of memory use trouble from periodic-update code that keeps setting innerHTML to the same value as it was before, unnecessarily. (this also
causes other problems - cannot cut-and-paste affected parts of the web page, high cpu use to redraw the (unchanged) page). K.O.
‌
Edit Pin to top Mark as spam Delete 2019-09-15
Stefan Ritt
For setting innerHTML we should always use
if (text !== control.inner HTML)
control.innerHTML = text;
‌
I thought I caught most of the cases, but I might have missed some. Please add as needed.
‌
Stefan
Pin to top Mark as spam Delete 2019-09-15
dd1 REPORTER
Strange things continue. Just say huge CPU usage from 3 midas web pages (odb editor, programs page and the new sequencer page). All 3 pages are tabs in
an iconized browser window. Suddenly machine feels slow, and I see all 3 use 25% CPU each (by the chrome-browser task manager window). Opened the
browser window, sent to the offending tabs, nothing looks amiss, CPU usage went back to 0%. WTH? (all 3 pages have 100 Mbyte memory use, all 3 pages
update at 1 Hz). K.O.
Edit Pin to top Mark as spam Delete 2019-09-16
dd1 REPORTER
looked at the “programs” page. learned how to use the google-chrome “performance” tool. I was definitely leaking html nodes. The leak was in an
unexpected place - innerHTML with a link was miscomparing because of unexpected string transformation:
xbad: "<a href='?cmd=odb&odb_path=System/Clients/" + key + "'>" + host + "</a>";
good: "<a href=\"?cmd=odb&odb_path=System/Clients/" + key + "\">" + host + "</a>";
Now node leak from my periodic update went from 35 nodes to 2 nodes per update. The performance tool fails to identify where these last 2 nodes are
coming from.
K.O.
Edit Pin to top Mark as spam Delete 2019-09-17
dd1 REPORTER
Forgot to add - the periodic update from mhttpd_init() is also leaking nodes. I will look at it some other time. K.O.
Edit Pin to top Mark as spam Delete 2019-09-17
dd1 REPORTER
after improvement to the “programs” page, the tab is staying at 50-60 Mbytes. promising… K.O.
‌
Edit Pin to top Mark as spam Delete 2019-09-18
dd1 REPORTER
Fixed node leak in mhttpd_refresh(): the alarm display was setting e.innerHTML even if it did not change.
There only remains an unavoidable node leak with “mheader_last_updated” where we set the current time every 1 second. If I comment this out, there is no
node leak on the “programs” page.
K.O.
‌
Edit Pin to top Mark as spam Delete 2019-09-18
dd1 REPORTER
“programs” page memory use now sits around 40 Mbytes. K.O.
Edit Pin to top Mark as spam Delete 2019-09-18
dd1 REPORTER
Stefan points me to the use of e.firstChild.data instead of e.innerHTML, per https://medium.com/@ok.bayat/fixing-memory-leak-problem-in-javascript-
application-ed3a2d9d92df
K.O.
Edit Pin to top Mark as spam Delete 2019-09-18
dd1 REPORTER
implemented this for the timestamp update and (i.e.) the “programs” page now leaks 0 nodes. memory use for all pages sits around 40-60 Mbytes. K.O.
Edit Pin to top Mark as spam Delete 2019-09-26
dd1 REPORTER
see problem of high cpu usage again, after google-chrome restarted after an update to latest version. for example, “program” page is 65 Mbytes, uses 20%
CPU. (in an inactive tab). If I open this tab, for maybe 10 seconds, it goes to 100+ Mbytes with big CPU usage (>100%), then drops down to 90 Mbytes, 0%
CPU usage. I do not see any other web pages or tabs doing this. Only our midas pages. WTH!?! K.O.
‌
Edit Pin to top Mark as spam Delete 2019-10-13
dd1 REPORTER
figured out high cpu usage reported as “rendering”. Open “devtools”, goto “performance”, press Command-Shift-P, start typing “rendering”, select “fps
meter”. A black square will open in top-left, showing graphics activity (frame rate, GPU usage, etc).
Now wait for new message to appear in the top status bar. It will be “yellow” at first, that it will fade to “gray”. During this fading, GPU use is 100% during
about 1 second, FPS is about 50 frames/sec.
K.O.
Edit Pin to top Mark as spam Delete 2019-10-14
dd1 REPORTER
quick google search shows much discussion about css animations using “too much CPU”, i.e. google “css pulsing background”, but no clear way to tell the
browser to slow down. It looks to me like the background-color animation tries to run at maximum possible frame-rate, as if electricity is free. (Since I am
debugging high-cpu and high-memory use of inactive tabs, there is nobody looking at these animations). K.O.
Edit Pin to top Mark as spam Delete 2019-10-14
Stefan Ritt
New messages are displayed with a yellow background and fade to grey after 5 seconds. This is handeled in mhttpd.js around line 2144. You can try to
remove the lines
d.style.setProperty("-webkit-transition", "background-color 3s", "");
d.style.setProperty("transition", "background-color 3s", "");
and see if the CPU load goes down.
Pin to top Mark as spam Delete 2019-10-14
dd1 REPORTER
captured another trace of midas page using 20% cpu in an inactive tab, iconized browser window. capturing is difficult, requires very fast mousing to: select
the right tab, right-click to “inspect”, select “performance” tab, click on “start capture”, and hope that by this time the web page activity does not complete.
this time I got the last 200 ms or so.
what I see is again is “media activity” (only identified as “task”), GPU activity (only identified as “GPU activity”) and main thread activity (identified as an
infinitely repeating sequence of “receive response 206 audio/mpeg”, “receive data 39287 bytes”, “finish loading”, then the same sequence again. 39287 is
the file size of resources/beep.mp3. There is no corresponding network activity, so the loading of beep.mp3 must be coming from cache. On the javascript
console, there are the usual “not allowed to play audio because user did not interact” messages repeating about every 1-2-3 minutes.
I read this as: for reasons unknown, a huge number of audio requests becomes queued (the tab was inactive/iconized for many days) then they start trying to
play (load beep.mp3, do not play it because “not allowed”, move on to the next audio object, load … etc). This is consistent with the cpu use, with the
captured traces and with the quick growth in memory size (beep.mp3 objects are created, consume memory, cannot be free’d until garbage collector runs
later. much later).
The above scenario is impossible with how the current audio playing code is written (only one audio object can exist at a time, new audio object can only be
created after the previous one finished playing).
Two possible explanations: (a) the code running in the web page is not the same code as in mhttpd.js (running an old version from cache) or (b) the code
“one audio object at time” is not working correctly if javascript code is throttled /delayed/stopped in inactive tabs.
Following code will have this problem:
var only_one = null;
function foo() { /* runs from periodic timer */
if (!only_one) {
a = new Audio(“beep.mp3”);
/* throttled/suspended/delayed here */
/* multiple Audio objects created because "only_one" is still null */
only_one = a;
}
}
K.O.
‌
Edit Pin to top Mark as spam Delete 2019-10-17
dd1 REPORTER
fading background - yes, I found the code. pretty neat. I moved it around to remove the timer - I am suspicious of how the timers run in inactive tabs. but no
time to study it.
but the current problem is clearly with audio objects, and the only audio we have is the periodic playing of beep.mp3. who knew there will be so much trouble.
there is still the unexplained use of GPU, but maybe playing/decoding mp3 files uses the GPU.
I am also puzzled why the status page from midas-2019-03 does not show any of these problems. it just sits there using no memory (50 Mbytes) and no
CPU. perhaps we changed something in the playing of audio files since last March (when midas-2019-03 was tagged).
K.O.
‌
Edit Pin to top Mark as spam Delete 2019-10-17
dd1 REPORTER
For the first time I saw my message “mhttpd_alarm_play: Cannot play alarm sound: previous alarm sound did not finish playing yet” reported on the javascript
console. This confirms my guess that playing of audio is actually delayed and indeed we need to check that the previous audio finished playing before
creating new audio objects. But the check in the current code has a race condition. If the delay/stall is inside “new Audio()”, we will create multiple audio
objects as “last_audio” is still in the “finished playing” state, we only change it after the return from “new Audio()”. K.O.
Edit Pin to top Mark as spam Delete 2019-10-28
dd1 REPORTER
see big improvement. now inactive tabs grow from 50-ish Mbytes to 170-ish Mbytes, then when I open them, there is some cpu use (GC, I guess) and
memory use drops back to 50-ish Mbytes. So we are not leaking any memory anymore. Looking at the console messages, I see that my fixes are helping -
there is messages about attempts to create new Audio() when previous one did not finish yet. K.O.
Edit Pin to top Mark as spam Delete 2019-11-04
dd1 REPORTER
I guess, inactive tabs are throttled by google-chrome so much that their GC (memory garbage collection) does not keep up with our 1/sec data updates. I do
not think we need to keep updating inactive tabs at this high frequency, but I am not sure how to detect if we are active or inactive. Maybe I can detect the
throttling instead. K.O.
‌
Edit Pin to top Mark as spam Delete 2019-11-04
dd1 REPORTER
see consistent behaviour from google-chrome:
have all these midas tabs open, inactive, window iconized, typical tab size is 50-ish Mbytes.
google-chrome update arrives
update is installed, all windows and tabs automatically closed, then reopened.
the midas tabs are still inactive, window is iconized
after a few days, see behaviour as described before:
midas tabs use 20-30% CPU, size is 100-ish Mbytes
if I open one of these tabs, it’s cpu usage goes up to 160%, size grows to 250-ish Mbytes, then within 5-10 seconds drops to 100 Mbytes, CPU usage goes
from 160% to zero.
when looking at this, if I am quick enough, I can right-click “inspect”, go to the “performance” tab, and press the “start collecting data” button and I capture
the very tail end of all this strange activity. This is the traces I have been describing so far.
K.O.
Edit Pin to top Mark as spam Delete 2019-11-07
dd1 REPORTER
see big blob of activity:
timer activation
mhttpd_message()
first call to mhttpd_fit_message()
a long cycle of (maybe 10-20) “recalculate size”, “layout”, “parse html”
second call to mhttpd_fit_message()
same long cycle of …
The way I understand this, mhttpd_fit_message() changes the size of some html element that causes the whole window to be re-layed-out.
K.O.
‌
Edit Pin to top Mark as spam Delete 2019-11-07
dd1 REPORTER
trying to figure out what triggers a long run of the “rasterizer” thread. see a very strange call sequence
timer fires
mhttpd_alarm_play()
mhttpdConfig()
“Layout”
mhttpd_alarm_play() calls mhttpdConfig() (3 times) to find out if alarm sound is enabled, the period, the file name, etc. so far so good. but mhttpdConfig()
does not touch any DOM objects, so why is it shown as calling “Layout”?!?
other than this trace, I see nothing else that would trigger the rasterizer thread…
(note) this time, mhttpd_alarm_play() does not call mhttpd_alarm_play_now(), so “new Audio” and stuff does not enter this picture.
K.O.
Edit Pin to top Mark as spam Delete 2019-11-07
dd1 REPORTER
in the early part of the trace, where I think the meat of “tab is using cpu and memory” is, I see the audio events firing in rapid sequence: loadeddata, canplay,
canplaythrough, rinse, repeat.
It turns out that promise rejection from audio.play() does not stop the loading of the sound file. This is easy to see by attaching the event handlers to these
events and by observing these event handlers print something to the javascript console.
If that is what is happening, it explains what I see: all my previous attempts to prevent the piling up of sound files are unsuccessful, and when I open the
previously inactive tab, all the queued sound files start loading (and not playing per “user did not interact” policy).
google docs suggest using audio.src=”” to cancel loading of sound files and it does seem to work. testing it now.
K.O.
‌
Edit Pin to top Mark as spam Delete 2019-11-07
dd1 REPORTER
gotcha. came back home, found one tab using about 10% cpu. audio.src=”” is commented out, javascript console is full of this:
Thu Nov 07 2019 22:17:30 GMT-0800 (Pacific Standard Time): mhttpd_audio_loadeddata: counter 234
mhttpd.js:2763 Thu Nov 07 2019 22:17:30 GMT-0800 (Pacific Standard Time): mhttpd_audio_canplay: counter 234
mhttpd.js:2767 Thu Nov 07 2019 22:17:30 GMT-0800 (Pacific Standard Time): mhttpd_audio_canplaythrough: counter 234
mhttpd.js:2759 Thu Nov 07 2019 22:17:30 GMT-0800 (Pacific Standard Time): mhttpd_audio_loadeddata: counter 235
mhttpd.js:2763 Thu Nov 07 2019 22:17:30 GMT-0800 (Pacific Standard Time): mhttpd_audio_canplay: counter 235
mhttpd.js:2767 Thu Nov 07 2019 22:17:30 GMT-0800 (Pacific Standard Time): mhttpd_audio_canplaythrough: counter 235
mhttpd.js:2759 Thu Nov 07 2019 22:17:30 GMT-0800 (Pacific Standard Time): mhttpd_audio_loadeddata: counter 236
mhttpd.js:2763 Thu Nov 07 2019 22:17:30 GMT-0800 (Pacific Standard Time): mhttpd_audio_canplay: counter 236
mhttpd.js:2767 Thu Nov 07 2019 22:17:30 GMT-0800 (Pacific Standard Time): mhttpd_audio_canplaythrough: counter 236
mhttpd.js:2759 Thu Nov 07 2019 22:17:30 GMT-0800 (Pacific Standard Time): mhttpd_audio_loadeddata: counter 237
mhttpd.js:2763 Thu Nov 07 2019 22:17:30 GMT-0800 (Pacific Standard Time): mhttpd_audio_canplay: counter 237
mhttpd.js:2767 Thu Nov 07 2019 22:17:30 GMT-0800 (Pacific Standard Time): mhttpd_audio_canplaythrough: counter 237
mhttpd.js:2759 Thu Nov 07 2019 22:17:30 GMT-0800 (Pacific Standard Time): mhttpd_audio_loadeddata: counter 238
mhttpd.js:2763 Thu Nov 07 2019 22:17:30 GMT-0800 (Pacific Standard Time): mhttpd_audio_canplay: counter 238
mhttpd.js:2767 Thu Nov 07 2019 22:17:30 GMT-0800 (Pacific Standard Time): mhttpd_audio_canplaythrough: counter 238
mhttpd.js:2759 Thu Nov 07 2019 22:17:30 GMT-0800 (Pacific Standard Time): mhttpd_audio_loadeddata: counter 239
mhttpd.js:2763 Thu Nov 07 2019 22:17:30 GMT-0800 (Pacific Standard Time): mhttpd_audio_canplay: counter 239
mhttpd.js:2767 Thu Nov 07 2019 22:17:30 GMT-0800 (Pacific Standard Time): mhttpd_audio_canplaythrough: counter 239
the timestamp is exactly when I opened this tab. so confirmed, a whole bunch of audio files got queued, when I open the tab they all try to play. (there is no
actual sound, all tabs are muted).
now I uncomment audio.src=”” and see what happens.
K.O.
‌
Edit Pin to top Mark as spam Delete 2019-11-07
dd1 REPORTER
looks good. an update to google-chrome came in and after installing, I no longer see midas tabs show high cpu usage or high memory use. I think the
audio.src=”” fix is it. I will be committing these fixes to midas. K.O.
Edit Pin to top Mark as spam Delete 2019-11-12
Stefan Ritt
The loop in mhttpd_fit_message() is there for a good reason: I want to display the message in a single line. If it’s too long, I want first to cut the time stamp
and then display it. If it’s still too long, I want to truncate the message and display “…” at the end. The problem is what “too long” is. Nobody can tell you how
much pixel a message on your browser take, because this depends on the installed fonts, the exact character spacing of your browser and so on. So the only
way I could make this happen is to add one char at a time, until we get close to the maximum allowed space. If course this requires a re-layout of the page for
10-20 times, but when your window is in the foreground this is not a problem, since a browser can do this with small CPU load. The “scope” application I use
does 70 frames per second at 30% CPU load. So one could make the loop a bit smarter, like binary search, which would drop the 10-20 iterations to log2(10-
20) ~ 4-5, but still there would be a loop.
How that the update of the messages in the background is suppressed with the hidden API, do you still have that problem or can we consider it fixed?
Stefan
Pin to top Mark as spam Delete 2019-11-18
dd1 REPORTER
see new behaviour - after many days, inactive page size is ~180 Mbytes. 0% CPU use (an improvement from before where there was large CPU use). activate
the tab, nothing much happens, 0% CPU use (again an improvement from before). after about 30 seconds, memory use drops down to the normal 50-70-80
Mbytes. I think what we see is the garbage collector is throttled down and does not keep up with our allocations. Stefan’s new fix reducing polling in inactive
pages from 1/sec to 1/10sec should help with this. K.O.
‌
Edit Pin to top Mark as spam Delete 2019-11-19
dd1 REPORTER
mhttpd_fit_message() - confirmed. I was confused about the function argument.
I thought it is passed an array of messages. no, it is one message string and the loop is over the message string length. The loop is done twice (second time,
with the time/date stamp removed). google-chrome debugger does show that this uses large amount of CPU, mainly to compute d.offsetWidth.
I think I will refactor these loops - instead of growing the message, I will shrink it.
K.O.
Edit Pin to top Mark as spam Delete 4 hours ago
dd1 REPORTER
rewrote mhttpd_fit_message() to reduce CPU use: try to fit complete message, if too long, try to fit message without timestamp, if too long, guess the desired
length assuming all chars have same width, then grow or shrink the message until the size is right. K.O.
Edit Pin to top Mark as spam Delete 17 minutes ago
dd1 REPORTER
changed status to resolved
The main fix is to set audio.src="" in the promise rejection. K.O.
Edit Pin to top Mark as spam Delete just now
What would you like to say?
Assignee
–
Type
bug
Priority
major
Status
resolved
Votes
0 Vote for this issue
Watchers
1 Stop watching
Dismiss this bannerJira Software: the preferred issue tracker for Bitbucket. Join the team! |
|