> Very good idea. And thanks for finding the document.hidden solution. I put it in, so give it a try.

Hi, Stefan - I did not look at your code, if all midas tabs are inactive, will the alarm sound still play?

K.O.


> 
> Best,
> Stefan
> 
> 
> > I am currently testing the new history system on the mhttpd side and stumbled over the following issue: typically our user open a lot of midas web-page tabs and keep them open. With the current version this leads after a night typically to a state where the browser is busy with itself and not reacting anymore.
> > 
> > One important reason seems to be that ALL tabs trying to communicate all the time which is totally unnecessary, since I think a hidden tab should stay in a sleeping mode. 
> > 
> > I was browsing if there is a way to find out if a tab is active or not, and found the following API which exactly does this:
> > 
> > https://developer.mozilla.org/en-US/docs/Web/API/Page_Visibility_API
> > 
> > Furthermore, the simple
> > 
> > document.hidden 
> > 
> > tag, could be used to find out if the page is currently active.
> > 
> > Wouldn't it a good idea to send all midas tabs which are not active into a sleep mode and only reactivate them if they come into focus?
> > 
> > I had a quick look at the JavaScript libs of midas, but I am not quite certain where to best inject this. 

> Hi, Stefan - I did not look at your code, if all midas tabs are inactive, will the alarm sound still play?

Nope. All updates are done in mhhtpd_refresh(), and I changed it such that nothing is updated if hidden.

I agree however that this is bad. You want to hear alarms always. So I added some code to do ONLY alarm updates if in the background. 
No GUI changes, only audio playing. Checking is reduced from 1 Hz to 0.1 Hz (once every 10 seconds). I don't have however a solution 
to your problem "not enough interaction with page" which I have never seen so far.

I wonder if we should try push notifications: https://developers.google.com/web/fundamentals/codelabs/push-notifications
which seems a bit complicated to me. It might also be too subtle if someone is sleeping in front of the computer.

Stefan

> a) google chrome slows down the execution of javascript in inactive tabs, leading to trouble
> with memory management - midas pages poll at 1/sec, each poll allocates memory for processing RPC messages,
> and (until recently) allocates memory for new DOM objects to update the web page - but the garbage collector
> gets slowed down and does not keep up - leading to huge memory use (up to 200 Mbytes) for inactive midas pages
> that normally consume 50-ish Mbytes.

Try my latest change, which drops any update if hidden, except the alarm sound. At the moment this only works on the main "status" page.

> most midas web pages poll in two places - mhttpd_refresh() updates the current date timestamp, alarms, currently active midas.log message;
> and each page has it's own loop for updating it's own data (i.e. "alarms" page, "programs" page).

That's correct. I changed mhttpd_refresh() now and the main loop for the "status" page. If that works for everybody, we can do that also for "programs" and other pages. The code is here:

      if (document.hidden) {
         // don't update page if hidden
         setTimeout(update_page, 500);
         return;
      }

where "update_page" has to be replaced with the proper function.

Stefan

> > Hi, Stefan - I did not look at your code, if all midas tabs are inactive, will the alarm sound still play?
> I added some code to do ONLY alarm updates if in the background (once every 10 seconds)

Ok, good.

> I don't have however a solution to your problem "not enough interaction with page" which I have never seen so far.

When it happens, you should see my messages about it in the javascript console. (rejected promise from audio.play()).

How to make it happen so one can see it and be sure it is handled correctly (audio.src has to be set to blank, at the least), I have no idea.

What I have been looking at for the huge memory and cpu use problem, I would call "a bad interaction between two kludges",
the first kludge is the throttling javascript in inactive tabs, the second kludge is the "not interacted" rejection of audio.play() promise.
(audio.play() returning a promise, and the strange sequencing between this promise success/reject and
the audio events loaded/canplay/done/etc smells of a kludge, too).

> I wonder if we should try push notifications: https://developers.google.com/web/fundamentals/codelabs/push-notifications
> which seems a bit complicated to me. It might also be too subtle if someone is sleeping in front of the computer.

I did not read all the explanation, but if it requires use of 3rd party services, I think we cannot use it, we do not want
to miss "experiment is on fire" alarms just because google is down.

K.O.

We're using the History Logger MIDAS feature and writing to mySQL tables, but 
in some cases have run into issues installing mySQL on centos7 systems.

Has anyone ever tried running this MIDAS feature with MariaDB rather than 
mySQL?

> We're using the History Logger MIDAS feature and writing to mySQL tables, but 
> in some cases have run into issues installing mySQL on centos7 systems.
> 
> Has anyone ever tried running this MIDAS feature with MariaDB rather than 
> mySQL?

Hi

I've used MIDAS Logger feature with MariaDB and had no issue thus far .

> We're using the History Logger MIDAS feature and writing to mySQL tables, but 
> in some cases have run into issues installing mySQL on centos7 systems.
> 
> Has anyone ever tried running this MIDAS feature with MariaDB rather than 
> mySQL?

The best I can tell, MariaDB is *the* mysql. "the other thing" is an abandonware fork.

I personally at this moment do not run any daq stations with mysql logging, and I accidentally
removed all the 25 versions and flavours of mysql from my home laptop, so if there
is any problems with mysql, please holler, and I will reinstall mysql (ahem, mariadb) and restore
my ability to test midas against it.

As a connected question, is there any need to have postgres support in midas as well?

K.O.

Hi, Stefan & co - now that midas is c++11 and c++11 comes with a threads library, should we 
switch midas to use the c++11 threads instead of pthreads? (Of course on Linux c++11 
threads are a layer on top of pthreads, the best I know).

This should remove the dependency on pthreads.h and use a more native implementation of 
threads on MacOS and Windows. (again, the best I can tell).

Of course this depends on c++11 threads having all the functions we need. Specifically, "lock 
with timeout" is useful to deal with "gah! everything stopped! what do I do!", a problem 
bedeviling midas in the early days (and still happens today!). Current midas kills everything 
after 5 minutes of deadlock - then the user knows how to restart everything and the developer 
has core dumps to look at. (to see which program/thread holds the lock and would not give it 
up).

Any thoughts on this?

K.O.

I'm thinking of this already since some time, and it was part of my motivation switching to C++11.
I was delighted to see that what we do in system.c (encapsulate system functions such as threads
and shared memory) is now done natively in C++11, and it's done by experts and not amateurs like us.

I see that with std::timed_mutex and try_lock_for we have the desired timeout function. Pity that
C++11 does not contain inter-process communication like semaphores, so there we still have to use
our old functions. 

But for threads switching to std::thread, I'm all in.

Stefan

> Hi, Stefan & co - now that midas is c++11 and c++11 comes with a threads library, should we 
> switch midas to use the c++11 threads instead of pthreads? (Of course on Linux c++11 
> threads are a layer on top of pthreads, the best I know).
> 
> This should remove the dependency on pthreads.h and use a more native implementation of 
> threads on MacOS and Windows. (again, the best I can tell).
> 
> Of course this depends on c++11 threads having all the functions we need. Specifically, "lock 
> with timeout" is useful to deal with "gah! everything stopped! what do I do!", a problem 
> bedeviling midas in the early days (and still happens today!). Current midas kills everything 
> after 5 minutes of deadlock - then the user knows how to restart everything and the developer 
> has core dumps to look at. (to see which program/thread holds the lock and would not give it 
> up).
> 
> Any thoughts on this?
> 
> K.O.

> Hi, Stefan & co - now that midas is c++11 and c++11 comes with a threads library, should we 
> switch midas to use the c++11 threads instead of pthreads? (Of course on Linux c++11 
> threads are a layer on top of pthreads, the best I know).
> 
> This should remove the dependency on pthreads.h and use a more native implementation of 
> threads on MacOS and Windows. (again, the best I can tell).
> 
> Of course this depends on c++11 threads having all the functions we need. Specifically, "lock 
> with timeout" is useful to deal with "gah! everything stopped! what do I do!", a problem 
> bedeviling midas in the early days (and still happens today!). Current midas kills everything 
> after 5 minutes of deadlock - then the user knows how to restart everything and the developer 
> has core dumps to look at. (to see which program/thread holds the lock and would not give it 
> up).
> 
> Any thoughts on this?
> 
> K.O.

Hi, I just wanted to say that I have seen this post and maybe that is the solution to the pthreads compiler problem in OS 10.15, but 
of course I am a total amateur in here. Thanks for thinking about this and I will wait and hold to see what gets decided. Thanks

Berta  

I am using the Midas sequencer to run subsequent measurements in a loop, without 
knowing how many iterations in advance. Therefore, I am using the "infinity" 
option. Since I have other commands after the loop, it would be nice to have the 
possibility to break the loop, but let the sequencer then finish the rest of the 
commands. 
Cheers,
Ivo

> I am using the Midas sequencer to run subsequent measurements in a loop, without 
> knowing how many iterations in advance. Therefore, I am using the "infinity" 
> option. Since I have other commands after the loop, it would be nice to have the 
> possibility to break the loop, but let the sequencer then finish the rest of the 
> commands. 
> Cheers,
> Ivo

You can do that with the "GOTO" statement, jumping to the first line after the loop.

Here is a working example:


LOOP runs, 5
     WAIT Seconds 3
     IF $runs > 2
         GOTO 7
     ENDIF
ENDLOOP
MESSAGE "Finished", 1

Best,
Stefan

> You can do that with the "GOTO" statement, jumping to the first line after the loop.
> 
> Here is a working example:
> 
> 
> LOOP runs, 5
>      WAIT Seconds 3
>      IF $runs > 2
>          GOTO 7
>      ENDIF
> ENDLOOP
> MESSAGE "Finished", 1
> 
> Best,
> Stefan

Hoi Stefan

Thanks for your answer. As I understand it, this has to be in the sequence script before 
running. So, in the end, it is not different than just saying "LOOP runs, 2" and 
therefore the number of runs has do be known in advance as well. Or is there an option to 
change the script on runtime? What I would like, is to start a sequence with "LOOP runs, 
infinite" and when I come back to the experiment after falling asleep being able to break 
the loop after the next iteration, but still execute everything after ENDLOOP, i.e. the 
MESSAGE statement in your example. Because if I do a "Stop after current run", this seems 
not to happen. 

Best, Ivo

> > You can do that with the "GOTO" statement, jumping to the first line after the loop.
> > 
> > Here is a working example:
> > 
> > 
> > LOOP runs, 5
> >      WAIT Seconds 3
> >      IF $runs > 2
> >          GOTO 7
> >      ENDIF
> > ENDLOOP
> > MESSAGE "Finished", 1
> > 
> > Best,
> > Stefan
> 
> Hoi Stefan
> 
> Thanks for your answer. As I understand it, this has to be in the sequence script before 
> running. So, in the end, it is not different than just saying "LOOP runs, 2" and 
> therefore the number of runs has do be known in advance as well. Or is there an option to 
> change the script on runtime? What I would like, is to start a sequence with "LOOP runs, 
> infinite" and when I come back to the experiment after falling asleep being able to break 
> the loop after the next iteration, but still execute everything after ENDLOOP, i.e. the 
> MESSAGE statement in your example. Because if I do a "Stop after current run", this seems 
> not to happen. 
> 
> Best, Ivo

First, you have the sequencer button "Stop after current run", but that does of course ot
execute anything after ENDLOOP. 

Second, you can put anything in the IF statement. Like create a variable on the ODB like
/Experiment/Run parameters/Stop loop and set this to zero. Then put in your script:

...
ODBGET /Experiment/Run parameters/Stop loop, flag
IF $flag == 1
   GOTO 7
...

So once you want to stop the loop, set the flag in the ODB to one.

Best,
Stefan

> LOOP runs, 5
> ...
> ENDLOOP

Classical loop constructs usually include "break" (exit the loop) and "continue" (loop again!) 
constructs. I would say it is an unfortunate omission if they are not present in the midas sequencer.

But Stefan is right, of course, both commands are just funny names for "goto".

K.O.

Hello everybody,

I am really appreciate the development of the new odb++ API. So I directly started to rewrite the code for the Mu3e DAQ system.

I have a view questions / suggestions which came up during my work so fare:

1. The documentation seems to be quite new so there are some variables wrong named and small typo stuff. I would like to fix them. Should I request for an account or what else is needed to change them?

2. When I create an ODB structure with the new API I do for example:

    midas::odb stream_settings = {
            {"Test_odb_api", {
                                      {"Divider", 1000},     // int
                                      {"Enable", false},     // bool
                              }},
    };
    stream_settings.connect("/Equipment/Test/Settings", true);

and with 

midas::odb datagen("/Equipment/Test/Settings/Test_odb_api");
std::cout << "Datagenerator Enable is " << datagen["Enable"] << std::endl;

I am getting back false. Which looks nice but when I look into the odb via the browser the value is actually "y" meaning true which is stange. I added my frontend where I cleaned all function leaving only the frontend_init() one where I create this key. Its a cuda program but since I clean everything no cuda function is called anymore.

Thank you again for the nice development!

Cheers,
Marius 

> 2. When I create an ODB structure with the new API I do for example:
> 
>     midas::odb stream_settings = {
>             {"Test_odb_api", {
>                                       {"Divider", 1000},     // int
>                                       {"Enable", false},     // bool
>                               }},
>     };
>     stream_settings.connect("/Equipment/Test/Settings", true);
> 
> and with 
> 
> midas::odb datagen("/Equipment/Test/Settings/Test_odb_api");
> std::cout << "Datagenerator Enable is " << datagen["Enable"] << std::endl;
> 
> I am getting back false. Which looks nice but when I look into the odb via the browser the value is actually "y" meaning true which is stange. I added my frontend where I cleaned all function leaving only the frontend_init() one where I create this key. Its a cuda program but since I clean everything no cuda function is called anymore.

I cannot confirm this behaviour. Just put following code in a standalone program:

cm_connect_experiment(NULL, NULL, "test", NULL);
   midas::odb::set_debug(true);

   midas::odb stream_settings = {
           {"Test_odb_api", {
               {"Divider", 1000},     // int
               {"Enable", false},     // bool
           }},
   };
   stream_settings.connect("/Equipment/Test/Settings", true);

   midas::odb datagen("/Equipment/Test/Settings/Test_odb_api");
   std::cout << "Datagenerator Enable is " << datagen["Enable"] << std::endl;

and run it. The result is:

...
Get ODB key "/Equipment/Test/Settings/Test_odb_api/Enable": false
Datagenerator Enable is Get ODB key "/Equipment/Test/Settings/Test_odb_api/Enable": false
false

Looking in the ODB, I also see 

[local:Online:S]/>cd Equipment/Test/Settings/Test_odb_api/
[local:Online:S]Test_odb_api>ls
Divider                         1000
Enable                          n
[local:Online:S]Test_odb_api>


So not sure what is different in your case. Are you looking to the same ODB? Maybe you have one remote, and local? 
Note that the "true" flag in stream_settings.connect(..., true); forces all default values into the ODB. 
So if the ODB value is "y", it will be cdhanged to "n".

Best,
Stefan

Marius, has the problem been fixed in meantime?

Stefan

> I am getting back false. Which looks nice but when I look into the odb via the browser the value is actually "y" meaning true which is stange. 
> I added my frontend where I cleaned all function leaving only the frontend_init() one where I create this key. Its a cuda program but since 
> I clean everything no cuda function is called anymore.

Hi Stefan,

your test program was only working for me after I changed the following lines inside the odbxx.cpp

diff --git a/src/odbxx.cxx b/src/odbxx.cxx
index 24b5a135..48edfd15 100644
--- a/src/odbxx.cxx
+++ b/src/odbxx.cxx
@@ -753,7 +753,12 @@ namespace midas {
          }
       } else {
          u_odb u = m_data[index];
-         status = db_set_data_index(m_hDB, m_hKey, &u, rpc_tid_size(m_tid), index, m_tid);
+         if (m_tid == TID_BOOL) {
+             BOOL ss = bool(u);
+             status = db_set_data_index(m_hDB, m_hKey, &ss, rpc_tid_size(m_tid), index, m_tid);
+         } else {
+             status = db_set_data_index(m_hDB, m_hKey, &u, rpc_tid_size(m_tid), index, m_tid);
+         }
          if (m_debug) {
             std::string s;
             u.get(s);

Likely not the best fix but otherwise I was always getting after running the test program:

[ODBEdit,INFO] Program ODBEdit on host localhost started
[local:Default:S]/>cd Equipment/Test/Settings/Test_odb_api/
key not found
makoeppe@office ~/mu3e/online/online (git)-[odb++_api] % test_connect
Created ODB key /Equipment/Test/Settings
Created ODB key /Equipment/Test/Settings/Test_odb_api
Created ODB key /Equipment/Test/Settings/Test_odb_api/Divider
Set ODB key "/Equipment/Test/Settings/Test_odb_api/Divider" = 1000
Created ODB key /Equipment/Test/Settings/Test_odb_api/Enable
Set ODB key "/Equipment/Test/Settings/Test_odb_api/Enable" = false
Get definition for ODB key "/Equipment/Test/Settings/Test_odb_api"
Get definition for ODB key "/Equipment/Test/Settings/Test_odb_api/Divider"
Get ODB key "/Equipment/Test/Settings/Test_odb_api/Divider": 1000
Get definition for ODB key "/Equipment/Test/Settings/Test_odb_api/Enable"
Get ODB key "/Equipment/Test/Settings/Test_odb_api/Enable": false
Get definition for ODB key "/Equipment/Test/Settings/Test_odb_api/Divider"
Get ODB key "/Equipment/Test/Settings/Test_odb_api/Divider": 1000
Get definition for ODB key "/Equipment/Test/Settings/Test_odb_api/Enable"
Get ODB key "/Equipment/Test/Settings/Test_odb_api/Enable": false
Datagenerator Enable is Get ODB key "/Equipment/Test/Settings/Test_odb_api/Enable": false
false
makoeppe@office ~/mu3e/online/online (git)-[odb++_api] % odbedit
[ODBEdit,INFO] Program ODBEdit on host localhost started
[local:Default:S]/>cd Equipment/Test/Settings/Test_odb_api/
[local:Default:S]Test_odb_api>ls
Divider                         1000
Enable                          y 

> > I am getting back false. Which looks nice but when I look into the odb via the browser the value is actually "y" meaning true which is stange. 
> > I added my frontend where I cleaned all function leaving only the frontend_init() one where I create this key. Its a cuda program but since 
> > I clean everything no cuda function is called anymore.