> /code/midas/manalyzer/manalyzer.cxx:799:27: error: ‘pow’ was not declared in this scope
>   799 |       bins[i] = TimeRange*pow(1.1,i)/pow(1.1,Nbins);

math.h added, pushed. nice catch.

implicit include of math.h came through TFile.h (ROOT v6.34.02), perhaps you have a newer ROOT
and they jiggled the include files somehow.

TFile.h -> TDirectoryFile.h -> TDirectory.h -> TNamed.h -> TString.h -> TMathBase.h -> cmath -> math.h

K.O.

> Thanks for reporting this. It was caused by a
> 
>   MJsonNode* node = MJsonNode::Parse(str.c_str());
> 
> not followed by a 
> 
>   delete node;
> 
> I added that now in odb::odb_from_json_string(). Can you try again?
> 
> Stefan

Close, but no cigar, node you delete is not the node you got from Parse(), see "node = subnode;".

If I delete the node returned by Parse(), I confirm the memory leak is gone.

BTW, it looks like we are parsing the whole JSON ODB dump (200k+) on every odbxx access. Can you parse it just once?

K.O.

> BTW, it looks like we are parsing the whole JSON ODB dump (200k+) on every odbxx access. Can you parse it just once?

You are right. I changed the code so that the dump is only parsed once. Please give it a try.

Stefan

Thanks. Are you happy for me to update the submodule commit in Midas to use this fix? I should have sufficient permission if you agree.

> > /code/midas/manalyzer/manalyzer.cxx:799:27: error: ‘pow’ was not declared in this scope
> >   799 |       bins[i] = TimeRange*pow(1.1,i)/pow(1.1,Nbins);
> 
> math.h added, pushed. nice catch.
> 
> implicit include of math.h came through TFile.h (ROOT v6.34.02), perhaps you have a newer ROOT
> and they jiggled the include files somehow.
> 
> TFile.h -> TDirectoryFile.h -> TDirectory.h -> TNamed.h -> TString.h -> TMathBase.h -> cmath -> math.h
> 
> K.O.

> > BTW, it looks like we are parsing the whole JSON ODB dump (200k+) on every odbxx access. Can you parse it just once?
> You are right. I changed the code so that the dump is only parsed once. Please give it a try.

Confirmed fixed, thanks! There are 2 small changes I made in odbxx.h, please pull.

K.O.

> TFile.h -> TDirectoryFile.h -> TDirectory.h -> TNamed.h -> TString.h -> TMathBase.h -> cmath -> math.h

reading ROOT release notes, 6.38 removed TMathBase.h from TString.h, with a warning "This change may cause errors during compilation of 
ROOT-based code". Upright citizens, nice guys!

> Thanks. Are you happy for me to update the submodule commit in Midas to use this fix? I should have sufficient permission if you agree.

I am doing some last minute tests, will pull it into midas and rootana later today.

K.O.

> Confirmed fixed, thanks! There are 2 small changes I made in odbxx.h, please pull.

There was one missing enable_jsroot in manalyzer, please pull yourself.

Stefan

> > Confirmed fixed, thanks! There are 2 small changes I made in odbxx.h, please pull.
> There was one missing enable_jsroot in manalyzer, please pull yourself.

pulled, pushed to rootana, thanks for fixing it!

K.O.

I have an application where I want to (optionally) send my own debugging messages to the midas.log file, but am having some problems with this: 

* Messages with type MT_DEBUG don't show up in midas.log or on the messages page (calling cm_msg from python)
* messages.html is missing the DEBUG filter option
* Messages sent to other log files (not midas.log) don't get message banners on the web page. Is this intentional? 

So I think either there is a bug, or I need to start MIDAS with some flag to enable debugging. Looking at the source, I don't see why these messages wouldn't get logged.

Any insight would be appreciated!

MT_DEBUG messages are there for debugging, not logging. They only go into the SYSMSG buffer and NOT to the log file. If you want anything logged, just use MT_INFO.

Not sure if that's missing in the documentation. Anyhow, there are my original ideas (from 1995 ;-) )

MT_ERROR  
Error message, to be displayed in red

MT_INFO    
 Info or status message

MT_DEBUG 
Only sent to SYSMSG buffer, not to midas.log file. Handy if you produce lots of message and don't want to flood the message file. Plus it does not change the timing of your app, since the SYSMSG buffer is much faster than writing 
to a file.

MT_USER
Message generated interactively by a user, like in the chat window or via the odbedit "msg" command

MT_LOG
Messages with are only logged but not put into the SYSMSG buffer

MT_TALK
Messages which should go through the speech synthesis in the browser and are "spoken"

MT_CALL
Message which would be forwarded to the user via a messaging app (historically this was an actual analog telephone call via a modem ;-) )

If that is missing in the documentation, please feel free to copy/paste it to the appropriate place.


Stefan

Ok thanks for the quick and clear response!

Derek

> MT_DEBUG messages are there for debugging, not logging. They only go into the SYSMSG buffer and NOT to the log file. If you want anything logged, just use MT_INFO.
> 
> Not sure if that's missing in the documentation. Anyhow, there are my original ideas (from 1995 ;-) )
> 
> MT_ERROR  
> Error message, to be displayed in red
> 
> MT_INFO    
>  Info or status message
> 
> MT_DEBUG 
> Only sent to SYSMSG buffer, not to midas.log file. Handy if you produce lots of message and don't want to flood the message file. Plus it does not change the timing of your app, since the SYSMSG buffer is much faster than writing 
> to a file.
> 
> MT_USER
> Message generated interactively by a user, like in the chat window or via the odbedit "msg" command
> 
> MT_LOG
> Messages with are only logged but not put into the SYSMSG buffer
> 
> MT_TALK
> Messages which should go through the speech synthesis in the browser and are "spoken"
> 
> MT_CALL
> Message which would be forwarded to the user via a messaging app (historically this was an actual analog telephone call via a modem ;-) )
> 
> If that is missing in the documentation, please feel free to copy/paste it to the appropriate place.
> 
> 
> Stefan

We finished running the DarkLight experiment and I am reporting accumulated bugs that we have run into.

1) history plots on 12 hrs, 24 hrs tend to hang with "page not responsive". most plots have 16-20 variables, 
which are recorded at 1/sec interval. (yes, we must see all the variables at the same time and yes, we want to 
record them with fine granularity).

2) starting runs gets into a funny mode if a GEM frontend aborts (hardware problems), transition page reports 
"wwrrr, timeout 0", and stays stuck forever, "cancel transition" does nothing. observe it goes from "w" 
(waiting) to "r" (RPC running) without a "c" (connecting...) and timeout should never be zero (120 sec in 
ODB).

3) ODB editor clicking on hex number versus decimal number no longer allows editing in hex, Stefan implemented 
this useful feature and it worked for a while, but now seems broken.

4) ODB editor "right click" to "delete" or "rename" key does not work, the right-click menu disappears 
immediately before I can use it (dl-server-2), click on item (it is now blue), right-click menu disappears 
before I can use it (daq17). it looks like a timing or race condition.

5) ODB editor "create link" link target name is limited to 32 bytes, links cannot be created (dl-server-2), ok 
on daq17 with current MIDAS.

6) MIDAS on dl-server-2 is "installed" in such a way that there is no connection to the git repository, no way 
to tell what git checkout it corresponds to. Help page just says "branch master", git-revision.h is empty. We 
should discourage such use of MIDAS and promote our "normal way" where for all MIDAS binary programs we know 
what source code and what git commit was used to build them.

6a) MIDAS on dl-server-2 had a pretty much non-functional history display, I reported it here, Stefan provided 
a fix, I manually retrofitted it into dl-server-2 MIDAS and we were able to run the experiment. (good)

6b) bug (5) suggests that there is more bugs being introduced and fixed without any notice to other midas 
users (via this forum or via the bitbucket bug tracker).

K.O.

Thanks for the detailed report. Let me reply one-by-one.

> 1) history plots on 12 hrs, 24 hrs tend to hang with "page not responsive". most plots have 16-20 variables, 
> which are recorded at 1/sec interval. (yes, we must see all the variables at the same time and yes, we want to 
> record them with fine granularity).

Attached is a similar plot. 8 values recorded every second, displayed for 24h. The backend is actually a Raspberry Pi! I see no issues there. Do you have 
the current history version which does the re-binning? Actually the plot below is still without rebinding (see the "1" at the top right), and it contains ~72000 points x 8. The browser does not have any issue 
with it.

Stefan

> 3) ODB editor clicking on hex number versus decimal number no longer allows editing in hex, Stefan implemented 
> this useful feature and it worked for a while, but now seems broken.

I cannot confirm. See below. There was some issue some time ago, but that's fixed since a while. Please pull on develop and try again.

Here is the change: https://bitbucket.org/tmidas/midas/commits/882974260876529c43811c63a16b4a32395d416a

Stefan

> 4) ODB editor "right click" to "delete" or "rename" key does not work, the right-click menu disappears 
> immediately before I can use it (dl-server-2), click on item (it is now blue), right-click menu disappears 
> before I can use it (daq17). it looks like a timing or race condition.

Confirmed and fixed: https://bitbucket.org/tmidas/midas/commits/4ba30761683ac9aa558471d2d2d35ce05e72096a

/Stefan

> 5) ODB editor "create link" link target name is limited to 32 bytes, links cannot be created (dl-server-2), ok 
> on daq17 with current MIDAS.

Works for me with the current version.


> 6) MIDAS on dl-server-2 is "installed" in such a way that there is no connection to the git repository, no way 
> to tell what git checkout it corresponds to. Help page just says "branch master", git-revision.h is empty. We 
> should discourage such use of MIDAS and promote our "normal way" where for all MIDAS binary programs we know 
> what source code and what git commit was used to build them.

Not sure if you have seen it. I make a "install" script to clone, compile and install midas. Some people use this already. Maybe give it a shot. Might need 
adjustment for different systems, I certainly haven't covered all corner cases. But on a RaspberryPi it's then just one command to install midas, modify 
the environment, install mhttpd as a service and load the ODB defaults. I know that some people want it "their way" and that's ok, but for the novice user 
that might be a good starting point. It's documented here: https://daq00.triumf.ca/MidasWiki/index.php/Install_Script

The install script is plain shell, so should be easy to be understandable. 

> 6a) MIDAS on dl-server-2 had a pretty much non-functional history display, I reported it here, Stefan provided 
> a fix, I manually retrofitted it into dl-server-2 MIDAS and we were able to run the experiment. (good)
> 
> 6b) bug (5) suggests that there is more bugs being introduced and fixed without any notice to other midas 
> users (via this forum or via the bitbucket bug tracker).

If I would notify everybody about a new bug I introduced, I would know that it's a bug and I would not introduce it ;-)

For all the fixes I encourage people to check the commit log. Doing an elog entry for every bug fix would be considered spam by many people because 
that can be many emails per week. The commit log is here: https://bitbucket.org/tmidas/midas/commits/branch/develop

If somebody volunteers to consolidate all commits and make a monthly digest to be posted here, I'm all in favor, but I'm not that individual.

Stefan

I commited a fix to mhttpd logic that decides which messages should be shown in
"red" colour- before, any message with square brackets and colons would be
highlighted in red. Now only messages matching the pattern [...:...] are
highlighted. The decision logic was moved into a function message_red(). K.O.

Problem:
If cm_set_transition_sequence() is used for changing the sequence number, the 
command odbedit> start/stop/resume/pause -v report the propre sequence but the
action on the client side is actually not performed!

Fix:
Local transition table updated in midas.c (1.226)

Note:
The transition number under /system/clients/<pid>/transition...
is used internally. Changing it won't have any effect on the client action
if sequence number is not registered.

We stumbled upon and fixed a "double free" bug in src/ybos.c causing crashes in
lazylogger writing .mid files in the FORMAT_MIDAS format (why does it use
ybos.c? Pierre says- for generic file i/o). Why this code had ever worked before
remains a mystery. K.O.

I fixed some minor bit rot in the example experiment: a few minor Makefile
problems, make the analyzer use the current histogram creation macros, etc. I
also added startup and shutdown scripts. These will be documented as we work
through them with our Summer student. K.O.