> I recently upgraded to MIDAS version midas-2019-06-b. I had to make a few changes 
> to get our custom page running again, but am a little confused on starting and 
> stopping runs.

So far so good.

> When I click on my "Start" button, it now redirects to a 
> Transition page rather than reloading the status page.

Are you sure? The "start" button redirects to the "start" page (start.html) which redirects
to the "transition" page (transition.html), which does not redirect anywhere so you can see
the result of the transition.

> Could someone explain the reasoning for the current behavior?

It's been like this for years now. Stefan suggest that we implement the "start" page
and the "transition" page as overlays on top of the status page, but it did not happen yet.

> Furthermore my "Stop" button is now broken with the following error:
> Error: Invalid URL "CS/EngeRun&" or query "cmd=Stop&redir=EngeRun%26" or command  "Stop"

I grep for "EngeRun" and I do not see it anywhere in the midas sources. Can you grep for it
to see if it is coming from one of your pages?

If you want to start/stop runs from your custom page, look at start.html and transition.html - you will
need to make the run transition RPC calls (cut-and-paste the code to your page) and (obviously)
you will not have any redirects to some strange pages.

> For example, start calls:
> location.search = "cmd=Transition";
> whereas stop does:
> mhttpd_goto_page("Transition"); // DOES NOT RETURN

It's the same thing, look at mhttpd_goto_page().

> Can anyone offer any insights or advice? I can change the former to "cmd=Status", but 
> the latter doesn't allow it.

I am not sure what you are trying to do. If you need the "start" button on the status page
to do something different from what it does now, just hack status.html until it does so.
If you need some specific help with that, I am happy to help. I think I answered all questions
you asked so far.

K.O.

> online monitor would show (almost in real-time) the 
> gain, the dark noise, and the pedestal for all the channels, the 2D tracks 
> inside the detectors for each spill and so on.

Hmm... I now realize that the midas distribution does not include an example web page that 
integrates all these elements into one easy to understand html file.

I think an example page that would answer your questions and the questions from the other 
thread about starting/stopping runs, should include following elements:

- the general midas web page framework (the midas left-hand side menu, the top side 
status display, Stefan's new odb tags)
- buttons to start and stop runs (javascript code to call the run transition RPCs)
- embedded images for history display (old style gif and new style canvas)
- embedded images for ROOT histograms (via the ROOT http server and jsroot)
- code to live-update all these elements independantly from each other (to allow history 
plots and ROOT histograms to update at different frequencies).

As for the web page code for showing a mini-event-display, I think we do not know yet how 
to do - the event data lives inside the analyzer as C++ data structures, so somehow it 
needs to be encoded as json (this code is missing - but one can use the ROOT C++ to json 
encoder/streamer), needs to be transported to the web browser (we know how to do this) 
and at the end, plotting the json data on a canvas is the easy part.

I know some experiments have done all of this, and I think we should have such a pipeline 
available as part of the ROOTANA package. Maybe some day...

K.O.

Dear Thomas and Konstantin,

thank you very much for the feedback. I found the ROOTANA javascript display a good source of 
information and references.

As Thomas said, maybe the simplest thing would be to use the ROOT THttpServer. Honestly, I do 
not think that ROOT was ever meant to act as an online monitor due to its wacky memory 
management and abysmal multithread support. In other words, I think that by using ROOT we would 
inevitably lose some performance. 

Perhaps there are better ways of achieving the same goal. For example, I was leaning towards a 
plotly.js based approach where I would encode a series of vectors in base64 strings (for better 
transmission performance), send them to the client through the MJSONRPC mechanism, decode them 
and then feed them to plotly.js. But in this case, I should study many new libraries 
(plotly.js, the library for the base64 encoding, the Gaussian fitting, etc...) and I do not 
have the time to do that now: "beam is coming".

So ROOT it is. I will use the ROOTANA javascript display as a reference. Do you happen to know 
who wrote that part? 

In that example, you have some "static" histograms that you keep always in memory, while in our 
case the number of channels is so big that we have to dynamically generate the histograms only 
when needed (when the user select a single channel).

Best regards
Giorgio

> 
> As Thomas said, maybe the simplest thing would be to use the ROOT THttpServer. Honestly, I do 
> not think that ROOT was ever meant to act as an online monitor due to its wacky memory 
> management and abysmal multithread support. In other words, I think that by using ROOT we would 
> inevitably lose some performance.
>

Yes. The previous data analysis frameworks - PAW/PAW++/CERNlib (CERN), NOVA (TRIUMF) - certainly had
support for online monitoring. In CERNlib/PAW the histograms were stored in shared memory,
the analyzer running in the background was filling them, the PAW/PAW++ display was displaying
them "live". I was very surprised to find this function removed/not implemented in ROOT, given
that the same people were behind both projects (Rene Brun & co).

We tried to roll our own implementation of this in ROOTANA/ROODY, with mixed success.

I am glad the JSROOT project finally gained traction and web based "I can see the data" is now
available in ROOT.

> 
> Perhaps there are better ways of achieving the same goal. For example, I was leaning towards a 
> plotly.js based approach where 
>

There are many web/javascript graphics libraries out there, all have the weak spot - how do you
get your data into them?

Going forward, I see us standardizing on JSROOT: https://root.cern.ch/js/

>
> ... send them to the client through the MJSONRPC mechanism ...
>

I am not sure JSROOT have any support for interacting from the web page to the back-end analyzer. Perhaps
we can use the MIDAS MJSONRPC library for this. Hmm... (Note that the ROOT HTTP server is a derivative
of the mongoose web server library, which we use in mhttpd, so I already know how to work it)

>
> I would encode a series of vectors in base64 strings (for better transmission performance)
>

We looked into this when deciding on the data encoding for the midas history data. There is a tradeoff
between network use and cpu use - to save on the network, you try to reduce the data size by using
compressed binary data - to save on the CPU you try to minimize data encoding.

For history data, we gave up on binary json (extra decoding needed), gave up on text json (extra decoding 
needed), gave up on compression (extra cpu use for decompression) and use javascript native binary processing
("arraybuffer").

Our thinking is that network bandwidth is usually quite big and is getting bigger, but cpu resource is limited
and is expensive. (mobile devices seems to be stuck with ~2 GHz CPUs; cpu use means battery use and
battery capacity is limited, not improving quickly)

> 
> So ROOT it is. I will use the ROOTANA javascript display as a reference. Do you happen to know 
> who wrote that part? 
> 

Yes. See "Contact" at https://root.cern.ch/js/

>
> In that example, you have some "static" histograms that you keep always in memory, while in our 
> case the number of channels is so big that we have to dynamically generate the histograms only 
> when needed (when the user select a single channel).
> 

This requires interaction with the analyzer, requires some kind of RPC mechanism. I am now curious what jsroot 
have, also it would not be too hard to add the mjsonrpc library to rootana. Cooperation from ROOT multithreading
is not required: I can queue the RPC requests in a separate (thread safe, non-ROOT) buffer, then process
them in the ROOT main event loop (this is how the ROOTANA histogram server worked in the days when
ROOT had no multithread support at all).

K.O.

Hi,

I have been looking at the 2019 workshop slides, I am interested in the C++ future of MIDAS. 

I am quite interested in using the object oriented 


ALPHA will start data taking in 2021

I keep on getting messages like this:

16:25:35 [fecaen,ERROR] [odb.c:4567:db_get_data,ERROR] data for key
"/DAQ/params/VX1730/custom/Board 0/Channel 0/Input range" truncated

whenever I start my frontend. Input range is defined to be a BOOL and using
odbedit to read it shows:

Key name                        Type    #Val  Size  Last Opn Mode Value
---------------------------------------------------------------------------
Input range                     BOOL    1     4     75h  0   RWD  y

without any error message. The entry is read using

         size = sizeof(fInputRange);
         db_get_data(hDb, hSubKey, &fInputRange, &size, TID_BOOL);

where fInputRange is a bool.

Where does this message come from and how can I resolve this?

> I keep on getting messages like this:
> 16:25:35 [fecaen,ERROR] [odb.c:4567:db_get_data,ERROR] data for key
> "/DAQ/params/VX1730/custom/Board 0/Channel 0/Input range" truncated
>
>  [  bool fInputRange... ]
>          size = sizeof(fInputRange);
>          db_get_data(hDb, hSubKey, &fInputRange, &size, TID_BOOL);
>

The error is correct. size of TID_BOOL is 4 byte (uint32_t) and you give is sizeof(bool) instead which is probably not 4.

Note that sizeof(bool) is not well defined, sometimes it is 1 (you need 4), sometimes something else, see
https://stackoverflow.com/questions/4897844/is-sizeofbool-defined-in-the-c-language-standard

A good fix would be to change fInputRange from bool to uint32_t (which is always 4 byte size).

#include <stdint.h>
...
uint32_t fInputRange;

K.O.

Hello,

I am quite new in both Linux and MIDAS.
I have install MIDAS on my desktop by going through this link: 
https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux 

in the last step when I send "mhttpd" command and try to open the link 
https://localhost:8443 (of course, changing the localhost with my host name), it 
failed to connect and shows this error: SSL_ERROR_NO_CYPHER_OVERLAP (please see 
attached file includes a screenshot of the error).

I have tried many ways to solve this problem: In Firefox: going to option/privacy 
and security/ security and uncheck the option "Block dangerous and deceptive 
content". but it does not help.

Looking forward your help
Thanks
Mehran

> I am quite new in both Linux and MIDAS.
> I have install MIDAS on my desktop by going through this link: 
> https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux 
> 
> in the last step when I send "mhttpd" command and try to open the link 
> https://localhost:8443 (of course, changing the localhost with my host name), it 
> failed to connect and shows this error: SSL_ERROR_NO_CYPHER_OVERLAP (please see 
> attached file includes a screenshot of the error).

What Linux? (on most linuxes, run "lsb_release -a")
What version of midas? (run odbedit "ver" command)
What version of firefox? (from the "about firefox" menu)

> I have tried many ways to solve this problem: In Firefox: going to option/privacy 
> and security/ security and uncheck the option "Block dangerous and deceptive 
> content". but it does not help.

No you cannot fix it from inside firefox. The issue is that the overlap of encryption methods
supported by your firefox and by your openssl library (used by mhttpd) is an empty set.
No common language, so to say, communication is impossible.

So either you have a very old openssl but very new firefox, or a very new openssl but very old 
firefox. Both very old or both very new can talk to each other, difficulties start with greater  
difference in age, as new (better) encryption methods are added and old (no-longer-secure) 
methods are banished.

BTW, for good security we recommend using apache httpd as the https proxy (instead of built-in 
https support in mhttpd). (I am not sure what it says in the current documentation). (But apache 
httpd will use the same openssl library, so this may not solve your problem. Let's see what 
versions of software you are using, per questions above, first).

K.O.

Hi Konstantin,
Thanks for your reply, 

> What Linux? (on most linuxes, run "lsb_release -a")
> What version of midas? (run odbedit "ver" command)
I am using CentOS 8

> What version of firefox? (from the "about firefox" menu)
Firefox 71.0

Thanks 
Mehran

> No you cannot fix it from inside firefox. The issue is that the overlap of encryption methods
> supported by your firefox and by your openssl library (used by mhttpd) is an empty set.
> No common language, so to say, communication is impossible.
> 
> So either you have a very old openssl but very new firefox, or a very new openssl but very old 
> firefox. Both very old or both very new can talk to each other, difficulties start with greater  
> difference in age, as new (better) encryption methods are added and old (no-longer-secure) 
> methods are banished.
> 
> BTW, for good security we recommend using apache httpd as the https proxy (instead of built-in 
> https support in mhttpd). (I am not sure what it says in the current documentation). (But apache 
> httpd will use the same openssl library, so this may not solve your problem. Let's see what 
> versions of software you are using, per questions above, first).
> 
> K.O.

Hi, I have not run midas on Centos-8 yet. Maybe there is a problem with the openssl library there. The Centos-7 
instructions for setting up apache httpd proxy are here, with luck they work on centos-8:
https://daq.triumf.ca/DaqWiki/index.php/SLinstall#Configure_HTTPS_server_.28CentOS7.29

K.O.


> Hi Konstantin,
> Thanks for your reply, 
> 
> > What Linux? (on most linuxes, run "lsb_release -a")
> > What version of midas? (run odbedit "ver" command)
> I am using CentOS 8
> 
> > What version of firefox? (from the "about firefox" menu)
> Firefox 71.0
> 
> Thanks 
> Mehran
> 
> > No you cannot fix it from inside firefox. The issue is that the overlap of encryption methods
> > supported by your firefox and by your openssl library (used by mhttpd) is an empty set.
> > No common language, so to say, communication is impossible.
> > 
> > So either you have a very old openssl but very new firefox, or a very new openssl but very old 
> > firefox. Both very old or both very new can talk to each other, difficulties start with greater  
> > difference in age, as new (better) encryption methods are added and old (no-longer-secure) 
> > methods are banished.
> > 
> > BTW, for good security we recommend using apache httpd as the https proxy (instead of built-in 
> > https support in mhttpd). (I am not sure what it says in the current documentation). (But apache 
> > httpd will use the same openssl library, so this may not solve your problem. Let's see what 
> > versions of software you are using, per questions above, first).
> > 
> > K.O.

Hi,
As, the link suggests, I perform "yum install -y mod_ssl certwatch crypto-utils" but it complains as:
No match for argument: certwatch
No match for argument: crypto-utils

You may have a look on this link: https://blog.cloudware.bg/en/whats-new-in-centos-linux-8/
What’s gone?
In with the new, out with the old. CentOS 8 also says goodbye to some features. The OS removes several security functionalities. Among them is the Clevis HTTP pin, Coolkey and crypto-utils.
Cent OS 8 comes with securetty disabled by default. The configuration file is no longer included. You can add it back, but you will have to do it yourself. Another change is that shadow-utils no longer allow all-numeric user and group names.

Thanks
Mehran

> Hi, I have not run midas on Centos-8 yet. Maybe there is a problem with the openssl library there. The Centos-7 
> instructions for setting up apache httpd proxy are here, with luck they work on centos-8:
> https://daq.triumf.ca/DaqWiki/index.php/SLinstall#Configure_HTTPS_server_.28CentOS7.29
> 
> K.O.
> 

> I am using CentOS 8 [and]
> Firefox 71.0

I now have a centos-8 machine, I successfully built midas and I confirm that there is a problem.

But I get different errors from you:

- google chrome - does not connect at all (without any useful error message: "This site can’t be reached. The 
connection was reset.")
- firefox complains about the self-signed certificate, but connects ok, I see the midas status page and it works. "page 
info" reports connection is TLS 1.3, encryption TLS_AES_128_GCM_SHA256. However, the function "view certificate" 
does not work (without any useful error message).

I tried to run the SSLlabs tool to get some more information from mhttpd, but it does not want to run against mhttpd on 
port 8443... I do have a port redirect program somewhere... need to find it...

K.O.

> > The Centos-7 instructions for setting up apache httpd proxy are here, with luck they work on centos-8:
> > https://daq.triumf.ca/DaqWiki/index.php/SLinstall#Configure_HTTPS_server_.28CentOS7.29

I now have a centos-8 computer, I followed my instructions and they generally worked.

There is a number of problems with the certbot package that prevent me from writing coherent production quality instructions for centos-8.

But at the end I was successful, httpd runs, gets "A+" rating from SSLlabs, forwards the requests to mhttpd, I can access the midas status page etc.

With luck the certbot packages for centos-8 will be sorted out soon (the apache plugin seems to be missing, this causes the automatic
certificate renewal to not work) and I will update my instructions to include centos-8.

Until then, I recommend that people continue to use centos-7 or the current Ubuntu LTS release.

K.O.

While upgrading to the latest MIDAS version

MIDAS version: 2.1 GIT revision: Tue Dec 31 17:40:14 2019 +0100 - midas-2019-09-i-1-gd93944ce-dirty on branch develop

ODB version: 3

I encountered two issues using cmake

1. on machines with NVIDIA drivers:

(nvml.h doesn't seem to be part of the standard nvidia driver package.)

2. Complile including ROOT throws an error with ROOT 6.12/06 on Centos7 and with ROOT 6.18/04 on Fedora 31:

A workaround (which works for me) is to compile with

cmake .. -DNO_ROOT=1 -DNO_NVIDIA=1

MIDAS version:      2.1
GIT revision:       Tue Dec 31 17:40:14 2019 +0100 - midas-2019-09-i-1-gd93944ce-dirty on branch develop

/Logger/Channels/0/Settings

With the settings above the file last.json generated for a new run is empty and the events 0x8000 and 0x8001 are missing in the .mid file.

When setting "ODB dump format" to "xml", events 0x8000 and 0x8001 are included in the .mid file, however, the file last.xml is not created.

After upgrading to the lastes MIDAS version I got the DAQ frontend of my application running by changing all compiler directives from cc to g++ and using

#include "mfe.h"

extern HNDLE hDB

 extern "C" { 
 #include <CAENComm.h>  
 }

With these changes everything seems to work fine.

However, I'm having trouble with a slow control frontend using a tcpip driver. It compiled well with the older MIDAS version. Even though all the functions in question are defined in the frontend code, the following error comes up:

 I guess the the aforementioned DAQ frontend compiles because its equipment definitions don't call on the function `initialize_equipment()', but I can't figure out why it doesn't work. Help is appreciated. P.K.

(please post messages in "plain" mode, they are much easier to answer)

- nvidia problems - this code was contributed by Joseph (I think?), with luck he will look into 
this problem.

- ROOT problem - it looks like the error is thrown by the ROOT header files, has nothing to do 
with MIDAS?

So what ROOT are you using? I recommend installing ROOT by following instructions at 
root.cern.ch.

Perhaps you used the ROOT packages from the EPEL repository? I have seen trouble with 
those packages before (miscompiled; important optional features turned off; very old 
versions; etc).

P.S.

Historically, ROOT has caused so many reports of "cannot build midas" that I consistently 
vote to "remove ROOT support from MIDAS". But Stefan's code for writing MIDAS data into 
ROOT files is so neat, cannot throw it away. And some people do use it. So at the latest MIDAS 
bash this Summer we decided to keep it.

(Only build targets to use ROOT are the rmlogger executable and the rmana.o object file (and 
it's one-man-army library)).

But.

In the past, one could use "make -k" to get past the errors caused by ROOT, everything will 
get built and installed, except for the code that failed to build.

Now with cmake, it is "all or nothing", if there is any compilation error, nothing gets installed 
into the "bin" directory. So one must discover and use "NO_ROOT=1" (which becomes sticky 
until the next "make cclean". Some people are not used to sticky "make" options, I just got 
burned by this very thing last week).

Perhaps there is a way to tell cmake to ignore compile errors for rmlogger and rmana.


K.O.



<p>&nbsp;</p>

<table align="center" cellspacing="1" style="border:1px solid #486090; width:98%">
	<tbody>
		<tr>
			<td style="background-color:#486090">Peter Kunz wrote:</td>
		</tr>
		<tr>
			<td style="background-color:#FFFFB0">
			<p>While upgrading to the latest MIDAS version</p>

			<p>MIDAS version: 2.1 GIT revision: Tue Dec 31 17:40:14 2019 +0100 - midas-
2019-09-i-1-gd93944ce-dirty on branch develop</p>

			<p>ODB version: 3</p>

			<p>I encountered two issues using cmake</p>

			<p>1. on machines with NVIDIA drivers:</p>

			<div style="background:#eee;border:1px solid #ccc;padding:5px 
10px;">nvml.h: no such file or directory</div>

			<p>(nvml.h doesn&#39;t seem to be part of the standard nvidia driver 
package.)</p>

			<p>2. Complile including ROOT throws an error with ROOT 6.12/06 on Centos7 
and with ROOT 6.18/04 on Fedora 31:</p>

			<div style="background:#eee;border:1px solid #ccc;padding:5px 10px;">[ 29%] 
Building CXX object CMakeFiles/rmana.dir/src/mana.cxx.o In file included from 
/usr/include/root/TString.h:28, from /usr/include/root/TCollection.h:29, from 
/usr/include/root/TSeqCollection.h:25, from /usr/include/root/TList.h:25, from 
/usr/include/root/TQObject.h:40, from /usr/include/root/TApplication.h:30, from 
/home/pkunz/packages/midas/src/mana.cxx:60:</div>

			<div style="background:#eee;border:1px solid #ccc;padding:5px 
10px;">/usr/include/root/ROOT/RStringView.hxx:32:37: error: &lsquo;experimental&rsquo; in 
namespace &lsquo;std&rsquo; does not name a type 32 | using basic_string_view = 
::std::experimental::basic_string_view&lt;_CharT,_Traits&gt;;</div>

			<p>A workaround (which works for me) is to compile with</p>

			<p><tt>cmake .. -DNO_ROOT=1 -DNO_NVIDIA=1</tt></p>
			</td>
		</tr>
	</tbody>
</table>

<p>&nbsp;</p>