ELOG Midas

> online monitor would show (almost in real-time) the > gain, the dark noise, and the pedestal for all the channels, the 2D tracks > inside the detectors for each spill and so on. Hmm... I now realize that the midas distribution does not include an example web page that integrates all these elements into one easy to understand html file. I think an example page that would answer your questions and the questions from the other thread about starting/stopping runs, should include following elements: - the general midas web page framework (the midas left-hand side menu, the top side status display, Stefan's new odb tags) - buttons to start and stop runs (javascript code to call the run transition RPCs) - embedded images for history display (old style gif and new style canvas) - embedded images for ROOT histograms (via the ROOT http server and jsroot) - code to live-update all these elements independantly from each other (to allow history plots and ROOT histograms to update at different frequencies). As for the web page code for showing a mini-event-display, I think we do not know yet how to do - the event data lives inside the analyzer as C++ data structures, so somehow it needs to be encoded as json (this code is missing - but one can use the ROOT C++ to json encoder/streamer), needs to be transported to the web browser (we know how to do this) and at the end, plotting the json data on a canvas is the easy part. I know some experiments have done all of this, and I think we should have such a pipeline available as part of the ROOTANA package. Maybe some day... K.O.

Dear Thomas and Konstantin, thank you very much for the feedback. I found the ROOTANA javascript display a good source of information and references. As Thomas said, maybe the simplest thing would be to use the ROOT THttpServer. Honestly, I do not think that ROOT was ever meant to act as an online monitor due to its wacky memory management and abysmal multithread support. In other words, I think that by using ROOT we would inevitably lose some performance. Perhaps there are better ways of achieving the same goal. For example, I was leaning towards a plotly.js based approach where I would encode a series of vectors in base64 strings (for better transmission performance), send them to the client through the MJSONRPC mechanism, decode them and then feed them to plotly.js. But in this case, I should study many new libraries (plotly.js, the library for the base64 encoding, the Gaussian fitting, etc...) and I do not have the time to do that now: "beam is coming". So ROOT it is. I will use the ROOTANA javascript display as a reference. Do you happen to know who wrote that part? In that example, you have some "static" histograms that you keep always in memory, while in our case the number of channels is so big that we have to dynamically generate the histograms only when needed (when the user select a single channel). Best regards Giorgio

> > As Thomas said, maybe the simplest thing would be to use the ROOT THttpServer. Honestly, I do > not think that ROOT was ever meant to act as an online monitor due to its wacky memory > management and abysmal multithread support. In other words, I think that by using ROOT we would > inevitably lose some performance. > Yes. The previous data analysis frameworks - PAW/PAW++/CERNlib (CERN), NOVA (TRIUMF) - certainly had support for online monitoring. In CERNlib/PAW the histograms were stored in shared memory, the analyzer running in the background was filling them, the PAW/PAW++ display was displaying them "live". I was very surprised to find this function removed/not implemented in ROOT, given that the same people were behind both projects (Rene Brun & co). We tried to roll our own implementation of this in ROOTANA/ROODY, with mixed success. I am glad the JSROOT project finally gained traction and web based "I can see the data" is now available in ROOT. > > Perhaps there are better ways of achieving the same goal. For example, I was leaning towards a > plotly.js based approach where > There are many web/javascript graphics libraries out there, all have the weak spot - how do you get your data into them? Going forward, I see us standardizing on JSROOT: https://root.cern.ch/js/ > > ... send them to the client through the MJSONRPC mechanism ... > I am not sure JSROOT have any support for interacting from the web page to the back-end analyzer. Perhaps we can use the MIDAS MJSONRPC library for this. Hmm... (Note that the ROOT HTTP server is a derivative of the mongoose web server library, which we use in mhttpd, so I already know how to work it) > > I would encode a series of vectors in base64 strings (for better transmission performance) > We looked into this when deciding on the data encoding for the midas history data. There is a tradeoff between network use and cpu use - to save on the network, you try to reduce the data size by using compressed binary data - to save on the CPU you try to minimize data encoding. For history data, we gave up on binary json (extra decoding needed), gave up on text json (extra decoding needed), gave up on compression (extra cpu use for decompression) and use javascript native binary processing ("arraybuffer"). Our thinking is that network bandwidth is usually quite big and is getting bigger, but cpu resource is limited and is expensive. (mobile devices seems to be stuck with ~2 GHz CPUs; cpu use means battery use and battery capacity is limited, not improving quickly) > > So ROOT it is. I will use the ROOTANA javascript display as a reference. Do you happen to know > who wrote that part? > Yes. See "Contact" at https://root.cern.ch/js/ > > In that example, you have some "static" histograms that you keep always in memory, while in our > case the number of channels is so big that we have to dynamically generate the histograms only > when needed (when the user select a single channel). > This requires interaction with the analyzer, requires some kind of RPC mechanism. I am now curious what jsroot have, also it would not be too hard to add the mjsonrpc library to rootana. Cooperation from ROOT multithreading is not required: I can queue the RPC requests in a separate (thread safe, non-ROOT) buffer, then process them in the ROOT main event loop (this is how the ROOTANA histogram server worked in the days when ROOT had no multithread support at all). K.O.

I keep on getting messages like this: 16:25:35 [fecaen,ERROR] [odb.c:4567:db_get_data,ERROR] data for key "/DAQ/params/VX1730/custom/Board 0/Channel 0/Input range" truncated whenever I start my frontend. Input range is defined to be a BOOL and using odbedit to read it shows: Key name Type #Val Size Last Opn Mode Value --------------------------------------------------------------------------- Input range BOOL 1 4 75h 0 RWD y without any error message. The entry is read using size = sizeof(fInputRange); db_get_data(hDb, hSubKey, &fInputRange, &size, TID_BOOL); where fInputRange is a bool. Where does this message come from and how can I resolve this?

> I keep on getting messages like this: > 16:25:35 [fecaen,ERROR] [odb.c:4567:db_get_data,ERROR] data for key > "/DAQ/params/VX1730/custom/Board 0/Channel 0/Input range" truncated > > [ bool fInputRange... ] > size = sizeof(fInputRange); > db_get_data(hDb, hSubKey, &fInputRange, &size, TID_BOOL); > The error is correct. size of TID_BOOL is 4 byte (uint32_t) and you give is sizeof(bool) instead which is probably not 4. Note that sizeof(bool) is not well defined, sometimes it is 1 (you need 4), sometimes something else, see https://stackoverflow.com/questions/4897844/is-sizeofbool-defined-in-the-c-language-standard A good fix would be to change fInputRange from bool to uint32_t (which is always 4 byte size). #include <stdint.h> ... uint32_t fInputRange; K.O.

Hello, I am quite new in both Linux and MIDAS. I have install MIDAS on my desktop by going through this link: https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux in the last step when I send "mhttpd" command and try to open the link https://localhost:8443 (of course, changing the localhost with my host name), it failed to connect and shows this error: SSL_ERROR_NO_CYPHER_OVERLAP (please see attached file includes a screenshot of the error). I have tried many ways to solve this problem: In Firefox: going to option/privacy and security/ security and uncheck the option "Block dangerous and deceptive content". but it does not help. Looking forward your help Thanks Mehran

> I am quite new in both Linux and MIDAS. > I have install MIDAS on my desktop by going through this link: > https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux > > in the last step when I send "mhttpd" command and try to open the link > https://localhost:8443 (of course, changing the localhost with my host name), it > failed to connect and shows this error: SSL_ERROR_NO_CYPHER_OVERLAP (please see > attached file includes a screenshot of the error). What Linux? (on most linuxes, run "lsb_release -a") What version of midas? (run odbedit "ver" command) What version of firefox? (from the "about firefox" menu) > I have tried many ways to solve this problem: In Firefox: going to option/privacy > and security/ security and uncheck the option "Block dangerous and deceptive > content". but it does not help. No you cannot fix it from inside firefox. The issue is that the overlap of encryption methods supported by your firefox and by your openssl library (used by mhttpd) is an empty set. No common language, so to say, communication is impossible. So either you have a very old openssl but very new firefox, or a very new openssl but very old firefox. Both very old or both very new can talk to each other, difficulties start with greater difference in age, as new (better) encryption methods are added and old (no-longer-secure) methods are banished. BTW, for good security we recommend using apache httpd as the https proxy (instead of built-in https support in mhttpd). (I am not sure what it says in the current documentation). (But apache httpd will use the same openssl library, so this may not solve your problem. Let's see what versions of software you are using, per questions above, first). K.O.

Hi Konstantin, Thanks for your reply, > What Linux? (on most linuxes, run "lsb_release -a") > What version of midas? (run odbedit "ver" command) I am using CentOS 8 > What version of firefox? (from the "about firefox" menu) Firefox 71.0 Thanks Mehran > No you cannot fix it from inside firefox. The issue is that the overlap of encryption methods > supported by your firefox and by your openssl library (used by mhttpd) is an empty set. > No common language, so to say, communication is impossible. > > So either you have a very old openssl but very new firefox, or a very new openssl but very old > firefox. Both very old or both very new can talk to each other, difficulties start with greater > difference in age, as new (better) encryption methods are added and old (no-longer-secure) > methods are banished. > > BTW, for good security we recommend using apache httpd as the https proxy (instead of built-in > https support in mhttpd). (I am not sure what it says in the current documentation). (But apache > httpd will use the same openssl library, so this may not solve your problem. Let's see what > versions of software you are using, per questions above, first). > > K.O.

Hi, I have not run midas on Centos-8 yet. Maybe there is a problem with the openssl library there. The Centos-7 instructions for setting up apache httpd proxy are here, with luck they work on centos-8: https://daq.triumf.ca/DaqWiki/index.php/SLinstall#Configure_HTTPS_server_.28CentOS7.29 K.O. > Hi Konstantin, > Thanks for your reply, > > > What Linux? (on most linuxes, run "lsb_release -a") > > What version of midas? (run odbedit "ver" command) > I am using CentOS 8 > > > What version of firefox? (from the "about firefox" menu) > Firefox 71.0 > > Thanks > Mehran > > > No you cannot fix it from inside firefox. The issue is that the overlap of encryption methods > > supported by your firefox and by your openssl library (used by mhttpd) is an empty set. > > No common language, so to say, communication is impossible. > > > > So either you have a very old openssl but very new firefox, or a very new openssl but very old > > firefox. Both very old or both very new can talk to each other, difficulties start with greater > > difference in age, as new (better) encryption methods are added and old (no-longer-secure) > > methods are banished. > > > > BTW, for good security we recommend using apache httpd as the https proxy (instead of built-in > > https support in mhttpd). (I am not sure what it says in the current documentation). (But apache > > httpd will use the same openssl library, so this may not solve your problem. Let's see what > > versions of software you are using, per questions above, first). > > > > K.O.

Hi, As, the link suggests, I perform "yum install -y mod_ssl certwatch crypto-utils" but it complains as: No match for argument: certwatch No match for argument: crypto-utils You may have a look on this link: https://blog.cloudware.bg/en/whats-new-in-centos-linux-8/ What�s gone? In with the new, out with the old. CentOS 8 also says goodbye to some features. The OS removes several security functionalities. Among them is the Clevis HTTP pin, Coolkey and crypto-utils. Cent OS 8 comes with securetty disabled by default. The configuration file is no longer included. You can add it back, but you will have to do it yourself. Another change is that shadow-utils no longer allow all-numeric user and group names. Thanks Mehran > Hi, I have not run midas on Centos-8 yet. Maybe there is a problem with the openssl library there. The Centos-7 > instructions for setting up apache httpd proxy are here, with luck they work on centos-8: > https://daq.triumf.ca/DaqWiki/index.php/SLinstall#Configure_HTTPS_server_.28CentOS7.29 > > K.O. >

> I am using CentOS 8 [and] > Firefox 71.0 I now have a centos-8 machine, I successfully built midas and I confirm that there is a problem. But I get different errors from you: - google chrome - does not connect at all (without any useful error message: "This site can�t be reached. The connection was reset.") - firefox complains about the self-signed certificate, but connects ok, I see the midas status page and it works. "page info" reports connection is TLS 1.3, encryption TLS_AES_128_GCM_SHA256. However, the function "view certificate" does not work (without any useful error message). I tried to run the SSLlabs tool to get some more information from mhttpd, but it does not want to run against mhttpd on port 8443... I do have a port redirect program somewhere... need to find it... K.O.

> > The Centos-7 instructions for setting up apache httpd proxy are here, with luck they work on centos-8: > > https://daq.triumf.ca/DaqWiki/index.php/SLinstall#Configure_HTTPS_server_.28CentOS7.29 I now have a centos-8 computer, I followed my instructions and they generally worked. There is a number of problems with the certbot package that prevent me from writing coherent production quality instructions for centos-8. But at the end I was successful, httpd runs, gets "A+" rating from SSLlabs, forwards the requests to mhttpd, I can access the midas status page etc. With luck the certbot packages for centos-8 will be sorted out soon (the apache plugin seems to be missing, this causes the automatic certificate renewal to not work) and I will update my instructions to include centos-8. Until then, I recommend that people continue to use centos-7 or the current Ubuntu LTS release. K.O.

After upgrading to the lastes MIDAS version I got the DAQ frontend of my application running by changing all compiler directives from cc to g++ and using

However, I'm having trouble with a slow control frontend using a tcpip driver. It compiled well with the older MIDAS version. Even though all the functions in question are defined in the frontend code, the following error comes up:

g++ -o feMotor -DOS_LINUX -Dextname -g -O2 -fPIC -Wall -Wuninitialized -fpermissive -I/home/pkunz/packages/midas/include -I. -I/home/pkunz/packages/midas/drivers/bus /home/pkunz/packages/midas/lib/libmidas.a feMotor.o /home/pkunz/packages/midas/drivers/bus/tcpip.o cd_Galil.o /home/pkunz/packages/midas/lib/libmidas.a /home/pkunz/packages/midas/lib/mfe.o -lm -lz -lutil -lnsl -lpthread -lrt
/usr/bin/ld: /home/pkunz/packages/midas/lib/mfe.o: in function `initialize_equipment()':
/home/pkunz/packages/midas/src/mfe.cxx:687: undefined reference to `interrupt_configure(int, int, long)'
/usr/bin/ld: /home/pkunz/packages/midas/lib/mfe.o: in function `readout_enable(unsigned int)':
/home/pkunz/packages/midas/src/mfe.cxx:1236: undefined reference to `interrupt_configure(int, int, long)'
/usr/bin/ld: /home/pkunz/packages/midas/src/mfe.cxx:1238: undefined reference to `interrupt_configure(int, int, long)'
/usr/bin/ld: /home/pkunz/packages/midas/lib/mfe.o: in function `main':
/home/pkunz/packages/midas/src/mfe.cxx:2791: undefined reference to `interrupt_configure(int, int, long)'
/usr/bin/ld: /home/pkunz/packages/midas/src/mfe.cxx:2792: undefined reference to `interrupt_configure(int, int, long)'
collect2: error: ld returned 1 exit status
make: *** [Makefile:36: feMotor] Error 1

I guess the the aforementioned DAQ frontend compiles because its equipment definitions don't call on the function `initialize_equipment()', but I can't figure out why it doesn't work. Help is appreciated. P.K.

(please post messages in "plain" mode, they are much easier to answer) - nvidia problems - this code was contributed by Joseph (I think?), with luck he will look into this problem. - ROOT problem - it looks like the error is thrown by the ROOT header files, has nothing to do with MIDAS? So what ROOT are you using? I recommend installing ROOT by following instructions at root.cern.ch. Perhaps you used the ROOT packages from the EPEL repository? I have seen trouble with those packages before (miscompiled; important optional features turned off; very old versions; etc). P.S. Historically, ROOT has caused so many reports of "cannot build midas" that I consistently vote to "remove ROOT support from MIDAS". But Stefan's code for writing MIDAS data into ROOT files is so neat, cannot throw it away. And some people do use it. So at the latest MIDAS bash this Summer we decided to keep it. (Only build targets to use ROOT are the rmlogger executable and the rmana.o object file (and it's one-man-army library)). But. In the past, one could use "make -k" to get past the errors caused by ROOT, everything will get built and installed, except for the code that failed to build. Now with cmake, it is "all or nothing", if there is any compilation error, nothing gets installed into the "bin" directory. So one must discover and use "NO_ROOT=1" (which becomes sticky until the next "make cclean". Some people are not used to sticky "make" options, I just got burned by this very thing last week). Perhaps there is a way to tell cmake to ignore compile errors for rmlogger and rmana. K.O. <p> </p> <table align="center" cellspacing="1" style="border:1px solid #486090; width:98%"> <tbody> <tr> <td style="background-color:#486090">Peter Kunz wrote:</td> </tr> <tr> <td style="background-color:#FFFFB0"> <p>While upgrading to the latest MIDAS version</p> <p>MIDAS version: 2.1 GIT revision: Tue Dec 31 17:40:14 2019 +0100 - midas- 2019-09-i-1-gd93944ce-dirty on branch develop</p> <p>ODB version: 3</p> <p>I encountered two issues using cmake</p> <p>1. on machines with NVIDIA drivers:</p> <div style="background:#eee;border:1px solid #ccc;padding:5px 10px;">nvml.h: no such file or directory</div> <p>(nvml.h doesn't seem to be part of the standard nvidia driver package.)</p> <p>2. Complile including ROOT throws an error with ROOT 6.12/06 on Centos7 and with ROOT 6.18/04 on Fedora 31:</p> <div style="background:#eee;border:1px solid #ccc;padding:5px 10px;">[ 29%] Building CXX object CMakeFiles/rmana.dir/src/mana.cxx.o In file included from /usr/include/root/TString.h:28, from /usr/include/root/TCollection.h:29, from /usr/include/root/TSeqCollection.h:25, from /usr/include/root/TList.h:25, from /usr/include/root/TQObject.h:40, from /usr/include/root/TApplication.h:30, from /home/pkunz/packages/midas/src/mana.cxx:60:</div> <div style="background:#eee;border:1px solid #ccc;padding:5px 10px;">/usr/include/root/ROOT/RStringView.hxx:32:37: error: ‘experimental’ in namespace ‘std’ does not name a type 32 | using basic_string_view = ::std::experimental::basic_string_view<_CharT,_Traits>;</div> <p>A workaround (which works for me) is to compile with</p> <p><tt>cmake .. -DNO_ROOT=1 -DNO_NVIDIA=1</tt></p> </td> </tr> </tbody> </table> <p> </p>

(Please post messages in "plain" mode, they are much easier to answer) Thank you for reporting this problem. I will try to reproduce it. In addition, I will say a few words about your version of midas: > GIT revision: midas-2019-09-i-1-gd93944ce-dirty on branch develop I recommend that for production systems one used the tagged release versions of midas. (i.e. see https://midas.triumf.ca/elog/Midas/1750). (Your midas is "1 commit after the latest tag" - the "-1" in the git revision). I apply bug fixes to both the release branch and the develop branch, but for you to get these fixes, on the develop branch you will also "get" all the unrelated changes that may come with new bugs. On the release branch, you will only get the bug fixes. In your midas version it says "-dirty" which means that you have local modifications to the midas sources. With luck those changes are not related to the bug that you see. (but I cannot tell). You can do "git status" and "git diff" to see what the local changes are. It is much better if bugs are reported against "clean" builds of MIDAS (no "-dirty"). K.O. <p> </p> <table align="center" cellspacing="1" style="border:1px solid #486090; width:98%"> <tbody> <tr> <td style="background-color:#486090">Peter Kunz wrote:</td> </tr> <tr> <td style="background-color:#FFFFB0"> <p>MIDAS version:      2.1<br /> GIT revision:       Tue Dec 31 17:40:14 2019 +0100 - midas-2019-09-i-1-gd93944ce-dirty on branch develop</p> <p>/Logger/Channels/0/Settings</p> <table border="3" cellpadding="1" class="dialogTable"> <tbody> <tr> <td>ODB dump</td> <td>y</td> </tr> <tr> <td>ODB dump format</td> <td>json</td> </tr> </tbody> </table> <p>With the settings above the file last.json generated for a new run is empty and the events 0x8000 and 0x8001 are missing in the .mid file.</p> <p>When setting "ODB dump format" to "xml", events 0x8000 and 0x8001 are included in the .mid file, however, the file last.xml is not created. </p> <p> </p> <p> </p> <p> </p> </td> </tr> </tbody> </table> <p> </p>

(please use the "plain" text, much easier to answer). Hi, Peter, I think you misread the error message. There is no error about initialize_equipment(), the error is about interrupt_configure(). initialize_equipment() is just one of the functions that calls it. The cause of the error most likely is a mismatch between the declaration of interrupt_configure() in mfe.h and the definition of this function in your program (in feMotor.c, I guess). Sometimes this mismatch is hard to identify just by looking at the code. One fool-proof method to debug this is to extract the actual function prototypes from your object files, both the declaration ("U") in mfe.o and the definition ("T") in your program should be identical: nm feMotor.o | grep -i interrupt | c++filt 0000000000000f90 T interrupt_configure(int, int, long) <--- should be this nm ~/packages/midas/lib/libmfe.a | grep -i interrupt | c++filt U interrupt_configure(int, int, long) If they are different, you adjust your program until they match. One way to ensure the match is to copy the declaration from mfe.h into your program. K.O. <p> </p> <table align="center" cellspacing="1" style="border:1px solid #486090; width:98%"> <tbody> <tr> <td style="background-color:#486090">Peter Kunz wrote:</td> </tr> <tr> <td style="background-color:#FFFFB0"> <p>After upgrading to the lastes MIDAS version I got the DAQ frontend of my application running by changing all compiler directives from cc to g++ and using</p> <p>#include "mfe.h"</p> <p>extern HNDLE hDB</p> <p> extern "C" { <br />  #include <CAENComm.h>  <br />  }</p> <p>With these changes everything seems to work fine.</p> <p>However, I'm having trouble with a slow control frontend using a tcpip driver. It compiled well with the older MIDAS version. Even though all the functions in question are defined in the frontend code, the following error comes up:</p> <div style="background:#eee;border:1px solid #ccc;padding:5px 10px;">g++ -o feMotor -DOS_LINUX -Dextname -g -O2 -fPIC -Wall -Wuninitialized  -fpermissive - I/home/pkunz/packages/midas/include -I. -I/home/pkunz/packages/midas/drivers/bus   /home/pkunz/packages/midas/lib/libmidas.a feMotor.o /home/pkunz/packages/midas/drivers/bus/tcpip.o cd_Galil.o /home/pkunz/packages/midas/lib/libmidas.a /home/pkunz/packages/midas/lib/mfe.o -lm -lz -lutil -lnsl - lpthread -lrt<br /> /usr/bin/ld: /home/pkunz/packages/midas/lib/mfe.o: in function `initialize_equipment()':<br /> /home/pkunz/packages/midas/src/mfe.cxx:687: undefined reference to `interrupt_configure(int, int, long)'<br /> /usr/bin/ld: /home/pkunz/packages/midas/lib/mfe.o: in function `readout_enable(unsigned int)':<br /> /home/pkunz/packages/midas/src/mfe.cxx:1236: undefined reference to `interrupt_configure(int, int, long)'<br /> /usr/bin/ld: /home/pkunz/packages/midas/src/mfe.cxx:1238: undefined reference to `interrupt_configure(int, int, long)'<br /> /usr/bin/ld: /home/pkunz/packages/midas/lib/mfe.o: in function `main':<br /> /home/pkunz/packages/midas/src/mfe.cxx:2791: undefined reference to `interrupt_configure(int, int, long)'<br /> /usr/bin/ld: /home/pkunz/packages/midas/src/mfe.cxx:2792: undefined reference to `interrupt_configure(int, int, long)'<br /> collect2: error: ld returned 1 exit status<br /> make: *** [Makefile:36: feMotor] Error 1</div> <p> I guess the the aforementioned DAQ frontend compiles because its equipment definitions don't call on the function `initialize_equipment()', but I can't figure out why it doesn't work. Help is appreciated. P.K.</p> </td> </tr> </tbody> </table> <p> </p>