The following program handles regular directories in a file, or folders (ugh).
Most histograms are added bin by bin.

For scaler events it is convenient to see the counts as a function of time (ala
sclaer history plots in mhttpd).  If the histogram looks like a scaler plot versus
time, then new bins are added on to the end (or into the middle!) of the histogram.

All different versions of cuts are kept.

TTrees are not explicitly supported, so probably don't do the right thing...

John.

> Dear MIDAS users,
> 
> I want to add several .root-files produced by the MIDAS analyzer, in a fast 
> and convenient way. ROOT's hadd fails because it does not know how to treat 
> TFolders. I guess this problem is not unique to me, so I hope that somebody of 
> you might already have found a solution.
> 
> Why don't I just run "analyzer -r 1 10000"?
> We have taken lots of runs under (rapidly) varying conditions, so it would be 
> lots of "-r". And the analysis is quite involved, so rerunning all data takes 
> about one hour on a fast PC making this quite painful.
> Therefore, I would like to rerun all data only once, and then add the result 
> files depending on different criteria.
> 
> Of course, I tried to write a script that does the adding. But somehow it is 
> incredibly slow. And I am not the Master Of C++, too.
> 
> Is there any deeper reason for MIDAS using TFolders, not TDirectorys? ROOT's 
> hadd can treat TDirectory. Can I simply patch "my" MIDAS? Is there general 
> interest in a change like this? (Does anyone have experience with the speed of 
> hadd?)
> 
> Looking forward to comments from the Forum.
> 
> Cheers,
> 
> Randolf

some semi-random thoughts:

no templates strictly means you can't use std::string, std::vector etc.

printf is in any case part of C++ (#include <cstdio>), but std::ostreams can be faster (for std::cout, endl line causes buffer flushing, whereas "\n" does not flush the buffer but printf
always flushes the buffer), and formatting is possible (though very long winded).  printf does not allow to print things other than simple data, e.g. BANK_HEADER* bh; printf( "%?", *bh);

I've been writing all our DAQ code in C++ for a while now.

> > >
> > > With the removal of the requirement to make it possible to write MIDAS frontends in C, we can switch the MIDAS 
> > > default build to C++ and start using C++ features in the MIDAS API (std::string, std::vector, etc).
> > > 
> 
> C++ is a big animal. Obviously we want to use std::string, std::vector and similar improvements over plain C (we already use "//" for comments).
> 
> But in keeping with the Camel's nose fable (https://en.wikipedia.org/wiki/Camel%27s_nose), there are some parts of C++ we definitely do not want to use in MIDAS. Even the C++ FAQ talks 
> about "evil features", see https://isocpp.org/wiki/faq/big-picture#use-evil-things-sometimes
> 
> Here is my list of things to use and to avoid. Comments on this are very welcome - as everybody's experience with C++ is different (and everybody's experience is very valuable and very 
> welcome).
> 
> - std::string, std:vector, etc are in. I am already using them in the MIDAS API (midas.h)
> - extern "C" is out, everything has to be C++, will remove "extern "C"" from all midas header files.
> - exceptions are out, see https://stackoverflow.com/questions/1736146/why-is-exception-handling-bad
> - std::thread and std::mutex are in, at least for writing new frontends, but see discussion of "cannot use c++11". (maybe replace ss_mutex_xxx() with out own std::mutex look-alike).
> - heavy use of templates and heavy use of argument overloading is out - just by looking at the code, impossible to tell what function will be called
> - "auto" is on probation. I need to know if "auto v=f()" is an integer or a double when I write "auto w=v/2" or "auto w=v/2.0". see 
> https://softwareengineering.stackexchange.com/questions/180216/does-auto-make-c-code-harder-to-understand
> - unreadable gibberish is out (lambdas, etc)
> - C-style malloc()/free() is in. C++ new and delete are okey, but "delete[]" confuses me.
> - C-style printf() is in. C++ cout and "<<" gunk provide no way to easily format the output for easy reading.
> 
> K.O.

We recently had an instance where mlogger lost its connection to the database resulting in 317 write errors on mysql_insert and (Good) did report connection errors to the Midas log but (Bad) didn't attempt to restore the connection. With some help from Gemini, I've drafted a potential solution to reheal. Our copy uses mysql_query_debug line 1903 in our copy that inserts rows, creates tables, etc:

   /* execut sql query */
   status = mysql_query(db, query);

   if (status)
      cm_msg(MERROR, "mysql_query_debug", "SQL error: %s", mysql_error(db));

   return status;

to add a reconnection attempt like the following (untested) block:

    /* execute sql query */
    status = mysql_query(db, query);
    
    if (status != 0) {
        unsigned int err_num = mysql_errno(db);
        cm_msg(MERROR, "mysql_query_debug", "SQL query execution failed (Error %u: %s).", err_num, mysql_error(db));
        cm_msg(MTRCE, "mysql_query_debug", "Network drop suspected. Backing off 5 seconds to heal handle socket...");
        
        #ifdef OS_WIN64
        Sleep(5000);
        #else
        sleep(5);
        #endif

        cm_msg(MTRCE, "mysql_query_debug", "Initiating handle re-authentication to server...");

        // Setting MYSQL_OPT_PROTOCOL to MYSQL_PROTOCOL_TCP ensures that even when passing NULL for host configuration
        // fallbacks, the client library strictly forces a network TCP connection instead of a local Unix socket.
        unsigned int protocol = MYSQL_PROTOCOL_TCP;
        mysql_options(db, MYSQL_OPT_PROTOCOL, &protocol);

        if (!mysql_real_connect(db, NULL, NULL, NULL, NULL, 0, NULL, 0)) {
            cm_msg(MERROR, "mysql_query_debug", "Automatic handle reconciliation failed: %s. Logging halted.", mysql_error(db));
        } else {
            cm_msg(MINFO, "mysql_query_debug", "Database connection handle successfully restored.");
            
            // Retry the blocked execution string
            status = mysql_query(db, query);
            if (status != 0) {
                cm_msg(MERROR, "mysql_query_debug", "Query retry failed after reconnection (Error %u: %s).", mysql_errno(db), mysql_error(db));
            } else {
                cm_msg(MINFO, "mysql_query_debug", "SQL query successfully recovered and executed on retry.");
            }
        }
    }

   return status;

Hi All,

I'm testing MIDAS SVN rev-4113 on Scientific Linux 5.1 (i386) and the Analyzer 
can't start as Daemon. What I mean "can't" is that it stops running 
immediately without leaving any error messages. However, it can run offline or 
without becoming a Daemon. I have tested with ROOT 5.14e/5.16/5.18 and 
the "Experiment" example coming with MIDAS and this problem always happens. 
Any ideas?

Best Regards,
Jimmy

Hi All,

Is there any example code for using MIDAS with the CAEN VME-USB Bridge V1718? 
Thanks.

Regards,
Jimmy

Hi All,

I am testing the libraries provided by CAEN with the sample softwares in the 
bundle CD. The Windows sample program works fine, but I cannot get started with 
the Linux sample program. When I run CAENVMEDemo in Scientific Linux 5.1, it 
gives me a message "Error opening the device". I have followed the instructions 
in CAENVMElibReadme.txt: 
- compile and load the device driver v1718.ko
- install the library libCAENVME.so

Does anyone have any experience of using V1718 in Scientific Linux? Thanks.

Regards,
Jimmy


> Hi All,
> 
> Is there any example code for using MIDAS with the CAEN VME-USB Bridge V1718? 
> Thanks.
> 
> Regards,
> Jimmy

Dear All,

I have a problem when testing the V792N 16 CH QDC with the V1718 VME-USB 
Bridge on Scientific Linux 5.1 i386 (kernel 2.6.18-53.1.21.e15).

The problem is that the V792N does not response normally after a few minutes 
of continuous polling and readout of data. It seems like the V792N is hanged 
and a hardware reset of the VME system is required to bring it working again. 
If I do not poll for DREADY first and directly read the Output Buffer 
continuously, the system can work properly.

I have worked on this problem many days but I cannot find any clues to solve 
it. I have tried to use the CAENVMEDemo program (with some modifications) to 
do the same thing (polling and readout) and it works fine. CAEN technical 
support also doesn't know why the VME system is hanged. I think it might be a 
problem of MIDAS itself. I have tried with MIDAS revision 4132 and the trunk 
version, but the problem is still there. Is there any parameter in MIDAS 
(buffer size etc?) which may cause this problem? I have attached my frontend 
code and drivers for your reference.

Thank you for your kind attention.

Best Regards,
Jimmy

Dear All,

Is there any recommended platforms for running MIDAS? Have anyone encountered 
problems when running MIDAS on Scientific Linux?

Thanks.

Jimmy

Dear All,

We have a temperature/humidity sensor in MIDAS now and will add a liquid level 
sensor to MIDAS soon. We want the operators to get alerted ASAP when the 
laboratory environment or the liquid level reached some critical levels. Can 
MIDAS send email alerts or SMS alerts to cell phones when the alarms are 
triggered? If yes, how can I config it?

Many thanks!

Best Regards,
Jimmy

Dear All,

How can I add a button at the top of the "Status" webpage which will show a 
page similar to the "CNAF" one after I click on it? and how can I make a 
custom page similar to "CNAF" which allow me to call some custom funtions? I 
want to make a page which is particularly for doing calibration.

Thank you for your attention!

Best Regards,
Jimmy Ngai

Dear All,

I have created a custom page (please see the attachment) and imported into 
MIDAS with key name "Control panel&" (without the ""). I have the following 
two questions:

1) I display the status of the run with <odb src="/Runinfo/State">, but it 
returns numbers which is not user friendly. How can I make something 
like "Running" with green background and "Stopped" with red background in the 
default status page?

2) When I click either Start/Stop/Pause/Resume, it can performs the right 
things, but afterward it jumps to the page "http://domain.name:8081/CS/" which 
shows "Invalid custom page: NULL path". How can I make it returns to the 
correct page "http://domain.name:8081/CS/Control%20panel"?

Thank you for your attention.

Best Regards,
Jimmy

Dear All,

I have one more question. I use <odb src="odb field" edit=1> to display an 
editable ODB value, but how can I show this value in hexadecimal?

Thanks.

Best Regards,
Jimmy


> Dear All,
> 
> I have created a custom page (please see the attachment) and imported into 
> MIDAS with key name "Control panel&" (without the ""). I have the following 
> two questions:
> 
> 1) I display the status of the run with <odb src="/Runinfo/State">, but it 
> returns numbers which is not user friendly. How can I make something 
> like "Running" with green background and "Stopped" with red background in the 
> default status page?
> 
> 2) When I click either Start/Stop/Pause/Resume, it can performs the right 
> things, but afterward it jumps to the page "http://domain.name:8081/CS/" which 
> shows "Invalid custom page: NULL path". How can I make it returns to the 
> correct page "http://domain.name:8081/CS/Control%20panel"?
> 
> Thank you for your attention.
> 
> Best Regards,
> Jimmy

Dear All,

Can one set a time limit for each run? I can only find event limit in ODB. 
Thanks.

Jimmy

Dear All,

I'm using SL5 and MIDAS rev 4528. Occasionally, when I stop a run in odbedit, 
a timeout would occur: 
[midas.c:9496:rpc_client_call,ERROR] rpc timeout after 121 sec, routine 
= "rc_transition", host = "computerB", connection closed
Error: Unknown error 504 from client 'Frontend' on host computerB

This error seems to be random without any reason or pattern. After this error 
occurs, I cannot start or stop any run. Sometime restarting MIDAS can bring 
the system working again, but sometime not.

Another transition timeout occurs after I change any ODB value using the web 
interface:
[midas.c:8291:rpc_client_connect,ERROR] timeout on receive remote computer 
info: 
[midas.c:3642:cm_transition,ERROR] cannot connect to client "Frontend" on host 
computerB, port 36255, status 503
Error: Cannot connect to client 'Frontend'

This error is reproducible: start run -> change ODB value within webpage -> 
stop run -> timeout!

Any idea?

Thanks,
Jimmy

Hi All,

I set "/Analyzer/ODB Load" to true and analyzed a run in offline mode. After
that, I found the start time and stop time in /RunInfo did not reflect the
correct time as in online. How do I retrieve the correct start/stop time from
the ODB in offline mode?

Thanks!

Jimmy

Dear All,

I want to run two frontend programs (one for trigger and one for slow control)
concurrently on the same computer, but I failed. The second frontend said: 

Semaphore already present
 There is another process using the semaphore.
 Or a process using the semaphore exited abnormally.
 In That case try to manually release the semaphore with:
   ipcrm sem XXX.

The two frontends are connected to the same experiment. Is there any way I can
overcome this problem?

Thanks!

Jimmy

Dear Stefan,

Thanks for the reply. I have tried your patch and it didn't solve my problem. Maybe I 
have not written my question clearly. The two frontends could run on the same computer 
if I use the remote method, i.e. by setting up the mserver and connect to the 
experiment by specifying "-h localhost", also the frontend programs need to be put in 
different directory. What I want to know is whether I can simply start multiple 
frontends in the same directory without setting up the mserver etc. I noticed that 
there are several *.SHM files, I'm not familiar with semaphore, but I guess they are 
the key to the problem. Please correct me if I misunderstood something.

Best Regards,
Jimmy


> > Dear All,
> > 
> > I want to run two frontend programs (one for trigger and one for slow control)
> > concurrently on the same computer, but I failed. The second frontend said: 
> > 
> > Semaphore already present
> >  There is another process using the semaphore.
> >  Or a process using the semaphore exited abnormally.
> >  In That case try to manually release the semaphore with:
> >    ipcrm sem XXX.
> > 
> > The two frontends are connected to the same experiment. Is there any way I can
> > overcome this problem?
> 
> That might be related to the RPC mutex, which gets created system wide now. I 
> modified this in midas.c rev. 4628, so there will be one mutex per process. Can you 
> try that temporary patch and tell me if it works for you?

Dear Stefan,

I followed your suggestion to try the sample front-ends from the distribution and 
they work fine. They also work fine with any one of my front-ends. Only my two 
front-ends cannot run concurrently in the same directory. I later found that the 
problem is in the CAEN HV wrapper library. The problem arises when the front-ends 
are both linked to that library and it is solved now.

Thanks & Best Regards,
Jimmy


> Hi Jimmy,
> 
> ok, now I understand. Well, I don't see your problem. I just tried with the 
> current SVN 
> version to start
> 
> midas/examples/experiment/frontend
> midas/examples/slowcont/scfe
> 
> in the same directory (without "-h localhost") and it works just fine (see 
> attachemnt). I even started them from the same directory. Yes there are *.SHM 
> files and they correspond to shared memory, but both front-ends use this shared 
> memory together (that's why it's called 'shared').
> 
> Your error message 'Semaphore already present' is strange. The string is not 
> contained in any midas program, so it must come from somewhere else. Do you 
> maybe try to access the same hardware with the two front-end programs?
> 
> I would propose you do the following: Use the two front-ends from the 
> distribution (see above). They do not access any hardware. See if you can run 
> them with the current SVN version of midas. If not, report back to me.
> 
> Best regards,
> 
>   Stefan
> 
> 
> > Dear Stefan,
> > 
> > Thanks for the reply. I have tried your patch and it didn't solve my problem. 
> Maybe I 
> > have not written my question clearly. The two frontends could run on the same 
> computer 
> > if I use the remote method, i.e. by setting up the mserver and connect to the 
> > experiment by specifying "-h localhost", also the frontend programs need to be 
> put in 
> > different directory. What I want to know is whether I can simply start 
> multiple 
> > frontends in the same directory without setting up the mserver etc. I noticed 
> that 
> > there are several *.SHM files, I'm not familiar with semaphore, but I guess 
> they are 
> > the key to the problem. Please correct me if I misunderstood something.
> > 
> > Best Regards,
> > Jimmy
> > 
> > 
> > > > Dear All,
> > > > 
> > > > I want to run two frontend programs (one for trigger and one for slow 
> control)
> > > > concurrently on the same computer, but I failed. The second frontend said: 
> > > > 
> > > > Semaphore already present
> > > >  There is another process using the semaphore.
> > > >  Or a process using the semaphore exited abnormally.
> > > >  In That case try to manually release the semaphore with:
> > > >    ipcrm sem XXX.
> > > > 
> > > > The two frontends are connected to the same experiment. Is there any way I 
> can
> > > > overcome this problem?
> > > 
> > > That might be related to the RPC mutex, which gets created system wide now. 
> I 
> > > modified this in midas.c rev. 4628, so there will be one mutex per process. 
> Can you 
> > > try that temporary patch and tell me if it works for you?

Hi Exaos,

This may help: https://ladd00.triumf.ca/elog/Midas/645

You need to set the following keys:
/Logger/Run duration
/Logger/Auto restart
/Logger/Auto restart delay

Regards,
Jimmy


> I want to let the run stop and start periodically. But I looked through the ODB
> and didn't find anything may help. I also checked the FAQ online and didn't find
> answer either. Who can help me? Thank you.

Dear All,

After clicking the "Start" button, there is a page for the operator to change some 
ODB values. I have created "/Experiment/Edit on start" and added some links there. 
If the link is pointed to a boolean type key, a check box will appear in the 
"Start" page, which is great. But how about if I want to have some radio buttons 
or pull-down menus for the operator to select among different calibration sources 
or running modes?

Thanks,

Jimmy