Hi all,

Having different histogram class templates to make processing easier.

My idea here was to create a parser which can take json input and creates automated default histograms from this. So here are a toy example how this could look like:

{
    "default1D": {
        "title": "bla",
        "x": "bla",
        "y": "#",
        "minX": "0",
        "maxX": 128,
        "nBins": 128,
    },
    "default2D": {
        "title": "bla",
        "x": "bla",
        "y": "#",
        "minX": "0",
        "maxX": 128,
        "nBinsX": 128,
        "minY": "0",
        "maxY": 128,
        "nBinsY": 128,
    },
    "detectorX": {
        "1DHistos":
            [
                {
                    "name": "FPGAID",
                    "type": "count",
                    "y": "#",
                    "minX": 0,
                    "maxX": 12,
                    "nBins": 12,
                    "value": "fpgaID",
                    "startChipID": 0,
                    "endChipID": 128,
                    "condition": "ROOT style: tree->Draw("X","X>10","");"
                },
                {
                    "name": "timeDiff",
                    "type": "diff",
                    "y": "#",
                    "xaxis": "col",
                    "minX": -256,
                    "maxX": 256,
                    "nBins": 512,
                    "value": "chipID",
                    "startChipID": 0,
                    "endChipID": 128,
                    "condition": "ROOT style: tree->Draw("X","X>10","");"
                },
           ]
....
}

Best,
Marius

Hi all,

again an idea from the mail conversation:

Mail me: Having an „easy way“ to explore the data - maybe some simple Python bindings. Especially when the data protocol is not fully settled in the experiment it’s maybe nice to have a simple way of plotting histograms. But also in general more and more people go to Python.

Mail Konstantin: I am not sure how python can come into this. manalyzer is a C++ analysis framework for experiments with needs beyound "toy" analyzers written in scripting languages (in performance and in complexity). if we want a python analysis framework, we can create one, but I am not sure how much of manalyzer we can reuse, python != C++. if you are thinking of a mixed framework, lets see your examples and use cases.

So a bit of what we did in the past. At some point MIDAS did not have this nice REST API to get events directly by request. At that time we had a python flask server running which used the Python MIDAS binding and gets some events from the buffer and created a REST API so our event display could display the events. However, my idea for the analyzer would be to further extend this idea of having an REST API which provides the user analyzed data. So my "python binding" can be anything at the end (also still a c++ application). So the focus is more from an experiment side: A few people develop the an anlyzer for an experiment but when it's going to be complex users have a hard time to even plot a "simple" histogram if they are non-experts. But if each module could export its flow events via a REST API debugging and easy development for new histograms / checks etc. could be quite easy.

What do you thing?

Best,
Marius

I can add here my five cents:

Since a while, there is an API to request individual events. Have a look at the new "Event Dump" page (which is a pure custom page using mjsonrpc_call("bm_receive_event") ).
In the mjsonrpc call you can specify the "midas even buffer", which is "SYSTEM" by default where all front-end dump their data. Nothing prevents you to create a new
buffer "ANALYZER" and send analyzed events there from your analyzer. The logger only listens to SYSTEM, so does not store these analyzed events. But you can use
the bm_receive_event call to request those analyzed events in a custom page and make a nice single event display from it. This technique would not require any 
modification in midas, just add code to your analyzer to send events to a new buffer.

Best,
Stefan 

Hi,

> Since a while, there is an API to request individual events. Have a look at the new "Event Dump" page (which is a pure custom page using mjsonrpc_call("bm_receive_event") ).
> In the mjsonrpc call you can specify the "midas even buffer", which is "SYSTEM" by default where all front-end dump their data. Nothing prevents you to create a new
> buffer "ANALYZER" and send analyzed events there from your analyzer. The logger only listens to SYSTEM, so does not store these analyzed events. But you can use
> the bm_receive_event call to request those analyzed events in a custom page and make a nice single event display from it. This technique would not require any 
> modification in midas, just add code to your analyzer to send events to a new buffer.

Question would be if this functionality should be implemented into the manalyzer framework so that everyone can use it if needed.

Best,
Marius

> Question would be if this functionality should be implemented into the manalyzer framework so that everyone can use it if needed.

You need to function calls for that bm_open_buffer and bm_send_event (both part of the MIDAS C API). The contents of the event
is anyhow up to the user. There also might be cases where one want not one but several different buffers, like for different
event types. So I would recommend start outside the manalyzer framework with that (since it's just two lines of code ... plus maybe
some error checking), and once we all know what is good we can promote this to the manalyzer framework.

Stefan

Hi,

> The contents of the event is anyhow up to the user.
Correct me but the bm_send_event wants a event in the MIDAS format. I was more thinking about a json format.

BTW: at my current MIDAS setup I get: Error: Invalid URL "" or query "cmd=Event%20Dump" or command "Event Dump" and Error: Invalid URL "" or query "cmd=OldODB" or command "OldODB" when I try to open the Event Dump or OldODB pages - rest of the pages work.

Best,
Marius

Hi,

I have now a mini working setup to get the events from the new buffer:

TEST(Analyzer, GetBuffer)
{
  INT status, request_id;
  HNDLE hBufEvent;
  INT EVENT_BUFFER_SIZE = 10000 * (1 << 25);

  status = cm_connect_experiment("", "Mu3e", "Simple Analyzer", NULL);
  if (status != CM_SUCCESS)
    ASSERT_TRUE(false);

  bm_open_buffer("ANALYZER", EVENT_BUFFER_SIZE, &hBufEvent);
  bm_request_event(hBufEvent, 1, TRIGGER_ALL, GET_ALL, &request_id, process_event);

  cm_disconnect_experiment();

  ASSERT_TRUE(true);
}

For sending the events I am not sure how to do this in the correct way. Since all the methods like bk_init etc. require some kind of frontend logic. Is there an easy way to create a MIDAS event without having a frontend?

Also when I created the new buffer I saw that there is no option to create an uint ODB entry (see file).

Best,
Marius


 

> For sending the events I am not sure how to do this in the correct way. 

bm_compose_event()
bm_send_event()

You can use bk_init (better bk_init32) to create a bank structure inside an event, but that's not mandatory.
Nothing prevents you from putting a JSON string into your event, as long as the midas even header (created
via bm_compose_event() is correct).

Best,
Stefan

> Also when I created the new buffer I saw that there is no option to create an uint ODB entry (see file).

> BTW: at my current MIDAS setup I get: Error: Invalid URL "" or query "cmd=Event%20Dump" or command "Event Dump" and Error: Invalid URL "" or query "cmd=OldODB" or command "OldODB" when I try to open the Event Dump or OldODB pages - rest of the pages work.

You have to update to the develop branch of midas and refresh your browser cache.

Stefan

> You have to update to the develop branch of midas and refresh your browser cache.
I already tried it and my local installation is 2 days old. I pulled again the dev branch now and I also got another error (actually a while ago but I forgot to report it):

Commit d5263e gives me the following error:

/home/makoeppe/mu3e/midas/progs/mhdump.cxx:393:42: error: ‘ctime’ was not declared in this scope
  393 |                            printf(" %s", ctime(&t));
      |                                          ^~~~~
/home/makoeppe/mu3e/midas/progs/mhdump.cxx:25:1: note: ‘ctime’ is defined in header ‘<ctime>’; did you forget to ‘#include <ctime>’?
   24 | #include <vector>
  +++ |+#include <ctime>
   25 | 

gcc (GCC) 12.2.0
Linux office 5.15.86-1-lts #1 SMP Sun, 01 Jan 2023 16:03:00 +0000 x86_64 GNU/Linux

After adding #include <ctime> it compiles fine.

However, after cleaning the ODB and setting up everything fresh I don't even have the pages on the left bar anymore.

Best,
Marius

> > You have to update to the develop branch of midas and refresh your browser cache.
> I already tried it and my local installation is 2 days old. I pulled again the dev branch now and I also got another error (actually a while ago but I forgot to report it):
> 
> Commit d5263e gives me the following error:
> 
> /home/makoeppe/mu3e/midas/progs/mhdump.cxx:393:42: error: ‘ctime’ was not declared in this scope
>   393 |                            printf(" %s", ctime(&t));
>       |                                          ^~~~~
> /home/makoeppe/mu3e/midas/progs/mhdump.cxx:25:1: note: ‘ctime’ is defined in header ‘<ctime>’; did you forget to ‘#include <ctime>’?
>    24 | #include <vector>
>   +++ |+#include <ctime>
>    25 | 
> 
> gcc (GCC) 12.2.0
> Linux office 5.15.86-1-lts #1 SMP Sun, 01 Jan 2023 16:03:00 +0000 x86_64 GNU/Linux
> 
> After adding #include <ctime> it compiles fine.


mhdump.cxx is written by Konstantin, so he should give his ok to add the include. 


> However, after cleaning the ODB and setting up everything fresh I don't even have the pages on the left bar anymore.

The menu on the left side is controlled via the ODB flags /Experiment/Menu. Probably the "Event dump" is of by default.

Stefan

> > > You have to update to the develop branch of midas and refresh your browser cache.
> > I already tried it and my local installation is 2 days old. I pulled again the dev branch now and I also got another error (actually a while ago but I forgot to report it):
> > 
> > Commit d5263e gives me the following error:
> > 
> > /home/makoeppe/mu3e/midas/progs/mhdump.cxx:393:42: error: ‘ctime’ was not declared in this scope
> >   393 |                            printf(" %s", ctime(&t));
> >       |                                          ^~~~~
> > /home/makoeppe/mu3e/midas/progs/mhdump.cxx:25:1: note: ‘ctime’ is defined in header ‘<ctime>’; did you forget to ‘#include <ctime>’?
> >    24 | #include <vector>
> >   +++ |+#include <ctime>
> >    25 | 
> > 
> > gcc (GCC) 12.2.0
> > Linux office 5.15.86-1-lts #1 SMP Sun, 01 Jan 2023 16:03:00 +0000 x86_64 GNU/Linux
> > 
> > After adding #include <ctime> it compiles fine.
> 
> 
> mhdump.cxx is written by Konstantin, so he should give his ok to add the include. 
Any news on this? We had this issue now on couple of more PCs and operating systems (Arch, OpenSuse and Fedora)

Best,
Marius

> > mhdump.cxx is written by Konstantin, so he should give his ok to add the include. 
> Any news on this? We had this issue now on couple of more PCs and operating systems (Arch, OpenSuse and Fedora)

Since I didn't get any complaint in the last two months, I added the include.

Stefan

In running alphaAnalysis.exe, a instance of manalyser, I would like to use the ---usetimerange option. When I include it on the commandline I get back the messages including this option which suggests that it is input incorrectly, but I also don't see it anymore in the manalyzer.cxx code.
Has it been deliberately removed, and, if so, should not the help be amended to reflect this. This option would be highly useful to me.