Back Midas Rome Roody Rootana
  Midas DAQ System, Page 38 of 121  Not logged in ELOG logo
New entries since:Wed Dec 31 16:00:00 1969
ID Date Author Topic Subject
  1675   06 Sep 2019 Andreas SuterInfoNew history plot facility
> > 2) the background of a history plot is very handy if you only show one measure.
> > If you have multiple ones (see attachment 2), this is not the case anymore. It
> > would be nice if the background could be enabled/disabled.
> 
> Looking at your plot, even without the background things look messy. Please note
> that you can display only a single curve by double clicking on it (and back with Escape).
> If all curves are on top of each other, you can get them apart a bit by zooming
> in to the vertical axis, then double click. Let ma know if that works for you.

This I found out, yet the attachment here shows another case where it would be useful to be
able to disable the background, namely if you have positive and negative measures in one
plot. Somehow it suggests that CH1 and CH2 show very different values, whereas it is only a
difference in the sign of this variables.

It's not all the important but I would like to mention this is the early stage before
everything is fully frozen.
Attachment 1: plot_plus_minus.png
plot_plus_minus.png
  1674   06 Sep 2019 Stefan RittInfoNew history plot facility
> 1) it would be nice to have an option to format the label output (see attachment 1)

That's clearly a bug, I will fix it.

 
> 2) the background of a history plot is very handy if you only show one measure.
> If you have multiple ones (see attachment 2), this is not the case anymore. It
> would be nice if the background could be enabled/disabled.

Looking at your plot, even without the background things look messy. Please note
that you can display only a single curve by double clicking on it (and back with Escape).
If all curves are on top of each other, you can get them apart a bit by zooming
in to the vertical axis, then double click. Let ma know if that works for you.

Best regards,
Stefan
  1673   06 Sep 2019 Andreas SuterInfoNew history plot facility
I like the new history system very much, but I stumbled over a couple of issues.
I used the version "Thu Aug 29 08:24:29 2019 +0200 -
midas-2019-06-b-244-gdd6585bb on branch develop":

1) it would be nice to have an option to format the label output (see attachment 1)

2) the background of a history plot is very handy if you only show one measure.
If you have multiple ones (see attachment 2), this is not the case anymore. It
would be nice if the background could be enabled/disabled.

> During my visit at TRIUMF we rewrote the history plotting functionality of
midas. Instead of 
> static GIF images, we have now interactive JavaScript panels where we can
scroll, zoom, 
> inspect values and much more (example is attached). We are now in a state
where this is still 
> work in progress, but already at this stage it might be useful for others to
report any 
> feedback.
> 
> Simply upgrade the the newest develop branch of midas, and you will see two
menu items 
> "OldHistory" which is the old system and "History" which is the new system. In
the new 
> system, you can drag with the mouse to scroll, use the mouse wheel to zoom in
and out the 
> time axis, and hover with your mouse over data points to see its value. If you
zoom out, 
> old data is loaded automatically in the background.
> 
> Following items are planned, but not yet implemented:
> 
> - Printing of run markers as in the old history
> 
> - Delete old data in the buffer to limit memory consumption if the browser
window is 
>    open for very long (weeks)
> 
> - Implement time interval selector (clock icon, select "last day", "last 8
hours" etc.)
> 
> - New settings dialog as a floating dialog box. At the moment, the setting
page of the 
>    old history system is used
> 
> - Export / Printing / Sending to ELOG any history plot
> 
> - Implement a formula for plotting data, such as "y = 12 * (x-14) +32". This
will replace 
>    the old "offset" and "factor" and is more flexible. The formula can be
passed directly 
>    to the JavaScript engine and will be executed on the web page. It should be
also 
>    possible to combine different channels, like the difference of two history
values.
> 
> - Determine the number of digits for variable display from the axis limits.
Like if a value 
>    changes between 520001 and 520002 only, we need more digits than the usual 6.
> 
> Many of these things will be implemented in the next weeks. If you have any
more idea 
> or find some bugs, please report back to me.
> 
> Best,
> Stefan for the midas team
Attachment 1: label_issue.png
label_issue.png
Attachment 2: many_labels.png
many_labels.png
  1672   01 Sep 2019 Nick HastingsForumHistory plot problems for frontend with multiple indicies
Hi Ben,

thanks for your reply. I can confirm that your suggested workaround does indeed
make the problem dissapear.

I guess this issue hasn't been seen at T2K since we use MYSQL for the history.

Thanks,

Nick.
  Draft   29 Aug 2019 Nick HastingsForumHistory plot problems for frontend with multiple indicies
Hi Ben,

thanks for your reply. I can confirm that your suggested workaround does indeed
make the problem dissapear.

I guess this issue hasn't been seen at T2K since we use MYSQL for the history.

Thanks,

Nick.
  Draft   29 Aug 2019 Nick HastingsForumHistory plot problems for frontend with multiple indicies
Hi LCP,

thanks for the suggestion and link. Unfortunatly I don't think this explains it.

Nick.
  1669   29 Aug 2019 Ben SmithForumHistory plot problems for frontend with multiple indicies
Hi Nick,

I confirm that this issue appears when using the MIDAS history driver. The issue does not appear when using the MYSQL history driver.

One solution is to give each frontend instance a different Event ID (see example code below for doing this in frontend_init). The history system did still seem to be confused by the existing FeDummy equipments/events even after making this change. However, after changing EQ_NAME from FeDummy to FeDum (i.e. starting from a clean state history-wise) things behaved normally.

I will note that some experiments definitely have a need for the "-i" method, especially those that run on distributed clusters.

Ben


```
INT frontend_init()
{
   sprintf(eq_name, "%s%02d", EQ_NAME, get_frontend_index());

   // Ensure each FE gets a different Event ID in the history system (951, 952 etc)
   char keyname[128];
   HNDLE hkey;
   int status;
   sprintf(keyname, "/Equipment/%s/Common/Event ID", eq_name);
   status = db_find_key(hDB, 0, keyname, &hkey);
   if (status != DB_SUCCESS) abort();

   WORD new_ev_id = 950 + get_frontend_index();
   status = db_set_data_index(hDB, hkey, &new_ev_id, 2, 0, TID_WORD);
   if (status != DB_SUCCESS) abort();
   return SUCCESS;
}
```
  1668   28 Aug 2019 lcpForumHistory plot problems for frontend with multiple indicies
hi, 

> > That makes things more 
> > complicated than needed. In the normal FE framework, you can define either several equipment 
> > served by one frontend, or even one equipment linked to several devices. In the MEG experiment 
> > we have one slow control frontend controlling ~100 devices without problem. In the old days 
> there 
> > was a problem that some slow devices could throttle the readout, but since the invention of 
> multi-
> > threaded slow control equipment, each device gets its own thread so they don't block each 
> other.
> 

I agree with Stefan, that it's probably better to run a multi-threaded setup, than individual frontends.

The only place I've ever used the frontend index on startup is when I was testing and building
an eventbuilder.

https://midas.triumf.ca/MidasWiki/index.php/Event_Builder#Example

This might explain, why your history is swapping between frontends, as in the event builder, it gets
reconstructed.

Maybe this helps...

LCP


> Perhaps things have changed in the 10 years since the FGD SC code was written. I can do it 
> differently but doing it that way seemed naturual since around 90% of the frontend code that I
> have see does it that way.
  1667   28 Aug 2019 Nick HastingsForumHistory plot problems for frontend with multiple indicies
Hi Stefan,

thanks for you quick reply.

> My first question would be why are you using several font-ends at all?

Becuase I was following the model used for many of the frontends for the ND280 FGD.

> That makes things more 
> complicated than needed. In the normal FE framework, you can define either several equipment 
> served by one frontend, or even one equipment linked to several devices. In the MEG experiment 
> we have one slow control frontend controlling ~100 devices without problem. In the old days 
there 
> was a problem that some slow devices could throttle the readout, but since the invention of 
multi-
> threaded slow control equipment, each device gets its own thread so they don't block each 
other.

Perhaps things have changed in the 10 years since the FGD SC code was written. I can do it 
differently but doing it that way seemed naturual since around 90% of the frontend code that I
have see does it that way.

Nick.
  1666   28 Aug 2019 Stefan RittForumHistory plot problems for frontend with multiple indicies
My first question would be why are you using several font-ends at all? That makes things more 
complicated than needed. In the normal FE framework, you can define either several equipment 
served by one frontend, or even one equipment linked to several devices. In the MEG experiment 
we have one slow control frontend controlling ~100 devices without problem. In the old days there 
was a problem that some slow devices could throttle the readout, but since the invention of multi-
threaded slow control equipment, each device gets its own thread so they don't block each other.

Stefan
  1665   28 Aug 2019 Nick HastingsForumHistory plot problems for frontend with multiple indicies
Hello experts,

I have been writing a SC frontend for a powersupply. I have used the model 
where the frontend can be started with "-i n" option so that each fe can 
control a different supply. During the development/testing of the program I 
would normally only run a single instance with "-i 1". However when I started
a second instance with "-i 2" I found problems with the history plots that
were being made for the original "-i 1" instance. The variable being plotted
seemed to randomly jump between the value from the "-i 1" instance and 
the "-i 2" instance.  confirmed that the "correct" values exist for each 
frontend in the odb under /Equipment/Foo01/Variables and 
/Equipment/Foo02/Variables

This is also not just a plotting artifact since I was also
able to see the two different values by running mhist.

I saw this behaviour using midas-2019-03 and also the head of the development
branch (686e4de2b55023b0d1936c60bcf4767c5e6caac0 from just under 48 hours ago). 

I was able to reproduce this with a stripped down frontend that just 
sets a variable that is equal to its frontend_index. Please find the code 
and Makefile attached. Presumably I've done something wrong in my 
implementation that hopefully a more experienced person can spot quite 
quickly, but please let me know if any more information is needed.

I have seen this behaviour on both Debian 10 and on a CentOS 7 Singularity 
image running on top of Debian 10.

Thanks,

Nick.

P.S. I made the topic of this post "Forum" and not "Bug Report" since I
expect the root of this problem is somewhere between the keyboard and chair.
Attachment 1: fedummy.cxx
/*******************************************************************\

  Name:         fetest.cxx
  Created by:   

  Contents:     Front end for testing MIDAS. Just sets a variable to
                the value of the frontend index.

\********************************************************************/

#undef NDEBUG // midas required assert() to be always enabled

#include <cassert>
#include "mfe.h"

/*-- Globals -------------------------------------------------------*/

/* The frontend name (client name) as seen by other MIDAS clients   */
const char *frontend_name = "fedummy";

/* The frontend file name, don't change it */
const char *frontend_file_name = __FILE__;

/* frontend_loop is called periodically if this variable is TRUE    */
BOOL frontend_call_loop = FALSE;

/* a frontend status page is displayed with this frequency in ms */
INT display_period = 0;

/* maximum event size produced by this frontend */
INT max_event_size      = 4*1024*1024;
INT max_event_size_frag = 4*1024*1024;

/* buffer size to hold events */
INT event_buffer_size = 10*1024*1024;

/*-- Function declarations -----------------------------------------*/

INT frontend_init();
INT frontend_exit();
INT begin_of_run(INT run_number, char *error);
INT end_of_run(INT run_number, char *error);
INT pause_run(INT run_number, char *error);
INT resume_run(INT run_number, char *error);
INT frontend_loop();
INT poll_event(INT source, INT count, BOOL test);
INT interrupt_configure(INT cmd, INT source, PTYPE adr);

int read_dummy_event(char *pevent, int off);

/*-- Equipment list ------------------------------------------------*/

#ifndef FE_NAME
#define FE_NAME "fedummy"
#endif

#ifndef EQ_NAME
#define EQ_NAME "Dummy"
#endif

#ifndef EQ_EVID
#define EQ_EVID 1
#endif

char eq_name[256];

EQUIPMENT equipment[] = {

  { EQ_NAME "%02d"   ,         /* equipment name */
    {
      EQ_EVID, (1<<EQ_EVID),/* event ID, trigger mask */
      "SYSTEM",             /* event buffer */
      EQ_PERIODIC,          /* equipment type */
      0,                    /* event source */
      "MIDAS",              /* format */
      TRUE,                 /* enabled */
      RO_ALWAYS,            /* Read when running */
      1000,                 /* poll every so milliseconds */
      0,                    /* stop run after this event limit */
      0,                    /* number of sub events */
      1,                    /* history period */
      "", "", ""
    },
    read_dummy_event,/* readout routine */
    NULL,
    NULL,
    NULL,       /* bank list */
  }
  ,
  {""}
};
int event_size = 10*1024;

/*-- Frontend Init -------------------------------------------------*/
#include "msystem.h"

INT frontend_init()
{
   sprintf(eq_name, "%s%02d", EQ_NAME, get_frontend_index());
   return SUCCESS;
}

INT frontend_exit()
{
   return SUCCESS;
}

INT begin_of_run(INT run_number, char *error)
{
   return SUCCESS;
}

INT end_of_run(INT run_number, char *error)
{
   return SUCCESS;
}

INT pause_run(INT run_number, char *error)
{
   return SUCCESS;
}

INT resume_run(INT run_number, char *error)
{
   return SUCCESS;
}

INT frontend_loop()
{
   return SUCCESS;
}

INT poll_event(INT source, INT count, BOOL test)
{
   return 0;
}

INT interrupt_configure(INT cmd, INT source, PTYPE adr)
{
  return 0;
}

INT read_dummy_event(char *pevent, INT off)
{
   static HNDLE hVar = 0;
   if ( hVar == 0 ) {
      char str[1024];
      sprintf(str, "/Equipment/%s/Variables", eq_name);
      
      int status = db_find_key(hDB, 0, str, &hVar);
      if ( status == DB_NO_KEY ) {
         status = db_create_key(hDB, 0, str, TID_KEY);
         if (status == DB_SUCCESS) {
            status = db_find_key(hDB, 0, str, &hVar);
         }
      }
      
      if (status != SUCCESS) {
         cm_msg(MERROR, frontend_name, "read_dummy_event: Can\'t find or create %s, status %d, exiting", str, status);
         exit(1);
      }
   }
   const float data = static_cast<float>(get_frontend_index());
   const int status = db_set_value( hDB, hVar, "Data", &data,
                          sizeof(float), 1, TID_FLOAT);
   assert(status == DB_SUCCESS);
   return 0;
}

/* emacs
 * Local Variables:
 * tab-width: 8
 * c-basic-offset: 3
 * indent-tabs-mode: nil
 * End:
 */
Attachment 2: Makefile
OSFLAGS  = -DOS_LINUX -Dextname
CFLAGS   = -g -O2 -fPIC -Wall -Wno-unused-function -I$(MIDASSYS)/include
CXXFLAGS = -std=c++11 $(CFLAGS)

LIBS = -lm -lz -lutil -lnsl -lpthread -lrt
MIDASLIBS = $(MIDASSYS)/lib/libmidas.a
MFE = $(MIDASSYS)/lib/mfe.o

all:: fedummy.exe

fedummy.exe: %.exe: fedummy.cxx $(MIDASLIBS) $(MFE)
	$(CXX) -o $@ $(CXXFLAGS) $(OSFLAGS) $^ $(MIDASLIBS) $(LIBS) -DFE_NAME=\"fedummy\" -DEQ_NAME=\"FeDummy\" -DEQ_EVID=1


%.o: %.cxx
	$(CXX) $(CXXFLAGS) $(OSFLAGS) -c $<

%.d: %.cxx
	$(CXX) -MM -MD $(CXXFLAGS) $(OSFLAGS) -c $<

depend:

-include *.d

clean::
	-rm -f *.o *.exe
	-rm -f *.d

  1664   14 Aug 2019 Konstantin OlchanskiBug ReportROOTANA bug?
> -  ss_suspend_set_dispatch_ipc(NULL);
> +  //  ss_suspend_set_dispatch_ipc(NULL);
> 
> This compiles and at least runs for me; so maybe that is helpful for you.  But Konstantin will provide a longer term solution.

I now understand why this fix worked. Around December 2018 timeframe, I reworked the MIDAS event buffer code
and one improvement was to only send UDP buffer notifications if somebody is waiting for them. This probably
reduced to zero the probability of recursive calls to the user event handler - the problem originally fixed by the monkey
work against the midas ipc handler.

After looking at it, I now understand that the correct solution is to call ss_suspend(MSG_BM), but it turns out
inside MIDAS, handling of MSG_BM was incomplete and recursive calls to the user event handler were still
possible. (but most likely not actually happening anymore because of those changes to the event buffer code).

So.

a) ss_suspend(MSG_BM) inside midas now works correctly, recursive call to the user event handler will not happen.
b) TMidasOnline::sleep() now calls ss_suspend(MSG_BM), monkey business with ss_suspend_set_dispatch_ipc() is removed.

The problem of recursive call to the analyzer event handler is now fixed, both rootana and manalyzer (both use the same TMidasOnline code).

Read more about this here:
https://midas.triumf.ca/elog/Midas/1663

K.O.
  1663   14 Aug 2019 Konstantin OlchanskiBug Fixincorrect recursion in ss_suspend() via the user event handler
The ROOTANA midas analyzer uncovered a problem with recursive use of ss_suspend().

When running in graphical mode, the ROOT graphics main event loop was calling 
ss_suspend(0) (no MSG_BM) which would sometimes call the user event handler (if a new 
event arrives into the midas event buffer). Because this loop was already running in the 
user event handler, there was a crash.

This is the calling sequence leading to the incorrect recursion: (from system.cxx comments 
to ss_suspend())

analyzer ->
     -> cm_yield() in the main loop
     -> ss_suspend(0)
     -> MSG_BM message arrives in the UDP socket
     -> ss_suspend_process_ipc(0)
     -> cm_dispatch_ipc()
     -> bm_push_event()
     -> bm_push_buffer()
     -> bm_read_buffer()
     -> bm_dispatch_event()
     -> user event handler
     -> user event handler ROOT graphics main loop needs to sleep
     -> ss_suspend(0) <--- should be ss_suspend(MSG_BM)!!!     
     -> MSG_BM message arrives in the UDP socket
     -> ss_suspend_process_ipc(0) <- should be ss_suspend_process_ipc(MSG_BM)!!!
     -> cm_dispatch_ipc() <- without MSG_BM, calling cm_dispatch_ipc() again
     -> bm_push_event()
     -> bm_push_buffer()
     -> bm_read_buffer()
     -> bm_dispatch_event()
     -> user event handler <---- called recursively, very bad!

The proper fix for this is to always call ss_suspend(MSG_BM) from the user event handler 
and ss_suspend(MSG_ODB) from the user db_watch handler.

In this second case, ss_suspend(MSG_OBM) will lose/ignore/discard db_watch notifications, 
if you do not want that, call ss_suspend(0) and be ready for recursive calls to your 
db_watch handler. (this can happen in a frontend program that acts on changes in ODB and 
where some of these actions may require sleeping via ss_suspend()).

ss_suspend(MSG_BM) will discard MSG_BM messages, which is not a problem as 
bm_wait_for_events() and cm_yield() will immediately poll the event buffer and there will be 
no delay in receiving new events.

Until commit 99d6e90 there were problems in ss_suspend(MSG_BM) and recursive calls to 
the user event handler were still possible. It is now fixed in this and the previous commits.

K.O.
  1662   14 Aug 2019 Stefan RittInfoNew history plot facility
During my visit at TRIUMF we rewrote the history plotting functionality of midas. Instead of 
static GIF images, we have now interactive JavaScript panels where we can scroll, zoom, 
inspect values and much more (example is attached). We are now in a state where this is still 
work in progress, but already at this stage it might be useful for others to report any 
feedback.

Simply upgrade the the newest develop branch of midas, and you will see two menu items 
"OldHistory" which is the old system and "History" which is the new system. In the new 
system, you can drag with the mouse to scroll, use the mouse wheel to zoom in and out the 
time axis, and hover with your mouse over data points to see its value. If you zoom out, 
old data is loaded automatically in the background.

Following items are planned, but not yet implemented:

- Printing of run markers as in the old history

- Delete old data in the buffer to limit memory consumption if the browser window is 
   open for very long (weeks)

- Implement time interval selector (clock icon, select "last day", "last 8 hours" etc.)

- New settings dialog as a floating dialog box. At the moment, the setting page of the 
   old history system is used

- Export / Printing / Sending to ELOG any history plot

- Implement a formula for plotting data, such as "y = 12 * (x-14) +32". This will replace 
   the old "offset" and "factor" and is more flexible. The formula can be passed directly 
   to the JavaScript engine and will be executed on the web page. It should be also 
   possible to combine different channels, like the difference of two history values.

- Determine the number of digits for variable display from the axis limits. Like if a value 
   changes between 520001 and 520002 only, we need more digits than the usual 6.

Many of these things will be implemented in the next weeks. If you have any more idea 
or find some bugs, please report back to me.

Best,
Stefan for the midas team
Attachment 1: Screenshot_2019-08-14_at_8.50.53_.png
Screenshot_2019-08-14_at_8.50.53_.png
  1661   13 Aug 2019 Stefan RittInfoPrecedence of equipment/common structure
> Lacking any ideas for improvements, I vote for the status quo. (plus a review of the documentation to ensure we have clearly
> written up what each entry in "common" does and whether the user is permitted to edit it in odb).

I agree with that.

Stefan
  1659   09 Aug 2019 Konstantin OlchanskiForumHow to convert C midas frontends to C++
> How do I solve mismatched declarations in the mfe (or other places in the midas code)?

I run into such problems all the time. My solution? I grep for the function name in my code and in the header file,
then look very carefully at the definition to confirm that all the argument declarations are the same in both
places. Sometimes my eyes do not see the difference and I ask for a "second pair of eyes".

In your case, you have a mismatch between functions in mfe.h and in your frontend. The difference
is "int source" in mfe.h and "int source[]" in your code.

Because C++ permits functions with identical namesm but different arguments, the compiler thinks
you did this on purpose and does not complain. Later, of course, the linker bombs,
but all it can report at this stage, is what you see "function not found"... Then you grep your code
for the missing function, check arguments, rinse, repeat.

Before C++, the C compiler would probably had complained about the mismatch, except that MIDAS
did not have an mfe.h header file definitions for all this stuff until just now, so again, the mismatch would
have gone unnoticed, unfixed.

K.O.



> It is having issues with the midas defined BOOL/... types. This 
> is what I get for a minimal scfe:
> 
> [ 12%] Building CXX object CMakeFiles/sc_fe_mini.dir/sc_fe_mini.cpp.o
> [ 25%] Building CXX object CMakeFiles/sc_fe_mini.dir/home/frederik/packages/midas/drivers/class/hv.cxx.o
> [ 37%] Building CXX object CMakeFiles/sc_fe_mini.dir/home/frederik/packages/midas/drivers/class/multi.cxx.o
> [ 50%] Building CXX object CMakeFiles/sc_fe_mini.dir/home/frederik/packages/midas/drivers/device/nulldev.cxx.o
> [ 62%] Building CXX object CMakeFiles/sc_fe_mini.dir/home/frederik/packages/midas/drivers/bus/null.cxx.o
> [ 75%] Building CXX object CMakeFiles/sc_fe_mini.dir/home/frederik/packages/midas/drivers/device/mscbdev.cxx.o
> [ 87%] Building CXX object CMakeFiles/sc_fe_mini.dir/home/frederik/packages/midas/mscb/src/mscb.cxx.o
> [100%] Linking CXX executable sc_fe_mini
> /home/frederik/packages/midas/build/libmfe.a(mfe.cxx.o): In function `_readout_thread':
> /home/frederik/packages/midas/src/mfe.cxx:1271: undefined reference to `poll_event(int, int, unsigned int)'
> /home/frederik/packages/midas/build/libmfe.a(mfe.cxx.o): In function `check_polled_events':
> /home/frederik/packages/midas/src/mfe.cxx:1601: undefined reference to `poll_event(int, int, unsigned int)'
> /home/frederik/packages/midas/src/mfe.cxx:1643: undefined reference to `poll_event(int, int, unsigned int)'
> /home/frederik/packages/midas/build/libmfe.a(mfe.cxx.o): In function `readout_enable(unsigned int)':
> /home/frederik/packages/midas/src/mfe.cxx:1158: undefined reference to `interrupt_configure(int, int, long)'
> /home/frederik/packages/midas/src/mfe.cxx:1156: undefined reference to `interrupt_configure(int, int, long)'
> /home/frederik/packages/midas/build/libmfe.a(mfe.cxx.o): In function `initialize_equipment':
> /home/frederik/packages/midas/src/mfe.cxx:614: undefined reference to `interrupt_configure(int, int, long)'
> /home/frederik/packages/midas/src/mfe.cxx:649: undefined reference to `poll_event(int, int, unsigned int)'
> /home/frederik/packages/midas/build/libmfe.a(mfe.cxx.o): In function `scheduler':
> /home/frederik/packages/midas/src/mfe.cxx:1890: undefined reference to `poll_event(int, int, unsigned int)'
> /home/frederik/packages/midas/src/mfe.cxx:1932: undefined reference to `poll_event(int, int, unsigned int)'
> /home/frederik/packages/midas/build/libmfe.a(mfe.cxx.o): In function `main':
> /home/frederik/packages/midas/src/mfe.cxx:2701: undefined reference to `interrupt_configure(int, int, long)'
> /home/frederik/packages/midas/src/mfe.cxx:2702: undefined reference to `interrupt_configure(int, int, long)'
> collect2: error: ld returned 1 exit status
> make[2]: *** [sc_fe_mini] Error 1
> make[1]: *** [CMakeFiles/sc_fe_mini.dir/all] Error 2
> make: *** [all] Error 2
> 
> 
> This is my cmakelists for my user code:
> 
> #
> # cmake for the muX software
> #
> cmake_minimum_required(VERSION 3.3)
> 
> project(muX)
> 
> #
> # find installations
> #
> set(MIDAS_DIR $ENV{MIDASSYS})
> message("MIDAS dir: " ${MIDAS_DIR})
> 
> #
> # set directories
> #
> set(MIDASBUILD_DIR ${MIDAS_DIR}/build)
> set(MIDASINCLUDE_DIR ${MIDAS_DIR}/include)
> set(MXML_DIR ${MIDAS_DIR}/mxml)
> set(MSCB_DIR ${MIDAS_DIR}/mscb)
> set(DRV_DIR ${MIDAS_DIR}/drivers)
> 
> 
> #
> # drivers, libs
> #
> set(DRIVERS
>     ${MIDAS_DIR}/drivers/class/hv
>     ${MIDAS_DIR}/drivers/class/multi
>     ${MIDAS_DIR}/drivers/device/nulldev
>     ${MIDAS_DIR}/drivers/bus/null
> )
> set(MIDASLIB ${MIDASBUILD_DIR}/libmidas.a)
> set(FELIB ${MIDASBUILD_DIR}/libmfe.a)
> 
> #
> # sc_fe
> #
> add_executable(sc_fe_mini
>         sc_fe_mini.cpp
>         ${DRIVERS}
>         ${MIDAS_DIR}/drivers/device/mscbdev
>         ${MIDAS_DIR}/mscb/src/mscb)
> 
> target_include_directories(sc_fe_mini PRIVATE ${DRV_DIR} ${MIDAS_DIR}/mscb/include ${MIDAS_DIR}/include)
> target_link_libraries(sc_fe_mini ${LIBS} ${MIDASLIB} ${FELIB} rt pthread util)
> 
> 
> 
> I seem to be able to compile the current midas distributions, including the scfe frontend
> 
> 
> 
> > To convert a MIDAS frontend to C++ follow this checklist:
> > 
> > a) add #include "mfe.h" after include of midas.h and fix all compilation errors.
> > 
> > NOTE: there should be no "extern C"  brackets around MIDAS include files.
> > 
> > NOTE: Expect to see following problems:
> > 
> > a1) duplicate or mismatched declarations of functions defined in mfe.h
> > a2) frontend_name and frontend_file_name should be "const char*" instead of "char*"
> > a3) duplicate "HNDLE hDB" collision with hDB from mfe.c - not sure why it worked before, either use HNDLE hDB from mfe.h or use "extern HNDLE hDB".
> > a4) poll_event() and interrupt_configure() have "source" as "int[]" instead of "int" (why did this work before?)
> > a5) use of "extern int frontend_index" instead of get_frontend_index() from mfe.h
> > a6) bk_create() last argument needs to be cast to (void**)
> > a7) "bool debug" collides with "debug" from mfe.h (why did this work before?)
> > 
> > b) remove no longer needed "extern C" brackets around mfe related code. Ideally there should be no "extern C" brackets anywhere.
> > 
> > c) in the Makefile, change CC=gcc to CC=g++ for compiling and linking everything as C++
> > 
> > c1) fix all compilation problems. most valid C code will compile as valid C++, but there is some known trouble:
> > - return value of malloc() & co needs to be cast to the correct data type: "char* s = (char*)malloc(...)"
> > - some C++ compilers complain about mismatch between signed and unsigned values
> > 
> > If you need help with converting your frontend from C to C++, I will be most happy
> > to assist you - post your compiler error messages to this forum or email them to me privately.
> > 
> > Good luck,
> > K.O.
  1658   09 Aug 2019 Konstantin OlchanskiBug ReportFetest History Plot
> Hi, our logger was running.

Please do these simple tests:

- run "mlogger -v", it should report that it is writing slow/slow data into the history with rate 1 Hz (fetest 
should be running at this point, yes?)
- normally the history files are written into the experiment directory (where ODB is, etc) and have file names 
"*.hst". Observe that the files are growing. Use "ls -ltr". (mlogger and fetest should be running at this point, 
yes?)
- if all if this is happening, you can try to run "mhist" to see the history data

If all of the above works, but you still get nothing from the history plots in mhttpd, then we probably have a 
bug in midas and we would like very much to fix it. For this we will need some more information from you. I 
hope you have some time available to help us with this.

Hmm... the fetest history plots are not defined automatically, you have to create the history plot manually,
maybe this is where the problem happens. One thing to check here, the correct variable to plot
is "slow/slow", if I remember right.


K.O.


> I have tried restarting mlogger (even though we haven't
> changed variable names). We ran the following commands one after another and still no
> luck with history plot. Is there anything else that could be causing these problems?
> 
> Kind regards,
> Hassan 
> 
> 
================================================================================
==
> 
> [lm17773@it038146 ~]$ cd /opt/midas_software/midas/bin/
> [lm17773@it038146 bin]$ mhttpd
> [mhttpd,ERROR] [odb.cxx:1646:db_open_database,ERROR] Removed ODB client 'mhttpd',
> index 0 because process pid 20094 does not exists
> [mhttpd,ERROR] [odb.cxx:1646:db_open_database,ERROR] Removed ODB client 'Logger',
> index 1 because process pid 20214 does not exists
> [mhttpd,INFO] Removed open record flag from "/Experiment/Security/RPC hosts/Allowed hosts"
> [mhttpd,INFO] Removed exclusive access mode from "/Experiment/Security/RPC
> hosts/Allowed hosts"
> [mhttpd,INFO] Removed open record flag from "/Experiment/Security/mhttpd hosts/Allowed
> hosts"
> [mhttpd,INFO] Removed exclusive access mode from "/Experiment/Security/mhttpd
> hosts/Allowed hosts"
> [mhttpd,INFO] Removed open record flag from "/Logger/History"
> [mhttpd,INFO] Removed exclusive access mode from "/Logger/History"
> [mhttpd,INFO] Removed open record flag from "/Sequencer/State"
> [mhttpd,INFO] Removed exclusive access mode from "/Sequencer/State"
> [mhttpd,INFO] Removed open record flag from "/History/LoggerHistoryChannel"
> [mhttpd,INFO] Removed exclusive access mode from "/History/LoggerHistoryChannel"
> [mhttpd,INFO] Removed open record flag from "/Equipment/slow/Variables"
> [mhttpd,INFO] Removed exclusive access mode from "/Equipment/slow/Variables"
> [mhttpd,INFO] Removed open record flag from "/Equipment/Trigger/Statistics/Events per
> sec."
> [mhttpd,INFO] Removed exclusive access mode from "/Equipment/Trigger/Statistics/Events
> per sec."
> [mhttpd,INFO] Removed open record flag from "/Equipment/Trigger/Statistics/kBytes per
> sec."
> [mhttpd,INFO] Removed exclusive access mode from "/Equipment/Trigger/Statistics/kBytes
> per sec."
> [mhttpd,INFO] Removed open record flag from "/Equipment/Periodic/Variables"
> [mhttpd,INFO] Removed exclusive access mode from "/Equipment/Periodic/Variables"
> [mhttpd,INFO] Removed open record flag from "/Equipment/Scaler/Variables"
> [mhttpd,INFO] Removed exclusive access mode from "/Equipment/Scaler/Variables"
> [mhttpd,INFO] Corrected 10 ODB entries
> [mhttpd,INFO] Deleted entry '/System/Clients/20094' for client 'mhttpd' because it is
> not connected to ODB
> Mongoose web server will use SSL certificate file "/home/lm17773/online/ssl_cert.pem"
> Mongoose web server will use authentication realm "sampleexpt", password file
> "/home/lm17773/online/htpasswd.txt"
> mongoose web server is redirecting HTTP port 8080 to
> https://it038146.users.bris.ac.uk:8443
> mongoose web server is listening on the HTTP port 8080
> mongoose web server is listening on the HTTPS port 8443
> 
================================================================================
====
> 
> [lm17773@it038146 bin]$ mlogger
> [Logger,INFO] Deleted entry '/System/Clients/20214' for client 'Logger' because it is
> not connected to ODB
> Log     directory is /home/lm17773/online/
> Data    directory is same as Log unless specified in /Logger/channels/
> History directory is same as Log unless specified in /Logger/history/
> ELog    directory is same as Log
> SQL     database is localhost/sampleexpt/Runlog
> MIDAS logger started. Stop with "!"
> 
================================================================================
====
> [lm17773@it038146 bin]$ fetest
> Frontend name          :     fetest
> Event buffer size      :     10485760
> User max event size    :     4194304
> User max frag. size    :     4194304
> # of events per buffer :     2
> 
> Connect to experiment sampleexpt...
> OK
> Init hardware...frontend_init!
> Event size set to 10240 bytes
> Ring buffer wait sleep 1 ms
> OK
> time 1564131394, data 97.814758
> time 1564131395, data 96.592583
> time 1564131396, data 95.105652
> time 1564131397, data 93.358040
> time 1564131398, data 91.354546
> time 1564131399, data 89.100655
> time 1564131400, data 86.602539
> time 1564131401, data 83.867058
> time 1564131402, data 80.901703
> time 1564131403, data 77.714592
> Warning: bank RND4 has zero size
> time 1564131404, data 74.314484
> time 1564131405, data 70.710678
> time 1564131406, data 66.913063
> time 1564131407, data 62.932041
> 
================================================================================
====
> 
> 
> 
> 
> 
> 
> > > Hi,
> > > 
> > > We've been trying to run Fetest in the attempt of plotting the sine wave data on
> > > the history page on the web server. However each time we've tried running a new
> > > plot we have come across the error of 'no data' from the variables. In the
> > > status page we are clearly obtaining data from the frontend and it is updating
> > > the variable as expected in SLOW.
> > > 
> > > When setting up MIDAS we managed to produce a history plot from Fetest but are
> > > unable to do so any longer. We did have a go at modifying the Fetest code but
> > > created a backup before doing so and are now running the original backup.
> > > 
> > > What could be causing the Fetest data not to be showing in the history plot?
> > 
> > Is the logger running? (this application is handling the history data).
> > If yes: Did you change the variable names? If yes: restart the logger.
  1657   09 Aug 2019 Konstantin OlchanskiBug ReportFetest History Plot
> Hi, our logger was running.


One more thing is to check that the history files are actually being written to. Be default
the history files are written into the same directory where you have ODB, etc and
the file names are "*.hst".

Second thing, you can run "mlogger -v" and it will report that it is writing history events
into the history.

If all these things are happening,

Third, you can run "mhist" to see the data directly from the data files.

If all of these work, but you still get nothing in mhttpd, that would be very strange. If I remember
right, to see the sine wave, the history variable to plot is equipment "slow", variable "slow".


K.O.



> I have tried restarting mlogger (even though we haven't
> changed variable names). We ran the following commands one after another and still no
> luck with history plot. Is there anything else that could be causing these problems?
> 
> Kind regards,
> Hassan 
> 
> ==================================================================================
> 
> [lm17773@it038146 ~]$ cd /opt/midas_software/midas/bin/
> [lm17773@it038146 bin]$ mhttpd
> [mhttpd,ERROR] [odb.cxx:1646:db_open_database,ERROR] Removed ODB client 'mhttpd',
> index 0 because process pid 20094 does not exists
> [mhttpd,ERROR] [odb.cxx:1646:db_open_database,ERROR] Removed ODB client 'Logger',
> index 1 because process pid 20214 does not exists
> [mhttpd,INFO] Removed open record flag from "/Experiment/Security/RPC hosts/Allowed hosts"
> [mhttpd,INFO] Removed exclusive access mode from "/Experiment/Security/RPC
> hosts/Allowed hosts"
> [mhttpd,INFO] Removed open record flag from "/Experiment/Security/mhttpd hosts/Allowed
> hosts"
> [mhttpd,INFO] Removed exclusive access mode from "/Experiment/Security/mhttpd
> hosts/Allowed hosts"
> [mhttpd,INFO] Removed open record flag from "/Logger/History"
> [mhttpd,INFO] Removed exclusive access mode from "/Logger/History"
> [mhttpd,INFO] Removed open record flag from "/Sequencer/State"
> [mhttpd,INFO] Removed exclusive access mode from "/Sequencer/State"
> [mhttpd,INFO] Removed open record flag from "/History/LoggerHistoryChannel"
> [mhttpd,INFO] Removed exclusive access mode from "/History/LoggerHistoryChannel"
> [mhttpd,INFO] Removed open record flag from "/Equipment/slow/Variables"
> [mhttpd,INFO] Removed exclusive access mode from "/Equipment/slow/Variables"
> [mhttpd,INFO] Removed open record flag from "/Equipment/Trigger/Statistics/Events per
> sec."
> [mhttpd,INFO] Removed exclusive access mode from "/Equipment/Trigger/Statistics/Events
> per sec."
> [mhttpd,INFO] Removed open record flag from "/Equipment/Trigger/Statistics/kBytes per
> sec."
> [mhttpd,INFO] Removed exclusive access mode from "/Equipment/Trigger/Statistics/kBytes
> per sec."
> [mhttpd,INFO] Removed open record flag from "/Equipment/Periodic/Variables"
> [mhttpd,INFO] Removed exclusive access mode from "/Equipment/Periodic/Variables"
> [mhttpd,INFO] Removed open record flag from "/Equipment/Scaler/Variables"
> [mhttpd,INFO] Removed exclusive access mode from "/Equipment/Scaler/Variables"
> [mhttpd,INFO] Corrected 10 ODB entries
> [mhttpd,INFO] Deleted entry '/System/Clients/20094' for client 'mhttpd' because it is
> not connected to ODB
> Mongoose web server will use SSL certificate file "/home/lm17773/online/ssl_cert.pem"
> Mongoose web server will use authentication realm "sampleexpt", password file
> "/home/lm17773/online/htpasswd.txt"
> mongoose web server is redirecting HTTP port 8080 to
> https://it038146.users.bris.ac.uk:8443
> mongoose web server is listening on the HTTP port 8080
> mongoose web server is listening on the HTTPS port 8443
> 
====================================================================================
> 
> [lm17773@it038146 bin]$ mlogger
> [Logger,INFO] Deleted entry '/System/Clients/20214' for client 'Logger' because it is
> not connected to ODB
> Log     directory is /home/lm17773/online/
> Data    directory is same as Log unless specified in /Logger/channels/
> History directory is same as Log unless specified in /Logger/history/
> ELog    directory is same as Log
> SQL     database is localhost/sampleexpt/Runlog
> MIDAS logger started. Stop with "!"
> 
====================================================================================
> [lm17773@it038146 bin]$ fetest
> Frontend name          :     fetest
> Event buffer size      :     10485760
> User max event size    :     4194304
> User max frag. size    :     4194304
> # of events per buffer :     2
> 
> Connect to experiment sampleexpt...
> OK
> Init hardware...frontend_init!
> Event size set to 10240 bytes
> Ring buffer wait sleep 1 ms
> OK
> time 1564131394, data 97.814758
> time 1564131395, data 96.592583
> time 1564131396, data 95.105652
> time 1564131397, data 93.358040
> time 1564131398, data 91.354546
> time 1564131399, data 89.100655
> time 1564131400, data 86.602539
> time 1564131401, data 83.867058
> time 1564131402, data 80.901703
> time 1564131403, data 77.714592
> Warning: bank RND4 has zero size
> time 1564131404, data 74.314484
> time 1564131405, data 70.710678
> time 1564131406, data 66.913063
> time 1564131407, data 62.932041
> 
====================================================================================
> 
> 
> 
> 
> 
> 
> > > Hi,
> > > 
> > > We've been trying to run Fetest in the attempt of plotting the sine wave data on
> > > the history page on the web server. However each time we've tried running a new
> > > plot we have come across the error of 'no data' from the variables. In the
> > > status page we are clearly obtaining data from the frontend and it is updating
> > > the variable as expected in SLOW.
> > > 
> > > When setting up MIDAS we managed to produce a history plot from Fetest but are
> > > unable to do so any longer. We did have a go at modifying the Fetest code but
> > > created a backup before doing so and are now running the original backup.
> > > 
> > > What could be causing the Fetest data not to be showing in the history plot?
> > 
> > Is the logger running? (this application is handling the history data).
> > If yes: Did you change the variable names? If yes: restart the logger.
  1656   09 Aug 2019 Konstantin OlchanskiInfoPrecedence of equipment/common structure
> Today I fixed a long-annoying problem. ...
> /Equipment/<name>/Common
> In the past, the ODB setting took precedence over the frontend structure...
> We defined this like 25 years ago and I forgot what the exact reason was.
> It causes however many people (including myself) to fall into this trap: ...

There is good number of confusions regarding entries in /eq/xxx/common:

- for some of them, the frontend code settings take precedence and overwrite settings in odb ("frontend file name")
- for some of them, ODB takes precedence and frontend code values are ignored ("read on" and "period")
- for some of them, changes in ODB take effect immediately (via db_watch) ("period")
- for some of them, frontend restart is required for changes to take effect (output event buffer name "buffer")
- some of them continuously update the odb values ("status", "status color")

I do not think there is a simple way to improve on this.

(One solution would replace the single "common" with several subdirectories, "per function",
one would have items where the code takes precedence, one would have items where odb takes
precedence (in effect, "standard settings"), one will have items that the frontend always updates
and that should not be changes via odb ("frontend name", etc). I am not sure this one solution
is necessarily an "improvement").

Lacking any ideas for improvements, I vote for the status quo. (plus a review of the documentation to ensure we have clearly
written up what each entry in "common" does and whether the user is permitted to edit it in odb).

K.O.
  1655   08 Aug 2019 Stefan RittSuggestionmidas cmake migration
I just tried CMake 3.1.0 and it worked with midas. So I believe all versions between 3.1.0 and 3.6.1 are ok.

Actually playing around with different versions I realized that 3.0.0 is also ok, so I changed the requirement of midas down to 3.0

Stefan
ELOG V3.1.4-2e1708b5