| ID |
Date |
Author |
Topic |
Subject |
|
1682
|
12 Sep 2019 |
Pintaudi Giorgio | Info | History panels in custom pages | > > A new tag has been implemented to display history panels in custom pages, integrated in the
> > new custom page design from 2017. The full documentation can be found at
> >
>
> As part of consolidating/cleaning the MIDAS Wiki documentation, the "New Custom Pages" was folded into the main "Custom Page". So to see a
> description of Stefan's new functionality please go to
>
> https://midas.triumf.ca/MidasWiki/index.php/Custom_Page#mhistory
Hello!
I am trying to use the new mhistory panels in the WAGASCI slow control custom page, but I cannot get them to work.
All I get is an empty frame. Anyway, in the History tab I can see the history plots correctly.
Here is a minimal example:<html>
<head>
<title>Test</title>
<link rel="stylesheet" href="midas.css">
<script src="controls.js"></script>
<script src="midas.js"></script>
<script src="mhttpd.js"></script>
</head>
<body class="mcss" onload="mhttpd_init('Test');">
<div id="mheader"></div>
<div id="msidenav"></div>
<div id="mmain">
<div name="mhistory" data-group="Test" data-panel="Test" data-scale="1m" style="width:600px;border:1px solid black;"></div>
</div>
</body>
</html>
Of course, the "Test" group and "Test" panel exist in the ODB and are correctly shown in the History tab. No error is shown in the console of the web browser.
I am using the latest version of MIDAS as of September 12.
Can you confirm that this feature is working in the latest MIDAS? If yes, how can I troubleshoot the problem?
Regards
Giorgio |
|
1681
|
10 Sep 2019 |
Andreas Suter | Info | New history plot facility | Our typical use case is that a lot of people are connected to the experiment
having some history tabs open most of the time. Hence, I setup a test system and
connect to it from all kind of systems/browsers. What I see currently quite
often is the error hs_read_arraybuffer (see the attachement).
For firefox 60.8.0esr this can result into a full freeze of the tab and only
closing it is possible.
For chromium based browsers you eventually get a popup informing that it is not
responsive anymore.
The worst though is safari 12.1.2 which not only freezes the tab, but
reproducibly crashes the mhttpd on the server side.
Are there ways to get a log which would document where the problems start? |
| Attachment 1: history_hangs.PNG
|
|
|
1680
|
08 Sep 2019 |
Vinzenz Bildstein | Bug Report | https redirect and ODB access | I'm not sure if these issues are related or not, but I'm getting an error
message when I want to access the root of the ODB via the webserver:
[mhttpd,ERROR] [mhttpd.cxx:563:rread,ERROR] Cannot read file '/root', read of
4096 returned -1, errno 21 (Is a directory)
I also tried turning the re-direct from http to https off, but this does not
seem to work. I also noticed that the redirect changes the localhost into a
hostname. Where does mongoose take this hostname from?
EDIT: Seems that the change of the hostname is due to a setting in /etc/hosts,
i.e. all my fault ...
EDIT: I think there was some issue with the mhttpd. When I checked the output (I
used screen to run it), it was full of these messages:
ss_semaphore_wait_for: semop/semtimedop(2588679) returned -1, errno 43
(Identifier removed)
al_check: Something is wrong with our semaphore, ss_semaphore_wait_for()
returned 408, aborting.
al_check: Cannot abort - this will lock you out of odb. From this point, MIDAS
will not work correctly. Please read the discussion at
https://midas.triumf.ca/elog/Midas/945
Restarted it and it stopped redirecting. So accessing the root of the ODB via
the webserver is the only issue now. |
|
1679
|
08 Sep 2019 |
Stefan Ritt | Info | New history plot facility | > 1) it would be nice to have an option to format the label output (see attachment 1)
I fixed that in the current version.
Stefan |
| Attachment 1: Screenshot_2019-09-08_at_12.29.12_.png
|
|
|
1678
|
07 Sep 2019 |
Stefan Ritt | Info | New history plot facility | > This I found out, yet the attachment here shows another case where it would be useful to be
> able to disable the background, namely if you have positive and negative measures in one
> plot. Somehow it suggests that CH1 and CH2 show very different values, whereas it is only a
> difference in the sign of this variables.
Ok, I added an option which lets you switch off the background.
I also changed the background drawing such that it only goes to the y=0 axis, not the bottom of the screen.
That should help displaying negative values.
Stefan |
| Attachment 1: Screenshot_2019-09-07_at_13.52.49_.png
|
|
| Attachment 2: Slow-Sine_3-20198107-132905-20198107-135305.png
|
|
|
Draft
|
07 Sep 2019 |
Stefan Ritt | Info | New history plot facility | > This I found out, yet the attachment here shows another case where it would be useful to be
> able to disable the background, namely if you have positive and negative measures in one
> plot. Somehow it suggests that CH1 and CH2 show very different values, whereas it is only a
> difference in the sign of this variables.
Ok, I added
- a correction which does the fill not to the bottom of the window, but only to the y=0 axis.
- a flag "Show graph fille" which lets you turn on and off the filling for each plot
Best,
Stefan |
|
1676
|
06 Sep 2019 |
Pintaudi Giorgio | Forum | Open a hotlink to a single element in an ODB array | Hello!
Just a little question about the ODB hotlinks. Is it possible to open a hotlink
to a single element in and ODB array?
I have searched through the documentation and I have taken a look at the source
code but I could not find any piece of code to use as a reference (maybe I have
not searched deeply enough).
This is more or less what I would like to achieve (without error checking):
for (INT i = 0; i < hv_info->num_channels; i++) {
char element[HKEY_STRING_LENGTH];
snprintf(element, HKEY_STRING_LENGTH, "%s[%d]", path, i);
if (db_find_key(hDB, hv_info->hKeyRoot, element, &hKey) == DB_SUCCESS) {
if ((hv_info->driver[i]->flags & flag) == 0)
db_open_record(hDB, hKey, &array[i], sizeof(double), MODE_READ, callback, pequipment);
else
db_open_record(hDB, hKey, &array[i], sizeof(double), MODE_READ, NULL, NULL);
} else {
cm_msg(MERROR, __func__, "Key %s not found", element);
}
}
But it is not working because the key is not found ...
Thank you
Giorgio |
|
1675
|
06 Sep 2019 |
Andreas Suter | Info | New history plot facility | > > 2) the background of a history plot is very handy if you only show one measure.
> > If you have multiple ones (see attachment 2), this is not the case anymore. It
> > would be nice if the background could be enabled/disabled.
>
> Looking at your plot, even without the background things look messy. Please note
> that you can display only a single curve by double clicking on it (and back with Escape).
> If all curves are on top of each other, you can get them apart a bit by zooming
> in to the vertical axis, then double click. Let ma know if that works for you.
This I found out, yet the attachment here shows another case where it would be useful to be
able to disable the background, namely if you have positive and negative measures in one
plot. Somehow it suggests that CH1 and CH2 show very different values, whereas it is only a
difference in the sign of this variables.
It's not all the important but I would like to mention this is the early stage before
everything is fully frozen. |
| Attachment 1: plot_plus_minus.png
|
|
|
1674
|
06 Sep 2019 |
Stefan Ritt | Info | New history plot facility | > 1) it would be nice to have an option to format the label output (see attachment 1)
That's clearly a bug, I will fix it.
> 2) the background of a history plot is very handy if you only show one measure.
> If you have multiple ones (see attachment 2), this is not the case anymore. It
> would be nice if the background could be enabled/disabled.
Looking at your plot, even without the background things look messy. Please note
that you can display only a single curve by double clicking on it (and back with Escape).
If all curves are on top of each other, you can get them apart a bit by zooming
in to the vertical axis, then double click. Let ma know if that works for you.
Best regards,
Stefan |
|
1673
|
06 Sep 2019 |
Andreas Suter | Info | New history plot facility | I like the new history system very much, but I stumbled over a couple of issues.
I used the version "Thu Aug 29 08:24:29 2019 +0200 -
midas-2019-06-b-244-gdd6585bb on branch develop":
1) it would be nice to have an option to format the label output (see attachment 1)
2) the background of a history plot is very handy if you only show one measure.
If you have multiple ones (see attachment 2), this is not the case anymore. It
would be nice if the background could be enabled/disabled.
> During my visit at TRIUMF we rewrote the history plotting functionality of
midas. Instead of
> static GIF images, we have now interactive JavaScript panels where we can
scroll, zoom,
> inspect values and much more (example is attached). We are now in a state
where this is still
> work in progress, but already at this stage it might be useful for others to
report any
> feedback.
>
> Simply upgrade the the newest develop branch of midas, and you will see two
menu items
> "OldHistory" which is the old system and "History" which is the new system. In
the new
> system, you can drag with the mouse to scroll, use the mouse wheel to zoom in
and out the
> time axis, and hover with your mouse over data points to see its value. If you
zoom out,
> old data is loaded automatically in the background.
>
> Following items are planned, but not yet implemented:
>
> - Printing of run markers as in the old history
>
> - Delete old data in the buffer to limit memory consumption if the browser
window is
> open for very long (weeks)
>
> - Implement time interval selector (clock icon, select "last day", "last 8
hours" etc.)
>
> - New settings dialog as a floating dialog box. At the moment, the setting
page of the
> old history system is used
>
> - Export / Printing / Sending to ELOG any history plot
>
> - Implement a formula for plotting data, such as "y = 12 * (x-14) +32". This
will replace
> the old "offset" and "factor" and is more flexible. The formula can be
passed directly
> to the JavaScript engine and will be executed on the web page. It should be
also
> possible to combine different channels, like the difference of two history
values.
>
> - Determine the number of digits for variable display from the axis limits.
Like if a value
> changes between 520001 and 520002 only, we need more digits than the usual 6.
>
> Many of these things will be implemented in the next weeks. If you have any
more idea
> or find some bugs, please report back to me.
>
> Best,
> Stefan for the midas team |
| Attachment 1: label_issue.png
|
|
| Attachment 2: many_labels.png
|
|
|
1672
|
01 Sep 2019 |
Nick Hastings | Forum | History plot problems for frontend with multiple indicies | Hi Ben,
thanks for your reply. I can confirm that your suggested workaround does indeed
make the problem dissapear.
I guess this issue hasn't been seen at T2K since we use MYSQL for the history.
Thanks,
Nick. |
|
Draft
|
29 Aug 2019 |
Nick Hastings | Forum | History plot problems for frontend with multiple indicies | Hi Ben,
thanks for your reply. I can confirm that your suggested workaround does indeed
make the problem dissapear.
I guess this issue hasn't been seen at T2K since we use MYSQL for the history.
Thanks,
Nick. |
|
Draft
|
29 Aug 2019 |
Nick Hastings | Forum | History plot problems for frontend with multiple indicies | Hi LCP,
thanks for the suggestion and link. Unfortunatly I don't think this explains it.
Nick. |
|
1669
|
29 Aug 2019 |
Ben Smith | Forum | History plot problems for frontend with multiple indicies | Hi Nick,
I confirm that this issue appears when using the MIDAS history driver. The issue does not appear when using the MYSQL history driver.
One solution is to give each frontend instance a different Event ID (see example code below for doing this in frontend_init). The history system did still seem to be confused by the existing FeDummy equipments/events even after making this change. However, after changing EQ_NAME from FeDummy to FeDum (i.e. starting from a clean state history-wise) things behaved normally.
I will note that some experiments definitely have a need for the "-i" method, especially those that run on distributed clusters.
Ben
```
INT frontend_init()
{
sprintf(eq_name, "%s%02d", EQ_NAME, get_frontend_index());
// Ensure each FE gets a different Event ID in the history system (951, 952 etc)
char keyname[128];
HNDLE hkey;
int status;
sprintf(keyname, "/Equipment/%s/Common/Event ID", eq_name);
status = db_find_key(hDB, 0, keyname, &hkey);
if (status != DB_SUCCESS) abort();
WORD new_ev_id = 950 + get_frontend_index();
status = db_set_data_index(hDB, hkey, &new_ev_id, 2, 0, TID_WORD);
if (status != DB_SUCCESS) abort();
return SUCCESS;
}
``` |
|
1668
|
28 Aug 2019 |
lcp | Forum | History plot problems for frontend with multiple indicies | hi,
> > That makes things more
> > complicated than needed. In the normal FE framework, you can define either several equipment
> > served by one frontend, or even one equipment linked to several devices. In the MEG experiment
> > we have one slow control frontend controlling ~100 devices without problem. In the old days
> there
> > was a problem that some slow devices could throttle the readout, but since the invention of
> multi-
> > threaded slow control equipment, each device gets its own thread so they don't block each
> other.
>
I agree with Stefan, that it's probably better to run a multi-threaded setup, than individual frontends.
The only place I've ever used the frontend index on startup is when I was testing and building
an eventbuilder.
https://midas.triumf.ca/MidasWiki/index.php/Event_Builder#Example
This might explain, why your history is swapping between frontends, as in the event builder, it gets
reconstructed.
Maybe this helps...
LCP
> Perhaps things have changed in the 10 years since the FGD SC code was written. I can do it
> differently but doing it that way seemed naturual since around 90% of the frontend code that I
> have see does it that way. |
|
1667
|
28 Aug 2019 |
Nick Hastings | Forum | History plot problems for frontend with multiple indicies | Hi Stefan,
thanks for you quick reply.
> My first question would be why are you using several font-ends at all?
Becuase I was following the model used for many of the frontends for the ND280 FGD.
> That makes things more
> complicated than needed. In the normal FE framework, you can define either several equipment
> served by one frontend, or even one equipment linked to several devices. In the MEG experiment
> we have one slow control frontend controlling ~100 devices without problem. In the old days
there
> was a problem that some slow devices could throttle the readout, but since the invention of
multi-
> threaded slow control equipment, each device gets its own thread so they don't block each
other.
Perhaps things have changed in the 10 years since the FGD SC code was written. I can do it
differently but doing it that way seemed naturual since around 90% of the frontend code that I
have see does it that way.
Nick. |
|
1666
|
28 Aug 2019 |
Stefan Ritt | Forum | History plot problems for frontend with multiple indicies | My first question would be why are you using several font-ends at all? That makes things more
complicated than needed. In the normal FE framework, you can define either several equipment
served by one frontend, or even one equipment linked to several devices. In the MEG experiment
we have one slow control frontend controlling ~100 devices without problem. In the old days there
was a problem that some slow devices could throttle the readout, but since the invention of multi-
threaded slow control equipment, each device gets its own thread so they don't block each other.
Stefan |
|
1665
|
28 Aug 2019 |
Nick Hastings | Forum | History plot problems for frontend with multiple indicies | Hello experts,
I have been writing a SC frontend for a powersupply. I have used the model
where the frontend can be started with "-i n" option so that each fe can
control a different supply. During the development/testing of the program I
would normally only run a single instance with "-i 1". However when I started
a second instance with "-i 2" I found problems with the history plots that
were being made for the original "-i 1" instance. The variable being plotted
seemed to randomly jump between the value from the "-i 1" instance and
the "-i 2" instance. confirmed that the "correct" values exist for each
frontend in the odb under /Equipment/Foo01/Variables and
/Equipment/Foo02/Variables
This is also not just a plotting artifact since I was also
able to see the two different values by running mhist.
I saw this behaviour using midas-2019-03 and also the head of the development
branch (686e4de2b55023b0d1936c60bcf4767c5e6caac0 from just under 48 hours ago).
I was able to reproduce this with a stripped down frontend that just
sets a variable that is equal to its frontend_index. Please find the code
and Makefile attached. Presumably I've done something wrong in my
implementation that hopefully a more experienced person can spot quite
quickly, but please let me know if any more information is needed.
I have seen this behaviour on both Debian 10 and on a CentOS 7 Singularity
image running on top of Debian 10.
Thanks,
Nick.
P.S. I made the topic of this post "Forum" and not "Bug Report" since I
expect the root of this problem is somewhere between the keyboard and chair. |
| Attachment 1: fedummy.cxx
|
/*******************************************************************\
Name: fetest.cxx
Created by:
Contents: Front end for testing MIDAS. Just sets a variable to
the value of the frontend index.
\********************************************************************/
#undef NDEBUG // midas required assert() to be always enabled
#include <cassert>
#include "mfe.h"
/*-- Globals -------------------------------------------------------*/
/* The frontend name (client name) as seen by other MIDAS clients */
const char *frontend_name = "fedummy";
/* The frontend file name, don't change it */
const char *frontend_file_name = __FILE__;
/* frontend_loop is called periodically if this variable is TRUE */
BOOL frontend_call_loop = FALSE;
/* a frontend status page is displayed with this frequency in ms */
INT display_period = 0;
/* maximum event size produced by this frontend */
INT max_event_size = 4*1024*1024;
INT max_event_size_frag = 4*1024*1024;
/* buffer size to hold events */
INT event_buffer_size = 10*1024*1024;
/*-- Function declarations -----------------------------------------*/
INT frontend_init();
INT frontend_exit();
INT begin_of_run(INT run_number, char *error);
INT end_of_run(INT run_number, char *error);
INT pause_run(INT run_number, char *error);
INT resume_run(INT run_number, char *error);
INT frontend_loop();
INT poll_event(INT source, INT count, BOOL test);
INT interrupt_configure(INT cmd, INT source, PTYPE adr);
int read_dummy_event(char *pevent, int off);
/*-- Equipment list ------------------------------------------------*/
#ifndef FE_NAME
#define FE_NAME "fedummy"
#endif
#ifndef EQ_NAME
#define EQ_NAME "Dummy"
#endif
#ifndef EQ_EVID
#define EQ_EVID 1
#endif
char eq_name[256];
EQUIPMENT equipment[] = {
{ EQ_NAME "%02d" , /* equipment name */
{
EQ_EVID, (1<<EQ_EVID),/* event ID, trigger mask */
"SYSTEM", /* event buffer */
EQ_PERIODIC, /* equipment type */
0, /* event source */
"MIDAS", /* format */
TRUE, /* enabled */
RO_ALWAYS, /* Read when running */
1000, /* poll every so milliseconds */
0, /* stop run after this event limit */
0, /* number of sub events */
1, /* history period */
"", "", ""
},
read_dummy_event,/* readout routine */
NULL,
NULL,
NULL, /* bank list */
}
,
{""}
};
int event_size = 10*1024;
/*-- Frontend Init -------------------------------------------------*/
#include "msystem.h"
INT frontend_init()
{
sprintf(eq_name, "%s%02d", EQ_NAME, get_frontend_index());
return SUCCESS;
}
INT frontend_exit()
{
return SUCCESS;
}
INT begin_of_run(INT run_number, char *error)
{
return SUCCESS;
}
INT end_of_run(INT run_number, char *error)
{
return SUCCESS;
}
INT pause_run(INT run_number, char *error)
{
return SUCCESS;
}
INT resume_run(INT run_number, char *error)
{
return SUCCESS;
}
INT frontend_loop()
{
return SUCCESS;
}
INT poll_event(INT source, INT count, BOOL test)
{
return 0;
}
INT interrupt_configure(INT cmd, INT source, PTYPE adr)
{
return 0;
}
INT read_dummy_event(char *pevent, INT off)
{
static HNDLE hVar = 0;
if ( hVar == 0 ) {
char str[1024];
sprintf(str, "/Equipment/%s/Variables", eq_name);
int status = db_find_key(hDB, 0, str, &hVar);
if ( status == DB_NO_KEY ) {
status = db_create_key(hDB, 0, str, TID_KEY);
if (status == DB_SUCCESS) {
status = db_find_key(hDB, 0, str, &hVar);
}
}
if (status != SUCCESS) {
cm_msg(MERROR, frontend_name, "read_dummy_event: Can\'t find or create %s, status %d, exiting", str, status);
exit(1);
}
}
const float data = static_cast<float>(get_frontend_index());
const int status = db_set_value( hDB, hVar, "Data", &data,
sizeof(float), 1, TID_FLOAT);
assert(status == DB_SUCCESS);
return 0;
}
/* emacs
* Local Variables:
* tab-width: 8
* c-basic-offset: 3
* indent-tabs-mode: nil
* End:
*/
|
| Attachment 2: Makefile
|
OSFLAGS = -DOS_LINUX -Dextname
CFLAGS = -g -O2 -fPIC -Wall -Wno-unused-function -I$(MIDASSYS)/include
CXXFLAGS = -std=c++11 $(CFLAGS)
LIBS = -lm -lz -lutil -lnsl -lpthread -lrt
MIDASLIBS = $(MIDASSYS)/lib/libmidas.a
MFE = $(MIDASSYS)/lib/mfe.o
all:: fedummy.exe
fedummy.exe: %.exe: fedummy.cxx $(MIDASLIBS) $(MFE)
$(CXX) -o $@ $(CXXFLAGS) $(OSFLAGS) $^ $(MIDASLIBS) $(LIBS) -DFE_NAME=\"fedummy\" -DEQ_NAME=\"FeDummy\" -DEQ_EVID=1
%.o: %.cxx
$(CXX) $(CXXFLAGS) $(OSFLAGS) -c $<
%.d: %.cxx
$(CXX) -MM -MD $(CXXFLAGS) $(OSFLAGS) -c $<
depend:
-include *.d
clean::
-rm -f *.o *.exe
-rm -f *.d
|
|
1664
|
14 Aug 2019 |
Konstantin Olchanski | Bug Report | ROOTANA bug? | > - ss_suspend_set_dispatch_ipc(NULL);
> + // ss_suspend_set_dispatch_ipc(NULL);
>
> This compiles and at least runs for me; so maybe that is helpful for you. But Konstantin will provide a longer term solution.
I now understand why this fix worked. Around December 2018 timeframe, I reworked the MIDAS event buffer code
and one improvement was to only send UDP buffer notifications if somebody is waiting for them. This probably
reduced to zero the probability of recursive calls to the user event handler - the problem originally fixed by the monkey
work against the midas ipc handler.
After looking at it, I now understand that the correct solution is to call ss_suspend(MSG_BM), but it turns out
inside MIDAS, handling of MSG_BM was incomplete and recursive calls to the user event handler were still
possible. (but most likely not actually happening anymore because of those changes to the event buffer code).
So.
a) ss_suspend(MSG_BM) inside midas now works correctly, recursive call to the user event handler will not happen.
b) TMidasOnline::sleep() now calls ss_suspend(MSG_BM), monkey business with ss_suspend_set_dispatch_ipc() is removed.
The problem of recursive call to the analyzer event handler is now fixed, both rootana and manalyzer (both use the same TMidasOnline code).
Read more about this here:
https://midas.triumf.ca/elog/Midas/1663
K.O. |
|
1663
|
14 Aug 2019 |
Konstantin Olchanski | Bug Fix | incorrect recursion in ss_suspend() via the user event handler | The ROOTANA midas analyzer uncovered a problem with recursive use of ss_suspend().
When running in graphical mode, the ROOT graphics main event loop was calling
ss_suspend(0) (no MSG_BM) which would sometimes call the user event handler (if a new
event arrives into the midas event buffer). Because this loop was already running in the
user event handler, there was a crash.
This is the calling sequence leading to the incorrect recursion: (from system.cxx comments
to ss_suspend())
analyzer ->
-> cm_yield() in the main loop
-> ss_suspend(0)
-> MSG_BM message arrives in the UDP socket
-> ss_suspend_process_ipc(0)
-> cm_dispatch_ipc()
-> bm_push_event()
-> bm_push_buffer()
-> bm_read_buffer()
-> bm_dispatch_event()
-> user event handler
-> user event handler ROOT graphics main loop needs to sleep
-> ss_suspend(0) <--- should be ss_suspend(MSG_BM)!!!
-> MSG_BM message arrives in the UDP socket
-> ss_suspend_process_ipc(0) <- should be ss_suspend_process_ipc(MSG_BM)!!!
-> cm_dispatch_ipc() <- without MSG_BM, calling cm_dispatch_ipc() again
-> bm_push_event()
-> bm_push_buffer()
-> bm_read_buffer()
-> bm_dispatch_event()
-> user event handler <---- called recursively, very bad!
The proper fix for this is to always call ss_suspend(MSG_BM) from the user event handler
and ss_suspend(MSG_ODB) from the user db_watch handler.
In this second case, ss_suspend(MSG_OBM) will lose/ignore/discard db_watch notifications,
if you do not want that, call ss_suspend(0) and be ready for recursive calls to your
db_watch handler. (this can happen in a frontend program that acts on changes in ODB and
where some of these actions may require sleeping via ss_suspend()).
ss_suspend(MSG_BM) will discard MSG_BM messages, which is not a problem as
bm_wait_for_events() and cm_yield() will immediately poll the event buffer and there will be
no delay in receiving new events.
Until commit 99d6e90 there were problems in ss_suspend(MSG_BM) and recursive calls to
the user event handler were still possible. It is now fixed in this and the previous commits.
K.O. |
|