> Would it be possible to add something like the following?
>  343 if (!isdigit(value1_var[i]) && value1_var[i] != '.')
>  344          break;

Actually isdigit() is completely wrong here, because it also fails the minus sign and the 'E' exponent, like in -1.2E-3 

So I changed it to strchr("0123456789.+-Ee", var[i]) which should cover this case. If you put 1.2.3, it takes it as 1.2. 

Stefan

Today I fixed a long-annoying problem. We have in each front-end an equipment structure 
which defined the event id, event type, readout frequency etc. This is mapped to the ODB 
subtree

/Equipment/<name>/Common

In the past, the ODB setting took precedence over the frontend structure. We defined this 
like 25 years ago and I forgot what the exact reason was. It causes however many people 
(including myself) to fall into this trap: You change something in the front-end EQUIPMENT 
structure, you restart the front-end, but the new setting does not take effect since the 
(old) ODB value took precedence. After some debugging you find out that you have to both 
change the EQUIPMENT structure (which defines the default value for a fresh ODB) and the 
ODB value itself.

So I changed it in the current develop tree that the front-end structure takes precedence. 
You still have a hot-link, so if you want to change anything while the front-end is running 
(like the readout period), you can do that in the ODB and it takes effect immediately. But 
when you start the front-end the next time, the value from the EQUIPMENT structure is 
taken again. So please be aware of this new feature.

Happy BC day,
Stefan

Hi Stefan,

This change does not sound like a good idea to me.  I think that this change will cause just as much confusion as before; probably more since you are changing established behaviour.

It is common that MIDAS frontends usually have a Settings directory in the ODB where details about the frontend behaviour are set.  The Settings directory might get initialized from strings in the frontend code, but after initialization the Settings in the ODB have precedence and define how the frontend will behave.  Indeed, most of my custom webpages are designed to control my frontend programs through their Settings ODB tree.

So you have created a situation where Frontend/Settings in the ODB has precedence and is the main place for changing frontend behaviour; but Frontend/Common in the ODB is essentially meaningless and will get overwritten the next time the frontend restarts.  That seems likely to confuse people. 

If you really want to make this change I suggest that you delete the Frontend/Common directory entirely; or make it read-only so that people aren't fooled into changing it.

Thomas 



> Today I fixed a long-annoying problem. We have in each front-end an equipment structure 
> which defined the event id, event type, readout frequency etc. This is mapped to the ODB 
> subtree
> 
> /Equipment/<name>/Common
> 
> In the past, the ODB setting took precedence over the frontend structure. We defined this 
> like 25 years ago and I forgot what the exact reason was. It causes however many people 
> (including myself) to fall into this trap: You change something in the front-end EQUIPMENT 
> structure, you restart the front-end, but the new setting does not take effect since the 
> (old) ODB value took precedence. After some debugging you find out that you have to both 
> change the EQUIPMENT structure (which defines the default value for a fresh ODB) and the 
> ODB value itself.
> 
> So I changed it in the current develop tree that the front-end structure takes precedence. 
> You still have a hot-link, so if you want to change anything while the front-end is running 
> (like the readout period), you can do that in the ODB and it takes effect immediately. But 
> when you start the front-end the next time, the value from the EQUIPMENT structure is 
> taken again. So please be aware of this new feature.
> 
> Happy BC day,
> Stefan

Hi Thomas,

the change only affects Eqipment/<name>/common not the Equipment/<name>/Settings. 

The Common subtree is still hot-linked into the frontend, so when running things can be changed if needed. This mainly concerns the readout period of periodic events. 
Sometimes you want to change this quickly without restarting the frontend. Changing the other settings are kind of dangerous. If you change the ID of an event on the fly
you won't be able to analyze your data. So having this read-only in the ODB might be a good idea (you still need it in the ODB for the status page), except for the values
you want to change (like the readout period). 

Let's see what other people have to say.

Stefan

> Hi Stefan,
> 
> This change does not sound like a good idea to me.  I think that this change will cause just as much confusion as before; probably more since you are changing established behaviour.
> 
> It is common that MIDAS frontends usually have a Settings directory in the ODB where details about the frontend behaviour are set.  The Settings directory might get initialized from strings in the frontend code, but after initialization the Settings in the ODB have precedence and define how the frontend will behave.  Indeed, most of my custom webpages are designed to control my frontend programs through their Settings ODB tree.
> 
> So you have created a situation where Frontend/Settings in the ODB has precedence and is the main place for changing frontend behaviour; but Frontend/Common in the ODB is essentially meaningless and will get overwritten the next time the frontend restarts.  That seems likely to confuse people. 
> 
> If you really want to make this change I suggest that you delete the Frontend/Common directory entirely; or make it read-only so that people aren't fooled into changing it.
> 
> Thomas 
> 
> 
> 
> > Today I fixed a long-annoying problem. We have in each front-end an equipment structure 
> > which defined the event id, event type, readout frequency etc. This is mapped to the ODB 
> > subtree
> > 
> > /Equipment/<name>/Common
> > 
> > In the past, the ODB setting took precedence over the frontend structure. We defined this 
> > like 25 years ago and I forgot what the exact reason was. It causes however many people 
> > (including myself) to fall into this trap: You change something in the front-end EQUIPMENT 
> > structure, you restart the front-end, but the new setting does not take effect since the 
> > (old) ODB value took precedence. After some debugging you find out that you have to both 
> > change the EQUIPMENT structure (which defines the default value for a fresh ODB) and the 
> > ODB value itself.
> > 
> > So I changed it in the current develop tree that the front-end structure takes precedence. 
> > You still have a hot-link, so if you want to change anything while the front-end is running 
> > (like the readout period), you can do that in the ODB and it takes effect immediately. But 
> > when you start the front-end the next time, the value from the EQUIPMENT structure is 
> > taken again. So please be aware of this new feature.
> > 
> > Happy BC day,
> > Stefan

After some internal discussion, I decided to undo my previous change again, in order not to break existing habits. Instead, I created a new function

set_odb_equipment_common(equipment, name);

which should be called from frontend_init() which explicitly copies all data from the equipment structure in the front-end into the ODB.

Stefan

The default el6 (RHEL/SL/CentOS-6) compiler is gcc-4.4.7, it does not support c++11, not even a little bit.

Do this to install newer c++ compilers and build MIDAS with c++11:

ssh root@sl6machine
# yum install centos-release-scl-rh
# yum install devtoolset-8
# yum install cmake3
# scl -l
devtoolset-8
...

$ ssh user@sl6machine
$ scl enable devtoolset-8 bash
$ gcc -v
COLLECT_LTO_WRAPPER=/opt/rh/devtoolset-8/root/usr/libexec/gcc/x86_64-redhat-linux/8/lto-wrapper
gcc version 8.3.1 20190311 (Red Hat 8.3.1-3) (GCC) 
$ cd git/midas
$ make cclean
$ make cmake3
$ ls -l bin/odbedit

K.O.

After much discussion, and following the MIDAS workshop at TRIUMF, we made the decision to use C++11 in MIDAS.

There are many benefits, and only one drawback - no c++11 compilers in the default OS install on older computers (i.e. 
RHEL/SL/CentOS before el7). (the same applies to our use of cmake).

Specifically for el6, the solution is to use c++11 compatible gcc-8 from devtoolset-8, see 
https://midas.triumf.ca/elog/Midas/1649

The c++11 features we most welcome - initialization of class members at declaration time (no more forgetting to add initialization to 
each and every constructor), c++ threads and mutexes, lambdas and "auto".

K.O.

> Today I fixed a long-annoying problem. ...
> /Equipment/<name>/Common
> In the past, the ODB setting took precedence over the frontend structure...
> We defined this like 25 years ago and I forgot what the exact reason was.
> It causes however many people (including myself) to fall into this trap: ...

There is good number of confusions regarding entries in /eq/xxx/common:

- for some of them, the frontend code settings take precedence and overwrite settings in odb ("frontend file name")
- for some of them, ODB takes precedence and frontend code values are ignored ("read on" and "period")
- for some of them, changes in ODB take effect immediately (via db_watch) ("period")
- for some of them, frontend restart is required for changes to take effect (output event buffer name "buffer")
- some of them continuously update the odb values ("status", "status color")

I do not think there is a simple way to improve on this.

(One solution would replace the single "common" with several subdirectories, "per function",
one would have items where the code takes precedence, one would have items where odb takes
precedence (in effect, "standard settings"), one will have items that the frontend always updates
and that should not be changes via odb ("frontend name", etc). I am not sure this one solution
is necessarily an "improvement").

Lacking any ideas for improvements, I vote for the status quo. (plus a review of the documentation to ensure we have clearly
written up what each entry in "common" does and whether the user is permitted to edit it in odb).

K.O.

> Lacking any ideas for improvements, I vote for the status quo. (plus a review of the documentation to ensure we have clearly
> written up what each entry in "common" does and whether the user is permitted to edit it in odb).

I agree with that.

Stefan

During my visit at TRIUMF we rewrote the history plotting functionality of midas. Instead of 
static GIF images, we have now interactive JavaScript panels where we can scroll, zoom, 
inspect values and much more (example is attached). We are now in a state where this is still 
work in progress, but already at this stage it might be useful for others to report any 
feedback.

Simply upgrade the the newest develop branch of midas, and you will see two menu items 
"OldHistory" which is the old system and "History" which is the new system. In the new 
system, you can drag with the mouse to scroll, use the mouse wheel to zoom in and out the 
time axis, and hover with your mouse over data points to see its value. If you zoom out, 
old data is loaded automatically in the background.

Following items are planned, but not yet implemented:

- Printing of run markers as in the old history

- Delete old data in the buffer to limit memory consumption if the browser window is 
   open for very long (weeks)

- Implement time interval selector (clock icon, select "last day", "last 8 hours" etc.)

- New settings dialog as a floating dialog box. At the moment, the setting page of the 
   old history system is used

- Export / Printing / Sending to ELOG any history plot

- Implement a formula for plotting data, such as "y = 12 * (x-14) +32". This will replace 
   the old "offset" and "factor" and is more flexible. The formula can be passed directly 
   to the JavaScript engine and will be executed on the web page. It should be also 
   possible to combine different channels, like the difference of two history values.

- Determine the number of digits for variable display from the axis limits. Like if a value 
   changes between 520001 and 520002 only, we need more digits than the usual 6.

Many of these things will be implemented in the next weeks. If you have any more idea 
or find some bugs, please report back to me.

Best,
Stefan for the midas team

I like the new history system very much, but I stumbled over a couple of issues.
I used the version "Thu Aug 29 08:24:29 2019 +0200 -
midas-2019-06-b-244-gdd6585bb on branch develop":

1) it would be nice to have an option to format the label output (see attachment 1)

2) the background of a history plot is very handy if you only show one measure.
If you have multiple ones (see attachment 2), this is not the case anymore. It
would be nice if the background could be enabled/disabled.

> During my visit at TRIUMF we rewrote the history plotting functionality of
midas. Instead of 
> static GIF images, we have now interactive JavaScript panels where we can
scroll, zoom, 
> inspect values and much more (example is attached). We are now in a state
where this is still 
> work in progress, but already at this stage it might be useful for others to
report any 
> feedback.
> 
> Simply upgrade the the newest develop branch of midas, and you will see two
menu items 
> "OldHistory" which is the old system and "History" which is the new system. In
the new 
> system, you can drag with the mouse to scroll, use the mouse wheel to zoom in
and out the 
> time axis, and hover with your mouse over data points to see its value. If you
zoom out, 
> old data is loaded automatically in the background.
> 
> Following items are planned, but not yet implemented:
> 
> - Printing of run markers as in the old history
> 
> - Delete old data in the buffer to limit memory consumption if the browser
window is 
>    open for very long (weeks)
> 
> - Implement time interval selector (clock icon, select "last day", "last 8
hours" etc.)
> 
> - New settings dialog as a floating dialog box. At the moment, the setting
page of the 
>    old history system is used
> 
> - Export / Printing / Sending to ELOG any history plot
> 
> - Implement a formula for plotting data, such as "y = 12 * (x-14) +32". This
will replace 
>    the old "offset" and "factor" and is more flexible. The formula can be
passed directly 
>    to the JavaScript engine and will be executed on the web page. It should be
also 
>    possible to combine different channels, like the difference of two history
values.
> 
> - Determine the number of digits for variable display from the axis limits.
Like if a value 
>    changes between 520001 and 520002 only, we need more digits than the usual 6.
> 
> Many of these things will be implemented in the next weeks. If you have any
more idea 
> or find some bugs, please report back to me.
> 
> Best,
> Stefan for the midas team

> 1) it would be nice to have an option to format the label output (see attachment 1)

That's clearly a bug, I will fix it.

 
> 2) the background of a history plot is very handy if you only show one measure.
> If you have multiple ones (see attachment 2), this is not the case anymore. It
> would be nice if the background could be enabled/disabled.

Looking at your plot, even without the background things look messy. Please note
that you can display only a single curve by double clicking on it (and back with Escape).
If all curves are on top of each other, you can get them apart a bit by zooming
in to the vertical axis, then double click. Let ma know if that works for you.

Best regards,
Stefan

> > 2) the background of a history plot is very handy if you only show one measure.
> > If you have multiple ones (see attachment 2), this is not the case anymore. It
> > would be nice if the background could be enabled/disabled.
> 
> Looking at your plot, even without the background things look messy. Please note
> that you can display only a single curve by double clicking on it (and back with Escape).
> If all curves are on top of each other, you can get them apart a bit by zooming
> in to the vertical axis, then double click. Let ma know if that works for you.

This I found out, yet the attachment here shows another case where it would be useful to be
able to disable the background, namely if you have positive and negative measures in one
plot. Somehow it suggests that CH1 and CH2 show very different values, whereas it is only a
difference in the sign of this variables.

It's not all the important but I would like to mention this is the early stage before
everything is fully frozen.

> This I found out, yet the attachment here shows another case where it would be useful to be
> able to disable the background, namely if you have positive and negative measures in one
> plot. Somehow it suggests that CH1 and CH2 show very different values, whereas it is only a
> difference in the sign of this variables.

Ok, I added 

- a correction which does the fill not to the bottom of the window, but only to the y=0 axis.
- a flag "Show graph fille" which lets you turn on and off the filling for each plot

Best,
Stefan

> This I found out, yet the attachment here shows another case where it would be useful to be
> able to disable the background, namely if you have positive and negative measures in one
> plot. Somehow it suggests that CH1 and CH2 show very different values, whereas it is only a
> difference in the sign of this variables.

Ok, I added an option which lets you switch off the background. 

I also changed the background drawing such that it only goes to the y=0 axis, not the bottom of the screen. 
That should help displaying negative values.

Stefan

> 1) it would be nice to have an option to format the label output (see attachment 1)

I fixed that in the current version.

Stefan

Our typical use case is that a lot of people are connected to the experiment
having some history tabs open most of the time. Hence, I setup a test system and
connect to it from all kind of systems/browsers. What I see currently quite
often is the error hs_read_arraybuffer (see the attachement).

For firefox 60.8.0esr this can result into a full freeze of the tab and only
closing it is possible.

For chromium based browsers you eventually get a popup informing that it is not
responsive anymore.

The worst though is safari 12.1.2 which not only freezes the tab, but
reproducibly crashes the mhttpd on the server side.

Are there ways to get a log which would document where the problems start?  

<html>
<head>
   <title>Test</title>
   <link rel="stylesheet" href="midas.css">
   <script src="controls.js"></script>
   <script src="midas.js"></script>
   <script src="mhttpd.js"></script>
</head>
<body class="mcss" onload="mhttpd_init('Test');">

<div id="mheader"></div>
<div id="msidenav"></div>

<div id="mmain">
  <div name="mhistory" data-group="Test" data-panel="Test" data-scale="1m" style="width:600px;border:1px solid black;"></div>
</div>
</body>
</html>