ID |
Date |
Author |
Topic |
Subject |
2385
|
15 Apr 2022 |
Stefan Ritt | Info | New midas sequencer version |
I prepared some slides about the new features of the sequencer and post it here so
people can have a quick look at get some inspiration.
Stefan |
Attachment 1: sequencer.pdf
|
|
2394
|
02 May 2022 |
Stefan Ritt | Info | added web page for "mdump" |
Here are some of my thoughts:
- I volunteer to write the JavaScript midas bank decoder. Just a couple of pure javascript functions, no
midasio.cxx library needed.
- If different javascript connections "steal" events from each other, I would not be concerned. Actually I
would rather like that all connections see the SAME event. So mhttpd keeps one event, serves it to all
links, so displays are consistent. If a browser wants to see the "next" event, it send the old serial
number and days "please send next event AFTER serial number". If the serial number is larger than the
event in the buffer, mhttpd fetches a new event and puts it into its buffer.
- Since javascript connections are connectionless, I would rather pass event_id and trigger_mask with each
request. Then mhttpd can retrieve events until event_id and trigger_mask match, then serve that event.
Since reading events from a midas buffer is fast (many 10'000s of events per second), the won't be much of
a delay.
- GET_ALL does not make sense for browsers, you don't want to slow down any frontend. If someone wants to
do histogramming in the browser, then GET_SOME (which is kind of GET_OLD) would make sense, but most of
the cases we have some single event display, and there a GET_RECENT is most appropriate. |
2395
|
04 May 2022 |
Stefan Ritt | Info | added web pages for "show odb clients" and "show open records" |
Concerning the "scl" page, we are currently having a discussion. At the moment, one can
see midas clients in three different places:
1) the main status page at the bottom, only names and hosts are there
2) the programs page, where one can also start/stop program
3) now the new page "Show ODB clients" in the ODB editor page, which shows also the
alive status, PID and timeout
I'm thinking that three locations are two too much, so we are considering to merge the
tree pages into one. That would mean that 1) goes away, and the "Programs" page will
show more information. We have some rare cases that programs are removed from
/System/Clients in the ODB but still attached to the ODB. For those "zombies" we would
add a "hard kill" function.
I would like to hear feedback from the midas community before we proceed with the
plans. Anybody desperately in need of the programs shown on the status page?
Best,
Stefan |
2397
|
06 May 2022 |
Stefan Ritt | Info | Increased timeout for program shut down |
We had the problem in our lab that a frontend took about 6 seconds to gracefully
shut down, mainly it needed to park some motors. I found that the shutdown command
had a hard-coded timeout of 5 seconds, after which the frontend gets killed, and
cannot finish the park operation. I change the code so that the client timeout
stored in the ODB is taken instead of the hard-coded 5 seconds. This allows each
client to fine-tune its timeout, to allow graceful shutdown, but also not let the
user wait too long if the client gets stuck and needs a hard kill.
The default timeout for mfe.cxx based frontends has been changed to 10 seconds
now, but in the frontend_init function this can be changed by the user code
easily.
I hope this char does not trigger any bad side effects, but if it does, please
report here.
Stefan |
2398
|
08 May 2022 |
Stefan Ritt | Info | RO_STOPPED with triggered events |
We had issues in one of our experiment that people used RO_STOPPED in the
equipment list together with triggered events (EQ_USER). If events are sent when
a run is stopped, this leads to many unexpected results, so I added a check in
the mfe.cxx code which prevents RO_STOPPED (or RO_ALWAYS which includes
RO_STOPPED) together with EQ_TRIGGERED, EQ_INTERRUPT, EQ_MULTITHREAD and EQ_USER
type of events.
I got now complaints that some old front-end are not running any more since they
do use RO_ALWAYS together with triggered events. Can the author of these frontend
please tell me the rationale why this is needed, then I can maybe add a better
fix for that.
Stefan |
2406
|
17 May 2022 |
Stefan Ritt | Info | RO_STOPPED with triggered events |
> > > some old front-end are not running any more since they do use RO_ALWAYS together with
> > triggered events.
> >
> > I confirm, if you have mfe.c frontends that have RO_ALWAYS, after you update MIDAS,
> > some of these frontends will fail to start.
> > https://bitbucket.org/tmidas/midas/commits/1961af0d657e4f76ab9db17f9b70c0c492172b6d
> >
> > tmfe c++ frontends do not have this restriction but by default only read data when run
> > is active (per-equipment fEqConfReadOnlyWhenRunning default is true).
>
> As of commit
> https://bitbucket.org/tmidas/midas/commits/28d9c96bd6d4f65346ebcd6a04492ea764c90823 mfe.c
> frontends will no longer fail to start. an error will still be issued "Equipment \"%s\"
> contains RO_STOPPED or RO_ALWAYS. This can lead to undesired side-effect and should be
> removed."
>
> BTW 1:
>
> Some of our old frontends use EQ_MULTITHREAD to implement multithreaded periodic equipments.
> They do not generate any events when there is no run (some of them do not generate any
> events at all). Now they will start printing this error message, for no reason. (no we will
> not be rewriting them justy to get rid of this message. life is too short).
>
> BTW 2:
>
> the c++ tmfe frontend does not have any protections against these "undersired side-effects".
>
> What are these undesired side effects and should we add protection against them?
>
> K.O.
The undesired side-effects are the following: The logger tries to collect all events at the end of
the run by emptying the SYSTEM buffer. If events keep coming after the run is stopped, this loop in
the logger might be an endless loop, crashing the whole experiment in the end.
Another issue (and actually the reason for this change) is the funciton receive_trigger_event() in
mfe.cxx which will get confused if events are still coming in after a run has been stopped and
actually enters an infinite loop.
Combining EQ_MULTITHREAD with EQ_PERIODIC or EQ_SLOW is a wrong parameter combination as written in
the documentation. If one wants to have multi-threaded slow control events, one has to use the
DF_MULTITHREAD flag in the DEVICE_DRIVER structure.
Having triggered events being sent to the system after a run has been stopped I would consider
simply wrong. Why should we ever use a run start/stop if events are always flowing? Adding
protections in all places for this case is certainly much more work than just changing one flag for
frontends which produce this error message now for a wrong parameter combination. |
2413
|
20 Jun 2022 |
Stefan Ritt | Forum | Alarm on variable not updating |
There are two functions to do that, one check the last write access, the other the last write access if the run is running. The alarm condition looks like:
access(/Equipment/.../Variables/Input[10]) > 60
which will cause an alarm if the Input[10] is not written for more than 60 seconds. The other function which checks the run status as well is like:
access_running(...odb key...) > 60
You can actually see an example on the MEG alarm page.
Rather than having an alarm for that I would however recommend that you program you frontend such that it realizes if it looses connections, then tries automatically to reconnect or trigger an alarm itself (so-called "internal" alarm). This is also how the MSCB system is working and is much more robust.
Stefan |
2417
|
05 Aug 2022 |
Stefan Ritt | Info | Information for midas updates though git |
Several submodules of midas have been re-organized, so if you want to pull the
newest version, you need a
git pull --recurse-submodules
git submodule update --init --recursive
before you can build again. To do this automatically the next time, you can do
git config submodule.recurse true
which needs git 2.14 or later. I hope this works for everybody. If there is a
better way to do that (I'm not a big expert on git) please reply here.
Stefan |
2418
|
06 Aug 2022 |
Stefan Ritt | Info | Improvement of odbxx API |
While the odbxx API has been successfully used since the last months, a potential
problem with large ODBs surfaced. If you have lots of data in the ODB and load it
into an object like
midas::odb o("/Equipment");
this might take quite long, since each ODB value is fetched separately, which is
very quick on a local machine but can take long over a client-server connection.
For large experiments this can take up to minutes (!).
To get rid of this problem, the underlying object model has been modified. When an
object is instantiated like above, then the whole ODB tree is fetched in an XML
buffer in a single transfer, which even for large ODBs usually takes much less
than a second. Then the XML buffer is decomposed on the client side and converted
into the proper midas::odb objects. In one case this gave an improvement from 35
seconds to 0.5 seconds which is significant. To enable the new method, the object
can be created with a flag like
midas::odb o("/Equipment", true);
which then switches to the new method. One has to take care not to fool oneself
(like I did) by printing the object like
midas::odb o("/Equipment", true);
std::cout << o << std::endl;
because each read access to any sub-object of o causes a separate read request to
the server which again can take long. Therefore, one has to switch off the auto
refresh via
midas::odb o("/Equipment", true);
o.set_auto_refresh_read(false);
std::cout << o << std::endl;
Accessing any sub-object of o then does not cause a client-server request, which
is not necessary if all objects just have been pulled from the server before. If
one keeps the object however for a long time in memory, one has to be aware that
it only contains "old" values from the time if instantiation. If one needs more
current ODB values, the auto read refresh has to be turned on again.
Stefan |
2419
|
08 Aug 2022 |
Stefan Ritt | Info | Improvement of odbxx API |
After some thought, I changed the API again and removed the flag in the constructor,
so the system now automatically choses the best algorithm depending if the client
is connected to a local or a remote API. So in all cases you use again the old syntax:
midas::odb o("/Equipment");
Stefan |
2423
|
08 Aug 2022 |
Stefan Ritt | Info | Information for midas updates though git |
> after I set "submodule.recurse true", I still have to type "git submodule update --
> init --recursive", without --recursive, mscb/mxml is empty and the build bombs.
Indeed, doesn't work for me either. If some git guru has some more insight, please post
here!
> P.S. the underlying issue is that the mxml submodule is now included twice
> (midas/mxml and midas/mscb/mxml) and there is nothing to enforce that both copies are
> the same. (No idea what happens if the two mxml's are different).
The version of each mxml is defined by last commit of the parent repository, which contains
the hash of the submodule version. If we have to update mxml for some reason, we have to
commit also mscb with the new version, and then midas with the same version of mxml. If one
checks out midas then with
git clone https://bitbucket.org/tmidas/midas --recursive
one gets the same versions for mxml. |
2425
|
15 Aug 2022 |
Stefan Ritt | Bug Report | firefox hangs due to mhistory |
> Firefox is hanging/becoming unresponsive due to javascript code. After stopping the script manually to get firefox back in control I have the following message in the console
>
> 17:21:28.821 Script terminated by timeout at:
> MhistoryGraph.prototype.drawTAxis@http://lem03.psi.ch:8081/mhistory.js:2828:7
> MhistoryGraph.prototype.draw@http://lem03.psi.ch:8081/mhistory.js:1792:9
> mhistory.js:2828:7
>
> Any ideas how to resolve this??
I have to reproduce the problem. Can you send me the full URL from your browser when you see that problem? Probably you have some "special" axis limits, so we don't see that
problem anywhere else.
Stefan |
2429
|
17 Aug 2022 |
Stefan Ritt | Bug Report | firefox hangs due to mhistory |
The problem lies in your function mhistory_init_one() in Mudas.js:1965. You can only call "new MhistoryGraph(e)" with an element "e" which is something like
<div class="mjshistory" data-group="..." data-panel="..." data-base-u-r-l="https://host.psi.ch/?cmd=history" title="">
Please note the "data-base-u-r-l". This gets automatically added by the function mhistory_init() in mhistory.js:48. The URL is necessary sot that the upper right button in a history graph works which goes to a history page only showing the current graph.
In you function mhistory_init_one() you forgot the call
mhist.dataset.baseURL = baseURL;
where baseURL has to come from the current address bar like
let baseURL = window.location.href;
if (baseURL.indexOf("?cmd") > 0)
baseURL = baseURL.substr(0, baseURL.indexOf("?cmd"));
baseURL += "?cmd=history";
If you duplicate some functionality from mhistory.js, please make sure to duplicate it completely.
Best,
Stefan |
Draft
|
17 Aug 2022 |
Stefan Ritt | Bug Report | firefox hangs due to mhistory |
> Some of my histories are placed in an IFrame object. I eventually realized that my code fails
> when it tries to resize a history which is placed in an invisible IFrame. I resolved the issue
> by making sure that I am resizing plots only if they are in a visible IFrame.
Just to be clear: You could resolve everything on your side, or do you need to change anything in mhistory.js?
Just a tip: IFrames are not good to put anything in. I recommend just to dynamically crate a <div> element,
append it to the document body, make it floating and initially invisible. Then put all inside that div. Have
a look how control.js do it. T |
2432
|
17 Aug 2022 |
Stefan Ritt | Bug Report | firefox hangs due to mhistory |
> Some of my histories are placed in an IFrame object. I eventually realized that my code fails
> when it tries to resize a history which is placed in an invisible IFrame. I resolved the issue
> by making sure that I am resizing plots only if they are in a visible IFrame.
Just to be clear: You could resolve everything on your side, or do you need to change anything in mhistory.js?
Just a tip: IFrames are not good to put anything in. I recommend just to dynamically crate a <div> element,
append it to the document body, make it floating and initially invisible. Then put all inside that div. Have
a look how control.js do it. This takes less resources than a complete IFrame and is much easier to handle.
Stefan |
2435
|
22 Aug 2022 |
Stefan Ritt | Suggestion | mvodb functionality to get the 'LastWritten' property of a key |
> I want to read data from the ODB with the mvodb interface in one of my frontends, it's useful to know how old that data is, so I prototyped functionality in a pull request to mvodb:
>
> https://bitbucket.org/tmidas/mvodb/pull-requests/2/add-readkeylastwritten-function-to-extract
Thanks for raising that point. I realized that the odbxx API was also missing that functionality, so I added it:
https://bitbucket.org/tmidas/midas/commits/6991a92c19292eaf67721cb80f182c61db077f45
Best,
Stefan |
2443
|
24 Oct 2022 |
Stefan Ritt | Suggestion | read_only odbxx? |
> I really like the concept of the odbxx interface.
> I think it would be a nice feature if one could have a read_only connection, e.g. by declaring a "const midas::odb".
> Just for fun I tried if this already works, but the compiler doesn't allow const midas::odb for e.g. the [] operator. I'm guessing this would be non-trivial to implement, but I like the idea of certain Midas clients being able to read the odb without risking corruption.
Having a "const midas::odb" probably does not work (at least I would not know how to implement that).
But I could make an internal flag analog to the auto refresh flags. So you would have
o.set_write_protect(true);
to turn on write protection. Would that work for you?
Best,
Stefan |
2445
|
29 Oct 2022 |
Stefan Ritt | Suggestion | read_only odbxx? |
Ok, I implemented and committed that. Just call
o.set_write_protect(true)
on a key you don't want to modify. If you do so, an exception gets thrown.
Best,
Stefan |
2448
|
12 Nov 2022 |
Stefan Ritt | Bug Fix | O_CREAT in open in split.cxx |
> midas currently does not compile on linux
>
> /usr/include/x86_64-linux-gnu/bits/fcntl2.h:50:24: error: call to ‘__open_missing_mode’ declared with attribute error: open with O_CREAT or O_TMPFILE in second argument needs 3 arguments
> 50 | __open_missing_mode ();
>
> giving the mode is mandatory: https://man7.org/linux/man-pages/man2/open.2.html
>
> fix is to give open in midas/examples/lowlevel/split.cxx a default mode, e.g. 006600
Thanks. Fixed.
Stefan |
2452
|
13 Jan 2023 |
Stefan Ritt | Suggestion | Debug printf remaining in mhttpd.cxx |
These debug statements were added by K.O. on June 24, 2021. He should remove it.
https://bitbucket.org/tmidas/midas/commits/21f7ba89a745cfb0b9d38c66b4297e1aa843cffd
Best,
Stefan |