02 Jan 2024, Konstantin Olchanski, Forum, midas.triumf.ca alias moved to daq00.triumf.ca
|
the DNS alias for midas.triumf.ca moved from old ladd00.triumf.ca to new
daq00.triumf.ca. same as before it redirects to the MidasWiki and to the midas
forum (elog) that moved from ladd00 to daq00 quite some time ago. if you see any
anomalies in accessing them (broken links, bad https certificates), please report
them to this forum or to me directly at olchansk@triumf.ca. K.O. |
03 Jan 2024, Stefan Ritt, Forum, midas.triumf.ca alias moved to daq00.triumf.ca
|
> the DNS alias for midas.triumf.ca moved from old ladd00.triumf.ca to new
> daq00.triumf.ca. same as before it redirects to the MidasWiki and to the midas
> forum (elog) that moved from ladd00 to daq00 quite some time ago. if you see any
> anomalies in accessing them (broken links, bad https certificates), please report
> them to this forum or to me directly at olchansk@triumf.ca. K.O.
I found the first issue: The link to
https://midas.triumf.ca/MidasWiki/index.php/Custom_plots_with_mplot
does not work any more. The link
https://daq00.triumf.ca/MidasWiki/index.php/Custom_plots_with_mplot
however does work. Same with
https://midas.triumf.ca/MidasWiki/index.php/Sequencer
and
https://daq00.triumf.ca/MidasWiki/index.php/Sequencer
I have a few cases in mhttpd where I link directly to our documentation. I prefer
to have those link with "midas.triumf.ca" instead of "daq00.triumf.ca" in case you
change the machine again in the future.
Best,
Stefan |
03 Jan 2024, Konstantin Olchanski, Forum, midas.triumf.ca alias moved to daq00.triumf.ca
|
> I found the first issue: The link to
> https://midas.triumf.ca/MidasWiki/index.php/Custom_plots_with_mplot
fixed.
https://midas.triumf.ca/Custom_plots_with_mplot
also works.
I tried to get rid of redirect to daq00 completely and make the whole MidasWiki show up
under midas.triumf.ca, but discovered/remembered that I cannot do this without changing
MidasWiki config [$wgServer = "https://daq00.triumf.ca";] which causes mediawiki to
redirect everything to daq00 (using the 301 "moved permanently" reply, ouch!). In theory,
if I change it to "https://midas.triumf.ca" it will redirect everything there instead,
but I am hesitant to make this change. It has been like this since forever and I have no
idea what else will break if I change it.
K.O. |
10 Jan 2024, Pavel Murat, Forum, slow control frontends - how much do they sleep and how often their drivers are called?
|
Dear all,
I have implemented a number of slow control frontends which are directed to update the
history once in every 10 sec, and they do just that.
I expected that such frontends would be spending most of the time sleeping and waking up
once in ten seconds to call their respective drivers and send the data to the server.
However I observe that each frontend process consumes almost 100% of a single core CPU time
and the frontend driver is called many times per second.
Is that the expected behavior ?
So far, I couldn't find the place in the system part of the frontend code (is that the right
place to look for?) which regulates the frequency of the frontend driver calls, so I'd greatly
appreciate if someone could point me to that place.
I'm using the following commit:
commit 30a03c4c develop origin/develop Make sure line numbers and sequence lines are aligned.
-- many thanks, regards, Pasha |
11 Jan 2024, Stefan Ritt, Forum, slow control frontends - how much do they sleep and how often their drivers are called?
|
Put a
ss_sleep(10);
into your frontend_loop(), then you should be fine.
The event loop runs as fast as possible in order not to miss any (triggered) event, so no seep in the
event loop, because this would limit the (triggered) event rate to 100 Hz (minimum sleep is 10 ms).
Therefore, you have to slow down the event loop manually with the method described above.
Best,
Stefan |
11 Jan 2024, Pavel Murat, Forum, slow control frontends - how much do they sleep and how often their drivers are called?
|
Hi Stefan, thanks a lot !
I just thought that for the EQ_SLOW type equipment calls to sleep() could be hidden in mfe.cxx
and handled based on the requested frequency of the history updates.
Doing the same in the user side is straighforward - the important part is to know where the
responsibility line goes (: smile :)
-- regards, Pasha |
12 Jan 2024, Stefan Ritt, Forum, slow control frontends - how much do they sleep and how often their drivers are called?
|
> Hi Stefan, thanks a lot !
>
> I just thought that for the EQ_SLOW type equipment calls to sleep() could be hidden in mfe.cxx
> and handled based on the requested frequency of the history updates.
Most people combine EQ_SLOW with EQ_POLLED, so they want to read out as quickly as possible. Since
the framework cannot "guess" what the users want there, I removed all sleep() in the framework.
> Doing the same in the user side is straighforward - the important part is to know where the
> responsibility line goes (: smile :)
Pushing this to the user gives you more freedom. Like you can add sleep() for some frontends, but not
for others, only when the run is stopped and more.
Stefan |
15 Jan 2024, Frederik Wauters, Forum, dump history FILE files
|
We switched from the history files from MIDAS to FILE, so we have *.dat files now (per variable), instead of the old *.hst.
How shoul
d one now extract data from these data files? With the old *,hst files I can e.g. mhdump -E 102 231010.hst
but with the new *.dat files I get
...2023/history$ mhdump -E 0 -T "Run number" mhf_1697445335_20231016_run_transitions.dat | head -n 15
event name: [Run transitions], time [1697445335]
tag: tag: /DWORD 1 4 /timestamp
tag: tag: UINT32 1 4 State
tag: tag: UINT32 1 4 Run number
record size: 12, data offset: 1024
record 0, time 1697557722, incr 112387
record 1, time 1697557783, incr 61
record 2, time 1697557804, incr 21
record 3, time 1697557834, incr 30
record 4, time 1697557888, incr 54
record 5, time 1697558318, incr 430
record 6, time 1697558323, incr 5
record 7, time 1697558659, incr 336
record 8, time 1697558668, incr 9
record 9, time 1697558753, incr 85
not very intelligible
Yes, I can do csv export on the webpage. But it would be nice to be able to extract from just the files. Also, the webpage export only saves the data shown ( range limited and/or downsampled) |
16 Jan 2024, Pavel Murat, Forum, a scroll option for "add history variables" window?
|
Dear all,
I have a "slow control" frontend which reads out 100 slow control parameters.
When I'm interactively adding a parameter to a history plot,
a nice "Add history variable" pops up .. , but with 100 parameters in the list,
it doesn't fit within the screen...
The browser becomes passive, and I didn't find any easy way of scrolling.
In the attached example, adding a channel 32 variable becomes rather cumbersome,
not speaking about channel 99.
Two questions:
a) how do people get around this "no-scrolling" issue? - perhaps there is a workaround
b) how big of a deal is it to add a scroll bar to the "Add history variables" popup ?
- I do not know javascript myself, but could find help to contribute..
-- many thanks, regards, Pasha |
16 Jan 2024, Stefan Ritt, Forum, a scroll option for "add history variables" window?
|
Have you updated to the current midas version? This issue has been fixed a while ago. Below
you see a screenshot of a long list scrolled all the way to the bottom.
Revision: Thu Dec 7 14:26:37 2023 +0100 - midas-2022-05-c-762-g1eb9f627-dirty on branch
develop
Chrome on MacOSX 14.2.1
The fix is actually in "controls.js", so make sure your browser does not cache an old
version of that file. I usually have to clear my browser history to get the new file from
mhttpd.
Best regards,
Stefan |
17 Jan 2024, Francesco Renga, Forum, History tags
|
Dear experts,
I would like to have some clarification about the meaning and use of the
tags in the ODB under /History/Tags.
I noticed that, if a history plot is created, but the name of the corresponding
variable is changed later and the plot is modified accordingly, the old name
persists in the /History/Tags list along with the new one. So, it appears in the
list of variables when a new history plot is created.
It seems not to compromise the functionalities of the history system, but it is
prone to create confusion.
Is it the expected behavior? What is the correct procedure to follow if the name
of a variable has to be changed?
Thank you,
Francesco |
17 Jan 2024, Pavel Murat, Forum, a scroll option for "add history variables" window?
|
> Have you updated to the current midas version? This issue has been fixed a while ago.
Hi Stefan, thanks a lot! I pulled from the head, and the scrolling works now. -- regards, Pasha |
18 Jan 2024, Andreas Suter, Forum, mhttpd eqtable
|
I have two more questions related to Units, Format for Equipment/Settings:
1) It looks as if I can have units per channel only for the Input/Output channels but not for Demand/Measured channels.
For instance we do have HV FE which collect devices with kV and V demand settings. It looks like this is not possible (see attachment) to have per channel units.
Is this right, or do I miss something here?
2) This new functionality needs entries under /Equipment/<eq-name>/Settings. The class driver generates the necessary structures if they are missing at the startup
of the scfe. It would be nice that the new, additional entries would be generate as well: Editable, Unit Input, Unit Format, etc. Perhaps optionally, if a DD is providing it?
Best,
Andreas |
18 Jan 2024, Stefan Ritt, Forum, History tags
|
This part of the system has been designed by KO, so he should reply here.
Stefan |
18 Jan 2024, Stefan Ritt, Forum, mhttpd eqtable
|
I fixed both in the current version, so please give it a try.
Stefan |
28 Jan 2024, Pavel Murat, Forum, number of entries in a given ODB subdirectory ?
|
Dear MIDAS experts,
- I have a detector configuration with a variable number of hardware components - FPGA's receiving data
from the detector. They are described in ODB using a set of keys ranging
from "/Detector/FPGAs/FPGA00" .... to "/Detector/FPGAs/FPGA68".
Each of "FPGAxx" corresponds to an ODB subdirectory containing parameters of a given FPGA.
The number of FPGAs in the detector configuration is variable - [independent] commissioning
of different detector subsystems involves different number of FPGAs.
In the beginning of the data taking one needs to loop over all of "FPGAxx",
parse the information there and initialize the corresponding FPGAs.
The actual question sounds rather trivial - what is the best way to implement a loop over them?
- it is certainly possible to have the number of FPGAs introduced as an additional configuration parameter,
say, "/Detector/Number_of_FPGAs", and this is what I have resorted to right now.
However, not only that loooks ugly, but it also opens a way to make a mistake
and have the Number_of_FPGAs, introduced separately, different from the actual number
of FPGA's in the detector configuration.
I therefore wonder if there could be a function, smth like
int db_get_n_keys(HNDLE hdb, HNDLE hKeyParent)
returning the number of ODB keys with a common parent, or, to put it simpler,
a number of ODB entries in a given subdirectory.
And if there were a better solution to the problem I'm dealing with, knowing it might be helpful
for more than one person - configuring detector readout may require to deal with a variable number
of very different parameters.
-- many thanks, regards, Pasha |
28 Jan 2024, Konstantin Olchanski, Forum, number of entries in a given ODB subdirectory ?
|
Very good question. It exposes a very nasty problem, the race condition between "ls" and "rm". While you are
looping over directory entries, somebody else is completely permitted to remove one of the files (or add more
files), making the output of "ls" incorrect (contains non-existant/removed files, does not contain newly added
files). even the simple count of number of files can be wrong.
Exactly the same problem exists in ODB. As you loop over directory entries, some other ODB client can remove or
add new entries.
To help with this, I considered adding an db_ls() function that would take the odb lock, atomically iterate over
a directory and return an std::vector<std::string> with names of all entries. (current odb iterator returns ODB
handles that may be invalid if corresponding entry was removed while we were iterating). Unfortunately the
delete/add race condition remains, some returned entries may be invalid or missing.
For your specific application, you can swear that you will never add/delete files "at the wrong time", and you
will not see this problem until one of your users writes a script that uses odbedit to add/remove subdirectory
entries exactly at the wrong time. (you run your "ls" in the BeginRun() handler of your frontend, they run their
"rm" from their's, so both run at the same time, a race condition.
Closer to your question, I think it is simplest to always iterate over the subdirectory, collect names of all
entries, then work with them:
std::vector<std::string> names;
iterate over odb {
names.push_back(name);
}
foreach (name in names)
work_on(name);
instead of:
size_t n = db_get_num_entries();
for (size_t i=0; i<n; i++) {
std::string name = sprintf("FPGA%d", i);
work_on(name);
}
K.O.
> Dear MIDAS experts,
>
> - I have a detector configuration with a variable number of hardware components - FPGA's receiving data
> from the detector. They are described in ODB using a set of keys ranging
> from "/Detector/FPGAs/FPGA00" .... to "/Detector/FPGAs/FPGA68".
> Each of "FPGAxx" corresponds to an ODB subdirectory containing parameters of a given FPGA.
>
> The number of FPGAs in the detector configuration is variable - [independent] commissioning
> of different detector subsystems involves different number of FPGAs.
>
> In the beginning of the data taking one needs to loop over all of "FPGAxx",
> parse the information there and initialize the corresponding FPGAs.
>
> The actual question sounds rather trivial - what is the best way to implement a loop over them?
>
> - it is certainly possible to have the number of FPGAs introduced as an additional configuration parameter,
> say, "/Detector/Number_of_FPGAs", and this is what I have resorted to right now.
>
> However, not only that loooks ugly, but it also opens a way to make a mistake
> and have the Number_of_FPGAs, introduced separately, different from the actual number
> of FPGA's in the detector configuration.
>
> I therefore wonder if there could be a function, smth like
>
> int db_get_n_keys(HNDLE hdb, HNDLE hKeyParent)
>
> returning the number of ODB keys with a common parent, or, to put it simpler,
> a number of ODB entries in a given subdirectory.
>
> And if there were a better solution to the problem I'm dealing with, knowing it might be helpful
> for more than one person - configuring detector readout may require to deal with a variable number
> of very different parameters.
>
> -- many thanks, regards, Pasha |
28 Jan 2024, Konstantin Olchanski, Forum, History tags
|
> This part of the system has been designed by KO, so he should reply here.
That's right. Some of this stuff is historical gibberish that is no longer needed
for FILE and SQL histories.
/History/Events is needed to create persistent mapping between history event names
and history event id's (at some point history event id was same equipment event
id, with the obvious problems when equipment event ids are duplicated, reused,
renamed, deleted).
/History/Tags was used by the history editor to speed up "give me all tag names
for this history event name". With the "MIDAS" history storage this required
reading a lot of data from disk. With the "FILE" history and cached ZFS SSD, disk
access is much cheaper and caching history event names and tags in odb is no
longer necessary.
/History/Tags should probably be removed (be check that nobody uses it first).
/History/Events has to remain as long as "MIDAS" history storage is still used.
K.O. |
28 Jan 2024, Konstantin Olchanski, Forum, a scroll option for "add history variables" window?
|
> > Have you updated to the current midas version? This issue has been fixed a while ago.
>
> Hi Stefan, thanks a lot! I pulled from the head, and the scrolling works now. -- regards, Pasha
Right, I remember running into this problem, too.
If you have some ideas on how to better present 100500 history variables, please shout out!
K.O. |
28 Jan 2024, Konstantin Olchanski, Forum, dump history FILE files
|
$ cat mhf_1697445335_20231016_run_transitions.dat
event name: [Run transitions], time [1697445335]
tag: tag: /DWORD 1 4 /timestamp
tag: tag: UINT32 1 4 State
tag: tag: UINT32 1 4 Run number
record size: 12, data offset: 1024
...
data is in fixed-length record format. from the file header, you read "record size" is 12 and data starts at offset 1024.
the 12 bytes of the data record are described by the tags:
4 bytes of timestamp (DWORD, unix time)
4 bytes of State (UINT32)
4 bytes of "Run number" (UINT32)
endianess is "local endian", which means "little endian" as we have no big-endian hardware anymore to test endian conversions.
file format is designed for reading using read() or mmap().
and you are right mhdump, does not work on these files, I guess I can write another utility that does what I just described and spews the numbers to stdout.
K.O. |
|