ID |
Date |
Author |
Topic |
Subject |
2643
|
04 Dec 2023 |
Stefan Ritt | Forum | run number from an external (*SQL) db? |
> - how does one communicate with an external shell script from MSL ? I looked at the MIDAS Sequencer page
>
> https://daq00.triumf.ca/MidasWiki/index.php/Sequencer
>
> and didn't find an immediately obvious candidate among the MSL commands.
> The closest seems to be
>
> 'SCRIPT script [, a, b, c, ...]'
>
> but I couldn't easily figure how to propagate the output of the script back to MIDAS.
> Let say, the script creates an ASCII file with the next run number. What is the easiest
> way to import the run number into ODB? - Should an external script spawn a [short-lived]
> MIDAS client ? - That would work, but I'm almost sure there is a more straightforward solution.
The output of the SCRTIP command is stored in the variable $SCRIPT_RESULT. Please pull midas to get this
new functionality.
Stefan |
2644
|
04 Dec 2023 |
Stefan Ritt | Forum | run number from an external (*SQL) db? |
> [that was not obvious from the documentation on MIDAS wiki, and adding a couple of clarifying
> sentences there would go long ways]
I added a sentence there. Please have a look. If you like more info, please write it yourself and send it to me.
It's always better if that comes from users than from me.
> 2. if a script produces an ascii file with a known name, for example, 'a.odb', with the following two lines:
Use $SCRIPT_RESULT as described before.
Best,
Stefan |
2645
|
07 Dec 2023 |
Stefan Ritt | Info | Midas Holiday Update |
Dear beloved MIDAS users,
I'm happy to announce a "holiday update" for MIDAS. In countless hours, Zaher from
PSI worked hard to introduce syntax highlighting in the midas script editor. In
addition, there are additional features like a cleaner user interface, the option
to see all variables also in non-debug mode and more. Have a look at the picture
below, doesn't it beginning to look a lot like Christmas?
We have tested this quite a bit and went through many iterations, but no guarantee
that it's flawless. So please report any issue here.
I wish you all a happy holiday season,
Stefan |
Attachment 1: Screenshot_2023-12-08_at_08.19.48.png
|
|
2649
|
10 Dec 2023 |
Stefan Ritt | Info | Midas Holiday Update |
> If I understand the msequencer code correctly:
> Under '/Sequencer/State/Path' the path can be defined from where the msequencer gets the files, generates the xml, etc.
> However, the new javascript code reads/writes the files to '<exp>/userfiles/sequencer/'
>
> If the path in the ODB is different to '<exp>/userfiles/sequencer/', it leads to quite some unexpected behavior. If '<exp>/userfiles/sequencer/' is the place where things should go, the ODB entry of the msequencer and the internal handling should probably adopted, no?
Indeed there is a change in philosophy. Previously, /Sequencer/State/Path could point anywhere in the file system. This was considered a security problem, since one could access system files under /etc for example via the midas interface. When the new file API was
introduced recently, it has therefor been decided that all files accessible remotely should reside under <exp>/userfiles. If an experiment needs some files outside of that directory, the experiment could define some symbolic link, but that's then in the responsibility of
the experiment.
To resolve now the issue between the sequencer path and the userfiles, we have different options, and I would like to get some feedback from the community, since *all experiments* have to do that change.
1) Leave thins as they are, but explain that everybody should modify /Sequencer/Stat/Path to some subdirectory of <exp>/userfiles/sequencer
2) Drop /Sequencer/State/Path completely and "hard-wire" it to <exp>/usefiles/sequencer
3) Make /Sequencer/State/Path relative to <exp>/userfiles. Like if /Sequencer/State/Path=test would then result to a final directory <exp>/userfiles/sequencer/test
I'm kind of tempted to go with 3), since this allows the experiment to define different subdirectories under <exp>/userfiles/sequencer/... depending on the situation of the experiment.
Best,
Stefan |
2650
|
10 Dec 2023 |
Stefan Ritt | Forum | history plotting: where to convert the ADC readings into temps/voltages? |
> to plot time dependencies of the monitored detector parameters, say, voltages or temperatures,
> one needs to convert the coresponging ADC readings into floats.
>
> One could think of two ways of doing that:
>
> - one can perform the ADC-->T or ADC-->V conversion in the MIDAS frontend,
> store their [float] values in the data bank, and plot precalculated parameters vs time
>
> - one can also store in the data bank the ADC readings which typically are short's
> and convert them into floats (V's or T's) at the plotting time
>
> The first approach doubles the storage space requirements, and I couldn't find the place where
> one would do the conversion, if stored were the 16-bit ADC readings.
>
> I'm sure this issue has been thought about, so what is the "recommended MIDAS way" of performing
> the ADC -> monitored_number conversion when making MIDAS history plots ?
Most experiment go with the second method. The front-end program converts all ADC reading into physicsl
units, i.e. not only Volt, but even Degrees Centigrade or Tesla or whatever. The slow control part of
midas then puts these number into /Equipment/<name>/Variables as "float", and the history system picks
them up from there. This way your history is shown in physical units and not ADC count. Actually the
recommended slow control framework (check the examples direcotory) does not rely on data banks, but
puts values directly into the ODB. This is typically done faster, like once per second if a value
changes, rather than slow control events which are generated maybe once per 10 seconds or once per
minute. Usually the slow control values are only few compared with trigger data, so a factor of two
there does not really matter. In the MEG experiment, we have like 400 GB of slow control data per year,
but 400 TB of trigger data per year.
Best,
Stefan |
2653
|
11 Dec 2023 |
Stefan Ritt | Forum | the logic of handling history variables ? |
First of all it's important to understand that the slow control system has nothing to do
with events. So if you look at event statistics, these are the events with the slow control
data sent to the midas data file, not the history database. So the logging period (the one you
decreased from 60s to 10s to 6s) only affect the generation of events.
What is important in your case is the number of events sent to the ODB. You see these in the
screen output of the slow control frontend (see attachment). This number show increase every
second.
I tried your modification (change nulldev with a sine function), and left the sc_fe.cxx
otherwise untouched. I then started with a fresh ODB ("rm /"). Started logger, mhttpd, sc_fe
and started a run. In the attachments is what I see. So I don't understand what your problem
is. |
Attachment 1: Screenshot_2023-12-12_at_09.03.13.png
|
|
Attachment 2: Screenshot_2023-12-12_at_08.57.45.png
|
|
Attachment 3: Screenshot_2023-12-12_at_08.55.14.png
|
|
Attachment 4: Screenshot_2023-12-12_at_08.57.37.png
|
|
2654
|
12 Dec 2023 |
Stefan Ritt | Info | Midas Holiday Update |
> > 3) Make /Sequencer/State/Path relative to <exp>/userfiles. Like if /Sequencer/State/Path=test would then result to a final directory <exp>/userfiles/sequencer/test
> >
> > I'm kind of tempted to go with 3), since this allows the experiment to define different subdirectories under <exp>/userfiles/sequencer/... depending on the situation of the experiment.
> >
> > Best,
> > Stefan
>
> For me the option 3) seems the most coherent one.
> Andreas
Ok, I implemented option 3) above. This means everybody using the midas sequencer has to change /Sequencer/State/Path to an empty string and move the
sequencer files under <exp>/userfiles/sequencer as a starting point. I tested most thing, including the INCLUDE statements, but there could still be
a bug here or there, so please give it a try and report any issue to me.
Best,
Stefan |
2657
|
13 Dec 2023 |
Stefan Ritt | Forum | the logic of handling history variables ? |
> Also, I think that having a sine wave displayed by midas/examples/slowcont/scfe.cxx
> would make this example even more helpful.
Indeed. I reworked the example to have a out-of-the-box sine wave plotter, including the
automatic creation of a history panel. Thanks for the hint.
Best,
Stefan |
2660
|
15 Dec 2023 |
Stefan Ritt | Info | Implementation of custom scatter, histogram and color map plots |
Custom plots including scatter, histogram and color map plots have been
implemented. This lets you plot graphs of X/Y data or histogram data stored in the
ODB on a custom page. For some examples and documentation please go to
https://daq00.triumf.ca/MidasWiki/index.php/Custom_plots_with_mplot
Enjoy!
Stefan |
Attachment 1: plots.png
|
|
2664
|
03 Jan 2024 |
Stefan Ritt | Forum | midas.triumf.ca alias moved to daq00.triumf.ca |
> the DNS alias for midas.triumf.ca moved from old ladd00.triumf.ca to new
> daq00.triumf.ca. same as before it redirects to the MidasWiki and to the midas
> forum (elog) that moved from ladd00 to daq00 quite some time ago. if you see any
> anomalies in accessing them (broken links, bad https certificates), please report
> them to this forum or to me directly at olchansk@triumf.ca. K.O.
I found the first issue: The link to
https://midas.triumf.ca/MidasWiki/index.php/Custom_plots_with_mplot
does not work any more. The link
https://daq00.triumf.ca/MidasWiki/index.php/Custom_plots_with_mplot
however does work. Same with
https://midas.triumf.ca/MidasWiki/index.php/Sequencer
and
https://daq00.triumf.ca/MidasWiki/index.php/Sequencer
I have a few cases in mhttpd where I link directly to our documentation. I prefer
to have those link with "midas.triumf.ca" instead of "daq00.triumf.ca" in case you
change the machine again in the future.
Best,
Stefan |
2665
|
03 Jan 2024 |
Stefan Ritt | Bug Report | Compilation error on RPi |
> > git pull
> > git submodule update
>
> confirmed. just run into this myself. I think "make" should warn about out of
> date git modules. Also check that the build git version is tagged with "-dirty".
>
> K.O.
The submodule business becomes kind of annoying. I updated the documentation at
https://daq00.triumf.ca/MidasWiki/index.php/Quickstart_Linux#MIDAS_Package_Installatio
n
to tell people to use
1) "git clone ... --recurse-submodules" during the first clone
2) "git submodule update --init --recursive" in case they forgot 1)
3) "git pull --recurse-submodules" for each update or to use
4) "git config submodule.recurse true" to make the --recurse-submodules the default
I use 4) since a while and it works nicely, so one does not have to remember to pull
recursively each time.
Stefan |
2668
|
11 Jan 2024 |
Stefan Ritt | Forum | slow control frontends - how much do they sleep and how often their drivers are called? |
Put a
ss_sleep(10);
into your frontend_loop(), then you should be fine.
The event loop runs as fast as possible in order not to miss any (triggered) event, so no seep in the
event loop, because this would limit the (triggered) event rate to 100 Hz (minimum sleep is 10 ms).
Therefore, you have to slow down the event loop manually with the method described above.
Best,
Stefan |
2670
|
12 Jan 2024 |
Stefan Ritt | Forum | slow control frontends - how much do they sleep and how often their drivers are called? |
> Hi Stefan, thanks a lot !
>
> I just thought that for the EQ_SLOW type equipment calls to sleep() could be hidden in mfe.cxx
> and handled based on the requested frequency of the history updates.
Most people combine EQ_SLOW with EQ_POLLED, so they want to read out as quickly as possible. Since
the framework cannot "guess" what the users want there, I removed all sleep() in the framework.
> Doing the same in the user side is straighforward - the important part is to know where the
> responsibility line goes (: smile :)
Pushing this to the user gives you more freedom. Like you can add sleep() for some frontends, but not
for others, only when the run is stopped and more.
Stefan |
2673
|
16 Jan 2024 |
Stefan Ritt | Forum | a scroll option for "add history variables" window? |
Have you updated to the current midas version? This issue has been fixed a while ago. Below
you see a screenshot of a long list scrolled all the way to the bottom.
Revision: Thu Dec 7 14:26:37 2023 +0100 - midas-2022-05-c-762-g1eb9f627-dirty on branch
develop
Chrome on MacOSX 14.2.1
The fix is actually in "controls.js", so make sure your browser does not cache an old
version of that file. I usually have to clear my browser history to get the new file from
mhttpd.
Best regards,
Stefan |
Attachment 1: Screenshot_2024-01-17_at_08.17.30.png
|
|
2675
|
17 Jan 2024 |
Stefan Ritt | Bug Report | mhttpd eqtable |
> 1) In the attached snapshot you see that the values shown from our vacuum Pirani and Penning cells are all zero, which of course is not true.
> It would be nice to have under the equipment settings some formatting options, like the possibility to add units.
You have a
/Equipment/LEMVAC/Settings/Format Input
array where you can specify the format for every value. Default is "%f2" for two digits after the period. For vacuum levels you might want to
consider "%e3" which give you exponential format with three significant digits. The "format" setting is described at
https://daq00.triumf.ca/MidasWiki/index.php//Equipment_ODB_tree#Format_%3Cvariable%3E
and the details are at
https://daq00.triumf.ca/MidasWiki/index.php/Custom_Page#Formatting
The was a bug with the format handling, so please pull the current develop branch.
> 2) If one of the number evaluates to Infinity, the table is not shown properly anymore.
I fixed that as well in the current version.
Best,
Stefan |
Attachment 1: Screenshot_2024-01-17_at_14.09.30.png
|
|
2680
|
18 Jan 2024 |
Stefan Ritt | Forum | History tags |
This part of the system has been designed by KO, so he should reply here.
Stefan |
2681
|
18 Jan 2024 |
Stefan Ritt | Forum | mhttpd eqtable |
I fixed both in the current version, so please give it a try.
Stefan |
2683
|
22 Jan 2024 |
Stefan Ritt | Bug Report | Warnings about ODB keys that haven't been touched for 10+ years |
> What's the best way to make these messages go away?
> - Change the logic in db_validate_and_repair_key_wlocked() to not worry if keys are 10+ years old?
> - Write a script to "touch" all the old keys so they've been modified recently?
> - Something else?
The function db_validate_and_repair_key_wlocked() has been written by KO so he should reply here.
In my opinion, I would go with the first one. Changing the function is easier than to write a script
and teach everybody how to use it. This would be one more thing not to forget.
Now changing the function is not so obvious. We could extend the check to let's say 20 years, but
then we meet here again in ten years. Maybe the best choice would be to just check that the time
is not in the future.
Anyhow, most people don't realize, but we all will have fun on Jan 19, 2038, when the Unix time
overflows in 32-bit signed integers. I don't know if midas will be around by then (I will be 74 years),
but before that date one has to worry about many places in midas where we use Unix time. At that time
your date stamps from 2013 would be 25 years old, so we either remove the date check (just keep
the check of not being in the future), or extend it to 26 years.
Stefan |
2694
|
28 Jan 2024 |
Stefan Ritt | Forum | number of entries in a given ODB subdirectory ? |
I guess you won't change your FPGA configuration just when you start a run, so I don't consider the race
condition very crucial (although KO is correct, it it there).
I guess rather than any pseudo code you want to see real working code (db_get_num_entries() does not exist!), right?
The easiest these day is to ask ChatGPT. MIDAS has been open source since a long time, so it has been used
to train modern Large Language Models. Attached is the result. Here is the direct link from where you can
copy the code:
https://chat.openai.com/share/d927c78d-9914-4413-ab5e-3b0e5d173132
Please note that you never can be 100% sure that the code from a LLM is correct, so always compile and debug it.
But nevertheless, it's always easier to start from some existing code, even if there is a danger that it's not perfect.
Best,
Stefan |
Attachment 1: Screenshot_2024-01-29_at_07.20.50.png
|
|
2695
|
28 Jan 2024 |
Stefan Ritt | Bug Report | Warnings about ODB keys that haven't been touched for 10+ years |
> Please run "git blame" to find out who added that check.
OK ok, was me. But actually 2003. I hope that this being more than 20y ago excuses me not remembering it ;-)
> I think this age check should be removed, but there must be *some* check for invalid/bogus timestamps. Or
> not, we should check if MIDAS cares about timestamps at all, if ODB functions never use/look at timestamp,
> maybe we are okey with bogus timestamps. They may look funny in the odb editor, but that's it.
I changed the code to only check for timestamps more than 1h in the future and then complain. This should
avoid glitches when switching daylight savings time.
Stefan |