Back Midas Rome Roody Rootana
  Midas DAQ System, Page 125 of 136  Not logged in ELOG logo
ID Date Authorup Topic Subject
  2589   17 Aug 2023 Stefan RittBug Reportmidas wants to show notification?
> > This feature was asked by some people ...
> 
> "show notifications" popups are strongly associated with disreputable web sites (presumably to 
> push spam), it was surprising to see it from midas.
> 
> K.O.

I agree. But unlike emails (where you get lots of spam as well), you can nicely blacklist/whitelist 
desktop notifications. I suppress all of them except the one for MIDAS. This allows me to watch our 
experiment without staring on the web page all the time.

The main question here is maybe if the desktop notification should be on or off by default (for a 
fresh browser). While you always can change that via the mhttpd "Config" page, the default value is 
chosen by the system. I thought I put it to "on" so people can experience it, and then turn it off if 
they don't like. Having them off by default, most people never would notice this possibility. But I'm 
open to a discussion here.

Stefan
  2593   19 Aug 2023 Stefan RittBug ReportError accessing history files
Still get the same error with the latest version:

3:28 [mhttpd,ERROR] [history_schema.cxx:2913:FileHistory::read_data,ERROR] Cannot read 
'/data2/history/mhf_1692391703_20230818_hv_tc.dat', read() errno 2 (No such file or directory)

Stefan
  2598   12 Sep 2023 Stefan RittSuggestionSyntax highlighting for sequencer scripts
I like the idea of syntax highlighting, but your solution is just for one editor which not everybody
is using. It would be better if the editor built into mhttpd for MSL files would have the possibility.

I looked at highlighting in an HTML <textarea> tag, and found that we can do it with a 

<div contenteditable="true" style="font-family: monospace"> ... </div>

tag where we can change the color of individual words. If you translate your existing rules of syntax
highlighting into JavaScript, I'm happy to put that into the mhttpd sequencer editor. So I would need
a function which receives a MSL text, then replaces all keywords with some color tagging, like

ODBSET -> <span style="color:red">ODBSET</span>

Best,
Stefan
  2599   13 Sep 2023 Stefan RittForumHide start and stop buttons
Indeed the ODB settings are obsolete. Now that the status page is fully dynamic 
(JavaScript), it's much more powerful to modify the status.html page directly. You 
can not only hide the buttons, but also remove the run numbers, the running time, 
and so on. This is much more flexible than steering things through the ODB.

If there is a general need for that, I can draft a "non-run" based status page, but 
it's a bit hard to make a one-fits-all. Like some might even remove the logging 
channels and the clients, but add certain things like if their slow control front-
end is running etc.

Best,
Stefan
  2601   13 Sep 2023 Stefan RittForumHide start and stop buttons
> Hi Stefan,
> 
> > Indeed the ODB settings are obsolete.
> 
> I just applied for an account for the wiki.
> I'll try add a note regarding this change.

Please coordinate with Ben Smith at TRIUMF <bsmith@triumf.ca>, who coordinates the documentation. 


> Very true. Currently I copied the resources/midas.css into the experiment directory and appended:
> 
> #runNumberCell { display: none;}
> #runStatusStartTime { display: none;}
> #runStatusStopTime { display: none;}
> #runStatusSequencer { display: none;}
> #logChannel { display: none;}
> 
> See screenshot attached. :-)
> 
> But if feels a little clunky to copy the whole file just to add five lines.
> It might be more elegant if status.html looked for a user css file in addition
> to the default ones.

I would not go to change the CSS file. You only can hide some tables. But in a while I'm sure you
want to ADD new things, which you only can do by editing the status.html file. You don't have to
change midas/resources/status.html, but can make your own "custom status", name it differently, and
link /Custom/Default in the ODB to it. This way it does not get overwritten if you pull midas.


> The logging channels are easily removed with the css (see attachment), but it might be
> nice if the string "Run Status" table title was also configurable by css. For this
> slow control system I'd probably change it to something like "GSC Status". Again
> this is a minor thing, I could trivially do this by copying the resources/status.html
> to the experiment directory and editing it.

See above. I agree that the status.html file is a bit complicated and not so easy to understand
as the CSS file, but you can do much more by editing it.

> Lots of fun new stuff migrating from circa 2012 midas to midas-2022-05-c :-)

I always advise people to frequently pull, they benefit from the newest features and avoid the
huge amount of work to migrate from a 10 year old version.

Best,
Stefan
  2604   14 Sep 2023 Stefan RittForumHide start and stop buttons
> I believe the original "hide run start / stop" was added specifically for ND280 GSC MIDAS. I do not know 
> why it was removed. "hide pause / resume" is still there. I will restore them. Hiding logger channel 
> section should probably be automatic of there is no /logger/channels, I can check if it works and what 
> happens if there is more than one logger channel. K.O.

Very likely it was "forgotten" when the status page was converted to a dynamic page by Shouyi Ma. Since he is 
not around any more, it's up to us to adapt status.html if needed.

Stefan
  2605   15 Sep 2023 Stefan RittForumHide start and stop buttons
> I believe the original "hide run start / stop" was added specifically for ND280 GSC MIDAS. I do not know 
> why it was removed. "hide pause / resume" is still there. I will restore them. Hiding logger channel 
> section should probably be automatic of there is no /logger/channels, I can check if it works and what 
> happens if there is more than one logger channel. K.O.

Actually one thing is the functionality of the /Experiment/Start-Stop button in status.html, but the other is 
the warning we get from mhttpd:

[mhttpd,ERROR] [mhttpd.cxx:1957:init_mhttpd_odb,ERROR] ODB "/Experiment/Start-Stop Buttons" is obsolete, please 
delete it.

This was added by KO on Nov. 29, 2019 (commit 2366eefc). So we have to decide re-enable this feature (and 
remove the warning above), or keep it dropped and work on changes of status.hmtl.

Stefan
  2607   20 Sep 2023 Stefan RittBug Reportepics fe "Start Command"
Thanks for reporting this problem. It has been fixed today, so the start command is only written if it's emtpy.

Stefan
  2609   26 Sep 2023 Stefan RittSuggestionscroll when browsing for a link
> When making a link in the odb (web interface) a nice browser window pop's up. There is however not scrolling possible in the window. As a result, you can not reach a odb key if it is nested to deeply. 
> 
> Trying to type out the Link target in the field only allows for 32 characters

Thanks for reporting the bug with the pop-up not being able to scroll, I fixed that and committed the change.

I do however not understand the issue with 32 characters. The link NAME should not be more than 32 chars (which applies to all ODB keys). 
But if I try I can write more than 32 chars in the link target.

Stefan
  2610   26 Sep 2023 Stefan RittInfomjsonrpc_db_save / mjsonrpc_db_load have been dropped
The JavaScript function

mjsonrpc_db_save / mjonrpc_db_load

have been dropped from the API because they were not considered safe. Users 
should use now the new function

file_save_ascii()

and

file_load_ascii()

These function have the additional advantage that the file is not loaded 
directly into the ODB but goes into the JavaScript code in the browser, which 
can check or modify it before sending it to the ODB via mjsonrpc_db_paste(). 

Access of these functions is limited to <experiment>/userfiles/* where 
<experiment> is the normal MIDAS experiment directory defined by "exptab" or 
"MIDAS_DIR". This ensures that there is no access to e.g. system-level files. If 
you need to access a directory not under "userfile", us symbolic links.

These files can be combined with file_picker(), which lets you select files on 
the server interactively.

Stefan
  2612   01 Oct 2023 Stefan RittBug ReportODB page and hex values
Thanks for reporting this bug, I fixed it in the last commit.

Best,
Stefan
  2619   06 Oct 2023 Stefan RittInfoNew equipment display
Since a long time we tried to convert all "static" mhttpd-generated pages to 
dynamic JavaScript. With the new history panel editor we were almost there. Now I 
committed the last missing piece - the equipment display. This is shown when you 
click on some equipment on the main status page, or if you define some Alias with 

?cmd=eqtable&eq=Trigger

This is now a dynamic display, so the values change if they change in the ODB. The 
also flash briefly in yellow to visually highlight any change. In addition, these 
pages have a unit display, and some values can be edited. This is controlled by 
following settings:

/Equipment/<name>/settings/Unit <variable>

where <name> is the name of the equipment and <variable> the variable array name 
under /Equipment/<name>/Variables/<variable>

If the unit setting is not present, just a blank column is shown.

The other setting is

/Equipment/<name>/settings/Editable 

which may contain a comma-separated string of variables which can be editied on 
the equipment page.

In addition, one can save/export the equipment in a json file, which is the same 
as a ODB save of that branch. A load or import however only loads values into the 
ODB which are under the "Editable" setting above. This allows a simple editor for 
HV values etc.

Stefan
  2620   09 Oct 2023 Stefan RittInfoNew equipment display
An additional functionality has been implemented on the equipment table:

You can now select several elements by Ctrl/Shift-Click on their names, then change the 
first one. After a confirmation dialog, all selected variables are then set to the new 
value. This way one can very easily change all values to zero etc.

Stefan
Attachment 1: Screenshot_2023-10-09_at_21.56.25.png
Screenshot_2023-10-09_at_21.56.25.png
  2623   24 Oct 2023 Stefan RittForumDevice with inputs and outputs
The "multi" class driver takes care of that. It properly calls the SET and GET functions 
with the correct index. The code for that is in multi.cxx:105:

 device_driver(m_info->driver_input[i], CMD_GET,
               i - m_info->channel_offset_input[i],
               &m_info->var_input[i]);

The "channel_offset_input" and "channel_offset_output" store the first index of the 
channel in the overall ODB array (where inputs and outputs are staggered together), so 
the device_driver is always called with an index 0...n each for input and output, but 
with different commands CMD_GET and CMD_SET. You can take the mscbdev.cxx device driver 
as a working example.

Stefan
  2626   13 Nov 2023 Stefan RittForummlogger does not HAVE_ROOT
When you do "cmake .." in the build directory, you will see

-- MIDAS: Found ROOT version xxx in yyy

which will tell you that ROOT has been found. Then you should check if it has been turned off manually by doing\

ccmake ..

in the build directory. You will then see all the control variables. Make sure NO_ROOT is turned OFF (meaning ROOT is enabled).

Finally, make sure you start "rmlogger" and not "mlogger". Only "rmlogger" contains the ROOT binding.

Stefan
  2629   14 Nov 2023 Stefan RittForummlogger does not HAVE_ROOT
No, I'm not aware of this problem, but I suspect that your events somehow got corrupted. You can try the mdump utility
or the "Event Dump" web page to peek into your events, maybe you see an issue there. To give you more detailed information,
I would have to reproduce your problem, which is probably hard without your hardware.

Stefan
  2632   22 Nov 2023 Stefan RittForumPolled frontend writes data to ODB without RO_ODB
I cannot confirm that. I just tried myself with examples/experiment/frontend.cxx, removed the RO_ODB, and the trigger events did NOT get copied to the ODB.

Actually you can debug the code yourself. The relevant line is in mfe.cxx:2075:

   /* send event to ODB */
   if (pevent->data_size && (eq_info->read_on & RO_ODB)) {
      if (actual_millitime - eq->last_called > ODB_UPDATE_TIME) {
         eq->last_called = actual_millitime;
         update_odb(pevent, eq->hkey_variables, eq->format);
         eq->odb_out++;
      }
   }

so if read_on is equal 1, the function update_odb should never be called.

So the problem must be on your side.

Best,
Stefan
  2635   22 Nov 2023 Stefan RittForumrun number from an external (*SQL) db?
> - multiple subdetectors are taking test data during early commissioning 
> - a postgres db is a single sorce of run numbers.
> - test runs taken by different subsystems are assigned different [unique] run numbers and 
>   the data taken by the subsystem are identified not by the run number/dataset name , but 
>   by the run type, different for different susbsystems.

For that purpose I would not "mis-use" run numbers. Run number are meant to be incremented 
sequentially, like if you have a time-stamp in seconds since 1.1.1970 (Unix time). Intead, I 
would add additional attributes under /Experiment/Run Parameters like "Subsystem type", "Run 
mode (production/commissioning)" etc. You have much more freedom in choosing any number of 
attributes there. Then, send this attributes to your postgred db via "/Logger/Runlog/SQL/Links 
BOR". Then you can query your database to give you all runs of a certain subtype or mode.

See https://daq00.triumf.ca/MidasWiki/index.php/Logging_to_a_mySQL_database

Stefan
  2638   02 Dec 2023 Stefan RittForumrun number from an external (*SQL) db?
> Stefan, I don't think we're talking 'mis-use' - rather different subdetectors being commisisoned 
> at different locations, on an uncorrelated schedule, using independent run control (RC) instances. 
> At this point in time, we can't use a common RC instance. 
> The collected data, however, are written back into a common storage, and we need to avoid two 
> subdetectors  using the same run number. As all RC instances can connect to the same database and request a 
> run number from there, an external DB serving run numbers to multiple clients looks as a reasonable solution, 
> which provides unique run numbers for everyone. Of course, the run number gets incremented (although on the DB 
> server side), and of course different susbystems are assigned different subsystem types. 
> 
> So, in essense, it is about _where_ the run number is incremented - the RC vs the DB. 
> If there were a good strategy to implement a DB-based solution that w/o violating 
> first principles of Midas:), I'd be happy to contribute. It looks like a legitimate use case.

Ok, maybe attitude comes from the fact that I never used such a scheme in the last 30 years with midas.

If you go in this direction, there is an alternative to what Ben wrote: Use the sequencer to start a run.
The sequencer script can obtain a new run number from a central instance (e.g. by calling a shell script 
like 'curl ...' to obtain the new run number, then put it into /Runinfo/Run number as Ben wrote. This has
the advantage that the run is _started_ already with the correct number, so the history system is fine.

The script can then wait for n events, then stop the run etc. A sequencer script will also be necessary if
you want to configure your electronics (see next answer...)

Stefan
  2639   02 Dec 2023 Stefan RittForumMIDAS state machine : how to get around w/o 'configured' state?
> The MIDAS state machine doesn't seem to have a state in between 'initialized' and 
> 'running'.
> In a larger detectors with multiple subsystems, the DAQ systems often have one more state:
> after ending a previous run and before starting a new one from the 'stopped' state, 
> one needs to make sure that all subdetectors are ready, or 'configured' for the new run. 
> So theat calls for a 'configure' step during which the detector (all subsystems in 
> parallel, to save the time) transitions from 'initialized'/'stopped' to 'configured' state, 
> from which it transitions to the 'running' state.
> 
> If one of the subdetectors fails to get configured, it could be excluded from the run 
> configuration and another attempt to reconfigure the system could be made without 
> starting a new run. Or an attempt could be made to troubleshoot and configure the 
> failed subsytem individually , with the rest subsystems waiting in a 'configured' state.
> 
> How does the logic of configuring the detector for the new run is implemented in MIDAS? 
> - it is a fairly common operational procedure, so I'm sure there should be a way 
> of doing that.

We have a similar requirement in our MEG experiment. Configuring your subdetectors can
be quite complex and therefore it's almost impossible to define a 'configure' step in
the run transition system to accommodate all corner cases.

Instead of a new state, we do everything through the sequencer:

- To start a run, we start a special sequencer script. We have different scripts for
calibration runs, data runs, special runs.

- When the user starts the script, they are asked for certain parameters, like number
of events, number of runs to take, how to configure the subdetectors, which subdetectors
to read out etc.

- The script then configures the whole experiment by setting everything in the ODB for
each equipment.

- The frontends connected to their equipment get a hotlink from the ODB and start the
configuration of the trigger etc. based on the parameters from the ODB

- The progress of the configuration is indicated by the frontend by writing back the
progress (like 0...100) into the ODB

- The script now waits for the progress to reach 100. It shows the current progress
on the sequencer page, so you see exactly where we are.

- If we have several subdetectors, each of them can publish a progress, and the script
can wait for an AND of all progress, or exclude one if it fails etc. Any logic is
possible there.

- Once all progresses are at 100, the run is finally started.

- If the mechanics of configuration become more elaborate, one can 'hide' it in
sub-modules of the script.

This scheme allows us to configure very different run modes, we use it in MEG since
many years (about 0.5M runs) and it works very nice.

Attached is our main script to start a full data run. You don't have to understand
all details, but it can give you a glimpse of what it's possible with the sequencer.
The function "ApplySettings" is the one waiting for the configuration flag in the ODB
(we simply use a boolean flag there). The code is:

SUBROUTINE ApplySettings
  ODBSET "/Equipment/Trigger/Settings/Reload all", y, 1
  WAIT seconds, 2
  WAIT ODBValue, "/Equipment/Trigger/Variables/Config busy", ==, y
ENDSUBROUTINE

Best,
Stefan
Attachment 1: muegamma.msl
COMMENT "MEG Run"
INCLUDE /home/meg/online/sequencer/lib/WDAQPMT
INCLUDE /home/meg/online/sequencer/lib/WDAQMPPC
INCLUDE /home/meg/online/sequencer/lib/WDAQTC
INCLUDE /home/meg/online/sequencer/lib/WDAQCDCH
INCLUDE /home/meg/online/sequencer/lib/WDAQSystem
INCLUDE /home/meg/online/sequencer/run23/setswzerosuppression
INCLUDE /home/meg/online/sequencer/run23/xec/subscripts/ledcontrol
INCLUDE /home/meg/online/sequencer/run23/cdch/dcmon

PARAM enableDCMON, "enable DCMON readout", true, false

# Run parameters
SET Nevent, 5000
SET Nrun, infinite

# LXe Thresholds
# QH: 43.5 MeV
SET QHEnergy, 43.5
# QL: 39 MeV
SET QLEnergy, 39
# QC: 90 MeV
SET VetoEnergy, 90

# Time Window
# Narrow +-8 ns
Set NarrowThreshold, 10
# Wide +-25 ns
Set WideThreshold, 32

# Time Offset 
Set TimeOffset, 8

#make sure "Physics" is set
ODBSET "/Experiment/Run Parameters/Physics", 1, 1

#reload all default settings
CALL ReloadAll
CALL ApplySettings

#Apply Zero Suppression
CALL SetZeroSuppression

CALL SetLEDAmpl, 0.7, 4

# Trigger setting
CALL DisableAllTriggers

# 12.5 Hz of MEG trigger (12.5 Hz)
CALL EnableTrigger, 0, 1, 0

# 0.14 Hz of MEG LowQ (31 Hz)
CALL EnableTrigger, 1, 217, 0

# 0.14 Hz of MEG WideAngle (57 Hz)
CALL EnableTrigger, 2, 401, 0

# 0.14 Hz of MEG WideTime (36 Hz)
CALL EnableTrigger, 3, 253, 0

# 0.42 Hz of QH (1600 Hz)
CALL EnableTrigger, 10, 3755, 0

# 0.14 Hz of QL (3850 Hz)
CALL EnableTrigger, 11, 27100, 0

# 0.3 Hz of Cosmics (168 Hz)
CALL EnableTrigger, 15, 560, 0

# 0.1 Hz of LEDs
CALL EnableTrigger, 13, 100, 21

# 0.42 Hz of TCTrack
CALL EnableTrigger, 20, 6000000, 9

# 1 Hz of TCSingle 
#CALL EnableTrigger, 21, 1800000, 12

# 0.5Hz of Pedestal
CALL EnableTrigger, 63, 100000, 0

CALL SetHighThreshold, $QHEnergy
CALL SetLowThreshold, $QLEnergy
CALL SetVetoThreshold, $VetoEnergy

ODBGET "/Equipment/Trigger/Settings/WaveDAQ/TRG/MASTER/XecHighThreshold", QHThreshold
ODBGET "/Equipment/Trigger/Settings/WaveDAQ/TRG/MASTER/XecLowThreshold", QLThreshold
ODBGET "/Equipment/Trigger/Settings/WaveDAQ/TRG/MASTER/XecVetoThreshold", VetoThreshold
ODBSET "/Equipment/Trigger/Settings/WaveDAQ/TRG/MASTER/TimeNarrowThreshold", $NarrowThreshold
ODBSET "/Equipment/Trigger/Settings/WaveDAQ/TRG/MASTER/TimeWideThreshold", $WideThreshold
ODBSET "/Equipment/Trigger/Settings/WaveDAQ/TRG/MASTER/TcTimeOffset", $timeOffset

# Reload and wait for configuration
CALL ApplySettings

# Set "XXX Data" flags
CALL UpdateDataFlags

# Loop for runs
LOOP $Nrun

   # Set Run description
   CAT description, "MEG run. QH: " $QHThreshold, ", QL: " $QLThreshold, ", QC: " $VetoThreshold, ", TimeNarrow: ", $NarrowThreshold, ", TimeWide: ", $WideThreshold, ", Offset: ", $timeOffset
   ODBSET "/Experiment/Run Parameters/Run Description", $description, 1

   CALL dcmon, $enableDCMON

   TRANSITION start
   WAIT events, $Nevent
   TRANSITION stop

ENDLOOP

#reload settings
CALL ReloadAll
ELOG V3.1.4-2e1708b5