After some iterations, we merged the branch with the new build scheme. Now you can compile any midas program as described at

  https://bitbucket.org/tmidas/midas/pull-requests/48?link_source=email

A default CMakeLists.txt file can look like this:

cmake_minimum_required(VERSION 3.17)
project(example)
find_package(Midas REQUIRED PATHS $ENV{MIDASSYS})
add_executable(example example.cpp)
target_link_libraries(example midas::midas)


Which is much simpler than what we had before. The trick now is that the find_package() retrieves all include and link files automatically. 
There are different targets:

midas::midas              - normal midas program
midas::midas-shared       - normal midas programs using the shared midas library
midas::mfe                - old style mfe.cxx frontend
midas::mfed               - newer style frontend using mfed.cxx
midas::mscb               - programs using MSCB system
midas::drivers            - slow control program using any of the standard midas drivers

We are not absolutely sure that all midas installations will work that way, so far we have tested it on RH8, MacOSX with cmake version 
3.29.5.

Comments and bug reports are welcome as usual.

Alex and Stefan

currently to link Midas to project one has to do several steps in cmake script:
- do `find_package`
- get Midas location from MIDASSYS, or from MIDAS_LIBRARY_DIRS
- set MIDAS_INCLUDE_DIRS, MIDAS_LIBRARY_DIRS and MIDAS_LIBRARIES to your target
- add sources from Midas for mfe, drivers, etc.

in general cmake already can to all of this automatically, and the only lines you would need are:
- do `find_package(Midas ... PATHS ~/midas_install_location)`
- and do `target_link_libraries(... midas::mfe)`
  (and all include dirs, libs, and deps are propagated automatically)

see PR https://bitbucket.org/tmidas/midas/pull-requests/48
- nothing should break with current setups
- if you want to try new `midas::` targets, try to link e.g. `midas::mfed` to your frontend

Hi Pavel,

have you looked into 

  cm_set_transition_sequence()

which let's you define the sequence number for every midas client. You give any number between 1 and 1000 (default is 500 for frontends I believe).

The default value of 500 is defined in mfe.cxx:2641 where you have 500 for all four transitions, but it can be overwritten in the frontend_init function via 
cm_set_transition_sequence(). Since you have separate values for the start and stop transition, you can get the different sequencing for both transitions as you need. Like set 
all type A to 400, type B to 600 for TR_START, and set type A to 600 and type B to 400 for TR_STOP.

Since this works on the midas client level, it should also work for the tmfe.cxx framework. I'm however not sure if you have a similar default of 500 as in mfe.cxx:2641. But K.O. 
should know.

Best,
Stefan 

Hi K.O., your clarification is much appreciated! 
"
> I am not sure what you are trying to do. It is always easier to suggest a solution to a specific problem.

I think, I owe you an explanation :) :

Consider ~ 40 nodes with two FPGAs (PCIE cards) per node, talking to the detector hardware. 
One of those FPGAs, in addition to reading the data, performs the global timing synchronization.
The high-bandwidth data readout is not controlled by MIDAS, so all frontends perform only 'slow control'-type functions.
In MIDAS language, an FPGA implements two different units of slow control equipment: 
one - configuring and controlling a single FPGA (equipment type A), and another one - synchronizing 
multiple FPGAs (equipment type B). On one of the nodes, unit A and unit B share the FPGA card, 
so they better be controlled by the same frontend. 

For one, I need to make sure that all type A equipment units, managed by multiple frontends, 
are initialized before the [single] type B unit which shares the frontend with the type A unit. 
And, of course, the end of a run transition has to be handled in the opposite order - type B unit 
shuts down first. 

As 'periodic' actions for all registered pieces of equipment are performed in the same loop [thread], 
registering the equipment in the needed order - first A, then B - should give a solution - thanks for making that clear.     

> 
> 1) "time ordering of run transitions" - of course midas transitions are ordered by transition sequence numbers 
> and the tmfe class provides methods to control this. ditto for the mfe.cxx frontends.
> 
> 2) for one TMFrontend, the order of calling HandleBeginRun() is the order in which equipments were added to the 
> equipment using FeAddEquipment(). HandleEndRun() is called in reverse order. (I better check this).

the ordering of the rpc handler calls in tmfe's tr_stop/tr_pause/tr_resume functions is ok.

> 
> 3) to have multiple TMFrontends in one program would be unusual (mfe.cxx frontends completely do not support 
> this), but should work. Everything was coded to support this, but it was never tested in practice because we 
> cannot invent any useful use-case for it. HandleBeginRun() handlers are likely to be called in the frontends are 
> created. (I could check this and confirm it works, as long as you have a valid use-case for this configuration).

agreed, I don't think there is a good use case for that, so no need to spend time checking.

> 
> 4) Frontend X has EquipmentA and EquipmentB, you want EqA::HandleBeginRun() to be called at run transition 200 
> and EqB::HandleBeginRun() to be called at run transition 400.
> 
> This is not directly supported by mfe.cxx frontends (the begin_run() handler is a global function) and I did not 
> directly implement it in the TMFE frontend.
> 
> But I think this would be a useful improvement. I will look into this. 

In the simplest case, registering the equipment units in the right order is definitely the answer. 
However a single FPGA can perform multiple logically independent tasks and thus represent 
multiple logical units of equipment. Those units however are not independent: they share the hardware (FPGA) 
and thus do depend on each other. Giving users a full control over the sequence in which those logical units 
execute their run transitions is quite likely to be needed, for example, to work around peculiarities of the 
custom-made kernel drivers.
 
> 
> Likely I will add per-equipment data members fEqConfBeginRunSeqNo, fEqConfEndRunSeqno, etc. Value 0 would 
> unregister the corresponding run transition handler. This would cleanup the code quite a bit, a bunch
> of RegisterTranstionXXX functions could go away.

this also makes sense. -- thanks again, regards, Pasha

> 
> K.O.

> I have a question about "tmfe approach" to implementing MIDAS frontends. If I read the code correctly, 
> within this approach it is the TMFeEquipment things, not the TMFrontend's themselves, 
> which handle the run transitions - the TMFrontend class

that's correct and it is documented so in https://bitbucket.org/tmidas/midas/src/develop/tmfe.md

> So how does a user control the sequence in which TMFeEquipment::HandleBeginRun functions of different 
> TMFeEquipment pieces are called at begin run? - there are two cases to consider: TMFeEquipment things 
> defined by the same TMFrontend and by different TMFrontend's.

I am not sure what you are trying to do. It is always easier to suggest a solution to a specific problem.

But I will try to answer anyway:

1) "time ordering of run transitions" - of course midas transitions are ordered by transition sequence numbers 
and the tmfe class provides methods to control this. ditto for the mfe.cxx frontends.

2) for one TMFrontend, the order of calling HandleBeginRun() is the order in which equipments were added to the 
equipment using FeAddEquipment(). HandleEndRun() is called in reverse order. (I better check this).

3) to have multiple TMFrontends in one program would be unusual (mfe.cxx frontends completely do not support 
this), but should work. Everything was coded to support this, but it was never tested in practice because we 
cannot invent any useful use-case for it. HandleBeginRun() handlers are likely to be called in the frontends are 
created. (I could check this and confirm it works, as long as you have a valid use-case for this configuration).

4) Frontend X has EquipmentA and EquipmentB, you want EqA::HandleBeginRun() to be called at run transition 200 
and EqB::HandleBeginRun() to be called at run transition 400.

This is not directly supported by mfe.cxx frontends (the begin_run() handler is a global function) and I did not 
directly implement it in the TMFE frontend.

But I think this would be a useful improvement. I will look into this.

Likely I will add per-equipment data members fEqConfBeginRunSeqNo, fEqConfEndRunSeqno, etc. Value 0 would 
unregister the corresponding run transition handler. This would cleanup the code quite a bit, a bunch
of RegisterTranstionXXX functions could go away.

K.O.

Dear MIDAS experts, 

I have a question about "tmfe approach" to implementing MIDAS frontends. If I read the code correctly, 
within this approach it is the TMFeEquipment things, not the TMFrontend's themselves, 
which handle the run transitions - the TMFrontend class 

https://bitbucket.org/tmidas/midas/src/423082fb67c7711813fcda61f7cd03784c398f49/include/tmfe.h#lines-306:378

simply doesn't have methods to handle those directly. 

So how does a user control the sequence in which TMFeEquipment::HandleBeginRun functions of different 
TMFeEquipment pieces are called at begin run? - there are two cases to consider: TMFeEquipment things 
defined by the same TMFrontend and by different TMFrontend's.

Many thanks and happy holidays to everyone! 

-- regards, Pasha
 

Another modification has been done to the alarm system. 

We have often cases where an alarm is triggered on some readout noise. Like an analog voltage just over the alarm threshold for a very short period of time, triggered sometimes from environmental 
electromagnetic effects like turning on the light. 

To mitigate this problem, an "alarm trigger count" has been implemented. Every alarm has now a variable "Trigger count required". If this value is zero (the default), the alarm system works as before. If this 
value is nowever set to a non-zero value N, the alarm limit has to be met on N consecutive periods in order to trigger the alarm. Each alarm has a "Check interval" which determines the period of the alarm 
checking. If one has for example:

Check interval = 10
Trigger count required = 3

then the alarm condition has to be met for 3 consecutive periods of 10 seconds each, or for 30 seconds in total. 

The modification has been merged into the develop branch, and people have to be aware that the alarm structures in the ODB changed. The current code tries to fix this automatically, but it is important 
that ALL MIDAS CLIENTS GET RE-COMPILED after the new code is applied. Otherwise we could have "new" clients expecting the new ODB structure, and some "old" clients expecting the old structure, 
then both types of clients would fight against each other and change the ODB structure every few seconds, leading to tons of error messages. So if you pull the current develop branch, make sure to re-
compile ALL midas clients.

/Stefan

I had put in a check which limits the range to 7d into the past if you press the "play" button, but now I'm not sure why this was needed. I removed it again and 
things seem to be fine. Change is committed to develop.

Stefan

To reproduce:
- Open a history plot, click [-] a few times until the x axis shows more than 7 days.
- Scroll to the past (left)
- Click "Jump to current time" (the triangle)

Expected result:
The upper limit of the x axis is at the current time and the lower range is now - whatever range you had before 
(>7d)

Actual result:
The upper limit is the current time, the lower limit is now - 7d

(The interval seems unchanged if the range was < 7d before clicking "Jump to current time")

Dear all,

a new feature was implemented which allows to search the log messages in MIDAS. Attached one can find a more detailed explanation of how to use the feature.

If you see any issues / bugs feel don't hesitate to report them. For now the code was tested on Linux / Mac OS using Chrome, Firefox and Safari.

Best,
Marius 

We had the case in MEG that some alarms were actually just warnings, not very severe. This happens for example if we calibrate our detector 
once every other day and modify the hardware which actually triggers the alarm for about an hour or so.

The problem with this is now that the alarm sounds every few minutes, and people get annoyed during that hour. They turn down the volume 
of their speakers, or even disable the alarm sound. If the detector gets back into the default mode again, they might forget to re-enable the 
alarm, which causes some risk. 

Turning down the volume is also not good, since during that hour we could have a "real" alarm on which people have to react quickly in order 
not destroy the detector.

The art is now to configure the alarm system in a way that "normal" changes do not annoy people or cover up really severe alarms. After long 
discussions we came to following conclusion: We need a special class of alarm (let's call it 'warning') which does not annoy people. The 
warning should be visible on the screen, but not ring the alarm bell. 

While we have different alarm classes in midas, which let us customize the frequency of alarms and the screen colors, all alarms or warnings 
ring the alarm sound right now. This can be changed in the browser under "Config/Alarm sound" but that switch affects ALL alarms, which is 
not what we want.

The idea we came up with was to add a flag "Alarm sound" to the alarm classes. For the 'warning' we can then turn off the alarm sound, so 
only the banner is shown on top of the screen, and the spoken message is generated every 15 mins to remind people, but not to annoy them.

I added this "Alarm sound" flag in the branch feature/alarm_sound so everybody can test it. The downside is that all "Alarm/Classs/xxx" need 
to be modified to add this flag. While the new code will add this flag automatically (with a default value of 'true'), the size of the alarm class 
record changes by four bytes (one bool). Therefore, all running midas programs will complain about the changed size, until they get 
recompiled. 

Therefore, to test the new feature, you have to checkout the branch and  re-compile all midas programs you use, otherwise you will get errors 
like 

  Fixing ODB "/Alarms/Classes/Alarm" struct size mismatch (expected 352, odb size 348)

I will keep the branch for a few days for people to try it out and report any issue, and later merge it to develop.

Stefan

These kind of alarm conditions have been implemented and committed. The documentation at 

  https://daq00.triumf.ca/MidasWiki/index.php/Alarm_System

has been updated.

/Stefan

> Commit [develop 06735d29] improve TMFE::Sleep()

Report of test_sleep on Ubuntu-22 (Intel E-2236) and MacOS 14.6.1 (M1 MAX Mac Studio).

It is easy to see Ubuntu-22 (kernel 6.2.x) sleep granularity is ~60 usec, MacOS sleep granularity is <2 usec. (Sub 1 usec sleep likely measures the syscall() speed, 500 ns on Intel and 200 
ns on ARM M1 MAX). (NOTE: long sleep is interrupted by an alarm roughly 10 seconds into the sleep, see progs/test_sleep.cxx)

daq00:midas$ uname -a
Linux daq00.triumf.ca 6.2.0-39-generic #40~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Nov 16 10:53:04 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
daq00:midas$ ./bin/test_sleep 
test short sleep:
sleep      10 loops,   100000.000 usec per loop,  1.000000000 sec total,  1.002568007 sec actual total,   100256.801 usec actual per loop, oversleep  256.801 usec,    0.3%
sleep     100 loops,    10000.000 usec per loop,  1.000000000 sec total,  1.025897980 sec actual total,    10258.980 usec actual per loop, oversleep  258.980 usec,    2.6%
sleep    1000 loops,     1000.000 usec per loop,  1.000000000 sec total,  1.169670105 sec actual total,     1169.670 usec actual per loop, oversleep  169.670 usec,   17.0%
sleep   10000 loops,      100.000 usec per loop,  1.000000000 sec total,  1.573357105 sec actual total,      157.336 usec actual per loop, oversleep   57.336 usec,   57.3%
sleep   99999 loops,       10.000 usec per loop,  0.999990000 sec total,  6.963442087 sec actual total,       69.635 usec actual per loop, oversleep   59.635 usec,  596.4%
sleep 1000000 loops,        1.000 usec per loop,  1.000000000 sec total, 60.939687967 sec actual total,       60.940 usec actual per loop, oversleep   59.940 usec, 5994.0%
sleep 1000000 loops,        0.100 usec per loop,  0.100000000 sec total,  0.613572121 sec actual total,        0.614 usec actual per loop, oversleep    0.514 usec,  513.6%
sleep 1000000 loops,        0.010 usec per loop,  0.010000000 sec total,  0.576359987 sec actual total,        0.576 usec actual per loop, oversleep    0.566 usec, 5663.6%
test long sleep: requested 126.000000000 sec ... sleeping ...
alarm!
test long sleep: requested 126.000000000 sec, actual 126.000875950 sec

bash-3.2$ uname -a
Darwin send.triumf.ca 23.6.0 Darwin Kernel Version 23.6.0: Mon Jul 29 21:14:30 PDT 2024; root:xnu-10063.141.2~1/RELEASE_ARM64_T6000 arm64
bash-3.2$ ./bin/test_sleep 
test short sleep:
sleep      10 loops,   100000.000 usec per loop,  1.000000000 sec total,  1.032556057 sec actual total,   103255.606 usec actual per loop, oversleep 3255.606 usec,    3.3%
sleep     100 loops,    10000.000 usec per loop,  1.000000000 sec total,  1.245460033 sec actual total,    12454.600 usec actual per loop, oversleep 2454.600 usec,   24.5%
sleep    1000 loops,     1000.000 usec per loop,  1.000000000 sec total,  1.331466913 sec actual total,     1331.467 usec actual per loop, oversleep  331.467 usec,   33.1%
sleep   10000 loops,      100.000 usec per loop,  1.000000000 sec total,  1.281141996 sec actual total,      128.114 usec actual per loop, oversleep   28.114 usec,   28.1%
sleep   99999 loops,       10.000 usec per loop,  0.999990000 sec total,  1.410759926 sec actual total,       14.108 usec actual per loop, oversleep    4.108 usec,   41.1%
sleep 1000000 loops,        1.000 usec per loop,  1.000000000 sec total,  2.400593996 sec actual total,        2.401 usec actual per loop, oversleep    1.401 usec,  140.1%
sleep 1000000 loops,        0.100 usec per loop,  0.100000000 sec total,  0.188431025 sec actual total,        0.188 usec actual per loop, oversleep    0.088 usec,   88.4%
sleep 1000000 loops,        0.010 usec per loop,  0.010000000 sec total,  0.188102007 sec actual total,        0.188 usec actual per loop, oversleep    0.178 usec, 1781.0%
test long sleep: requested 126.000000000 sec ... sleeping ...
alarm!
test long sleep: requested 126.000000000 sec, actual 126.001244068 sec

K.O.

> >       status = select(1, &fdset, NULL, NULL, &timeout);
> >       On error, -1 is returned, ... timeout becomes undefined.

I confirm Linux and MacOS man pages and select() with EINTR work as I remember, Linux updates "timeout" to account for the 
time already slept, MacOS does not ("timeout" is unchanged).

So the original code is roughly correct, but long sleeps will not work right if SIGALRM fires during sleeping.

Note that MIDAS no longer uses SIGALRM to fire cm_watchdog() (it was moved to a thread) and MIDAS does not use signals,
so handling of EINTR is now moot.

(Please correct me if I missed something).

The original bug report was about EINVAL, and best I can tell, it was caused by calls to TMFE::Sleep()
with strange sleep times that caused invalid values to be computed into the select() timeout.

To improve on this, I make these changes:

1) TMFE::Sleep(0) will report an error and will not sleep
2) TMFE::Sleep(negative number) will report an error and will not sleep

(please check the sleep time before calling TMFE::Sleep())

3) TMFE::Sleep(1 sec or less) will sleep using select(). (I also looked into using poll(), ppoll() and pselect()).
4) TMFE::Sleep(more than 1 second) will use a loop to sleep in increments of 1 second and will use one additional syscall to 
read the current time to decide how much more to sleep.

5) if select() returns EINVAL, the error message will reporting the sleep time and the values in "timeout".

A side effect of this is that on both Linux and MacOS long sleeps work correctly if interrupted by SIGALRM,
because SIGALRM granularity is 1 sec and our sleep time is also 1 sec.

Commit [develop 06735d29] improve TMFE::Sleep()

K.O.

A new slow control mini-framework has been developed for MIDAS and been successfully tested in the Mu3e experiment. It 
might be suited for other experiments as well.

Background

Since the late 90’s we have the three-tier bases slow control framework in MIDAS with class drivers, device drivers and bus 
drivers. While it was used successfully since many years, it is complicated to understand and limited in its flexibility. If we 
have a HV device with a demand value, a measured voltage and a current it’s fine, but if we want to control more things like 
trip voltage, temperature and status readout etc. it soon hits its limits. With the development of the new odbxx API 
(https://daq00.triumf.ca/MidasWiki/index.php/Odbxx) there is now an opportunity to make everything much simpler.

Design principles

Instead of a three-tier system, the new “mdev” framework (“m”idas “dev”ices) uses a simple base class which is attached to 
a certain MIDAS equipment. It implements five simple functions:

- odb_setup() to setup /Equipment/<name>/Settings and /Equipment/<name>/Variables to its desired structure

- init() to initialize the slow control device

- exit() to close the connection to the device

- loop() which is called periodically to read the device

- read_event() which returns a MIDAS event going to the data stream

A device driver inherits from this base class and implements the functions. A simple example can be found in 

  midas/drivers/mdev/mdev_mscb.[h,cxx]

for the MSCB field bus system used at TRIUMF and PSI. It basically boils down to two calls:

Init:
   m_variables.connect(“/Equipment/<name>/Variables”);
   m_variables[“Output”].watch(midas::odb &o) {
      m_mscb[“HV”] = o[0]; // transfer value from ODB to MSCB device
   }

Reading a value in the loop function:
   m_variables[“Input”][0] = m_mscb[“HVMeas”];

The member variable m_variables is a midas::odb variable attached to the “Input” and “Output” variables in the ODB. The 
watch() functions executes the lambda function whenever the “Output” in the ODB changes. It then simply transfers the new 
value to the device. The reading of measured values just work in the other direction from the device to the ODB.

If you look at the mdev_mscb.cxx code, you see of course some more things like connecting to the MSCB device with proper 
error handling, looping over several devices and variables, setting up the “Setting” directory in the ODB to define labels for 
all variables. In addition we have a mirror for output variables, so that new values are only sent to the device if they differ 
from the previous variable (needed to reduce some communication traffic). 

The midas/drivers/mdev directory contains also an example frontend in the mfe.cxx framework, but this is no a requirement. 
The mdev framework can also be used in the tmfe framework and others as well. Please note how compact the frontend 
code now looks.

User interface

Since the beginning, MIDAS allows access to the the slow control devices through the “equipment” page (on the main status 
page, click on one equipment). A few more options can control now the behavior of this page, allowing quite some flexibility 
without having to write a dedicated custom page (which of course can still be done). Attached is an example from Mu3e where 
the details of the equipment display are controlled through some options in the setting subdirectory as described in 
https://daq00.triumf.ca/MidasWiki/index.php//Equipment_ODB_tree (especially the “grid display”, “Editable” and “Format” 
flags).

Conclusions

The new “mdev” framework offers a compact and effective way to communicate from MIDAS to slow control devices. Since 
all interface code is now not “hidden” any more in system class and device drivers, the user has much higher flexibility in 
controlling different devices. If a device has a new parameter, the user can add a single line of code to connect this 
parameter to an ODB entry.

The framework is very simple and misses some features of the old system. Ramping of HV voltages and current trips are not 
available in the framework (like with the old HV class driver), but modern devices usually implement this in hardware which 
is much better. The new framework is not multi-threaded, but modern devices are these day much faster than in the ‘90s. 
Since the ODB is thread save, nothing prevents us from putting a device readout into its own thread in the frontend.

We will use the new system for all devices in Mu3e, with probably some new features being added soon, so stay tuned.

/Stefan

> > Another more minor visual problem is the edit-on-start dialog. There seems to be no upper bound to the 
> > size of the text box. In the attached screenshot, ShortString has a maximum length of 32 characters, 
> > LongString has 255. Both are empty at the time of the screenshot. Maybe, the size should be limited to a 
> > reasonable width.
> 
> I limited the input size now to (arbitrarily) 100 chars. The string can still be longer than 100 chars, and you start then scrolling inside the input box. Let me know if 
> that's ok this way.

I am moving the dragon experiment to the new midas and we see this problem on the begin-of-run page.

Old midas: no horizontal scroll bar, edit-on-start names, values and comments are all squeezed in into the visible frame.

New midas: page is very wide, values entry fields are very long and there is a horizontal scroll bar.

So something got broken in the htlm formatting or sizing. I should be able to spot the change by doing
a diff between old resources/start.html and the new one.

K.O.

> > Actual result:
> > The key picker does not close.
> 
> Thanks for reporting that bug. It has been fixed in the current commit (installed already on megon02)
>

Stefan, thank you for fixing both problems, I have seen them, too, but no time to deal with them.

K.O.

The atexit() function has been implemented in the current develop branch of midas, see

  https://daq00.triumf.ca/MidasWiki/index.php/Sequencer#ATEXIT_subroutine


Stefan

> Another more minor visual problem is the edit-on-start dialog. There seems to be no upper bound to the 
> size of the text box. In the attached screenshot, ShortString has a maximum length of 32 characters, 
> LongString has 255. Both are empty at the time of the screenshot. Maybe, the size should be limited to a 
> reasonable width.

I limited the input size now to (arbitrarily) 100 chars. The string can still be longer than 100 chars, and you start then scrolling inside the input box. Let me know if 
that's ok this way.

Stefan

> Actual result:
> The key picker does not close.

Thanks for reporting that bug. It has been fixed in the current commit (installed already on megon02)

Stefan