Back Midas Rome Roody Rootana
  Midas DAQ System, Page 116 of 136  Not logged in ELOG logo
ID Date Author Topic Subjectdown
  173   04 Nov 2004 Stefan RittForumFrontend code and the ODB
Hi Jan,

I usually keep under /Experiment/Run Parameters only those settings which are kind of "global" and thus of
interest to frontend *and* analyzer, like a run mode (data/calibration/cosmic/...). Settings more specific to a
frontend I keep under /Equipment/<name>/Settings where <name> is the equipment name the specific frontend
produces. In your case each frontend will then get its own tree (related to each fragment). Please note that
both discussed trees can contain a whole tree with subdirectories, which lets you organize your data better.

Best regards, Stefan.
  598   24 Jun 2009 Razvan Stefan GorneaForumFrontend and manual trigger question
Hi,

I have a question related to the frontend and I would need some suggestions
about the proper way of doing things in Midas.

I have some CAEN ADC boards and a VME interface and I made a simple frontend
that configures and reads the system and it works great ... Now I would like to
add a feature and it seems to me I am going the wrong way.

I would like to add manual trigger capability and so I added the EQ_MANUAL_TRIG
flag to the "CAEN" equipment type but the problem is that the framework calls
directly the readout function on "Midas manual trigger". To trigger manually the
CAEN ADC's I have to write some registers and therefore I either need to have a
function called before the readout function or be able in the readout function
to know if the call has been triggered by the poll function or "Midas manual
trigger". I tried to check the value *((DWORD *)pevent) but it seems to be a
well defined and meaningful value only when the readout function call is
triggered by the poll function.

So my question is what's the proper "Midas way" of doing this? Should I create a
new equipment which is of EQ_MANUAL_TRIG type and its readout function writes
the registers on the CAEN ADC's to trigger manually the boards? Is there a way
of "mapping" the Midas manual trigger to a "trigger generator function"? Because
I am a little bit confused ... Is the Midas manual trigger on the new equipment
(let's say "Manual trigger manager") going to increment the event ID? Then when
the event is really read through the readout function of the "CAEN" equipment
the event ID is going to be incremented again obviously ... 

Thanks a lot,
Razvan
  600   25 Jun 2009 Stefan RittForumFrontend and manual trigger question
> I would like to add manual trigger capability and so I added the EQ_MANUAL_TRIG
> flag to the "CAEN" equipment type but the problem is that the framework calls
> directly the readout function on "Midas manual trigger". To trigger manually the
> CAEN ADC's I have to write some registers and therefore I either need to have a
> function called before the readout function or be able in the readout function
> to know if the call has been triggered by the poll function or "Midas manual
> trigger". I tried to check the value *((DWORD *)pevent) but it seems to be a
> well defined and meaningful value only when the readout function call is
> triggered by the poll function.

Actually there is no way to figure out if your readout function is called normally or 
manually triggered. So I modified the framework to add this functionality. In your 
readout routing you can how call

  flag = DATA_SIZE(pevent);

If flag is zero, this is a normal call, if it's one, it's a manual trigger. To get 
this functionality, you have to update midas.h and mfe.c from the repository (rev. 
4519).
  440   19 Feb 2008 Petr NomokonovInfoFrontend - Backend c onnection
Backend computer with SLC4.4 Linux did'not work as mserver because some security
protection under iptables service (could not connect with frontend computers).
The connection established if to make ( under root ) iptables disable
by command: service iptables stop, or much more gently
just to accept mserver port with command (under root):
iptables -I INPUT -p tcp --dport 1175 -j ACCEPT
( in /etc/service 
  midas    1175/tcp       #Midas server)
To check which ports is open
it is possible to use the command: "netstat -n" to see digital numbers of ports.
  332   26 Jan 2007 Carl MetelkoForumFront end electronics broadcast data over ethernet, can midas read this in
Hi,
   the system I'm building will have data read into the frontend nodes
via ethernet (optic). Is this possible?>
  352   26 Feb 2007 Stefan RittInfoFragmented polled events
Fragmented polled events have been implemented in SVN revision 3625.
Fragmentation is a method of breaking down large (>MB) events into smaller
pieces and send them through the shared memory buffers, reassembling them at the
output. In the past this was only possible for periodic events (such as large
histograms read out once every few seconds), but now this is also possible for
polled events.
  300   05 Sep 2006 Konstantin OlchanskiForumForums moved from dasdevpc.triumf.ca to ladd00.triumf.ca
For the record, the MIDAS (& co) forums have been physically moved from
dasdevpc.triumf.ca to our new server machine ladd00.triumf.ca. This change
should be transparent to all users, but if anything stops working, please let me
know at olchansk-at-triumf-dot-ca. K.O.
  1799   29 Jan 2020 Pintaudi GiorgioInfoForce triggering of idle routine of a frontend
Hello!
As you know, the generic MIDAS frontend has a class driver, device driver, bus driver
structure. Assuming a slow device frontend, its class driver should have a routine of type
INT idle (EQUIPMENT * pequipment)
This routine is called with a rate controlled by the
"/Equipment/<frontend name>/Common/Event limit" parameter.
The idle routine usually reads one channel of the frontend and stores the results
in the "/Equipment/<frontend name>/Variables" ODB folder.

My question is: it is possible to force (from the code) the frontend to call the idle routine at a
certain point. This is because I need to update the "/Equipment/<frontend name>/Variables"
variables inside the "begin_of_run" routine, at a very specific time.


One dirty solution would be to increase a lot the reading rate ... but I need this
increased reading rate only during the run start while I need a low reading rate
during the run. So the question: is it possible to increase and decrease the reading
rate (event limit) of a frontend without stopping and restarting it?

If you need more info, please let me know.
Thank you
Giorgio
  1802   02 Feb 2020 Konstantin OlchanskiInfoForce triggering of idle routine of a frontend
Hi, Giorgio - I think you encountered a fundamental problem with what to do at the begin of 
run. There are two ways of thinking about it.

Some experiments want to start the run as quickly as possible, so they do not want 
begin_of_run() to do too much stuff.

Other experiments want to record all the current settings and conditions before starting a 
run, their begin_of_run() will read all the slow controls, interrogate all the power supplies, 
read all the voltages, temperatures, pressures, etc. By necessity this will slow down the 
starting of the run quite significantly.

The best I understand the midas class driver structure, it is more geared for the first case - 
fast starting of runs.

The thinking behind this choice considers the nature of most slow control data in typical 
physics experiments:
- if the data does not change quickly (say, room temperature, atmospheric pressure, etc), 
and you read it say every 1 minute, then you do not need to read it again at begin run time - 
the 1 minute old measurement is still good enough - nothing changed much since then
- if the opposite is true, the data changes wildly (i.e. detector high voltage current goes up 
and down in response to the quickly changing beam current), measuring it at the start of 
the run does us no good - by the time the first event comes around, it has already changed 
completely.

Hopefully Stefan can help you with your specific problem, he has better understanding of 
the midas class drivers.


K.O.


[quote="Pintaudi Giorgio"]Hello!
As you know, the generic MIDAS frontend has a class driver, device driver, bus driver
structure. Assuming a slow device frontend, its class driver should have a routine of type   
[CODE]INT idle (EQUIPMENT * pequipment)[/CODE]
This routine is called with a rate controlled by the 
"[I]/Equipment/<frontend name>/Common/Event limit[/I]" parameter.
The idle routine usually reads one channel of the frontend and stores the results
in the "[I]/Equipment/<frontend name>/Variables[/I]" ODB folder.

[B]My question is: it is possible to force (from the code) the frontend to call the idle routine 
at a 
certain point. This is because I need to update the "[I]/Equipment/<frontend 
name>/Variables[/I]"
variables inside the "[I]begin_of_run[/I]" routine, at a very specific time.[/B]

One dirty solution would be to increase a lot the reading rate ... but I need this
increased reading rate only during the run start while I need a low reading rate
during the run. So the question: is it possible to increase and decrease the reading
rate (event limit) of a frontend without stopping and restarting it?

If you need more info, please let me know.
Thank you
Giorgio[/quote]
  1805   02 Feb 2020 Pintaudi GiorgioInfoForce triggering of idle routine of a frontend
Dear Konstantin,
thank you very much for the explanation. I already have an idea of how to solve my problem by bypassing the class driver altogether or by slightly modifying the mfe.cxx frontend.
But either way is not very elegant. If there was a way to do what I need easily and without writing much code, I would obviously choose that.
So let us wait for Stefan opinion!
Thanks again
Giorgio


Quote:
> Hi, Giorgio - I think you encountered a fundamental problem with what to do at the begin of
> run. There are two ways of thinking about it.
>
> Some experiments want to start the run as quickly as possible, so they do not want
> begin_of_run() to do too much stuff.
>
> Other experiments want to record all the current settings and conditions before starting a
> run, their begin_of_run() will read all the slow controls, interrogate all the power supplies,
> read all the voltages, temperatures, pressures, etc. By necessity this will slow down the
> starting of the run quite significantly.
>
> The best I understand the midas class driver structure, it is more geared for the first case -
> fast starting of runs.
>
> The thinking behind this choice considers the nature of most slow control data in typical
> physics experiments:
> - if the data does not change quickly (say, room temperature, atmospheric pressure, etc),
> and you read it say every 1 minute, then you do not need to read it again at begin run time -
> the 1 minute old measurement is still good enough - nothing changed much since then
> - if the opposite is true, the data changes wildly (i.e. detector high voltage current goes up
> and down in response to the quickly changing beam current), measuring it at the start of
> the run does us no good - by the time the first event comes around, it has already changed
> completely.
>
> Hopefully Stefan can help you with your specific problem, he has better understanding of
> the midas class drivers.
>
>
> K.O.
  1806   03 Feb 2020 Stefan RittInfoForce triggering of idle routine of a frontend
It is important to note that slow control readout and sending of midas events are two separate things. Readout is done as fast as possible, even multi-threaded if selected. On fast devices this can be 100 Hz readout rate and even more. This data is stored in an internal buffer. When one of the values changes by more than the update threshold, then the ODB gets updated. The midas events are composed from this internal buffer when a new event has to be sent. This is typically periodic (like every 10 seconds or so), or during run transitions. If you specify this in the equipment list with the RO_xxx flags. If you want an event at the begin-of-run, just add there RO_BOR. It should be noted however that this then creates and event during BOR from the last values in the internal buffer, which - depending on the readout speed - can be a few ms "old". I would recommend that you test the readout speed of your variables and then check if this delay is acceptable.

Best,
Stefan


Pintaudi Giorgio wrote:
Hello!
As you know, the generic MIDAS frontend has a class driver, device driver, bus driver
structure. Assuming a slow device frontend, its class driver should have a routine of type
INT idle (EQUIPMENT * pequipment)
This routine is called with a rate controlled by the
"/Equipment/<frontend name>/Common/Event limit" parameter.
The idle routine usually reads one channel of the frontend and stores the results
in the "/Equipment/<frontend name>/Variables" ODB folder.

My question is: it is possible to force (from the code) the frontend to call the idle routine at a
certain point. This is because I need to update the "/Equipment/<frontend name>/Variables"
variables inside the "begin_of_run" routine, at a very specific time.


One dirty solution would be to increase a lot the reading rate ... but I need this
increased reading rate only during the run start while I need a low reading rate
during the run. So the question: is it possible to increase and decrease the reading
rate (event limit) of a frontend without stopping and restarting it?

If you need more info, please let me know.
Thank you
Giorgio
  1808   04 Feb 2020 Pintaudi GiorgioInfoForce triggering of idle routine of a frontend
Dear Stefan,
thank you very much for the clarification. I knew about the DF_XXX flags and I am making good use of them in all my frontends. Anyway, what I really needed was to change the readout rate depending on the run status (in particular DF_RUNNING or DF_TRANSITION).

Moreover, currently, I am not using the MIDAS events framework at all. For the real DAQ, we have our way of acquiring and saving the raw data using the Pyrame software. For the slow control devices, we just use the information that MIDAS automatically saves in the history files .hst (very handy). But I am going to use the MIDAS events at some point in the future, so your explanation is very welcome.

However, I was able to solve my problem by slightly modifying the mfe.cxx file in this way:
@@ -411,6 +411,17 @@ static INT register_equipment(void)
             ss_sleep(3000);
             return 0;
          }
+#ifdef WAGASCI_OPEN_ODB_HOTLINK
+         status = db_open_record(hDB, hKey, eq_info, sizeof(EQUIPMENT), MODE_READ,
+                                 nullptr, nullptr);
+         if (status != DB_SUCCESS) {
+            printf("ERROR: Cannot open hotlink with equipment record \"%s\", db_open_record() status %d\n",
+                   str, status);
+            cm_disconnect_experiment();
+            ss_sleep(3000);
+            return 0;
+         }
+#endif
       } else if (status == DB_STRUCT_MISMATCH) {
          cm_msg(MINFO, "register_equipment", "Correcting \"%s\", db_check_record() status %d", str, status);
          db_create_record(hDB, 0, str, EQUIPMENT_COMMON_STR);

I was quite surprised that I could get things done by just opening a hotlink to the EQUIPMENT eq_info struct. That way I can change dynamically the readout rate (the rate at which the idle routine of a slow device frontend is called is tuned by the "/Equipment/<frontend name>/Common/Event Limit" variable). I change this variable temporarily during a transition to increase the reading rate. I have done some testing and it seems to have no collateral effect.
There is only one caveat.
  • Every change to the equipment "/Equipment/<frontend name>/Common" is instantaneously applied (and might crash the frontend?)

Just to give you an example of a situation where all of this might be useful, think about the ramping-up of the high voltage applied to APD or MPPC. When ramping up from 0 to X volts, you want to read out the voltage and current frequently (let's say once every second) to check for overcurrent and stuff. But as soon as the voltage is up and stable you do not need to monitor it every second and a reading every minute might be more than enough. In our case, the HV power supplies are connected through a serial bus (a nightmare to get it working) and once in a while, we have a transitory connection error. If we kept the reading rate very high continuously the log would be flooded with these innocuous errors (but every new shifter would panic every time he/she notices them). Anyway, this is just an example.



Stefan Ritt wrote:
It is important to note that slow control readout and sending of midas events are two separate things. Readout is done as fast as possible, even multi-threaded if selected. On fast devices this can be 100 Hz readout rate and even more. This data is stored in an internal buffer. When one of the values changes by more than the update threshold, then the ODB gets updated. The midas events are composed from this internal buffer when a new event has to be sent. This is typically periodic (like every 10 seconds or so), or during run transitions. If you specify this in the equipment list with the RO_xxx flags. If you want an event at the begin-of-run, just add there RO_BOR. It should be noted however that this then creates and event during BOR from the last values in the internal buffer, which - depending on the readout speed - can be a few ms "old". I would recommend that you test the readout speed of your variables and then check if this delay is acceptable.

Best,
Stefan


Pintaudi Giorgio wrote:
Hello!
As you know, the generic MIDAS frontend has a class driver, device driver, bus driver
structure. Assuming a slow device frontend, its class driver should have a routine of type
INT idle (EQUIPMENT * pequipment)
This routine is called with a rate controlled by the
"/Equipment/<frontend name>/Common/Event limit" parameter.
The idle routine usually reads one channel of the frontend and stores the results
in the "/Equipment/<frontend name>/Variables" ODB folder.

My question is: it is possible to force (from the code) the frontend to call the idle routine at a
certain point. This is because I need to update the "/Equipment/<frontend name>/Variables"
variables inside the "begin_of_run" routine, at a very specific time.


One dirty solution would be to increase a lot the reading rate ... but I need this
increased reading rate only during the run start while I need a low reading rate
during the run. So the question: is it possible to increase and decrease the reading
rate (event limit) of a frontend without stopping and restarting it?

If you need more info, please let me know.
Thank you
Giorgio
  1811   07 Feb 2020 Stefan RittInfoForce triggering of idle routine of a frontend
Dear Giorgio,

ok, now I'm slowly getting your point.

Dynamically changing the slow control readout rate is possible with your modification, but I consider this badd practice.

You mentioned the case of your HV over a quirky serial line. I had the same some years ago. Rather than reducing the readout rate to reduce the number of errors, I modified my device driver. If the connection is broken, the driver tries silently to reconnect. Only if the reconnect fails for more than a given period (like 1 min), then an error is produced. Otherwise the driver reads as fast as possible. Imagine you have some instabilities in your HV, which only last for a few seconds. If you read only once per minute, you might miss that. We worked hard to make the slow control system multi-threaded, so a slow many-times-retrying-to-reconnect driver does not slow any other equipment. On the other hand, if the re-connect fails for a minute, then you know that your HV unit really has a problem the shifter should follow up.

Best,
Stefan
  1812   07 Feb 2020 Pintaudi GiorgioInfoForce triggering of idle routine of a frontend
Dear Stefan,
Thank you for the advice. I will try to modify the driver as you say. As for the dynamical change of readout rate, basically you are telling me that is not achievable without dirty hacks like mine and it is better to find a way to avoid it.
Best regards
Giorgio


Stefan Ritt wrote:
Dear Giorgio,

ok, now I'm slowly getting your point.

Dynamically changing the slow control readout rate is possible with your modification, but I consider this badd practice.

You mentioned the case of your HV over a quirky serial line. I had the same some years ago. Rather than reducing the readout rate to reduce the number of errors, I modified my device driver. If the connection is broken, the driver tries silently to reconnect. Only if the reconnect fails for more than a given period (like 1 min), then an error is produced. Otherwise the driver reads as fast as possible. Imagine you have some instabilities in your HV, which only last for a few seconds. If you read only once per minute, you might miss that. We worked hard to make the slow control system multi-threaded, so a slow many-times-retrying-to-reconnect driver does not slow any other equipment. On the other hand, if the re-connect fails for a minute, then you know that your HV unit really has a problem the shifter should follow up.

Best,
Stefan
  1813   09 Feb 2020 Stefan RittInfoForce triggering of idle routine of a frontend
You dirty hacks will probably work, but what you REALLY want is to read out your HV always as fast as possible, not only during run transitions or ramping. We had a case where a detector produced electrostatic discharges which only lasted for a second or so, and we were happy to detect this in spikes in the HV current. With measurements of only one per minute we would not have realized that so quicky.

Stefan


Pintaudi Giorgio wrote:
Dear Stefan,
Thank you for the advice. I will try to modify the driver as you say. As for the dynamical change of readout rate, basically you are telling me that is not achievable without dirty hacks like mine and it is better to find a way to avoid it.
  1816   10 Feb 2020 Konstantin OlchanskiInfoForce triggering of idle routine of a frontend
> We had a case where a detector produced electrostatic discharges which only lasted for a second or so
> and we were happy to detect this in spikes in the HV current. With measurements of only one per minute
> we would not have realized that so quicky.

For the T2K/ND280 TPC we implemented something similar. The TPC uses MicroMegas detector which sparks during 
normal operation. We asked Wiener/ISEG to implement a "spark counting mode" for us (and they did). In this mode,
high voltage over-current (a micromegas spark) sets a special flag (does not trip the high voltage). Our midas frontend
reads this flag at rate about 1/min, if flag is set, clears it, increments the software spark counter, reads the flag again,
if the flag is still set (failed to clear), it means this was not a normal spark but a high voltage breakdown
and the offending channel is shut down. I believe this mode is still part of the ISEG normal firmware.

Because the Wiener/ISEG interface uses SNMP to "read all data in one operation", the MIDAS "device driver" structure
was not useful, the readout was a simple loop, the readout frequency was easy to control, and indeed,
we read the high voltage with increased frequency during ramping. This was easy to implement because we
did not have to fight the MIDAS "device driver" framework.

If you want a similar solution, talk to the device, interpret the data, record values to odb and history, generate
midas events - all without hand holding from (arm wrestling with the rest of) midas - I recommend
the new tmfe.h/tmfe.cxx c++ frontend - see the two examples in midas/progs/fetest_tmfe.cxx
and fetest_tmfe_thread.cxx (single-threaded and multi-threaded).

K.O.
  1821   12 Feb 2020 Stefan RittInfoForce triggering of idle routine of a frontend
I had a look again at the issue. If you sett the event limit to zero in the EQUIPMENT list, then the idle() routine of your class driver is called as often as possible. Typically with 100 Hz. It's then up to you what to do in the class driver. The hv_idle() routine of the HV class driver shipped in the distribution for example read a channel more often if it has been changed recently. Look at the lines

/* additionally read channel recently updated if not multithreaded */
if (!(hv_info->driver[hv_info->last_channel]->flags & DF_MULTITHREAD)) {

act_time = ss_millitime();

act = (hv_info->last_channel_updated + 1) % hv_info->num_channels;
while (!(act_time - hv_info->last_change[act] < 10000)) {
act = (act + 1) % hv_info->num_channels;
if (act == hv_info->last_channel_updated) {
/* non found, so return */
return status;
}
}

/* updated channel found, so read it additionally */
status = hv_read(pequipment, act);
hv_info->last_channel_updated = act;
}


You can do similar things there like if you are ramping. On an end-of-run, the class drivers cd_xx_read() routine is called from the framework, which in turn sends a full midas event down the stream, but getting the current slow control values from its local cache, not from the actual device (otherwise stopping a run could be very slow). So if you want all values at the end of the run with good precision, you have to read them DURING the run as fast as possible. That's why I posted my comment about fixing dropped serial connections automatically and reading as fast as possible.

Stefan


Pintaudi Giorgio wrote:
Dear Stefan,
Thank you for the advice. I will try to modify the driver as you say. As for the dynamical change of readout rate, basically you are telling me that is not achievable without dirty hacks like mine and it is better to find a way to avoid it.
  885   10 May 2013 Konstantin OlchanskiBug FixFixed: crash if alarm "write elog message" is enabled
If the MIDAS Alarm property "write elog message" is enabled, an uninitialized variable "tag" is passed to 
el_submit() and depending on your luck, cause a crash. "tag" is supposed to be and is now a NUL-
terminated string. The only other use of el_submit() is in mhttpd.cxx and mserver.c, where it is called 
correctly.

alarm.c svn rev 5361
K.O.
  948   15 Jan 2014 Konstantin OlchanskiBug FixFixed spurious symlinks to midas.log
In some experiments (i.e. DEAP), we see spurious symlinks to midas.log scattered just about everywhere. I 
now traced this to an uninitialized variable in cm_msg_log() and it should be fixed now. K.O.
  118   30 Oct 2003 Stefan Ritt Fixed several potential problems for ODB corruption
I just realized that db_set_value, db_set_data, db_set_num_values and 
db_merge_data do not check for num_values == 0. With such a parameter the 
ODB can become corrupted, since zero length ODB entries are not allowed. I 
fixed the according places in odb.c and committed the changes. Everyone 
with ODB corruption problems should update that code.
ELOG V3.1.4-2e1708b5