ID |
Date |
Author |
Topic |
Subject |
1963
|
15 Jul 2020 |
Stefan Ritt | Info | Makefile update | Please note that you can also compile midas in the standard cmake way with
$ mkdir build
$ cd build
$ cmake ..
$ make install
in the root midas directory. You might have to use "cmake3" on some systems.
Stefan |
1964
|
15 Jul 2020 |
Stefan Ritt | Info | Minimal CMakeLists.txt for your midas front-end | Since a few people asked me, here is a "minimal" CMakeLists.txt file for a user-written front-end
program "myfe":
---------------------------
cmake_minimum_required(VERSION 3.0)
project(myfe)
# Check for MIDASSYS environment variable
if (NOT DEFINED ENV{MIDASSYS})
message(SEND_ERROR "MIDASSYS environment variable not defined.")
endif()
set(CMAKE_CXX_STANDARD 11)
set(MIDASSYS $ENV{MIDASSYS})
if (${CMAKE_SYSTEM_NAME} MATCHES Linux)
set(LIBS -lpthread -lutil -lrt)
endif()
add_executable(myfe myfe.cxx)
target_include_directories(myfe PRIVATE ${MIDASSYS}/include)
target_link_libraries(crfe ${MIDASSYS}/lib/libmfe.a ${MIDASSYS}/lib/libmidas.a ${LIBS}) |
1967
|
10 Aug 2020 |
Stefan Ritt | Bug Report | data missing in runXXXXXX.mid | > Dear all
>
> We just started our beam time at ILL and just found yesterday that for certain
> settings of our detector the data is not saved into the .mid files. Running "mdump
> -l 10" online we see the data coming in as they should. Nevertheless, if we run
> "mdump -x runXXXXXX.mid" offline, the data file has no events and the banks are
> missing. Any ideas where the data could go lost?
>
> Thanks in advance,
> Ivo
Have you checked
/Logger/Channels/0/Settings/Event ID = -1
/Logger/Channels/0/Settings/Trigger mask = -1
If these settings are not -1, they filter the data stream for certain events and trigger
masks.
Stefan |
1969
|
10 Aug 2020 |
Stefan Ritt | Bug Report | data missing in runXXXXXX.mid | > Both set to -1. We only have one logging channel. If we run a sequence with a few runs and the
> same settings, sometimes data is in the .mid file and sometimes it is not.
Then I'm running out of ideas. Things I would check:
- Are the file sizes about the same?
- When you dump the .mid file, you do you see your bank names?
This would tell you if the events are really missing or if mdump would just not find them.
But I guess without being able to debug the system at ILL I cannot be of any more help. You are the
first one reporting such a problem, so it must have to do with your local setup.
Stefan |
1971
|
10 Aug 2020 |
Stefan Ritt | Bug Report | data missing in runXXXXXX.mid | > So I did a quick check. The file size is about the same (322K and 329K). When I dump the .mid I don't see
> the banks. It only prints two lines with "------ Event# 0 ------" and "------ Event# 1 ------" whereas for
> the file with data I get the two banks with all the data. Our online analyzer also fails to see the banks.
> Is there another way to check what is in the .mid file?
with "dump" I meant a true object dump like "hexdump -C run000001.mid". I produced a file with ADC0 and TDC0
banks (that's the example from the distribution under exampels/experiments/frontend.cxx), and I get
....
00024220 01 00 00 00 41 44 43 30 04 00 08 00 eb 06 35 04 |....ADC0......5.|
00024230 31 09 4f 06 54 44 43 30 04 00 08 00 93 04 fb 07 |1.O.TDC0........|
00024240 5c 09 88 0b 01 00 00 00 01 00 00 00 2a 0b 31 5f |\...........*.1_|
00024250 28 00 00 00 20 00 00 00 01 00 00 00 41 44 43 30 |(... .......ADC0|
00024260 04 00 08 00 c3 09 24 05 85 05 f3 06 54 44 43 30 |......$.....TDC0|
00024270 04 00 08 00 88 08 2d 03 3b 0d d6 02 01 00 00 00 |......-.;.......|
00024280 02 00 00 00 2a 0b 31 5f 28 00 00 00 20 00 00 00 |....*.1_(... ...|
00024290 01 00 00 00 41 44 43 30 04 00 08 00 a5 0a 69 09 |....ADC0......i.|
where you clearly see the ADC0 and TDC0 banks.
Stefan |
1973
|
10 Aug 2020 |
Stefan Ritt | Bug Report | data missing in runXXXXXX.mid | Have you tried longer files? Maybe a few 100 MB or so. Maybe a buffer is not flushed correctly at the end of a run. |
1978
|
10 Aug 2020 |
Stefan Ritt | Bug Report | data missing in runXXXXXX.mid | I have to reproduce the problem to fix it. Why don't you go and modify midas/examples/experiment/frontend.cxx in such a way that
it creates exactly the banks you have, just with random data. If you see the same problem, send me your frontend file so that I
can reproduce it. |
1982
|
13 Aug 2020 |
Stefan Ritt | Suggestion | adding db_get_mode ti check access mode for keys | > Hello,
>
> I am wondering if there is a function that checks the access mode for a key? I
> found the db_set_mode() function that allows me to set the access mode for a key,
> but failed to find its counterpart get function.
>
> Thanks in advance,
> Yan
KEY k;
db_get_key(hDB, handle, &k);
std::cout << k.access_mode << std::endl;
/Stefan |
1985
|
24 Aug 2020 |
Stefan Ritt | Forum | time information |
> 1. Is it possible to get "Running time" using, for example, jsonrpc? (please see
> the attached file)
You have in the ODB "/Runinfo/Start time binary" which is measured in seconds since
1970. By subtracting this from the current time, you get the running time.
> 2. Is it possible to configure "Start time" and "Stop time" with time zone? For
> example when I start a new run, value of "Start time" key is automatically changed
> to "Fri Aug 21 12:38:36 2020" without time zone.
"Start time binary" and "Stop time binary" are in seconds since the 1970 in UTC, so no
time zone involved there. The ASCII versions of the start/stop time are derived from
the binary time using the server's local time zone. If you want to display them in a
different time zone, you have to create a custom page and convert it to another time
zone using JavaScript like
var d = new Date(start_time_binary);
Stefan |
2007
|
20 Oct 2020 |
Stefan Ritt | Info | About remote control of front end part of MIDAS on chip | We also use a Zynq chip and boot in the following order:
1. SD Card
a. First Stage Bootloader
b. PL Firmware
c. UBOOT
2. NFS over Ethernet
a. Linux kernel
b. RootFS
c. Mounting home directories
If you need details I can bring you in contact with the person who actually implemented that.
Best,
Stefan |
2014
|
17 Nov 2020 |
Stefan Ritt | Info | Equipment "common" settings in ODB | Today I addressed a topic which bugged me since long time. The ODB contains
settings under /Equipment/<name>/Common which are a "mirror" of the equipment[]
setting in a frontend (using the mfe.cxx framework). If the "Common" entry in
the ODB is not present (fresh experiment), the equipment[] settings from the
frontend are copied to the ODB. But if it exists, it takes precedence over the
equipment[] entries, which is wrong in my opinion. Like if you change some
settings in equipment[] (like the logging period of the history), then recompile
and restart the frontend, the old values in the ODB are kept and your
modification in the frontend code has no effect.
Starting on commit c3017c6c on Nov. 17th 2020 I reversed the precedence: Now, on
each start of the frontend program, the values from equipment[] are written to
the ODB. They are still "live". If one changes them when the frontend is
running, that change takes effect immediately. But on the next restart of the
frontend, the old values from equipment[] is put back there.
I fell too many times into this trap, and I hope the modification helps
everybody. If there are however experiments which rely on the fact that the
common settings in the ODB are NOT overwritten by the frontend, please let me
know and I can put a flag "EQUIPMENT_FE_PRECEDENCE = FALSE" somewhere to restore
the old behaviour.
Stefan |
2016
|
19 Nov 2020 |
Stefan Ritt | Forum | History plot consuming too much memory | The history code is right now programmes in such a way that when you request
an old time window, then all data from that window until the present date
gets loaded. When we implemented that, this worked fine for data ranges of
several years with a delay of just a few seconds. Of course one can only
load that specific window, but when the user then scrolls right, one has to
append new data to the "right side" of the array stored in the browser. If the
user jumps to another location, then the browser has to keep track of which
windows are loaded and which windows not, making the history code much more
complicated. Therefore I'm only willing to spend a few days of solid work
if this really becomes a problem.
Are you sure that the delay comes from the browser or actually from mhttpd
digging through GBytes of history data? I realized that you need solid state
disks to get a real quick response.
Stefan |
2018
|
20 Nov 2020 |
Stefan Ritt | Forum | History plot consuming too much memory | > Taking this down a tangent, I have a mild concern that a user could temporarily
> flood our gigabit network if we do have faster disks to read the history data. Have
> there been any plans or thoughts on limiting the bandwidth users can pull from
> mhttpd?
I guess this will not be network limiting but CPU limiting of the mhttpd process. But I'm
not 100% sure, depends on the actual hardware. But even if we improve the history
retrieval to "window only", the user could request all data form 2010 to 2020. So one
would need some code which estimates the amount of data, then tell the user "do you really
want that?". But still, a novice user can simply click "yes" without much of a thought. So
in conclusion I believe proper user training is better than software limits. Like the
other guy "I did 'rm -rf /', and now nothing works any more, can you help?".
Stefan |
2036
|
27 Nov 2020 |
Stefan Ritt | Info | Equipment "common" settings in ODB | Ok, so what about the following proposal:
- I change back the mfe.cxx code to behave like before (ODB has precedence and does not get overwritten when the
front-end restarts)
- I add a global flag
BOOL equipment_common_overwrite;
and pre-set it to FALSE;
- So if nothing is changed the flag stays false and ODB keeps precedence
- If a frontend wants to overwrite equipment/common on each start, the user sets
BOOL equipment_common_overwrite = TRUE;
near the equipment[] structure in the front-end code.
- If the flag is true, the mfe.cxx init code copies the equipment[] structure to the ODB on each frontend start
I believe this way we can keep backward compatibility, and add the new way with minimal effort. The only downside
is that all frontends on this plane have to add at least "BOOL equipment_common_overwrite = FALSE;" in their
code.
I know global variables are evil, but this way the user can just add the line above to the equipment[] array, so
one sees this when one edits the equipment[] array, giving motivation to change as needed. So the code would be
BOOL equipment_common_overwrite = TRUE;
EQUIPMENT equipment[] = {
....
}
An alternative way would be to add a function
set_equipment_common_overwrite(TRUE);
into the frontend_init() code. That's somehow cleaner (still needs an internal global variable), but it has to go
into frontend_init() so won't be at the same place as the EQUIPMENT list in the frontend.
Thoughts?
Best,
Stefan |
2039
|
30 Nov 2020 |
Stefan Ritt | Info | Equipment "common" settings in ODB | Ok, I implemented it the following way:
- Added a boolean flag "equipment_common_overwrite", which must be contained in EACH frontend, preferably just
before the EQUIPMENT structure, such as:
BOOL equipment_common_overwrite = TRUE;
EQUIPMENT equipment[] = {
...
};
- If that flag is TRUE, then the contents of the "equipment" structure is copied to the ODB on each start of the
front-end
- If the flag is FALSE, then the ODB values are kept on the start of the front-end
The setting of the flag depends now on the philosophy of the experiment. Some experiments say that everything
needed should be in the front-end code, so when it starts everything gets set correctly. They don't change the
values in the ODB, but in the frontend code, which then goes into their repository. Other experiments just need
some default values from the frontend code, and the fine-tune things by changing values in the ODB. These
experiments should set this flag to FALSE.
*****
Please note that EVERY frontend now needs this flag, so all of you have to add it to all of your front-ends,
otherwise the front-end will not compile! I could not figure out how to this could be done without this
requirement, since you can define a global variable only once.
*****
Stefan |
2040
|
30 Nov 2020 |
Stefan Ritt | Suggestion | ODBSET wildcards with array keys in Sequencer files | Hi Konstantin,
we are considering to make the range selection uniform among json, sequencer and
odbedit "set" command. Having multiple ranges like [1,4-5] will be quite some work, so
my question is did you just implement it on the json side because it was easy, or are
there experiments who really need it? Wouldn't it be enough to have
[*]
[n]
[n-m]
This way we always have only one db_set_data() value behind that. Any set of indices
we have to split into several db_set_data(), which especially for the front-end
configuration can cause trouble by triggering a hot link on each access.
Stefan |
2041
|
30 Nov 2020 |
Stefan Ritt | Info | Equipment "common" settings in ODB | One more change:
After using the new code for some hours, we realized that the "enabled" flag should not come from the frontend code,
but always be defined by the ODB. So if you quickly have to disable some equipment because the associated hardware is
off, you want to change this flag only in the ODB and not have to recompile the frontend. So we exclude that flag from
being set by the frontend. It is anyhow special, because one sees all disable equipment in the main midas status page,
so one knows what's on and what's off.
Please comment here if you think that change causes problem. Anyhow it's working now for the enabled flag as before
all these changes.
Stefan |
2046
|
01 Dec 2020 |
Stefan Ritt | Forum | subrun | There is no "mechanism" foreseen to be executed after each subrun. But you could
run a shell script after each run which loops over all subruns and converts them
one after the other.
Stefan
> Hi,
>
> I was wondering if there is a "mechanism" to run an executable
> file after each subrun is closed...
>
> I need to convert .mid.lz4 subrun files to ROOT (TTree) files;
>
> Thanks,
> Gennaro |
2050
|
09 Dec 2020 |
Stefan Ritt | Forum | history and variables confusion | First, the writing of banks is completely independent of the history system. Banks go to the log file only,
while the history is only linked to the "Variables" section in the ODB.
Second, it's advisable to group similar equipment into one. Like if you have five power supplies powering
and experiment, you don't want to have five equipments Supply1, Supply2, ..., but only one equipment
"Power Supplies". In the frontend belonging to that equipment, you define a DEVICE_DRIVER list with
one entry for each power supply. If you interact with an mscb device, there are some helper functions
which simplify the definition of the equipment and which I can send you privately. So your device
driver looks a bit like the one attached.
If you cannot do that and absolutely want separate equipments, please post a complete ODB subtree of your
settings, and I can try to reproduce your problem.
Stefan
======================
DEVICE_DRIVER power_driver[] = {
{"Power Supply 1", mscbdev, 0, NULL, DF_INPUT | DF_MULTITHREAD},
{"Power Supply 2", mscbdev, 0, NULL, DF_INPUT | DF_MULTITHREAD},
{"Power Supply 3", mscbdev, 0, NULL, DF_INPUT | DF_MULTITHREAD},
{""}
};
...
INT frontend_init()
{
mscb_define("mscbxxx.psi.ch", "Power Supplies", "Power Supply 1", power_driver, 1, 0, "Output 1", 0.1);
mscb_define("mscbxxx.psi.ch", "Power Supplies", "Power Supply 1", power_driver, 1, 1, "Output 2", 0.1);
...
}
/*-- Function to define MSCB variables in a convenient way ---------*/
void mscb_define(const char *submaster, const char *equipment, const char *devname,
DEVICE_DRIVER *driver, int address, unsigned char var_index,
const char *name, double threshold)
{
int i, dev_index, chn_index, chn_total;
char str[256];
float f_threshold;
HNDLE hDB;
cm_get_experiment_database(&hDB, NULL);
if (submaster && submaster[0]) {
sprintf(str, "/Equipment/%s/Settings/Devices/%s/Device", equipment, devname);
db_set_value(hDB, 0, str, submaster, 32, 1, TID_STRING);
sprintf(str, "/Equipment/%s/Settings/Devices/%s/Pwd", equipment, devname);
db_set_value(hDB, 0, str, "meg", 32, 1, TID_STRING);
}
/* find device in device driver */
for (dev_index=0 ; driver[dev_index].name[0] ; dev_index++)
if (equal_ustring(driver[dev_index].name, devname))
break;
if (!driver[dev_index].name[0]) {
cm_msg(MERROR, "mscb_define", "Device \"%s\" not present in device driver list", devname);
return;
}
/* count total number of channels */
for (i=chn_total=0 ; i<=dev_index ; i++)
if (((driver[dev_index].flags & DF_INPUT) > 0 && (driver[i].flags & DF_INPUT)) ||
((driver[dev_index].flags & DF_OUTPUT) > 0 && (driver[i].flags & DF_OUTPUT)))
chn_total += driver[i].channels;
chn_index = driver[dev_index].channels;
sprintf(str, "/Equipment/%s/Settings/Devices/%s/MSCB Address", equipment, devname);
db_set_value_index(hDB, 0, str, &address, sizeof(int), chn_index, TID_INT, TRUE);
sprintf(str, "/Equipment/%s/Settings/Devices/%s/MSCB Index", equipment, devname);
db_set_value_index(hDB, 0, str, &var_index, sizeof(char), chn_index, TID_BYTE, TRUE);
if (threshold != -1 && (driver[dev_index].flags & DF_INPUT) > 0) {
sprintf(str, "/Equipment/%s/Settings/Update Threshold", equipment);
f_threshold = (float) threshold;
db_set_value_index(hDB, 0, str, &f_threshold, sizeof(float), chn_total, TID_FLOAT, TRUE);
}
if (name && name[0]) {
sprintf(str, "/Equipment/%s/Settings/Names %s", equipment, devname);
db_set_value_index(hDB, 0, str, name, 32, chn_total, TID_STRING, TRUE);
}
/* increment number of channels for this driver */
driver[dev_index].channels++;
} |
2059
|
16 Dec 2020 |
Stefan Ritt | Forum | Issues building banks. | > This is very hard to do using the mfe.c frontend. (the main reason I wrote the TMFE C++ frontend class).
Actually that's not true. Just look at
midas/examples/mtfe/mtfe.c
this is an example for a frontend with equipment with the EQ_USER flag, which allows you easily to run a separate
thread (or more) for event collection and processing. Of course all old-fashioned C style (code is from 2007) but it
works.
Stefan |
|