ID![up up](up.png) |
Date |
Author |
Topic |
Subject |
2707
|
12 Feb 2024 |
Konstantin Olchanski | Info | MIDAS and ROOT 6.30 |
Starting around ROOT 6.30, there is a new dependency requirement for nlohmann-json3-dev from https://github.com/nlohmann/json.
If you use a Ubuntu-22 ROOT binary kit from root.cern.ch, MIDAS build will bomb with errors: Could not find a package configuration file provided by "nlohmann_json"
Per https://root.cern/install/dependencies/ install it:
apt install nlohmann-json3-dev
After this MIDAS builds ok.
K.O. |
2708
|
13 Feb 2024 |
Stefan Ritt | Forum | number of entries in a given ODB subdirectory ? |
> > For ODB keys of type TID_KEY, the value num_values IS the number of subkeys.
>
> this logic makes sense, however it doesn't seem to be consistent with the printout of the test example
> at the end of https://daq00.triumf.ca/elog-midas/Midas/240203_095803/a.cc . The printout reports
>
> key.num_values = 1, but the actual number of subkeys = 6, and all subkeys being of TID_KEY type
>
> I'm certain that the ODB subtree in question was not accessed concurrently during the test.
You are right, num_values is always 1 for TID_KEYS. The number of subkeys is stored in
((KEYLIST *) ((char *)pheader + pkey->data))->num_keys
Maybe we should add a function to return this. But so far db_enum_key() was enough.
Stefan |
2709
|
13 Feb 2024 |
Konstantin Olchanski | Forum | forbidden equipment names ? |
> equipment names are 'mu2edaq09:DTC0' and 'mu2edaq09:DTC1'
I think all names permitted for ODB keys are allowed as equipment names, any valid UTF-8,
forbidden chars are "/" (ODB path separator) and "\0" (C string terminator). Maximum length
is 31 byte (plus "\0" string terminator). (Fixed length 32-byte names with implied terminator
are no longer permitted).
The ":" character is used in history plot definitions and we are likely eventually change that,
history event names used to be pairs of "equipment_name:tag_name" but these days with per-variable
history, they are triplets "equipment_name,variable_name,tag_name". The history plot editor
and the corresponding ODB entries need to be updated for this. Then, ":" will again be a valid
equipment name.
I think if you disable the history for your equipments, MIDAS will stop complaining about ":" in the name.
K.O. |
2710
|
13 Feb 2024 |
Konstantin Olchanski | Bug Fix | string --> int64 conversion in the python interface ? |
> > The symptoms are consistent with a string --> int64 conversion not happening
> > where it is needed.
>
> Thanks for the report Pasha. Indeed I was missing a conversion in one place. Fixed now!
>
Are we running these tests as part of the nightly build on bitbucket? They would be part of
the "make test" target. Correct python dependancies may need to be added to the bitbucket OS
image in bitbucket-pipelines.yml. (This is a PITA to get right).
K.O. |
2711
|
14 Feb 2024 |
Konstantin Olchanski | Bug Fix | added ubuntu-22 to nightly build on bitbucket, now need python! |
> Are we running these tests as part of the nightly build on bitbucket? They would be part of
> the "make test" target. Correct python dependancies may need to be added to the bitbucket OS
> image in bitbucket-pipelines.yml. (This is a PITA to get right).
I added ubuntu-22 to the nightly builds.
but I notice the build says "no python" and I am not sure what packages I need to install for
midas python to work.
Ben, can you help me with this?
https://bitbucket.org/tmidas/midas/pipelines/results/1106/steps/%7B9ef2cf97-bd9f-4fd3-9ca2-9c6aa5e20828%7D
K.O. |
2712
|
14 Feb 2024 |
Konstantin Olchanski | Info | bitbucket permissions |
I pushed some buttons in bitbucket user groups and permissions to make it happy
wrt recent changes.
The intended configuration is this:
- two user groups: admins and developers
- admins has full control over the workspace, project and repositories ("Admin"
permission)
- developers have push permission for all repositories (not the "create
repository" permission, this is limited to admins) ("Write" permission).
- there seems to be a quirk, admins also need to be in the developers group or
some things do not work (like "run pipeline", which set me off into doing all
this).
- admins "Admin" permission is set at the "workspace" level and is inherited
down to project and repository level.
- developers "Write" permission is set at the "project" level and is inherited
down to repository level.
- individual repositories in the "MIDAS" project also seem to have explicit
(non-inhertited) permissions, I think this is redundant and I will probably
remove them at some point (not today).
K.O. |
2713
|
15 Feb 2024 |
Konstantin Olchanski | Forum | number of entries in a given ODB subdirectory ? |
> > > For ODB keys of type TID_KEY, the value num_values IS the number of subkeys.
> >
> > this logic makes sense, however it doesn't seem to be consistent with the printout of the test example
> > at the end of https://daq00.triumf.ca/elog-midas/Midas/240203_095803/a.cc . The printout reports
> >
> > key.num_values = 1, but the actual number of subkeys = 6, and all subkeys being of TID_KEY type
> >
> > I'm certain that the ODB subtree in question was not accessed concurrently during the test.
>
> You are right, num_values is always 1 for TID_KEYS. The number of subkeys is stored in
>
> ((KEYLIST *) ((char *)pheader + pkey->data))->num_keys
>
> Maybe we should add a function to return this. But so far db_enum_key() was enough.
>
> Stefan
I would rather add a function that atomically returns an std::vector<KEY>. number of entries
is vector size, entry names are in key.name. If you need to do something with an entry,
like iterate a subdirectory, you have to go by name (not by HNDLE), and if somebody deleted
it, you get an error "entry deleted, tough!", (HNDLE becomes invalid without any error message about it,
subsequent db_get_data() likely returns gibberish, subsequent db_set_data() likely corrupts ODB).
K.O. |
2714
|
15 Feb 2024 |
Konstantin Olchanski | Forum | number of entries in a given ODB subdirectory ? |
> > You are right, num_values is always 1 for TID_KEYS. The number of subkeys is stored in
> > ((KEYLIST *) ((char *)pheader + pkey->data))->num_keys
> > Maybe we should add a function to return this. But so far db_enum_key() was enough.
Hmm... is there any use case where you want to know the number of directory entries, but you will not iterate
over them later?
K.O. |
2715
|
15 Feb 2024 |
Stefan Ritt | Forum | number of entries in a given ODB subdirectory ? |
> Hmm... is there any use case where you want to know the number of directory entries, but you will not iterate
> over them later?
I agree.
One more way to iterate over subkeys by name is by using the new odbxx API:
midas::odb tree("/Test/Settings");
for (midas::odb& key : tree)
std::cout << key.get_name() << std::endl;
Stefan |
2716
|
18 Feb 2024 |
Frederik Wauters | Forum | dump history FILE files |
> $ cat mhf_1697445335_20231016_run_transitions.dat
> event name: [Run transitions], time [1697445335]
> tag: tag: /DWORD 1 4 /timestamp
> tag: tag: UINT32 1 4 State
> tag: tag: UINT32 1 4 Run number
> record size: 12, data offset: 1024
> ...
>
> data is in fixed-length record format. from the file header, you read "record size" is 12 and data starts at offset 1024.
>
> the 12 bytes of the data record are described by the tags:
> 4 bytes of timestamp (DWORD, unix time)
> 4 bytes of State (UINT32)
> 4 bytes of "Run number" (UINT32)
>
> endianess is "local endian", which means "little endian" as we have no big-endian hardware anymore to test endian conversions.
>
> file format is designed for reading using read() or mmap().
>
> and you are right mhdump, does not work on these files, I guess I can write another utility that does what I just described and spews the numbers to stdout.
>
> K.O.
Thanks for the answer. As this FILE system is advertised as the new default (eog:2617), this format does merit some more WIKI info. |
2717
|
19 Feb 2024 |
Pavel Murat | Forum | number of entries in a given ODB subdirectory ? |
> > Hmm... is there any use case where you want to know the number of directory entries, but you will not iterate
> > over them later?
>
> I agree.
here comes the use case:
I have a slow control frontend which monitors several DAQ components - software processes.
The components are listed in the system configuration stored in ODB, a subkey per component.
Each component has its own driver, so the length of the driver list, defined by the number of components,
needs to be determined at run time.
I calculate the number of components by iterating over the list of component subkeys in the system configuration,
allocate space for the driver list, and store the pointer to the driver list in the equipment record.
The approach works, but it does require pre-calculating the number of subkeys of a given key.
-- regards, Pasha |
2718
|
26 Feb 2024 |
Maia Henriksson-Ward | Forum | mserver ERR message saying data area 100% full, though it is free |
> Hi,
>
> I have just installed Midas and set-up the ODB for a SuperCDMS test-facility (on
> a SL6.7 machine). All works fine except that I receive the following error message:
>
> [mserver,ERROR] [odb.c:944:db_validate_db,ERROR] Warning: database data area is
> 100% full
>
> Which is puzzling for the following reason:
>
> -> I have created the ODB with: odbedit -s 4194304
> -> Checking the size of the .ODB.SHM it says: 4.2M
> -> When I save the ODB as .xml and check the file's size it says: 1.1M
> -> When I start odbedit and check the memory usage issuing 'mem', it says:
> ...
> Free Key area: 1982136 bytes out of 2097152 bytes
> ...
> Free Data area: 2020072 bytes out of 2097152 bytes
> Free: 1982136 (94.5%) keylist, 2020072 (96.3%) data
>
> So it seems like nearly all memory is still free. As a test I created more
> instances of one of our front-ends and checked 'mem' again. As expected the free
> memory was decreasing. I did this ten times in fact, reaching
>
> ...
> Free Key area: 1440976 bytes out of 2097152 bytes
> ...
> Free Data area: 1861264 bytes out of 2097152 bytes
> Free: 1440976 (68.7%) keylist, 1861264 (88.8%) data
>
> So I could use another >20% of the database data area, which is according to the
> error message 100% (resp. >95%) full. Am I misunderstanding the error message?
> I'd appreciate any comments or ideas on that subject!
>
> Thanks, Belina
This is an old post, but I encountered the same error message recently and was looking for a
solution here. Here's how I solved it, for anyone else who finds this:
The size of .ODB.SHM was bigger than the maximum ODB size (4.2M > 4194304 in Belina's case). For us,
the very large odb size was in error and I suspect it happened because we forgot to shut down midas
cleanly before shutting the computer down. Using odbedit to load a previously saved copy of the ODB
did not help me to get .ODB.SHM back to a normal size. Following the instructions on the wiki for
recovery from a corrupted odb,
https://daq00.triumf.ca/MidasWiki/index.php/FAQ#How_to_recover_from_a_corrupted_ODB, (odbinit with --cleanup option) should
work, but didn't for me. Unfortunately I didn't save the output to figure out why. My solution was to manually delete/move/hide
the .ODB.SHM file, and an equally large file called .ODB.SHM.1701109528, then run odbedit again and reload that same saved copy of my ODB.
Manually changing files used by mserver is risky - for anyone who has the same problem, I suggest trying odbinit --cleanup -s
<yoursize> first. |
2719
|
27 Feb 2024 |
Pavel Murat | Forum | displaying integers in hex format ? |
Dear MIDAS Experts,
I'm having an odd problem when trying to display an integer stored in ODB on a custom
web page: the hex specifier, "%x", displays integers as if it were "%d" .
- attachment 1 shows the layout and the contents of the ODB sub-tree in question
- attachment 2 shows the web page as it is displayed
- attachment 3 shows the snippet of html/js producing the web page
I bet I'm missing smth trivial - an advice is greatly appreciated!
Also, is there an equivalent of a "0x%04x" specifier to have the output formatted
into a fixed length string ?
-- thanks, regards, Pasha |
Attachment 1: 2024_02_27_dtc_registers_in_odb.png
|
|
Attachment 2: 2024_02_27_custom_page.png
|
|
Attachment 3: 2024_02_27_custom_page_html.png
|
|
2720
|
27 Feb 2024 |
Stefan Ritt | Forum | displaying integers in hex format ? |
Thanks for reporting that bug. I fixed it and committed the change to the develop branch.
Stefan |
2721
|
27 Feb 2024 |
Pavel Murat | Forum | displaying integers in hex format ? |
Hi Stefan (and Ben),
thanks for reacting so promptly - your commits on Bitbucket fixed the problem.
For those of us who knows little about how the web browsers work:
- picking up the fix required flushing the cache of the MIDAS client web browser - apparently the web browser
I'm using - Firefox 115.6 - cached the old version of midas.js but wouldn't report it cached and wouldn't load
the updated file on its own.
-- thanks again, regards, Pasha |
2722
|
08 Mar 2024 |
Konstantin Olchanski | Info | MIDAS frontend for WIENER L.V. P.S. and VME crates |
Our MIDAS frontend for WIENER power supplies is now available as a standalone git repository.
https://bitbucket.org/ttriumfdaq/fewienerlvps/src/master/
This frontend use the snmpwalk and snmpset programs to talk to the power supply.
Also included is a simple custom web page to display power supply status and to turn things on and off.
This frontend was originally written for the T2K/ND280 experiment in Japan.
In addition to controlling Wiener low voltage power supplies, it was also used to control the ISEG MPOD high
voltage power supplies.
In Japan, ISEG MPOD was (still is) connected to the MicroMegas TPC and is operated in a special "spark counting"
mode. This spark counting code is still present in this MIDAS frontend and can be restored with a small amount of
work.
K.o. |
2723
|
10 Mar 2024 |
Zaher Salman | Bug Report | Autostart program |
Hello everyone,
It seems that if a frontend is started automatically by using Program->Auto start then the status page does not show it as started. This is since the FE name has a number after the name. If I stop and start manually then the status page shows the correct state of the FE. Am I doing something wrong or is this a bug somewhere?
thanks,
Zaher |
2724
|
11 Mar 2024 |
Konstantin Olchanski | Bug Report | Autostart program |
> It seems that if a frontend is started automatically by using Program->Auto start then the status page does not show it as started. This is since the FE name has a number after the name. If I stop and start manually then the status page shows the correct state of the FE. Am I doing something wrong or is this a bug somewhere?
Zaher, please read https://daq00.triumf.ca/elog-midas/Midas/919
K.O. |
2725
|
18 Mar 2024 |
Grzegorz Nieradka | Bug Report | Midas (manalyzer) + ROOT 6.31/01 - compilation error |
I tried to update MIDAS installation on Ubuntu 22.04.1 to the latest commit at
the bitbucket.
I have update the ROOT from source the latest version ROOT 6.31/01.
During the MIDAS compilation I have error:
/usr/bin/ld: *some_path_to_ROOT*/libRIO.so: undefined reference to
`std::condition_variable::wait(std::unique_lock<std::mutex>&)@GLIBCXX_3.4.30'
The longer version of this error is below.
Has anybody knows some simple solution of this error?
Thanks, GN
Consolidate compiler generated dependencies of target manalyzer_main
[ 32%] Building CXX object
manalyzer/CMakeFiles/manalyzer_main.dir/manalyzer_main.cxx.o
[ 33%] Linking CXX static library libmanalyzer_main.a
[ 33%] Built target manalyzer_main
Consolidate compiler generated dependencies of target manalyzer_test.exe
[ 33%] Building CXX object
manalyzer/CMakeFiles/manalyzer_test.exe.dir/manalyzer_main.cxx.o
[ 34%] Linking CXX executable manalyzer_test.exe
/usr/bin/ld: /home/astrocent/workspace/root/root_install/lib/libRIO.so: undefined
reference to
`std::condition_variable::wait(std::unique_lock<std::mutex>&)@GLIBCXX_3.4.30'
collect2: error: ld returned 1 exit status
make[2]: *** [manalyzer/CMakeFiles/manalyzer_test.exe.dir/build.make:124:
manalyzer/manalyzer_test.exe] Error 1
make[1]: *** [CMakeFiles/Makefile2:780:
manalyzer/CMakeFiles/manalyzer_test.exe.dir/all] Error 2 |
2726
|
18 Mar 2024 |
Konstantin Olchanski | Bug Report | Midas (manalyzer) + ROOT 6.31/01 - compilation error |
> [ 34%] Linking CXX executable manalyzer_test.exe
> /usr/bin/ld: /home/astrocent/workspace/root/root_install/lib/libRIO.so: undefined
> reference to
> `std::condition_variable::wait(std::unique_lock<std::mutex>&)@GLIBCXX_3.4.30'
> collect2: error: ld returned 1 exit status
the error is actually in ROOT, libRIO does not find someting in the standard library.
one possible source of trouble is mismatched compilation flags, to debug this, please
use "make cmake" and email me (or post here) the full output. (standard cmake hides
all compiler information to make it easier to debug such problems).
since this is a prerelease of ROOT 6.32 (which in turn fixes major breakage on MacOS)
and you built it from sources, can you confirm for me that it actually works, you can
run "root -l somefile.root", open the tbrowser, look at some plots? this is to
eliminate the possibility that your ROOR is miscompiled.
hmm... also please run "make cmake -k", let's see is linking of rmlogger also fails.
K.O. |