Back Midas Rome Roody Rootana
  Midas DAQ System  Not logged in ELOG logo
Entry  28 Jan 2024, Pavel Murat, Forum, number of entries in a given ODB subdirectory ? 
    Reply  28 Jan 2024, Konstantin Olchanski, Forum, number of entries in a given ODB subdirectory ? 
       Reply  28 Jan 2024, Stefan Ritt, Forum, number of entries in a given ODB subdirectory ? Screenshot_2024-01-29_at_07.20.50.png
          Reply  29 Jan 2024, Pavel Murat, Forum, number of entries in a given ODB subdirectory ? 
          Reply  29 Jan 2024, Konstantin Olchanski, Forum, number of entries in a given ODB subdirectory ? 
             Reply  03 Feb 2024, Pavel Murat, Forum, number of entries in a given ODB subdirectory ? a.cc
                Reply  08 Feb 2024, Stefan Ritt, Forum, number of entries in a given ODB subdirectory ? 
                   Reply  11 Feb 2024, Pavel Murat, Forum, number of entries in a given ODB subdirectory ? 
                      Reply  13 Feb 2024, Stefan Ritt, Forum, number of entries in a given ODB subdirectory ? 
                         Reply  15 Feb 2024, Konstantin Olchanski, Forum, number of entries in a given ODB subdirectory ? 
                            Reply  15 Feb 2024, Konstantin Olchanski, Forum, number of entries in a given ODB subdirectory ? 
                               Reply  15 Feb 2024, Stefan Ritt, Forum, number of entries in a given ODB subdirectory ? 
                                  Reply  19 Feb 2024, Pavel Murat, Forum, number of entries in a given ODB subdirectory ? 
Message ID: 2687     Entry time: 28 Jan 2024     In reply to: 2686     Reply to this: 2694
Author: Konstantin Olchanski 
Topic: Forum 
Subject: number of entries in a given ODB subdirectory ? 
Very good question. It exposes a very nasty problem, the race condition between "ls" and "rm". While you are 
looping over directory entries, somebody else is completely permitted to remove one of the files (or add more 
files), making the output of "ls" incorrect (contains non-existant/removed files, does not contain newly added 
files). even the simple count of number of files can be wrong.

Exactly the same problem exists in ODB. As you loop over directory entries, some other ODB client can remove or 
add new entries.

To help with this, I considered adding an db_ls() function that would take the odb lock, atomically iterate over 
a directory and return an std::vector<std::string> with names of all entries. (current odb iterator returns ODB 
handles that may be invalid if corresponding entry was removed while we were iterating). Unfortunately the 
delete/add race condition remains, some returned entries may be invalid or missing.

For your specific application, you can swear that you will never add/delete files "at the wrong time", and you 
will not see this problem until one of your users writes a script that uses odbedit to add/remove subdirectory 
entries exactly at the wrong time. (you run your "ls" in the BeginRun() handler of your frontend, they run their 
"rm" from their's, so both run at the same time, a race condition.

Closer to your question, I think it is simplest to always iterate over the subdirectory, collect names of all 
entries, then work with them:

std::vector<std::string> names;
iterate over odb {
  names.push_back(name);
}
foreach (name in names)
  work_on(name);

instead of:

size_t n = db_get_num_entries();
for (size_t i=0; i<n; i++) {
   std::string name = sprintf("FPGA%d", i);
   work_on(name);
}

K.O.


> Dear MIDAS experts, 
> 
> - I have a detector configuration with a variable number of hardware components - FPGA's receiving data 
>   from the detector. They are described in ODB using a set of keys ranging 
>   from "/Detector/FPGAs/FPGA00" .... to "/Detector/FPGAs/FPGA68".
>   Each of "FPGAxx" corresponds to an ODB subdirectory containing parameters of a given FPGA. 
> 
>   The number of FPGAs in the detector configuration is variable - [independent] commissioning 
>   of different detector subsystems involves different number of FPGAs.
> 
>   In the beginning of the data taking one needs to loop over all of "FPGAxx", 
>   parse the information there and initialize the corresponding FPGAs.
> 
> The actual question sounds rather trivial - what is the best way to implement a loop over them? 
> 
> - it is certainly possible to have the number of FPGAs introduced as an additional configuration parameter, 
>   say, "/Detector/Number_of_FPGAs", and this is what I have resorted to right now.
> 
>   However, not only that loooks ugly, but it also opens a way to make a mistake 
>   and have the Number_of_FPGAs, introduced separately, different from the actual number 
>   of FPGA's in the detector configuration.
>  
> I therefore wonder if there could be a function, smth like 
> 
>     int db_get_n_keys(HNDLE hdb, HNDLE hKeyParent)
> 
> returning the number of ODB keys with a common parent, or, to put it simpler, 
> a number of ODB entries in a given subdirectory.
> 
> And if there were a better solution to the problem I'm dealing with, knowing it might be helpful 
> for more than one person - configuring detector readout may require to deal with a variable number 
> of very different parameters.
> 
> -- many thanks, regards, Pasha
ELOG V3.1.4-2e1708b5