> I don't immediately see a reason for saying that if a DB key is older than 10 yrs, it may not be valid.
> 
> However, it would be worth learning what was the logic behind choosing 10 yrs as a threshold. 
> If 10 is just a more or less arbitrary number, changing 10 --> 100 seems to be the way to go.

Please run "git blame" to find out who added that check.

If I remember right, it was added to complain/correct dates in the future.

I think the oldest experiment at TRIUMF where we still can load an odb into current MIDAS is TWIST,
now about 25 years old. the purpose of loading odb would be to test the history function
to see if we can look at 10-15 year old histories. (TWIST history is in the latest FILE format,
so it will load).

I think this age check should be removed, but there must be *some* check for invalid/bogus timestamps. Or 
not, we should check if MIDAS cares about timestamps at all, if ODB functions never use/look at timestamp, 
maybe we are okey with bogus timestamps. They may look funny in the odb editor, but that's it.

K.O.

> This part of the system has been designed by KO, so he should reply here.

That's right. Some of this stuff is historical gibberish that is no longer needed 
for FILE and SQL histories.

/History/Events is needed to create persistent mapping between history event names 
and history event id's (at some point history event id was same equipment event 
id, with the obvious problems when equipment event ids are duplicated, reused, 
renamed, deleted).

/History/Tags was used by the history editor to speed up "give me all tag names 
for this history event name". With the "MIDAS" history storage this required 
reading a lot of data from disk. With the "FILE" history and cached ZFS SSD, disk 
access is much cheaper and caching history event names and tags in odb is no 
longer necessary.

/History/Tags should probably be removed (be check that nobody uses it first).

/History/Events has to remain as long as "MIDAS" history storage is still used.

K.O.

> > Have you updated to the current midas version? This issue has been fixed a while ago. 
> 
> Hi Stefan, thanks a lot! I pulled from the head, and the scrolling works now. -- regards, Pasha

Right, I remember running into this problem, too.

If you have some ideas on how to better present 100500 history variables, please shout out!

K.O.

$ cat mhf_1697445335_20231016_run_transitions.dat
event name: [Run transitions], time [1697445335]
tag: tag: /DWORD 1 4 /timestamp
tag: tag: UINT32 1 4 State
tag: tag: UINT32 1 4 Run number
record size: 12, data offset: 1024
...

data is in fixed-length record format. from the file header, you read "record size" is 12 and data starts at offset 1024.

the 12 bytes of the data record are described by the tags:
4 bytes of timestamp (DWORD, unix time)
4 bytes of State (UINT32)
4 bytes of "Run number" (UINT32)

endianess is "local endian", which means "little endian" as we have no big-endian hardware anymore to test endian conversions.

file format is designed for reading using read() or mmap().

and you are right mhdump, does not work on these files, I guess I can write another utility that does what I just described and spews the numbers to stdout.

K.O.

> I have implemented a number of slow control frontends which are directed to update the 
> history once in every 10 sec, and they do just that. 

I suggest that you switch from the old mfe.c frontend framework to the new tmfe framework that was 
designed to solve exactly this type of problems.

Look at .../midas/progs/tmfe_example*.cxx

You have a choice of:
- single threaded frontend, most robust, no race conditions, but readout is interrupted during 
begin/end of run.
- two-threaded frontend, your periodic equipments run in one thread, midas loop and rpc run in a 
different thread, you have to handle locking yourself.
- you can run each of your equipments in it's own thread without help from the framework, it is 
obvious how to do it if you can program c++ threads, "new std::thread" to create/start a thread, 
stop threads using a binary flag, thread->join() to reap them at the end (or thread sanitizer will 
complain).

K.O.

MIDAS history is very simple:

from your frontend, your write your history data to ODB /eq/xxx/variables (see below)
mlogger has a hotlink to all /eq/*/variables and it will "see" the new data, write it to history file (see below)
you should see the history file grow using "ls"

history web page in your browser sends a "give me more data" JSON-RPC request to mhttpd
mhttpd looks at the history file, if there is new data (file got bigger) it send it to the web page
web page shows the new data.

where things usually go wrong:

- mlogger only looks for new history variables on startup and on begin-of-run. if you add new stuff in your frontend, you 
will not see it until you restart mlogger or start a new run.
- mlogger only looks at history data if corresponding "/eq/xxx/common/log history" is non-zero. for best effect, set it to 
"1". (or "0" to turn history off).
- history file is not growing, likely mlogger does not "see" your new data
- timestamps of stuff in /eq/xxx/variables are not getting updated, likely frontend is not writing them, and there is no 
new data for mlogger to "see" and write to file.

Frontend has several ways of writing to /eq/xxx/variables:

- write to ODB directly using ODB API db_set_data(), mvodb->Wx(), etc. this is the most foolproof method. use in 
conjunction with a printf() statement to make sure you actually do write to ODB. Sometimes your frontend event loop fails 
to run, a bug/failure that has nothing to do with midas history.

- generate a midas event and set the per-equipment "write event to ODB" flag (RO_ODB for mfe.c frontends), the mfe/tmfe 
framework will write event data to ODB, each data bank will be written to /eq/xxx/variables/BANKNAME, data type is taken 
from the event data bank definition.

This second method sometimes malfunctions, typical problems are missing RO_ODB in the equipment table, equipment table in 
ODB overwriting the value in source code (this is confusing in mfe.c frontends).

Least likely failure is "/eq/xxx/common/log history" set to bogus value. Normal values are 0=history disabled, 1=history 
enabled, other values are only needed if you do not want mlogger to record history as often as you generate it, i.e. you 
update /eq/xxx/variables every 1/sec, but you want mlogger to only record it 1/minute.

I hope this helps.

P.S. I notice your equipment tables do not have RO_ODB, so if you use the 2nd method, write history via event data banks, 
it will not work.


K.O.


> Dear MIDAS developers,
> 
> I'm trying to understand handling of the history (slow control) variables in MIDAS,
> and it seems that the behavior I'm observing is somewhat counterintuitive. 
> Most likely, I just do not understand the implemented logic.
> 
> As it it rather difficult to report on the behavior of the interactive program,
> I'll describe what I'm doing and illustrate the report with the series of attached 
> screenshots showing the history plots and the status of the run control at different 
> consecutive points in time.
> 
> Starting with the landscape:
> 
> - I'm running MIDAS, git commit=30a03c4c (the latest, as of today).
> 
> - I have built the midas/examples/slowcont frontend with the following modifications.
>   (the diffs are enclosed below):
>   
>   1) the frequency of the history updates is increased from 60sec/10sec to 6sec/1sec
>      and, in hope to have updates continuos, I replaced (RO_RUNNING | RO_TRANSITIONS)
>      with RO_ALWAYS.
> 
>   2) for convenience of debugging, midas/drivers/nulldrv.cxx is replaced with its clone,
>      which instead of returning zeroes in each channel, generates a sine curve:
> 
>                   V(t) = 100*sin(t/60)+10*channel
> 
> - an active channel in /Logger/History is chosen to be FILE
> 
> - /History/LoggerHistoryChannel is also set to FILE 
> 
> - I'm running mlogger and modified, as described, 'scfe' frontend from midas/examples/slowcont
> 
> - the attached history plots include three (0,4 and 7) HV:MEASURED channels
> 
> 
> Now, the observations:
> 
> 1) the history plots are updated only when a new run starts, no matter how hard
>    I'm trying to update them by clicking on various buttons.
> 
>    The attached screenshots show the timing sequence of the run control states
>    (with the times printed) and the corresponding history plots. 
> 
>    The "measured voltages" change only when the next run starts - the voltage graphs 
>    break only at the times corresponding to the vertical green lines.
> 
> 2) No matter for how long I wait within the run, the history updates are not happening.
> 
> 3) if the time difference between the two run starts gets too large,
>    the plotted time dependence starts getting discontinuities
> 
> 4) finally, if I switch the logging channel from FILE to MIDAS (activate the MIDAS
>    channel in /Logger/History and set /History/LoggerHistoryChannel to MIDAS),
>    the updates of the history plots simply stop.
> 
> MIDAS feels as a great DAQ framework, so I would appreciate any suggestion on 
> what I could be doing wrong. I'd also be happy to give a demo in real time 
> (via ZOOM/SKYPE etc).
> 
> -- much appreciate your time, thanks, regards, Pasha
>     
> ------------------------------------------------------------------------------
> diff --git a/examples/slowcont/scfe.cxx b/examples/slowcont/scfe.cxx
> index 11f09042..c98d37e8 100644
> --- a/examples/slowcont/scfe.cxx
> +++ b/examples/slowcont/scfe.cxx
> @@ -24,9 +24,10 @@
>  #include "mfe.h"
>  #include "class/hv.h"
>  #include "class/multi.h"
> -#include "device/nulldev.h"
>  #include "bus/null.h"
>  
> +#include "nulldev.h"
> +
>  /*-- Globals -------------------------------------------------------*/
>  
>  /* The frontend name (client name) as seen by other MIDAS clients   */
> @@ -74,11 +75,11 @@ EQUIPMENT equipment[] = {
>       0,                         /* event source */
>       "FIXED",                   /* format */
>       TRUE,                      /* enabled */
> -     RO_RUNNING | RO_TRANSITIONS,        /* read when running and on transitions */
> -     60000,                     /* read every 60 sec */
> +     RO_ALWAYS,        /* read when running and on transitions */
> +     6000,                     /* read every 6 sec */
>       0,                         /* stop run after this event limit */
>       0,                         /* number of sub events */
> -     10000,                     /* log history at most every ten seconds */
> +     1000,                     /* log history at most every one second */
>       "", "", ""} ,
>      cd_hv_read,                 /* readout routine */
>      cd_hv,                      /* class driver main routine */
> @@ -93,8 +94,8 @@ EQUIPMENT equipment[] = {
>       0,                         /* event source */
>       "FIXED",                   /* format */
>       TRUE,                      /* enabled */
> -     RO_RUNNING | RO_TRANSITIONS,        /* read when running and on transitions */
> -     60000,                     /* read every 60 sec */
> +     RO_ALWAYS,        /* read when running and on transitions */
> +     6000,                     /* read every 6 sec */
>       0,                         /* stop run after this event limit */
>       0,                         /* number of sub events */
>       1,                         /* log history every event as often as it changes (max 1 Hz) */
> ------------------------------------------------------------------------------
> [test_001]$ diff ../midas/examples/slowcont/nulldev.cxx ../midas/drivers/device/nulldev.cxx 
> 13d12
> < #include <math.h>
> 150,154c149,150
> <    if (channel < info->num_channels) {
> <      // *pvalue = info->array[channel];
> <      time_t t = time(NULL);;
> <      *pvalue = 100*sin(M_PI*t/60)+10*channel;
> <    }
> ---
> >    if (channel < info->num_channels)
> >       *pvalue = info->array[channel];
> ------------------------------------------------------------------------------

> https://chat.openai.com/share/d927c78d-9914-4413-ab5e-3b0e5d173132
> 
> Please note that you never can be 100% sure that the code from a LLM is correct

yup, it's wrong allright. it should be looping until db_enum_key() returns "no more keys",
not from 0 to N. this is same as iterating over unix filesystem directory entries, opendir(),
loop readdir() until it returns EOF, closedir().

K.O.

familiar situation, "too much data", you dice t or slice it, still too much. BTW, you can try to generate history 
plot ODB entries from your program instead of from the history plot editor. K.O.

Starting around ROOT 6.30, there is a new dependency requirement for nlohmann-json3-dev from https://github.com/nlohmann/json.

If you use a Ubuntu-22 ROOT binary kit from root.cern.ch, MIDAS build will bomb with errors: Could not find a package configuration file provided by "nlohmann_json"

Per https://root.cern/install/dependencies/ install it:

apt install nlohmann-json3-dev

After this MIDAS builds ok.

K.O.

> equipment names are 'mu2edaq09:DTC0' and 'mu2edaq09:DTC1'

I think all names permitted for ODB keys are allowed as equipment names, any valid UTF-8,
forbidden chars are "/" (ODB path separator) and "\0" (C string terminator). Maximum length
is 31 byte (plus "\0" string terminator). (Fixed length 32-byte names with implied terminator
are no longer permitted).

The ":" character is used in history plot definitions and we are likely eventually change that,
history event names used to be pairs of "equipment_name:tag_name" but these days with per-variable
history, they are triplets "equipment_name,variable_name,tag_name". The history plot editor
and the corresponding ODB entries need to be updated for this. Then, ":" will again be a valid
equipment name.

I think if you disable the history for your equipments, MIDAS will stop complaining about ":" in the name.

K.O.

> > The symptoms are consistent with a string --> int64 conversion not happening 
> > where it is needed. 
> 
> Thanks for the report Pasha. Indeed I was missing a conversion in one place. Fixed now!
> 

Are we running these tests as part of the nightly build on bitbucket? They would be part of 
the "make test" target. Correct python dependancies may need to be added to the bitbucket OS 
image in bitbucket-pipelines.yml. (This is a PITA to get right).

K.O.

> Are we running these tests as part of the nightly build on bitbucket? They would be part of 
> the "make test" target. Correct python dependancies may need to be added to the bitbucket OS 
> image in bitbucket-pipelines.yml. (This is a PITA to get right).

I added ubuntu-22 to the nightly builds.

but I notice the build says "no python" and I am not sure what packages I need to install for 
midas python to work.

Ben, can you help me with this?

https://bitbucket.org/tmidas/midas/pipelines/results/1106/steps/%7B9ef2cf97-bd9f-4fd3-9ca2-9c6aa5e20828%7D

K.O.

I pushed some buttons in bitbucket user groups and permissions to make it happy 
wrt recent changes.

The intended configuration is this:

- two user groups: admins and developers
- admins has full control over the workspace, project and repositories ("Admin" 
permission)
- developers have push permission for all repositories (not the "create 
repository" permission, this is limited to admins) ("Write" permission).
- there seems to be a quirk, admins also need to be in the developers group or 
some things do not work (like "run pipeline", which set me off into doing all 
this).
- admins "Admin" permission is set at the "workspace" level and is inherited 
down to project and repository level.
- developers "Write" permission is set at the "project" level and is inherited 
down to repository level.
- individual repositories in the "MIDAS" project also seem to have explicit 
(non-inhertited) permissions, I think this is redundant and I will probably 
remove them at some point (not today).

K.O.

> > > For ODB keys of type TID_KEY, the value num_values IS the number of subkeys. 
> > 
> > this logic makes sense, however it doesn't seem to be consistent with the printout of the test example
> > at the end of https://daq00.triumf.ca/elog-midas/Midas/240203_095803/a.cc . The printout reports 
> > 
> > key.num_values = 1, but the actual number of subkeys = 6, and all subkeys being of TID_KEY type
> > 
> > I'm certain that the ODB subtree in question was not accessed concurrently during the test.
> 
> You are right, num_values is always 1 for TID_KEYS. The number of subkeys is stored in 
> 
>   ((KEYLIST *) ((char *)pheader + pkey->data))->num_keys
> 
> Maybe we should add a function to return this. But so far db_enum_key() was enough.
> 
> Stefan

I would rather add a function that atomically returns an std::vector<KEY>. number of entries
is vector size, entry names are in key.name. If you need to do something with an entry,
like iterate a subdirectory, you have to go by name (not by HNDLE), and if somebody deleted
it, you get an error "entry deleted, tough!", (HNDLE becomes invalid without any error message about it, 
subsequent db_get_data() likely returns gibberish, subsequent db_set_data() likely corrupts ODB).

K.O.

> > You are right, num_values is always 1 for TID_KEYS. The number of subkeys is stored in 
> >   ((KEYLIST *) ((char *)pheader + pkey->data))->num_keys
> > Maybe we should add a function to return this. But so far db_enum_key() was enough.

Hmm... is there any use case where you want to know the number of directory entries, but you will not iterate 
over them later?

K.O.

Our MIDAS frontend for WIENER power supplies is now available as a standalone git repository.

https://bitbucket.org/ttriumfdaq/fewienerlvps/src/master/

This frontend use the snmpwalk and snmpset programs to talk to the power supply.

Also included is a simple custom web page to display power supply status and to turn things on and off.

This frontend was originally written for the T2K/ND280 experiment in Japan.

In addition to controlling Wiener low voltage power supplies, it was also used to control the ISEG MPOD high 
voltage power supplies.

In Japan, ISEG MPOD was (still is) connected to the MicroMegas TPC and is operated in a special "spark counting" 
mode. This spark counting code is still present in this MIDAS frontend and can be restored with a small amount of 
work.

K.o.

> It seems that if a frontend is started automatically by using Program->Auto start then the status page does not show it as started. This is since the FE name has a number after the name. If I stop and start manually then the status page shows the correct state of the FE. Am I doing something wrong or is this a bug somewhere?

Zaher, please read https://daq00.triumf.ca/elog-midas/Midas/919

K.O.

> [ 34%] Linking CXX executable manalyzer_test.exe
> /usr/bin/ld: /home/astrocent/workspace/root/root_install/lib/libRIO.so: undefined 
> reference to 
> `std::condition_variable::wait(std::unique_lock<std::mutex>&)@GLIBCXX_3.4.30'
> collect2: error: ld returned 1 exit status

the error is actually in ROOT, libRIO does not find someting in the standard library.

one possible source of trouble is mismatched compilation flags, to debug this, please 
use "make cmake" and email me (or post here) the full output. (standard cmake hides 
all compiler information to make it easier to debug such problems).

since this is a prerelease of ROOT 6.32 (which in turn fixes major breakage on MacOS) 
and you built it from sources, can you confirm for me that it actually works, you can 
run "root -l somefile.root", open the tbrowser, look at some plots? this is to 
eliminate the possibility that your ROOR is miscompiled.

hmm... also please run "make cmake -k", let's see is linking of rmlogger also fails.

K.O.

ok, thank you for your information. I cannot reproduce this problem, I use vanilla Ubuntu 
LTS 22, ROOT binary kit root_v6.30.02.Linux-ubuntu22.04-x86_64-gcc11.4 from root.cern.ch 
and latest midas from git.

something is wrong with your ubuntu or with your c++ standard library or with your ROOT.

a) can you try with root_v6.30.02.Linux-ubuntu22.04-x86_64-gcc11.4 from root.cern.ch
b) for the midas build, please run "make cclean; make cmake -k" and email me (or post 
here) the complete output.

K.O.

> ok, thank you for your information. I cannot reproduce this problem, I use vanilla Ubuntu 
> LTS 22, ROOT binary kit root_v6.30.02.Linux-ubuntu22.04-x86_64-gcc11.4 from root.cern.ch 
> and latest midas from git.
> 
> something is wrong with your ubuntu or with your c++ standard library or with your ROOT.
> 
> a) can you try with root_v6.30.02.Linux-ubuntu22.04-x86_64-gcc11.4 from root.cern.ch
> b) for the midas build, please run "make cclean; make cmake -k" and email me (or post 
> here) the complete output.

also, please email me the output of these commands on your machine:

daq00:midas$ ls -l /lib/x86_64-linux-gnu/libstdc++*
lrwxrwxrwx 1 root root      19 May 13  2023 /lib/x86_64-linux-gnu/libstdc++.so.6 -> libstdc++.so.6.0.30
-rw-r--r-- 1 root root 2260296 May 13  2023 /lib/x86_64-linux-gnu/libstdc++.so.6.0.30
daq00:midas$ 

and

daq00:midas$ ldd $ROOTSYS/bin/rootreadspeed 
	linux-vdso.so.1 (0x00007ffe6c399000)
	libTree.so => /daq/cern_root/root_v6.30.02.Linux-ubuntu22.04-x86_64-gcc11.4/lib/libTree.so (0x00007f67e53b5000)
	libRIO.so => /daq/cern_root/root_v6.30.02.Linux-ubuntu22.04-x86_64-gcc11.4/lib/libRIO.so (0x00007f67e4fb9000)
	libCore.so => /daq/cern_root/root_v6.30.02.Linux-ubuntu22.04-x86_64-gcc11.4/lib/libCore.so (0x00007f67e4b08000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f67e48bd000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f67e489b000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f67e4672000)
	libNet.so => /daq/cern_root/root_v6.30.02.Linux-ubuntu22.04-x86_64-gcc11.4/lib/libNet.so (0x00007f67e458b000)
	libThread.so => /daq/cern_root/root_v6.30.02.Linux-ubuntu22.04-x86_64-gcc11.4/lib/libThread.so (0x00007f67e4533000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f67e444c000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f67e5599000)
	libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f67e43d6000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f67e43b8000)
	liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f67e438d000)
	libxxhash.so.0 => /lib/x86_64-linux-gnu/libxxhash.so.0 (0x00007f67e4378000)
	liblz4.so.1 => /lib/x86_64-linux-gnu/liblz4.so.1 (0x00007f67e4358000)
	libzstd.so.1 => /lib/x86_64-linux-gnu/libzstd.so.1 (0x00007f67e4289000)
	libssl.so.3 => /lib/x86_64-linux-gnu/libssl.so.3 (0x00007f67e41e3000)
	libcrypto.so.3 => /lib/x86_64-linux-gnu/libcrypto.so.3 (0x00007f67e3d9f000)
daq00:midas$ 

K.O.