ID |
Date |
Author |
Topic |
Subject |
Draft
|
17 May 2022 |
Ben Smith | Info | MIDAS switched to C++ | > - have you posted somewhere this guide about converting C frontends to C++?
See the instructions at:
https://daq00.triumf.ca/MidasWiki/index.php/Changelog#2019-06
> - it was mentioned previously that there will be a 'tag the last "C" midas', which version is it?
> - it means that even a simple example like odb_test.c cannot be compile anymore? Even when using g++?
> g++ -I $HOME/daq/packages/midas/include/ -L $HOME/daq/packages/midas/lib/ odb_test.c -l midas
Correct. Midas is built with C++, so names get mangled |
2409
|
17 May 2022 |
Konstantin Olchanski | Info | MIDAS switched to C++ | > Hi, I have three naive questions about this:
all good questions, ask more of them.
> - have you posted somewhere this guide about converting C frontends to C++?
yes, in this elog here I posted a guide for converting C mfe.c frontends to C++ and
a guide for converting mfe.c frontend to C++ TMFE frontend. please use the "find" function,
if you cannot find them, let me know, I will look for it for you.
> - it was mentioned previously that there will be a 'tag the last "C" midas', which version is it?
correct. please run "git tag", tags before "midas-2019-05-cxx"is "C", after is "C++".
> - it means that even a simple example like odb_test.c cannot be compile anymore? Even when using g++?
> g++ -I $HOME/daq/packages/midas/include/ -L $HOME/daq/packages/midas/lib/ odb_test.c -l midas
> is expected to fail or is just me glitching? Is it because of thread library differences?
yes, it is expected to fail, you have spaces after "-I", "-L" and "-l", incorrect g++ command syntax. after
correcting this, it may or may not work depending on what you have inside odb_test.c. I would be happy
to help you debug this, but please start a separate thread instead of necroposting into the C++ announcements.
K.O. |
2410
|
17 May 2022 |
Ben Smith | Info | MIDAS switched to C++ | > - have you posted somewhere this guide about converting C frontends to C++?
There's documentation in the wiki at:
https://daq00.triumf.ca/MidasWiki/index.php/Changelog#2019-06
It includes a step-by-step guide of how to upgrade, what changes need to be made to frontends, and common issues that people had. |
2417
|
05 Aug 2022 |
Stefan Ritt | Info | Information for midas updates though git | Several submodules of midas have been re-organized, so if you want to pull the
newest version, you need a
git pull --recurse-submodules
git submodule update --init --recursive
before you can build again. To do this automatically the next time, you can do
git config submodule.recurse true
which needs git 2.14 or later. I hope this works for everybody. If there is a
better way to do that (I'm not a big expert on git) please reply here.
Stefan |
2418
|
06 Aug 2022 |
Stefan Ritt | Info | Improvement of odbxx API | While the odbxx API has been successfully used since the last months, a potential
problem with large ODBs surfaced. If you have lots of data in the ODB and load it
into an object like
midas::odb o("/Equipment");
this might take quite long, since each ODB value is fetched separately, which is
very quick on a local machine but can take long over a client-server connection.
For large experiments this can take up to minutes (!).
To get rid of this problem, the underlying object model has been modified. When an
object is instantiated like above, then the whole ODB tree is fetched in an XML
buffer in a single transfer, which even for large ODBs usually takes much less
than a second. Then the XML buffer is decomposed on the client side and converted
into the proper midas::odb objects. In one case this gave an improvement from 35
seconds to 0.5 seconds which is significant. To enable the new method, the object
can be created with a flag like
midas::odb o("/Equipment", true);
which then switches to the new method. One has to take care not to fool oneself
(like I did) by printing the object like
midas::odb o("/Equipment", true);
std::cout << o << std::endl;
because each read access to any sub-object of o causes a separate read request to
the server which again can take long. Therefore, one has to switch off the auto
refresh via
midas::odb o("/Equipment", true);
o.set_auto_refresh_read(false);
std::cout << o << std::endl;
Accessing any sub-object of o then does not cause a client-server request, which
is not necessary if all objects just have been pulled from the server before. If
one keeps the object however for a long time in memory, one has to be aware that
it only contains "old" values from the time if instantiation. If one needs more
current ODB values, the auto read refresh has to be turned on again.
Stefan |
2419
|
08 Aug 2022 |
Stefan Ritt | Info | Improvement of odbxx API | After some thought, I changed the API again and removed the flag in the constructor,
so the system now automatically choses the best algorithm depending if the client
is connected to a local or a remote API. So in all cases you use again the old syntax:
midas::odb o("/Equipment");
Stefan |
2420
|
08 Aug 2022 |
Konstantin Olchanski | Info | midas on ubuntu LTS 22.04 | reporting that as of commit 78f707c0686d22f8329c7a1f1c46d7dccf35ceff, midas builds
without errors or warnings on Ubuntu LTS 22.04, 20.04, CentOS-7 and MacOS 12.4.
(except for some warnings from mscb and msc). K.O. |
2421
|
08 Aug 2022 |
Konstantin Olchanski | Info | Information for midas updates though git | > git pull --recurse-submodules
> git submodule update --init --recursive
> git config submodule.recurse true
does not work for me, macos 12.4 git 2.32.1.
after I set "submodule.recurse true", I still have to type "git submodule update --
init --recursive", without --recursive, mscb/mxml is empty and the build bombs.
P.S. the underlying issue is that the mxml submodule is now included twice
(midas/mxml and midas/mscb/mxml) and there is nothing to enforce that both copies are
the same. (No idea what happens if the two mxml's are different). |
2422
|
08 Aug 2022 |
Konstantin Olchanski | Info | odb disallow key names that start or end with spaces | while testing the new odb editor, we ran into a number of problems with key names
that start or end with spaces. we cannot think of any valid use case for such key
names (subdirectories and variables) and we think they could only have been
created by mistake. ODB now disallows such names. K.O. |
2423
|
08 Aug 2022 |
Stefan Ritt | Info | Information for midas updates though git | > after I set "submodule.recurse true", I still have to type "git submodule update --
> init --recursive", without --recursive, mscb/mxml is empty and the build bombs.
Indeed, doesn't work for me either. If some git guru has some more insight, please post
here!
> P.S. the underlying issue is that the mxml submodule is now included twice
> (midas/mxml and midas/mscb/mxml) and there is nothing to enforce that both copies are
> the same. (No idea what happens if the two mxml's are different).
The version of each mxml is defined by last commit of the parent repository, which contains
the hash of the submodule version. If we have to update mxml for some reason, we have to
commit also mscb with the new version, and then midas with the same version of mxml. If one
checks out midas then with
git clone https://bitbucket.org/tmidas/midas --recursive
one gets the same versions for mxml. |
2439
|
14 Oct 2022 |
Ben Smith | Info | Allow onchange to refer to arbitrary js function | > I would like to use a function defined in the <script> block of my custom page as an onchange callback.
>
> Is this a possibility?
Yes, this is already possible. An example was shown in the "modb" section of the custom page documentation, but not in the "Changing properties of controls dynamically" section. I've updated the wiki with an example.
https://daq00.triumf.ca/MidasWiki/index.php/Custom_Page#Changing_properties_of_controls_dynamically |
2440
|
22 Oct 2022 |
Lars Martin | Info | Allow onchange to refer to arbitrary js function | I figured I wasn't the first to have this idea.
Works great, thanks! |
2442
|
22 Oct 2022 |
Lars Martin | Info | Allow onchange to refer to arbitrary js function | Actually, now that I look again, there is a mistake in the instructions:
you say
onchange="this.dataset.color=check_therm(this)"
but check_therm doesn't return anything and instead changes the color itself. So you either want the function to return the string and use the above assignment, or use the function you provide and use
onchange="check_therm(this)" |
2458
|
22 Feb 2023 |
Stefano Piacentini | Info | connection to a MySQL server: retry procedure in the Logger | Dear all,
we are experiencing a connection problem to the MySQL server that we use to log informations. Is there an
option to retry multiple times the I/O on the MySQL?
The error we are experiencing is the following (hiding the IP address):
[Logger,ERROR] [mlogger.cxx:2455:write_runlog_sql,ERROR] Failed to connect to database: Error: Can't
connect to MySQL server on 'xxx.xxx.xxx.xxx:6033' (110)
Then the logger stops, and must be restarted. This eventually happens only during the BOR or the EOR.
Best,
Stefano. |
2459
|
22 Feb 2023 |
Stefan Ritt | Info | connection to a MySQL server: retry procedure in the Logger | > Dear all,
>
> we are experiencing a connection problem to the MySQL server that we use to log informations. Is there an
> option to retry multiple times the I/O on the MySQL?
>
> The error we are experiencing is the following (hiding the IP address):
>
> [Logger,ERROR] [mlogger.cxx:2455:write_runlog_sql,ERROR] Failed to connect to database: Error: Can't
> connect to MySQL server on 'xxx.xxx.xxx.xxx:6033' (110)
>
> Then the logger stops, and must be restarted. This eventually happens only during the BOR or the EOR.
What would you propose? If the connection does not work, most likely the server is down or busy. If we retry,
the connection still might not work. If we retry many times, people will complain that the run start or stop
takes very long. If we then just continue (without stopping the logger), the MySQL database will miss important
information and the runs probably cannot be analyzed later. So I believe it's better to really stop the logger
so that people get aware that there is a problem and fix the source, rather than curing the symptoms.
In the MEG experiment at PSI we run the logger with a MySQL database and we never see any connection issue,
except when the MySQL server gets in maintenance (once a year), but usually we don't take data then. Since we
use the same logger code, it cannot be a problem there. So I would try to fix the problem on the MySQL side.
Best,
Stefan |
2464
|
07 Mar 2023 |
Stefano Piacentini | Info | connection to a MySQL server: retry procedure in the Logger | > > Dear all,
> >
> > we are experiencing a connection problem to the MySQL server that we use to log informations. Is there an
> > option to retry multiple times the I/O on the MySQL?
> >
> > The error we are experiencing is the following (hiding the IP address):
> >
> > [Logger,ERROR] [mlogger.cxx:2455:write_runlog_sql,ERROR] Failed to connect to database: Error: Can't
> > connect to MySQL server on 'xxx.xxx.xxx.xxx:6033' (110)
> >
> > Then the logger stops, and must be restarted. This eventually happens only during the BOR or the EOR.
>
> What would you propose? If the connection does not work, most likely the server is down or busy. If we retry,
> the connection still might not work. If we retry many times, people will complain that the run start or stop
> takes very long. If we then just continue (without stopping the logger), the MySQL database will miss important
> information and the runs probably cannot be analyzed later. So I believe it's better to really stop the logger
> so that people get aware that there is a problem and fix the source, rather than curing the symptoms.
>
> In the MEG experiment at PSI we run the logger with a MySQL database and we never see any connection issue,
> except when the MySQL server gets in maintenance (once a year), but usually we don't take data then. Since we
> use the same logger code, it cannot be a problem there. So I would try to fix the problem on the MySQL side.
>
> Best,
> Stefan
Dear Stefan,
a possible solution could be to define the number of times to retry as a parameter that is 0 by default, as well as a wait time between two subsequent tries. This
would leave the decision on how to handle a possible failed connection to the user. In our case, for example, we would prefer to not stop the acquisition in case
of a failed connection to the external SQL. In addition, we have other software that, with a retry procedure, doesn’t fail: with 1 re-try and a sleep time of 0.5 s
we already recover 100% of the faults.
Anyway, we implemented a local database, which is a mirror of the external one, and the problems disappeared.
Thanks,
Stefano. |
2470
|
17 Mar 2023 |
Konstantin Olchanski | Info | T2K/ND280 - Many warning from ten year old variables in ODB | Forwarded from the T2K/ND280 elog:
Author : Nick Hastings
Subject : Many warning from ten year old variables in ODB
Logbook URL : http://elog.nd280.org/elog/FGD/2553
Midas does period checks that the variables in the ODB are ok.
One of these is a check to see if each variable was set with +/- 10
years. Since this experiment has been running for longer than 10
years there are *many* variables that fail this check.
As a result the midas.log and messages in mhttpd are spammed
with many warnings. Eg
Mon Feb 13 14:49:18 2023 [ODBEdit,ERROR] [odb.c:548:db_validate_key,ERROR] Warning: invalid access time, key
"/System/Prompt", time 1288763123
These can be removed by simply setting the variable again with its current value.
So I wonder if it would be best to just do a full odbdump and then load all the values
back in. Comments from MIDAS experts would be appreciated. Eg:
odbedit -c 'save fgddaq.odb'
odbedit -c 'load fgddaq.odb'
Note this problem is currently seen on both the FGD DAQ and the global slow control MIDAS instances.
It may also be a problem on the INGRID GSC and the DAQs of other ND280 systems but I did not check. |
2514
|
12 May 2023 |
Stefan Ritt | Info | New environment variable MIDAS_EXPNAME | A new environment variable MIDAS_EXPNAME has been introduced. This must be
used for cases where people use MIDAS_DIR, and is then equivalent for the
experiment name and directory usually used in the exptab file. This fixes
and issue with creating and deleting shared memory in midas as described in
https://bitbucket.org/tmidas/midas/issues/363/deletion-of-shared-memory-fails
The documentation has been updated at
https://daq00.triumf.ca/MidasWiki/index.php/MIDAS_environment_variables#MIDAS_EXPN
AME
/Stefan |
2515
|
12 May 2023 |
Konstantin Olchanski | Info | New environment variable MIDAS_EXPNAME | > A new environment variable MIDAS_EXPNAME has been introduced [to be used together with
MIDAS_DIR]
This is fixes an important buglet. If experiment uses MIDAS_DIR instead of exptab, at the time
of connecting to ODB, we do not know the experiment name and use name "Default" to create ODB
shared memory, instead of actual experiment name.
This creates an inconsistency, if some MIDAS programs in the same experiment use MIDAS_DIR while
others use exptab (this would be unusual, but not impossible) they would connect to two
different ODB shared memories, former using name "Default", latter using actual experiment name.
As an indication that something is not right, when stopping MIDAS programs, there is an error
message about failure to delete shared ODB shared memory (because it was created using name
"Default" and delete using the correct experiment name fails).
Also it can cause co-mingling between two different experiments, depending on the type of shared
memory used by MIDAS (see $MIDAS_DIR/.SHM_TYPE.TXT):
POSIX - (usually not used) not affected (experiment name is not used)
POSIXv2 - (usually not used) affected (shm name is "$EXP_NAME_ODB")
POSIXv3 - used on MacOS - affected (shm name is "$UID_$EXP_NAME_ODB" so "$UID_Default_ODB" will
collide)
POSIXv4 - used on Linux - not affected (shm name includes $MIDAS_DIR which is different for
different experiments)
K.O. |
2522
|
01 Jun 2023 |
Thomas Lindner | Info | MIDAS Workshop 2023 | Dear MIDAS users,
We would like to arrange another MIDAS workshop, following on from previous successful workshops in 2015, 2017 and 2019. The
goals of the workshop would include:
- Getting updates from MIDAS developers on new features and other changes.
- Getting reports from MIDAS users on how they are using MIDAS, what is working and what is not
- Making plans for future MIDAS changes and improvements
This would be a one-day virtual workshop, planned for about 4 hours length. The workshop will probably be after another of
Stefan's visits to TRIUMF.
If you would be interested in participating in such a workshop, please help us choose the date by filling out this doodle poll:
https://doodle.com/meeting/organize/id/dBPVMQJa
Please fill in the poll by June 9, if you are interested. We will announce the date soon after that.
Thanks,
Thomas |
|