| ID |
Date |
Author |
Topic |
Subject |
|
1223
|
01 Dec 2016 |
Thomas Lindner | Bug Report | control characters not sanitized by json_write - can cause JSON.parse of mhttpd result to fail | > > I've recently run into issues when using JSON.parse on ODB keys containing
> > 8-bit data.
>
> I am tempted to take a hard line and say that in general MIDAS TID_STRING data should be valid
> UTF-8 encoded Unicode. In the modern mixed javascript/json/whatever environment I think
> it is impractical to handle or permit invalid UTF-8 strings.
>
> Certainly in the general case, replacing all control characters with something else or escaping them or
> otherwise changing the value if TID_STRING data would wreck *valid* UTF-8 strings, which I would
> assume to be the normal use.
>
> In other words, non-UTF-8 strings are following non-IEEE-754 floating point values into oblivion - as
> we do not check the TID_FLOAT and TID_DOUBLE is valid IEEE-754 values, we should not check
> that TID_STRING is valid UTF-8.
I agree that I think we should start requiring strings to be UTF-8 encoded unicode.
I'd suggest that before worrying about the TID_STRING data, we should start by sanitizing the ODB key names.
I've seen a couple cases where the ODB key name is a non-UTF-8 string. It is very awkward to use odbedit
to delete these keys.
I attach a suggested modification to odb.c that rejects calls to db_create_key with non-UTF-8 key names. It
uses some random function I found on the internet that is supposed to check if a string is valid UTF-8. I
checked a couple of strings with invalid UTF-8 characters and it correctly identified them. But I won't
claim to be certain that this is really identifying all UTF-8 vs non-UTF-8 cases. Maybe others have a
better way of identifying this. |
| Attachment 1: odb_modifications.txt
|
diff --git a/src/odb.c b/src/odb.c
index 47ace8f..041080e 100755
--- a/src/odb.c
+++ b/src/odb.c
@@ -1818,6 +1818,90 @@ BOOL equal_ustring(const char *str1, const char *str2)
return TRUE;
}
+// Method to check if a given string is valid UTF-8. Returns 1 if it is.
+BOOL is_utf8(const char * string)
+{
+ if(!string)
+ return 0;
+
+ const unsigned char * bytes = (const unsigned char *)string;
+ while(*bytes)
+ {
+ if( (// ASCII
+ // use bytes[0] <= 0x7F to allow ASCII control characters
+ bytes[0] == 0x09 ||
+ bytes[0] == 0x0A ||
+ bytes[0] == 0x0D ||
+ (0x20 <= bytes[0] && bytes[0] <= 0x7E)
+ )
+ ) {
+ bytes += 1;
+ continue;
+ }
+
+ if( (// non-overlong 2-byte
+ (0xC2 <= bytes[0] && bytes[0] <= 0xDF) &&
+ (0x80 <= bytes[1] && bytes[1] <= 0xBF)
+ )
+ ) {
+ bytes += 2;
+ continue;
+ }
+
+ if( (// excluding overlongs
+ bytes[0] == 0xE0 &&
+ (0xA0 <= bytes[1] && bytes[1] <= 0xBF) &&
+ (0x80 <= bytes[2] && bytes[2] <= 0xBF)
+ ) ||
+ (// straight 3-byte
+ ((0xE1 <= bytes[0] && bytes[0] <= 0xEC) ||
+ bytes[0] == 0xEE ||
+ bytes[0] == 0xEF) &&
+ (0x80 <= bytes[1] && bytes[1] <= 0xBF) &&
+ (0x80 <= bytes[2] && bytes[2] <= 0xBF)
+ ) ||
+ (// excluding surrogates
+ bytes[0] == 0xED &&
+ (0x80 <= bytes[1] && bytes[1] <= 0x9F) &&
+ (0x80 <= bytes[2] && bytes[2] <= 0xBF)
+ )
+ ) {
+ bytes += 3;
+ continue;
+ }
+
+ if( (// planes 1-3
+ bytes[0] == 0xF0 &&
+ (0x90 <= bytes[1] && bytes[1] <= 0xBF) &&
+ (0x80 <= bytes[2] && bytes[2] <= 0xBF) &&
+ (0x80 <= bytes[3] && bytes[3] <= 0xBF)
+ ) ||
+ (// planes 4-15
+ (0xF1 <= bytes[0] && bytes[0] <= 0xF3) &&
+ (0x80 <= bytes[1] && bytes[1] <= 0xBF) &&
+ (0x80 <= bytes[2] && bytes[2] <= 0xBF) &&
+ (0x80 <= bytes[3] && bytes[3] <= 0xBF)
+ ) ||
+ (// plane 16
+ bytes[0] == 0xF4 &&
+ (0x80 <= bytes[1] && bytes[1] <= 0x8F) &&
+ (0x80 <= bytes[2] && bytes[2] <= 0xBF) &&
+ (0x80 <= bytes[3] && bytes[3] <= 0xBF)
+ )
+ ) {
+ bytes += 4;
+ continue;
+ }
+
+ return 0;
+ }
+
+ return 1;
+}
+
+
+
+
/********************************************************************/
/**
Create a new key in a database
@@ -1829,6 +1913,12 @@ Create a new key in a database
*/
INT db_create_key(HNDLE hDB, HNDLE hKey, const char *key_name, DWORD type)
{
+
+ if(!is_utf8(key_name)){
+ cm_msg(MERROR, "db_create_key", "invalid non-UTF-8 key name \'%s\'", key_name);
+ return DB_INVALID_PARAM;
+ }
+
if (rpc_is_remote())
return rpc_call(RPC_DB_CREATE_KEY, hDB, hKey, key_name, type);
|
|
1227
|
15 Jan 2017 |
Thomas Lindner | Bug Report | control characters not sanitized by json_write - can cause JSON.parse of mhttpd result to fail | > > In other words, non-UTF-8 strings are following non-IEEE-754 floating point values into oblivion - as
> > we do not check the TID_FLOAT and TID_DOUBLE is valid IEEE-754 values, we should not check
> > that TID_STRING is valid UTF-8.
> ...
> I attach a suggested modification to odb.c that rejects calls to db_create_key with non-UTF-8 key names. It
> uses some random function I found on the internet that is supposed to check if a string is valid UTF-8. I
> checked a couple of strings with invalid UTF-8 characters and it correctly identified them. But I won't
> claim to be certain that this is really identifying all UTF-8 vs non-UTF-8 cases. Maybe others have a
> better way of identifying this.
At Konstantin's suggestion, I committed the function I found for checking if a string was UTF-8 compatible to
odb.c. The function is currently not used; I commented out a proposed use in db_create_key. Experts can decide
if the code was good enough to use. |
|
1228
|
23 Jan 2017 |
Thomas Lindner | Bug Report | control characters not sanitized by json_write - can cause JSON.parse of mhttpd result to fail |
> At Konstantin's suggestion, I committed the function I found for checking if a string was UTF-8 compatible to
> odb.c. The function is currently not used; I commented out a proposed use in db_create_key. Experts can decide
> if the code was good enough to use.
After more discussion, I have enabled the parts of the ODB code that check that key names are UTF-8 compliant.
This check will show up in (at least) two ways:
1) Attempts to create a new ODB variable if the ODB key is not UTF-8 compliant. You will see error messages like
[fesimdaq,ERROR] [odb.c:572:db_validate_name,ERROR] Invalid name "Eur€" passed to db_create_key: UTF-8 incompatible
string
2) When a program first connects to the ODB, it runs a check to ensure that the ODB is valid. This will now include
a check that all key names are UTF-8 compliant. Any non-UTF8 compliant key names will be replaced by a string of the
pointer to the key. You will see error messages like:
[fesimdaq,ERROR] [odb.c:572:db_validate_name,ERROR] Invalid name "Eur€" passed to db_validate_key: UTF-8
incompatible string
[fesimdaq,ERROR] [odb.c:647:db_validate_key,ERROR] Warning: corrected key "/Equipment/SIMDAQ/Eur€": invalid name
"Eur€" replaced with "0x7f74be63f970"
This behaviour (checking UTF-8 compatibility and automatically fixing ODB names) can be disabled by setting an
environment variable
MIDAS_INVALID_STRING_IS_OK
It doesn't matter what the environment variable is set to; it just needs to be set. Note also that this variable is
only checked once, when a program starts. |
|
1229
|
30 Jan 2017 |
Stefan Ritt | Bug Report | control characters not sanitized by json_write - can cause JSON.parse of mhttpd result to fail | >
> > At Konstantin's suggestion, I committed the function I found for checking if a string was UTF-8 compatible to
> > odb.c. The function is currently not used; I commented out a proposed use in db_create_key. Experts can decide
> > if the code was good enough to use.
>
> After more discussion, I have enabled the parts of the ODB code that check that key names are UTF-8 compliant.
>
> This check will show up in (at least) two ways:
>
> 1) Attempts to create a new ODB variable if the ODB key is not UTF-8 compliant. You will see error messages like
>
> [fesimdaq,ERROR] [odb.c:572:db_validate_name,ERROR] Invalid name "Eur€" passed to db_create_key: UTF-8 incompatible
> string
>
> 2) When a program first connects to the ODB, it runs a check to ensure that the ODB is valid. This will now include
> a check that all key names are UTF-8 compliant. Any non-UTF8 compliant key names will be replaced by a string of the
> pointer to the key. You will see error messages like:
>
> [fesimdaq,ERROR] [odb.c:572:db_validate_name,ERROR] Invalid name "Eur€" passed to db_validate_key: UTF-8
> incompatible string
> [fesimdaq,ERROR] [odb.c:647:db_validate_key,ERROR] Warning: corrected key "/Equipment/SIMDAQ/Eur€": invalid name
> "Eur€" replaced with "0x7f74be63f970"
>
> This behaviour (checking UTF-8 compatibility and automatically fixing ODB names) can be disabled by setting an
> environment variable
>
> MIDAS_INVALID_STRING_IS_OK
>
> It doesn't matter what the environment variable is set to; it just needs to be set. Note also that this variable is
> only checked once, when a program starts.
I see you put some switches into the environment ("MIDAS_INVALID_STRING_IS_OK"). Do you think this is a good idea? Most variables are
sitting in the ODB (/experiment/xxx), except those which cannot be in the ODB because we need it before we open the ODB, like MIDAS_DIR.
Having them in the ODB has the advantage that everything is in one place, and we see a "list" of things we can change. From an empty
environment it is not clear that such a thing like "MIDAS_INVALID_STRING_IS_OK" does exist, while if it would be an ODB key it would be
obvious. Can I convince you to move this flag into the ODB? |
|
1230
|
01 Feb 2017 |
Konstantin Olchanski | Bug Report | control characters not sanitized by json_write - can cause JSON.parse of mhttpd result to fail | >
> I see you put some switches into the environment ("MIDAS_INVALID_STRING_IS_OK"). Do you think this is a good idea? Most variables are
> sitting in the ODB (/experiment/xxx), except those which cannot be in the ODB because we need it before we open the ODB, like MIDAS_DIR.
> Having them in the ODB has the advantage that everything is in one place, and we see a "list" of things we can change. From an empty
> environment it is not clear that such a thing like "MIDAS_INVALID_STRING_IS_OK" does exist, while if it would be an ODB key it would be
> obvious. Can I convince you to move this flag into the ODB?
>
Some additional explanation.
Time passed, the world turned, and the current web-compatible standard for text strings is UTF-8 encoded Unicode, see
https://en.wikipedia.org/wiki/UTF-8
(ObCanadianContent, UTF-8 was invented the Canadian Rob Pike https://en.wikipedia.org/wiki/Rob_Pike)
(and by some other guy https://en.wikipedia.org/wiki/Ken_Thompson).
It turns out that not every combination of 8-bit characters (char*) is valid UTF-8 Unicode.
In the MIDAS world we run into this when MIDAS ODB strings are exported to Javascript running inside web
browsers ("custom pages", etc). ODB strings (TID_STRING) and ODB key names that are not valid UTF-8
make such web pages malfunction and do not work right.
One solution to this is to declare that ODB strings (TID_STRING) and ODB key names *must* be valid UTF-8 Unicode.
The present commits implemented this solution. Invalid UTF-8 is rejected by db_create() & co and by the ODB integrity validator.
This means some existing running experiment may suddenly break because somehow they have "old-style" ODB entries
or they mistakenly use TID_STRING to store arbitrary binary data (use array of TID_CHAR instead).
To permit such experiments to use current releases of MIDAS, we include a "defeat" device - to disable UTF-8 checks
until they figure out where non-UTF-8 strings come from and correct the problem.
Why is this defeat device non an ODB entry? Because it is not a normal mode of operation - there is no use-case where
an experiment will continue to use non-UTF-8 compatible ODB indefinitely, in the long term. For example, as the MIDAS user
interface moves to more and more to HTML+Javascript+"AJAX", such experiments will see that non-UTF-8 compatible ODB entries
cause all sorts of problems and will have to convert.
K.O. |
|
1235
|
01 Feb 2017 |
Stefan Ritt | Bug Report | control characters not sanitized by json_write - can cause JSON.parse of mhttpd result to fail | > Some additional explanation.
>
> Time passed, the world turned, and the current web-compatible standard for text strings is UTF-8 encoded Unicode, see
> https://en.wikipedia.org/wiki/UTF-8
> (ObCanadianContent, UTF-8 was invented the Canadian Rob Pike https://en.wikipedia.org/wiki/Rob_Pike)
> (and by some other guy https://en.wikipedia.org/wiki/Ken_Thompson).
>
> It turns out that not every combination of 8-bit characters (char*) is valid UTF-8 Unicode.
>
> In the MIDAS world we run into this when MIDAS ODB strings are exported to Javascript running inside web
> browsers ("custom pages", etc). ODB strings (TID_STRING) and ODB key names that are not valid UTF-8
> make such web pages malfunction and do not work right.
>
> One solution to this is to declare that ODB strings (TID_STRING) and ODB key names *must* be valid UTF-8 Unicode.
>
> The present commits implemented this solution. Invalid UTF-8 is rejected by db_create() & co and by the ODB integrity validator.
>
> This means some existing running experiment may suddenly break because somehow they have "old-style" ODB entries
> or they mistakenly use TID_STRING to store arbitrary binary data (use array of TID_CHAR instead).
>
> To permit such experiments to use current releases of MIDAS, we include a "defeat" device - to disable UTF-8 checks
> until they figure out where non-UTF-8 strings come from and correct the problem.
>
> Why is this defeat device non an ODB entry? Because it is not a normal mode of operation - there is no use-case where
> an experiment will continue to use non-UTF-8 compatible ODB indefinitely, in the long term. For example, as the MIDAS user
> interface moves to more and more to HTML+Javascript+"AJAX", such experiments will see that non-UTF-8 compatible ODB entries
> cause all sorts of problems and will have to convert.
>
>
> K.O.
Ok, I agree.
Stefan |
|
2458
|
22 Feb 2023 |
Stefano Piacentini | Info | connection to a MySQL server: retry procedure in the Logger | Dear all,
we are experiencing a connection problem to the MySQL server that we use to log informations. Is there an
option to retry multiple times the I/O on the MySQL?
The error we are experiencing is the following (hiding the IP address):
[Logger,ERROR] [mlogger.cxx:2455:write_runlog_sql,ERROR] Failed to connect to database: Error: Can't
connect to MySQL server on 'xxx.xxx.xxx.xxx:6033' (110)
Then the logger stops, and must be restarted. This eventually happens only during the BOR or the EOR.
Best,
Stefano. |
|
2459
|
22 Feb 2023 |
Stefan Ritt | Info | connection to a MySQL server: retry procedure in the Logger | > Dear all,
>
> we are experiencing a connection problem to the MySQL server that we use to log informations. Is there an
> option to retry multiple times the I/O on the MySQL?
>
> The error we are experiencing is the following (hiding the IP address):
>
> [Logger,ERROR] [mlogger.cxx:2455:write_runlog_sql,ERROR] Failed to connect to database: Error: Can't
> connect to MySQL server on 'xxx.xxx.xxx.xxx:6033' (110)
>
> Then the logger stops, and must be restarted. This eventually happens only during the BOR or the EOR.
What would you propose? If the connection does not work, most likely the server is down or busy. If we retry,
the connection still might not work. If we retry many times, people will complain that the run start or stop
takes very long. If we then just continue (without stopping the logger), the MySQL database will miss important
information and the runs probably cannot be analyzed later. So I believe it's better to really stop the logger
so that people get aware that there is a problem and fix the source, rather than curing the symptoms.
In the MEG experiment at PSI we run the logger with a MySQL database and we never see any connection issue,
except when the MySQL server gets in maintenance (once a year), but usually we don't take data then. Since we
use the same logger code, it cannot be a problem there. So I would try to fix the problem on the MySQL side.
Best,
Stefan |
|
2464
|
07 Mar 2023 |
Stefano Piacentini | Info | connection to a MySQL server: retry procedure in the Logger | > > Dear all,
> >
> > we are experiencing a connection problem to the MySQL server that we use to log informations. Is there an
> > option to retry multiple times the I/O on the MySQL?
> >
> > The error we are experiencing is the following (hiding the IP address):
> >
> > [Logger,ERROR] [mlogger.cxx:2455:write_runlog_sql,ERROR] Failed to connect to database: Error: Can't
> > connect to MySQL server on 'xxx.xxx.xxx.xxx:6033' (110)
> >
> > Then the logger stops, and must be restarted. This eventually happens only during the BOR or the EOR.
>
> What would you propose? If the connection does not work, most likely the server is down or busy. If we retry,
> the connection still might not work. If we retry many times, people will complain that the run start or stop
> takes very long. If we then just continue (without stopping the logger), the MySQL database will miss important
> information and the runs probably cannot be analyzed later. So I believe it's better to really stop the logger
> so that people get aware that there is a problem and fix the source, rather than curing the symptoms.
>
> In the MEG experiment at PSI we run the logger with a MySQL database and we never see any connection issue,
> except when the MySQL server gets in maintenance (once a year), but usually we don't take data then. Since we
> use the same logger code, it cannot be a problem there. So I would try to fix the problem on the MySQL side.
>
> Best,
> Stefan
Dear Stefan,
a possible solution could be to define the number of times to retry as a parameter that is 0 by default, as well as a wait time between two subsequent tries. This
would leave the decision on how to handle a possible failed connection to the user. In our case, for example, we would prefer to not stop the acquisition in case
of a failed connection to the external SQL. In addition, we have other software that, with a retry procedure, doesn’t fail: with 1 re-try and a sleep time of 0.5 s
we already recover 100% of the faults.
Anyway, we implemented a local database, which is a mirror of the external one, and the problems disappeared.
Thanks,
Stefano. |
|
1071
|
15 Jul 2015 |
Konstantin Olchanski | Bug Fix | compiler warnings cleaned up | Latest C/C++ compilers (MacOS 10.10, GCC on RHEL7 and Ubuntu) generate a large number of new
warnings about unused variables, unused functions, dead code, failure to check return values of system
calls, etc.
Some of these warnings catch real bugs so we do not want to turn them off.
Most of these warnings have been cleaned out in the latest MIDAS code. On MacOS and RHEL6 Linux MIDA
S compiles without any warnings. On RHEL7 and Ubuntu Linux there are some warnings from a few
problematic files, history.c being the worst (it will be eventually cleaned out).
K.O. |
|
360
|
06 Mar 2007 |
Konstantin Olchanski | Info | commited mhttpd fixes & improvements | I commited the mhttpd fixes and improvements to the history code accumulated while running the ALPHA
experiment at CERN:
- fix crashes and infinite loops while generating history plots (also seen in TWIST)
- permit more than 10 variables per history plot
- let users set their own colours for variables on history plot
- (finally) add gui elements for setting mimimum and maximum values on a plot
- implement special "history" mode. In this mode, the master mhttpd does all the work, except for
generating of history plots, which is done in a separate mhttpd running in history mode, possibly on a
different computer (via ODB variable "/history/url").
I also have improvements to the mhttpd elog code (better formatting of email) and to the "export history
plot as CSV" function, which I will not be commiting: for elog, we switched to the standalone elogd; and
CSV export is still very broken, even with my fixes.
The commited fixes have been in use at CERN since last Summer, but I could have introduced errors
during the merge & commit. I am now using this new code, so any new errors should surface and get
squashed quickly.
K.O. |
|
264
|
08 Jun 2006 |
Konstantin Olchanski | Bug Fix | commit latest ccusb.c CAMAC-USB driver | I commited the latest driver for the Wiener CCUSB USB-CAMAC driver. It
implements all functions from mcstd.h and has been tested to be plug-compatible
with at least one of our CAMAC frontends. K.O. |
|
305
|
23 Sep 2006 |
Konstantin Olchanski | Bug Fix | commit latest ccusb.c CAMAC-USB driver | > I commited the latest driver for the Wiener CCUSB USB-CAMAC driver. It
> implements all functions from mcstd.h and has been tested to be plug-compatible
> with at least one of our CAMAC frontends. K.O.
This driver is known to not work with the latest CCUSB firmware (20x, 204, 30x, 303). I know what
modifications are required and an updated driver will be available shortly. If there is a delay, and you need the
driver ASAP, please drop me an email.
Also, I am thinking about dropping support for the very old CCUSB firmware revisions (before 204). (Any
comments?)
K.O. |
|
402
|
22 Aug 2007 |
Konstantin Olchanski | Bug Fix | commit latest ccusb.c CAMAC-USB driver | > > I commited the latest driver for the Wiener CCUSB USB-CAMAC driver. It
> > implements all functions from mcstd.h and has been tested to be plug-compatible
> > with at least one of our CAMAC frontends. K.O.
Well, it took almost a year to finish an updated driver, which has now been
commited to MIDAS SVN (see http://savannah.psi.ch/viewcvs/trunk/drivers/camac/ccusb/?root=midas).
This supports ccusb firmware release 0x402. With earlier firmware, simple CAMAC operations should work,
but to use the readout list feature one has to have the latest main firmware (0x402 as of today) and the latest CPLD
firmware.
The driver kit includes:
- the "ccusb" driver which implements the MIDAS mcstd.h CAMAC interface;
- test_ccusb to probe the interface and generally make the lights flash;
- ccusb_flash for updating the ccusb main firmware (assembled from bits and pieces found on the CCUSB driver CD);
- feccusb, an example midas frontend, which uses the ccusb readout list feature and has extensive error handling,
should be good enough for production use (unlike the Wiener libxxusb drivers, which lack basic error handling).
- analyzer.cxx, an rootana-based example on how to decode the ccusb data;
- README file with release notes.
If you use this driver, please drop me an email (even if it works perfectly for you, hah!) - the ccusb device is very
nice but can be hard to use and I would like to hear about problems other people have with it.
Today's version of the README files is attached below:
MIDAS driver for the Wiener/JTec CC-USB CAMAC-USB interface.
Date: 22-AUG-2007/KO
Note 1: The CC-USB interface comes with a CD which contains manuals,
firmware files, Windows and Linux software. The Wiener/JTec driver
is called "libxxusb". These MIDAS/musbstd drivers were written before
libxxusb bacame available and do not use libxxusb.
This driver implements the MIDAS CAMAC interafce "mcstd.h" using
the MIDAS USB interface musbstd.h.
Note 2: There exist many revisions of CCUSB firmware. Basic CAMAC
access works in all of them, but the "readout list" feature seems
to be only functional with firmware revision 0x402 or older and
with CPLD revisions CC_atmmgr_101406.jed, CC_datamgr_021905.jed,
CC_lammgr_brdcst_041906.jed or older.
To upgrade the main CCUSB firmware, follow instructions from
the CCUSB manual. On Linux, one can use the ccusb_flash
program included with these MIDAS drivers. It is a copy
of ccusb_flash from the Wiener CD, with all the pieces
assembled into one place and with a working Makefile. (I am too
lazy to add the flashing bits to the ccusb.c driver).
To upgrade the CPLD firmware, one needs a Xilinx JTag programmer
cable (we use a "parallel port to JTag" cable provided by Wiener),
and the Xilinx software (on Linux, we use Xilinx91i). For successful
upgrade, follow instructions from Xilinx and Wiener.
Note 3: Before starting to use the CCUSB interface, one should obtain
the latest version of the CCUSB manual and firmware by downloading
the latest version the CCUSB driver CD from the Wiener web
site (registration required)
Note 4: The example CCUSB frontend assumes this hardware configuration:
LeCroy 2249A 12 channel ADC in slot 20, Kinetic Systems 3615 6 channel
scaler in slot 12. NIM trigger input connected to CCUSB input "I1"
firing at 10-100 Hz. Without the external trigger CCUSB will not
generate any data and the frontend will only give "data timeout"
errors. With the trigger, the LED on the scaler should flash at 1 Hz
and the LEDs on the CCUSB should flash at the trigger rate.
Note 5: The CCUSB interface does not reliably power up in some CAMAC
crates (this has something to do with the sequence in which
different voltages start at different times with different CAMAC
power supplies). Some newer CCUSB modules may have this
problem fixed in the hardware and in the CPLD firmware. For modules
exhibiting this problem (i.e. no USB communication after power up),
try to cycle the power several time, or implement the "hardware reset
switch" (ask Wiener).
Note 6: The CCUSB firmware is very fickle and would crash if you look
at it the wrong way. This MIDAS driver tries to avoid all known crashers
and together with the example frontend, can recover from some
of them. Other crashes cannot be recovered from other than by
a hardware reset or power cycle.
//end |
|
2787
|
04 Jul 2024 |
Pavel Murat | Suggestion | cmake-installing more files ? | Dear all,
this posting results from the Fermilab move to a new packaging/build system called spack
which doesn't allow to use the MIDAS install procedure described at
https://daq00.triumf.ca/MidasWiki/index.php/Quickstart_Linux#MIDAS_Package_Installation
as is. Spack specifics aside, building MIDAS under spack took
a) adding cmake install for three directories: drivers, resources, and python/midas,
b) adding one more include file - include/tinyexpr.h - to the list of includes installed by cmake.
With those changes I was able to point MIDASSYS to the spack install area and successfully run mhttpd,
build experiment-specific C++ frontends and drivers, use experiment-specific python frontends etc.
I'm not using anything from MIDAS submodules though.
I'm wondering what the experts would think about accepting the changes above to the main tree.
Installation procedures and changed to cmake files are always a sensitive area with a lot of boundary
constraints coming from the existing use patterns, and even a minor change could have unexpected consequences
So I wouldn't be surprised if the fairly minor changes outlined above had side effects.
The patch file is attached for consideration.
-- regards, Pasha |
| Attachment 1: midas-spack.patch
|
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 3c6a4109..57dab96f 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -642,6 +642,7 @@ else()
include/musbstd.h
include/mvmestd.h
include/odbxx.h
+ include/tinyexpr.h
include/tmfe.h
include/tmfe_rev0.h
mxml/mxml.h
@@ -657,6 +658,9 @@ install(TARGETS midas midas-shared midas-c-compat mfe mana rmana
install(EXPORT ${PROJECT_NAME}-targets DESTINATION lib)
+install(DIRECTORY drivers DESTINATION . )
+install(DIRECTORY resources DESTINATION . )
+install(DIRECTORY python/midas DESTINATION python)
#####################################################################
# generate git revision file
#####################################################################
|
|
2797
|
06 Aug 2024 |
Stefan Ritt | Suggestion | cmake-installing more files ? | I don't see any bad side effects at the moment, so I accepted the changes and committed them.
Stefan |
|
2203
|
04 Jun 2021 |
Andreas Suter | Bug Report | cmake with CMAKE_INSTALL_PREFIX fails | Hi,
if I check out midas and try to configure it with
cmake ../ -DCMAKE_INSTALL_PREFIX=/usr/local/midas
I do get the error messages:
Target "midas" INTERFACE_INCLUDE_DIRECTORIES property contains path:
"<path>/tmidas/midas/include"
which is prefixed in the source directory.
Is the cmake setup not relocatable? This is new and was working until recently:
MIDAS version: 2.1
GIT revision: Thu May 27 12:56:06 2021 +0000 - midas-2020-08-a-295-gfd314ca8-dirty on branch HEAD
ODB version: 3 |
|
2204
|
04 Jun 2021 |
Konstantin Olchanski | Bug Report | cmake with CMAKE_INSTALL_PREFIX fails | > cmake ../ -DCMAKE_INSTALL_PREFIX=/usr/local/midas
good timing, I am working on cmake for manalyzer and rootana and I have not tested
the install prefix business.
now I know to test it for all 3 packages.
I will also change find_package(Midas) slightly, (see my other message here),
I hope you can confirm that I do not break it for you.
K.O. |
|
2206
|
04 Jun 2021 |
Konstantin Olchanski | Bug Report | cmake with CMAKE_INSTALL_PREFIX fails | > cmake ../ -DCMAKE_INSTALL_PREFIX=/usr/local/midas
> Is the cmake setup not relocatable? This is new and was working until recently:
Indeed. Not relocatable. This is because we do not install the header files.
When you use the CMAKE_INSTALL_PREFIX, you get MIDAS "installed" in:
prefix/lib
prefix/bin
$MIDASSYS/include <-- this is the source tree and so not "relocatable"!
Before, this was kludged and cmake did not complain about it.
Now I changed cmake to handle the include path "the cmake way", and now it knows to complain about it.
I am not sure how to fix this: we have a conflict between:
- our normal way of using midas (include $MIDASSYS/include, link $MIDASSYS/lib, run $MIDASSYS/bin)
- the cmake way (packages *must be installed* or else! but I do like install(EXPORT)!)
- and your way (midas include files are in $MIDASSYS/include, everything else is in your special location)
I think your case is strange. I am curious why you want midas libraries to be in prefix/lib instead of in
$MIDASSYS/lib (in the source tree), but are happy with header files remaining in the source tree.
K.O. |
|
2208
|
04 Jun 2021 |
Andreas Suter | Bug Report | cmake with CMAKE_INSTALL_PREFIX fails | > > cmake ../ -DCMAKE_INSTALL_PREFIX=/usr/local/midas
> > Is the cmake setup not relocatable? This is new and was working until recently:
>
> Indeed. Not relocatable. This is because we do not install the header files.
>
> When you use the CMAKE_INSTALL_PREFIX, you get MIDAS "installed" in:
>
> prefix/lib
> prefix/bin
> $MIDASSYS/include <-- this is the source tree and so not "relocatable"!
>
> Before, this was kludged and cmake did not complain about it.
>
> Now I changed cmake to handle the include path "the cmake way", and now it knows to complain about it.
>
> I am not sure how to fix this: we have a conflict between:
>
> - our normal way of using midas (include $MIDASSYS/include, link $MIDASSYS/lib, run $MIDASSYS/bin)
> - the cmake way (packages *must be installed* or else! but I do like install(EXPORT)!)
> - and your way (midas include files are in $MIDASSYS/include, everything else is in your special location)
>
> I think your case is strange. I am curious why you want midas libraries to be in prefix/lib instead of in
> $MIDASSYS/lib (in the source tree), but are happy with header files remaining in the source tree.
>
> K.O.
We do it this way, since the lib and bin needs to be in a place where standard users have no access to.
If I think an all other packages I am working with, e.g. ROOT, the includes are also installed under CMAKE_INSTALL_PREFIX.
Up until recently there was no issue to work with CMAKE_INSTALL_PREFIX, accepting that the includes stay under
$MIDASSYS/include, even though this is not quite the standard way, but no problem here. Anyway, since CMAKE_INSTALL_PREFIX
is a standard option from cmake, I think things should not "break" if you want to use it.
A.S. |
|