Back Midas Rome Roody Rootana
  Midas DAQ System, Page 16 of 146  Not logged in ELOG logo
New entries since:Wed Dec 31 16:00:00 1969
ID Date Author Topicdown Subject
  2754   03 May 2024 Stefan RittSuggestionPossible addition to IF Statements
The tinyexpr library I use to evaluate expressions does not support boolean operations. I would have to switch to the newer 
tineyexpr-plusplus version, which also has much richer functionality:

https://github.com/Blake-Madden/tinyexpr-plusplus/blob/tinyexpr%2B%2B/TinyExprChanges.md

Unfortunately it requires C++17, and at the moment we limit MIDAS to C++11, meaning we would break this requirement. I 
believe at the moment there are still some experiments (mainly at TRIUMF) which are stuck to older OS and therefore cannot 
switch to C++17, but hopefully this will change over time.

Stefan
  2783   22 Jun 2024 Joseph McKennaSuggestionmanalyzer thread safety and custom http IP binding
Hi all, I hope this is the right place to post two pull requests, if not, please let me know where I should be submitting them

Both are fairly small changes, please see them listed below (more details written on the PRs themselves)


- Enable ROOT's thread safety when running in multithreaded mode

This helps avoid users having to write their call to a global thread lock when calling ->Fill() on ROOT histograms and Trees
https://bitbucket.org/tmidas/manalyzer/pull-requests/5


- Add command argument to specify an IP of the root HTTP server to bind to

This was a problem I painted around when at ALPHA (quickly hardcoding the right external IP address into the local build. Obviously a bad habit)
https://bitbucket.org/tmidas/manalyzer/pull-requests/6
  2787   04 Jul 2024 Pavel MuratSuggestioncmake-installing more files ?
Dear all, 

this posting results from the Fermilab move to a new packaging/build system called spack 
which doesn't allow to use the MIDAS install procedure described at

https://daq00.triumf.ca/MidasWiki/index.php/Quickstart_Linux#MIDAS_Package_Installation

as is. Spack specifics aside, building MIDAS under spack took  
a) adding cmake install for three directories: drivers, resources, and python/midas, 
b) adding one more include file - include/tinyexpr.h - to the list of includes installed by cmake.

With those changes I was able to point MIDASSYS to the spack install area and successfully run mhttpd, 
build experiment-specific C++ frontends and drivers, use experiment-specific python frontends etc. 
I'm not using anything from MIDAS submodules though.

I'm wondering what the experts would think about accepting the changes above to the main tree. 

Installation procedures and changed to cmake files are always a sensitive area with a lot of boundary 
constraints coming from the existing use patterns, and even a minor change could have unexpected consequences
So I wouldn't be surprised if the fairly minor changes outlined above had side effects.

The patch file is attached for consideration.

-- regards, Pasha
Attachment 1: midas-spack.patch
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 3c6a4109..57dab96f 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -642,6 +642,7 @@ else()
       include/musbstd.h
       include/mvmestd.h
       include/odbxx.h
+      include/tinyexpr.h
       include/tmfe.h
       include/tmfe_rev0.h
       mxml/mxml.h
@@ -657,6 +658,9 @@ install(TARGETS midas midas-shared midas-c-compat mfe mana rmana
         
 install(EXPORT ${PROJECT_NAME}-targets DESTINATION lib)
 
+install(DIRECTORY drivers       DESTINATION .     )
+install(DIRECTORY resources     DESTINATION .     )
+install(DIRECTORY python/midas  DESTINATION python)
 #####################################################################
 # generate git revision file
 #####################################################################
  2790   05 Jul 2024 Joseph McKennaSuggestionshared pointers for more flexible memory managment of the analysis 'flow' and TMEvent
> Hi all, I hope this is the right place to post two pull requests, if not, please let me know where I should be submitting them
> 
> Both are fairly small changes, please see them listed below (more details written on the PRs themselves)
> 
> 
> - Enable ROOT's thread safety when running in multithreaded mode
> 
> This helps avoid users having to write their call to a global thread lock when calling ->Fill() on ROOT histograms and Trees
> https://bitbucket.org/tmidas/manalyzer/pull-requests/5
> 
> 
> - Add command argument to specify an IP of the root HTTP server to bind to
> 
> This was a problem I painted around when at ALPHA (quickly hardcoding the right external IP address into the local build. Obviously a bad habit)
> https://bitbucket.org/tmidas/manalyzer/pull-requests/6

Further to the pull manalyzer pull requests, I have another feature I would like to add. Took a little longer to test than planned... here I present an effort to use smart pointers to manage the lifetime of TMEvents and TAFlow.

I will be interested to discuss the implications of this pull request (its possible to return to previous 'raw' pointers via a cmake toggle)

https://bitbucket.org/tmidas/manalyzer/pull-requests/8
  2791   05 Jul 2024 Joseph McKennaSuggestionClean up compiler warning in manalyzer
This is a super small pull request, simple replace deprecated sprintf with snprintf

https://bitbucket.org/tmidas/manalyzer/pull-requests/9
  2797   06 Aug 2024 Stefan RittSuggestioncmake-installing more files ?
I don't see any bad side effects at the moment, so I accepted the changes and committed them.

Stefan
  2814   30 Aug 2024 Marius KoeppelSuggestionImprove Event Documentation
Hi,

I am writing a Rust based midas file reader however it was kind of hard to understand the full midas file 
structure from the documentation.

Only at the end of the page 
https://daq00.triumf.ca/MidasWiki/index.php/Event_Structure#MIDAS_Format_Event one finds under the 
headline “tape format” that there are special events which mark the start and the end of the run. It would 
be better to place this information more prominent maybe we a headline: “Special Events”. Maybe a link to 
this section at the top of the page could help. Also at the mlogger page there is no information about this.

Best,
Marius
  2816   01 Sep 2024 Stefan RittSuggestionImprove Event Documentation
> Hi,
> 
> I am writing a Rust based midas file reader however it was kind of hard to understand the full midas file 
> structure from the documentation.
> 
> Only at the end of the page 
> https://daq00.triumf.ca/MidasWiki/index.php/Event_Structure#MIDAS_Format_Event one finds under the 
> headline “tape format” that there are special events which mark the start and the end of the run. It would 
> be better to place this information more prominent maybe we a headline: “Special Events”. Maybe a link to 
> this section at the top of the page could help. Also at the mlogger page there is no information about this.
> 
> Best,
> Marius

Ben was so kind to update the event documentation:

  https://daq00.triumf.ca/MidasWiki/index.php/Event_Structure

Please have a look and let us know if that's better now.

Best,
Stefan
  2817   01 Sep 2024 Marius KoeppelSuggestionImprove Event Documentation
> > Hi,
> > 
> > I am writing a Rust based midas file reader however it was kind of hard to understand the full midas file 
> > structure from the documentation.
> > 
> > Only at the end of the page 
> > https://daq00.triumf.ca/MidasWiki/index.php/Event_Structure#MIDAS_Format_Event one finds under the 
> > headline “tape format” that there are special events which mark the start and the end of the run. It would 
> > be better to place this information more prominent maybe we a headline: “Special Events”. Maybe a link to 
> > this section at the top of the page could help. Also at the mlogger page there is no information about this.
> > 
> > Best,
> > Marius
> 
> Ben was so kind to update the event documentation:
> 
>   https://daq00.triumf.ca/MidasWiki/index.php/Event_Structure
> 
> Please have a look and let us know if that's better now.
> 
> Best,
> Stefan

Thank you Ben! Now its super clear!
  2818   02 Sep 2024 Daniel DuqueSuggestionImprove Event Documentation
> I am writing a Rust based midas file reader

You might find this library I wrote useful: https://crates.io/crates/midasio

It should "just work", and if it doesn't, I would be interested to know.
  2819   02 Sep 2024 Marius KoeppelSuggestionImprove Event Documentation
> > I am writing a Rust based midas file reader
> 
> You might find this library I wrote useful: https://crates.io/crates/midasio
> 
> It should "just work", and if it doesn't, I would be interested to know.

Nice! I did not know about this. I have now also one simple reader but yours looks much more advanced. My 
overall idea here is to connect directly to midas so having some frontend features to analyze the data etc. do 
you also have already a library for this? I can also extend your stuff.

Best,
Marius
  2820   02 Sep 2024 Daniel DuqueSuggestionImprove Event Documentation
> My overall idea here is to connect directly to midas so having some frontend features to analyze the data etc. do 
> you also have already a library for this? I can also extend your stuff.

No, sadly I don't have something like this yet. It has been on my "fun things to do at some point" list for too
long, but I haven't had the time.

If you start working on something like this, please keep me in the loop/link a repo here. I would be interested
on keeping an eye/contributing to something like this :)
  2835   11 Sep 2024 Konstantin OlchanskiSuggestionImprove Event Documentation
> I am writing a Rust based midas file reader however it was kind of hard to understand the full midas file 
> structure from the documentation.

MIDAS is old-school, when the code was the documentation.

This is very noticeable when you try to document things MIDAS (as I have done many times).

For MIDAS data format, file level and bank level, best if you look at my midasio library (included with MIDAS 
git clone) and translate it to Rust directly. I think a Rust version of C++ midasio would be very welcome.

Many data fields in MIDAS files are mysterious and I reverse-engineered them the best I could.

The main problems were:
- data padding
- "length" fields include padding or not?
- identification of big-endian vs little-endian data
- probably something I forget

K.O.
  2843   13 Sep 2024 Konstantin OlchanskiSuggestionmanalyzer thread safety and custom http IP binding
> - Enable ROOT's thread safety when running in multithreaded mode
> This helps avoid users having to write their call to a global thread lock when calling ->Fill() on ROOT histograms and Trees
> https://bitbucket.org/tmidas/manalyzer/pull-requests/5

merged by hand. (pull request shows a "rejected", bitbucket has no "merged manually" button).

also noted this change in the documentation: README.md

K.O.
  2844   13 Sep 2024 Konstantin OlchanskiSuggestionClean up compiler warning in manalyzer
> This is a super small pull request, simple replace deprecated sprintf with snprintf
> https://bitbucket.org/tmidas/manalyzer/pull-requests/9

sprintf() is not deprecated and "char buf[256]; sprintf(buf, "%05d", 64-bit-int);" is safe, will never overflow.

we could bulk-convert all these sprintf() to snprintf() but I would rather wait for this:

https://en.cppreference.com/w/cpp/utility/format/format

let me think on this for a bit.

K.O.
  2854   20 Sep 2024 Joseph McKennaSuggestionClean up compiler warning in manalyzer
> > This is a super small pull request, simple replace deprecated sprintf with snprintf
> > https://bitbucket.org/tmidas/manalyzer/pull-requests/9
> 
> sprintf() is not deprecated and "char buf[256]; sprintf(buf, "%05d", 64-bit-int);" is safe, will never overflow.
> 
> we could bulk-convert all these sprintf() to snprintf() but I would rather wait for this:
> 
> https://en.cppreference.com/w/cpp/utility/format/format
> 
> let me think on this for a bit.
> 
> K.O.

I completely agree that the 64-bit int is safe and will never overflow. Doing a little digging, both clang and gcc don't raise warnings on x86_64 (even with -Wall -Wextra -Wpedantic), even when I give it a buffer impossibly small (two bytes). However I've narrowed down the depreciation warning comes from: MacOS

https://developer.apple.com/documentation/kernel/1441083-sprintf

I like the look of std::format, looks cleaner than string streams
  2855   20 Sep 2024 Stefan RittSuggestionClean up compiler warning in manalyzer
> I like the look of std::format, looks cleaner than string streams

I fully agree. String streams is a pain if you want to do zero-leading hex output mixed with decimal output. Yes it's easier to read if you don't know printf syntax,
but 10-20 times more chars to write and not necessarily cleaner.

Proble is that we would have to convert about a few thousand of sprintf's() in midas.

Stefan
  2859   24 Sep 2024 Konstantin OlchanskiSuggestionClean up compiler warning in manalyzer
> > I like the look of std::format, looks cleaner than string streams
> 
> I fully agree. String streams is a pain if you want to do zero-leading hex output mixed with decimal output. Yes it's easier to read if you don't know printf syntax,
> but 10-20 times more chars to write and not necessarily cleaner.
>

IMO c++ string streams formatting is optimized for "hello world" and is useless for printing hex numbers, table-formatted data and generally anything real-life.

plus the borked std::to_string() (it takes a global lock for the "C" locale), "fixed" it by introducing std::to_chars() in C++17,
with "ultimate fix" in std::format in C++26.

no question why C++ has the bad reputation. for a "done right" example, take a look at the Go standard library.

> 
> Probable is that we would have to convert about a few thousand of sprintf's() in midas.
> 

surprising few bare sprintf() remaining in MIDAS, most of them overflow-safe and most of them to be converted to msprintf().

K.O.
  2871   09 Oct 2024 Lukas GerritzenSuggestionodbedit minor quality of life
I have made two minor quality of life changes to odbedit.
  • cd command: Typing cd without arguments now changes the directory to /, similar to the behaviour of the cd command in Linux sending you to the home directory.
  • Exit behavior: Upon exiting the program with Ctrl+C, a newline character is printed so that the command line starts on an empty line rather than the last line from odbedit.
Here's the diff:
@@ -1668,7 +1668,10 @@ int command_loop(char *host_name, char *exp_name, char *cmd, char *start_dir)

       /* cd */
       else if (param[0][0] == 'c' && param[0][1] == 'd') {
-         compose_name(pwd, param[1], str);
+         if (strlen(param[1]) == 0)
+            strcpy(str, "/");
+         else
+            compose_name(pwd, param[1], str);

          status = db_find_key(hDB, 0, str, &hKey);

@@ -2962,6 +2965,7 @@ void ctrlc_odbedit(INT i)

    cm_disconnect_experiment();

+   printf("\n");
    exit(EXIT_SUCCESS);
 }

Please consider incorporating those changes to odbedit.

Lukas
  2872   09 Oct 2024 Stefan RittSuggestionodbedit minor quality of life
Ok, accepted, done and pushed.

Stefan


Lukas Gerritzen wrote:
I have made two minor quality of life changes to odbedit.
  • cd command: Typing cd without arguments now changes the directory to /, similar to the behaviour of the cd command in Linux sending you to the home directory.
  • Exit behavior: Upon exiting the program with Ctrl+C, a newline character is printed so that the command line starts on an empty line rather than the last line from odbedit.
Here's the diff:
@@ -1668,7 +1668,10 @@ int command_loop(char *host_name, char *exp_name, char *cmd, char *start_dir)

       /* cd */
       else if (param[0][0] == 'c' && param[0][1] == 'd') {
-         compose_name(pwd, param[1], str);
+         if (strlen(param[1]) == 0)
+            strcpy(str, "/");
+         else
+            compose_name(pwd, param[1], str);

          status = db_find_key(hDB, 0, str, &hKey);

@@ -2962,6 +2965,7 @@ void ctrlc_odbedit(INT i)

    cm_disconnect_experiment();

+   printf("\n");
    exit(EXIT_SUCCESS);
 }

Please consider incorporating those changes to odbedit.

Lukas
ELOG V3.1.4-2e1708b5