The tinyexpr library I use to evaluate expressions does not support boolean operations. I would have to switch to the newer 
tineyexpr-plusplus version, which also has much richer functionality:

https://github.com/Blake-Madden/tinyexpr-plusplus/blob/tinyexpr%2B%2B/TinyExprChanges.md

Unfortunately it requires C++17, and at the moment we limit MIDAS to C++11, meaning we would break this requirement. I 
believe at the moment there are still some experiments (mainly at TRIUMF) which are stuck to older OS and therefore cannot 
switch to C++17, but hopefully this will change over time.

Stefan

Hi all, I hope this is the right place to post two pull requests, if not, please let me know where I should be submitting them

Both are fairly small changes, please see them listed below (more details written on the PRs themselves)


- Enable ROOT's thread safety when running in multithreaded mode

This helps avoid users having to write their call to a global thread lock when calling ->Fill() on ROOT histograms and Trees
https://bitbucket.org/tmidas/manalyzer/pull-requests/5


- Add command argument to specify an IP of the root HTTP server to bind to

This was a problem I painted around when at ALPHA (quickly hardcoding the right external IP address into the local build. Obviously a bad habit)
https://bitbucket.org/tmidas/manalyzer/pull-requests/6

Dear all, 

this posting results from the Fermilab move to a new packaging/build system called spack 
which doesn't allow to use the MIDAS install procedure described at

https://daq00.triumf.ca/MidasWiki/index.php/Quickstart_Linux#MIDAS_Package_Installation

as is. Spack specifics aside, building MIDAS under spack took  
a) adding cmake install for three directories: drivers, resources, and python/midas, 
b) adding one more include file - include/tinyexpr.h - to the list of includes installed by cmake.

With those changes I was able to point MIDASSYS to the spack install area and successfully run mhttpd, 
build experiment-specific C++ frontends and drivers, use experiment-specific python frontends etc. 
I'm not using anything from MIDAS submodules though.

I'm wondering what the experts would think about accepting the changes above to the main tree. 

Installation procedures and changed to cmake files are always a sensitive area with a lot of boundary 
constraints coming from the existing use patterns, and even a minor change could have unexpected consequences
So I wouldn't be surprised if the fairly minor changes outlined above had side effects.

The patch file is attached for consideration.

-- regards, Pasha

> Hi all, I hope this is the right place to post two pull requests, if not, please let me know where I should be submitting them
> 
> Both are fairly small changes, please see them listed below (more details written on the PRs themselves)
> 
> 
> - Enable ROOT's thread safety when running in multithreaded mode
> 
> This helps avoid users having to write their call to a global thread lock when calling ->Fill() on ROOT histograms and Trees
> https://bitbucket.org/tmidas/manalyzer/pull-requests/5
> 
> 
> - Add command argument to specify an IP of the root HTTP server to bind to
> 
> This was a problem I painted around when at ALPHA (quickly hardcoding the right external IP address into the local build. Obviously a bad habit)
> https://bitbucket.org/tmidas/manalyzer/pull-requests/6

Further to the pull manalyzer pull requests, I have another feature I would like to add. Took a little longer to test than planned... here I present an effort to use smart pointers to manage the lifetime of TMEvents and TAFlow.

I will be interested to discuss the implications of this pull request (its possible to return to previous 'raw' pointers via a cmake toggle)

https://bitbucket.org/tmidas/manalyzer/pull-requests/8

This is a super small pull request, simple replace deprecated sprintf with snprintf

https://bitbucket.org/tmidas/manalyzer/pull-requests/9

I don't see any bad side effects at the moment, so I accepted the changes and committed them.

Stefan

Hi,

I am writing a Rust based midas file reader however it was kind of hard to understand the full midas file 
structure from the documentation.

Only at the end of the page 
https://daq00.triumf.ca/MidasWiki/index.php/Event_Structure#MIDAS_Format_Event one finds under the 
headline “tape format” that there are special events which mark the start and the end of the run. It would 
be better to place this information more prominent maybe we a headline: “Special Events”. Maybe a link to 
this section at the top of the page could help. Also at the mlogger page there is no information about this.

Best,
Marius

> Hi,
> 
> I am writing a Rust based midas file reader however it was kind of hard to understand the full midas file 
> structure from the documentation.
> 
> Only at the end of the page 
> https://daq00.triumf.ca/MidasWiki/index.php/Event_Structure#MIDAS_Format_Event one finds under the 
> headline “tape format” that there are special events which mark the start and the end of the run. It would 
> be better to place this information more prominent maybe we a headline: “Special Events”. Maybe a link to 
> this section at the top of the page could help. Also at the mlogger page there is no information about this.
> 
> Best,
> Marius

Ben was so kind to update the event documentation:

  https://daq00.triumf.ca/MidasWiki/index.php/Event_Structure

Please have a look and let us know if that's better now.

Best,
Stefan

> > Hi,
> > 
> > I am writing a Rust based midas file reader however it was kind of hard to understand the full midas file 
> > structure from the documentation.
> > 
> > Only at the end of the page 
> > https://daq00.triumf.ca/MidasWiki/index.php/Event_Structure#MIDAS_Format_Event one finds under the 
> > headline “tape format” that there are special events which mark the start and the end of the run. It would 
> > be better to place this information more prominent maybe we a headline: “Special Events”. Maybe a link to 
> > this section at the top of the page could help. Also at the mlogger page there is no information about this.
> > 
> > Best,
> > Marius
> 
> Ben was so kind to update the event documentation:
> 
>   https://daq00.triumf.ca/MidasWiki/index.php/Event_Structure
> 
> Please have a look and let us know if that's better now.
> 
> Best,
> Stefan

Thank you Ben! Now its super clear!

> I am writing a Rust based midas file reader

You might find this library I wrote useful: https://crates.io/crates/midasio

It should "just work", and if it doesn't, I would be interested to know.

> > I am writing a Rust based midas file reader
> 
> You might find this library I wrote useful: https://crates.io/crates/midasio
> 
> It should "just work", and if it doesn't, I would be interested to know.

Nice! I did not know about this. I have now also one simple reader but yours looks much more advanced. My 
overall idea here is to connect directly to midas so having some frontend features to analyze the data etc. do 
you also have already a library for this? I can also extend your stuff.

Best,
Marius

> My overall idea here is to connect directly to midas so having some frontend features to analyze the data etc. do 
> you also have already a library for this? I can also extend your stuff.

No, sadly I don't have something like this yet. It has been on my "fun things to do at some point" list for too
long, but I haven't had the time.

If you start working on something like this, please keep me in the loop/link a repo here. I would be interested
on keeping an eye/contributing to something like this :)

> I am writing a Rust based midas file reader however it was kind of hard to understand the full midas file 
> structure from the documentation.

MIDAS is old-school, when the code was the documentation.

This is very noticeable when you try to document things MIDAS (as I have done many times).

For MIDAS data format, file level and bank level, best if you look at my midasio library (included with MIDAS 
git clone) and translate it to Rust directly. I think a Rust version of C++ midasio would be very welcome.

Many data fields in MIDAS files are mysterious and I reverse-engineered them the best I could.

The main problems were:
- data padding
- "length" fields include padding or not?
- identification of big-endian vs little-endian data
- probably something I forget

K.O.

> - Enable ROOT's thread safety when running in multithreaded mode
> This helps avoid users having to write their call to a global thread lock when calling ->Fill() on ROOT histograms and Trees
> https://bitbucket.org/tmidas/manalyzer/pull-requests/5

merged by hand. (pull request shows a "rejected", bitbucket has no "merged manually" button).

also noted this change in the documentation: README.md

K.O.

> This is a super small pull request, simple replace deprecated sprintf with snprintf
> https://bitbucket.org/tmidas/manalyzer/pull-requests/9

sprintf() is not deprecated and "char buf[256]; sprintf(buf, "%05d", 64-bit-int);" is safe, will never overflow.

we could bulk-convert all these sprintf() to snprintf() but I would rather wait for this:

https://en.cppreference.com/w/cpp/utility/format/format

let me think on this for a bit.

K.O.

> > This is a super small pull request, simple replace deprecated sprintf with snprintf
> > https://bitbucket.org/tmidas/manalyzer/pull-requests/9
> 
> sprintf() is not deprecated and "char buf[256]; sprintf(buf, "%05d", 64-bit-int);" is safe, will never overflow.
> 
> we could bulk-convert all these sprintf() to snprintf() but I would rather wait for this:
> 
> https://en.cppreference.com/w/cpp/utility/format/format
> 
> let me think on this for a bit.
> 
> K.O.

I completely agree that the 64-bit int is safe and will never overflow. Doing a little digging, both clang and gcc don't raise warnings on x86_64 (even with -Wall -Wextra -Wpedantic), even when I give it a buffer impossibly small (two bytes). However I've narrowed down the depreciation warning comes from: MacOS

https://developer.apple.com/documentation/kernel/1441083-sprintf

I like the look of std::format, looks cleaner than string streams

> I like the look of std::format, looks cleaner than string streams

I fully agree. String streams is a pain if you want to do zero-leading hex output mixed with decimal output. Yes it's easier to read if you don't know printf syntax,
but 10-20 times more chars to write and not necessarily cleaner.

Proble is that we would have to convert about a few thousand of sprintf's() in midas.

Stefan

> > I like the look of std::format, looks cleaner than string streams
> 
> I fully agree. String streams is a pain if you want to do zero-leading hex output mixed with decimal output. Yes it's easier to read if you don't know printf syntax,
> but 10-20 times more chars to write and not necessarily cleaner.
>

IMO c++ string streams formatting is optimized for "hello world" and is useless for printing hex numbers, table-formatted data and generally anything real-life.

plus the borked std::to_string() (it takes a global lock for the "C" locale), "fixed" it by introducing std::to_chars() in C++17,
with "ultimate fix" in std::format in C++26.

no question why C++ has the bad reputation. for a "done right" example, take a look at the Go standard library.

> 
> Probable is that we would have to convert about a few thousand of sprintf's() in midas.
> 

surprising few bare sprintf() remaining in MIDAS, most of them overflow-safe and most of them to be converted to msprintf().

K.O.

• ODB over Frontend precedence.
  When starting a FE client, the equipment settings are taken from the ODB if this equipment already existed. This meant the ODB has precedence over the EQUIPEMENT structure and whatever change you apply to the C-Structure, it will NOT be taken in consideration until you clean (remove) the equipment tree in ODB.
  
  
• Revived 64 bit support. This was required as more OS are already supporting such architecture. Originally Midas did support Alpha/OSF/1 which operated on 64 bit machine. This new code has been tested on SL4.2 with Dual-Core 64-bit AMD Opterons.
  
  
• Multi-threading in Slow Control equipments.
  Check entry 289 in Midas Elog from Stefan.
  
  
• mhttpd using external Elog.
  The standalone ELOG package can be coupled to an existing experiment and therefore supersede the internal elog functionality from mhttpd.
  This requires a particular configuration which is described in the documentation.
  
  
• MySQL test in mlogger
  A reminder that mlogger can generate entries in a MySQL database as long as the pre-compilation flag -HAVE_MYSQL is enabled during system built. The access and form filling is then defined from the ODB under Logger/SQL once the logger is running, see documentation.
  
  
• Directory destination for midas.log and odb dump files
  It is now possible to specify an individual directory to the default midas.log file as well as to the "ODB Dump file" destination. If either of these fields contains a preceding directory, it will take the string as an absolute path to the file.
  
  
• User defined "event Data buffer size" (ODB)
  The event buffer size has been until now defined at the system level in midas.h. It is now possible to optimize the memory allocation specific to the event buffer with an entry in the ODB under /experiment, see documentation.
  
  
• History group display
  It is now possible to display an individual group of history plots. No documentation on that topics as it should be self explanatory.
  
  
• History export option
  From the History web page, it is possible to export to a ASCII .csv file the history content. This file can later be imported into excel for example. No documentation on that topics as it should be self explanatory.
  
  
• Multiple "minor" corrections:
  - Alarm reset for multiple experiment (return directly to the experiment).
  - mdump -b option bug fixed.
  - Alarm evaluation function fixed.
  - mlogger/SQL boolean handling fixed.
  - bm_get_buffer_level() was returning a wrong value which has been fixed now.
  - Event buffer bug traced and exterminated (Thanks to Konstantin).
  

I committed an updated lazylogger with updated documentation. The new version supports subruns and 
can save to external storage arbitrary files (i.e. odb dump files). It also moves most book keeping out of 
odb to permit handling more files on bigger storage disks.

Example lazylogger scripts for castor (CERN) and dcache (TRIUMF) are in the directory "utils".

The lazylogger documentation was updated to remove obsolete information and to describe the new 
functions. As usual "make dox; cd doxfiles/html; firefox index.html" or see my copy at:

http://ladd00.triumf.ca/~olchansk/midas/Utilities.html#lazylogger_task

svn rev 4615, 4616.
K.O.