I reworked the alarm sounds in mhttpd - now you can turn off all sounds without disabling the 
alarm system for everybody.

a) new checkbox on the "alarms" page to turn off the alarm buzzer sound
b) fixed a bug where the status page will speak the last alarm even if the "speak" checkbox is 
unchecked on the "alarms" page (was coming through the TALK messages)
c) made sure the chat messages are only spoken if "speak" is enabled on the "chat" page
d) these speech and sounds settings are now stored in the browser "localStorage", which means 
they are shared across all open tabs and windows and are preserved across browser sessions and 
computer reboots.

I hope this is an improvement.

There is still one bug remaining - the first (last?) alarm is always spoken twice - 1st time in the loop 
over all alarms and 2nd time through the TALK messages. I do not know how to fix this.

K.O.

currently to link Midas to project one has to do several steps in cmake script:
- do `find_package`
- get Midas location from MIDASSYS, or from MIDAS_LIBRARY_DIRS
- set MIDAS_INCLUDE_DIRS, MIDAS_LIBRARY_DIRS and MIDAS_LIBRARIES to your target
- add sources from Midas for mfe, drivers, etc.

in general cmake already can to all of this automatically, and the only lines you would need are:
- do `find_package(Midas ... PATHS ~/midas_install_location)`
- and do `target_link_libraries(... midas::mfe)`
  (and all include dirs, libs, and deps are propagated automatically)

see PR https://bitbucket.org/tmidas/midas/pull-requests/48
- nothing should break with current setups
- if you want to try new `midas::` targets, try to link e.g. `midas::mfed` to your frontend

After some iterations, we merged the branch with the new build scheme. Now you can compile any midas program as described at

  https://bitbucket.org/tmidas/midas/pull-requests/48?link_source=email

A default CMakeLists.txt file can look like this:

cmake_minimum_required(VERSION 3.17)
project(example)
find_package(Midas REQUIRED PATHS $ENV{MIDASSYS})
add_executable(example example.cpp)
target_link_libraries(example midas::midas)


Which is much simpler than what we had before. The trick now is that the find_package() retrieves all include and link files automatically. 
There are different targets:

midas::midas              - normal midas program
midas::midas-shared       - normal midas programs using the shared midas library
midas::mfe                - old style mfe.cxx frontend
midas::mfed               - newer style frontend using mfed.cxx
midas::mscb               - programs using MSCB system
midas::drivers            - slow control program using any of the standard midas drivers

We are not absolutely sure that all midas installations will work that way, so far we have tested it on RH8, MacOSX with cmake version 
3.29.5.

Comments and bug reports are welcome as usual.

Alex and Stefan

> currently to link Midas to project one has to do several steps ...

this information is incorrect. please read https://daq00.triumf.ca/elog-midas/Midas/2258

a very simple way to use link MIDAS using midas-targets.cmake has been implemented a long time ago.

before proposing a new way of doing things, it would be nice to hear about shortcomings
of the existing stuff. A simple "Konstantin's way sucks" or "this is not the cmake way!"
would have been sufficient.

K.O.

> After some iterations, we merged the branch with the new build scheme.

the commit to implement this change in the manalyzer was not pushed, for reasons unknown.

fixed, commit f2b4dc87ca4830f6bed8667d6a4ee4afd6d242a1

K.O.

> > currently to link Midas to project one has to do several steps ...
> 
> this information is incorrect. please read https://daq00.triumf.ca/elog-midas/Midas/2258
> 
> a very simple way to use link MIDAS using midas-targets.cmake has been implemented a long time ago.

I admit that i did not see your post about targets import
via `include($ENV{MIDASSYS}/lib/midas-targets.cmake)`
before implementing changes to cmake scripts.
But in this respect the way you propose to do it via `include` should still work.
Note however that `include(...)` way is very unusual as one have to know exactly
where `...-targets.cmake` is located and standard way in cmake is via `find_package`
(similar to how e.g. ROOT, Geant4, etc. are found and linked).

The things that changed (and are incompatible with what was before)
is the naming of targets (in `midas-targets.cmake` with `midas::` namespace,
which is standard practice in cmake to distinguish cmake targets from bare library names
(e.g. when you do `link_libraries(midas)` it may be interpreted as linking with `-lmidas`
or if target is defined it does machinery to link actual cmake target; the namespace way
makes it unambiguous).
Though i again admit that maybe the namespace change was a bit too much as it may
have broken previous users of `include($ENV{MIDASSYS}/lib/midas-targets.cmake)`

> 
> before proposing a new way of doing things, it would be nice to hear about shortcomings
> of the existing stuff.

- shortcomings of what was before is usage of non-standard `include(...)`
- one shortcoming i see for new implementation is usage `midas::` namespace
  (mentioned above) that may have broken some setups

> A simple "Konstantin's way sucks" or "this is not the cmake way!"
> would have been sufficient.

- `find_package` is standard and recommended way of finding packages
    - note that `include($ENV{MIDASSYS}/lib/midas-targets.cmake)` should still work
      (but with usage of `midas::midas` instead of simply `midas`)
    - in the end `find_package` works by locating and loading `MidasConfig.cmake`,
      and it now actually does `include("${CMAKE_CURRENT_LIST_DIR}/../../midas-targets.cmake")`,
      so in this respect `find_package` is the same as `include(...)`,
      but it also preserves old behavior of exporting cmake vars for includes/libs
      such that prev uses are unaffected,
      and does a bit more checking such that it can be used for both in- and out-of-tree builds
    
- in addition `find_package` allows to handle components,
  e.g. now it is possible to do
  `find_package(Midas COMPONENTS manalyzer)`
  instead of also doing `include($ENV{MIDASSYS}/lib/manalyzer-targets.cmake)`

> 
> K.O.

Alex

> > > currently to link Midas to project one has to do several steps ...
> > this information is incorrect. please read https://daq00.triumf.ca/elog-midas/Midas/2258
> 
> I admit that i did not see your post about targets import
> via `include($ENV{MIDASSYS}/lib/midas-targets.cmake)`
> before implementing changes to cmake scripts.
>
> But in this respect the way you propose to do it via `include` should still work.
>

I proposed nothing, you did the proposing. I spent many hours trying to understand cmake (mission 
impossible!) and many more hours to implement the previously existing package scheme based
on the cmake "EXPORT" function.

> Note however that `include(...)` way is very unusual as one have to know exactly
> where `...-targets.cmake` is located and standard way in cmake is via `find_package`
> (similar to how e.g. ROOT, Geant4, etc. are found and linked).

Very difficult to cut-and-paste "include($ENV{MIDASSYS}/lib/midas-targets.cmake)".

You cannot simplify-out $ENV{MIDASSYS} because computer cannot read your mind, which of the 10 copies
of midas you want to use from which user account on which day.

Argument about "very unusual" I would buy, I am not a cmake expert and I do not know which package
finding method is in favour today.

>
> Though i again admit that maybe the namespace change was a bit too much as it may
> have broken previous users of `include($ENV{MIDASSYS}/lib/midas-targets.cmake)`
> 

I believe it did break at least one experiment, after updating MIDAS to latest version,
the analyzer would not build.

Speaking of which, did you implement your new scheme for the manalyzer so that it works
in standalone mode (without MIDAS)?

If you did not, now we have two schemes, your new scheme just for MIDAS and my old scheme
for manalyzer *and* MIDAS. xkcd 927.

> 
> - shortcomings of what was before is usage of non-standard `include(...)`
>

You should have started by posting a message spelling it out: Konstantin implemented
a scheme that uses the cmake "export" function to find midas, mfe and manalyzer,
it is very nice and works ok, but it is non-standard/obsoleted/obscure/frowned-upon/
unpopular/I-do-not-like-it/I-did-not-invent-it, and I propose implementing a new scheme
based on find_package().

>
> - one shortcoming i see for new implementation is usage `midas::` namespace
>   (mentioned above) that may have broken some setups
> 

If you think that your changes will break other people code, you should explicitely
say this in a message to this forum and hopefully provide instruction on fixing it,
i.e. in your makefile, please replace "midas" with "midas::midas".

> 
> - `find_package` is standard and recommended way of finding packages
>

Do you have a reference for this? When I look at cmake documentation, I do not see
any specific recommendation on creating packages and finding them. I do see
other people's code for finding packages and often spend hours fighting
them because said methods are designed to work only on the developer's laptop.

P.S. Did anybody ask Ben to update the MidasWiki documentation with the new find_package() information?

K.O.

> > > > currently to link Midas to project one has to do several steps ...
> > > this information is incorrect. please read https://daq00.triumf.ca/elog-midas/Midas/2258
> > 
> > I admit that i did not see your post about targets import
> > via `include($ENV{MIDASSYS}/lib/midas-targets.cmake)`
> > before implementing changes to cmake scripts.
> >
> > But in this respect the way you propose to do it via `include` should still work.
> >
> 
> I proposed nothing, you did the proposing. I spent many hours trying to understand cmake (mission 
> impossible!) and many more hours to implement the previously existing package scheme based
> on the cmake "EXPORT" function.
I agree that cmake is difficult, especially when it comes to creating cmake scripts for library
that should work for other people (as opposed to just using other libraries).
But that is why we should try to follow recommended way of using it.

> 
> > Note however that `include(...)` way is very unusual as one have to know exactly
> > where `...-targets.cmake` is located and standard way in cmake is via `find_package`
> > (similar to how e.g. ROOT, Geant4, etc. are found and linked).
> 
> Very difficult to cut-and-paste "include($ENV{MIDASSYS}/lib/midas-targets.cmake)".
> 
> You cannot simplify-out $ENV{MIDASSYS} because computer cannot read your mind, which of the 10 copies
> of midas you want to use from which user account on which day.
One can use `find_package(Midas PATH $ENV{MIDASSYS})` to set specific location of Midas
(this is mentioned in https://bitbucket.org/tmidas/midas/pull-requests/48)
and without `PATH` argument the default system/user locations are searched.

> 
> Argument about "very unusual" I would buy, I am not a cmake expert and I do not know which package
> finding method is in favour today.
> 
> >
> > Though i again admit that maybe the namespace change was a bit too much as it may
> > have broken previous users of `include($ENV{MIDASSYS}/lib/midas-targets.cmake)`
> > 
> 
> I believe it did break at least one experiment, after updating MIDAS to latest version,
> the analyzer would not build.
This unfortunately was not easy to avoid in this case, as both midas and manalyzer depend on each other:
midas should compile when manalyzer is enabled (as submodule)
and manalyzer should compile with midas as separate lib.
So in this case i would expect both midas and manalyzer be updated at same time to their matching versions.

> 
> Speaking of which, did you implement your new scheme for the manalyzer so that it works
> in standalone mode (without MIDAS)?
The only change on manalyzer i did was to use `find_package(Midas ...)` and `midas::midas` target
(see https://bitbucket.org/tmidas/manalyzer/commits/b219a916).
If there is interest to use same scheme in manalyzer as in Midas i can implement it.

> 
> If you did not, now we have two schemes, your new scheme just for MIDAS and my old scheme
> for manalyzer *and* MIDAS. xkcd 927.
> 
> > 
> > - shortcomings of what was before is usage of non-standard `include(...)`
> >
> 
> You should have started by posting a message spelling it out: Konstantin implemented
> a scheme that uses the cmake "export" function to find midas, mfe and manalyzer,
> it is very nice and works ok, but it is non-standard/obsoleted/obscure/frowned-upon/
> unpopular/I-do-not-like-it/I-did-not-invent-it, and I propose implementing a new scheme
> based on find_package().
As i mentioned i did not see you original post about usage of `include`
and otherwise i may have referenced it and though more about compatibility issues.

> 
> >
> > - one shortcoming i see for new implementation is usage `midas::` namespace
> >   (mentioned above) that may have broken some setups
> > 
> 
> If you think that your changes will break other people code, you should explicitely
> say this in a message to this forum and hopefully provide instruction on fixing it,
> i.e. in your makefile, please replace "midas" with "midas::midas".
In the original message to this thread i posted reference to PR
(https://bitbucket.org/tmidas/midas/pull-requests/48)
where it shows how to use `find_package` with this change.
As i did not expect the direct use of `include()` form
and assumed that manual linking was used (via specifying include/lib paths/names)
some scenarios where code for people broke were missed (not taken into account) by me.

> 
> > 
> > - `find_package` is standard and recommended way of finding packages
> >
> 
> Do you have a reference for this? When I look at cmake documentation, I do not see
> any specific recommendation on creating packages and finding them. I do see
> other people's code for finding packages and often spend hours fighting
> them because said methods are designed to work only on the developer's laptop.
see https://cmake.org/cmake/help/v3.27/guide/importing-exporting/index.html:
- about use of `find_package` see https://cmake.org/cmake/help/latest/guide/using-dependencies/index.html#guide:Using%20Dependencies%20Guide
- about double colon namespace for target see https://cmake.org/cmake/help/v3.27/guide/importing-exporting/index.html
  where it is mentioned "This convention of double-colons gives CMake a hint that the name is an IMPORTED target when it is used by downstream projects".

> 
> P.S. Did anybody ask Ben to update the MidasWiki documentation with the new find_package() information?
> 
> K.O.

> https://cmake.org/cmake/help/latest/guide/using-dependencies/index.html#guide:Using%20Dependencies%20Guide

thank you for providing a link to latest cmake find_package() guide.

I notice that this documentation was added in cmake 3.24 released circa Nov 2022
and does not exist in older versions. (It is easy to see in git history the last
time I touched any cmake stuff in midas).

I still see no documentation on "this is how you write a package that other people can import
using find_package()". Only see documentation on how to use find_package() on packages
that somebody who somehow knows how to do it already wrote.

> In the original message to this thread i posted reference to PR
> (https://bitbucket.org/tmidas/midas/pull-requests/48)

this pull request was rail-roaded through during the holidays without any
discussion on this forum. I was not given an opportunity to comment to it,
it was pushed and merged faster than I could blink.

bottom line. I voted against using cmake and was over-ruled. To me this cmake stuff
is only a source of wasted time and created bad feelings.

if midas is required to use cmake, we should have somebody on the team that at least
understands it and if not love it, at least does not hate it.

K.O.

I figured out the breakage, added a git tag to identify where the cmake incompatible change was made (roughly) 
and posted a note on how to fix it. Please reimburse me for the 2 hours I had to spend on this instead of doing 
useful work. K.O.

The mhttpd json-rpc interface now implements batch requests per
http://www.jsonrpc.org/specification#batch

In the nutshell, instead of a single request, one can send a json array of requests and receive a json 
array of replies.

As a variance from the spec, the midas implementation executes the requests strictly in-order and 
the array of replies corresponds exactly to the array of requests (the spec requires user to use the 
"id" field to match replies to requests, in midas json-rpc, the 1st reply is always to the 1st request,
2nd reply is to the 2nd request and so forth).

See this in action look at resources/example.html and in resources/transition.html

K.O.

I'm not sure if these issues are related or not, but I'm getting an error
message when I want to access the root of the ODB via the webserver:
[mhttpd,ERROR] [mhttpd.cxx:563:rread,ERROR] Cannot read file '/root', read of
4096 returned -1, errno 21 (Is a directory)

I also tried turning the re-direct from http to https off, but this does not
seem to work. I also noticed that the redirect changes the localhost into a
hostname. Where does mongoose take this hostname from?

EDIT: Seems that the change of the hostname is due to a setting in /etc/hosts,
i.e. all my fault ...

EDIT: I think there was some issue with the mhttpd. When I checked the output (I
used screen to run it), it was full of these messages:

ss_semaphore_wait_for: semop/semtimedop(2588679) returned -1, errno 43
(Identifier removed)
al_check: Something is wrong with our semaphore, ss_semaphore_wait_for()
returned 408, aborting.
al_check: Cannot abort - this will lock you out of odb. From this point, MIDAS
will not work correctly. Please read the discussion at
https://midas.triumf.ca/elog/Midas/945

Restarted it and it stopped redirecting. So accessing the root of the ODB via
the webserver is the only issue now.

> I'm not sure if these issues are related or not, but I'm getting an error
> message when I want to access the root of the ODB via the webserver:
> [mhttpd,ERROR] [mhttpd.cxx:563:rread,ERROR] Cannot read file '/root', read of
> 4096 returned -1, errno 21 (Is a directory)

This is an old bug. It was part of the "custom path" confusion. Fixed (I think) in all midas-2019 
releases.

To confirm, which version are you using (run "odbedit ver" or look on the mhttpd "help" page)?

If you have an older version, I recommend that you update to midas-2019-03 (cd midas; git pull; 
git checkout midas-2019-03; make clean; make).

If you feel adventurous, you can also update to the head of the development version
and see all the new features (cmake, c++11, new history pages).

If you do not feel adventurous, wait until we have midas-2019-09 ready, use midas-2019-03 
until then.

K.O.

Please, tell me how to choose a value of a "key" like DCM, pulser period,
presamples, upper thresholds to run a experiment? where can I find the related
informations? 

I am having problems with handling the end-of-run situation in my midas
frontend. I have a device that continuously sends data (over USB) and I read
this data in my "read_event" function.

Everything is good until the end-of-run, at which time this happens:
0) mfe.c calls my read_event() to read the data (loop until the end-of-run
transition)
1) mfe.c calls my end_of_run()
2) here, I tell the device "please stop sending data"
3) all seems good, but wait!!!
4) there is all this data generated between step 0 and step 2 still sitting
inside the device and it has nowhere to go: the run is ended, the output file is
closed, my read_event() will never be called ever again (well, until the next run).

It seems to me mfe.c needs to have one more function, something like
"pre_end_of_run()" that works like this:
0) mfe.c calls my read_event() to read the data (loop until the end-of-run
transition)
1) mfe.c calls pre_end_of_run(), here I tell the device to stop sending data
2) mfe.c calls read_event() for the very last time, to give me the opportunity
to read and send away any data I still may have.
3) mfe.c calls the end_of_run(). The run is truely finished.

Any thoughts?

K.O.

> I am having problems with handling the end-of-run situation in my midas
> frontend. I have a device that continuously sends data (over USB) and I read
> this data in my "read_event" function.
> 
> Everything is good until the end-of-run, at which time this happens:
> 0) mfe.c calls my read_event() to read the data (loop until the end-of-run
> transition)
> 1) mfe.c calls my end_of_run()
> 2) here, I tell the device "please stop sending data"
> 3) all seems good, but wait!!!
> 4) there is all this data generated between step 0 and step 2 still sitting
> inside the device and it has nowhere to go: the run is ended, the output file is
> closed, my read_event() will never be called ever again (well, until the next run).
> 
> It seems to me mfe.c needs to have one more function, something like
> "pre_end_of_run()" that works like this:
> 0) mfe.c calls my read_event() to read the data (loop until the end-of-run
> transition)
> 1) mfe.c calls pre_end_of_run(), here I tell the device to stop sending data
> 2) mfe.c calls read_event() for the very last time, to give me the opportunity
> to read and send away any data I still may have.
> 3) mfe.c calls the end_of_run(). The run is truely finished.
> 
> Any thoughts?

You can achieve the desired functionality without changing mfe.c:

0) mfe.c calls read_event
1) mfe.c calls end_of_run. Your end_of_run tells the device to stop data and flushes
the remaining data. At this point you have to re-make actually a part of the mfe.c
functionality, but basically you need a bm_compose_event() and a bm_send_event(), so
just a few lines of code. If you want to have the final event number right in your
equipment, you also need to update eq->events_sent accordingly. 

Given the fact that 99% of the experiments do not need this functionality, I propose
that we keep mfe.c and you add the few lines of code into your user part of the
specific frontend.

Stefan

[Dear All, I apologize in advance for spamming.]

1) I tried to login into the forum from the lab computer and realized 
   that I forgot my password
2) I tried to reset the password and found that when registering 
   I mistyped my email address, having typed '.giv' instead of '.gov'
   in the domain name, so the recovery email went into nowhere 

(still have one session open on the laptop so can post this question)

- how do I get my email address fixed so I'd be able to reset the password?

-- many thanks, Pasha

The code for storing midas history in an odbc sql database has been committed.
Changes:
include/history_odbc.h, src/history_odbc.cxx --- implementation
src/mlogger.c --- call the history_odbc functions
utils/mh2sql.cxx --- import existing midas history files (*.hst) into an odbc
sql database.

This new code is enabled by the HAVE_ODBC gunk in the Makefile. If compilation
bombs, please let me know and as a work around, comment out all instances of
HAVE_ODBC from your Makefile.

Limitations:
- mhttpd support for reading history data from odbc sql database is missing
- many sql functions are implemented in a very minimalistic form (i.e. when
defining a history event, we blindly ask sql to create the tables, even if they
already exist - this works, but spams the midas log with sql errors).
- error handling is incomplete: after any sql error, the odbc connection is closed.
- only MySQL (and ascii output) are supported: we use mysql-specific data types
as they match midas types exactly. Code to support PgSQL is present and it used
to work, but is commented out. (At TRIUMF/T2K, we intend to use MySQL exclusively).
- ODBC ascii interface is used, instead of the potentially more efficient binary
interface.

To enable:
- create a MySQL database,
- create $HOME/.odbc.ini (see attached example)
- set ODB "/History/PerVariableHistory" to "1" - the new code is intended to be
used with per-variable history. Per-equipment (traditional) history would work,
but will result in suboptimal layout of SQL tables.
- set ODB "/Logger/ODBC_DSN" to the DSN defined in .odc.ini.
- set ODB "/Logger/ODBC_Debug" to non-zero to enable debugging output from the
new code.

To use the "ascii output" mode:
Included is code to write "ascii" sql output into a text file, instead of using
an actual SQL database. To enable it, set "ODBC_DSN" to
"/path/to/some/text/file" and all SQL output will be written to this file. No
actual SQL database required. This mode exists mostly for debugging the SQL syntax.

Despite limitations, the committed code is fully functional - we are presently
using it to record history data from slow controls of T2K detector tests
(voltages, currents, temperatures).

Comments and suggestions on naming and mapping from odb structures to SQL tables
is very much welcome.

K.O.

I have a history that is embedded in a custom page using

<div class="mjshistory" data-group="SampleCryo" data-panel="SampleTemp" data-scale="30m" style="'+size+' position: relative;left: 640px;top: -205px;"></div>

this works fine when I load the page but seems to cause a timeout when reloading (F5) the page. It used to work fine last year but since a midas update this year it does not work. 

When I manually stop the script when firefox reports that it is slowing down the browse I get the following in the console:

Script terminated by timeout at:
binarySearch@http://xxx.psi.ch:8081/mhistory.js:1051:11
MhistoryGraph.prototype.findMinMax@http://xxx.psi.ch:8081/mhistory.js:1583:28
MhistoryGraph.prototype.loadInitialData/<@http://xxx.psi.ch:8081/mhistory.js:780:15

any ideas what may be causing this?

thanks.

Another hint to the problem, the custom page is accessible via 
http://xxx.psi.ch:8081/?cmd=custom&page=SampleCryo&
once loaded the address changes to where A and B change values as time passes (I guess B-A=30m).
http://xxx.psi.ch:8081/?cmd=custom&page=SampleCryo&&A=1614173369&B=1614175169

I have to reproduce the problem. Can you please send me the full link by direct email. As you know, I'm also at PSI.

Stefan