> The latest version of MIDAS puts libraries and executables in $MIDASSYS/lib and bin (the "linux" part of pathname is removed).
> 
> Some packages (rootana) have been already changed to use this new scheme and they will not build against older versions of midas. 
> I recommend that you create following symlinks to make old versions of midas compatible with the new scheme:
> 
> cd $MIDASSYS # (~/packages/midas)
> ln -s linux/bin .
> ln -s linux/lib .

If i'm not mistaken the proper commands are

cd $MIDASSYS
ln -s ../bin linux/bin
ln -s ../lib linux/lib

Alternatively, you can change your PATH to point to $MIDASSYS/bin instead of $MIDASSYS/linux/bin and link against $MIDASSYS/lib instead of 
$MIDASSYS/linux/lib

Stefan

I disagree. The modbset() function is used in many custom pages at PSI because people are tired of typing mjsonrpc_db_paste([path],[value]) vs. modbset(path, value). We need to keep 
modbset() which is well documented at 

https://midas.triumf.ca/MidasWiki/index.php/Custom_Page#modbset

Since modbset() does call the underlying mjsonrpc_db_paste(), it is as good or bad as that function. Plus it adds standard error handling to avoid the need of catching errors for each and 
every mjsonrpc_db_paste() call. If it is believed that modbset() has a problem, then this should be fixed in the source code of modbset(). Removing that function is not an option.

Stefan

> - it looks like cmake does not like building variant executables and object files, i.e. "with ROOT" and "without ROOT".
> 
> I need to set "-DHAVE_ROOT" for building "with ROOT" and unset it via remove_definitions() for building "without ROOT",
> but remove_definitions() and add_definitions() do not work on a per-target basis, instead they operate
> per-directory and per-project.

You should not use per-directory and per-project definitions, but per-target definitions, such as

target_compile_options(mhttpd PRIVATE -DMG_ENABLE_SSL)

> In midas, we build mlogger without ROOT (to avoid tangling it with the ROOT RPATH and ROOT shared libraries),
> but if ROOT is present, we build rmlogger "with ROOT support". Same for the analyzer (mana.o and rmana.o).
> 
> For now we have this:
> - mana.o is built with ROOT if ROOT is detected
> - rmana.o is not built
> - rmlogger is not built (not clear why)

I added rmlogger to the install instructions. I believe it was always built, but just not installed into the /bin directory.

Stefan

A ladder of promise event handlers is certainly one possibility to enforce the order of ODB writes, but I wonder if we could so something simpler:

- modbset creates an object remembering the status of the RPC request. Initially, this object receives the status "open request"
- when the rpc call got executed successfully, the callback sets the state of the above object to "request succeeded" or "request failed" (in case of error)
- if a new modbset comes BEFORE the previous one has completed, the function queues the new request in a data field of the above object
- if a rpc call finishes, and a queued new rpc request is present, it gets executed

This would be relatively easy to be implemented and keep the order of the rpc calls. Does that make sense?

Best,
Stefan

Just to make this point clear: The "write-to-odb-read-via-hotlink" was never meant to guarantee the receiving side to see each change. If changes happen too often, updates might get lost. If one relies on the 
sequence of updates, one should use direct RPC calls to the frontend or use a midas buffer and encode updates in events.

Stefan

Why don't your try the (yet undocumented) new installation procedure:

$ git clone https://bitbucket.com/tmidas/midas --recursive
$ cd midas
$ mkdir build
$ cd build
$ cmake ..
$ make
$ make install

In case your RPi does not have cmake pre-installed, you need

$ sudo apt-get install cmake.

Works for my RPi.

Best,
Stefan

Update: "make" instead of "make linux32" should also work. I believe the "linux32" target came 
from some special case at TRIUMF for some FPGA embedded linux, which is not applicable for 
the Raspberry Pi.

Note that the build process has to be initiated on the Raspberry Pi, NOT a host PC.

Stefan

Please note that 

"make cmake" / "make cmake3"

is an abbreviation for the "normal" cmake command chain. Users familiar with cmake can also do the standard command chain:

mkdir build
cd build
cmake ..
make
make install


- Stefan

> We are happy to the midas release "midas-2019-06" with the build system implemented in cmake and the midas, mxml and mscb 
> projects switched to C++.
> 
> Changes since midas-2019-03:
> 
> minor bug fixes
> switch of midas build to c++ with c++ linkage (no "extern C")
> switch of midas build to cmake
> removal of $(OS_DIR) from the midas library and bin paths (use $MIDASSYS/lib instead of $MIDASSYS/linux/lib)
> mxml and mscb are implemented as git submodules
> 
> Please review the following guide to update midas from previous release midas-2019-03 or older.
> 
> Update the code:
> 
> git checkout develop
> git pull
> git checkout feature/midas-2019-06
> git pull
> git submodule update --init # this will checkout correct versions of mxml and mscb
> make clean
> make cclean
> rm -rf linux/bin
> rm -rf linux/lib
> rmdir linux
> make cmake3 # or "make cmake" on ubuntu and macos
> ls -l bin/odbedit bin/mlogger
> 
> Update experiment environment:
> 
> - change PATH from $MIDASSYS/linux/bin to $MIDASSYS/bin
> 
> Cleanup unneeded stuff:
> 
> - remove $HOME/packages/mxml (new location $MIDASSYS/mxml)
> - remove $HOME/packages/mscb (new location $MIDASSYS/mscb)
> 
> Update experiment frontend build:
> 
> - change Makefile to remove $(OS_DIR) from library search path ($MIDASSYS/linux/lib becomes $MIDASSYS/lib)
> - change Makefile to set mxml include path from $MIDASSYS/../mxml to $MIDASSYS/mxml (to avoid including the wrong 
> version of mxml/strlcpy.h)
> - update frontend code to use mfe.h and build as C++, see https://midas.triumf.ca/elog/Midas/1526
> 
> K.O.

Note that the example experiment compiles a simple example frontend and a root-based analyzer. If you don't have 
ROOT installed, you of course cannot compile the analyzer. If you don't need the analyzer, remove it from the 
Makefile/CMakeLists.txt

It's not clear to me why the frontend did not compile on our server machine. You did not post the command how you 
initiated the build. Note that there are now two parallel build schemes: the traditional Makefile and the new 
CMakeFiles.txt. We try to maintain both of them, so you have to specify which one you use when you get an error.

I realize now that the CMakeLists.txt in the experiment example directory builds nicely under midas, but when you move 
it to another directory and extract it from the normal build scheme it breaks. I rewrote the CMakeLists.txt now that it 
looks for MIDASSYS and also build at different locations. Do

cd $HOME/online
cp $MIDASSYS/examples/experiment/* .
mkdir build
cd build
cmake ..
make

and it should work. Of course first pull the current develop version.

Stefan

Sure some existing scripting languages can be used, but they fall short of a few important items in larger experiments:

- they are typically run from a local terminal in the counting house. A remote observer of the experiment has no idea which script is running and at which state it is.

- if DAQ crashes during a script or is aborted, it has to be restarted from the beginning. If you run a sequence of let's say 100 runs taking 8 hours, and on run #98 something goes wrong, you are screwed if you have to start at run #1 again.

This are the main reasons why I developed the midas sequencer. Having everything web-based, everybody can watch remotely how far the sequence progressed. If the whole DAQ crashes, the sequence resumes from the crash point, not from the beginning. This is by saving the current state into the ODB. So even if the sequencer itself is stopped and restarted, that still works.

I agree that the MSL is missing a few calculations, and I was just waiting to get a few specific requests. I will either add new functions such as basic calculations like adding and subtracting variables, or I will create a way to call an external shell like bash to do calculations. I put this high on my todo list.

Stefan

> Yes, this has been the way to do it for years...

Calling odbedit -c ... ist certainly not the most effective way, but it works. I just tried on my Mac and found that I can call odbedit about 150 times per second. So not so bad if you have a limited operations to perform.

Stefan

The bug has been fixed. It was actually in the mxml library. So you have to go to the midas/mxml 
subdirectory and update that one via "git pull origin master".

Stefan

> Hi,
> the subject line describes the project already
> Suppose you have a file foo.msl. Somewhere in the file, you have the line 
> INCLUDE bar.msl
> 
> Once you click save in the sequencer page, mhttpd crashes:
> $ mhttpd
> free(): double free detected in tcache 2
> [1]    27590 abort (core dumped)  mhttpd
> 
> 
> GDB helps shed some light on the problem:
> 
> #0  0x00007ffff76b057f in raise () from /lib64/libc.so.6
> #1  0x00007ffff769a895 in abort () from /lib64/libc.so.6
> #2  0x00007ffff76f39d7 in __libc_message () from /lib64/libc.so.6
> #3  0x00007ffff76fa2ec in malloc_printerr () from /lib64/libc.so.6
> #4  0x00007ffff76fbdf5 in _int_free () from /lib64/libc.so.6
> #5  0x00000000004b8b41 in mxml_parse_entity (buf=buf@entry=0x7fffffffc2c8,
> file_name=file_name@entry=0x7fffffffc710
> "/home/luk/packages/mutrig_daq/online/foo.xml",
>             error=error@entry=0x7fffffffcd24 "XML read error in file
> \"/home/luk/packages/mutrig_daq/online/foo.xml\", line 2: bar.msl.xml is
> missing", error_size=error_size@entry=256,
>                 error_line=error_line@entry=0x7fffffffce24) at ../mxml/mxml.c:1996
> #6  0x00000000004b966d in mxml_parse_file
> (file_name=file_name@entry=0x7fffffffc710
> "/home/luk/packages/mutrig_daq/online/foo.xml",
>             error=error@entry=0x7fffffffcd24 "XML read error in file
> \"/home/luk/packages/mutrig_daq/online/foo.xml\", line 2: bar.msl.xml is
> missing", error_size=error_size@entry=256,
>                 error_line=error_line@entry=0x7fffffffce24) at ../mxml/mxml.c:2041
> #7  0x000000000041d9c2 in init_sequencer () at src/mhttpd.cxx:14321
> #8  0x000000000040c2b6 in main (argc=<optimized out>, argv=<optimized out>) at
> src/mhttpd.cxx:18028
> 
> Cheers
> Lukas
> 
> P. S. This problem reminds me of the old joke: A man goes to his doctor and says
> "Doc, it hurts when I do this" to which the doctor replies "Then don't do that".
> However, I think, mhttpd should not crash even if you're not supposed to include
> non-existent scripts in msequencer.

> You can confirm that you are linking against the correct ROOT by running cmake with VERBOSE=1
> and examine the linker command line to see what library link path is specified for ROOT.

Actually you don't call cmake with the verbose flag but specify it during the make phase

$ make VERBOSE=1

to see the command lines.

Stefan

> - this will activate the mhttpd password protection, so one would have to define a username and password 
> in the .htdigest file (this is done by the mongoose web server library).

Actually I'm thinking since a while to have user-level access to mhttpd, similarly to elog. Each user has to log in with a unique username/password. After some time of inactivity, you're logged out. This would have 
the advantage that one knows who is active where, like when using the chat functionality in mhttpd. Or who started/stopped a run etc. This might not be necessary for simple local installations, but if you have 20 
people controlling an experiment from three different continents simultaneously, this could be beneficial. Using the elog authentication libraries, one could even forward the login process to LDAP or KERBEROS, 
so you could log in with out institutional account, and don't have to remember an additional password.

Just some food for thought.

Stefan

Did you include mfe.h as written in elog:1526 ?

Stefan

Have you left any "extern C" in your frontend program or in any of the used header file. Seems 
like the linker cannot find the poll_event in your frontend code. If it's there, but it's compiled 
with C calling (instead of C++), the name mangling causes it to be invisible to the linker. That 
usually happens if there is somewhere

extern C {
  INT poll_event();
  ... 
}

while it is NOT defined as "extern C" in mfe.h which is used by mfe.cxx.

Stefan

> > Did you include mfe.h as written in elog:1526 ?
> > 
> > Stefan
> 
> 
> Yes I did
> 
> this is my include 
> 
> #include <stdio.h>
> #include <string.h>
> #include <assert.h>
> #include <math.h>
> #include <pthread.h>
> 
> 
> #include "midas.h"
> #include "mscb.h"
> #include "multi.h"
> #include "mscbdev.h"
> #include "mfe.h"
> 
> (I attach my dummy fe)
> 
> What confuses me is that I can compile examples/experiment/ if I copy that to a 
> fresh dir.
> 
> I also copied the CMakeLists from the example:
> 
> #
> # cmake for the muX software
> #
> cmake_minimum_required(VERSION 3.3)
> 
> project(muX)
> 
> #
> # find midas installation, from CMakeLists in examples/experiment
> #
> set(MIDAS_DIR $ENV{MIDASSYS})
> message("MIDAS dir: " ${MIDAS_DIR})
> if (NOT EXISTS $ENV{MIDASSYS})
>   message(FATAL_ERROR "Environment variable $MIDASSYS not defined, aborting.")
> endif()
> 
> set(INC_PATH ${MIDAS_DIR}/include ${MIDAS_DIR}/mxml ${MIDAS_DIR}/mscb/include 
> ${MIDAS_DIR}/drivers/class ${MIDAS_DIR}/drivers/device)
> link_directories($ENV{MIDASSYS}/lib)
> 
> # enable certain compile warnings
> add_compile_options(-Wall -Wformat=2 -Wno-format-nonliteral -Wno-strict-
> aliasing -Wuninitialized -Wno-unused-function)
> 
> set(LIBS -lpthread -lutil -lrt) 
> 
> 
> add_executable(sc_fe_mini sc_fe_mini.cpp)
> target_include_directories(sc_fe_mini PRIVATE ${INC_PATH})
> target_link_libraries(sc_fe_mini mfe midas ${LIBS})

> as a side comment, not all drivers are c++ compatible yet (e.g. mscbvr), so changes needed are small 

Right. We recently switched the whole midas to c++, but we could not cover all drivers. Most of them just need some type 
casting to compile under c++. I got already patches from several people which I'm happy to merge in. If you got mscbhvr 
or any other driver compile under c++, please send me the diff.

Stefan

> Would it be possible to add something like the following?
>  343 if (!isdigit(value1_var[i]) && value1_var[i] != '.')
>  344          break;

Actually isdigit() is completely wrong here, because it also fails the minus sign and the 'E' exponent, like in -1.2E-3 

So I changed it to strchr("0123456789.+-Ee", var[i]) which should cover this case. If you put 1.2.3, it takes it as 1.2. 

Stefan

> 
> Ok, so the detail that I missed was that the dummy functions
> 
> INT poll_event(INT source[], INT count, BOOL test)
> {
>    return 1;
> };
> 
> as shown in some of the older examples, work when you set
> 
> INT poll_event(INT source, INT count, BOOL test)
> {
>    return 1;
> };

If you would have read elog:1526 more carefully (point a4) you would have saved yourself a lot of time.

Stefan

Today I fixed a long-annoying problem. We have in each front-end an equipment structure 
which defined the event id, event type, readout frequency etc. This is mapped to the ODB 
subtree

/Equipment/<name>/Common

In the past, the ODB setting took precedence over the frontend structure. We defined this 
like 25 years ago and I forgot what the exact reason was. It causes however many people 
(including myself) to fall into this trap: You change something in the front-end EQUIPMENT 
structure, you restart the front-end, but the new setting does not take effect since the 
(old) ODB value took precedence. After some debugging you find out that you have to both 
change the EQUIPMENT structure (which defines the default value for a fresh ODB) and the 
ODB value itself.

So I changed it in the current develop tree that the front-end structure takes precedence. 
You still have a hot-link, so if you want to change anything while the front-end is running 
(like the readout period), you can do that in the ODB and it takes effect immediately. But 
when you start the front-end the next time, the value from the EQUIPMENT structure is 
taken again. So please be aware of this new feature.

Happy BC day,
Stefan