My TPC events are fairly large: 18 FEC cards * 128 channels per card * 2 Kbytes
per channel = about 4 Mbytes. In my
frontend, when I request this event size, MIDAS complaints (in mfe.c) that it is
bigger than MAX_EVENT_SIZE, which
is set to 0.5 Mbytes in midas.h. What is the best way to deal with this? Should
we increase MAX_EVENT_SIZE to
something bigger? Remove the MAX_EVENT_SIZE limitation altogether?
  
For now, I increased the value MAX_EVENT_SIZE & co to (10*1024*1024) and it
seems to work (I also had to bump the
sanity check in bm_open_buffer() from 10E6 to 100E6). With 1/4 of the FEC cards,
the event size is 1 Mbyte at ~6
ev/sec the machine is almost idle, with the biggest CPU user being the event
builder at 10% CPU utilization.

K.O.

> My TPC events are fairly large: 18 FEC cards * 128 channels per card * 2 Kbytes
> per channel = about 4 Mbytes. In my
> frontend, when I request this event size, MIDAS complaints (in mfe.c) that it is
> bigger than MAX_EVENT_SIZE, which
> is set to 0.5 Mbytes in midas.h. What is the best way to deal with this? Should
> we increase MAX_EVENT_SIZE to
> something bigger? Remove the MAX_EVENT_SIZE limitation altogether?

If you teach me how to remove the MAX_EVENT_SIZE, that would be perfect!

Unfortunately the limit comes from the shared memory on the back end (the so-called
"SYSTEM" shared memory). Due to the structure of the buffer manager, the shared
memory has to hold at least two events simultaneously. And once the shared memeory
is created, it's size cannot be changed without restarting all the clients. That's
the origin of the MAX_EVENT_SIZE. In former days, the total allowed shared memory on
a typical linux machine was 2MB. That's why I set MAX_EVENT_SIZE to 0.5 MB, so midas
takes 2*0.5MB=1MB plus 0.2MB for the ODB, leaving 0.8MB for other applications.
Nowadays, the shared memory might be bigger (actually it's a parameter during kernel
compilation), so one could consider increasing the default MAX_EVENT_SIZE. If you
make a survey of the shared memory sizes in some of the current distributions, we
can choose a safe value.

> For now, I increased the value MAX_EVENT_SIZE & co to (10*1024*1024) and it
> seems to work (I also had to bump the
> sanity check in bm_open_buffer() from 10E6 to 100E6). With 1/4 of the FEC cards,
> the event size is 1 Mbyte at ~6
> ev/sec the machine is almost idle, with the biggest CPU user being the event
> builder at 10% CPU utilization.

I made sure that there is no other limitation as the one given by MAX_EVENT_SIZE, so
it should work fine. Thanks for telling me the wrong sanity check, that should be
changed in the repository.

I try to do make in zlib folder and got  this
cc -O -o example example.o -L. -lz
/usr/bin/ld: errno: TLS definition in /lib/libc.so.6 section .tbss mismatches
non-TLS reference in ./libz.a(gzio.o)
/lib/libc.so.6: could not read symbols: Bad value
collect2: ld returned 1 exit status
make: *** [example] Error 1

Do I miss any package to be installed?
Thanks in advance,
Xelap

The midas forum software (elogd) was updated to latest version and moved from our old server 
(ladd00.triumf.ca) to our new server (daq00.triumf.ca).

The following URLs should work:

https://daq00.triumf.ca/elog-midas/Midas/ (new URL)
https://midas.triumf.ca/elog/Midas/ (old URL, redirects to daq00)
https://midas.triumf.ca/forum (link from midas wiki)

The configuration on the old server ladd00.triumf.ca is quite tangled between
several virtual hosts and several DNS CNAMEs. I think I got all the redirects
correct and all old URLs and links in old emails & etc still work.

If you see something wrong, please reply to this message here or email me directly.

K.O.

Per changes at TRIUMF, the MIDAS forum mail relay was changed from trmail.triumf.ca to 
smtp.triumf.ca. K.O.

the midas forum elog is updated to latest version from Stefan - ELOG V3.1.1-b4d2a37 built from git sources. K.O.

I updated the midas forum elog to the latest version from git: 083448f7

Also investigated elogd failure to start on reboot,
it turned out to be a crasher bug, see
https://elog.psi.ch/elogs/Forum/69919

K.O.

> I updated the midas forum elog to the latest version from git: 083448f7
> 
> Also investigated elogd failure to start on reboot,
> it turned out to be a crasher bug, see
> https://elog.psi.ch/elogs/Forum/69919
>

Also investigating elogd crash last Friday, looks like an elog bug, see

https://elog.psi.ch/elogs/Forum/69933

K.O.

Also a crash soon after startup:

https://elog.psi.ch/elogs/Forum/69934

K.O.

crashed again, restarted. gcc address sanitizer disables core dumps by default, to enable, do this:
export ASAN_OPTIONS=abort_on_error=1:disable_coredump=0:unmap_shadow_on_exit=1
K.O.

> crashed again, restarted. gcc address sanitizer disables core dumps by default, to enable, do this:
> export ASAN_OPTIONS=abort_on_error=1:disable_coredump=0:unmap_shadow_on_exit=1

midas forum elogd would not stay up, bot traffic crashes it within hours.

as temporary mitigation:
- elogd is now running from an autorestart script (15 second delay between restart attempts)
- address sanitizer and core dumps enabled to collect and fix crashes as they are identified (I fixed 4-5 crashers, so far)

K.O.

we are migrating the dragon experiment from an old mac to a new mac studio and we ran into a problem 
where one equipment format was set to "fixed" instead of "midas". lots of confusion, mdump crash, 
analyzer crash, etc. (mdump fixes for this are already committed).

it made us think whether equipment format is still needed. in the old days we had choice of MIDAS and 
YBOS formats, but YBOS was removed years ago, and I was surprised that format FIXED was permitted at 
all.

I did a midas source code review, this is what I found:

- remnants of YBOS support in a few places, commit to remove them pending.
- FORMAT_ROOT is used in mlogger for automatic conversion of MIDAS banks to ROOT trees
- FORMAT_FIXED is used in a few slow control drivers in drivers/class, instead of creating MIDAS 
banks, they copy raw data directly into an event (there is no bank header and no way to identify such 
events automatically)
- lots of code to support different formats in mdump (mostly dead code)
- the rest of the code does not care or use this format stuff

Current proposal is to remove support for all formats except FORMAT_MIDAS (and FORMAT_ROOT in 
mlogger).

- defines of FORMAT_XXX will be removed from midas.h
- "Format" will be removed from ODB Equipment/Common
- "Format" will be removed from ODB Logger/Channel
- to maintain binary compatibility, we can keep the "Format" ODB entries, but they will be ignored.

List of slow control drivers that support FORMAT_FIXED:

daq00:midas$ grep FORMAT_FIXED drivers/class/*
drivers/class/cd_fdg.cxx:   if (fgd_info->format == FORMAT_FIXED) {
drivers/class/cd_ivc32.cxx:   if (hv_info->format == XFORMAT_FIXED) {
drivers/class/cd_rf.cxx:	if (rf_info->format == XFORMAT_FIXED) 
drivers/class/generic.cxx:   if (gen_info->format == XFORMAT_FIXED) {
drivers/class/hv.cxx:   if (hv_info->format == XFORMAT_FIXED) {
drivers/class/multi.cxx:   if (m_info->format == XFORMAT_FIXED) {
drivers/class/slowdev.cxx:   if (gen_info->format == XFORMAT_FIXED) {
daq00:midas$ 

K.O.

Hi Konstantin,

In the PSI muSR laboratory, we are running about 140 slow control devices across six instruments using Format FIXED.

Could you please wait a bit with removing support for it so that we can assess if/how this will affect us?

Many thanks,
Jonas

> we are migrating the dragon experiment from an old mac to a new mac studio and we ran into a problem 
> where one equipment format was set to "fixed" instead of "midas". lots of confusion, mdump crash, 
> analyzer crash, etc. (mdump fixes for this are already committed).
> 
> it made us think whether equipment format is still needed. in the old days we had choice of MIDAS and 
> YBOS formats, but YBOS was removed years ago, and I was surprised that format FIXED was permitted at 
> all.
> 
> I did a midas source code review, this is what I found:
> 
> - remnants of YBOS support in a few places, commit to remove them pending.
> - FORMAT_ROOT is used in mlogger for automatic conversion of MIDAS banks to ROOT trees
> - FORMAT_FIXED is used in a few slow control drivers in drivers/class, instead of creating MIDAS 
> banks, they copy raw data directly into an event (there is no bank header and no way to identify such 
> events automatically)
> - lots of code to support different formats in mdump (mostly dead code)
> - the rest of the code does not care or use this format stuff
> 
> Current proposal is to remove support for all formats except FORMAT_MIDAS (and FORMAT_ROOT in 
> mlogger).
> 
> - defines of FORMAT_XXX will be removed from midas.h
> - "Format" will be removed from ODB Equipment/Common
> - "Format" will be removed from ODB Logger/Channel
> - to maintain binary compatibility, we can keep the "Format" ODB entries, but they will be ignored.
> 
> List of slow control drivers that support FORMAT_FIXED:
> 
> daq00:midas$ grep FORMAT_FIXED drivers/class/*
> drivers/class/cd_fdg.cxx:   if (fgd_info->format == FORMAT_FIXED) {
> drivers/class/cd_ivc32.cxx:   if (hv_info->format == XFORMAT_FIXED) {
> drivers/class/cd_rf.cxx:	if (rf_info->format == XFORMAT_FIXED) 
> drivers/class/generic.cxx:   if (gen_info->format == XFORMAT_FIXED) {
> drivers/class/hv.cxx:   if (hv_info->format == XFORMAT_FIXED) {
> drivers/class/multi.cxx:   if (m_info->format == XFORMAT_FIXED) {
> drivers/class/slowdev.cxx:   if (gen_info->format == XFORMAT_FIXED) {
> daq00:midas$ 
> 
> K.O.

> In the PSI muSR laboratory, we are running about 140 slow control devices across six instruments using Format FIXED.
> Could you please wait a bit with removing support for it so that we can assess if/how this will affect us?

Roger. I believe as long as these data do not go into the MIDAS event data stream, you should not see any difference from 
changing "FIXED" to "MIDAS", I hope you can test it and report how it works for you. If you see that things change in ODB 
or in history, perhaps we can implement some kind of workaround to make the transition transparent.

K.O.

Around Wednesday Noon, there was a power outage at triumf (loss of ups power in the triumf 
computing center) and after rebooting ladd00, https/ssl access stopped working with a complaint 
about mismatching server name and ssl certificate name. This configuration used to work, so one of the 
system updated must have broke it. This problem is now fixed and access to midas elog is restored. 
K.O.

I reworked the midas cmake files:
- install via CMAKE_INSTALL_PREFIX should work correctly now:
- installed are bin, lib and include - everything needed to build against the midas library
- if built without CMAKE_INSTALL_PREFIX, a special mode "MIDAS_NO_INSTALL_INCLUDE_FILES" is activated, and the include path 
contains all the subdirectories need for compilation
- -I$MIDASSYS/include and -L$MIDASSYS/lib -lmidas work in both cases
- to "use" midas, I recommend: include($ENV{MIDASSYS}/lib/midas-targets.cmake)
- config files generated for find_package(midas) now have correct information (a manually constructed subset of information 
automatically exported by cmake's install(export))
- people who want to use "find_package(midas)" will have to contribute documentation on how to use it (explain the magic used to 
find the "right midas" in /usr/local/midas or in /midas or in ~/packages/midas or in ~/pacjages/new-midas) and contribute an 
example superproject that shows how to use it and that can be run from the bitpucket automatic build. (features that are not part 
of the automatic build we cannot insure against breakage).

On my side, here is an example of using include($ENV{MIDASSYS}/lib/midas-targets.cmake). I posted this before, it is used in 
midas/examples/experiment and I will ask ben to include it into the midas wiki documentation.

Below is the complete cmake file for building the alpha-g event bnuilder and main control frontend. When presented like this, I 
have to agree that cmake does provide positive value to the user. (the jury is still out whether it balances out against the 
negative value in the extra work to "just support find_package(midas) already!").

#
# CMakeLists.txt for alpha-g frontends
#

cmake_minimum_required(VERSION 3.12)
project(agdaq_frontends)

include($ENV{MIDASSYS}/lib/midas-targets.cmake)

add_compile_options("-O2")
add_compile_options("-g")
#add_compile_options("-std=c++11")
add_compile_options(-Wall -Wformat=2 -Wno-format-nonliteral -Wno-strict-aliasing -Wuninitialized -Wno-unused-function)
add_compile_options("-DTMFE_REV0")
add_compile_options("-DOS_LINUX")

add_executable(feevb feevb.cxx TsSync.cxx)
target_link_libraries(feevb midas)

add_executable(fectrl fectrl.cxx GrifComm.cxx EsperComm.cxx JsonTo.cxx KOtcp.cxx $ENV{MIDASSYS}/src/tmfe_rev0.cxx)
target_link_libraries(fectrl midas)

#end

MIDAS git tag midas-2025-01-a introduced an incompatible change to "include midas-targets.cmake". Instead of "midas" one now has to 
say "midas::midas", as updated below. K.O.

> 
> #
> # CMakeLists.txt for alpha-g frontends
> #
> 
> cmake_minimum_required(VERSION 3.12)
> project(agdaq_frontends)
> 
> include($ENV{MIDASSYS}/lib/midas-targets.cmake)
> 
> add_compile_options("-O2")
> add_compile_options("-g")
> #add_compile_options("-std=c++11")
> add_compile_options(-Wall -Wformat=2 -Wno-format-nonliteral -Wno-strict-aliasing -Wuninitialized -Wno-unused-function)
> add_compile_options("-DTMFE_REV0")
> add_compile_options("-DOS_LINUX")
> 
> add_executable(feevb feevb.cxx TsSync.cxx)
> target_link_libraries(feevb midas::midas)
> 
> add_executable(fectrl fectrl.cxx GrifComm.cxx EsperComm.cxx JsonTo.cxx KOtcp.cxx $ENV{MIDASSYS}/src/tmfe_rev0.cxx)
> target_link_libraries(fectrl midas::midas)
> 
> #end

I want to report a bug in the ROOT build process that might be relevant to the midas implementation. I had an annoying failure to build root 6.18 (current pro version) with a misleading error message about a fault in the root code. It turned out this was a cmake problem, and the error was from my cmake version being older than 3.14, which is quite recent. Took a bit of searching to find this.

I recommend when the cmake version is distributed that the instructions include the required cmake version. Developers are generally working well ahead of what is available in the older OS's.

Each CMakeLists.txt should specify which version of CMake it requires. The MIDAS CMakeLists.txt requires CMake 3.1 or later. 
We deliberately stayed away from fancy cutting edge CMake features in order to make midas easier to compile. On top of that,
midas is a much simpler package compared to root, so things are not so complicated.

Stefan

> Each CMakeLists.txt should specify which version of CMake it requires. The MIDAS CMakeLists.txt requires CMake 3.1 or later. 
> We deliberately stayed away from fancy cutting edge CMake features in order to make midas easier to compile. On top of that,
> midas is a much simpler package compared to root, so things are not so complicated.

The oldest cmake I actually used is 3.6.1 (on SL6), so I do not know if cmake versions between 3.1 and 3.6 actually work for us. Perhaps we should set 
the CMakefile requirement to 3.6.1 to match the oldest version we know works. If somebody has an older cmake, they have a choice of updating it or 
trying it as as and reporting success/failure to the midas forum here.

K.O.