23 Jan 2020, Stefan Ritt, Bug Report, get an open ssl error while trying to compile Midas
|
I tried on my Mac (macOS 10.14.6, Xcode 11.3.1, current develop branch, openssl 1.1.1d) and it woks fine. Below is the transcript. I
see that your cmake output is much shorter (no C compiler listed etc.). Did you remove some lines? For such comparisons, it's
always good to start with an empty build directory.
OpenSSL libraries are the same (1.1.1d). Just for comparison, I run the build process with "make VERBOSE=1" and extract the line
which fails for you when linking mhttp, so maybe you can compare.
Linking mhttpd
---------------
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -O2 -g -DNDEBUG -isysroot
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -mmacosx-version-
min=10.14 -Wl,-search_paths_first -Wl,-headerpad_max_install_names CMakeFiles/mhttpd.dir/mhttpd.cxx.o
CMakeFiles/mhttpd.dir/mongoose6.cxx.o CMakeFiles/mhttpd.dir/mgd.cxx.o CMakeFiles/mhttpd.dir/__/mscb/src/mscb.cxx.o -o
mhttpd -L/opt/local/lib/mysql57/mysql -lmysqlclient -lz ../libmidas.a /opt/local/lib/libssl.dylib /opt/local/lib/libcrypto.dylib -lz -
L/opt/local/lib/mysql57/mysql -lmysqlclient -lz
Here is the full transcript
-------------------------
/midas/build$ cmake ..
-- MIDAS: cmake version: 3.16.3
-- The C compiler identification is AppleClang 11.0.0.11000033
-- The CXX compiler identification is AppleClang 11.0.0.11000033
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc --
works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- MIDAS: CMAKE_INSTALL_PREFIX: /midas
-- MIDAS: Found ROOT version 6.16/00
-- Found ZLIB: /opt/local/lib/libz.dylib (found version "1.2.11")
-- MIDAS: Found ZLIB version 1.2.11
-- Found OpenSSL: /opt/local/lib/libcrypto.dylib (found version "1.1.1d")
-- MIDAS: Found OpenSSL version 1.1.1d
-- MIDAS: Found MySQL version 5.7.26
-- MIDAS: ODBC not found
-- MIDAS: SQLITE not found
-- MIDAS: nvidia-smi not found
-- Setting default build type to "RelWithDebInfo"
-- Found Git: /usr/bin/git (found version "2.21.1 (Apple Git-122.3)")
-- MIDAS example/experiment: MIDAS in-tree-build
-- MIDAS: Found ZLIB version 1.2.11
-- MIDAS example/experiment: Found ROOT version 6.16/00
-- Configuring done
-- Generating done
-- Build files have been written to: /midas/build
/midas/build$ make
Scanning dependencies of target rmana
Scanning dependencies of target git_revision_h
Scanning dependencies of target mfe
Scanning dependencies of target manao
Scanning dependencies of target mfeo
Scanning dependencies of target objlib-c-compat
Scanning dependencies of target mana
Scanning dependencies of target rmanao
[ 0%] Building CXX object CMakeFiles/objlib-c-compat.dir/src/midas_c_compat.cxx.o
[ 2%] Building CXX object CMakeFiles/mfe.dir/src/mfe.cxx.o
[ 2%] Building CXX object CMakeFiles/mfeo.dir/src/mfe.cxx.o
[ 3%] Building CXX object CMakeFiles/mana.dir/src/mana.cxx.o
[ 4%] Building CXX object CMakeFiles/manao.dir/src/mana.cxx.o
[ 5%] Building CXX object CMakeFiles/rmanao.dir/src/mana.cxx.o
[ 6%] Building CXX object CMakeFiles/rmana.dir/src/mana.cxx.o
[ 6%] Built target git_revision_h
Scanning dependencies of target objlib
[ 7%] Building CXX object CMakeFiles/objlib.dir/src/midas.cxx.o
[ 7%] Built target objlib-c-compat
[ 8%] Building CXX object CMakeFiles/objlib.dir/src/midas_cxx.cxx.o
[ 8%] Linking CXX static library libmana.a
[ 8%] Built target manao
[ 8%] Building CXX object CMakeFiles/objlib.dir/src/odb.cxx.o
[ 8%] Built target mana
[ 9%] Building CXX object CMakeFiles/objlib.dir/src/device_driver.cxx.o
[ 9%] Built target mfeo
[ 10%] Linking CXX static library libmfe.a
[ 11%] Building CXX object CMakeFiles/objlib.dir/src/system.cxx.o
[ 11%] Built target mfe
[ 11%] Building CXX object CMakeFiles/objlib.dir/src/alarm.cxx.o
[ 12%] Building CXX object CMakeFiles/objlib.dir/src/elog.cxx.o
[ 13%] Building CXX object CMakeFiles/objlib.dir/src/mrpc.cxx.o
[ 13%] Building CXX object CMakeFiles/objlib.dir/src/mjson.cxx.o
[ 14%] Building CXX object CMakeFiles/objlib.dir/src/tmfe.cxx.o
[ 15%] Building CXX object CMakeFiles/objlib.dir/src/mvodb.cxx.o
[ 15%] Built target rmanao
[ 15%] Linking CXX static library librmana.a
[ 16%] Building CXX object CMakeFiles/objlib.dir/src/nullodb.cxx.o
[ 16%] Building CXX object CMakeFiles/objlib.dir/src/midasodb.cxx.o
[ 16%] Built target rmana
[ 17%] Building CXX object CMakeFiles/objlib.dir/src/mxmlodb.cxx.o
[ 18%] Building CXX object CMakeFiles/objlib.dir/src/mjsonodb.cxx.o
[ 18%] Building CXX object CMakeFiles/objlib.dir/src/json_paste.cxx.o
[ 19%] Building CXX object CMakeFiles/objlib.dir/src/mjsonrpc.cxx.o
[ 20%] Building CXX object CMakeFiles/objlib.dir/src/mjsonrpc_user.cxx.o
[ 21%] Building CXX object CMakeFiles/objlib.dir/src/history.cxx.o
[ 21%] Building CXX object CMakeFiles/objlib.dir/src/history_common.cxx.o
[ 22%] Building CXX object CMakeFiles/objlib.dir/src/history_odbc.cxx.o
[ 23%] Building CXX object CMakeFiles/objlib.dir/src/history_schema.cxx.o
[ 23%] Building CXX object CMakeFiles/objlib.dir/src/lz4.cxx.o
[ 24%] Building CXX object CMakeFiles/objlib.dir/src/lz4frame.cxx.o
[ 25%] Building CXX object CMakeFiles/objlib.dir/src/lz4hc.cxx.o
[ 26%] Building CXX object CMakeFiles/objlib.dir/src/xxhash.cxx.o
[ 26%] Building CXX object CMakeFiles/objlib.dir/src/crc32c.cxx.o
[ 27%] Building CXX object CMakeFiles/objlib.dir/src/sha256.cxx.o
[ 28%] Building CXX object CMakeFiles/objlib.dir/src/sha512.cxx.o
[ 28%] Building CXX object CMakeFiles/objlib.dir/src/ftplib.cxx.o
[ 29%] Building CXX object CMakeFiles/objlib.dir/src/mdsupport.cxx.o
[ 30%] Building CXX object CMakeFiles/objlib.dir/mxml/mxml.cxx.o
[ 31%] Building CXX object CMakeFiles/objlib.dir/mxml/strlcpy.cxx.o
[ 31%] Built target objlib
Scanning dependencies of target midas-shared
Scanning dependencies of target midas
[ 33%] Linking CXX shared library libmidas-shared.dylib
[ 33%] Linking CXX static library libmidas.a
[ 33%] Built target midas-shared
[ 33%] Built target midas
Scanning dependencies of target mhttpd
Scanning dependencies of target feudp
Scanning dependencies of target odb_lock_test
Scanning dependencies of target rmlogger
Scanning dependencies of target mchart
Scanning dependencies of target msysmon
Scanning dependencies of target midas-c-compat
Scanning dependencies of target mfe_link_test
[ 33%] Building CXX object progs/CMakeFiles/mfe_link_test.dir/mfe_link_test.cxx.o
[ 33%] Linking CXX shared library libmidas-c-compat.dylib
[ 34%] Building CXX object progs/CMakeFiles/feudp.dir/feudp.cxx.o
[ 35%] Building CXX object progs/CMakeFiles/msysmon.dir/msysmon.cxx.o
[ 36%] Building CXX object progs/CMakeFiles/mchart.dir/mchart.cxx.o
[ 37%] Building CXX object progs/CMakeFiles/odb_lock_test.dir/odb_lock_test.cxx.o
[ 38%] Building CXX object progs/CMakeFiles/rmlogger.dir/mlogger.cxx.o
[ 39%] Building CXX object progs/CMakeFiles/mhttpd.dir/mhttpd.cxx.o
[ 39%] Built target midas-c-compat
[ 40%] Building CXX object progs/CMakeFiles/mhttpd.dir/mongoose6.cxx.o
[ 41%] Linking CXX executable mfe_link_test
[ 42%] Linking CXX executable odb_lock_test
[ 42%] Built target mfe_link_test
Scanning dependencies of target mjson_test
[ 43%] Building CXX object progs/CMakeFiles/mjson_test.dir/mjson_test.cxx.o
[ 43%] Linking CXX executable mchart
[ 43%] Built target odb_lock_test
Scanning dependencies of target odbinit
[ 44%] Building CXX object progs/CMakeFiles/odbinit.dir/odbinit.cxx.o
[ 45%] Linking CXX executable feudp
[ 45%] Built target mchart
[ 46%] Building CXX object progs/CMakeFiles/mhttpd.dir/mgd.cxx.o
[ 46%] Built target feudp
[ 46%] Building CXX object progs/CMakeFiles/mhttpd.dir/__/mscb/src/mscb.cxx.o
[ 47%] Linking CXX executable mjson_test
[ 47%] Built target mjson_test
Scanning dependencies of target fetest_tmfe_thread
[ 47%] Building CXX object progs/CMakeFiles/fetest_tmfe_thread.dir/fetest_tmfe_thread.cxx.o
[ 47%] Linking CXX executable msysmon
[ 47%] Built target msysmon
[ 48%] Linking CXX executable odbinit
Scanning dependencies of target fetest_tmfe
[ 49%] Building CXX object progs/CMakeFiles/fetest_tmfe.dir/fetest_tmfe.cxx.o
[ 49%] Built target odbinit
[ 50%] Linking CXX executable fetest_tmfe_thread
Scanning dependencies of target mh2sql
[ 51%] Building CXX object progs/CMakeFiles/mh2sql.dir/mh2sql.cxx.o
Scanning dependencies of target odbhist
[ 51%] Building CXX object progs/CMakeFiles/odbhist.dir/odbhist.cxx.o
[ 51%] Built target fetest_tmfe_thread
Scanning dependencies of target melog
[ 51%] Building CXX object progs/CMakeFiles/melog.dir/melog.cxx.o
[ 52%] Linking CXX executable odbhist
[ 53%] Built target odbhist
[ 53%] Linking CXX executable melog
[ 54%] Linking CXX executable fetest_tmfe
Scanning dependencies of target mfe_link_test_cxx
[ 55%] Building CXX object progs/CMakeFiles/mfe_link_test_cxx.dir/mfe_link_test_cxx.cxx.o
[ 55%] Built target melog
Scanning dependencies of target crc32c_sum
[ 55%] Built target fetest_tmfe
Scanning dependencies of target odbedit
[ 56%] Building CXX object progs/CMakeFiles/crc32c_sum.dir/crc32c_sum.cxx.o
[ 56%] Building CXX object progs/CMakeFiles/odbedit.dir/odbedit.cxx.o
[ 57%] Linking CXX executable crc32c_sum
[ 58%] Linking CXX executable mh2sql
[ 58%] Built target crc32c_sum
[ 59%] Building CXX object progs/CMakeFiles/odbedit.dir/cmdedit.cxx.o
[ 59%] Linking CXX executable mfe_link_test_cxx
Scanning dependencies of target mdump
[ 59%] Built target mh2sql
Scanning dependencies of target mhdump
[ 60%] Building CXX object progs/CMakeFiles/mdump.dir/mdump.cxx.o
[ 60%] Building CXX object progs/CMakeFiles/mhdump.dir/mhdump.cxx.o
[ 60%] Built target mfe_link_test_cxx
Scanning dependencies of target lazylogger
[ 61%] Building CXX object progs/CMakeFiles/lazylogger.dir/lazylogger.cxx.o
Scanning dependencies of target mtransition
[ 61%] Building CXX object progs/CMakeFiles/mtransition.dir/mtransition.cxx.o
[ 62%] Linking CXX executable mdump
[ 63%] Linking CXX executable rmlogger
[ 63%] Built target mdump
Scanning dependencies of target mserver
[ 64%] Building CXX object progs/CMakeFiles/mserver.dir/mserver.cxx.o
[ 65%] Linking CXX executable mtransition
[ 65%] Built target rmlogger
Scanning dependencies of target mhist
[ 66%] Building CXX object progs/CMakeFiles/mhist.dir/mhist.cxx.o
[ 66%] Built target mtransition
Scanning dependencies of target get_record_test
[ 67%] Building CXX object progs/CMakeFiles/get_record_test.dir/get_record_test.cxx.o
[ 68%] Linking CXX executable mhdump
Scanning dependencies of target msequencer
[ 68%] Building CXX object progs/CMakeFiles/msequencer.dir/msequencer.cxx.o
[ 68%] Built target mhdump
Scanning dependencies of target fetest
[ 69%] Linking CXX executable odbedit
[ 70%] Building CXX object progs/CMakeFiles/fetest.dir/fetest.cxx.o
[ 70%] Built target odbedit
[ 70%] Linking CXX executable mserver
Scanning dependencies of target mstat
[ 70%] Linking CXX executable get_record_test
[ 71%] Building CXX object progs/CMakeFiles/mstat.dir/mstat.cxx.o
[ 71%] Built target mserver
Scanning dependencies of target mlogger
[ 71%] Built target get_record_test
[ 71%] Building CXX object progs/CMakeFiles/mlogger.dir/mlogger.cxx.o
Scanning dependencies of target analyzer
[ 71%] Building CXX object examples/experiment/CMakeFiles/analyzer.dir/analyzer.cxx.o
[ 71%] Linking CXX executable mhist
[ 71%] Linking CXX executable fetest
[ 72%] Linking CXX executable lazylogger
[ 72%] Built target mhist
[ 72%] Built target fetest
[ 73%] Building CXX object examples/experiment/CMakeFiles/analyzer.dir/adccalib.cxx.o
[ 74%] Building CXX object examples/experiment/CMakeFiles/analyzer.dir/adcsum.cxx.o
[ 74%] Built target lazylogger
Scanning dependencies of target frontend
[ 74%] Building CXX object examples/experiment/CMakeFiles/frontend.dir/frontend.cxx.o
[ 75%] Linking CXX executable mstat
Scanning dependencies of target mscb_fe
[ 76%] Building CXX object examples/slowcont/CMakeFiles/mscb_fe.dir/mscb_fe.cxx.o
[ 76%] Built target mstat
[ 76%] Building CXX object examples/experiment/CMakeFiles/analyzer.dir/scaler.cxx.o
[ 77%] Linking CXX executable frontend
Scanning dependencies of target scfe
[ 77%] Built target frontend
[ 77%] Building CXX object examples/slowcont/CMakeFiles/mscb_fe.dir/__/__/drivers/class/hv.cxx.o
[ 78%] Building CXX object examples/slowcont/CMakeFiles/mscb_fe.dir/__/__/drivers/class/multi.cxx.o
[ 79%] Building CXX object examples/slowcont/CMakeFiles/scfe.dir/scfe.cxx.o
Scanning dependencies of target mtfe
[ 80%] Building CXX object examples/mtfe/CMakeFiles/mtfe.dir/mtfe.cxx.o
[ 81%] Linking CXX executable analyzer
[ 81%] Building CXX object examples/slowcont/CMakeFiles/scfe.dir/__/__/drivers/class/hv.cxx.o
[ 82%] Linking CXX executable mtfe
[ 82%] Built target analyzer
[ 83%] Building CXX object examples/slowcont/CMakeFiles/mscb_fe.dir/__/__/drivers/device/nulldev.cxx.o
Scanning dependencies of target rpc_srvr
[ 83%] Built target mtfe
[ 84%] Building CXX object examples/lowlevel/CMakeFiles/rpc_srvr.dir/rpc_srvr.cxx.o
[ 85%] Building CXX object examples/slowcont/CMakeFiles/scfe.dir/__/__/drivers/class/multi.cxx.o
Scanning dependencies of target rpc_clnt
[ 85%] Building CXX object examples/lowlevel/CMakeFiles/rpc_clnt.dir/rpc_clnt.cxx.o
[ 85%] Building CXX object examples/slowcont/CMakeFiles/mscb_fe.dir/__/__/drivers/bus/null.cxx.o
[ 85%] Linking CXX executable rpc_srvr
[ 85%] Built target rpc_srvr
[ 86%] Building CXX object examples/slowcont/CMakeFiles/scfe.dir/__/__/drivers/device/nulldev.cxx.o
[ 87%] Linking CXX executable rpc_clnt
[ 88%] Building CXX object examples/slowcont/CMakeFiles/mscb_fe.dir/__/__/drivers/device/mscbdev.cxx.o
Scanning dependencies of target rpc_test
[ 89%] Building CXX object examples/lowlevel/CMakeFiles/rpc_test.dir/rpc_test.cxx.o
[ 89%] Built target rpc_clnt
[ 90%] Building CXX object examples/slowcont/CMakeFiles/mscb_fe.dir/__/__/mscb/src/mscb.cxx.o
[ 90%] Building CXX object examples/slowcont/CMakeFiles/scfe.dir/__/__/drivers/bus/null.cxx.o
[ 91%] Linking CXX executable mlogger
Scanning dependencies of target consume
[ 92%] Building CXX object examples/lowlevel/CMakeFiles/consume.dir/consume.cxx.o
[ 93%] Linking CXX executable rpc_test
[ 94%] Building CXX object examples/slowcont/CMakeFiles/scfe.dir/__/__/drivers/device/mscbdev.cxx.o
[ 94%] Built target mlogger
Scanning dependencies of target produce
[ 94%] Built target rpc_test
[ 94%] Building CXX object examples/lowlevel/CMakeFiles/produce.dir/produce.cxx.o
[ 95%] Building CXX object examples/slowcont/CMakeFiles/scfe.dir/__/__/mscb/src/mscb.cxx.o
[ 96%] Linking CXX executable msequencer
[ 96%] Built target msequencer
[ 96%] Linking CXX executable consume
[ 96%] Built target consume
[ 97%] Linking CXX executable produce
[ 97%] Built target produce
[ 98%] Linking CXX executable mscb_fe
[ 98%] Built target mscb_fe
[ 99%] Linking CXX executable scfe
[ 99%] Built target scfe
[100%] Linking CXX executable mhttpd
[100%] Built target mhttpd
/midas/build$ |
26 Jan 2020, Konstantin Olchanski, Bug Report, get an open ssl error while trying to compile Midas
|
>
> .../c++ ... /opt/local/lib/libssl.dylib
>
I get the same, libssl from /opt/local, so we are not using openssl shipped with mac os.
We still do not know were OP's libssl comes from and if there is a mismatch between
openssl header files and library (header files from one openssl, library from a different openssl).
OP's cmake output: (no VERBOSE=1)
> -- MIDAS: cmake version: 3.16.3
> -- MIDAS: Found OpenSSL version 1.1.1d
Stefan's and my cmake output: (VERBOSE=1)
> -- MIDAS: cmake version: 3.16.3
> -- Found OpenSSL: /opt/local/lib/libcrypto.dylib (found version "1.1.1d")
> -- MIDAS: Found OpenSSL version 1.1.1d
Without VERBOSE=1 cmake does not tell is which OpenSSL it found (not useful).
With VERBOSE=1 cmake outputs a flood of gunk (not useful).
My solution is to filter the cmake output with grep in "make cmake" ("make cmake3").
So please use that: "make cmake" - the output is roughly the same as normal make - compiler command lines (CFLAGS, library paths, etc),
compiler error message and all of the useful cmake output.
K.O. |
01 Dec 2008, Randolf Pohl, Bug Report, gcc warning in melog.c for midas 4401
|
Hi all,
I have just compiled midas 4401 using SuSE 11.0.
gcc is some odd SuSE version:
gcc version 4.3.1 20080507 (prerelease) [gcc-4_3-branch revision 135036] (SUSE
Linux)
Anyway, gcc stumbled over melog.c. I don't see the reason myself, but my
experience is that gcc is usually right when complaining about "array subscript
is above array bounds". So, just in case somebody knowlegeable wants to have a
look at this....
Cheers,
Randolf
The gcc output:
[...]
cc -g -O3 -Wall -Wuninitialized -Iinclude -Idrivers -I../mxml -Llinux/lib
-DINCLUDE_FTPLIB -D_LARGEFILE64_SOURCE -DHAVE_MYSQL -I/usr/include/mysql
-DHAVE_ROOT -pthread -m64 -I/usr/local/root/root_v5.20.00/include/root
-DHAVE_ZLIB -DOS_LINUX -fPIC -Wno-unused-function -o linux/bin/melog
utils/melog.c linux/lib/libmidas.a -lutil -lpthread -lz
utils/melog.c: In function 'submit_elog':
utils/melog.c:224: warning: array subscript is above array bounds
utils/melog.c:224: warning: array subscript is above array bounds
utils/melog.c:224: warning: array subscript is above array bounds
utils/melog.c:224: warning: array subscript is above array bounds
utils/melog.c:224: warning: array subscript is above array bounds
utils/melog.c:224: warning: array subscript is above array bounds
utils/melog.c:224: warning: array subscript is above array bounds
utils/melog.c:224: warning: array subscript is above array bounds
cc -g -O3 -Wall -Wuninitialized -Iinclude -Idrivers -I../mxml -Llinux/lib
-DINCLUDE_FTPLIB -D_LARGEFILE64_SOURCE -DHAVE_MYSQL -I/usr/include/mysql
-DHAVE_ROOT -pthread -m64 -I/usr/local/root/root_v5.20.00/include/root
-DHAVE_ZLIB -DOS_LINUX -fPIC -Wno-unused-function -o linux/bin/mlxspeaker
utils/mlxspeaker.c linux/lib/libmidas.a -lutil -lpthread -lz |
14 Jul 2004, Piotr Zolnierczuk, , future directions discussion?
|
Sorry the previous message got mangled:
Hi,
I think that rather than spending too much time on where to put files
and how to define the environment - I am guilty of that myself - we should
perhaps have some discussion on the future of MIDAS.
Are we ready for 2.0?
Stefan - do you have any ideas/enhancements?
1) For one I would like to explore memory mapping (mmap()) on Linux.
I've used it once upon a time on DEC OSF/1 and I found it really nice
compared to shared memory. From a user standpoint it behaves as a shared
memory but is mapped to a real file that can be easily "removed" when
neccessary. One really annoying thing in MIDAS is when it goes ballistic
the cleanup is somewhat tricky.
The question if there is any performance penalty associated
2) Expanding hardware support:
a) custom microcontrolers?
b) other hardware
c) how about a "standard" Linux device /dev/midas for various
PCI cards (PCI<->CAMAC) (PCI<->VME)
3) I have never really seen a midas deployment that uses interrupts.
I do understand the ease of polling and the fact that these days CPU's
are cheap but sometimes it is important to use interrupts.
Any examples/experience?
?)
Piotr |
14 Jul 2004, Piotr Zolnierczuk, , future direction discussion?
|
Hi,
I think that rather than spending too much time on where to
put files and how to define the environment - I am guilty of that myself.
We should be perhaps have some discussion on the future of MIDAS.
Are we ready for 2.0?
Stefan - do you have any ideas/enhancements?
1) For one I would like to explore memory mapping (mmap()) on Linux
- I've used it once upon a time on DEC OSF/1 and I found it really
nice compared to shared memory.
From a user standpoint it behaves as a shared memory but is mapped
to a real file that can be easily "removed" when neccessary.
One really annoying thing in MIDAS is when it goes ballistic
the cleanup which is somewhat tricky.
The question if there is any performance penalty associated
2) Expanding hardware support:
a) custom microcontrolers?
b) more hardware
c) how about a "standard" Linux device /dev/midas
for various PCI cards (PCI<->CAMAC) (PCI<->VME)
3) I have never really seen a midas deployment that uses interrupts.
I do understand the ease of polling and the fact that these days
CPU's are cheap but sometimes it is important to use interrupts.
Any examples/experience?
?)
Piotr |
14 Jul 2004, Stefan Ritt, , future direction discussion?
|
Have changed your entry as Non-HTML (easier to reply to...)
Here are some "initial" comments, by no means complete...
> Are we ready for 2.0?
> Stefan - do you have any ideas/enhancements?
A big thing along the horizon I see is the ROME environment
(http://midas.psi.ch/rome/). So we will move away from PAW to ROOT. Although
the DAQ part will stay untouched, the whole analysis back-end changes,
including some XML configuration and MySQL support. I guess that would justify
a 2.0. I will discuss this at TRIUMF when I come in September, see how useful
ROME is for other users...
> 1) For one I would like to explore memory mapping (mmap()) on Linux
> - I've used it once upon a time on DEC OSF/1 and I found it really
> nice compared to shared memory.
> From a user standpoint it behaves as a shared memory but is mapped
> to a real file that can be easily "removed" when neccessary.
> One really annoying thing in MIDAS is when it goes ballistic
> the cleanup which is somewhat tricky.
> The question if there is any performance penalty associated
I guess there are no performance penalties, since under the hood both
techniques are handled similarly. The problem is that besides share memories
you need also semaphores to controll exclusive access to the memory, either
shm() funcitons or mmap(), so this would only fix half of the problem. I seem
to remember that mmap() was not available on some Ultix systems or so, but I
guess that's obsolete by now...
> 2) Expanding hardware support:
> a) custom microcontrolers?
> b) more hardware
> c) how about a "standard" Linux device /dev/midas
> for various PCI cards (PCI<->CAMAC) (PCI<->VME)
Well, you cannot develop hardware support for hardware you don't have, so the
policy up to now was that everyone developing some special drivers or hardware
support contributed it to the package. About c), we have already a CAMAC
driver standard, but at the user space level, so I don't see the benefit of
having kernel-mode standardized drivers. The only difference will be that the
debugging will be harder. VME standard is there in a kind of poor start right
now, but I expect to finish it this fall.
As for a), there is the MSCB system (http://midas.psi.ch/mscb) which has midas
support on the device driver and bus driver level, but I learned that
distributing hardware (or PCB designs if you like) is much harder than sharing
software.
> 3) I have never really seen a midas deployment that uses interrupts.
> I do understand the ease of polling and the fact that these days
> CPU's are cheap but sometimes it is important to use interrupts.
> Any examples/experience?
It's not only the "ease" of polling, but also that it's faster (in almost all
cases) and less troublesome. But hey, interrupt support is included in mfe.c,
so if you are fanatic about interrupts, please feel free to use them. |
15 Jul 2004, Konstantin Olchanski, , future direction discussion?
|
> > Are we ready for 2.0?
I disapprove of version number inflation. Why not go straight for midas version
3000-Pro-Z?
> A big thing along the horizon I see is the ROME environment
> (http://midas.psi.ch/rome/). So we will move away from PAW to ROOT. Although
> the DAQ part will stay untouched, the whole analysis back-end changes,
> including some XML configuration and MySQL support.
I looked at the ROME slides from Pierre, and it seems to suffer badly from the
second-system syndrome (read The Mythical Man-Month).
For us, it is important to get the data into a form where we can process it with
ROOT and I would prefer if we could concentrate in improving the (embryonic) ROOT
online analysis capabilities.
> > 1) For one I would like to explore memory mapping (mmap())
It is trivial to replace System-V shared memory with mmap(). I am surprised that
it has not been done yet. System-V semaphores are a little bit harder to get rid of.
> > 2) Expanding hardware support:
> > a) custom microcontrolers?
> > b) more hardware
> > c) how about a "standard" Linux device /dev/midas
> > for various PCI cards (PCI<->CAMAC) (PCI<->VME)
We cannot expect MIDAS Authors to provide drivers for all possible hardware.
At best, we can mount an effort to collect exisiting drivers from all MIDAS users
"out there" and to integrate them into MIDAS.
Even that is highly non-trivial- many drivers use non-portable native hardware
access interfaces (direct bit-banging on PPCs, VMIC library on VMIC Linux
machines, etc). We have already failed to create an efficient generic portable VME
access library.
> As for a), there is the MSCB system (http://midas.psi.ch/mscb)
I never saw the point of having tons of MIDAS code for MSCB hardware that nobody
has and nobody will ever have.
> > 3) I have never really seen a midas deployment that uses interrupts.
>
> It's not only the "ease" of polling, but also that it's faster (in almost all
> cases) and less troublesome. But hey, interrupt support is included in mfe.c,
> so if you are fanatic about interrupts...
Interrupts are important when one cannot afford chewing up CPU cycles, memory
cycles, PCI cycles and VME cycles on polling.
As I understand, common CAMAC hardware does not generate interrupts- this explains
lack of examples and lack of interrupt use. At TRIUMF, our new VME hardware
supports interrupts and I have a VMIC-based setup where I can (and intend to) test
MIDAS support of interrupts.
K.O. |
13 Jan 2020, Peter Kunz, Forum, frontend issues with midas-2019-09
|
After upgrading to the lastes MIDAS version I got the DAQ frontend of my application running by changing all compiler directives from cc to g++ and using
#include "mfe.h"
extern HNDLE hDB
extern "C" {
#include <CAENComm.h>
}
With these changes everything seems to work fine.
However, I'm having trouble with a slow control frontend using a tcpip driver. It compiled well with the older MIDAS version. Even though all the functions in question are defined in the frontend code, the following error comes up:
g++ -o feMotor -DOS_LINUX -Dextname -g -O2 -fPIC -Wall -Wuninitialized -fpermissive -I/home/pkunz/packages/midas/include -I. -I/home/pkunz/packages/midas/drivers/bus /home/pkunz/packages/midas/lib/libmidas.a feMotor.o /home/pkunz/packages/midas/drivers/bus/tcpip.o cd_Galil.o /home/pkunz/packages/midas/lib/libmidas.a /home/pkunz/packages/midas/lib/mfe.o -lm -lz -lutil -lnsl -lpthread -lrt
/usr/bin/ld: /home/pkunz/packages/midas/lib/mfe.o: in function `initialize_equipment()':
/home/pkunz/packages/midas/src/mfe.cxx:687: undefined reference to `interrupt_configure(int, int, long)'
/usr/bin/ld: /home/pkunz/packages/midas/lib/mfe.o: in function `readout_enable(unsigned int)':
/home/pkunz/packages/midas/src/mfe.cxx:1236: undefined reference to `interrupt_configure(int, int, long)'
/usr/bin/ld: /home/pkunz/packages/midas/src/mfe.cxx:1238: undefined reference to `interrupt_configure(int, int, long)'
/usr/bin/ld: /home/pkunz/packages/midas/lib/mfe.o: in function `main':
/home/pkunz/packages/midas/src/mfe.cxx:2791: undefined reference to `interrupt_configure(int, int, long)'
/usr/bin/ld: /home/pkunz/packages/midas/src/mfe.cxx:2792: undefined reference to `interrupt_configure(int, int, long)'
collect2: error: ld returned 1 exit status
make: *** [Makefile:36: feMotor] Error 1
I guess the the aforementioned DAQ frontend compiles because its equipment definitions don't call on the function `initialize_equipment()', but I can't figure out why it doesn't work. Help is appreciated. P.K. |
13 Jan 2020, Konstantin Olchanski, Forum, frontend issues with midas-2019-09
|
(please use the "plain" text, much easier to answer).
Hi, Peter, I think you misread the error message. There is no error about initialize_equipment(), the error
is about interrupt_configure(). initialize_equipment() is just one of the functions that calls it.
The cause of the error most likely is a mismatch between the declaration of interrupt_configure() in
mfe.h and the definition of this function in your program (in feMotor.c, I guess).
Sometimes this mismatch is hard to identify just by looking at the code.
One fool-proof method to debug this is to extract the actual function prototypes from your object files,
both the declaration ("U") in mfe.o and the definition ("T") in your program should be identical:
nm feMotor.o | grep -i interrupt | c++filt
0000000000000f90 T interrupt_configure(int, int, long) <--- should be this
nm ~/packages/midas/lib/libmfe.a | grep -i interrupt | c++filt
U interrupt_configure(int, int, long)
If they are different, you adjust your program until they match. One way to ensure the match is to copy
the declaration from mfe.h into your program.
K.O.
<p> </p>
<table align="center" cellspacing="1" style="border:1px solid #486090; width:98%">
<tbody>
<tr>
<td style="background-color:#486090">Peter Kunz wrote:</td>
</tr>
<tr>
<td style="background-color:#FFFFB0">
<p>After upgrading to the lastes MIDAS version I got the DAQ frontend of my application
running by changing all compiler directives from cc to g++ and using</p>
<p>#include "mfe.h"</p>
<p>extern HNDLE hDB</p>
<p> extern "C" { <br />
#include <CAENComm.h> <br />
}</p>
<p>With these changes everything seems to work fine.</p>
<p>However, I'm having trouble with a slow control frontend using a tcpip driver. It
compiled well with the older MIDAS version. Even though all the functions in question are defined in the
frontend code, the following error comes up:</p>
<div style="background:#eee;border:1px solid #ccc;padding:5px 10px;">g++ -o feMotor
-DOS_LINUX -Dextname -g -O2 -fPIC -Wall -Wuninitialized -fpermissive -
I/home/pkunz/packages/midas/include -I. -I/home/pkunz/packages/midas/drivers/bus
/home/pkunz/packages/midas/lib/libmidas.a feMotor.o
/home/pkunz/packages/midas/drivers/bus/tcpip.o cd_Galil.o
/home/pkunz/packages/midas/lib/libmidas.a /home/pkunz/packages/midas/lib/mfe.o -lm -lz -lutil -lnsl -
lpthread -lrt<br />
/usr/bin/ld: /home/pkunz/packages/midas/lib/mfe.o: in function
`initialize_equipment()':<br />
/home/pkunz/packages/midas/src/mfe.cxx:687: undefined reference to
`interrupt_configure(int, int, long)'<br />
/usr/bin/ld: /home/pkunz/packages/midas/lib/mfe.o: in function
`readout_enable(unsigned int)':<br />
/home/pkunz/packages/midas/src/mfe.cxx:1236: undefined reference to
`interrupt_configure(int, int, long)'<br />
/usr/bin/ld: /home/pkunz/packages/midas/src/mfe.cxx:1238: undefined reference to
`interrupt_configure(int, int, long)'<br />
/usr/bin/ld: /home/pkunz/packages/midas/lib/mfe.o: in function `main':<br />
/home/pkunz/packages/midas/src/mfe.cxx:2791: undefined reference to
`interrupt_configure(int, int, long)'<br />
/usr/bin/ld: /home/pkunz/packages/midas/src/mfe.cxx:2792: undefined reference to
`interrupt_configure(int, int, long)'<br />
collect2: error: ld returned 1 exit status<br />
make: *** [Makefile:36: feMotor] Error 1</div>
<p> I guess the the aforementioned DAQ frontend compiles because its equipment
definitions don't call on the function `initialize_equipment()', but I can't figure out why
it doesn't work. Help is appreciated. P.K.</p>
</td>
</tr>
</tbody>
</table>
<p> </p> |
13 Jan 2020, Konstantin Olchanski, Forum, frontend issues with midas-2019-09
|
(please use the "plain" text, much easier to answer).
Hi, Peter, I think you misread the error message. There is no error about
initialize_equipment(), the error is about interrupt_configure(). initialize_equipment() is just
one of the functions that calls it.
The cause of the error most likely is a mismatch between the declaration of
interrupt_configure() in mfe.h and the definition of this function in your program (in
feMotor.c, I guess).
Sometimes this mismatch is hard to identify just by looking at the code.
One fool-proof method to debug this is to extract the actual function prototypes from your
object files, both the declaration ("U") in mfe.o and the definition ("T") in your program
should be identical:
nm feMotor.o | grep -i interrupt | c++filt
0000000000000f90 T interrupt_configure(int, int, long) <--- should be this
nm ~/packages/midas/lib/libmfe.a | grep -i interrupt | c++filt
U interrupt_configure(int, int, long)
If they are different, you adjust your program until they match. One way to ensure the
match is to copy the declaration from mfe.h into your program.
K.O.
<p> </p>
<table align="center" cellspacing="1" style="border:1px solid #486090; width:98%">
<tbody>
<tr>
<td style="background-color:#486090">Peter Kunz wrote:</td>
</tr>
<tr>
<td style="background-color:#FFFFB0">
<p>After upgrading to the lastes MIDAS version I got the DAQ frontend of my
application running by changing all compiler directives from cc to g++ and using</p>
<p>#include "mfe.h"</p>
<p>extern HNDLE hDB</p>
<p> extern "C" { <br />
#include <CAENComm.h> <br />
}</p>
<p>With these changes everything seems to work fine.</p>
<p>However, I'm having trouble with a slow control frontend using a
tcpip driver. It compiled well with the older MIDAS version. Even though all the functions in
question are defined in the frontend code, the following error comes up:</p>
<div style="background:#eee;border:1px solid #ccc;padding:5px 10px;">g++
-o feMotor -DOS_LINUX -Dextname -g -O2 -fPIC -Wall -Wuninitialized -fpermissive
-I/home/pkunz/packages/midas/include -I. -I/home/pkunz/packages/midas/drivers/bus
/home/pkunz/packages/midas/lib/libmidas.a feMotor.o
/home/pkunz/packages/midas/drivers/bus/tcpip.o cd_Galil.o
/home/pkunz/packages/midas/lib/libmidas.a /home/pkunz/packages/midas/lib/mfe.o -lm -lz
-lutil -lnsl -lpthread -lrt<br />
/usr/bin/ld: /home/pkunz/packages/midas/lib/mfe.o: in function
`initialize_equipment()':<br />
/home/pkunz/packages/midas/src/mfe.cxx:687: undefined reference to
`interrupt_configure(int, int, long)'<br />
/usr/bin/ld: /home/pkunz/packages/midas/lib/mfe.o: in function
`readout_enable(unsigned int)':<br />
/home/pkunz/packages/midas/src/mfe.cxx:1236: undefined reference to
`interrupt_configure(int, int, long)'<br />
/usr/bin/ld: /home/pkunz/packages/midas/src/mfe.cxx:1238: undefined
reference to `interrupt_configure(int, int, long)'<br />
/usr/bin/ld: /home/pkunz/packages/midas/lib/mfe.o: in function `main':
<br />
/home/pkunz/packages/midas/src/mfe.cxx:2791: undefined reference to
`interrupt_configure(int, int, long)'<br />
/usr/bin/ld: /home/pkunz/packages/midas/src/mfe.cxx:2792: undefined
reference to `interrupt_configure(int, int, long)'<br />
collect2: error: ld returned 1 exit status<br />
make: *** [Makefile:36: feMotor] Error 1</div>
<p> I guess the the aforementioned DAQ frontend compiles because
its equipment definitions don't call on the function `initialize_equipment()', but I
can't figure out why it doesn't work. Help is appreciated. P.K.</p>
</td>
</tr>
</tbody>
</table>
<p> </p> |
13 Jan 2020, Peter Kunz, Forum, frontend issues with midas-2019-09
|
Thanks for explaining this, Konstantin.
After updating the function to
INT interrupt_configure(INT cmd, INT source, POINTER_T adr)
{
return CM_SUCCESS;
}
it compiled without errors. In the original code the "INT source" variable was missing.
> (please use the "plain" text, much easier to answer).
>
> Hi, Peter, I think you misread the error message. There is no error about initialize_equipment(), the error
> is about interrupt_configure(). initialize_equipment() is just one of the functions that calls it.
>
> The cause of the error most likely is a mismatch between the declaration of interrupt_configure() in
> mfe.h and the definition of this function in your program (in feMotor.c, I guess).
>
> Sometimes this mismatch is hard to identify just by looking at the code.
>
> One fool-proof method to debug this is to extract the actual function prototypes from your object files,
> both the declaration ("U") in mfe.o and the definition ("T") in your program should be identical:
>
> nm feMotor.o | grep -i interrupt | c++filt
> 0000000000000f90 T interrupt_configure(int, int, long) <--- should be this
>
> nm ~/packages/midas/lib/libmfe.a | grep -i interrupt | c++filt
> U interrupt_configure(int, int, long)
>
> If they are different, you adjust your program until they match. One way to ensure the match is to copy
> the declaration from mfe.h into your program.
>
> K.O.
>
>
>
>
> <p> </p>
>
> <table align="center" cellspacing="1" style="border:1px solid #486090; width:98%">
> <tbody>
> <tr>
> <td style="background-color:#486090">Peter Kunz wrote:</td>
> </tr>
> <tr>
> <td style="background-color:#FFFFB0">
> <p>After upgrading to the lastes MIDAS version I got the DAQ frontend of my application
> running by changing all compiler directives from cc to g++ and using</p>
>
> <p>#include "mfe.h"</p>
>
> <p>extern HNDLE hDB</p>
>
> <p> extern "C" { <br />
> #include <CAENComm.h> <br />
> }</p>
>
> <p>With these changes everything seems to work fine.</p>
>
> <p>However, I'm having trouble with a slow control frontend using a tcpip driver. It
> compiled well with the older MIDAS version. Even though all the functions in question are defined in the
> frontend code, the following error comes up:</p>
>
> <div style="background:#eee;border:1px solid #ccc;padding:5px 10px;">g++ -o feMotor
> -DOS_LINUX -Dextname -g -O2 -fPIC -Wall -Wuninitialized -fpermissive -
> I/home/pkunz/packages/midas/include -I. -I/home/pkunz/packages/midas/drivers/bus
> /home/pkunz/packages/midas/lib/libmidas.a feMotor.o
> /home/pkunz/packages/midas/drivers/bus/tcpip.o cd_Galil.o
> /home/pkunz/packages/midas/lib/libmidas.a /home/pkunz/packages/midas/lib/mfe.o -lm -lz -lutil -lnsl -
> lpthread -lrt<br />
> /usr/bin/ld: /home/pkunz/packages/midas/lib/mfe.o: in function
> `initialize_equipment()':<br />
> /home/pkunz/packages/midas/src/mfe.cxx:687: undefined reference to
> `interrupt_configure(int, int, long)'<br />
> /usr/bin/ld: /home/pkunz/packages/midas/lib/mfe.o: in function
> `readout_enable(unsigned int)':<br />
> /home/pkunz/packages/midas/src/mfe.cxx:1236: undefined reference to
> `interrupt_configure(int, int, long)'<br />
> /usr/bin/ld: /home/pkunz/packages/midas/src/mfe.cxx:1238: undefined reference to
> `interrupt_configure(int, int, long)'<br />
> /usr/bin/ld: /home/pkunz/packages/midas/lib/mfe.o: in function `main':<br />
> /home/pkunz/packages/midas/src/mfe.cxx:2791: undefined reference to
> `interrupt_configure(int, int, long)'<br />
> /usr/bin/ld: /home/pkunz/packages/midas/src/mfe.cxx:2792: undefined reference to
> `interrupt_configure(int, int, long)'<br />
> collect2: error: ld returned 1 exit status<br />
> make: *** [Makefile:36: feMotor] Error 1</div>
>
> <p> I guess the the aforementioned DAQ frontend compiles because its equipment
> definitions don't call on the function `initialize_equipment()', but I can't figure out why
> it doesn't work. Help is appreciated. P.K.</p>
> </td>
> </tr>
> </tbody>
> </table>
>
> <p> </p> |
14 Jan 2020, Stefan Ritt, Forum, frontend issues with midas-2019-09
|
We updated midas/examples/experiment/frontend.cxx to correctly contain
/*-- Interrupt configuration ---------------------------------------*/
INT interrupt_configure(INT cmd, INT source, POINTER_T adr)
{
switch (cmd) {
case CMD_INTERRUPT_ENABLE:
break;
case CMD_INTERRUPT_DISABLE:
break;
case CMD_INTERRUPT_ATTACH:
break;
case CMD_INTERRUPT_DETACH:
break;
}
return SUCCESS;
}
but if you upgrade from C to C++ from your own old frontend code you might be hit by that issue.
Maybe Konstantin can update elog:1526 to contain a hint about "INT source".
Stefan |
14 Jan 2020, Stefan Ritt, Forum, frontend issues with midas-2019-09
|
Actually now I see that
a4) poll_event() and interrupt_configure() have "source" as "int[]" instead of "int" (why did this work before?)
mention this already, but maybe it's not completely clear that one has to change "int[] source" to "int source"
Stefan
> We updated midas/examples/experiment/frontend.cxx to correctly contain
>
> /*-- Interrupt configuration ---------------------------------------*/
>
> INT interrupt_configure(INT cmd, INT source, POINTER_T adr)
> {
> switch (cmd) {
> case CMD_INTERRUPT_ENABLE:
> break;
> case CMD_INTERRUPT_DISABLE:
> break;
> case CMD_INTERRUPT_ATTACH:
> break;
> case CMD_INTERRUPT_DETACH:
> break;
> }
> return SUCCESS;
> }
>
> but if you upgrade from C to C++ from your own old frontend code you might be hit by that issue.
>
> Maybe Konstantin can update elog:1526 to contain a hint about "INT source".
>
> Stefan |
05 Feb 2024, Pavel Murat, Forum, forbidden equipment names ?
|
Dear MIDAS experts,
I have multiple daq nodes with two data receiving FPGAs on the PCIe bus each.
The FPGAs come under the names of DTC0 and DTC1. Both FPGAs are managed by the same slow control frontend.
To distinguish FPGAs of different nodes from each other, I included the hostname to the equipment name,
so for node=mu2edaq09 the FPGA names are 'mu2edaq09:DTC0' and 'mu2edaq09:DTC1'.
The history system didn't like the names, complaining that
21:26:06.334 2024/02/05 [Logger,ERROR] [mlogger.cxx:5142:open_history,ERROR] Equipment name 'mu2edaq09:DTC1'
contains characters ':', this may break the history system
So the question is : what are the safe equipment/driver naming rules and what characters
are not allowed in them? - I think this is worth documenting, and the current MIDAS docs at
https://daq00.triumf.ca/MidasWiki/index.php/Equipment_List_Parameters#Equipment_Name
don't say much about it.
-- many thanks, regards, Pasha |
13 Feb 2024, Konstantin Olchanski, Forum, forbidden equipment names ?
|
> equipment names are 'mu2edaq09:DTC0' and 'mu2edaq09:DTC1'
I think all names permitted for ODB keys are allowed as equipment names, any valid UTF-8,
forbidden chars are "/" (ODB path separator) and "\0" (C string terminator). Maximum length
is 31 byte (plus "\0" string terminator). (Fixed length 32-byte names with implied terminator
are no longer permitted).
The ":" character is used in history plot definitions and we are likely eventually change that,
history event names used to be pairs of "equipment_name:tag_name" but these days with per-variable
history, they are triplets "equipment_name,variable_name,tag_name". The history plot editor
and the corresponding ODB entries need to be updated for this. Then, ":" will again be a valid
equipment name.
I think if you disable the history for your equipments, MIDAS will stop complaining about ":" in the name.
K.O. |
19 Aug 2006, Konstantin Olchanski, Bug Fix, fixes for minor mhttpd problems
|
I commited fix for minor mhttpd problems (rev 3314):
- for a newly created experiment, the "history" button gave the error [history
panel "" does not exist] (new problem introduced in revision 3150)
- for very long history panel names (close to the 32-character limit) history
plots produce the error "Cannot find /history/display/foo/bar/variables" (broke
in revision 3190 "use strlcpy()", in previous revisions, this bug was silent
stack corruption)
- elog attachments did not work for file names containing character plus (+)
(attachement URLs should be properly encoded to escape special CGI characters)
K.O. |
26 Aug 2006, Konstantin Olchanski, Bug Fix, fixes for minor mhttpd problems
|
> I commited fix for minor mhttpd problems (rev 3314):
> - elog attachments did not work for file names containing character plus (+)
> (attachement URLs should be properly encoded to escape special CGI characters)
I accidentally indirectly learned that the above change produced incorrect URLs
when more than one experiment is defined. I now commited a fix to this problem.
K.O. |
25 Feb 2005, Konstantin Olchanski, Bug Fix, fixed: double free in FORMAT_MIDAS ybos.c causing lazylogger crashes
|
We stumbled upon and fixed a "double free" bug in src/ybos.c causing crashes in
lazylogger writing .mid files in the FORMAT_MIDAS format (why does it use
ybos.c? Pierre says- for generic file i/o). Why this code had ever worked before
remains a mystery. K.O. |
29 Dec 2010, Konstantin Olchanski, Bug Report, fixed. odb corruption, odb race condition?
|
>
> The only remaining problem when running my script is some kind of deadlock between the ODB and SYSMSG semaphores...
>
I committed changes to odb.c and midas.c fixing a number of places that could corrupt ODB and SYSMSG data, and fixing a number of deadlocks. Without these
changes, on my Mac, MIDAS will reliably corrupt ODB or deadlock while running my odbedit fork-bomb torture test script. These changes still need to be tested on
Linux (but I do not expect any problems).
Because my changes do not fix the original race condition in client creation/removal/cleanup, you may still occasionally see messages like this:
13:35:14 [ODBEdit24,ERROR] [odb.c:2112:db_find_key,ERROR] hkey 169592 invalid key type 376
13:35:15 [ODBEdit28,ERROR] [odb.c:3268:db_get_value,ERROR] hkey 162072 entry "Name" is of type NULL, not STRING
For now, I am happy that we no longer corrupt ODB (nor deadlock) and I will work with Stefan on a permanent solution for this.
Special thanks go to the T2K/ND280 experiment, specifically, to Tim Nicholls and to the unnamed person who emailed me their script that executes many odbedit
commands to setup midas history plots.
svn rev 4930
K.O.
P.S. Below is my torture test script, I usually run many of them in a sequence "./test1.perl >& xxx1; ./test1.perl >& xxx2; ... etc".
#!/usr/bin/perl -w
for (my $i=0; $i<50; $i++)
{
#my $cmd = "odbedit -c \'scl -w\' &";
#my $cmd = "odbedit -c \'ls -l /system/clients\' >& xxx$i &";
my $cmd = "odbedit -c \'ls -l /system/clients\' &";
system $cmd;
}
#end
svn rev 4930
K.O. |
11 Feb 2011, Konstantin Olchanski, Bug Report, fixed. odb corruption, odb race condition?
|
> >
> > The only remaining problem when running my script is some kind of deadlock between the ODB and SYSMSG semaphores...
> >
>
> For now, I am happy that we no longer corrupt ODB (nor deadlock) ...
>
Found one more deadlock between ODB and SYSMSG semaphores, this time through cm_watchdog():
If cm_watchdog somehow runs while we are holding the ODB semaphore, it will eventually try to lock SYSMSG (through bm_cleanup & co) in
violation of our semaphore locking order. If at the same time another application tries to lock stuff using the correct order (SYSMSG first, ODB last),
the two programs will deadlock (wait for each other forever). I presently have two copies of gdb attached to two copies of odbedit
waiting for each other in a deadlock through this cm_watchdog scenario...
Solution shall follow quickly, I have been hunting this deadlock for the last couple of weeks...
K.O. |
|