ELOG Midas

Hi all, I have updated our daq computer to the latest OS 10.15 with the idea that then I will get all our daq programs running and will not need to update again in years to come. But now I am running unto even more problems trying to compile Midas. OS 10.15.3, Xcode 11.3 with commad line tools Darrens-Mac-mini:~ betacage$ cd packages/midas/build/ Darrens-Mac-mini:build betacage$ cmake .. -- MIDAS: cmake version: 3.16.3 -- The C compiler identification is AppleClang 11.0.0.11000033 -- The CXX compiler identification is AppleClang 11.0.0.11000033 -- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc -- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ -- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- MIDAS: CMAKE_INSTALL_PREFIX: /Users/betacage/packages/midas -- MIDAS: Found ROOT version 6.18/04 -- Found ZLIB: /usr/lib/libz.dylib -- MIDAS: Found ZLIB version -- MIDAS: Found OpenSSL version 1.1.1d -- MIDAS: MySQL not found -- MIDAS: ODBC not found -- MIDAS: SQLITE not found -- MIDAS: nvidia-smi not found -- Found Git: /usr/bin/git (found version "2.21.0 (Apple Git-122.2)") -- MIDAS example/experiment: MIDAS in-tree-build -- MIDAS: Found ZLIB version -- MIDAS example/experiment: Found ROOT version 6.18/04 -- Configuring done -- Generating done -- Build files have been written to: /Users/betacage/packages/midas/build Darrens-Mac-mini:build betacage$ make install [ 1%] Building CXX object CMakeFiles/mfeo.dir/src/mfe.cxx.o In file included from /Users/betacage/packages/midas/src/mfe.cxx:15: In file included from /Users/betacage/packages/midas/include/mfe.h:13: /Users/betacage/packages/midas/include/midas.h:159:10: fatal error: 'pthread.h' file not found #include <pthread.h> ^~~~~~~~~~~ 1 error generated. make[2]: *** [CMakeFiles/mfeo.dir/src/mfe.cxx.o] Error 1 make[1]: *** [CMakeFiles/mfeo.dir/all] Error 2 I guess that I am maybe the first one trying to install MIDAS in this OS, so I am willing to help as much as I can with getting this going. I have found a solution to this by downgrading Xcode to version 10 https://stackoverflow.com/questions/58524715/failing-to-compile-a-c-application-under-macos-catalina-10-15 If you don't have other solutions I will try to do that tomorrow. Thanks Berta

> I have updated our daq computer to the latest OS 10.15 ... FWIW, I do not have macos 10.15. I have 10.13 at home and 10.14 in the office. Maybe Stefan has it? > In file included from /Users/betacage/packages/midas/include/mfe.h:13: > /Users/betacage/packages/midas/include/midas.h:159:10: fatal error: 'pthread.h' file not found > #include <pthread.h> > ^~~~~~~~~~~ Hmm... pthread.h should be there, I do not see any notice of it's removal in macos. So I suspect a mis-installation of c++ compilers. In the past we had problems with this, in addition to installing Xcode from the app store, one most install some missing stuff manually ("XCode command line tools"), if I have it right, the command is "xcode-select --install". If you figure out how to build it, I think midas will work just fine on the latest macos. K.O.

> > I have updated our daq computer to the latest OS 10.15 ... > > FWIW, I do not have macos 10.15. I have 10.13 at home and 10.14 in the office. Maybe Stefan has it? No, I'm stuck with 10.14 due to the current lack of 64-bit support of some programs. I would try KOs proposal to do a "xcode-select --install". Stefan

> > > I have updated our daq computer to the latest OS 10.15 ... > > > > FWIW, I do not have macos 10.15. I have 10.13 at home and 10.14 in the office. Maybe Stefan has it? > > No, I'm stuck with 10.14 due to the current lack of 64-bit support of some programs. > Ok, in this case, I will update my office mac mini to 10.15. K.O.

> > Ok, in this case, I will update my office mac mini to 10.15. > > K.O. Hi Konstantin, Any luck with Midas in OS 10.15? I have downgraded to Xcode 10.2.1 as suggested in the post linked in my first post, without any luck, I keep getting the same pthead.h error. MY gcc uses the 10.15 sdk despite having v 10.2.1 for Xcode and command line tools Darrens-Mac-mini:~ betacage$ gcc --version Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include- dir=/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/c++/4.2.1 Apple LLVM version 10.0.1 (clang-1001.0.46.4) Target: x86_64-apple-darwin19.3.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin and that is where I suspect the problem is. Will keep investigating next week. Thanks for your help. Berta

> > Any luck with Midas in OS 10.15? > > Best I can tell, the problem is not in midas: pthread.h should be there, somewhere. > > K.O. HI Konstain, Thanks for your reply. I have been investigating this issue of pthread in OS 10.15. Found this forum post https://github.com/wjakob/nori/issues/16 that seem to suggest that pthread.h is in a different directory from OS 10.14 and on. And the end of the post the developer suggests that he has found a way to fix the Cmake to work so I have checked his code updates in here https://github.com/wjakob/nori/commit/be46cccc01a75b21dad1a3f61baa108fe644fc4b I have added his lines if(APPLE) # Try to auto-detect a suitable SDK execute_process(COMMAND bash -c "xcodebuild -version -sdk | grep MacOSX | grep Path | head -n 1 | cut -f 2 - d ' '" OUTPUT_VARIABLE CMAKE_OSX_SYSROOT) string(REGEX REPLACE "(\r?\n)+$" "" CMAKE_OSX_SYSROOT "${CMAKE_OSX_SYSROOT}") string(REGEX REPLACE "^.*X([0-9.]*).sdk$" "\\1" CMAKE_OSX_DEPLOYMENT_TARGET "${CMAKE_OSX_SYSROOT}") endif() to my local pakages/midas/CMakeLists.txt and run cmake .., make install again but now I get a different error [ 1%] Building CXX object CMakeFiles/mfeo.dir/src/mfe.cxx.o clang: error: invalid version number in '-mmacosx-version- min=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/DriverKit19.0.sdk' clang: warning: using sysroot for 'DriverKit' but targeting 'MacOSX' [-Wincompatible-sysroot] I will continue investigating this issue tomorrow, but please let me know if you have any suggestions. Also my version of cmake is 3.16.3 Thanks Berta

For your reference, here on my MacOSX 10.14.6 with XCode 11.3.1 the pthread.h file is present in locations listed below. Did you execute "xcode-select --install" ? ~$ locate pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/AppleTVOS.platform/Developer/SDKs/AppleTVOS.sdk/usr/include/pthread/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/AppleTVOS.platform/Developer/SDKs/AppleTVOS.sdk/usr/include/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/AppleTVSimulator.platform/Developer/SDKs/AppleTVSimulator.sdk/usr/include/pthread/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/AppleTVSimulator.platform/Developer/SDKs/AppleTVSimulator.sdk/usr/include/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/pthread/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/WatchOS.platform/Developer/SDKs/WatchOS.sdk/usr/include/pthread/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/WatchOS.platform/Developer/SDKs/WatchOS.sdk/usr/include/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/WatchSimulator.platform/Developer/SDKs/WatchSimulator.sdk/usr/include/pthread/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/WatchSimulator.platform/Developer/SDKs/WatchSimulator.sdk/usr/include/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk/usr/include/pthread/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk/usr/include/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator.sdk/usr/include/pthread/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator.sdk/usr/include/pthread.h /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/pthread/pthread.h /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/pthread.h

> For your reference, here on my MacOSX 10.14.6 with XCode 11.3.1 the pthread.h file is present in locations listed below. > > Did you execute "xcode-select --install" ? > > > ~$ locate pthread.h > /Applications/Xcode.app/Contents/Developer/Platforms/AppleTVOS.platform/Developer/SDKs/AppleTVOS.sdk/usr/include/pthread/pthread.h > /Applications/Xcode.app/Contents/Developer/Platforms/AppleTVOS.platform/Developer/SDKs/AppleTVOS.sdk/usr/include/pthread.h > /Applications/Xcode.app/Contents/Developer/Platforms/AppleTVSimulator.platform/Developer/SDKs/AppleTVSimulator.sdk/usr/include/pthread/pthread.h > /Applications/Xcode.app/Contents/Developer/Platforms/AppleTVSimulator.platform/Developer/SDKs/AppleTVSimulator.sdk/usr/include/pthread.h > /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/pthread/pthread.h > /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/pthread.h > /Applications/Xcode.app/Contents/Developer/Platforms/WatchOS.platform/Developer/SDKs/WatchOS.sdk/usr/include/pthread/pthread.h > /Applications/Xcode.app/Contents/Developer/Platforms/WatchOS.platform/Developer/SDKs/WatchOS.sdk/usr/include/pthread.h > /Applications/Xcode.app/Contents/Developer/Platforms/WatchSimulator.platform/Developer/SDKs/WatchSimulator.sdk/usr/include/pthread/pthread.h > /Applications/Xcode.app/Contents/Developer/Platforms/WatchSimulator.platform/Developer/SDKs/WatchSimulator.sdk/usr/include/pthread.h > /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk/usr/include/pthread/pthread.h > /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk/usr/include/pthread.h > /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator.sdk/usr/include/pthread/pthread.h > /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator.sdk/usr/include/pthread.h > /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/pthread/pthread.h > /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/pthread.h Hi Stefan, Thanks for looking into this. Yes, I have done "xcode-select --install" This is my output when I try to locate pthread.h Darrens-Mac-mini:~ betacage$ locate pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/AppleTVOS.platform/Developer/SDKs/AppleTVOS.sdk/usr/include/pthread/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/AppleTVOS.platform/Developer/SDKs/AppleTVOS.sdk/usr/include/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/AppleTVSimulator.platform/Developer/SDKs/AppleTVSimulator.sdk/usr/include/pthread/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/AppleTVSimulator.platform/Developer/SDKs/AppleTVSimulator.sdk/usr/include/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/pthread/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/WatchOS.platform/Developer/SDKs/WatchOS.sdk/usr/include/pthread/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/WatchOS.platform/Developer/SDKs/WatchOS.sdk/usr/include/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/WatchSimulator.platform/Developer/SDKs/WatchSimulator.sdk/usr/include/pthread/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/WatchSimulator.platform/Developer/SDKs/WatchSimulator.sdk/usr/include/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk/usr/include/pthread/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk/usr/include/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator.sdk/usr/include/pthread/pthread.h /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator.sdk/usr/include/pthread.h /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/pthread/pthread.h /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/pthread.h /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/pthread/pthread.h /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/pthread.h and this is what I see in my SDKs folder Darrens-Mac-mini:SDKs betacage$ pwd /Library/Developer/CommandLineTools/SDKs Darrens-Mac-mini:SDKs betacage$ ls MacOSX.sdk MacOSX10.14.sdk MacOSX10.15.sdk So it looks like in this computer one needs to read the version of the SDK as pthread.h is not in MacOSX.sdk but in MacOSX10.15.sdk Thanks Berta

Another thought: Can you delete the build directory and run cmake again? Like $ cd /midas/build $ rm -rf * $ cmake .. If you have the old build cache and upgraded the OS in meantime, the cache needs to be rebuild. Stefan

Another thought: Can you delete the midas build directory and run cmake again? Like $ cd midas/build $ rm -rf * $ cmake .. $ make VERBOSE=1 If you have the old build cache and upgraded your OS in meantime, the cache needs to be rebuild. The VERBOSE flag tells you the compiler options, and you see where the compiler looks for the SDK directory. Cheers, Stefan

> Another thought: Can you delete the midas build directory and run cmake again? Like > > $ cd midas/build > $ rm -rf * > $ cmake .. > $ make VERBOSE=1 > > If you have the old build cache and upgraded your OS in meantime, the cache needs to be rebuild. The VERBOSE > flag tells you the compiler options, and you see where the compiler looks for the SDK directory. > > Cheers, > Stefan Hi Stefan, Thanks again for your reply. Yes! the suggestion of rebuilding the cache was the right one, this is the output of cmake .. Darrens-Mac-mini:~ betacage$ cd packages/midas/build/ Darrens-Mac-mini:build betacage$ rm -rf * Darrens-Mac-mini:build betacage$ cmake .. -- MIDAS: cmake version: 3.16.3 -- The C compiler identification is AppleClang 11.0.0.11000033 -- The CXX compiler identification is AppleClang 11.0.0.11000033 -- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- MIDAS: CMAKE_INSTALL_PREFIX: /Users/betacage/packages/midas -- MIDAS: Found ROOT version 6.18/04 -- Found ZLIB: /usr/lib/libz.dylib (found version "1.2.11") -- MIDAS: Found ZLIB version 1.2.11 -- Found OpenSSL: /usr/lib/libcrypto.dylib (found version "1.1.1d") -- MIDAS: Found OpenSSL version 1.1.1d -- MIDAS: MySQL not found -- MIDAS: ODBC not found -- MIDAS: SQLITE not found -- MIDAS: nvidia-smi not found -- Setting default build type to "RelWithDebInfo" -- Found Git: /usr/bin/git (found version "2.21.1 (Apple Git-122.3)") -- MIDAS example/experiment: MIDAS in-tree-build -- MIDAS: Found ZLIB version 1.2.11 -- MIDAS example/experiment: Found ROOT version 6.18/04 -- Configuring done -- Generating done -- Build files have been written to: /Users/betacage/packages/midas/build Unfortunately running make VERBOSE=1 the compilation crashes, with the following error [ 39%] Linking CXX executable mhttpd cd /Users/betacage/packages/midas/build/progs && /Applications/CMake.app/Contents/bin/cmake -E cmake_link_script CMakeFiles/mhttpd.dir/link.txt --verbose=1 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -O2 -g -DNDEBUG -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -Wl,-search_paths_first -Wl,- headerpad_max_install_names CMakeFiles/mhttpd.dir/mhttpd.cxx.o CMakeFiles/mhttpd.dir/mongoose6.cxx.o CMakeFiles/mhttpd.dir/mgd.cxx.o CMakeFiles/mhttpd.dir/__/mscb/src/mscb.cxx.o -o mhttpd ../libmidas.a /usr/lib/libssl.dylib /usr/lib/libcrypto.dylib -lz ld: cannot link directly with dylib/framework, your binary is not an allowed client of /usr/lib/libcrypto.dylib for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) make[2]: *** [progs/mhttpd] Error 1 make[1]: *** [progs/CMakeFiles/mhttpd.dir/all] Error 2 make: *** [all] Error 2 Libcrypto seems to be part of the OpenSSL, so I am back to the original problem again. I did install OpenSSL via MacPorts. I have found this tread regarding this problem. But have to go and pick up my kids from school, so I don't have time to investigate today. https://stackoverflow.com/questions/58446253/xcode-11-ld-error-your-binary-is-not-an-allowed-client-of-usr-lib-libcrypto-dy Thanks again for staying with me and for responding so promptly. Berta

Now you are stuck with openssl, which is optional for mhttpd. If you only use mhttpd locally, you maybe don't need SSL support. In that case you can jus do [midas/build] $ cmake -D NO_SSL=1 .. To disable that. If you do need SSL, maybe you can try to install openssl11 via mac ports. Stefan

> Now you are stuck with openssl, which is optional for mhttpd. If you only use mhttpd locally, you maybe don't need SSL support. In that case you can jus do > > [midas/build] $ cmake -D NO_SSL=1 .. > > To disable that. If you do need SSL, maybe you can try to install openssl11 via mac ports. > > Stefan Thanks Stefan. If I run the compilation with the flag NO_SSL it works just fine. Still I think that Mhttpd is going to be important for us in the future as we may want to control the experiment remotely, so I will keep trying. But at least I can get started. Thanks Berta

> > [midas/build] $ cmake -D NO_SSL=1 .. > If I run the compilation with the flag NO_SSL it works just fine. ... FYI, the mongoose616 branch now has support for using the mbedtls https library, this library seems to build easily from sources and removes our dependency on where/how/which openssl library is installed. I hope to have this included in the next release of midas. K.O.

Thanks Konstantin, I will keep an eye for the next release so that I can update my Midas to include ssl libraries. Thanks > > > [midas/build] $ cmake -D NO_SSL=1 .. > > If I run the compilation with the flag NO_SSL it works just fine. ... > > FYI, the mongoose616 branch now has support for using the mbedtls https library, > this library seems to build easily from sources and removes our dependency > on where/how/which openssl library is installed. I hope to have this included > in the next release of midas. > > K.O.

Since commit bc227a8a34def271a598c0200ca30d73223c3373 I've been getting the compilation error below (on a Raspberry Pi 3 Model B Plus Rev 1.3).

The fix is obvious from the reported error, but I am wondering whether this should be fixed in the main git??

[ 7%] Building CXX object CMakeFiles/objlib.dir/src/json_paste.cxx.o
/home/nemu/nemu/tmidas/midas/src/json_paste.cxx: In function ‘int GetQWORD(const MJsonNode*
, const char*, UINT64*)’:
/home/nemu/nemu/tmidas/midas/src/json_paste.cxx:324:19: error: ‘const class MJsonNode’ has
no member named ‘GetLL’; did you mean ‘GetInt’?
      *qw = node->GetLL();
                  ^~~~~
                  GetInt
make[2]: *** [CMakeFiles/objlib.dir/build.make:271: CMakeFiles/objlib.dir/src/json_paste.cx

> git pull > git submodule update confirmed. just run into this myself. I think "make" should warn about out of date git modules. Also check that the build git version is tagged with "-dirty". K.O.