Back Midas Rome Roody Rootana
  Midas DAQ System, Page 36 of 50  Not logged in ELOG logo
New entries since:Wed Dec 31 16:00:00 1969
Entry  08 Oct 2009, Tim Nicholls, Bug Report, mserver linking fails when using shared library 
I have experienced a problem building MIDAS from the head of the SVN repository (rev 4458) when 
specifying the shared library flag. Whie the shared library appears to compile and link OK, the 
subsequent compilation of mserver fails as follows:

$ make ROOTSYS= NEED_SHLIB=1

<... snipped some lines ...>

ld -shared -o linux/lib/libmidas.so linux/lib/midas.o linux/lib/system.o linux/lib/mrpc.o 
linux/lib/odb.o linux/lib/ybos.o linux/lib/ftplib.o linux/lib/mxml.o linux/lib/history_midas.o 
linux/lib/history_sql.o linux/lib/history.o linux/lib/alarm.o linux/lib/elog.o linux/lib/strlcpy.o -lutil -
lpthread -lz -lc
cc -c -g -O2 -Wall -Wuninitialized -Iinclude -Idrivers -I../mxml -Llinux/lib -DINCLUDE_FTPLIB   -
D_LARGEFILE64_SOURCE -DHAVE_ZLIB -DOS_LINUX -fPIC -Wno-unused-function -o linux/lib/mana.o 
src/mana.c
cc -c -g -O2 -Wall -Wuninitialized -Iinclude -Idrivers -I../mxml -Llinux/lib -DINCLUDE_FTPLIB   -
D_LARGEFILE64_SOURCE -DHAVE_ZLIB -DOS_LINUX -fPIC -Wno-unused-function -o 
linux/lib/cnaf_callback.o src/cnaf_callback.c
cc -c -g -O2 -Wall -Wuninitialized -Iinclude -Idrivers -I../mxml -Llinux/lib -DINCLUDE_FTPLIB   -
D_LARGEFILE64_SOURCE -DHAVE_ZLIB -DOS_LINUX -fPIC -Wno-unused-function -o linux/lib/mfe.o 
src/mfe.c
g++ -Dextname -DMANA_LITE -c -g -O2 -Wall -Wuninitialized -Iinclude -Idrivers -I../mxml -
Llinux/lib -DINCLUDE_FTPLIB   -D_LARGEFILE64_SOURCE -DHAVE_ZLIB -DOS_LINUX -fPIC -Wno-
unused-function -o linux/lib/fal.o src/fal.c
cc -g -O2 -Wall -Wuninitialized -Iinclude -Idrivers -I../mxml -Llinux/lib -DINCLUDE_FTPLIB   -
D_LARGEFILE64_SOURCE -DHAVE_ZLIB -DOS_LINUX -fPIC -Wno-unused-function -o 
linux/bin/mserver src/mserver.c -lmidas -Wl,-rpath,/usr/local/lib -lutil -lpthread -lz
/usr/bin/ld: linux/bin/mserver: hidden symbol `__dso_handle' in /usr/lib/gcc/x86_64-redhat-
linux/4.1.2/crtbegin.o is referenced by DSO
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: ld returned 1 exit status
make: *** [linux/bin/mserver] Error 1

Having googled the error, it appears to be solved by modifying the linker statement for the shared 
library in the Makefile at line 464 to use g++ rather than ld:

463c463
< 	ld -shared -o $@ $^ $(LIBS) -lc
---
> 	$(CXX) -shared -o $@ $^ $(LIBS) -lc

Presumably this is because g++ knows better how to link in the appropriate system libraries required 
for some of the recently added C++ code? 

This was on Scientific Linux SL5.2 x86_64, gcc version 4.1.2, glibc version 2.5-24.

Tim
    Reply  09 Oct 2009, Konstantin Olchanski, Bug Report, change to building and linking libmidas.so, mserver linking fails when using shared library 
> --- Makefile
> < 	ld -shared -o $@ $^ $(LIBS) -lc
> ---
> > 	$(CXX) -shared -o $@ $^ $(LIBS) -lc

Will do. We also have a long standing request to change shared library name from lidmidas.so to libmidas-shared.so.

Different name for the .so file will permit us to build the shared library by default, but still link all MIDAS executables
with the static libmidas.a.

(there is no benefit from linking MIDAS executables - mlogger, mhttpd, etc - with the shared library,
and there is a significant cost in confusion from version skew between the executables and shared
libraries - I have had enough midnight calls "why did odbedit stop working? Oh, who changed LD_LIBRARY_PATH
and why is it now binding against this obsolete libmidas.so left over from 2 years ago?").

For user applications you can do whatever, but for MIDAS core applications I strongly suggest that they
be linked to the midas static library.

K.O.
       Reply  11 Oct 2009, Konstantin Olchanski, Bug Report, change to building and linking libmidas.so, mserver linking fails when using shared library 
> > > 	$(CXX) -shared -o $@ $^ $(LIBS) -lc

Done.
MIDAS shared library renamed from libmidas.so to libmidas-shared.so and always build on Linux (.so) and MacOS (.dylib and .so bundle).

Users who wish to link their applications to the shared library should change their Makefile from "gcc ... -lmidas" to "gcc ... -lmidas-shared".

MIDAS core applications (mhttpd, mlogger, etc) are linked to the static library to permit multiple versions of midas to be used
at the same time (LD_LIBRARY_PATH should point to the shared library for *which* midas?!?), avoid problems with wrong setting
of LD_LIBRARY_PATH and to avoid problems with version skew (which we found unavoidable once a midas daq is used for a year or
more - main reason we gave up on using the midas shared library in the first place).

svn rev 4594
K.O.
Entry  09 Oct 2009, Exaos Lee, Bug Report, Building error of history_midas.cxx due to missing declaration build-20091010.log
Platform: Debian Linux testing
Compiler: gcc 4.3.4 (Debian 4.3.4-2)
Arch: x86
Description:
  The "g++" is whining while compiling history_midas.cxx. Please see the attached log file.
This can be fixed by add "#include <cstdlib>" to the C++ source. You know, different versions
of g++ don't act the same way. I think, maybe in version 4.2 or before, g++ can automatically
include the C header "stdlib.h" (which in C++ should be <cstdlib> because of some confusion
between C and C++), but not in version 4.3 or later. I tested g++-4.4, the problem still exists.
And g++-4.2 gives no error.
    Reply  11 Oct 2009, Konstantin Olchanski, Bug Report, Building error of history_midas.cxx due to missing declaration 
>  The "g++" is whining while compiling history_midas.cxx. Please see the attached log file.

Fixed. svn 4594. K.O.
Entry  01 Oct 2009, Pierre-Andre Amaudruz, Bug Report, mfe.c: poll_event() before frontend_init() 
The latest version of mfe.c has a problem where poll_event() is called before
frontend_init() and this causes a crash because in poll_event() we try to access
VME before it is initialized in frontend_init(). K.O.
    Reply  01 Oct 2009, Stefan Ritt, Bug Report, mfe.c: poll_event() before frontend_init() 
> The latest version of mfe.c has a problem where poll_event() is called before
> frontend_init() and this causes a crash because in poll_event() we try to access
> VME before it is initialized in frontend_init(). K.O.

Oops, that sneaked in when doing the last modification to display the frontend status. 
I refactored register_equipment() so that frontend_init() gets called before 
poll_event().
Entry  29 Sep 2009, Exaos Lee, Bug Report, Error invoking 'odbedit': db_validate_size 
Revision: r4567
Error output:
$ odbedit -e expcvadc
odbedit: /opt/DAQ/repos/bot/midas/src/odb.c:651: db_validate_sizes: Assertion `sizeof(EQUIPMENT_INFO) == 400' failed.
zsh: abort      odbedit -e expcvadc
    Reply  29 Sep 2009, Exaos Lee, Bug Report, Error invoking 'odbedit': db_validate_size 
It seems to be fixed in svn-r4568:
-----------------------------------------------------------------------
r4568 | olchanski | 2009-09-27 23:56:39 +0800 (&#26085;, 27  9&#26376; 2009) | 5 lines

mhttpd: compile using the C++ compiler.
mhttpd: fix wrong initialization of /History/ODBC_DSN
odb.c: size of EQUIPMENT_INFO has changed.
Makefile: use "-O2" compiler flag instead of "-O3" - to fix SL5 gcc crash (ICE) 

But another compiling error:
Linking CXX executable bin/mh2sql
CMakeFiles/mh2sql.dir/utils/mh2sql.cxx.o: In function `main':
/opt/DAQ/repos/bot/midas/utils/mh2sql.cxx:150: undefined reference to `MakeMidasHistoryODBC()'
       Reply  30 Sep 2009, Konstantin Olchanski, Bug Report, mh2sql does not build, Error invoking 'odbedit': db_validate_size 
> Linking CXX executable bin/mh2sql
> CMakeFiles/mh2sql.dir/utils/mh2sql.cxx.o: In function `main':
> /opt/DAQ/repos/bot/midas/utils/mh2sql.cxx:150: undefined reference to `MakeMidasHistoryODBC()'

Yes, I am in the process of changing the midas history interface and accidentally committed a version of 
mh2sql (utility for converting MIDAS history .hst files to SQL database) that uses the new interface.

This is now fixed in svn rev 4571.

The new C++ interface to the MIDAS history is in include/history.h and implementations for data storage 
using both midas .hst files and SQL (ODBC/MySQL) database are also committed (history_midas.cxx
and history_sql.cxx). The file history_odbc.cxx will be removed after some more testing of the new 
interface.

(All the new code is not activated yet, pending more testing).

K.O.
    Reply  30 Sep 2009, Konstantin Olchanski, Bug Report, Error invoking 'odbedit': db_validate_size 
> $ odbedit -e expcvadc
> odbedit: /opt/DAQ/repos/bot/midas/src/odb.c:651: db_validate_sizes: Assertion 
`sizeof(EQUIPMENT_INFO) == 400' failed.

Yes, this is now fixed, svn rev 4571 should be okey. Sorry about causing this problem - Stefan added 
some useful additional data to EQUIPMENT_INFO and my check for binary compatibility caught it and 
complained. Unfortunately on Saturday Stefan had to abruptly go back to PSI and things have been a little 
bit chaotic because we did not complete the testing of all the new changes and additions.

K.O.
Entry  21 Sep 2009, Stefan Ritt, Info, New feature: Stop run after a certain time 
A new feature has been implemented in revision 4561 which allows runs with a 
certain duration. To use this, one has to set the variaable

/Logger/Run Duration

to a non-zero value in seconds. After a run lasted for this duration, it gets 
stopped automatically by the logger. If the auto-restart flag is on, this allows 
sequences of automatically started and stopped runs with all then have the same 
duration.
    Reply  22 Sep 2009, Stefan Ritt, Info, New feature: Stop run after a certain time 
> A new feature has been implemented in revision 4561 which allows runs with a 
> certain duration. To use this, one has to set the variaable
> 
> /Logger/Run Duration
> 
> to a non-zero value in seconds. After a run lasted for this duration, it gets 
> stopped automatically by the logger. If the auto-restart flag is on, this allows 
> sequences of automatically started and stopped runs with all then have the same 
> duration.

A similar scheme has been implemented to pose a certain duration on subruns. This can 
be controlled by the variable

/Logger/Subrun duration

when set to a non-zero value in seconds.
Entry  09 Sep 2009, Jimmy Ngai, Forum, Retrieve start/stop time in offline 
Hi All,

I set "/Analyzer/ODB Load" to true and analyzed a run in offline mode. After
that, I found the start time and stop time in /RunInfo did not reflect the
correct time as in online. How do I retrieve the correct start/stop time from
the ODB in offline mode?

Thanks!

Jimmy
    Reply  10 Sep 2009, Stefan Ritt, Forum, Retrieve start/stop time in offline 
> I set "/Analyzer/ODB Load" to true and analyzed a run in offline mode. After
> that, I found the start time and stop time in /RunInfo did not reflect the
> correct time as in online. How do I retrieve the correct start/stop time from
> the ODB in offline mode?

Most trees in the ODB are not loaded with "/Analyzer/ODB Load", since you might 
want to have the start/stop time of the offline analysis there for example 
(although I agree that the online start/stop time is more interesting). So you 
have several options:

- modify mana.c. There is a function odb_load(), which first locks the whole ODB 
and then unprotects "/Experiment/Run Parameters" for example. Just add three more 
lines for "/Runinfo".

- write a run summary when running online. After each run, write a summary with 
start/stop time, number of events, settings etc. into some file. I usually do this 
in the EOR routine of the online analyzer and write directly into a CSV file which 
I can import directly into Excel. There I can make filtering depending on certain 
parameters, like show me all runs with more than x events where setting y was 10.

- extract the ODB from the .mid file with "odbhist -e filename.mid" and look into 
that.

- The time stamp of each event is in UNIX time form (seconds since 1.1.1970), so 
you now exactly when each event was recorded.

Hope one of this helps...

- Stefan
Entry  18 Aug 2009, Denis Calvet, Suggestion, Could not create strings other than 32 characters with odbedit -c "..." command 
Hi,
I am writing shell scripts to create some tree structure in an ODB. When 
creating an array of strings, the default length of each string element is 32 
characters. If odbedit is used interactively to create the array of strings, 
the user is prompted to enter a different length if desired. But if the 
command odbedit is called from a shell script, I did not succeed in passing 
the argument to get a different length.
I tried:
odbedit -c "create STRING Test[8][40]"
Or:
odbedit -c "create STRING Test[8] 40"
Or:
odbedit -c "create STRING Test[8] \n 40"
etc. all produce an array of 8 strings with 32 characters each.
I haven't tried all possible syntaxes, but I suspect the length argument is 
dropped. If it has not been fixed in a later release than the one I am using, 
could this problem be looked at?
Thanks,
Denis.
  
    Reply  03 Sep 2009, Stefan Ritt, Suggestion, Could not create strings other than 32 characters with odbedit -c "..." command 
> Hi,
> I am writing shell scripts to create some tree structure in an ODB. When 
> creating an array of strings, the default length of each string element is 32 
> characters. If odbedit is used interactively to create the array of strings, 
> the user is prompted to enter a different length if desired. But if the 
> command odbedit is called from a shell script, I did not succeed in passing 
> the argument to get a different length.
> I tried:
> odbedit -c "create STRING Test[8][40]"
> Or:
> odbedit -c "create STRING Test[8] 40"
> Or:
> odbedit -c "create STRING Test[8] \n 40"
> etc. all produce an array of 8 strings with 32 characters each.
> I haven't tried all possible syntaxes, but I suspect the length argument is 
> dropped. If it has not been fixed in a later release than the one I am using, 
> could this problem be looked at?

Ok, I added a command

odbedit -c "create STRING Test[8][40]"

which works now. Please update to SVN revision 4555 of odbedit.c

- Stefan
       Reply  06 Sep 2009, Exaos Lee, Suggestion, Could not create strings other than 32 characters with odbedit -c "..." command 
> Ok, I added a command
> 
> odbedit -c "create STRING Test[8][40]"
> 
> which works now. Please update to SVN revision 4555 of odbedit.c
> 
> - Stefan

If I want to create only one string, should I write like this:

  odbedit -c "create STRING Test[] [256]"

OK. I need it. I will try the new odbedit.
          Reply  06 Sep 2009, Exaos Lee, Suggestion, Could not create strings other than 32 characters with odbedit -c "..." command 
> > Ok, I added a command
> > 
> > odbedit -c "create STRING Test[8][40]"
> > 
> > which works now. Please update to SVN revision 4555 of odbedit.c
> > 
> > - Stefan
> 
> If I want to create only one string, should I write like this:
> 
>   odbedit -c "create STRING Test[] [256]"
> 
> OK. I need it. I will try the new odbedit.

"create STRING test[1][256]" works.
Entry  06 Sep 2009, Exaos Lee, Bug Report, Delete key "/A_Str" problem odbedit.png
Another problem while using odbedit.
I tried the batch mode of "odbedit". I created a key as "/A_Str" by mistake and 
wanted to delete it. Then "odbedit" failed to accept the "Return" key. Please see 
the screen-shot attached. :-(
    Reply  06 Sep 2009, Exaos Lee, Bug Report, Delete key "/A_Str" problem 
> Another problem while using odbedit.
> I tried the batch mode of "odbedit". I created a key as "/A_Str" by mistake and 
> wanted to delete it. Then "odbedit" failed to accept the "Return" key. Please see 
> the screen-shot attached. :-(

This bug has been fixed in the latest repository.
I encountered it in svn-r4488.
Entry  03 Sep 2009, Exaos Lee, Suggestion, Building MIDAS using CMake cmake.zip
I write some configure file to build MIDAS using CMake. The usage is simple:
1. Unzip the attachment, copy "CMakeLists.txt" and directory "cmake" into the
midas source tree.
   $ cp -rp CMakeLists.txt cmake/  <PATH-TO-MIDAS>/
2. make a separate directory, such as "build". It's a good habit to build a
project without polluting the source tree. :-)
   $ mkdir build
3. Executing cmake
   $ cd build && cmake <PATH-TO-MIDAS>
4. Make
   $ make

Or, you can generate Xcode project files:
  $ cmake -G Xcode <PATH-TO-MIDAS>
or using visual studio
  $ cmake -G "Visual Studio" <PATH-TO-MIDAS>
(I havn't Visual Studio and windows, so the above command is not tested.)
or using other IDEs, such as KDevelop3, Eclipse, etc, just type:
  $ cmake -G "KDevelop3" <PATH-TO-MIDAS>
or
  $ cmake -G "Eclipse CDT4" <PATH-TO-MIDAS>


I test the configure file with GNU make and CMake 2.6.4 on Debian Lenny. I
havn't add installation commands now. Maybe later. If anyone interests in it, I
may check it again. Anyway, I'm using it.
    Reply  03 Sep 2009, Exaos Lee, Suggestion, Some screenshot using CMake with MIDAS Screenshot-11.pngScreenshot-13.png
I didn't add optimization flags to compile, so I got link error while generating mcnaf as I reported before.
The screen-shots show that the configure files works because I have modified the "driver/camac/camacrpc.c".
    Reply  06 Sep 2009, Exaos Lee, Suggestion, Updated "CMakeLists.txt" CMakeLists.txt
Add installation commands. Please see the attachment.
Entry  06 Sep 2009, Exaos Lee, Bug Report, Compiling error of "src/history_odbc.cxx" build-err.log
Version svn-r4556, I got a compiling error as below:
/opt/DAQ/bot/midas/src/history_odbc.cxx: In member function 'virtual int 
SqlODBC::GetNumRows()':
/opt/DAQ/bot/midas/src/history_odbc.cxx:589: error: cannot convert 'SQLINTEGER*' 
to 'long int*' for argument '2' to 'SQLRETURN SQLRowCount(void*, long int*)'
/opt/DAQ/bot/midas/src/history_odbc.cxx: In member function 'virtual const char* 
SqlODBC::GetColumn(int)':
/opt/DAQ/bot/midas/src/history_odbc.cxx:638: error: cannot convert 'SQLINTEGER*' 
to 'long int*' for argument '6' to 'SQLRETURN SQLGetData(void*, SQLUSMALLINT, 
SQLSMALLINT, void*, long int, long int*)'
make[2]: *** [CMakeFiles/midas-static.dir/src/history_odbc.cxx.o] Error 1
make[2]: Leaving directory `/opt/DAQ/bot/midas/build'
make[1]: *** [CMakeFiles/midas-static.dir/all] Error 2
make[1]: Leaving directory `/opt/DAQ/bot/midas/build'

The detail error log is attached. I used my CMake script without any optimization flags. I will try the default Makefile again.
    Reply  06 Sep 2009, Exaos Lee, Bug Report, Compiling error of "src/history_odbc.cxx" 

Exaos Lee wrote:
Version svn-r4556, I got a compiling error as below:
The detail error log is attached. I used my CMake script without any optimization flags. I will try the default Makefile again.


BUG is confirmed using the default "Makefile".
       Reply  06 Sep 2009, Exaos Lee, Bug Fix, Maybe a fix history_odbc.cxx.diff
Changing "SQLINTEGER" to "SQLLEN" maybe let the compiling pass. See the attached diff.

But I failed in another error. It was the problem in CMakeLists.txt. (FIXED)
Entry  03 Sep 2009, Exaos Lee, Bug Report, Prompt problem about odbedit Screenshot-10.png
I tried to use odbedit to set the "/System/Prompt" to "%h:%e:%s %p> " and got a
problem: pressing "Return" doesn't work any more. But "[%h:%e:%s]%p> " works fine.
Please see the attachment.
    Reply  03 Sep 2009, Stefan Ritt, Bug Report, Prompt problem about odbedit 
> I tried to use odbedit to set the "/System/Prompt" to "%h:%e:%s %p> " and got a
> problem: pressing "Return" doesn't work any more. But "[%h:%e:%s]%p> " works fine.
> Please see the attachment.

I fixed that problem in SVN revision 4556. It occurred when the prompt does start with a 
'%' which nobody tried before...
Entry  21 Aug 2009, Exaos Lee, Forum, Link error of "mcnaf" 
The "utils/mcnaf.c" uses "camop()",
180:         printf("camop\n");
181:         camop();
But "drivers/camac/camacrpc.c" provides "cam_op()":
void cam_op()
{
}

If you compile each source into an object, you may encounter a link error as
mcnaf.c:(.text+0x3b1): undefined reference to `camop'
collect2: ld returned 1 exit status
The "mcstd.h" provides "camop". So, we need to change "camacrpc.c" to match the definition and fix the link error.
    Reply  21 Aug 2009, Konstantin Olchanski, Forum, Link error of "mcnaf" 
> If you compile each source into an object, you may encounter a link error ...

Also camac rpc did not work at all last time we tried to use it at triumf, maybe 4 month ago in the Dragon 
experiment (upgrade from older version of midas). Never got around to trace down why. YMMV.

K.O.
    Reply  31 Aug 2009, Exaos Lee, Forum, Link error of "mcnaf" 
I repeated the link error again. I also found the almost all sources located in "driver/camac/" using "cam_op()" but not "camop()". Please see the grep result below:
drivers/camac/camaclx.c:760:INLINE void cam_op()
drivers/camac/camacnul.c:200:INLINE void cam_op()
drivers/camac/camacrpc.c:563:void cam_op()
drivers/camac/cc7700pci.c:744:INLINE void cam_op()
drivers/camac/ces2117.c:227:void cam_op()
drivers/camac/ces8210.c:553:void cam_op(void)
drivers/camac/ces8210.c:576:    cam_op();
drivers/camac/ces8210.c:625:    cam_op();
drivers/camac/dsp004.c:692:void cam_op()
drivers/camac/hyt1331.c:1125:INLINE void cam_op()
drivers/camac/jorway73a.c:563:INLINE void cam_op()
drivers/camac/kcs2926.c:618:INLINE void cam_op()
drivers/camac/kcs2927.c:677:INLINE void cam_op()
drivers/camac/wecc32.c:554:INLINE void cam_op()

I also found why the default Makefile can pass away this problem. I found the "make" using the following command to compile the "mcnaf":
cc -g -O3 -Wall -Wuninitialized -Iinclude -Idrivers -I../mxml -Llinux/lib -DINCLUDE_FTPLIB   -D_LARGEFILE64_SOURCE -DHAVE_MYSQL -I/usr/include/mysql -DHAVE_ODBC -DHAVE_ZLIB -DOS_LINUX -fPIC -Wno-unused-function -o linux/bin/mcnaf utils/mcnaf.c drivers/camac/camacrpc.c linux/lib/libmidas.a -lutil -lpthread -lodbc -lz
I try to re-implement the link error again. So I wrote some test code.

1. anullf.h
#ifndef _ANULLF_H_
#define _ANULLF_H_ 1

#define EXTERNAL extern

#if defined( _MSC_VER )
#define INLINE __inline
#elif defined(__GNUC__)
#define INLINE __inline__
#else
#define INLINE
#endif
/* make functions under WinNT dll exportable */
#if defined(_MSC_VER) && defined(MIDAS_DLL)
#define EXPRT __declspec(dllexport)
#else
#define EXPRT
#endif

EXTERNAL INLINE void EXPRT camop() {  };

#endif

2. ma_1.c
#include "anullf.h"

void cam_op() {}

3. ma.c
#include "anullf.h"
#include <stdio.h>

int main(int argc, char** argv)
{
  camop();
}

Then, only compiling as below can pass:
$ gcc -O -o ma ma_1.c ma.c
If you remove the option "-O", compiling immediately fails in a link error. Whether you compile into objects then link them or just compile them together, you can pass with option "-O" or the similar options. It is because the compile could solve linking to a dummy function automatically with option "-O".

Anyway, we should fix this no matter changing the "mcstd.h" or codes located in "driver/camac/".
       Reply  31 Aug 2009, Exaos Lee, Forum, Why should we use "INLINE" here? 
There are many "INLINE" definitions in "include/*.h". Both GNU C and C99 permit using inline functions. I still wonder why. Smile
          Reply  03 Sep 2009, Stefan Ritt, Forum, Why should we use "INLINE" here? 

Exaos Lee wrote:
There are many "INLINE" definitions in "include/*.h". Both GNU C and C99 permit using inline functions. I still wonder why. Smile


The INLINE statements are a relict from times where a subroutine call was in the order of a few microseconds. This was when you probably were pretty young, and we had MS DOS PCs running at 66 MHz and 640 kB (not MB!) of memory. At that time, inlining the CAMAC functions gave a 50% speedup, believe it or not! I guess now this is completely obsolete, plus modern compilers can do inlining automatically if they realize that there is a benefit. So we should remove all the inline business. We plan some major rework later in September, so I will include that on the todo list there.

- Stefan
Entry  01 Sep 2009, Jimmy Ngai, Forum, Timeout during run transition 
Dear All,

I'm using SL5 and MIDAS rev 4528. Occasionally, when I stop a run in odbedit, 
a timeout would occur: 
[midas.c:9496:rpc_client_call,ERROR] rpc timeout after 121 sec, routine 
= "rc_transition", host = "computerB", connection closed
Error: Unknown error 504 from client 'Frontend' on host computerB

This error seems to be random without any reason or pattern. After this error 
occurs, I cannot start or stop any run. Sometime restarting MIDAS can bring 
the system working again, but sometime not.

Another transition timeout occurs after I change any ODB value using the web 
interface:
[midas.c:8291:rpc_client_connect,ERROR] timeout on receive remote computer 
info: 
[midas.c:3642:cm_transition,ERROR] cannot connect to client "Frontend" on host 
computerB, port 36255, status 503
Error: Cannot connect to client 'Frontend'

This error is reproducible: start run -> change ODB value within webpage -> 
stop run -> timeout!

Any idea?

Thanks,
Jimmy
    Reply  03 Sep 2009, Stefan Ritt, Forum, Timeout during run transition 
> Dear All,
> 
> I'm using SL5 and MIDAS rev 4528. Occasionally, when I stop a run in odbedit, 
> a timeout would occur: 
> [midas.c:9496:rpc_client_call,ERROR] rpc timeout after 121 sec, routine 
> = "rc_transition", host = "computerB", connection closed
> Error: Unknown error 504 from client 'Frontend' on host computerB
> 
> This error seems to be random without any reason or pattern. After this error 
> occurs, I cannot start or stop any run. Sometime restarting MIDAS can bring 
> the system working again, but sometime not.
> 
> Another transition timeout occurs after I change any ODB value using the web 
> interface:
> [midas.c:8291:rpc_client_connect,ERROR] timeout on receive remote computer 
> info: 
> [midas.c:3642:cm_transition,ERROR] cannot connect to client "Frontend" on host 
> computerB, port 36255, status 503
> Error: Cannot connect to client 'Frontend'
> 
> This error is reproducible: start run -> change ODB value within webpage -> 
> stop run -> timeout!

A few hints for debugging:

- do the run stop via odbedit and the "-v" flag, like

[local:Online:R]/> stop -v

then you see which computer is contacted when.

- Then put some debugging code into your front-end end_of_run() routine at the 
beginning and the end of that routine, so you see when it's executed and how long 
this takes. If you do lots of things in your EOR routine, this could maybe cause a 
timeout.

- Then make sure that cm_yield() in mfe.c is called periodically by putting some 
debugging code there. This function checks for any network message, such as the 
stop command from odbedit. If you trigger event readout has an endless loop for 
example, cm_yield() will never be called and any transition will timeout.

- Make sure that not 100% CPU is used on your frontend. Some OSes have problems 
handling incoming network connections if the CPU is completely used of if 
input/output operations are too heavy.

- Stefan
Entry  29 Aug 2009, Exaos Lee, Forum, At last, I'm here again! 
I always got a 503 server error while I tried to connect this log book the latest 
weeks. I don't know why. I hope it is not due to the network censorship because 
of the coming National Day of China. Anyway, good luck to me when I want to paste 
something here.
Entry  10 Aug 2009, Konstantin Olchanski, Info, misc changes from PIENU and T2K 
FYI - committed the last changes from TRIUMF DAQ systems for PIENU and T2K/ND280 FGD and TPC 
tests:

- mhttpd: add <odb xxx format="%d">xxx</odb>, similar to AJAX ODBget() method
- alarm.c: if alarm stops the run, log a message (sometimes it is hard to tell "why did this run stop?!?") 
use DETACH transition (was ASYNC - does not follow requested transition sequencing, now calls 
mtransition helper). Also verified that alarm handler always runs on the main computer - for remote 
clients, alarms are processed inside the corresponding mserver process.
- midas.c: event buffer fixes:
-- mserver 100% cpu busy loop if event buffer is full
-- consolidate event buffer cleanup into one routine. do things similar to odb cleanup - check for client 
pid, etc.
-- do not kill clients that have the watchdog timeout set to zero.

svn rev 4541
K.O.
    Reply  10 Aug 2009, Konstantin Olchanski, Info, misc changes from PIENU and T2K 
> FYI - committed the last changes from TRIUMF DAQ systems for PIENU and T2K/ND280 FGD and TPC 
> tests:
> svn rev 4541

Also:

- add traps to event buffer code to catch event buffer (shared memory) corruption observed in PIENU
- dynamically allocate some RPC network data buffers to permit better communication between MIDAS clients built with different values of 
MAX_EVENT_SIZE (in T2K/ND280 the default 4 Mbytes is too small for some users, while other users use the default size - this change permits all 
these programs to talk to each other).

K.O.
Entry  04 Aug 2009, Exaos Lee, Forum, VME-related codes contribution vme4midas.zip
Hi, all

I have some codes while using MIDAS. I upload them here. They are tested with
SIS3100. I haven't other VME controllers, so I don't know whether they work with
other controllers. I just hope that they are helpful. You may find information
from the file "00README.txt" in the package vme4midas.zip. My English is limited, I just hope
that you may catch my ideas. Smile

All my best.

Exaos Lee
    Reply  04 Aug 2009, Exaos Lee, Forum, About python interface 
Coding in Python is faster than in C (but running slower). So, some python interfaces are useful for testing purpose. I hope you may like the PyMVME module for VME bus testing.
    Reply  04 Aug 2009, Exaos Lee, Forum, The contents of the attachment 
As requested from K.O., I paste the "00README.txt" as the following:
#-*- mode: outline -*-
#-*- encoding: utf-8 -*-
#AUTHOR: Exaos Lee <Exaos DOT Lee AT gmail DOT com>

* Directories
  +--> 00README.txt : This file
  |
  +--> bustester : Directory contains utilities for VME bus testing
  |
  +--> modules   : APIs to handle VME modules
  |
  +--> pyutil    : Uitilies in Python, including PyMVME
  |
  +--> sis3100   : Provide lib_sis3100mvme.a/so using with "mvmestd.h"

* Utilities in Python

** PyMVME module

   The module "PyMVME" provides the following stuff:
      a. class StdVME
      	 -- contains standard VME informations.
      b. class MVME_INTERFACE
      	 -- the C structure MVME_INTERFACE wrapped in Python
      c. dict MVME_STATUS
      	 -- the return information defined in "mvmestd.h"
      d. the related useful aliases from "mvmestd.h"
      	 -- including "mvme_addr_t", "mvme_locaddr_t", "mvme_size_t"
      e. class MvmeDev
      	 -- the major class which provides methods to access VME bus.

   You may find examples of how to use module "PyMVME" from "find_caen.py" or
   scripts in dir "test". All of the examples are using "lib_sis3100mvme.so".
   You may find information later in this introduction.

** find_caen.py

   The script to find VME modules from CAEN. Now, it is still in test status
   and can only find ADCs, TDCs or QDCs.

* SIS3100 library to be used togather with "mvmestd.h"

  The directory "sis3100" contains sources to build libraries as the following:
  a. lib_sis3100.a     -- APIs declared in "sis3100_vme_calls.h"
  b. lib_sis3100mvme.a -- APIs declared in "mvmestd.h". It also contains the
     		       	  same APIs from lib_sis3100.a

  If you want to use shared libraries, especially when you are using utilities
  wrote in Python, you may rebuild the libraries as the following:

    $ cd sis3100
    $ make shared

* APIs to handle VME modules

** vadc_caen.h/c

   Provides APIs to handle ADC-type modules from CAEN, including:
      a. ADCs --- V785, V785N
      b. TDCs --- V775, V775N
      c. QDCs --- V792, V792N

* VME bus testers

  Still under development.


Entry  04 Aug 2009, Exaos Lee, Forum, Scripts to handle MIDAS sessions utils4midas.zip
Hi, all again

I have some scripts in "bash" and "Python" to handle MIDAS sessions. Please see the attached utils4midas.zip. I didn't write instructions in detail of how to use them. But I think they are very simple. You may find how to use them by reading the codes and example files.

Best wishes.

Exaos Lee
Entry  03 Aug 2009, Exaos Lee, Forum, How to distinguish the status and value returned from "mvme_read_value(...)" 
The definition of mvme_read_value is as the following:
unsigned int EXPRT mvme_read_value(MVME_INTERFACE * vme, mvme_addr_t vme_addr);

Read single data from VME bus. Useful for register access. See example in
mvme_open()

Parameters:
    	*vme 	  VME structure
    	vme_addr  source address (VME location).

Returns:
    MVME_SUCCESS


Question: How to distinguish the status and value returned? Should the definition be something like
int EXPRT mvme_read_value(MVME_INTERFACE *mvme, mvme_addr_t vme_addr, unsigned int *var);
    Reply  03 Aug 2009, Stefan Ritt, Forum, How to distinguish the status and value returned from "mvme_read_value(...)" 

Exaos Lee wrote:
The definition of mvme_read_value is as the following:
unsigned int EXPRT mvme_read_value(MVME_INTERFACE * vme, mvme_addr_t vme_addr);

Read single data from VME bus. Useful for register access. See example in
mvme_open()

Parameters:
    	*vme 	  VME structure
    	vme_addr  source address (VME location).

Returns:
    MVME_SUCCESS


Question: How to distinguish the status and value returned? Should the definition be something like
int EXPRT mvme_read_value(MVME_INTERFACE *mvme, mvme_addr_t vme_addr, unsigned int *var);


This function is a shortcut when you want something like
  printf("%d\n", mvme_read_value(...)); 

and you know that the status is ok. Without this function, you would need to define a variable
  unsigned long d;
  mvme_read_value(..., &d);
  printf("%d\n", d);

so the above function is just a handy shortcut. If you want to see the status however, you can call the "normal" function mvme_read:
  status = mvme_read(..., &d, adr, 4); 
    Reply  03 Aug 2009, Konstantin Olchanski, Forum, How to distinguish the status and value returned from "mvme_read_value(...)" 
> uint32_t mvme_read_value(MVME_INTERFACE * vme, mvme_addr_t vme_addr);
> Question: How to distinguish the status and value returned?

On VME interfaces using the Universe and tsi148 PCI-VME bridges, your question has no meaning.

The VME address space is directly mapped into the PCI address space, then mmap()ed into your 
program address space. Internally mvme_read_value() is "return *(uint32_t*)(mmap_base + vme_addr);" 
and there is no such thing as "status".

Physically on the VME bus, for single-word VME cycles (mvme_read_value), there are only 2 error 
conditions, an AS or a DS timeout, and these bridges return the bit pattern 0xFFFFFFFF for either error, 
the same as the traditional VME bus always worked (i.e. before PCI, before ISA, back when the VME bus 
*was* the main CPU-memory-IO bus).

So the answer to your question is "yes". If mvme_read_value() returned 0xFFFFFFFF, there was a VME 
bus timeout because the board you are trying to address a) is not installed, b) was unplugged, c) does 
not decode the address you tried to access (maybe you used the wrong AM code or the wrong data 
width).

With Universe and tsi148 PCI-VME bridges, the mvme_read() call runs the DMA interface that can issue 
block transfer cycles on the VME bus. These DMA interfaces have interesting error handling, but 
basically, they only tell you the estimated VME address at which the AS or DS timeout or BERR has 
occurred. For sane VME boards, DMA errors mean very basic breakage of the VME crate and VME board.

With non-directly attached VME interfaces, i.e. the SIS3100, you can also have communication errors. I 
do not know how those are reported by the SIS3100 Linux drivers, and I do not know how the MIDAS 
driver reports them. But I do know that if you see those errors, your interface is very broken and VME 
bus errors are the least of your worries.

P.S. There also exist PCI bus errors, they also return the bit pattern 0xFFFFFFFF and mean basic 
breakage inside your computer. PCI-PCI and PCI-host bridges have special registers you can read to 
find out the exact cause of the error ("your computer is broken").

K.O.
Entry  02 Jul 2009, Dawei Liu, Forum, Data taking hangs in the middle of run 
Hi,

We are using midas to read ADC. It sometimes hung in the middle of data taking.
We tried to disable analyzer and only run with frontend. The problem still
exists. We tried to use different crate, different CAMAC controller and
different ADC module. All these did not solve the problem. We use polled method
to read data. We have dataway display unit so we know that it hung always after
it executed CAMAC command F9, which is after finishing one data taking and clear
the ADC for the next data taking. The data rate is about 1 KHz. It is random for
how long it takes for the system to hang.

Any ideas ?

Thanks,

Dawei Liu
    Reply  03 Jul 2009, Pierre-Andre Amaudruz, Forum, Data taking hangs in the middle of run 
Hi Dawei,

Could you give more info on your setup:
- CAMAC controller model
- ADC model
- LAM setting
- Mode of polling (on module or on CC)
- Are you still going through the poll_event() after hang up?
- Do you have the same problem at low rate (100Hz)?

Pierre-André
> Hi,
> 
> We are using midas to read ADC. It sometimes hung in the middle of data taking.
> We tried to disable analyzer and only run with frontend. The problem still
> exists. We tried to use different crate, different CAMAC controller and
> different ADC module. All these did not solve the problem. We use polled method
> to read data. We have dataway display unit so we know that it hung always after
> it executed CAMAC command F9, which is after finishing one data taking and clear
> the ADC for the next data taking. The data rate is about 1 KHz. It is random for
> how long it takes for the system to hang.
> 
> Any ideas ?
> 
> Thanks,
> 
> Dawei Liu
       Reply  06 Jul 2009, Dawei Liu, Forum, Data taking hangs in the middle of run 
Hi Pierr-Andre,


> Hi Dawei,
> 
> Could you give more info on your setup:
> - CAMAC controller model

Jorway 73A, we have three in hand and the problem doesn't depend on which controller
we were using.

> - ADC model

LeCroy 2249W. We also tried two other modules LeCroy 2249A. Same problem.

> - LAM setting

The poll and ADC reading codes are basically from Midas distribution.

> - Mode of polling (on module or on CC)

Polling on CC. I also tried to add a timeout code reading ADC, didn't solve the problem.

> - Are you still going through the poll_event() after hang up?

That's I don't know. I believe the problem happens between finishing reading one event
and passing the control back to poll_event.

> - Do you have the same problem at low rate (100Hz)?

The rate we are currently running is about 400 Hz, it has the same problem. We will
try lower rate more.

Thanks,

Dawei

> 
> Pierre-André
> > Hi,
> > 
> > We are using midas to read ADC. It sometimes hung in the middle of data taking.
> > We tried to disable analyzer and only run with frontend. The problem still
> > exists. We tried to use different crate, different CAMAC controller and
> > different ADC module. All these did not solve the problem. We use polled method
> > to read data. We have dataway display unit so we know that it hung always after
> > it executed CAMAC command F9, which is after finishing one data taking and clear
> > the ADC for the next data taking. The data rate is about 1 KHz. It is random for
> > how long it takes for the system to hang.
> > 
> > Any ideas ?
> > 
> > Thanks,
> > 
> > Dawei Liu
          Reply  08 Jul 2009, Konstantin Olchanski, Forum, jorway73a.c, Data taking hangs in the middle of run 
> > Could you give more info on your setup:
> > - CAMAC controller model
> Jorway 73A, we have three in hand and the problem doesn't depend on which controller
> we were using.

Dawei sent me a copy of his jorway73a.c scsi-camac driver. It is quite different from the
file in the MIDAS distribution. Dawei tells me that the file from the MIDAS distribution
does not compile. Stack traces from Dawei indicate a hang in this modified jorway73a.c
scsi-camac driver.

K.O.
             Reply  18 Jul 2009, Exaos Lee, Forum, jorway73a.c, Data taking hangs in the middle of run 
> > > Could you give more info on your setup:
> > > - CAMAC controller model
> > Jorway 73A, we have three in hand and the problem doesn't depend on which controller
> > we were using.
> 
> Dawei sent me a copy of his jorway73a.c scsi-camac driver. It is quite different from the
> file in the MIDAS distribution. Dawei tells me that the file from the MIDAS distribution
> does not compile. Stack traces from Dawei indicate a hang in this modified jorway73a.c
> scsi-camac driver.
> 
> K.O.

I encountered too that the jorway73a.c cannot work for my SCM-301 CAMAC driver. The
"jorway73a.c" distributed with MIDAS seems to work with Jorway 73a reversion > 300. But my
module has the reversion number 203. :-( I hope you can paste the modified version here so
that I can try it if I have spare time.
Regards.
Entry  07 May 2009, Konstantin Olchanski, Info, mhttpd now uses mtransition 
mhttpd function for starting and stopping runs now uses cm_transition(DETACH) which spawns an 
external helper program called mtransition to handle the transition sequencing. This helps with the old 
problem of looking at a blank screen for a long time if some frontends take a long time to process run 
transitions. Now mhttpd returns right back and shows start "starting run", "stopping run", etc as 
appropriate.

svn rev 4484 (some bits of this feature are present in rev 4473 and later).
K.O.

P.S. In one of our experiments, I sometimes see mhttpd getting "stuck" when starting or stopping a run 
using this feature. strace shows it is stuck in repeated calls to wait(), but I am unable to reproduce this 
problem in a test system and it happens only sometimes in the experiment. When it does, mhttpd has to 
be restarted. Replacing system("mtransition ...") with ss_sysem("mtransition ...") seems to fix this problem, 
but there are downsides to this (mtransition debug output vanishes) so I am not committing this yet.
K.O.
    Reply  21 May 2009, Konstantin Olchanski, Info, mhttpd now uses mtransition 
> mhttpd function for starting and stopping runs now uses cm_transition(DETACH) which spawns an 
> external helper program called mtransition to handle the transition sequencing.
> 
> P.S. In one of our experiments, I sometimes see mhttpd getting "stuck" when starting or stopping a run 
> using this feature. strace shows it is stuck in repeated calls to wait(), but I am unable to reproduce this 
> problem in a test system and it happens only sometimes in the experiment. When it does, mhttpd has to 
> be restarted. Replacing system("mtransition ...") with ss_sysem("mtransition ...") seems to fix this problem, 
> but there are downsides to this (mtransition debug output vanishes) so I am not committing this yet.
> K.O.

Found the problem. As observed on SL5 systems, the GLIBC "system()" function breaks if the user application
installs a SIGCHLD handler that "steals" wait() notifications. Such a handler is installed by the MIDAS ss_exec()
function in system.c.

I would count this as a GLIBC bug - their "system()" function should survive in the presence of non-default signal
handlers installed by the user, and in fact my copy of "man signal" talks about the "system()" doing something
special about SIGCHLD. Obviously whatever they do is broken, at least in the SL5 GLIBC.

I am now testing an implementation using MIDAS ss_spawnvp().

The simplest way to reproduce the problem: start mhttpd; start/stop runs - mtransition works perfectly; start some
program from the MIDAS "programs" page (this calls "ss_exec()"), try to start a run - mhttpd will hang inside the
system() GLIBC function, every time. mhttpd has to be killed with "kill -KILL" to recover.

K.O.
       Reply  02 Jun 2009, Konstantin Olchanski, Info, mhttpd now uses mtransition 
> > mhttpd function for starting and stopping runs now uses cm_transition(DETACH) which spawns an 
> > external helper program called mtransition to handle the transition sequencing.
>
> ... the GLIBC "system()" function breaks if the user application
> installs a SIGCHLD handler that "steals" wait() notifications. Such a handler is installed by the MIDAS ss_exec()
> function in system.c.
>
> I am now testing an implementation using MIDAS ss_spawnvp().

cm_transition() starting mtransition helper using ss_spawnvp() committed svn rev 4495.

K.O.
          Reply  26 Jun 2009, Konstantin Olchanski, Info, mhttpd now uses mtransition 
> > > mhttpd function for starting and stopping runs now uses cm_transition(DETACH) which spawns an 
> > > external helper program called mtransition to handle the transition sequencing.

Problem reported by Stefan - user presses the "stop the run" button, and the web page comes back saying "running" as if the button did not work. This is 
confusing. It happens because mtransition did not start yet - we have a race condition against it.

To improve this situation, mhttpd now remembers that a start/stop button was pushed and displays a message "Run start/stop requested" until it detects 
that mtransition started and set "runinfo/transition in progress" (or the run state changed).

svn rev 4520
K.O.
ELOG V3.1.4-2e1708b5