Back Midas Rome Roody Rootana
  Midas DAQ System, Page 67 of 143  Not logged in ELOG logo
New entries since:Wed Dec 31 16:00:00 1969
ID Date Authordown Topic Subject
  69   16 Jan 2004 Konstantin Olchanski First try- midas on darwin/macosx
> Great, I got already questions about MacOSX support...
> Once it's working, you should commit the changes.

With the ALIGN8() change ODB works, mhttpd works. ALIGN8 change now commited to cvs, verified that "make all" builds 
on Linux.

ROOT stuff still blows up because of more namespace pollution (/usr/include/sys/something does #define Free(x) 
free(blah...)). Arguably, it is not Apple's fault- portable programs should not include any <sys/foo.h> header files. I 
think I can fix it by moving "#include <sys/mount.h>" from midasinc.h to system.h.

Also figured out why PVM is defined- more pollution from "#include <sys/blah...>". This is only in mana.c and I will 
repace every "#ifdef PVM" with "#ifdef HAVE_PVM". Is there documentation that should be updated as well? Alternatively I 
can try to play games with header files...


> But take into account that using "//" for comments might cause problems for the VxWorks compiler (talk to Pierre 
about that!).

Yes, "// comments" stay out of midas. I used them to make the modification more visible.

> You can rename ALIGN to ALIGN8 all over the place.

Done, commited.

> > - "timezone" in mhttpd.c. On linux, it's an "int", on darwin, it's a function. What gives?
> Wrap it into a function get_timezone(). Under linux, just return "timezone", under OSX, 
> return timezone() via conditional compiling.

Right. Still on the todo list.

> > - building libmidas.a requires running ranlib

I still have to cleanup the Makefile. Not commiting it yet.

Then, a new problem- on MacOSX, pthread_t is not an "INT" and system.c:ss_thread_create() whines about it. I want to 
introduce a system dependant THREAD_T (or whatever) and make ss_thread_create() return that, rather than INT.

ROOT stuff is still not fully tested- it takes a little while to build ROOT on a 600MHz laptop.

Attached is my current CVS diff.

K.O.
Attachment 1: xxx
? .ALARM.SHM
? .ELOG.SHM
? .ODB.SHM
? .SYSMSG.SHM
? darwin
? midas.log
? xx
? xxx
Index: Makefile
===================================================================
RCS file: /usr/local/cvsroot/midas/Makefile,v
retrieving revision 1.50
diff -r1.50 Makefile
0a1
> 
218a220,224
> #
> # Uncomment the next line to build the midas shared library
> #
> NEED_SHLIB=1
> 
268a275,290
> # MacOSX/Darwin is just a funny Linux
> #
> ifeq ($(OSTYPE),Darwin)
> OSTYPE = darwin
> endif
> 
> ifeq ($(OSTYPE),darwin)
> OS_DIR = darwin
> OSFLAGS = -DOS_LINUX -DOS_DARWIN -DHAVE_STRLCPY -fPIC -Wno-unused-function
> LIBS = -lpthread
> SPECIFIC_OS_PRG = $(BIN_DIR)/mlxspeaker
> NEED_RANLIB=1
> NEED_SHLIB=
> endif
> 
> #-----------------------
340a363,364
> LIB    =$(LIBNAME)
> ifdef NEED_SHLIB
342,344c366,367
< LIB =   -lmidas
< # Uncomment this for static linking of midas executables
< #LIB =   $(LIBNAME)
---
> LIB   = $(SHLIB)
> endif
351c374
< 	$(LIB_DIR)/fal.o $(PROGS)
---
>  	$(LIB_DIR)/fal.o $(PROGS)
431a455,457
> ifdef NEED_RANLIB
> 	ranlib $@
> endif
432a459
> ifdef NEED_SHLIB
435a463
> endif
Index: include/midas.h
===================================================================
RCS file: /usr/local/cvsroot/midas/include/midas.h,v
retrieving revision 1.126
diff -r1.126 midas.h
464c464
< #if defined(OS_LINUX) || defined(OS_OSF1) || defined(OS_ULTRIX) || defined(OS_FREEBSD) || defined(OS_SOLARIS) || defined(OS_IRIX)
---
> #if defined(OS_LINUX) || defined(OS_OSF1) || defined(OS_ULTRIX) || defined(OS_FREEBSD) || defined(OS_SOLARIS) || defined(OS_IRIX) || defined(OS_DARWIN)
534a535,544
> #endif
> 
> /* need system-dependant thread type */
> #if defined(OS_WINNT)
> typedef HANDLE midas_thread_t;
> #elif defined(OS_UNIX)
> #include <pthread.h>
> typedef pthread_t midas_thread_t;
> #else
> typedef INT midas_thread_t;
Index: include/midasinc.h
===================================================================
RCS file: /usr/local/cvsroot/midas/include/midasinc.h,v
retrieving revision 1.11
diff -r1.11 midasinc.h
50a51
> #include <assert.h>
157d157
< #include <sys/mount.h>
163a164,165
> #ifdef OS_DARWIN
> #else
164a167
> #endif
166a170,172
> #ifdef OS_DARWIN
> #include <util.h>
> #else
167a174
> #endif
Index: include/msystem.h
===================================================================
RCS file: /usr/local/cvsroot/midas/include/msystem.h,v
retrieving revision 1.37
diff -r1.37 msystem.h
719,720c719,720
<    INT EXPRT ss_thread_create(INT(*func) (void *), void *param);
<    INT EXPRT ss_thread_kill(INT thread_id);
---
>    midas_thread_t EXPRT ss_thread_create(INT(*func) (void *), void *param);
>    INT EXPRT ss_thread_kill(midas_thread_t thread_id);
721a722
>    INT ss_timezone(void);
Index: src/mhttpd.c
===================================================================
RCS file: /usr/local/cvsroot/midas/src/mhttpd.c,v
retrieving revision 1.262
diff -r1.262 mhttpd.c
6983c6983
<    x_act = (int) floor((double) (xmin - timezone) / label_dx) * label_dx + timezone;
---
>    x_act = (int) floor((double) (xmin - ss_timezone()) / label_dx) * label_dx + ss_timezone();
6995,6996c6995,6996
<          if ((x_act - timezone) % major_dx == 0) {
<             if ((x_act - timezone) % label_dx == 0) {
---
>          if ((x_act - ss_timezone()) % major_dx == 0) {
>             if ((x_act - ss_timezone()) % label_dx == 0) {
Index: src/system.c
===================================================================
RCS file: /usr/local/cvsroot/midas/src/system.c,v
retrieving revision 1.78
diff -r1.78 system.c
306a307,310
> #ifdef OS_UNIX
> #include <sys/mount.h>
> #endif
> 
895c899
<     INT              thread handle
---
>     INT thread handle
914c918
<    return (int) hThread;
---
>    return hThread;
1653c1657
< thread_id = ss_thread_spawn((void *) taskWatch, &tsWatch);
---
> midas_thread_t thread_id = ss_thread_create((void *) taskWatch, &tsWatch);
1662c1666
< thread_id = ss_thread_spawn((void *) taskWatch, pDevice);
---
> midas_thread_t thread_id = ss_thread_create((void *) taskWatch, pDevice);
1673c1677
< INT ss_thread_create(INT(*thread_func) (void *), void *param)
---
> midas_thread_t ss_thread_create(INT(*thread_func) (void *), void *param)
1675c1679
< #ifdef OS_WINNT
---
> #if defined(OS_WINNT)
1689,1690c1693
< #endif                          /* OS_WINNT */
< #ifdef OS_MSDOS
---
> #elif defined(OS_MSDOS)
1694,1695c1697
< #endif                          /* OS_MSDOS */
< #ifdef OS_VMS
---
> #elif defined(OS_VMS)
1699c1701
< #endif                          /* OS_VMS */
---
> #elif defined(OS_VXWORKS)
1701d1702
< #ifdef OS_VXWORKS
1719d1719
< #endif                          /* OS_VXWORKS */
1721c1721,1722
< #ifdef OS_UNIX
---
> #elif defined(OS_UNIX)
> 
1728c1729,1730
< #endif                          /* OS_UNIX */
---
> 
> #endif
1738c1740
< thread_id = ss_thread_create((void *) taskWatch, pDevice);
---
> midas_thread_t thread_id = ss_thread_create((void *) taskWatch, pDevice);
1749c1751
< INT ss_thread_kill(INT thread_id)
---
> INT ss_thread_kill(midas_thread_t thread_id)
1751c1753
< #ifdef OS_WINNT
---
> #if defined(OS_WINNT)
1755c1757
<    status = TerminateThread((HANDLE) thread_id, 0);
---
>    status = TerminateThread(thread_id, 0);
1759,1760c1761
< #endif                          /* OS_WINNT */
< #ifdef OS_MSDOS
---
> #elif defined(OS_MSDOS)
1764,1765c1765
< #endif                          /* OS_MSDOS */
< #ifdef OS_VMS
---
> #elif defined(OS_VMS)
1769c1769
< #endif                          /* OS_VMS */
---
> #elif defined(OS_VXWORKS)
1771d1770
< #ifdef OS_VXWORKS
1773d1771
< 
1775d1772
< 
1777d1773
< #endif                          /* OS_VXWORKS */
1779,1782c1775
< #ifdef OS_UNIX
<    INT status;
< 
<    status = pthread_kill((pthread_t) thread_id, SIGKILL);
---
> #elif defined(OS_UNIX)
1783a1777,1778
>    INT status;
>    status = pthread_kill(thread_id, SIGKILL);
1785c1780,1781
< #endif                          /* OS_UNIX */
---
> 
> #endif
2339c2335
< #ifdef OS_WINNT
---
> #if defined(OS_WINNT)
2356,2357c2352,2358
< #endif
< #ifdef OS_UNIX
---
> #elif defined(OS_DARWIN)
> 
>    assert(!"ss_settime() is not supported");
>    /* not reached */
>    return SS_NO_DRIVER;
> 
> #elif defined(OS_UNIX)
2361,2362c2362
< #endif
< #ifdef OS_VXWORKS
---
> #elif defined(OS_VXWORKS)
2411a2412,2438
> INT ss_timezone()
> /********************************************************************\
> 
>   Routine: ss_timezone
> 
>   Purpose: Returns what?!?
> 
>   Input:
>     none
> 
>   Output:
>     what the heck does it return?!?
> 
>   Function value:
>     INT what is it?!?
> 
> \********************************************************************/
> {
> #ifdef OS_DARWIN
>   return 0;
> #else
>   return timezone; /* on Linux, comes from "#include <time.h>". What is it ?!? */
> #endif
> }
> 
> 
> /*------------------------------------------------------------------*/
4850c4877
<    INT status;
---
> #if defined(OS_DARWIN)
4852c4879,4883
< #ifdef OS_UNIX
---
>    return 0;
> 
... 14 more lines ...
  71   18 Jan 2004 Konstantin Olchanski First try- midas on darwin/macosx
> I would like to keep all OS specific #includes in midasinc.h

No go. Here is the problem:

midasinc.h includes sys/mount.h, which #defines Free(x) to be something else
mana.c includes msystem.h, which includes midasinc.h
mana.c includes ROOT header files, which blow up because Free(x) is redefined.

I want this:

mana.c does *not* include sys/mount.h
system.c does include sys/mount.h

Simplest solution is to take sys/mount.h out of midasinc.h and include it in system.c

> Right, PVM should be replaced by HAVE_PVM.

Commited.

> > Then, a new problem- on MacOSX, pthread_t is not an "INT" and system.c:ss_thread_create() whines about it. I want to 
> > introduce a system dependant THREAD_T (or whatever) and make ss_thread_create() return that, rather than INT.
> Good. If you have a OS_MACOSX, that should help you there.

Okey. In Darwin, pthread_t is not an int. It is a pointer to a struct. In midas.c I typedef midas_pthread_t to HANDLE on Windows and to pthread_t n OS_UNIX.

This uncovered a problem with ss_getthandle(). What is it supposed to do? On Windows it returns a handle to the current thread, on OS_UNIX, it returns getpid(). 
What gives? I am leaving it alone for now.

Attached is the current diff. Most changes are in system.c: ss_timezone() and midas_pthread_t. The Makefile part is already commited. Building the shared 
library was made dependant on NEED_SHLIB. Now, building static midas applications is very simple, use "make SHLIB="

K.O.
Attachment 1: xxx
? .ALARM.SHM
? .ELOG.SHM
? .ODB.SHM
? .SYSMSG.SHM
? darwin
? midas.log
? xx
? xxx
Index: Makefile
===================================================================
RCS file: /usr/local/cvsroot/midas/Makefile,v
retrieving revision 1.50
diff -r1.50 Makefile
0a1
> 
218a220,224
> #
> # Uncomment the next line to build the midas shared library
> #
> NEED_SHLIB=1
> 
268a275,290
> # MacOSX/Darwin is just a funny Linux
> #
> ifeq ($(OSTYPE),Darwin)
> OSTYPE = darwin
> endif
> 
> ifeq ($(OSTYPE),darwin)
> OS_DIR = darwin
> OSFLAGS = -DOS_LINUX -DOS_DARWIN -DHAVE_STRLCPY -fPIC -Wno-unused-function
> LIBS = -lpthread
> SPECIFIC_OS_PRG = $(BIN_DIR)/mlxspeaker
> NEED_RANLIB=1
> NEED_SHLIB=
> endif
> 
> #-----------------------
340a363,364
> LIB    =$(LIBNAME)
> ifdef NEED_SHLIB
342,344c366,367
< LIB =   -lmidas
< # Uncomment this for static linking of midas executables
< #LIB =   $(LIBNAME)
---
> LIB   = $(SHLIB)
> endif
351c374
< 	$(LIB_DIR)/fal.o $(PROGS)
---
>  	$(LIB_DIR)/fal.o $(PROGS)
431a455,457
> ifdef NEED_RANLIB
> 	ranlib $@
> endif
432a459
> ifdef NEED_SHLIB
435a463
> endif
Index: include/midas.h
===================================================================
RCS file: /usr/local/cvsroot/midas/include/midas.h,v
retrieving revision 1.126
diff -r1.126 midas.h
464c464
< #if defined(OS_LINUX) || defined(OS_OSF1) || defined(OS_ULTRIX) || defined(OS_FREEBSD) || defined(OS_SOLARIS) || defined(OS_IRIX)
---
> #if defined(OS_LINUX) || defined(OS_OSF1) || defined(OS_ULTRIX) || defined(OS_FREEBSD) || defined(OS_SOLARIS) || defined(OS_IRIX) || defined(OS_DARWIN)
534a535,544
> #endif
> 
> /* need system-dependant thread type */
> #if defined(OS_WINNT)
> typedef HANDLE midas_thread_t;
> #elif defined(OS_UNIX)
> #include <pthread.h>
> typedef pthread_t midas_thread_t;
> #else
> typedef INT midas_thread_t;
Index: include/midasinc.h
===================================================================
RCS file: /usr/local/cvsroot/midas/include/midasinc.h,v
retrieving revision 1.11
diff -r1.11 midasinc.h
50a51
> #include <assert.h>
157d157
< #include <sys/mount.h>
163a164,165
> #ifdef OS_DARWIN
> #else
164a167
> #endif
166a170,172
> #ifdef OS_DARWIN
> #include <util.h>
> #else
167a174
> #endif
Index: include/msystem.h
===================================================================
RCS file: /usr/local/cvsroot/midas/include/msystem.h,v
retrieving revision 1.37
diff -r1.37 msystem.h
719,720c719,720
<    INT EXPRT ss_thread_create(INT(*func) (void *), void *param);
<    INT EXPRT ss_thread_kill(INT thread_id);
---
>    midas_thread_t EXPRT ss_thread_create(INT(*func) (void *), void *param);
>    INT EXPRT ss_thread_kill(midas_thread_t thread_id);
721a722
>    INT ss_timezone(void);
Index: src/mhttpd.c
===================================================================
RCS file: /usr/local/cvsroot/midas/src/mhttpd.c,v
retrieving revision 1.262
diff -r1.262 mhttpd.c
6983c6983
<    x_act = (int) floor((double) (xmin - timezone) / label_dx) * label_dx + timezone;
---
>    x_act = (int) floor((double) (xmin - ss_timezone()) / label_dx) * label_dx + ss_timezone();
6995,6996c6995,6996
<          if ((x_act - timezone) % major_dx == 0) {
<             if ((x_act - timezone) % label_dx == 0) {
---
>          if ((x_act - ss_timezone()) % major_dx == 0) {
>             if ((x_act - ss_timezone()) % label_dx == 0) {
Index: src/system.c
===================================================================
RCS file: /usr/local/cvsroot/midas/src/system.c,v
retrieving revision 1.78
diff -r1.78 system.c
306a307,310
> #ifdef OS_UNIX
> #include <sys/mount.h>
> #endif
> 
895c899
<     INT              thread handle
---
>     INT thread handle
914c918
<    return (int) hThread;
---
>    return hThread;
1653c1657
< thread_id = ss_thread_spawn((void *) taskWatch, &tsWatch);
---
> midas_thread_t thread_id = ss_thread_create((void *) taskWatch, &tsWatch);
1662c1666
< thread_id = ss_thread_spawn((void *) taskWatch, pDevice);
---
> midas_thread_t thread_id = ss_thread_create((void *) taskWatch, pDevice);
1673c1677
< INT ss_thread_create(INT(*thread_func) (void *), void *param)
---
> midas_thread_t ss_thread_create(INT(*thread_func) (void *), void *param)
1675c1679
< #ifdef OS_WINNT
---
> #if defined(OS_WINNT)
1689,1690c1693
< #endif                          /* OS_WINNT */
< #ifdef OS_MSDOS
---
> #elif defined(OS_MSDOS)
1694,1695c1697
< #endif                          /* OS_MSDOS */
< #ifdef OS_VMS
---
> #elif defined(OS_VMS)
1699c1701
< #endif                          /* OS_VMS */
---
> #elif defined(OS_VXWORKS)
1701d1702
< #ifdef OS_VXWORKS
1719d1719
< #endif                          /* OS_VXWORKS */
1721c1721,1722
< #ifdef OS_UNIX
---
> #elif defined(OS_UNIX)
> 
1728c1729,1730
< #endif                          /* OS_UNIX */
---
> 
> #endif
1738c1740
< thread_id = ss_thread_create((void *) taskWatch, pDevice);
---
> midas_thread_t thread_id = ss_thread_create((void *) taskWatch, pDevice);
1749c1751
< INT ss_thread_kill(INT thread_id)
---
> INT ss_thread_kill(midas_thread_t thread_id)
1751c1753
< #ifdef OS_WINNT
---
> #if defined(OS_WINNT)
1755c1757
<    status = TerminateThread((HANDLE) thread_id, 0);
---
>    status = TerminateThread(thread_id, 0);
1759,1760c1761
< #endif                          /* OS_WINNT */
< #ifdef OS_MSDOS
---
> #elif defined(OS_MSDOS)
1764,1765c1765
< #endif                          /* OS_MSDOS */
< #ifdef OS_VMS
---
> #elif defined(OS_VMS)
1769c1769
< #endif                          /* OS_VMS */
---
> #elif defined(OS_VXWORKS)
1771d1770
< #ifdef OS_VXWORKS
1773d1771
< 
1775d1772
< 
1777d1773
< #endif                          /* OS_VXWORKS */
1779,1782c1775
< #ifdef OS_UNIX
<    INT status;
< 
<    status = pthread_kill((pthread_t) thread_id, SIGKILL);
---
> #elif defined(OS_UNIX)
1783a1777,1778
>    INT status;
>    status = pthread_kill(thread_id, SIGKILL);
1785c1780,1781
< #endif                          /* OS_UNIX */
---
> 
> #endif
2339c2335
< #ifdef OS_WINNT
---
> #if defined(OS_WINNT)
2356,2357c2352,2358
< #endif
< #ifdef OS_UNIX
---
> #elif defined(OS_DARWIN)
> 
>    assert(!"ss_settime() is not supported");
>    /* not reached */
>    return SS_NO_DRIVER;
> 
> #elif defined(OS_UNIX)
2361,2362c2362
< #endif
< #ifdef OS_VXWORKS
---
> #elif defined(OS_VXWORKS)
2411a2412,2438
> INT ss_timezone()
> /********************************************************************\
> 
>   Routine: ss_timezone
> 
>   Purpose: Returns what?!?
> 
>   Input:
>     none
> 
>   Output:
>     what the heck does it return?!?
> 
>   Function value:
>     INT what is it?!?
> 
> \********************************************************************/
> {
> #ifdef OS_DARWIN
>   return 0;
> #else
>   return timezone; /* on Linux, comes from "#include <time.h>". What is it ?!? */
> #endif
> }
> 
> 
> /*------------------------------------------------------------------*/
4850c4877
<    INT status;
---
> #if defined(OS_DARWIN)
4852c4879,4883
< #ifdef OS_UNIX
---
>    return 0;
> 
... 14 more lines ...
  73   19 Jan 2004 Konstantin Olchanski First try- midas on darwin/macosx
> > Simplest solution is to take sys/mount.h out of midasinc.h and include it in system.c
> Agree.

Done.

With this, I commited the rest of my changes: midas_thread_t in midas.h, change ss_thread_xxx() prototypes in msystem.h
, implementation in system.c

My cvs diff is now empty.

Midas should compile on Darwin aka macosx, I tested "odbedit" and "mhttpd"- they seem to work.
 
> > This uncovered a problem with ss_getthandle(). 
> The Unix version of ss_getthandle() returns the pid since at the time when I wrote that function (many years ago) there were no threads under Unix. It should now 
> be replaces with a function which returns the real thread id (at least under Linux).

I do not want to touch this. Sorry.

K.O.
  66   19 Jan 2004 Konstantin Olchanski darwin aka macosx changes
I commited the final bits to make Midas build on Darwin aka macosx.

Here is the summary:

1) I treat Darwin as a funny linux, so OS_LINUX is always defined
2) OS_DARWIN is defined for places where the two differ
3) system dependant directory is "midas/darwin/{bin,lib}"
4) a few header files had to be moved around to dodge namespace pollution by Apple system 
header files (i.e. one of the PowerPC header files #defines PVM- collision with PVM in mana.c, 
another #defines Free(x)- collision with ROOT header files)
5) ss_thread_create() and ss_thread_kill() now use midas_thread_t. On Darwin ptherad_t is not 
an "int".
6) the Makefile has no support for building the midas shared library on macosx.
7) on my Mac OS 10.2.8 machine, "make all" works, "odbedit" and "mhttpd" run. This is the 
full extent of my testing. Status on Mac OS 10.3.x is unknown.

K.O.
  64   30 Mar 2004 Konstantin Olchanski elog fixes
I am about to commit the mhttpd Elog fixes we have been using in TWIST since
about October. The infamous Elog "last N days" problem is fixed, sundry
memory overruns are caught and assert()ed.

For the curious, the "last N days" problem was caused by uninitialized data
in the elog handling code. A non-zero-terminated string was read from a file
and passed to atoi(). Here is a simplifed illustration:

char str[256]; // uninitialized, filled with whatever happens on the stack
read(file,str,6); // read 6 bytes, non-zero terminated
// str now looks like this: "123456UUUUUUUUU....", "U" is uninitialized memory
int len = atoi(str); // if the first "U" happens to be a number, we lose.

The obvious fix is to add "str[6]=0" before the atoi() call.

Attached is the CVS diff for the proposed changes. Please comment.

K.O.
Attachment 1: elog-fixes.txt
Index: src/midas.c
===================================================================
RCS file: /usr/local/cvsroot/midas/src/midas.c,v
retrieving revision 1.203
diff -u -r1.203 midas.c
--- src/midas.c	19 Mar 2004 09:58:22 -0000	1.203
+++ src/midas.c	31 Mar 2004 05:11:00 -0000
@@ -14814,8 +14814,9 @@
 \********************************************************************/
 
 /********************************************************************/
-void el_decode(char *message, char *key, char *result)
+void el_decode(char *message, char *key, char *result, int size)
 {
+   char *rstart = result;
    char *pc;
 
    if (result == NULL)
@@ -14828,6 +14829,8 @@
          *result++ = *pc++;
       *result = 0;
    }
+
+   assert(strlen(rstart) < size);
 }
 
 /**dox***************************************************************/
@@ -15020,9 +15023,9 @@
          size = atoi(str + 9);
          read(fh, message, size);
 
-         el_decode(message, "Date: ", date);
-         el_decode(message, "Thread: ", thread);
-         el_decode(message, "Attachment: ", attachment);
+         el_decode(message, "Date: ", date, sizeof(date));
+         el_decode(message, "Thread: ", thread, sizeof(thread));
+         el_decode(message, "Attachment: ", attachment, sizeof(attachment));
 
          /* buffer tail of logfile */
          lseek(fh, 0, SEEK_END);
@@ -15092,7 +15095,7 @@
       sprintf(message + strlen(message), "========================================\n");
       strcat(message, text);
 
-      assert(strlen(message) < sizeof(message));        // bomb out on array overrun.
+      assert(strlen(message) < sizeof(message)); /* bomb out on array overrun. */
 
       size = 0;
       sprintf(start_str, "$Start$: %6d\n", size);
@@ -15104,6 +15107,9 @@
          sprintf(tag, "%02d%02d%02d.%d", tms->tm_year % 100, tms->tm_mon + 1,
                  tms->tm_mday, (int) TELL(fh));
 
+      /* size has to fit in 6 digits */
+      assert(size < 999999);
+
       sprintf(start_str, "$Start$: %6d\n", size);
       sprintf(end_str, "$End$:   %6d\n\f", size);
 
@@ -15339,13 +15345,20 @@
          return EL_FILE_ERROR;
       }
 
-      if (strncmp(str, "$End$: ", 7) == 0) {
-         size = atoi(str + 7);
-         lseek(*fh, -size, SEEK_CUR);
-      } else {
+      if (strncmp(str, "$End$: ", 7) != 0) {
          close(*fh);
          return EL_FILE_ERROR;
       }
+        
+      /* make sure the input string to atoi() is zero-terminated:
+       * $End$:      355garbage
+       * 01234567890123456789 */
+      str[15] = 0;
+
+      size = atoi(str + 7);
+      assert(size > 15);
+
+      lseek(*fh, -size, SEEK_CUR);
 
       /* adjust tag */
       sprintf(strchr(tag, '.') + 1, "%d", (int) TELL(*fh));
@@ -15364,14 +15377,21 @@
       }
       lseek(*fh, -15, SEEK_CUR);
 
-      if (strncmp(str, "$Start$: ", 9) == 0) {
-         size = atoi(str + 9);
-         lseek(*fh, size, SEEK_CUR);
-      } else {
+      if (strncmp(str, "$Start$: ", 9) != 0) {
          close(*fh);
          return EL_FILE_ERROR;
       }
 
+      /* make sure the input string to atoi() is zero-terminated
+       * $Start$:    606garbage
+       * 01234567890123456789 */
+      str[15] = 0;
+
+      size = atoi(str+9);
+      assert(size > 15);
+
+      lseek(*fh, size, SEEK_CUR);
+
       /* if EOF, goto next day */
       i = read(*fh, str, 15);
       if (i < 15) {
@@ -15444,7 +15464,7 @@
 
 \********************************************************************/
 {
-   int size, fh, offset, search_status;
+   int size, fh = 0, offset, search_status, rd;
    char str[256], *p;
    char message[10000], thread[256], attachment_all[256];
 
@@ -15462,10 +15482,24 @@
 
    /* extract message size */
    offset = TELL(fh);
-   read(fh, str, 16);
-   size = atoi(str + 9);
+   rd = read(fh, str, 15);
+   assert(rd == 15);
+   
+   /* make sure the input string is zero-terminated before we call atoi() */
+   str[15] = 0;
+
+   /* get size */
+   size = atoi(str+9);
+   
+   assert(strncmp(str,"$Start$:",8) == 0);
+   assert(size > 15);
+   assert(size < sizeof(message));
+   
    memset(message, 0, sizeof(message));
-   read(fh, message, size);
+
+   rd = read(fh, message, size);
+   assert(rd > 0);
+   assert((rd+15 == size)||(rd == size));
 
    close(fh);
 
@@ -15473,14 +15507,14 @@
    if (strstr(message, "Run: ") && run)
       *run = atoi(strstr(message, "Run: ") + 5);
 
-   el_decode(message, "Date: ", date);
-   el_decode(message, "Thread: ", thread);
-   el_decode(message, "Author: ", author);
-   el_decode(message, "Type: ", type);
-   el_decode(message, "System: ", system);
-   el_decode(message, "Subject: ", subject);
-   el_decode(message, "Attachment: ", attachment_all);
-   el_decode(message, "Encoding: ", encoding);
+   el_decode(message, "Date: ",     date,     80); /* size from show_elog_submit_query() */
+   el_decode(message, "Thread: ", thread, sizeof(thread));
+   el_decode(message, "Author: ",   author,   80); /* size from show_elog_submit_query() */
+   el_decode(message, "Type: ",     type,     80); /* size from show_elog_submit_query() */
+   el_decode(message, "System: ",   system,   80); /* size from show_elog_submit_query() */
+   el_decode(message, "Subject: ",  subject, 256); /* size from show_elog_submit_query() */
+   el_decode(message, "Attachment: ", attachment_all, sizeof(attachment_all));
+   el_decode(message, "Encoding: ", encoding, 80); /* size from show_elog_submit_query() */
 
    /* break apart attachements */
    if (attachment1 && attachment2 && attachment3) {
@@ -15496,6 +15530,10 @@
                strcpy(attachment3, p);
          }
       }
+
+      assert(strlen(attachment1) < 256); /* size from show_elog_submit_query() */
+      assert(strlen(attachment2) < 256); /* size from show_elog_submit_query() */
+      assert(strlen(attachment3) < 256); /* size from show_elog_submit_query() */
    }
 
    /* conver thread in reply-to and reply-from */
  62   28 Apr 2004 Konstantin Olchanski mhttpd "start run" input field length?
I am setting up a new experiment and I added a "comment" field to "/
Experiment/Edit on start". When I start the run, I see this field, but I
cannot enter anything: the HTML "maxlength" is zero (or 1?). I traced this
to mhttpd.c: if (this is a string) maxlength = key.item_size. But what is
key.item_size for a string? The current length? If so, how do I enter a
string that is longer than the current one (zero in case I start from
scratch). I am stumped! K.O.
  58   07 May 2004 Konstantin Olchanski min(a,b) in mana.c and mlogger.c
When I compile current cvs-head midas, I get errors about undefined function
min(). I do not think min() is in the list of standard C functions, so
something else should be used instead, like a MIN(a,b) macro. To make life
more interesting, in a few places, there is also a variable called "min".
Here is the error:

src/mana.c: In function `INT write_event_ascii(FILE*, EVENT_HEADER*, 
   ANALYZE_REQUEST*)':
src/mana.c:2571: `min' undeclared (first use this function)
src/mana.c:2571: (Each undeclared identifier is reported only once for each 
   function it appears in.)
make: *** [linux/lib/rmana.o] Error 1

K.O.
  61   06 Jun 2004 Konstantin Olchanski Makefile: set -rpath
I commited Makefile bits to set the RPATH on dynamically linked executables
to find libmidas.so and ROOT shared libraries without setting
LD_LIBRARY_PATH , etc. K.O.
  52   22 Jun 2004 Konstantin Olchanski How to compile under Darwin-gcc? (MacOS X)
The current (cvs) version of MIDAS should build on Mac OS X right out of the
box- I fixed all the problems you report back in February(?)- see the macosx
thread in this forum. A few weeks ago I verified that it still compiles on Mac
OS 10.3.4. The Mac OS port received minimal testing- I checked that "odbedit"
and "mhttpd" run, that's about it. K.O.

> I add the following to makefile and try to treat Darwin as FreeBSD/Linux.
> But I failed.
> ============ 
> #-----------------------
> # This is for MacOS X
> #
> ifeq ($(OSTYPE), Darwin)
> CC = gcc
> OS_DIR = Darwin
> OSFLAGS = -DOS_DARWIN -DOS_LINUX
> LIBS = -lbsd -lcompat
> SPECIFIC_OS_PRG =
> endif
> ============
> 
> I got the following errors:
> =============
> gcc -c -g -O2 -Wall -Iinclude -Idrivers -LDarwin/lib -DINCLUDE_FTPLIB  
> -DOS_DARWIN -DOS_FREEBSD -o Darwin/lib/midas.o src/midas.c
> In file included from include/midasinc.h:45,
>                  from include/msystem.h:114,
>                  from src/midas.c:623:
> /usr/include/string.h:112: error: conflicting types for `strlcat'
> include/midas.h:1701: error: previous declaration of `strlcat'
> /usr/include/string.h:113: error: conflicting types for `strlcpy'
> include/midas.h:1700: error: previous declaration of `strlcpy'
> In file included from include/msystem.h:114,
>                  from src/midas.c:623:
> include/midasinc.h:161:21: sys/vfs.h: No such file or directory
> include/midasinc.h:164:17: pty.h: No such file or directory
> src/midas.c:780: error: conflicting types for `dbg_malloc'
> include/midas.h:1478: error: previous declaration of `dbg_malloc'
> src/midas.c:817: error: conflicting types for `dbg_calloc'
> include/midas.h:1479: error: previous declaration of `dbg_calloc'
> src/midas.c:858: error: conflicting types for `strlcpy'
> /usr/include/string.h:113: error: previous declaration of `strlcpy'
> src/midas.c:892: error: conflicting types for `strlcat'
> /usr/include/string.h:112: error: previous declaration of `strlcat'
> gmake: *** [Darwin/lib/midas.o] Error 1
> ==========
> 
> Could anyone give me some hints. Thanks!
  57   28 Jun 2004 Konstantin Olchanski Linking Error: g++ -rpath?
> ld: unknown flag: -rpath
> gmake: *** [darwin/bin/mlogger] Error 1

Fixed. Good catch.

> What does '-rpath' mean?

You will have to read the "ld" manual. In the nutshell, it tells the executable where to look for shared libraries. 
Aparently it is not supported by Mac OS X.

K.O.
  15   29 Jun 2004 Konstantin Olchanski FAQ: anonymous cvs access?
> Is the midas CVS server set-up so that I can pull the newest 
> version off the CVS server?
> 
> What would be my CVSROOT?
> pserver:anoncvs@midas.psi.ch:/cvs/midas *this did not work* :)

I use:

setenv CVS_RSH ssh
cvs -d cvs@midas.psi.ch:/usr/local/cvsroot checkout midas

It works most of the time. Sometimes I get locking and "permission denied"
errors on some subdirectories, presumably when CVS is being reorganized or
when there are stale locks. Does not happen too often.

K.O.
  48   14 Jul 2004 Konstantin Olchanski install problem of Makefile on MacOS X (Darwin 7.4.0, gcc 3.3)
> There are not such a file "io.h" inside my MacOS X. In fact, I didn't find any file containing function iopl().
> So what is the equivalent function of iopl() under MacOS X? The utility dio should be modified in order to be compiled under Darwin-
> gcc platform. 

"dio" is not supported under MacOSX. It is used to grant user programs access to PCI and ISA cards (usually CAMAC interfaces). We have
no MacOSX hardware with PCI or ISA slots so we cannot test and support this functionality.

The MacOSX Makefile should not try to build "dio". I will accept a patch to fix this Makefile bug.

K.O.
  27   15 Jul 2004 Konstantin Olchanski future direction discussion?
> > Are we ready for 2.0? 

I disapprove of version number inflation. Why not go straight for midas version
3000-Pro-Z?

> A big thing along the horizon I see is the ROME environment
> (http://midas.psi.ch/rome/). So we will move away from PAW to ROOT. Although
> the DAQ part will stay untouched, the whole analysis back-end changes,
> including some XML configuration and MySQL support.

I looked at the ROME slides from Pierre, and it seems to suffer badly from the
second-system syndrome (read The Mythical Man-Month).

For us, it is important to get the data into a form where we can process it with
ROOT and I would prefer if we could concentrate in improving the (embryonic) ROOT
online analysis capabilities.

> > 1) For one I would like to explore memory mapping (mmap())

It is trivial to replace System-V shared memory with mmap(). I am surprised that
it has not been done yet. System-V semaphores are a little bit harder to get rid of.

> > 2) Expanding hardware support: 
> >    a) custom microcontrolers?
> >    b) more hardware
> >    c) how about a "standard" Linux device /dev/midas
> >    for various PCI cards (PCI<->CAMAC) (PCI<->VME) 

We cannot expect MIDAS Authors to provide drivers for all possible hardware.

At best, we can mount an effort to collect exisiting drivers from all MIDAS users
"out there" and to integrate them into MIDAS.

Even that is highly non-trivial- many drivers use non-portable native hardware
access interfaces (direct bit-banging on PPCs, VMIC library on VMIC Linux
machines, etc). We have already failed to create an efficient generic portable VME
access library.

> As for a), there is the MSCB system (http://midas.psi.ch/mscb)

I never saw the point of having tons of MIDAS code for MSCB hardware that nobody
has and nobody will ever have.

> > 3) I have never really seen a midas deployment that uses interrupts. 
>
> It's not only the "ease" of polling, but also that it's faster (in almost all
> cases) and less troublesome. But hey, interrupt support is included in mfe.c,
> so if you are fanatic about interrupts...

Interrupts are important when one cannot afford chewing up CPU cycles, memory
cycles, PCI cycles and VME cycles on polling.

As I understand, common CAMAC hardware does not generate interrupts- this explains
lack of examples and lack of interrupt use. At TRIUMF, our new VME hardware
supports interrupts and I have a VMIC-based setup where I can (and intend to) test
MIDAS support of interrupts.

K.O.
  35   20 Jul 2004 Konstantin Olchanski HOWTO setup MIDAS ROOT tree analysis
Updating the instructions to ROOT version 3.10.2. Example is from TRIUMF-KOPIO
tree analysis.

shell> root -l
root> TFile *f = new TFile("run00064.root")
root> Trigger->MakeSelector("TriggerSelector")  // "Trigger" is the tree name
inside the root file. Generates TriggerSelector.h and TriggerSelector.cpp

= edit run.C, the main program:

{
gROOT->Reset();
TSelector *s = TSelector::GetSelector("TriggerSelector.C");

TChain chain("Trigger"); // "Trigger" is the tree name inside the root files
chain.Add("run03016.root"); // can chain multiple files

TH1D* tdc2 = new TH1D("tdc2","TDC2",1500,0,1500-1);

chain.Process(s,"",500); // process 500 events
//chain.Process(s); // or process all events

tdc2->Draw();
}

= edit TriggerSelector.h:

in the TriggerSelector class members, i.e. "UInt_t TDC1_TDC1[47];" edit the
array size to be bigger than the maximum possible bank size

= edit TriggerSelector.C:

Bool_t TriggerSelector::Process(Int_t entry)
{
  fChain->GetTree()->GetEntry(entry);

  if (entry%100 == 0)
    printf("process %d, nTDC %3d, 0x%08x\n",entry,TDC1_nTDC1,TDC1_TDC1[1]);

  tdc2->Fill(TDC1_nTDC1);
  return kTRUE;
}

= Run the analysis:

shell> root -l
root> .x run.C
 
K.O.
  33   20 Jul 2004 Konstantin Olchanski HOWTO setup MIDAS ROOT tree analysis
Updating the instructions to ROOT version 3.10.2. Example is from TRIUMF-KOPIO
tree analysis.

shell> root -l
root> TFile *f = new TFile("run00064.root")
root> Trigger->MakeSelector("TriggerSelector")  // "Trigger" is the tree name
inside the root file. Generates TriggerSelector.h and TriggerSelector.cpp

= edit run.C, the main program:

{
gROOT->Reset();
TSelector *s = TSelector::GetSelector("TriggerSelector.C");

TChain chain("Trigger"); // "Trigger" is the tree name inside the root files
chain.Add("run03016.root"); // can chain multiple files

TH1D* tdc2 = new TH1D("tdc2","TDC2",1500,0,1500-1);

chain.Process(s,"",500); // process 500 events
//chain.Process(s); // or process all events

tdc2->Draw();
}

= edit TriggerSelector.h:

in the TriggerSelector class members, i.e. "UInt_t TDC1_TDC1[47];" edit the
array size to be bigger than the maximum possible bank size

= edit TriggerSelector.C:

Bool_t TriggerSelector::Process(Int_t entry)
{
  fChain->GetTree()->GetEntry(entry);

  if (entry%100 == 0)
    printf("process %d, nTDC %3d, 0x%08x\n",entry,TDC1_nTDC1,TDC1_TDC1[1]);

  tdc2->Fill(TDC1_nTDC1);
  return kTRUE;
}

= Run the analysis:

shell> root -l
root> .x run.C
 
K.O.
  31   20 Jul 2004 Konstantin Olchanski HOWTO setup MIDAS ROOT tree analysis
Updating the instructions to ROOT version 3.10.2. Example is from TRIUMF-KOPIO
tree analysis.

shell> root -l
root> TFile *f = new TFile("run00064.root")
root> Trigger->MakeSelector("TriggerSelector")  // "Trigger" is the tree name
inside the root file. Generates TriggerSelector.h and TriggerSelector.cpp

= edit run.C, the main program:

{
gROOT->Reset();
TSelector *s = TSelector::GetSelector("TriggerSelector.C");

TChain chain("Trigger"); // "Trigger" is the tree name inside the root files
chain.Add("run03016.root"); // can chain multiple files

TH1D* tdc2 = new TH1D("tdc2","TDC2",1500,0,1500-1);

chain.Process(s,"",500); // process 500 events
//chain.Process(s); // or process all events

tdc2->Draw();
}

= edit TriggerSelector.h:

in the TriggerSelector class members, i.e. "UInt_t TDC1_TDC1[47];" edit the
array size to be bigger than the maximum possible bank size

= edit TriggerSelector.C:

Bool_t TriggerSelector::Process(Int_t entry)
{
  fChain->GetTree()->GetEntry(entry);

  if (entry%100 == 0)
    printf("process %d, nTDC %3d, 0x%08x\n",entry,TDC1_nTDC1,TDC1_TDC1[1]);

  tdc2->Fill(TDC1_nTDC1);
  return kTRUE;
}

= Run the analysis:

shell> root -l
root> .x run.C
 
K.O.
  28   20 Jul 2004 Konstantin Olchanski Introduction of environment variable MIDASSYS
> > Starting from midas version 1.9.4 on, the environment variable 'MIDASSYS' ...
> 2. What will the entire structure tree look like?
> 
> Here's my suggestion
> MIDASSYS=/opt/midas-1.9.4 (for example)   

Where should MIDAS be installed?

After looking at the LSB and at the FHS, it appears that the standards permit all of:
1) /opt/midas...
2) /usr/{bin,lib,...}
3) /usr/local/{bin,lib,...}

Some handy references:
http://www.pathname.com/fhs/pub/fhs-2.3.html
http://www.linuxbase.org/spec/

The "example LSB-compliant packages" appear to install into /opt/lsb, but I do not see
 any guidance as to where "my" packages should go.

Then, after some googling, I see that IBM "recommends" /opt (see
http://www-106.ibm.com/developerworks/linux/library/l-lsb.html):

begin-quote---
To avoid name space collisions when installing LSB-conforming applications, the
applications belonging to the base operating system or the distribution are to be
installed in /sbin/, /bin/, or /usr/. System administrators can build packages from
source and install them into the /usr/local/ directory. However, third-party packages
of add-on software must be installed in /opt/<package>/, where <package> is the name
that describes a software suite.
end-quote---


K.O.
  40   31 Aug 2004 Konstantin Olchanski midas odb locking
One of our experiments is suffering from periodic ODB corruption and I
suspected that there might be a problem with ODB locking. In the last few
days, I finally had time to read the ODB locking code, to write a little
test program and to play with ODB. This is what I found:

1) ODB locking appears to be sound, my test program failed to find any
locking flaws, except for a big problem, described below. Please read on.

2) ODB locking is "unfair". A program "while (1) { lock(); do_stuff();
unlock(); /* no sleep here */ }" would lock out other users of ODB
(including odbedit) for seconds and minutes at a time. I see this as a flaw
in the semop() implementation in the Linux kernel and I cannot think of an
easy way to fix it in our code. (I tested only on RHL9 2.4.20-31.9smp on a
dual CPU machine. 2.6 kernels may work better).

3) presently, we use an infinite timeout waiting for the ODB lock. I suggest
we set the timeout to, say, 5 minutes, to protect against dead (or live)
locks that we saw a few times here at TRIUMF- every ODB client would hang
without any error messages or explanations forever waiting for the ODB lock
that is held by some rogue ODB client stuck in an infinite loop in corrupted
ODB.

4) while reading the locking code in db_{lock,unlock}_database(), I thought
that there is a race condition against the "lock_cnt" variable, until I
realized that this variable is local and there is no race condition. I would
like to comment this in the code?

5) I found a failure mode where db_close_database() erroneously deletes the
lock semaphore. Once the semaphore is deleted, ODB locking silently fails
(in db_lock_database() we do not check for success status of
mutex_wait_for()) and remaining ODB clients operate without locking protection.

This failure happens after ODB undercounts active clients after losing track
of clients removed by "idle timeouts" (and by other checks?). At some point,
db_close_database() decides that there are no more clients left, attempts to
delete the shared memory (this fails because there are still active clients
attached) and deletes the lock semaphore. Afterwards, the remaining "lost"
active clients operate without lock protection. This would tend to happen
while shutting down all clients, a time when they all rush-in to delete
themselves from "/system/clients", unsuring ODB corruption.

A quick solution I just coded would not work for mmap()-based shared memory
(I destroy the lock semaphore after the ODB shared memory is destroyed) as
this relies on "shm_nattch" counting feature of System-V shared memories,
absent in the mmap() based shared memories. Since the Windows implementation
uses mmap(), my "solution" is an obvious no-go. Alternatives would be to add
a second semaphore, just for counting active ODB clients (kluncky); or never
delete the semaphore in the first place (dirty, and how does one clear it if
it gets stuck in the locked state?).

For now, I would like to add a check to ss_mutex_waitfor() call in
db_lock_database() and crash if we can't get the mutex. Returning an error
code would be cleaner, but would not work because nobody checks the return
status of db_lock_database(). If can't get the mutex for (say) 5 minutes, I
think we should crash, too- something is very wrong and it is pointless to
continue waiting.

K.O.
  43   31 Aug 2004 Konstantin Olchanski mlogger crash if using mserver.
Our users keep making a simple mistake- they set MIDAS_SERVER_HOST in their
environement. Most midas programs do not mind this- they go through the
mserver, inefficient but benign- except for the mlogger, which dumps core
about 10 seconds after starting. This mightily confuses the users-
everything works perfectly, except for the mlogger, (for most users) the
most obscure and magical part of midas. Obviously they can't take data
without the mlogger and they fail to correlate this crash with editing their
.cshrc file, so we get panic calls at midnight or whenever. And every time,
while debugging midas malfunctions, changes to .cshrc is absolutely the last
place we look for. Ouch!

As it turns out, mlogger does crash if it uses the mserver-
log_system_history() calls db_lock_database(), with a prompt crash because
the mlogger is not directly connected to any ODB (it's mserver is).

Obviously, running the mlogger via the mserver makes no sense, but we should
warn about this rather than dump core.

I propose this patch to src/mlogger.c::log_system_history():

-   db_lock_database(hDB);
-   db_notify_clients(hDB, hist_log[index].hKeyVar, FALSE);
-   db_unlock_database(hDB);
-
+   if (!rpc_is_remote())
+     {
+       db_lock_database(hDB);
+       db_notify_clients(hDB, hist_log[index].hKeyVar, FALSE);
+       db_unlock_database(hDB);
+     }
+   else
+     {
+       cm_msg(MERROR, "log_system_history", "Warning: mlogger is running
remotely via the mserver. This is an unsupported configuration. Please unset
MIDAS_SERVER_HOST and restart the mlogger");
+     }

K.O.
  45   15 Sep 2004 Konstantin Olchanski mlogger crash if using mserver.
> I trapped myself into that problem recently so it's the right time to fix it (;-).
> We have two options: 
> a) Make the logger work remotely, even if it's suboptimal and 
> b) Make the logger refuse to run remotely.

After some discussion between Stefan, Pierre and myself, it was decided to disallow
running mlogger remotely via the mserver.

K.O.
ELOG V3.1.4-2e1708b5