> With the ALIGN8() change ODB works, mhttpd works. ALIGN8 change now commited to cvs, verified that "make all" builds 
> on Linux.

Verified that "make all" still works under Windows.

> ROOT stuff still blows up because of more namespace pollution (/usr/include/sys/something does #define Free(x) 
> free(blah...)). Arguably, it is not Apple's fault- portable programs should not include any <sys/foo.h> header files. I 
> think I can fix it by moving "#include <sys/mount.h>" from midasinc.h to system.h.

I would like to keep all OS specific #includes in midasinc.h. In worst case put another section there for OSX, like

in midas.h:

#if !defined(OS_MACOSX)
#if defined ( __????__ ) <- put the proper thing here
#define OS_MACOSX
#endif
#endif

then make a new seciton in midasinc.h

#ifdef OS_MACOSX
#include <...>
#endif

> Also figured out why PVM is defined- more pollution from "#include <sys/blah...>". This is only in mana.c and I will 
> repace every "#ifdef PVM" with "#ifdef HAVE_PVM". Is there documentation that should be updated as well? Alternatively I 
> can try to play games with header files...

Right, PVM should be replaced by HAVE_PVM. This is only for the analyzer. I planned at some point to run the analyzer in 
parallel on a linux cluster, but it was never really used. Going to ROOT, that facility should be replaces by PROOF.

> Then, a new problem- on MacOSX, pthread_t is not an "INT" and system.c:ss_thread_create() whines about it. I want to 
> introduce a system dependant THREAD_T (or whatever) and make ss_thread_create() return that, rather than INT.

Good. If you have a OS_MACOSX, that should help you there.

-SR

> The result is strange because the get function is called all the time very 
> fast (much faster then the 9 seconds as set in the equipment) and even 
> before starting the run (I just put the flag RO_RUNNING).

This is on purpose. When the frontend is idle, it loops over the slow control 
equipment as fast as possible. This way, you see changes in your hardware very 
quickly. I see no reason to waste CPU cycles in the frontend when there are 
better things to do like reading slow control equipment. Presume you have the 
alarm system running, which turns off some equipment in case of an over 
current. You better do this as quickly as possible, not wasting up to 9 
seconds each time.

The 9 seconds you mention are for reading *EVENTS*. You have double 
functionality: First, reading the slow control system, writing updated values 
to the ODB, where someone else can display or evaluate them (in the alarm 
system for example). Second, assemble events and sending them with the other 
data to disk or tape. Only the second one gets controlled by RO_RUNNING and 
the 9 seconds. You can see this by the updating event statists on your 
frontend display, which increments only when running and then every 9 seconds.

> I would like to keep all OS specific #includes in midasinc.h

No go. Here is the problem:

midasinc.h includes sys/mount.h, which #defines Free(x) to be something else
mana.c includes msystem.h, which includes midasinc.h
mana.c includes ROOT header files, which blow up because Free(x) is redefined.

I want this:

mana.c does *not* include sys/mount.h
system.c does include sys/mount.h

Simplest solution is to take sys/mount.h out of midasinc.h and include it in system.c

> Right, PVM should be replaced by HAVE_PVM.

Commited.

> > Then, a new problem- on MacOSX, pthread_t is not an "INT" and system.c:ss_thread_create() whines about it. I want to 
> > introduce a system dependant THREAD_T (or whatever) and make ss_thread_create() return that, rather than INT.
> Good. If you have a OS_MACOSX, that should help you there.

Okey. In Darwin, pthread_t is not an int. It is a pointer to a struct. In midas.c I typedef midas_pthread_t to HANDLE on Windows and to pthread_t n OS_UNIX.

This uncovered a problem with ss_getthandle(). What is it supposed to do? On Windows it returns a handle to the current thread, on OS_UNIX, it returns getpid(). 
What gives? I am leaving it alone for now.

Attached is the current diff. Most changes are in system.c: ss_timezone() and midas_pthread_t. The Makefile part is already commited. Building the shared 
library was made dependant on NEED_SHLIB. Now, building static midas applications is very simple, use "make SHLIB="

K.O.

> I want this:
> 
> mana.c does *not* include sys/mount.h
> system.c does include sys/mount.h
> 
> Simplest solution is to take sys/mount.h out of midasinc.h and include it in system.c

Agree.

> This uncovered a problem with ss_getthandle(). What is it supposed to do? On Windows it returns a handle to the current thread, on OS_UNIX, it returns getpid(). 
> What gives? I am leaving it alone for now.

The Unix version of ss_getthandle() returns the pid since at the time when I wrote that function (many years ago) there were no threads under Unix. It should now 
be replaces with a function which returns the real thread id (at least under Linux).

> > Simplest solution is to take sys/mount.h out of midasinc.h and include it in system.c
> Agree.

Done.

With this, I commited the rest of my changes: midas_thread_t in midas.h, change ss_thread_xxx() prototypes in msystem.h
, implementation in system.c

My cvs diff is now empty.

Midas should compile on Darwin aka macosx, I tested "odbedit" and "mhttpd"- they seem to work.
 
> > This uncovered a problem with ss_getthandle(). 
> The Unix version of ss_getthandle() returns the pid since at the time when I wrote that function (many years ago) there were no threads under Unix. It should now 
> be replaces with a function which returns the real thread id (at least under Linux).

I do not want to touch this. Sorry.

K.O.

I commited the final bits to make Midas build on Darwin aka macosx.

Here is the summary:

1) I treat Darwin as a funny linux, so OS_LINUX is always defined
2) OS_DARWIN is defined for places where the two differ
3) system dependant directory is "midas/darwin/{bin,lib}"
4) a few header files had to be moved around to dodge namespace pollution by Apple system 
header files (i.e. one of the PowerPC header files #defines PVM- collision with PVM in mana.c, 
another #defines Free(x)- collision with ROOT header files)
5) ss_thread_create() and ss_thread_kill() now use midas_thread_t. On Darwin ptherad_t is not 
an "int".
6) the Makefile has no support for building the midas shared library on macosx.
7) on my Mac OS 10.2.8 machine, "make all" works, "odbedit" and "mhttpd" run. This is the 
full extent of my testing. Status on Mac OS 10.3.x is unknown.

K.O.

Dear Midas Team,

I have run into a problem with Midas and was wondering if you could explain what I 
am doing wrong.  I have included a simple demo to illustrate what I am doing and 
can send a small input data file if needed.

WHAT I AM TRYING TO DO:
Every midas event for the DANCE experiment consists of many physics events.  I am 
trying to create a secondary mid file where the event boundaries are now the 
physics events rather than the midas events.  This secondary mid file will be 
analyzed using a second stage midas analyzer.

For the demo, I use the data from EV02 (one of our 15 frontends), which consists of a 
variable number of fixed length structures where each structure contains the data for 
one crystal from the DANCE detector. 
 I treat each crystal as a separate physics event and write it out in the TREK bank, 
which is a demo calculated output bank, as a separate event.   

(The only difference between this demo and our real system is that we would include 
all the crystals from the other frontends that have approximately the same time stamp 
in the output bank.  Thus the output bank would consist of a varing number of 
crystals in one event rather than the fixed one crystal per event used in this demo.)

THE CHANGES TO analyzer.c AND adccalib.c
I loop through the EV02 bank examining each crystal structure in turn.  I calculate 
"calibrated" parameters and put them into an output bank called TREK.  The unusual 
part of this example is that the TREK bank is no longer part of the main list of input 
banks, ana_trigger_bank_list[].   Instead it is now part of a new bank list called 
ana_physics_bank_list[].  See the analyzer.c file for this definition.

In adccalib.c I  create the space for this new bank as follows. 

	EVENT_HEADER 	gPhysicsEventHeaders[ MAX_EVENT_SIZE / sizeof( 
EVENT_HEADER ) ];  
	WORD* 		gPhysicsEventData = ( WORD * )( gPhysicsEventHeaders + 1 );		

In the adc_calib routine I create the bank header as follows.  Note that the serial 
numbers will restart at 0 at the beginning of each midas event.  Should I let the serial 
number increment monotonically until the end of the run?:

	gPhysicsEventHeaders->serial_number = (DWORD) - 1;
	gPhysicsEventHeaders->event_id = 2;
	gPhysicsEventHeaders->trigger_mask = 0;
	gPhysicsEventHeaders->time_stamp = pheader->time_stamp;

In a loop that loops through all the crystals contained in EV02,  I extract each crystal, 
calibrate it, and store it in a TREK structure.  In creating the TREK bank I assume that 
each one will be a separate physics event thus I update the event serial number and 
use bk_init32 to initialize the memory.   

   	for ( short i = 0; i < nItems; i++ )
  	{	++(gPhysicsEventHeaders->serial_number);  	// Update serial number.
  		bk_init32( gPhysicsEventData );		// Initialize storage.
  		bk_create( gPhysicsEventData, "TREK", TID_STRUCT, &trek );
  	
  		trek->one = (double) pev->areahg * 1.0;
  		trek->two = (float) pev->timelo * 1.0;

  		bk_close( gPhysicsEventData, trek+1 );
  		
  		pev++; 					// Loop to next crystal's data. 
	}	

The output bank should consist of multiple events for each individual EV02 midas 
input event. 

 As far as I can tell the code compiles and runs fine, but I get no data in the .mid 
output file except for the ODB. I have a print statement at the beginning of each 
midas event stating how many crystals were found in the EV02 bank.  I also print out 
the calibrated value for each crystal as it is being placed in its own TREK output 
bank.  The data appears correct.

 I cannot place TREK in the input bank the way it normally is done in the examples 
because there is not a one-to-one correspondence between a midas event and a 
true physics event.  Instead one midas event has many physics events.  Thus the 
output bank needs to be in a new memory area so that I can create a custom header 
and increment the serial number properly for each event.  Our follow-on analysis 
using a second Midas analyzer only needs to analyze one physics event at a time 
rather than one Midas event at a time, which is why we are going to all the trouble to 
get this paradigm working.

I include all the code for this very simple example. 

RUNNING THE CODE:
To run the example just use the run01220.mid file I will send:

./analyzer -i run01220.mid.gz -o run01220out.mid -c settings.odb_cfg -n 50

The only thing done by the settings.odb_cfg file is to turn on the TREK output bank.  I 
have verified that the bank is on.

SUMMARY:
I believe that I must not be creating the new TREK output bank correctly so that 
midas understands that the event-by-event calculated physics data should be written 
out event-by-event.  I have pointed out several places in the above discussion where 
I might be making a mistake.

I would like to get both this example running and a similar which create Root trees, 
though the Root trees are of secondary importance.  With this example I can finish 
writing the second stage analyzer and get the DANCE collaboration moving forward 
with their analysis.  Currently, we cannot use this paradigm because I cannot create 
a secondary mid file in our stage one analysis.  I would be very grateful if you could 
take a look at this example and tell me what I am doing incorrectly.

Jan

Dear Jan,

I had a look at your code. You create a gPhysicsEventHeader array, fill it, and expect the 
framework to write it to disk. But how can the framework "guess" that you want your private 
global array being written? Unfortunately it cannot do magic!

Do do what you want, you have to write a "secondary" midas file yourself. I modified your 
code to do that. First, I define the event storage like

BYTE           gSecEvent[ MAX_EVENT_SIZE ];
EVENT_HEADER   *gPhysicsEventHeader = (EVENT_HEADER *) gSecEvent;
WORD* 	       gPhysicsEventData = ( WORD * )( gPhysicsEventHeader + 1 );		

I use gSecEvent as a BYTE array, since it only contains one avent at a time, so this is more 
appropriate. Then, in the BOR routine, I open a file:

  sprintf(str, "sec%05d.mid", run_number);
  sec_fh = open(str, O_CREAT | O_RDWR | O_BINARY, 0644);

and close it in the EOR routine

  close(sec_fh);

The event routine now manually fills events into the secondary file:

      /* write event to secondary .mid file */
      gPhysicsEventHeader->data_size = bk_size(gPhysicsEventData);
      write(sec_fh, gPhysicsEventHeader, sizeof(EVENT_HEADER)+bk_size(gPhysicsEventData));

Note that this code is placed *inside* the for() loop over nItems, so for each detector you 
create and event and write it.

That's all you need, the full file adccalib.c is attached. I tried to produce a sec01220.mid 
file and was able to read it back with the mdump utility.

Best regards,

  Stefan

Jan , 

Do you need to log this stage 1 output?  If not, you would use the 
eventbuilder mechanism to create your stage 2 events.  
I use the eventbuilder mechanism with success for my TWIST experiment.

Renee

I am about to commit the mhttpd Elog fixes we have been using in TWIST since
about October. The infamous Elog "last N days" problem is fixed, sundry
memory overruns are caught and assert()ed.

For the curious, the "last N days" problem was caused by uninitialized data
in the elog handling code. A non-zero-terminated string was read from a file
and passed to atoi(). Here is a simplifed illustration:

char str[256]; // uninitialized, filled with whatever happens on the stack
read(file,str,6); // read 6 bytes, non-zero terminated
// str now looks like this: "123456UUUUUUUUU....", "U" is uninitialized memory
int len = atoi(str); // if the first "U" happens to be a number, we lose.

The obvious fix is to add "str[6]=0" before the atoi() call.

Attached is the CVS diff for the proposed changes. Please comment.

K.O.

Thanks for fixing these long lasting bugs. The code is much cleaner now, please
commit it.

I am setting up a new experiment and I added a "comment" field to "/
Experiment/Edit on start". When I start the run, I see this field, but I
cannot enter anything: the HTML "maxlength" is zero (or 1?). I traced this
to mhttpd.c: if (this is a string) maxlength = key.item_size. But what is
key.item_size for a string? The current length? If so, how do I enter a
string that is longer than the current one (zero in case I start from
scratch). I am stumped! K.O.

> I am setting up a new experiment and I added a "comment" field to "/
> Experiment/Edit on start". When I start the run, I see this field, but I
> cannot enter anything: the HTML "maxlength" is zero (or 1?). I traced this
> to mhttpd.c: if (this is a string) maxlength = key.item_size. But what is
> key.item_size for a string? The current length? If so, how do I enter a
> string that is longer than the current one (zero in case I start from
> scratch). I am stumped! K.O.

Your problem is that you created a ODB string with zero length. If you do this
through ODBEdit, a default length of 32 is used:

[local:Test:S]Edit on start>cr string Comment
String length [32]:
[local:Test:S]Edit on start>ls -l
Key name                        Type    #Val  Size  Last Opn Mode Value
---------------------------------------------------------------------------
Comment                         STRING  1     32    2s   0   RWD
[local:Test:S]Edit on start>

which then results in a maxlength of 32 as well during run start. I presume
you used mhttpd itself to create the string. Trying to reporduce this, I found
that mhttpd creates strings with zero length. I will fix this soon. Until
then, use ODBEdit to create your strings.

When I compile current cvs-head midas, I get errors about undefined function
min(). I do not think min() is in the list of standard C functions, so
something else should be used instead, like a MIN(a,b) macro. To make life
more interesting, in a few places, there is also a variable called "min".
Here is the error:

src/mana.c: In function `INT write_event_ascii(FILE*, EVENT_HEADER*, 
   ANALYZE_REQUEST*)':
src/mana.c:2571: `min' undeclared (first use this function)
src/mana.c:2571: (Each undeclared identifier is reported only once for each 
   function it appears in.)
make: *** [linux/lib/rmana.o] Error 1

K.O.

> When I compile current cvs-head midas, I get errors about undefined function
> min(). I do not think min() is in the list of standard C functions, so
> something else should be used instead, like a MIN(a,b) macro. To make life
> more interesting, in a few places, there is also a variable called "min".
> Here is the error:
> 
> src/mana.c: In function `INT write_event_ascii(FILE*, EVENT_HEADER*, 
>    ANALYZE_REQUEST*)':
> src/mana.c:2571: `min' undeclared (first use this function)
> src/mana.c:2571: (Each undeclared identifier is reported only once for each 
>    function it appears in.)
> make: *** [linux/lib/rmana.o] Error 1

This is really a miracle to me. The min/max macros are defined both in midas.h
and msystem.h and worked the last ten years or so. However, I agree that macros
should follow the standard and use capital letters, so I changed that.

I commited Makefile bits to set the RPATH on dynamically linked executables
to find libmidas.so and ROOT shared libraries without setting
LD_LIBRARY_PATH , etc. K.O.

> > When I compile current cvs-head midas, I get errors about undefined function
> > min(). I do not think min() is in the list of standard C functions, so
> > something else should be used instead, like a MIN(a,b) macro. To make life
> > more interesting, in a few places, there is also a variable called "min".
> > Here is the error:
> > 
> > src/mana.c: In function `INT write_event_ascii(FILE*, EVENT_HEADER*, 
> >    ANALYZE_REQUEST*)':
> > src/mana.c:2571: `min' undeclared (first use this function)
> > src/mana.c:2571: (Each undeclared identifier is reported only once for each 
> >    function it appears in.)
> > make: *** [linux/lib/rmana.o] Error 1
> 
> This is really a miracle to me. The min/max macros are defined both in midas.h
> and msystem.h and worked the last ten years or so. However, I agree that macros
> should follow the standard and use capital letters, so I changed that.

The problem is that /usr/include/c++/3.*/bits/stl_algobase.h contains 
#undef min
#undef max

and in C++ with STL one should really use something like this
    std::min<INT>(a,b)


Cheers
  Piotr

Is the midas CVS server set-up so that I can pull the newest 
version off the CVS server?

What would be my CVSROOT?
pserver:anoncvs@midas.psi.ch:/cvs/midas *this did not work* :)

Piotr

 

> Is the midas CVS server set-up so that I can pull the newest 
> version off the CVS server?
> 
> What would be my CVSROOT?
> pserver:anoncvs@midas.psi.ch:/cvs/midas *this did not work* :)
> 
> Piotr
> 
>  

In the Midas doc under "Quick Start"
http://midas.triumf.ca/doc/html/quickstart.html
you will find the proper cvs command for accessing the latest cvs Midas
version. The public pwd is cvs. You will only be able to checkout/update the
package.

I add the following to makefile and try to treat Darwin as FreeBSD/Linux.
But I failed.
============ 
#-----------------------
# This is for MacOS X
#
ifeq ($(OSTYPE), Darwin)
CC = gcc
OS_DIR = Darwin
OSFLAGS = -DOS_DARWIN -DOS_LINUX
LIBS = -lbsd -lcompat
SPECIFIC_OS_PRG =
endif
============

I got the following errors:
=============
gcc -c -g -O2 -Wall -Iinclude -Idrivers -LDarwin/lib -DINCLUDE_FTPLIB  
-DOS_DARWIN -DOS_FREEBSD -o Darwin/lib/midas.o src/midas.c
In file included from include/midasinc.h:45,
                 from include/msystem.h:114,
                 from src/midas.c:623:
/usr/include/string.h:112: error: conflicting types for `strlcat'
include/midas.h:1701: error: previous declaration of `strlcat'
/usr/include/string.h:113: error: conflicting types for `strlcpy'
include/midas.h:1700: error: previous declaration of `strlcpy'
In file included from include/msystem.h:114,
                 from src/midas.c:623:
include/midasinc.h:161:21: sys/vfs.h: No such file or directory
include/midasinc.h:164:17: pty.h: No such file or directory
src/midas.c:780: error: conflicting types for `dbg_malloc'
include/midas.h:1478: error: previous declaration of `dbg_malloc'
src/midas.c:817: error: conflicting types for `dbg_calloc'
include/midas.h:1479: error: previous declaration of `dbg_calloc'
src/midas.c:858: error: conflicting types for `strlcpy'
/usr/include/string.h:113: error: previous declaration of `strlcpy'
src/midas.c:892: error: conflicting types for `strlcat'
/usr/include/string.h:112: error: previous declaration of `strlcat'
gmake: *** [Darwin/lib/midas.o] Error 1
==========

Could anyone give me some hints. Thanks!