The reason to call analyzer_init in odb_load is the following:

Assume you run the analyzer offline, analyzing many files in series. Then assume
that you have /Experiment/Run Parameters, which is actively used by the analyzer
(like beam settings etc.). In this case you do a db_open_record() to map
/Experiment/Run Parameters to the exp_param C structure. For this mapping to work,
the ODB structure and the C structure have to be exactly the same. Now assume that
you changed your run parameters over time, like you added some comment later. Now
you want to analyzer several runs, some before and some after the modification.
Both sets have a different structure in /Experiment/Run Parameters, which is a
problem, since the compiled analyzer can only have a single C structure. My "poor"
solution was to call analyzer_init after each loading of the ODB from the *.mid
file. The db_create_record() call matches the C structure to the ODB structure by
modifying the ODB structure if necessary. So if you added one parameter later, this
(modified) structure gets loaded by odb_load, but then it gets adjusted in
analyzer_init().

I understand now that this case might not happen so often, and you are more
bothered by the fact that analyzer_init gets called several time. There must
however be a hook for offline analysis that the user code can correct the ODB
structure. So I propose to add a flag to analyzer_init, such as

INT analyzer_init(BOOL bFirst)
{
}

If bFirst equals TRUE, the function got called from mana_init(), if FALSE, it got
called from odb_load. Then you can put code like

INT analyzer_init(BOOL bFirst)
{
   if (bFirst) {
      p = malloc()
      ...
   }
}

If you agree, I will modify the code and commit the change.

- Stefan

Thanks for the quick reply, Stefan.

Please don't change anything in the code unless you find it really important. I guess 
changing the analyzer_init prototype will break a lot of code out there?

In fact, I think I do understand this behavior now.
And even without your suggested fix there is a simple workaround: I add a static 
variable to my analyzer_init.cxx file, and do something similar to your bFirst fix.

In conclusion, commit your fix if it does not harm others. Postpone this commit to a 
future new version of midas which breaks a lot of things anyway...

A last question, for me to understand: Why not call db_open_record in 
ana_begin_of_run then?

Cheers,

Randolf

> Thanks for the quick reply, Stefan.
> 
> Please don't change anything in the code unless you find it really important.
I guess 
> changing the analyzer_init prototype will break a lot of code out there?
> 
> In fact, I think I do understand this behavior now.
> And even without your suggested fix there is a simple workaround: I add a static 
> variable to my analyzer_init.cxx file, and do something similar to your bFirst
fix.
> 
> In conclusion, commit your fix if it does not harm others. Postpone this
commit to a 
> future new version of midas which breaks a lot of things anyway...
> 
> A last question, for me to understand: Why not call db_open_record in 
> ana_begin_of_run then?

I fully agree with you that db_open_record would better go into ana_begin_of_run
(and
analyzer_init not being called in odb_load), and I fully agree with you that
changing the
code would break many experiments. ;-)

So I guess we leave it as it is right now as you suggested.

I am having problems with handling the end-of-run situation in my midas
frontend. I have a device that continuously sends data (over USB) and I read
this data in my "read_event" function.

Everything is good until the end-of-run, at which time this happens:
0) mfe.c calls my read_event() to read the data (loop until the end-of-run
transition)
1) mfe.c calls my end_of_run()
2) here, I tell the device "please stop sending data"
3) all seems good, but wait!!!
4) there is all this data generated between step 0 and step 2 still sitting
inside the device and it has nowhere to go: the run is ended, the output file is
closed, my read_event() will never be called ever again (well, until the next run).

It seems to me mfe.c needs to have one more function, something like
"pre_end_of_run()" that works like this:
0) mfe.c calls my read_event() to read the data (loop until the end-of-run
transition)
1) mfe.c calls pre_end_of_run(), here I tell the device to stop sending data
2) mfe.c calls read_event() for the very last time, to give me the opportunity
to read and send away any data I still may have.
3) mfe.c calls the end_of_run(). The run is truely finished.

Any thoughts?

K.O.

> I am having problems with handling the end-of-run situation in my midas
> frontend. I have a device that continuously sends data (over USB) and I read
> this data in my "read_event" function.
> 
> Everything is good until the end-of-run, at which time this happens:
> 0) mfe.c calls my read_event() to read the data (loop until the end-of-run
> transition)
> 1) mfe.c calls my end_of_run()
> 2) here, I tell the device "please stop sending data"
> 3) all seems good, but wait!!!
> 4) there is all this data generated between step 0 and step 2 still sitting
> inside the device and it has nowhere to go: the run is ended, the output file is
> closed, my read_event() will never be called ever again (well, until the next run).
> 
> It seems to me mfe.c needs to have one more function, something like
> "pre_end_of_run()" that works like this:
> 0) mfe.c calls my read_event() to read the data (loop until the end-of-run
> transition)
> 1) mfe.c calls pre_end_of_run(), here I tell the device to stop sending data
> 2) mfe.c calls read_event() for the very last time, to give me the opportunity
> to read and send away any data I still may have.
> 3) mfe.c calls the end_of_run(). The run is truely finished.
> 
> Any thoughts?

You can achieve the desired functionality without changing mfe.c:

0) mfe.c calls read_event
1) mfe.c calls end_of_run. Your end_of_run tells the device to stop data and flushes
the remaining data. At this point you have to re-make actually a part of the mfe.c
functionality, but basically you need a bm_compose_event() and a bm_send_event(), so
just a few lines of code. If you want to have the final event number right in your
equipment, you also need to update eq->events_sent accordingly. 

Given the fact that 99% of the experiments do not need this functionality, I propose
that we keep mfe.c and you add the few lines of code into your user part of the
specific frontend.

Stefan

Hi,
    I found that midas banks can be given an extra 32 bits of zeros when
trying to keep to 32bit boundary on my x86_64. 

This can be fixed by changing (in midas.h)
#define ALIGN8(x)  (((x)+7) & ~7)
to
#define ALIGN8(x)  (((x)+3) & ~3)

Is there any bad consequences doing this?

> Hi,
>     I found that midas banks can be given an extra 32 bits of zeros when
> trying to keep to 32bit boundary on my x86_64. 
> 
> This can be fixed by changing (in midas.h)
> #define ALIGN8(x)  (((x)+7) & ~7)
> to
> #define ALIGN8(x)  (((x)+3) & ~3)
> 
> Is there any bad consequences doing this?

Yes. ALIGN8 means 'align to 8-byte boundary' (64-bit), and if you change that, you
break the code at various locations. Furthermore, 8-byte aligned access is faster
on x86_64 than 4-byte aligned access, so you will get a performance penalty. If
course if you have very many small banks, the zero padding can cause some
overhead, but in that case you could combine some data into a single bank.

Dear Stephan,

I am writting on behalf of the LiBeRACE collaboration
at Berkeley/Livermore.

We are trying to use midas (2.0.0) for our acquisition system.
However we had some difficulties to compile it on LINUX Fedora
Core 6 with gcc 4.1.1
I tried to trace back the problem and I found that _syscall0 in
system.c is actually an obsolete call (since gcc 4.x apparently).
Playing with assembly language being behond my competence, I would 
like to know if you ever came across this situation recently and
if you have any suggestion(s).

With my best regards
Julien GIBELIN


------------------------------------------------------
GIBELIN Julien

Lawrence Berkeley National Laboratory
Nuclear Science Division
One Cyclotron Rd.
MS 88R0192
BERKELEY, CA 94720-8101

Tel: +1 (510) 495-2695
Fax: +1 (510) 486-7983
------------------------------------------------------

> Dear Stephan,
> 
> I am writting on behalf of the LiBeRACE collaboration
> at Berkeley/Livermore.
> 
> We are trying to use midas (2.0.0) for our acquisition system.
> However we had some difficulties to compile it on LINUX Fedora
> Core 6 with gcc 4.1.1
> I tried to trace back the problem and I found that _syscall0 in
> system.c is actually an obsolete call (since gcc 4.x apparently).
> Playing with assembly language being behond my competence, I would 
> like to know if you ever came across this situation recently and
> if you have any suggestion(s).

The '_syscall0' function call was replaced by 'syscall' in SVN revision 3583. I
would recommend that you switch to the current SVN version (see
http://ladd00.triumf.ca/~daqweb/doc/midas/html/quickstart.html on how to obtain
the SVN version). If the problem still persists, please let us know.

- Stefan

I now encountered a new problem with mhttpd - I connect using the Safari 3.0.4 browser, go to the 
"Programs" page, press the button "Start feplc" (or any other "start" button) and instead of starting this 
program, I get an error in the browser, funny entries in ODB in "/Programs", corrupted ODB and a spew 
of messages in the midas log file about the mess. ODB has to be reloaded from backup to recover.

Investigation shows that the culprit is odd bahaviour of the "redirect()" function:

    /* start command */
    if (*getparam("Start")) {
       /* for NT: close reply socket before starting subprocess */
-      redirect2("?cmd=programs");
+      redirect2("/?cmd=programs");

The version without "/" makes Safari explode - it appends the "?cmd..." stuff to the existing URL, which 
already has the "?cmd..." tags, making a mess.

Firefox accepts either version.

ODB corruption happens here:
 
       sprintf(str, "/Programs/%s/Start command", name);
-      db_get_value(hDB, 0, str, command, &size, TID_STRING, TRUE);
+      db_get_value(hDB, 0, str, command, &size, TID_STRING, FALSE);
       if (command[0]) {
          ss_system(command);

It looks like db_get_value() would corrupt ODB if given funny "str". When Safari explodes,
funny strings are generated.

The simple fix is to replace "TRUE" with "FALSE", then at least db_get_value() does not try to make bogus 
entries in ODB.

The "Stop" command has the same problem, but does not currupt ODB - there is no db_get_value() in 
that code path.

I am reporting this "fresh" as I made one of our daq systems work again.

I did not investigate the history of changes to this "redirect" command (perhaps it was broken in the 
recent reorganisation of midas urls?), what versions of Safari work or not.

K.O.

It looks like ODB function extract_key() will overwrite the array pointed to by "key_name" if given an odb 
path with very long names (as seems to happen when redirection explodes in the Safari web browser, via 
db_get_value(TRUE) via mhttpd "start program" button). All  callers of this function seem to provide 256 
byte strings, so the problem would not show up in normal use - only when abnormal odb paths are being 
parsed. Proposed solution is to add a "length" argument to this function. (Actually ODB path elements 
should be restricted to NAME_LENGTH (32 bytes), right?). K.O.

Traceback (most recent call last):
  File "runtest.py", line 42, in <module>
    import midas
  File "/opt/MIDAS.PSI/Resources/PyMIDAS/pymidas/midas/__init__.py", line 140, in
<module>
    cmidas = ctypes.cdll.LoadLibrary('libmidas.so')
  File "/usr/lib/python2.5/site-packages/PIL/__init__.py", line 431, in LoadLibrary
    
  File "/usr/lib/python2.5/site-packages/PIL/__init__.py", line 348, in __init__
    
OSError: /opt/MIDAS.PSI/Versions/Current/lib/libmidas.so: undefined symbol:
cam16i_rq

Hi All,

I'm testing MIDAS SVN rev-4113 on Scientific Linux 5.1 (i386) and the Analyzer 
can't start as Daemon. What I mean "can't" is that it stops running 
immediately without leaving any error messages. However, it can run offline or 
without becoming a Daemon. I have tested with ROOT 5.14e/5.16/5.18 and 
the "Experiment" example coming with MIDAS and this problem always happens. 
Any ideas?

Best Regards,
Jimmy

>     /* start command */
>     if (*getparam("Start")) {
>        /* for NT: close reply socket before starting subprocess */
> -      redirect2("?cmd=programs");
> +      redirect2("/?cmd=programs");

The second version won't work if mhttpd is run under an Apache proxy. Assume the proxy redirects

http://proxy.ca/midas

to

http://daq.ca:8080

If you now do a redirect to "/?cmd=programs", you will end up at

http://proxy.ca/?cmd=programs

which is now what you want. I tried to put a "./?cmd=programs", and that bings you to

http://proxy.ca/midas/./?cmd=programs

which is correctly redirected to

http://daq.ca:8080/?cmd=programs

I tried with the windows version (ughhh) of Safari and it worked for me. So give it a try, the change is committed.

> ODB corruption happens here:
>  
>        sprintf(str, "/Programs/%s/Start command", name);
> -      db_get_value(hDB, 0, str, command, &size, TID_STRING, TRUE);
> +      db_get_value(hDB, 0, str, command, &size, TID_STRING, FALSE);
>        if (command[0]) {
>           ss_system(command);
> 
> It looks like db_get_value() would corrupt ODB if given funny "str". When Safari explodes,
> funny strings are generated.

What happes is an endless redirect from xxxx -> xxxx?cmd=Programs. So in the end you have

http://url.ca?cmd=programs?cmd=programs?cmd=programs?cmd=programs....

and in the end you get a stack overflow, which busts all.

> The simple fix is to replace "TRUE" with "FALSE", then at least db_get_value() does not try to make bogus 
> entries in ODB.

I changed both butting FALSE there and adding

   if (strchr(name, '?'))
      *strchr(name, '?') = 0;

which keeps the URL short.

So for me it looks fine at the moment, but I cannot guarantee that everything works, so keep an eye open on that.

> It looks like ODB function extract_key() will overwrite the array pointed to by "key_name" if given an odb 
> path with very long names (as seems to happen when redirection explodes in the Safari web browser, via 
> db_get_value(TRUE) via mhttpd "start program" button). All  callers of this function seem to provide 256 
> byte strings, so the problem would not show up in normal use - only when abnormal odb paths are being 
> parsed. Proposed solution is to add a "length" argument to this function. (Actually ODB path elements 
> should be restricted to NAME_LENGTH (32 bytes), right?). K.O.

This is fixed in svn revision 4129.

K.O.

Index: Makefile
===================================================================
--- Makefile    (revision 4129)
+++ Makefile    (working copy)
@@ -270,7 +270,7 @@

 OBJS =  $(LIB_DIR)/midas.o $(LIB_DIR)/system.o $(LIB_DIR)/mrpc.o \
        $(LIB_DIR)/odb.o $(LIB_DIR)/ybos.o $(LIB_DIR)/ftplib.o \
-       $(LIB_DIR)/mxml.o $(LIB_DIR)/cnaf_callback.o \
+       $(LIB_DIR)/mxml.o \
        $(LIB_DIR)/history.o $(LIB_DIR)/alarm.o $(LIB_DIR)/elog.o

 ifdef NEED_STRLCPY

--- Makefile    (revision 4129)
+++ Makefile    (working copy)
-       $(LIB_DIR)/mxml.o $(LIB_DIR)/cnaf_callback.o \
+       $(LIB_DIR)/mxml.o \
> i.e. remove cnaf_callback.o which causes the link errors.


Hi, Denis - I confirm that cnaf_callback.c is only used by MIDAS frontends that implement CAMAC
functions and that it should not required for building the MIDAS library. I am now looking at removing 
it from libmidas.

> I propose that libmidas.so is built by default, so when something breaks it won't go unnoticed

We have been through this before and decided that shared libraries are bad and we do not want to use 
them. The option for building libmidas.so was preserved, though.

Not to refight old wars, on reason against using shared libraries was version skew - one could never be 
sure what version of midas is being used - depending on the PATH, LD_LIBRARY_PATH, rpath settings, 
etc. There were other reasons, perhaps practical, perhaps with the mserver.

The main problem with "just build it", is that then the rest of midas will link against it bringing back all 
the problems we solved by going away from using shared libraries.

So back to your proposal about building libmidas.so - can you look and see if you can do the Python 
bindings with a statically linked midas library?

I know it is possible with Perl bindings - perl creates it's own shared library containing perl api glue 
linked against a foreign static library libfoo.a , so in theory, the shared library is not needed.

But perhaps, Python do things differently...

K.O.

From "history" pages, the "create elog" button stopped working - it takes us to the elog entry form, but 
then, the "submit" button does not create any elog entries, instead dumps us into an invalid history 
display. This is using the internal elog.

This change in mhttpd.c::show_elog_new() makes it work again:

-       ("<body><form method=\"POST\" action=\"./\" enctype=\"multipart/form-data\">\n");
+       ("<body><form method=\"POST\" action=\"/EL/\" enctype=\"multipart/form-data\">\n");

Problem and fix confirmed with Linux/firefox and MacOS/firefox and Safari.

K.O.

> From "history" pages, the "create elog" button stopped working - it takes us to the elog entry form, but 
> then, the "submit" button does not create any elog entries, instead dumps us into an invalid history 
> display. This is using the internal elog.
> 
> This change in mhttpd.c::show_elog_new() makes it work again:
> -       ("<body><form method=\"POST\" action=\"./\" enctype=\"multipart/form-data\">\n");
> +       ("<body><form method=\"POST\" action=\"/EL/\" enctype=\"multipart/form-data\">\n");

This was a problem with relative URLs and it is now fixed. Svn revision 4131, fixes: delete elog, make elog from odb, make elog from history.

K.O.

Having libmidas.so is absolutely necessary for pymidas to work. If there was no such
option in Makefile pymidas users would have to build it themselves.

What I proposed though is that you change Makefile so it builds libmidas.so in
addition to (not instead of) static library. So if someone prefer to build
binaries statically they
may continue to do so. On other hand when someone needs a shared library they won't 
discover that it can't be easily built.