> > I commited the latest driver for the Wiener CCUSB USB-CAMAC driver. It
> > implements all functions from mcstd.h and has been tested to be plug-compatible
> > with at least one of our CAMAC frontends. K.O.

Well, it took almost a year to finish an updated driver, which has now been
commited to MIDAS SVN (see http://savannah.psi.ch/viewcvs/trunk/drivers/camac/ccusb/?root=midas).

This supports ccusb firmware release 0x402. With earlier firmware, simple CAMAC operations should work,
but to use the readout list feature one has to have the latest main firmware (0x402 as of today) and the latest CPLD
firmware.

The driver kit includes:
- the "ccusb" driver which implements the MIDAS mcstd.h CAMAC interface;
- test_ccusb to probe the interface and generally make the lights flash;
- ccusb_flash for updating the ccusb main firmware (assembled from bits and pieces found on the CCUSB driver CD);
- feccusb, an example midas frontend, which uses the ccusb readout list feature and has extensive error handling,
should be good enough for production use (unlike the Wiener libxxusb drivers, which lack basic error handling).
- analyzer.cxx, an rootana-based example on how to decode the ccusb data;
- README file with release notes.

If you use this driver, please drop me an email (even if it works perfectly for you, hah!) - the ccusb device is very
nice but can be hard to use and I would like to hear about problems other people have with it.

Today's version of the README files is attached below:

MIDAS driver for the Wiener/JTec CC-USB CAMAC-USB interface.

Date: 22-AUG-2007/KO

Note 1: The CC-USB interface comes with a CD which contains manuals,
firmware files, Windows and Linux software. The Wiener/JTec driver
is called "libxxusb". These MIDAS/musbstd drivers were written before
libxxusb bacame available and do not use libxxusb.

This driver implements the MIDAS CAMAC interafce "mcstd.h" using
the MIDAS USB interface musbstd.h.

Note 2: There exist many revisions of CCUSB firmware. Basic CAMAC
access works in all of them, but the "readout list" feature seems
to be only functional with firmware revision 0x402 or older and
with CPLD revisions CC_atmmgr_101406.jed, CC_datamgr_021905.jed,
CC_lammgr_brdcst_041906.jed or older.

To upgrade the main CCUSB firmware, follow instructions from
the CCUSB manual. On Linux, one can use the ccusb_flash
program included with these MIDAS drivers. It is a copy
of ccusb_flash from the Wiener CD, with all the pieces
assembled into one place and with a working Makefile. (I am too
lazy to add the flashing bits to the ccusb.c driver).

To upgrade the CPLD firmware, one needs a Xilinx JTag programmer
cable (we use a "parallel port to JTag" cable provided by Wiener),
and the Xilinx software (on Linux, we use Xilinx91i). For successful
upgrade, follow instructions from Xilinx and Wiener.

Note 3: Before starting to use the CCUSB interface, one should obtain
the latest version of the CCUSB manual and firmware by downloading
the latest version the CCUSB driver CD from the Wiener web
site (registration required)

Note 4: The example CCUSB frontend assumes this hardware configuration:
LeCroy 2249A 12 channel ADC in slot 20, Kinetic Systems 3615 6 channel
scaler in slot 12. NIM trigger input connected to CCUSB input "I1"
firing at 10-100 Hz. Without the external trigger CCUSB will not
generate any data and the frontend will only give "data timeout"
errors. With the trigger, the LED on the scaler should flash at 1 Hz
and the LEDs on the CCUSB should flash at the trigger rate.

Note 5: The CCUSB interface does not reliably power up in some CAMAC
crates (this has something to do with the sequence in which
different voltages start at different times with different CAMAC
power supplies). Some newer CCUSB modules may have this
problem fixed in the hardware and in the CPLD firmware. For modules
exhibiting this problem (i.e. no USB communication after power up),
try to cycle the power several time, or implement the "hardware reset
switch" (ask Wiener).

Note 6: The CCUSB firmware is very fickle and would crash if you look
at it the wrong way. This MIDAS driver tries to avoid all known crashers
and together with the example frontend, can recover from some
of them. Other crashes cannot be recovered from other than by
a hardware reset or power cycle.

//end

> While executing "make install" under MacOS 10.4.7, you may encounter errors about "dio". It is the 
> problem of "Makefile". I did some change to it and attach the diff file here.

Thank you very much for your instructions for installing Midas on MacOSX.
I followed your instructions to change the Makefile but I still get the following error message:

... 
... Installing programs and utilities to /usr/local/bin
... 
install: darwin/bin/lazylogger exists but is not a directory
install: darwin/bin/mchart exists but is not a directory
install: darwin/bin/mcnaf exists but is not a directory
install: darwin/bin/mdump exists but is not a directory
install: darwin/bin/melog exists but is not a directory
install: darwin/bin/mhdump exists but is not a directory
install: darwin/bin/mhist exists but is not a directory
install: darwin/bin/mhttpd exists but is not a directory
install: darwin/bin/mlogger exists but is not a directory
install: darwin/bin/mlxspeaker exists but is not a directory
install: darwin/bin/mserver exists but is not a directory
install: darwin/bin/mstat exists but is not a directory
install: darwin/bin/mtape exists but is not a directory
install: darwin/bin/odbedit exists but is not a directory
install: darwin/bin/odbhist exists but is not a directory
install: darwin/bin/stripchart.tcl exists but is not a directory
install: darwin/bin/webpaw exists but is not a directory
make: *** [install] Error 71

Could you help me solve this problem? Thank you in advance =)

I forgot to mention that, the following (and similar) lines:
           install -v -D -m 755 $$file $(SYSBIN_DIR)/`basename $$file` ; \
are changed into
           install -v -d -m 755 $$file $(SYSBIN_DIR)/`basename $$file` ; \

since -D is an illegal option for install. I am not sure whether -D in Linux means the same thing for -d in MacOSX install. 


> > While executing "make install" under MacOS 10.4.7, you may encounter errors about "dio". It is the 
> > problem of "Makefile". I did some change to it and attach the diff file here.
> 
> Thank you very much for your instructions for installing Midas on MacOSX.
> I followed your instructions to change the Makefile but I still get the following error message:
> 
> ... 
> ... Installing programs and utilities to /usr/local/bin
> ... 
> install: darwin/bin/lazylogger exists but is not a directory
> install: darwin/bin/mchart exists but is not a directory
> install: darwin/bin/mcnaf exists but is not a directory
> install: darwin/bin/mdump exists but is not a directory
> install: darwin/bin/melog exists but is not a directory
> install: darwin/bin/mhdump exists but is not a directory
> install: darwin/bin/mhist exists but is not a directory
> install: darwin/bin/mhttpd exists but is not a directory
> install: darwin/bin/mlogger exists but is not a directory
> install: darwin/bin/mlxspeaker exists but is not a directory
> install: darwin/bin/mserver exists but is not a directory
> install: darwin/bin/mstat exists but is not a directory
> install: darwin/bin/mtape exists but is not a directory
> install: darwin/bin/odbedit exists but is not a directory
> install: darwin/bin/odbhist exists but is not a directory
> install: darwin/bin/stripchart.tcl exists but is not a directory
> install: darwin/bin/webpaw exists but is not a directory
> make: *** [install] Error 71
> 
> Could you help me solve this problem? Thank you in advance =)

> I forgot to mention that, the following (and similar) lines:
>            install -v -D -m 755 $$file $(SYSBIN_DIR)/`basename $$file` ; \
> are changed into
>            install -v -d -m 755 $$file $(SYSBIN_DIR)/`basename $$file` ; \
> 
> since -D is an illegal option for install. I am not sure whether -D in Linux means the same thing for -d in MacOSX install. 

-D under linux means:

       -D     create all leading components of DEST except the last, then
              copy SOURCE to DEST; useful in the 1st format

This means if you install the first time, and eithe SYSBIN_DIR or `basename is not existing, it will be created on-the-fly from
the install program. If OSX does not support this, you somehow have to crate these subdirectories manually.

Thank you for your help =)

Since SYSBIN_DIR is defined as /usr/local/bin in the Makefile and it exists in my computer, so I deleted the -D in the Makefile and tried to "make install" again and the 
error message becomes:

... 
... Installing programs and utilities to /usr/local/bin
... 
/bin/sh: -c: line 2: syntax error: unexpected end of file
make: *** [install] Error 2

Can anyone help me solve this problem? 


> > I forgot to mention that, the following (and similar) lines:
> >            install -v -D -m 755 $$file $(SYSBIN_DIR)/`basename $$file` ; \
> > are changed into
> >            install -v -d -m 755 $$file $(SYSBIN_DIR)/`basename $$file` ; \
> > 
> > since -D is an illegal option for install. I am not sure whether -D in Linux means the same thing for -d in MacOSX install. 
> 
> -D under linux means:
> 
>        -D     create all leading components of DEST except the last, then
>               copy SOURCE to DEST; useful in the 1st format
> 
> This means if you install the first time, and eithe SYSBIN_DIR or `basename is not existing, it will be created on-the-fly from
> the install program. If OSX does not support this, you somehow have to crate these subdirectories manually.

Since the very beginning midas had the problem that the last event of a run was
sometimes missing in the data. While for most experiments this is not an issue,
it starts to hurt on experiments using event building (front-end 1 and front-end
2 in the example below). A missing event can screw up the event builder on the
next begin of run, where the "missing event" would show up as the first event of
the new run, triggering an event mismatch error in the event builder.

After some analysis, we identified the problem as follows. Assume FE1 controls
the trigger, while FE2 generates the second event fragment.

1) Stop is requested to FE1
2) tr_stop gets called on FE1
3) tr_stop calls end_of_run() use code
4) end_of_run() disables the trigger
5) FE1 finishes stop transition
6) Stop is requested to FE2
7) FE2 finishes stop transition

What can now happen is the following: An additional event occurs between 2) and
4). This event triggers ADCs and TDCs, and is then stored in the front-end
hardware. FE2 sees this event, since it has not yet done the stop transition,
and reads it out. FE1 is however already in the end_of_run() routine, and simply
disables the trigger, without reading this last event, and thus causing the
event mismatch at the beginning of the next run.

To fix the problem, the framework in mfe.c was changed:

1) Stop is requested to FE1
2) tr_stop gets called on FE1
3) tr_stop calls end_of_run() use code
4) end_of_run() disables the trigger
4b) tr_stop calls check_polled_events()
5) FE1 finishes stop transition
6) Stop is requested to FE2
7) FE2 finishes stop transition

The new routine check_polled_events checks if there is any more event in the
hardware by calling the user polling routine. If there is one more event, calls
the user readout routine and sends it to the back-end before concluding the run
transition. 

This modification solved our problem at the MEG experiment at PSI, but it might
be good that all experiments using event building update midas to revision 4225.

I do not expect any bad side effect, but one never knows. So if there are new
problems caused by this modification, please report.

- Stefan

It turns out that on 32-bit Linux, ss_file_size() returns the wrong answer for
files bigger than 2 GB (4GB?). The Linux stat() system call returns an error
(which is ignored) and bogus file size data (returned to the caller).

On 64-bit Linux (compiled with -m64), stat() appears to return correct data.

Related functions ss_disk_size() and ss_disk_free() return correct answers on
both 32-bit and 64-bit Linux (biggest disk I tried was 5.5 TB).

I now fixed this problem by using the stat64() system call for "#ifdef OS_LINUX".

I also changed ss_file_size(), ss_disk_size() and ss_disk_free() to return -1 if
the system call returns an error. I also added a test program
utils/test_ss_file_size.c.

svn revision 4397.
K.O.

> I also changed ss_file_size(), ss_disk_size() and ss_disk_free() to return -1 if
> the system call returns an error. I also added a test program
> utils/test_ss_file_size.c.

The test program gave under 64-bit SL5:

For [(null)], file size: -1, disk size: -0.001, disk free -0.001
sh: -c: line 0: syntax error near unexpected token `('
sh: -c: line 0: `/bin/ls -ld (null)'
sh: -c: line 0: syntax error near unexpected token `('
sh: -c: line 0: `/bin/df -k (null)'

Anyhow I guess that this test program just accidentally slipped into the repository.
Test programs for the developers should not be in the repository since they are of
not much use for the average user. If I would have added every test I made as an
individual test program, we would by now have tons of test programs making the whole
distribution pretty bulky, which nobody would know how to use now. So I removed the
test program again. If people do not agree, I suggest to make a central "main" test
program which combines all tests. I know there are also some C structure alignment
tests etc., which then could all be combined into a single, well documented, test
program.

> I now fixed this problem by using the stat64() system call for "#ifdef OS_LINUX".

That does not work if _LARGEFILE64_SOURCE is not defined. In that case, the compiler 
complains that stat64 is undefined. Since many Makefiles for front-ends out there do 
not have _LARGEFILE64_SOURCE defined, I changed system.c so that stat64 is only used 
if that flag is defined:

#ifdef _LARGEFILE64_SOURE
   struct stat64 stat_buf;
   int status;

   /* allocate buffer with file size */
   status = stat64(path, &stat_buf);
   if (status != 0)
      return -1;
   return (double) stat_buf.st_size;
#else
   ...

> > I now fixed this problem by using the stat64() system call for "#ifdef OS_LINUX".
> That does not work if _LARGEFILE64_SOURCE is not defined.
> #ifdef _LARGEFILE64_SOURE
>    struct stat64 stat_buf;

This does not work (observe the typoe in the #ifdef). But you cannot know this because
you already deleted the test program I wrote and committed to svn exactly to detect and
prevent this kind of breakage (+ plus to give the Solaris, BSD and other wierdo users
some way to check that ss_file_size() works on their systems).

K.O.

Committed minor bug fixes and improvements to mhttpd:
1) when generating history plots, use type "double" instead of "float" because "float" does not have enough 
significant digits to plot values of large integer numbers. For example, serial numbers of T2K FGD FEB 
cards are large integers, i.e. 99000001, 99000002, etc, but when we plot them with offset "-99000000", 
the plots show "0" for all cards because when these numbers are converted to "float", they are truncated to 
about 5 digits and the least significant digit (the only one of interest, the "1", "2", etc) is lost. Switching to 
type "double" makes the plots come out with correct values.
2) fixed breakage of "/History/URL" ODB setting used to offload generation of history plots to a separate 
mhttpd process, greatly improving responsiveness of the main mhttpd.
3) fixed memory leak in processing the new javascript requests (jset, jget & co).
svn revisions 4415-4417
K.O.

mhttpd did not like it when the equipment "Names" arrays had different length compared to the 
corresponding "Variables" arrays. These limitations are now removed.
svn rev 4469
K.O.

Fixed problems with mlogger starting and stopping runs.

Basic difficulty was with the mlogger using ASYNC transitions, which did not implement proper 
transition sequencing according to transition sequence numbers. Basically all clients were called at the 
same time, regardless of how long they took to process the transitions.

Switching from ASYNC to SYNC transitions introduces a deadlock between mlogger (not reading data 
from SYSTEM buffer while inside cm_transition) and any program trying to write into the SYSTEM buffer 
(buffer is full, does not listen for transition requests while waiting for mlogger which tries to call it's 
transition handler).

Then we invented the mtransition helper program. In the original implemtation for t2k it was spawned 
directly from the mlogger to stop the run (avoiding the deadlock). Then cm_transition(DETACHED) was 
introduced, but the mlogger start/stop/restart run logic became broken. One problem was with when 
auto restart delay is zero, mtransition tries to restart the run before previous run is stopped (instead, 
mlogger should restart the run from it's tr_stop() handler). Another problem was with the auto restart 
delay counting from the time when we start stopping the run - because stopping the run can take an 
unpredictable time, depending on when various frontends have to do - it is impossible to have a 
predictable delay between runs (again this is fixed by restarting the run from mlogger.c::tr_stop()).

All this has been straightened out by svn revision 4484. Basically the old run stop/restart logic was 
restored in mlogger.c, using cm_transition(DETACH) to avoid the deadlocks.

To remind all, these are the present controls for transitions initiated by mlogger:

/experiment/transition debug flag - set to "2" to capture transition sequences into midas.log
/experiment/transition timeout and transition connect timeout - one can change default timeouts as 
needed to accommodate non cooperative frontends.
/logger/async transitions - do not use mtransition - do ASYNC transitions, as before.
/logger/auto restart delay - delay between stopping the run (mlogger.c::tr_stop) and starting the next 
run.

svn rev 4484
K.O.

The odb hot-link function db_open_record() did not return an error when the system limit for hotlinks is 
exceeded and no more hot links could be added (silent failure). This is now fixed.
odb.c svn rev 4500
K.O.

odb.c rev 4622 fixes ODB corruption by db_connect_database() if client_name is
too long. Also fixed is potential ODB corruption by too long key names in
db_create_key(). Problem kindly reported by Tim Nichols of T2K/ND280 experiment.
K.O.

> odb.c rev 4622 fixes ODB corruption by db_connect_database() if client_name is
> too long. Also fixed is potential ODB corruption by too long key names in
> db_create_key(). Problem kindly reported by Tim Nichols of T2K/ND280 experiment.


Related bug fix - db_connect_database() should not permit client names that contain
the slash (/) character. Names like "aaa/bbb" create entries /Programs/aaa/bbb (aaa
is a subdirectory) and names like "../aaa" create entries in the ODB root directory.

svn rev 4623.
K.O.

Please be aware of mlogger.c update rev 4566 on Sept 23rd 2009, when Stefan
fixed a buglet in the subrun file size computations. Before this fix, the first
subrun could be of a short length. If you use subruns, please update your
mlogger to at least rev 4566 (or newer, Stefan added the run and subrun time
limits just recently).
K.O.