> In midas.c, ...
>
> 1) _net_send_buffer is not set to NULL when declared.

_net_send_buffer is a global variable. All global variables are automatically initialized to zero before the program 
starts.

static char*x; // = NULL; is redundant
char*y=realloc(x, 100);  // x is NULL, usage is correct

> 2) cm_disconect_experiment() calls free(_net_send_buffer) but does not set its 
> value to NULL.

My copy of midas.c (svn rev 5256) sets _net_send_buffer to NULL:

   if (_net_send_buffer_size > 0) { 
      M_FREE(_net_send_buffer); 
      _net_send_buffer_size = 0; 
   } 
 
What version of midas do you have? (svn info .)

K.O.

> > In midas.c, ...
> >
> > 1) _net_send_buffer is not set to NULL when declared.
> 
> _net_send_buffer is a global variable. All global variables are automatically 
initialized to zero before the program 
> starts.
> 
> static char*x; // = NULL; is redundant
> char*y=realloc(x, 100);  // x is NULL, usage is correct
>

Ah,okay. I was not aware of this feature of global variables.

 
> > 2) cm_disconect_experiment() calls free(_net_send_buffer) but does not set 
its 
> > value to NULL.
> 
> My copy of midas.c (svn rev 5256) sets _net_send_buffer to NULL:
> 
>    if (_net_send_buffer_size > 0) { 
>       M_FREE(_net_send_buffer); 
>       _net_send_buffer_size = 0; 
>    } 
>  
> What version of midas do you have? (svn info .)
> 
> K.O.

I have version 5256 also (matches what you posted), but I only see 
_net_send_buffer_size being set to 0, not _net_send_buffer itself. In midas.h, 
M_FREE(x) only expands to free(x) if _MEM_DBG is not defined.

> > > In midas.c, ...
> > >
> > > 1) _net_send_buffer is not set to NULL when declared.
> 
> Ah,okay. I was not aware of this feature of global variables.
> 

RTFM K&R "The C programming language". 
http://en.wikipedia.org/wiki/The_C_Programming_Language

>  
> > > 2) cm_disconect_experiment() calls free(_net_send_buffer) but does not set 
> its value to NULL.
>

Confirmed. Sorry for confusion in my previous message. Set the pointer to NULL after free() is good practice.

But note that calling cm_connect and cm_disconnect multiple times is unusual use of MIDAS and you will most 
likely find more breakage.

K.O.

> 2) cm_disconect_experiment() calls free(_net_send_buffer) but does not set its 
> value to NULL.

Set pointer to NULL after free() in these files:

M       odb.c
M       sequencer.cxx
M       mlogger.cxx
M       mhttpd.cxx
M       midas.c

svn rev 5294
K.O.

Dear guys,

I’m Zhi Li from China, and I’m now working on my graduation project, which now
basically gets stuck in the part of preparing the frontend for my FADC (CAEN
VX1721) using Midas.

Now the current set-up includes a VME crate, a CAEN v2718 (Optical Bridge and
Controller) and a CAEN VX1721(8ch 8bit 500MS/s Waveform digitizer). The hardware
set-up has been finished and I could capture the analog waveform using CAEN
software(wavedump). 

Could anyone please tell me what are the basic things to do for using MIDAS?
I’ve installed MIDAS in PC and it works well for CAMAC, but do I need any extra
hardware module on using VME crate? Also, how to download
Universe-II VME driver?

Thanks,
Li

Hi Li,

You mention that you've got the wavedump working. It suggests that you have a A3818 
interface, can you confirm that?

If so, you can make a Midas frontend using the CAEN libraries to access your VX1721. I can provide you with a frontend example used for the V1720 or V1740. The 
modifications for the VX1721 shouldn't be too hard as most of the CAEN digitizers 
are fortunately based on a similar configuration mechanism.
If you have a Midas CAMAC frontend, the trick would be to replace the CAMAC calls by 
the appropriate CAENComm_xxx() for the equivalent functionality.

Can you remind me what hardware do you have in your lab for acquisition?
CAMAC controller, VME controller etc.

Cheers, PAA

> Dear guys,
> 
> I’m Zhi Li from China, and I’m now working on my graduation project, which now
> basically gets stuck in the part of preparing the frontend for my FADC (CAEN
> VX1721) using Midas.
> 
> Now the current set-up includes a VME crate, a CAEN v2718 (Optical Bridge and
> Controller) and a CAEN VX1721(8ch 8bit 500MS/s Waveform digitizer). The hardware
> set-up has been finished and I could capture the analog waveform using CAEN
> software(wavedump). 
> 
> Could anyone please tell me what are the basic things to do for using MIDAS?
> I’ve installed MIDAS in PC and it works well for CAMAC, but do I need any extra
> hardware module on using VME crate? Also, how to download
> Universe-II VME driver?
> 
> Thanks,
> Li

Hi Pierre,

Thanks for your instructions. Before I run the wavedump software, I need to load a driver file for A2818, thus I think I've got this interface of A2818.

I would be grateful to have a look at the frontend example used for v1720 (closer to v1721 I suppose), would you be so kind to offer me the Makefile as well? I
really want to have a compilable/executable DAQ frontend for vme modules, and know better how to link to CAEN library in the Makefile.

About hardware currently used in the vme crate(A2818), there is a VME controller(V2718, CONET VME Bridge), and a FADC(VX1721 waveform digitizer). I'm now preparing
this DAQ system to compare relative quantum efficiency, timing resolution, 1 pe distribution of photomultipliers, also measure decay time of cosmic muons, and
electron spectrum. Humbly, I want to know your opinion on whether I need additional hardware to finish these experiments.

Thanks,
Li

> Hi Li,
> 
> You mention that you've got the wavedump working. It suggests that you have a A3818 
> interface, can you confirm that?
> 
> If so, you can make a Midas frontend using the CAEN libraries to access your VX1721. I can provide you with a frontend example used for the V1720 or V1740. The 
> modifications for the VX1721 shouldn't be too hard as most of the CAEN digitizers 
> are fortunately based on a similar configuration mechanism.
> If you have a Midas CAMAC frontend, the trick would be to replace the CAMAC calls by 
> the appropriate CAENComm_xxx() for the equivalent functionality.
> 
> Can you remind me what hardware do you have in your lab for acquisition?
> CAMAC controller, VME controller etc.
> 
> Cheers, PAA
> 
> > Dear guys,
> > 
> > I’m Zhi Li from China, and I’m now working on my graduation project, which now
> > basically gets stuck in the part of preparing the frontend for my FADC (CAEN
> > VX1721) using Midas.
> > 
> > Now the current set-up includes a VME crate, a CAEN v2718 (Optical Bridge and
> > Controller) and a CAEN VX1721(8ch 8bit 500MS/s Waveform digitizer). The hardware
> > set-up has been finished and I could capture the analog waveform using CAEN
> > software(wavedump). 
> > 
> > Could anyone please tell me what are the basic things to do for using MIDAS?
> > I’ve installed MIDAS in PC and it works well for CAMAC, but do I need any extra
> > hardware module on using VME crate? Also, how to download
> > Universe-II VME driver?
> > 
> > Thanks,
> > Li

Hi,

I had to apply the attached patch to convince SuSE Linux 12.2 to compile mana.c
gcc version is "(SUSE Linux) 4.6.2"

Problem is that gz{write,close, etc.} expect a 1st argument of type gzFile (see
zlib.h), whereas out_file is FILE*. In fact, out_file is a cast to FILE*, even
in the case when we work on a gzfile (HAVE_ZLIB).

Could you please confirm that the patch is correct, and possibly apply it to trunk?

I haven't checked if mana works as advertised now.

Cheers,


Randolf

> Hi,
> 
> I had to apply the attached patch to convince SuSE Linux 12.2 to compile mana.c
> gcc version is "(SUSE Linux) 4.6.2"
> 
> Problem is that gz{write,close, etc.} expect a 1st argument of type gzFile (see
> zlib.h), whereas out_file is FILE*. In fact, out_file is a cast to FILE*, even
> in the case when we work on a gzfile (HAVE_ZLIB).
> 
> Could you please confirm that the patch is correct, and possibly apply it to trunk?
> 
> I haven't checked if mana works as advertised now.
> 
> Cheers,
> 
> 
> Randolf

I applied your patch to the trunk.

Best,
Stefan

To reproduce:
- Open a history plot, click [-] a few times until the x axis shows more than 7 days.
- Scroll to the past (left)
- Click "Jump to current time" (the triangle)

Expected result:
The upper limit of the x axis is at the current time and the lower range is now - whatever range you had before 
(>7d)

Actual result:
The upper limit is the current time, the lower limit is now - 7d

(The interval seems unchanged if the range was < 7d before clicking "Jump to current time")

I had put in a check which limits the range to 7d into the past if you press the "play" button, but now I'm not sure why this was needed. I removed it again and 
things seem to be fine. Change is committed to develop.

Stefan

As of svn rev 4800, YBOS support was made optional, disabled by default. (But note that ybos.c is still used 
by mdump). See HAVE_YBOS in the Makefile.
K.O.

> As of svn rev 4800, YBOS support was made optional, disabled by default. (But note that ybos.c is still used 
> by mdump). See HAVE_YBOS in the Makefile.


It looks like some example drivers in .../drivers/class want to link against YBOS libraries. This fails because ybos.o is missing from the MIDAS library.

After discussions with SR and PAA, we think YBOS support can be removed or made optional, but there are too many of these drivers for me to fix 
them all right now in five minutes. Please accept my apology and use these workarounds:

If you get linker errors because of missing YBOS functions:

1) enable YBOS suport in the Makefile (uncomment HAVE_YBOS=1), or
2) "#ifdef HAVE_YBOS" all places that call YBOS functions

Solution (2) is preferable as it permits us to eventually remove YBOS completely. If you fix files from MIDAS svn, please do send me patches or diffs (or 
post them here).


K.O.

> It looks like some example drivers in .../drivers/class want to link against YBOS libraries. 
> This fails because ybos.o is missing from the MIDAS library.

I fixed the class drivers in meantime (SVN 4814).

There is however another problem: The lazylogger needs YBOS support compiled in if the FTP transfer mode is used. 
At PSI we are stuck at the moment to FTP, so we still need YBOS there (although none of the data is in YBOS format). 
Maybe there is a chance that this will be fixed some time and we can get rid of YBOS.

Hi,
with the recent fix of the CMakeLists.txt, it seems like another bug surfaced. 
In midas/progs/mlogger.cxx:48/49, the mysql header files are included without a 
prefix. However, mysql.h and mysqld_error.h are in a subdirectory, so for our 
systems, the lines should be
  48 #include <mysql/mysql.h>
  49 #include <mysql/mysqld_error.h>
This is the case with MariaDB 10.5.5 on OpenSuse Leap 15.2, MariaDB 10.5.5 on 
Fedora Workstation 34 and MySQL 5.5.60 on Raspbian 10.

If this problem occurs for other Linux/MySQL versions as well, it should be 
fixed in mlogger.cxx and midas/src/history_schema.cxx.
If this problem only occurs on some distributions or MySQL versions, it needs 
some more differentiation than #ifdef OS_UNIX. 

Also, this somehow seems familiar, wasn't there such a problem in the past?

Hi,

> with the recent fix of the CMakeLists.txt, it seems like another bug 
surfaced. 
> In midas/progs/mlogger.cxx:48/49, the mysql header files are included without 
a 
> prefix. However, mysql.h and mysqld_error.h are in a subdirectory, so for our 
> systems, the lines should be
>   48 #include <mysql/mysql.h>
>   49 #include <mysql/mysqld_error.h>
> This is the case with MariaDB 10.5.5 on OpenSuse Leap 15.2, MariaDB 10.5.5 on 
> Fedora Workstation 34 and MySQL 5.5.60 on Raspbian 10.
> 
> If this problem occurs for other Linux/MySQL versions as well, it should be 
> fixed in mlogger.cxx and midas/src/history_schema.cxx.
> If this problem only occurs on some distributions or MySQL versions, it needs 
> some more differentiation than #ifdef OS_UNIX.

What does "mariadb_config --cflags" or "mysql_config --cflags" return on 
these systems? For mariadb 10.3.27 on Debian 10 it returns both paths:

% mariadb_config --cflags
-I/usr/include/mariadb -I/usr/include/mariadb/mysql

Note also that mysql.h and mysqld_error.h reside in /usr/include/mariadb *not* 
/usr/include/mariadb/mysql so using "#include <mysql/mysql.h>" would not work.

On CentOS 7 with mariadb  5.5.68:

%  mysql_config --include
-I/usr/include/mysql
% ls -l /usr/include/mysql/mysql*.h
-rw-r--r--. 1 root root 38516 May  6  2020 /usr/include/mysql/mysql.h
-r--r--r--. 1 root root 76949 Oct  2  2020 /usr/include/mysql/mysqld_ername.h
-r--r--r--. 1 root root 28805 Oct  2  2020 /usr/include/mysql/mysqld_error.h
-rw-r--r--. 1 root root 24717 May  6  2020 /usr/include/mysql/mysql_com.h
-rw-r--r--. 1 root root  1167 May  6  2020 /usr/include/mysql/mysql_embed.h
-rw-r--r--. 1 root root  2143 May  6  2020 /usr/include/mysql/mysql_time.h
-r--r--r--. 1 root root   938 Oct  2  2020 /usr/include/mysql/mysql_version.h

So this seems to be the correct setup for both Debian and RHEL. If this is to 
be worked around in Midas I would think it would be better to do it at the 
cmake level than by putting another #ifdef in the code.

Cheers,

Nick.

> % mariadb_config --cflags
> -I/usr/include/mariadb -I/usr/include/mariadb/mysql

I get similar, both .../include and .../include/mysql are in my include path,
so both #include "mysql/mysql.h" and #include "mysql.h" work.

I added a message to cmake to report the MySQL CFLAGS and libraries, so next time
this is a problem, we can see what happened from the cmake output:

4ed0:midas olchansk$ make cmake | grep MySQL
...
-- MIDAS: Found MySQL version 10.4.16
-- MIDAS: MySQL CFLAGS: -I/opt/local/include/mariadb-10.4/mysql;-I/opt/local/include/mariadb-
10.4/mysql/mysql and libs: -L/opt/local/lib/mariadb-10.4/mysql/ -lmariadb

K.O.

Dear all,

we creating Midas events directly inside a FPGA and send them off via DMA into the PC RAM. For reading out this RAM via Midas the FPGA sends as a pointer where it has written the last 4kB of data. We use this pointer for telling the ring buffer of midas where the new events are. The buffer looks something like:

// event 1
dma_buf[0] = 0x00000001; // Trigger and Event ID
dma_buf[1] = 0x00000001; // Serial number
dma_buf[2] = TIME; // time
dma_buf[3] = 18*4-4*4; // event size
dma_buf[4] = 18*4-6*4; // all bank size
dma_buf[5] = 0x11; // flags
// bank 0
dma_buf[6] = 0x46454230; // bank name
dma_buf[7] = 0x6; // bank type TID_DWORD
dma_buf[8] = 0x3*4; // data size
dma_buf[9] = 0xAFFEAFFE; // data
dma_buf[10] = 0xAFFEAFFE; // data
dma_buf[11] = 0xAFFEAFFE; // data
// bank 1
dma_buf[12] = 0x1; // bank name
dma_buf[12] = 0x46454231; // bank name
dma_buf[13] = 0x6; // bank type TID_DWORD
dma_buf[14] = 0x3*4; // data size
dma_buf[15] = 0xAFFEAFFE; // data
dma_buf[16] = 0xAFFEAFFE; // data
dma_buf[17] = 0xAFFEAFFE; // data

// event 2
.....

dma_buf[fpga_pointer] = 0xXXXXXXXX;


And we do something like:

while{true}
   // obtain buffer space
   status = rb_get_wp(rbh, (void **)&pdata, 10);
   fpga_pointer = fpga.read_last_data_add();

   wlen = last_fpga_pointer - fpga_pointer; \\ in 32 bit words
   copy_n(&dma_buf[last_fpga_pointer], wlen, pdata);
   rb_status = rb_increment_wp(rbh, wlen * 4); \\ in byte

   last_fpga_pointer = fpga_pointer;

Leaving the case out where the dma_buf wrap around this works fine for a small data rate. But if we increase the rate the fpga_pointer also increases really fast and wlen gets quite big. Actually it gets bigger then max_event_size which is checked in rb_increment_wp leading to an error. 

The problem now is that the event size is actually not to big but since we have multi events in the buffer which are read by midas in one step. So we think in this case the function rb_increment_wp is comparing actually the wrong thing. Also increasing the max_event_size does not help.

Remark: dma_buf is volatile so memcpy is not possible here.

Cheers,
Marius

The rb_xxx functions are (thoroughly tested) robust against high data rate given that you use them as intended:

1) Once you create the ring buffer via rb_create(), specify the maximum event size (midas event, not bank size!). Later there is not protection any more, so if you obtain pdata from rb_get_wp, you can of course write 4GB to pdata, overwriting everything in your memory, causing a fatal crash. It's your duty not to write more bytes into pdata then what you specified in rb_create()

2) Once you obtain a write pointer to the ring buffer via rb_get_wp(), this function might fail when the receiving side reads data slower than the producing side, simply because the buffer is full. In that case the producing side was to wait to get new buffer space. If you call to rb_get_wp() returns DB_TIMEOUT, it means that the function did not obtain enough free space for the next event. In that case you have to wait (like ss_sleep(10)) and try again. Only when rb_get_wp() returns DB_SUCCESS, you are allowed to write into pdata (up to the maximum event size specified in rb_create() of course).

/Stefan


> Dear all,
> 
> we creating Midas events directly inside a FPGA and send them off via DMA into the PC RAM. For reading out this RAM via Midas the FPGA sends as a pointer where it has written the last 4kB of data. We use this pointer for telling the ring buffer of midas where the new events are. The buffer looks something like:
> 
> // event 1
> dma_buf[0] = 0x00000001; // Trigger and Event ID
> dma_buf[1] = 0x00000001; // Serial number
> dma_buf[2] = TIME; // time
> dma_buf[3] = 18*4-4*4; // event size
> dma_buf[4] = 18*4-6*4; // all bank size
> dma_buf[5] = 0x11; // flags
> // bank 0
> dma_buf[6] = 0x46454230; // bank name
> dma_buf[7] = 0x6; // bank type TID_DWORD
> dma_buf[8] = 0x3*4; // data size
> dma_buf[9] = 0xAFFEAFFE; // data
> dma_buf[10] = 0xAFFEAFFE; // data
> dma_buf[11] = 0xAFFEAFFE; // data
> // bank 1
> dma_buf[12] = 0x1; // bank name
> dma_buf[12] = 0x46454231; // bank name
> dma_buf[13] = 0x6; // bank type TID_DWORD
> dma_buf[14] = 0x3*4; // data size
> dma_buf[15] = 0xAFFEAFFE; // data
> dma_buf[16] = 0xAFFEAFFE; // data
> dma_buf[17] = 0xAFFEAFFE; // data
> 
> // event 2
> .....
> 
> dma_buf[fpga_pointer] = 0xXXXXXXXX;
> 
> 
> And we do something like:
> 
> while{true}
>    // obtain buffer space
>    status = rb_get_wp(rbh, (void **)&pdata, 10);
>    fpga_pointer = fpga.read_last_data_add();
> 
>    wlen = last_fpga_pointer - fpga_pointer; \\ in 32 bit words
>    copy_n(&dma_buf[last_fpga_pointer], wlen, pdata);
>    rb_status = rb_increment_wp(rbh, wlen * 4); \\ in byte
> 
>    last_fpga_pointer = fpga_pointer;
> 
> Leaving the case out where the dma_buf wrap around this works fine for a small data rate. But if we increase the rate the fpga_pointer also increases really fast and wlen gets quite big. Actually it gets bigger then max_event_size which is checked in rb_increment_wp leading to an error. 
> 
> The problem now is that the event size is actually not to big but since we have multi events in the buffer which are read by midas in one step. So we think in this case the function rb_increment_wp is comparing actually the wrong thing. Also increasing the max_event_size does not help.
> 
> Remark: dma_buf is volatile so memcpy is not possible here.
> 
> Cheers,
> Marius