> Done to all in-tree programs, except for mana.c (not using it), sequencer.cxx (cannot test it) and a few places where watching a TID_INT.
> Nothing more needs to be done, other than turn off the check for hotlink in db_create_record() & co (removed #define CHECK_OPEN_RECORD in odb.c).

Fixed a bug in mfe.c - it was overwriting odb /eq/xxx/common with default values. fixed now.

Running with CHECK_OPEN_RECORD seems to work okey so far. Will test some more before proposing to make it the default.

K.O.

Hi Midas forum,

I'm having a problem with odb hotlinks after increasing sub-directories in an 
odb. I now get the error code DB_NO_MEMORY after some db_open_record() calls. I 
tried 

1) increasing the parameter DEFAULT_ODB_SIZE in midas.h and make clean, make
but got the same error

2) increasing the parameter  MAX_OPEN_RECORDS in midas.h and make clean, make
but got fatal errors from odbedit and my midas FE and couldnt run anything

3) deleting my expts SHM files and starting odbedit with "odbedit -e SLAC -s 
0x1000000" to increse the odb size but got the same error?

4) I tried a different computer and got the same error code DB_NO_MEMORY

Maybe I running into some system limit that restricts the humber of open records? 
Or maybe I've not increased the correct midas parameter?

Best ,Tim.

oOne additional comment. I was able to trace the setting of the error code DB_NO_MEMORY 
to a call to the db_add_open_record() by mserver that is initiated during the start-up 
of my frontend via an RPC call. I checked with a debug printout that I have indeed 
reached the number of MAX_OPEN_RECORDS

> Hi Midas forum,
> 
> I'm having a problem with odb hotlinks after increasing sub-directories in an 
> odb. I now get the error code DB_NO_MEMORY after some db_open_record() calls. I 
> tried 
> 
> 1) increasing the parameter DEFAULT_ODB_SIZE in midas.h and make clean, make
> but got the same error
> 
> 2) increasing the parameter  MAX_OPEN_RECORDS in midas.h and make clean, make
> but got fatal errors from odbedit and my midas FE and couldnt run anything
> 
> 3) deleting my expts SHM files and starting odbedit with "odbedit -e SLAC -s 
> 0x1000000" to increse the odb size but got the same error?
> 
> 4) I tried a different computer and got the same error code DB_NO_MEMORY
> 
> Maybe I running into some system limit that restricts the humber of open records? 
> Or maybe I've not increased the correct midas parameter?
> 
> Best ,Tim.

Hi Tim,

I reproduced your problem and then managed to go through a procedure to increase the number of allowable open records.  The following is the procedure that I used 

1) Use odbedit to save current ODB

odbedit
save current_odb.odb

2) Stop all the running MIDAS processes, including mlogger and mserver using the web interface. Then stop mhttpd as well.

3) Remove your old ODB (we will recreate it after modifying MIDAS, using the backup you just made).

mv .ODB.SHM .ODB.SHM.20161104
rm /dev/shm/thomas_ODB_SHM

4) Make the following modifications to midas.  In this particular case I have increased the max number of open records from 256 to 1024.  You would need to change the constants if you want to change to other values

diff --git a/include/midas.h b/include/midas.h
index 02b30dd..33be7be 100644
--- a/include/midas.h
+++ b/include/midas.h
@@ -254,7 +254,7 @@ typedef std::vector<std::string> STRING_LIST;
-#define MAX_OPEN_RECORDS       256           /**< number of open DB records   */
+#define MAX_OPEN_RECORDS       1024           /**< number of open DB records   */
diff --git a/src/odb.c b/src/odb.c
index 47ace8f..ac1bef3 100755
--- a/src/odb.c
+++ b/src/odb.c
@@ -699,8 +699,8 @@ static void db_validate_sizes()
-   assert(sizeof(DATABASE_CLIENT) == 2112);
-   assert(sizeof(DATABASE_HEADER) == 135232);
+   assert(sizeof(DATABASE_CLIENT) == 8256);
+   assert(sizeof(DATABASE_HEADER) == 528448);

The calculation is as follows (in case you want a different number of open records): 
DATABASE_CLIENT = 64 + 8*MAX_OPEN_ERCORDS = 64 + 8*1024 = 8256
DATABASE_HEADER = 64 + 64*DATABASE_CLIENT = 64 + 64*8256 = 528448

5) Rebuild MIDAS

make clean; make

6) Create new ODB

odbedit -s 1000000

Change the size of the ODB to whatever you want.

7) reload your original ODB 

load  current_odb.odb

8) Rebuild your frontend against new MIDAS; then it should work and you should be able to produce more open records.

8.5*) Actually, I had a weird error where I needed to remove my .SYSTEM.SHM file as well when I first restarted my front-end.  Not sure if that was some unrelated error, but I mention it here for completeness.

This was a procedure based on something that originally was used for T2K (procedure by Renee Poutissou).  It is possible that not all steps are necessary and that there is a better way.  But this worked for me.

Also, any objections from other developers to tweaking the assert checks in odb.c so that the values are calculated automatically and MIDAS only needs to be touched in one place to modify the number of open records?

Let me know if it worked for you and I'll add these instructions to the Wiki.

Thomas


> oOne additional comment. I was able to trace the setting of the error code DB_NO_MEMORY 
> to a call to the db_add_open_record() by mserver that is initiated during the start-up 
> of my frontend via an RPC call. I checked with a debug printout that I have indeed 
> reached the number of MAX_OPEN_RECORDS
> 
> > Hi Midas forum,
> > 
> > I'm having a problem with odb hotlinks after increasing sub-directories in an 
> > odb. I now get the error code DB_NO_MEMORY after some db_open_record() calls. I 
> > tried 
> > 
> > 1) increasing the parameter DEFAULT_ODB_SIZE in midas.h and make clean, make
> > but got the same error
> > 
> > 2) increasing the parameter  MAX_OPEN_RECORDS in midas.h and make clean, make
> > but got fatal errors from odbedit and my midas FE and couldnt run anything
> > 
> > 3) deleting my expts SHM files and starting odbedit with "odbedit -e SLAC -s 
> > 0x1000000" to increse the odb size but got the same error?
> > 
> > 4) I tried a different computer and got the same error code DB_NO_MEMORY
> > 
> > Maybe I running into some system limit that restricts the humber of open records? 
> > Or maybe I've not increased the correct midas parameter?
> > 
> > Best ,Tim.

Hi Tim,

I reproduced your problem and then managed to go through a procedure to increase the number
of allowable open records.  The following is the procedure that I used 

1) Use odbedit to save current ODB

odbedit
save current_odb.odb

2) Stop all the running MIDAS processes, including mlogger and mserver using the web
interface. Then stop mhttpd as well.


3) Remove your old ODB (we will recreate it after modifying MIDAS, using the backup you just
made).

mv .ODB.SHM .ODB.SHM.20161104
rm /dev/shm/thomas_ODB_SHM

4) Make the following modifications to midas.  In this particular case I have increased the
max number of open records from 256 to 1024.  You would need to change the constants if you
want to change to other values

diff --git a/include/midas.h b/include/midas.h
index 02b30dd..33be7be 100644
--- a/include/midas.h
+++ b/include/midas.h
@@ -254,7 +254,7 @@ typedef std::vector<std::string> STRING_LIST;
-#define MAX_OPEN_RECORDS       256           /**< number of open DB records   */
+#define MAX_OPEN_RECORDS       1024           /**< number of open DB records   */
diff --git a/src/odb.c b/src/odb.c
index 47ace8f..ac1bef3 100755
--- a/src/odb.c
+++ b/src/odb.c
@@ -699,8 +699,8 @@ static void db_validate_sizes()
-   assert(sizeof(DATABASE_CLIENT) == 2112);
-   assert(sizeof(DATABASE_HEADER) == 135232);
+   assert(sizeof(DATABASE_CLIENT) == 8256);
+   assert(sizeof(DATABASE_HEADER) == 528448);

The calculation is as follows (in case you want a different number of open records): 
DATABASE_CLIENT = 64 + 8*MAX_OPEN_ERCORDS = 64 + 8*1024 = 8256
DATABASE_HEADER = 64 + 64*DATABASE_CLIENT = 64 + 64*8256 = 528448

5) Rebuild MIDAS

make clean; make

6) Create new ODB

odbedit -s 1000000

Change the size of the ODB to whatever you want.

7) reload your original ODB 

load  current_odb.odb

8) Rebuild your frontend against new MIDAS; then it should work and you should be able to
produce more open records.

8.5*) Actually, I had a weird error where I needed to remove my .SYSTEM.SHM file as well
when I first restarted my front-end.  Not sure if that was some unrelated error, but I
mention it here for completeness.

This was a procedure based on something that originally was used for T2K (procedure by Renee
Poutissou).  It is possible that not all steps are necessary and that there is a better way.
 But this worked for me.

Also, any objections from other developers to tweaking the assert checks in odb.c so that
the values are calculated automatically and MIDAS only needs to be touched in one place to
modify the number of open records?

Let me know if it worked for you and I'll add these instructions to the Wiki.

Thomas



> oOne additional comment. I was able to trace the setting of the error code DB_NO_MEMORY 
> to a call to the db_add_open_record() by mserver that is initiated during the start-up 
> of my frontend via an RPC call. I checked with a debug printout that I have indeed 
> reached the number of MAX_OPEN_RECORDS
> 
> > Hi Midas forum,
> > 
> > I'm having a problem with odb hotlinks after increasing sub-directories in an 
> > odb. I now get the error code DB_NO_MEMORY after some db_open_record() calls. I 
> > tried 
> > 
> > 1) increasing the parameter DEFAULT_ODB_SIZE in midas.h and make clean, make
> > but got the same error
> > 
> > 2) increasing the parameter  MAX_OPEN_RECORDS in midas.h and make clean, make
> > but got fatal errors from odbedit and my midas FE and couldnt run anything
> > 
> > 3) deleting my expts SHM files and starting odbedit with "odbedit -e SLAC -s 
> > 0x1000000" to increse the odb size but got the same error?
> > 
> > 4) I tried a different computer and got the same error code DB_NO_MEMORY
> > 
> > Maybe I running into some system limit that restricts the humber of open records? 
> > Or maybe I've not increased the correct midas parameter?
> > 
> > Best ,Tim.

The procedure I wrote seemed to work for Tim too, so I added a page to the wiki about it here:

https://midas.triumf.ca/MidasWiki/index.php/FAQ


> Hi Tim,
> 
> I reproduced your problem and then managed to go through a procedure to increase the number
> of allowable open records.  The following is the procedure that I used 
> 
> 1) Use odbedit to save current ODB
> 
> odbedit
> save current_odb.odb
> 
> 2) Stop all the running MIDAS processes, including mlogger and mserver using the web
> interface. Then stop mhttpd as well.
> 
> 
> 3) Remove your old ODB (we will recreate it after modifying MIDAS, using the backup you just
> made).
> 
> mv .ODB.SHM .ODB.SHM.20161104
> rm /dev/shm/thomas_ODB_SHM
> 
> 4) Make the following modifications to midas.  In this particular case I have increased the
> max number of open records from 256 to 1024.  You would need to change the constants if you
> want to change to other values
> 
> diff --git a/include/midas.h b/include/midas.h
> index 02b30dd..33be7be 100644
> --- a/include/midas.h
> +++ b/include/midas.h
> @@ -254,7 +254,7 @@ typedef std::vector<std::string> STRING_LIST;
> -#define MAX_OPEN_RECORDS       256           /**< number of open DB records   */
> +#define MAX_OPEN_RECORDS       1024           /**< number of open DB records   */
> diff --git a/src/odb.c b/src/odb.c
> index 47ace8f..ac1bef3 100755
> --- a/src/odb.c
> +++ b/src/odb.c
> @@ -699,8 +699,8 @@ static void db_validate_sizes()
> -   assert(sizeof(DATABASE_CLIENT) == 2112);
> -   assert(sizeof(DATABASE_HEADER) == 135232);
> +   assert(sizeof(DATABASE_CLIENT) == 8256);
> +   assert(sizeof(DATABASE_HEADER) == 528448);
> 
> The calculation is as follows (in case you want a different number of open records): 
> DATABASE_CLIENT = 64 + 8*MAX_OPEN_ERCORDS = 64 + 8*1024 = 8256
> DATABASE_HEADER = 64 + 64*DATABASE_CLIENT = 64 + 64*8256 = 528448
> 
> 5) Rebuild MIDAS
> 
> make clean; make
> 
> 6) Create new ODB
> 
> odbedit -s 1000000
> 
> Change the size of the ODB to whatever you want.
> 
> 7) reload your original ODB 
> 
> load  current_odb.odb
> 
> 8) Rebuild your frontend against new MIDAS; then it should work and you should be able to
> produce more open records.
> 
> 8.5*) Actually, I had a weird error where I needed to remove my .SYSTEM.SHM file as well
> when I first restarted my front-end.  Not sure if that was some unrelated error, but I
> mention it here for completeness.
> 
> This was a procedure based on something that originally was used for T2K (procedure by Renee
> Poutissou).  It is possible that not all steps are necessary and that there is a better way.
>  But this worked for me.
> 
> Also, any objections from other developers to tweaking the assert checks in odb.c so that
> the values are calculated automatically and MIDAS only needs to be touched in one place to
> modify the number of open records?
> 
> Let me know if it worked for you and I'll add these instructions to the Wiki.
> 
> Thomas
> 
> 
> 
> > oOne additional comment. I was able to trace the setting of the error code DB_NO_MEMORY 
> > to a call to the db_add_open_record() by mserver that is initiated during the start-up 
> > of my frontend via an RPC call. I checked with a debug printout that I have indeed 
> > reached the number of MAX_OPEN_RECORDS
> > 
> > > Hi Midas forum,
> > > 
> > > I'm having a problem with odb hotlinks after increasing sub-directories in an 
> > > odb. I now get the error code DB_NO_MEMORY after some db_open_record() calls. I 
> > > tried 
> > > 
> > > 1) increasing the parameter DEFAULT_ODB_SIZE in midas.h and make clean, make
> > > but got the same error
> > > 
> > > 2) increasing the parameter  MAX_OPEN_RECORDS in midas.h and make clean, make
> > > but got fatal errors from odbedit and my midas FE and couldnt run anything
> > > 
> > > 3) deleting my expts SHM files and starting odbedit with "odbedit -e SLAC -s 
> > > 0x1000000" to increse the odb size but got the same error?
> > > 
> > > 4) I tried a different computer and got the same error code DB_NO_MEMORY
> > > 
> > > Maybe I running into some system limit that restricts the humber of open records? 
> > > Or maybe I've not increased the correct midas parameter?
> > > 
> > > Best ,Tim.

It looks like ODB function extract_key() will overwrite the array pointed to by "key_name" if given an odb 
path with very long names (as seems to happen when redirection explodes in the Safari web browser, via 
db_get_value(TRUE) via mhttpd "start program" button). All  callers of this function seem to provide 256 
byte strings, so the problem would not show up in normal use - only when abnormal odb paths are being 
parsed. Proposed solution is to add a "length" argument to this function. (Actually ODB path elements 
should be restricted to NAME_LENGTH (32 bytes), right?). K.O.

> It looks like ODB function extract_key() will overwrite the array pointed to by "key_name" if given an odb 
> path with very long names (as seems to happen when redirection explodes in the Safari web browser, via 
> db_get_value(TRUE) via mhttpd "start program" button). All  callers of this function seem to provide 256 
> byte strings, so the problem would not show up in normal use - only when abnormal odb paths are being 
> parsed. Proposed solution is to add a "length" argument to this function. (Actually ODB path elements 
> should be restricted to NAME_LENGTH (32 bytes), right?). K.O.

This is fixed in svn revision 4129.

K.O.

Hi all,

I'm currently trying to see if I can speed up polling in a frontend I'm testing. 
Currently it seems like I can't get 'lam's to happen faster than 120 times/second. 
There must be a way to make this faster. From what I understand, changing the poll 
time (500ms by default) won't affect the frequency of polling just the 'lam' 
period.

Any suggestions?

Thanks for your help!

Isaac

Hi,

What is the actual readout time, event size?
Do you have multiple equipment and of what type if any?

PAA

> 
> I'm currently trying to see if I can speed up polling in a frontend I'm testing. 
> Currently it seems like I can't get 'lam's to happen faster than 120 times/second. 
> There must be a way to make this faster. From what I understand, changing the poll 
> time (500ms by default) won't affect the frequency of polling just the 'lam' 
> period.
> 
> Any suggestions?
> 

You could switch from the traditional midas mfe.c frontend to the C++ TMFE frontend,
where all this "lam" and "poll" business is removed.

At the moment, there are two example programs using the C++ TMFE frontend,
single threaded (progs/fetest_tmfe.cxx) and multithreaed (progs/fetest_tmfe_thread.cxx).

K.O.

> Hi all,
> 
> I'm currently trying to see if I can speed up polling in a frontend I'm testing. 
> Currently it seems like I can't get 'lam's to happen faster than 120 times/second. 
> There must be a way to make this faster. From what I understand, changing the poll 
> time (500ms by default) won't affect the frequency of polling just the 'lam' 
> period.
> 
> Any suggestions?
> 
> Thanks for your help!
> 
> Isaac

Hi,

How many equipment do you have and of what type?
What is the measured readout time of your equipment?

As you mentioned the polling time define the maximum time you spend in the in polling call before checking other equipment and system activities. But as soon as you get a LAM during the polling loop, the event is readout. The readout time of this equipment is obviously to be considered as well.

In case you have multiple equipment, the readout time of the other equipment is to be taken in account as you wont return to your polling prior the completion of them.

Something must be wrong on your side. If you take the example frontend under

midas/examples/experiment/frontend.cxx

and let it run to produce dummy events, you get about 90 Hz. This is because we have a

  ss_sleep(10);

in the read_trigger_event() routine to throttle things down. If you remove that sleep, 
you get an event rate of about 500'000 Hz. So the framework is really quick.

Probably your routine which looks for a 'lam' takes really long and should be fixed.

Stefan

> Something must be wrong on your side. If you take the example frontend under
> 
> midas/examples/experiment/frontend.cxx
> 
> and let it run to produce dummy events, you get about 90 Hz. This is because we have a
> 
>   ss_sleep(10);
> 
> in the read_trigger_event() routine to throttle things down. If you remove that sleep, 
> you get an event rate of about 500'000 Hz. So the framework is really quick.
> 
> Probably your routine which looks for a 'lam' takes really long and should be fixed.
> 
> Stefan

Sorry if I am going off-topic but, because the ss_sleep function was mentioned here, I 
would like to take the chance and report an issue that I am having.

In all my slow control frontends, the CPU usage for each frontend is close to 100%. This 
means that each frontend is monopolizing a single core. When I did some profiling, I 
noticed that 99% of the time is spent inside the ss_sleep function. Now, I would expect 
that the ss_sleep function should not require any CPU usage at all or very little.

So my two questions are:
Is this a bug or a feature?
Would you able to check/reproduce this behavior or do you need additional info from my 
side?

> Something must be wrong on your side. If you take the example frontend under
> 
> midas/examples/experiment/frontend.cxx
> 
> and let it run to produce dummy events, you get about 90 Hz. This is because we have a
> 
>   ss_sleep(10);
> 
> in the read_trigger_event() routine to throttle things down. If you remove that sleep, 
> you get an event rate of about 500'000 Hz. So the framework is really quick.
> 
> Probably your routine which looks for a 'lam' takes really long and should be fixed.
> 
> Stefan

Hi Stefan,

I should mention that I was using midas/examples/Triumf/c++/fevme.cxx. I was trying to see 
the max speed so I had the 'lam' always = 1 with nothing else to add overhead in the 
poll_event(). I was getting <200 Hz. I am assuming that this is a bug. There is no 
ss_sleep() in that function.

Thanks for your quick response!

Isaac

> > 
> > I'm currently trying to see if I can speed up polling in a frontend I'm testing. 
> > Currently it seems like I can't get 'lam's to happen faster than 120 times/second. 
> > There must be a way to make this faster. From what I understand, changing the poll 
> > time (500ms by default) won't affect the frequency of polling just the 'lam' 
> > period.
> > 
> > Any suggestions?
> > 
> 
> You could switch from the traditional midas mfe.c frontend to the C++ TMFE frontend,
> where all this "lam" and "poll" business is removed.
> 
> At the moment, there are two example programs using the C++ TMFE frontend,
> single threaded (progs/fetest_tmfe.cxx) and multithreaed (progs/fetest_tmfe_thread.cxx).
> 
> K.O.

Ok. I did not know that there was a C++ OOD frontend example in MIDAS. I'll take a look at 
it. Is there any documentation on it works?

Thanks for the support!

Isaac

> I should mention that I was using midas/examples/Triumf/c++/fevme.cxx

this is correct, the fevme frontend is written to do 100% CPU-busy polling.

there is several reasons for this:
- on our VME processors, we have 2 core CPUs, 1st core can poll the VME bus, 2nd core can run 
mfe.c and the ethernet transmitter.
- interrupts are expensive to use (in latency and in cpu use) because kernel handler has to call 
use handler, return back etc
- sub-millisecond sleep used to be expensive and unreliable (on 1-2GHz "core 1" and "core 2" 
CPUs running SL6 and SL7 era linux). As I understand, current linux and current 3+GHz CPUs can 
do reliable microsecond sleep.

K.O.

I'm getting segfaults in yb_any_file_rclose (closing a file opened with
yb_any_file_ropen with type MIDAS).

I think there are bugs with freeing from uninitialized pointers my.pmagta,
my.pyh, and my.pylrl (which are only set when opening a YBOS file).  These
should be set to NULL in yb_any_file_ropen (case MIDAS).  Likewise, the MIDAS
format pointers my.pmp and my.pmrd should be NULLed for YBOS opens. 

It might be wise to also initialize the pointers in the "my" structure to null.

--Donald              

> I'm getting segfaults in yb_any_file_rclose (closing a file opened with
> yb_any_file_ropen with type MIDAS).
> 
> I think there are bugs with freeing from uninitialized pointers my.pmagta,
> my.pyh, and my.pylrl (which are only set when opening a YBOS file).  These
> should be set to NULL in yb_any_file_ropen (case MIDAS).  Likewise, the MIDAS
> format pointers my.pmp and my.pmrd should be NULLed for YBOS opens. 
> 
> It might be wise to also initialize the pointers in the "my" structure to null.

Do you see this crash even after my fix to (another?) double free?

K.O.

I am getting a scary compiler warning from MSCB code. I generally avoid touching MSCB code because I have no MSCB hardware to test it, so I am 
asking the persons unnamed who wrote this code to fix it. Stock g++ on Ubuntu LTS 24.04. Thanks in advance!

In function ‘ssize_t read(int, void*, size_t)’,
    inlined from ‘int mscb_interprete_file(const char*, unsigned char**, unsigned int*, unsigned char**, unsigned int*, unsigned char*)’ at 
/home/olchansk/git/midas/mscb/src/mscb.cxx:2666:13:
/usr/include/x86_64-linux-gnu/bits/unistd.h:28:10: warning: ‘ssize_t __read_alias(int, void*, size_t)’ specified size 18446744073709551614 
exceeds maximum object size 9223372036854775807 [-Wstringop-overflow=]
   28 |   return __glibc_fortify (read, __nbytes, sizeof (char),
      |          ^~~~~~~~~~~~~~~
/usr/include/x86_64-linux-gnu/bits/unistd-decl.h: In function ‘int mscb_interprete_file(const char*, unsigned char**, unsigned int*, unsigned 
char**, unsigned int*, unsigned char*)’:
/usr/include/x86_64-linux-gnu/bits/unistd-decl.h:29:16: note: in a call to function ‘ssize_t __read_alias(int, void*, size_t)’ declared with 
attribute ‘access (write_only, 2, 3)’
   29 | extern ssize_t __REDIRECT_FORTIFY (__read_alias, (int __fd, void *__buf,
      |                ^~~~~~~~~~~~~~~~~~

K.O.

I hopefully fixed the waring (narrowing down from size_t to int). Please double check with your compiler.

Stefan