A split of src/midas.cxx was discussed during Stefan & Thomas R. visit to TRIUMF.

Main reason for the split is to simplify analysis using AI tools, they work best on smaller files that contain closely related 
code.

Main difficulty with the split is "git" tooling, which does not have a concept of file copy or file split. (still this is better 
than older tools like cvd and svn, which had no concept of branching and merging).

My first step is to do a strawman split of midas.cxx on a branch, push it out and ask people to check that they git tooling 
(vscode, clion, bitbucket, github, gitea, etc) see the split correctly.

Ideally, git log and git blame should be able to trace the revision of each split file
back to the beginning of time, instead of to the point of split from midas.cxx.

According to google AI, I should do this:

step 1: midas.cxx removed, split files are exact copies of midas.cxx, this allows git to identify the split point

cp midas.cxx split1.cxx
cp midas.cxx split2.cxx
...
git rm midas.cxx
git commit

step 2: edit the split files to remove duplicated code, commit.

step 3: check

git blame split1.cxx -> shows correct revision history for me. maybe need to use "-C"
git log split1.cxx -> shows correct revision history to split commit
git log --follow split1.cxx -> shows correct revision history for me

TD;DR continue below.

Identify all function names in midas.cxx:

cat midas.cxx | grep -v -e "^ " -e "^{" -e "^}" -e "^$" -e "^/" -e '^\\' -e "^@" -e "^*" -e "^#" | sort

^void functions:

void bk_xxx()
void bm_xxx()
void cm_xxx()
void dbg_xxx()
void rpc_xxx()

^std::string functions:

cm_xxx()
msprintf()
rpc_xxx()

^static functions:

bm_xxx
rpc_xxx
cm_xxx

^int functions:

bk_xxx
bm_xxx
cm_xxx
cm_msg_xxx
rb_xxx
rpc_xxx

^const data

cm_xxx
rpc_xxx

^bool functions:

rpc_xxx

^INT functions

bk_xxx
bm_xxx
cm_xxx
cm_msg_xxx
rpc_xxx

^BOOL functions

bk_xxx
cm_xxx

LC_ALL=C sort midas.cxx | grep -v -e "^/" -e "^\s" -e "^W" -e "^w" -e ^} -e ^{ -e ^[a-v] -e '^\\' -e ^[A-U] -e ^@ -e ^* -e ^# -e ^$

Stawman split:

midas.cxx -> deleted

cm_msg.cxx - all the cm_msg functions
bm.cxx     - all the event buffer functions
bk.cxx     - all the event bank functions
rb.cxx     - ring buffer functions (obsolete?)
rpc.cxx    - all the RPC functions

cm.cxx     - everything else

K.O.

> A split of src/midas.cxx was discussed during Stefan & Thomas R. visit to TRIUMF.

A test of split src/midas.cxx is now pushed to branch test/split-midas-cxx

midas.cxx -> deleted, as required by git (see below)

cm_msg.cxx - will be all the cm_msg functions
bm.cxx     - all the event buffer functions
bk.cxx     - will be all the event bank functions
rb.cxx     - ring buffer functions (obsolete?)
rpc.cxx    - will be all the RPC functions
cm.cxx     - everything else

when I do the split for real, I think I will split run transitions to cm_transition.cxx

please take a look at this branch, run git log and git blame using your preferred tool of choice (vscode, clion, etc)
and let me know if tracking of revision history works through the split commit and if there
is any other oddities.

also, better names for the split files are most welcome.

only rb.cxx and bm.cxx are actually reduced in size, the rest of the files are still copies of midas.cxx (this is only a test of the 
split).

K.O.

P.S. More details below:

> 
> Main reason for the split is to simplify analysis using AI tools, they work best on smaller files that contain closely related 
> code.
> 
> Main difficulty with the split is "git" tooling, which does not have a concept of file copy or file split. (still this is better 
> than older tools like cvd and svn, which had no concept of branching and merging).
> 
> My first step is to do a strawman split of midas.cxx on a branch, push it out and ask people to check that they git tooling 
> (vscode, clion, bitbucket, github, gitea, etc) see the split correctly.
> 
> Ideally, git log and git blame should be able to trace the revision of each split file
> back to the beginning of time, instead of to the point of split from midas.cxx.
> 
> According to google AI, I should do this:
> 
> step 1: midas.cxx removed, split files are exact copies of midas.cxx, this allows git to identify the split point
> 
> cp midas.cxx split1.cxx
> cp midas.cxx split2.cxx
> ...
> git rm midas.cxx
> git commit
> 
> step 2: edit the split files to remove duplicated code, commit.
> 
> step 3: check
> 
> git blame split1.cxx -> shows correct revision history for me. maybe need to use "-C"
> git log split1.cxx -> shows correct revision history to split commit
> git log --follow split1.cxx -> shows correct revision history for me
> 
> TD;DR continue below.
> 
> Identify all function names in midas.cxx:
> 
> cat midas.cxx | grep -v -e "^ " -e "^{" -e "^}" -e "^$" -e "^/" -e '^\\' -e "^@" -e "^*" -e "^#" | sort
> 
> ^void functions:
> 
> void bk_xxx()
> void bm_xxx()
> void cm_xxx()
> void dbg_xxx()
> void rpc_xxx()
> 
> ^std::string functions:
> 
> cm_xxx()
> msprintf()
> rpc_xxx()
> 
> ^static functions:
> 
> bm_xxx
> rpc_xxx
> cm_xxx
> 
> ^int functions:
> 
> bk_xxx
> bm_xxx
> cm_xxx
> cm_msg_xxx
> rb_xxx
> rpc_xxx
> 
> ^const data
> 
> cm_xxx
> rpc_xxx
> 
> ^bool functions:
> 
> rpc_xxx
> 
> ^INT functions
> 
> bk_xxx
> bm_xxx
> cm_xxx
> cm_msg_xxx
> rpc_xxx
> 
> ^BOOL functions
> 
> bk_xxx
> cm_xxx
> 
> LC_ALL=C sort midas.cxx | grep -v -e "^/" -e "^\s" -e "^W" -e "^w" -e ^} -e ^{ -e ^[a-v] -e '^\\' -e ^[A-U] -e ^@ -e ^* -e ^# -e ^$
> 
> Stawman split:
> 
> midas.cxx -> deleted
> 
> cm_msg.cxx - all the cm_msg functions
> bm.cxx     - all the event buffer functions
> bk.cxx     - all the event bank functions
> rb.cxx     - ring buffer functions (obsolete?)
> rpc.cxx    - all the RPC functions
> 
> cm.cxx     - everything else
> 
> K.O.

> > A split of src/midas.cxx was discussed during Stefan & Thomas R. visit to TRIUMF.
> 
> A test of split src/midas.cxx is now pushed to branch test/split-midas-cxx
> 
> please take a look at this branch, run git log and git blame using your preferred tool of choice (vscode, clion, etc)
> and let me know if tracking of revision history works through the split commit and if there
> is any other oddities.
>

bitbucket web gui for src/rb.cxx
- show history - only to split commit (ouch!)
- show blame - all the way to the beginning of time (good)

K.O.

The history of at least bm.cxx seems to be screwed up. Bitbucket shows this:

  https://bitbucket.org/tmidas/midas/annotate/fa649e6e9bb7447b4da01e964c6dc14b01e64052/src/bm.cxx?at=test%2Fsplit-midas-cxx

and CLion shows what I attached here. Most lines are tagged "Today Olchanski" which cannot be.

Best,
Stefan

> The history of at least bm.cxx seems to be screwed up. Bitbucket shows this:
>   https://bitbucket.org/tmidas/midas/annotate/fa649e6e9bb7447b4da01e964c6dc14b01e64052/src/bm.cxx?at=test%2Fsplit-midas-cxx
> and CLion shows what I attached here. Most lines are tagged "Today Olchanski" which cannot be.

yup, I see the same nonsense for bm.cxx from gitea.

back to the drawing board, will try branch, rename, merge as described here:
https://devblogs.microsoft.com/oldnewthing/20190916-00/?p=102892/

the naming of split files looks ok or we prefer more descriptive names "event_buffer.cxx",
"transition.cxx", "message.cxx", etc?

K.O.

> It turns out that unlike CentOS-7, Ubuntu LTS 18.04 does not run a restrictive firewall
> and access to mhttpd ports 8080 and 8443 is not blocked
>
> As result, anybody from anywhere can access mhttpd without a password.
> 

elogd can suffer from the same problem, but not as badly, one can connect to elogd and attempt to run 
exploits, but one cannot access elog entries without a password:

a) default configuration is to ask for a password
b) elogd almost immediately redirects to the https URL specified in the URL entry of the config file, which 
normally points to the https proxy, which also immediately asks for a password.

In the absence of firewall protection (as on Ubuntu), 
add "Interface = 127.0.0.1" to the elog config file or run elogd with "-n localhost",
per instructions at https://elog.psi.ch/elog/config.html 

K.O.

We installed recent mhttpd on a ubuntu machine and discovered a number of problems
with the default mhttpd settings.

We did follow the normal instructions to install and configure an apache https proxy
with a certbot certificate and password protection, this part worked ok. Big thanks
to Lars M. for providing the Ubuntu instructions for apache.

Then we started seeing errors from mhttpd about access to URLs like "manager/html" 
(google "manager/html exploit") that did not go through the proxy.

It turns out that unlike CentOS-7, Ubuntu LTS 18.04 does not run a restrictive firewall
and access to mhttpd ports 8080 and 8443 is not blocked. Then, it turns out that by 
default, the mhttpd access controls are also disabled, and it accepts http requests from 
anywhere/everywhere. Also by default, the mhttpd password is also disabled.

As result, anybody from anywhere can access mhttpd without a password.

One fix for this is to activate the mhttpd access control list by setting ODB 
/Experiment/Security/allowed hosts[0] to "localhost".

K.O.

Hello,
Our experiment based on MIDAS 2.x DAQ.
I'm using several identical frontend-%d  with only lam source & event id changed, 
running on 2 computers(~3frontends per one).
Each recieve about 10k Events (Max_SIZE =8*1024, but usually it is less then 
sizeof(DWORD)*400) per 7sec.
With no mlogger running it works just fine, but when I'm starting mlogger (on 3-d 
computer with mserver running)... looking at ethernet stat graph first 2-3 spills 
goes well, with one peak per 7 sec, then it becomes junky and everithing crushed 
(mlogger and frontends).
I tried to increase SYSTEM buffer and restart everything. What I saw was Logger 
writes only half of recieved events from sum of frontends, it stays running for 
awhile ~15minutes. If I push STOP button  before crashing, mlogger continious 
writing data on disk enough priod of time.
I will try to look at disk usage for bad sectors @HDD, but may be there is an easy 
way to fix this problem and i did something wrong. 
structure of frontend has code like
EQ_POLLED , POLL for 500,

frontend_loop{
read big buffer with 10k events;bufferread=true;
}

poll_event{
for (i=0;i<count;i++){
 if (bufferread) lam=1;
 if (!test) return lam;
 }
return 0;
}

read_trigger{
bk_init32();
//fill event with buffer until current word!=0xffffffff
if (currentposition+2 >buffer_size) bufferread=false
}
|
Help needed, please. Suggestions.
Thanks, Alexey.

Hello, once again.
What I found is when I tryed to stop the run, mlogger still working and writing some 
data, that i'm sure is not right, because frontend's are in stopped state
( for ex. every 3*frontend got 50k, mlogger showes 120k . Stop button pushed, but data 
in .mid file collect more then 150k~300k ev)
. And it continue writing until it crashes by the default waiting period 10s.

Hello, 
I'm in deppression.
I removed Everything from computer with mserver and reinstall system and midas.
Then I tried to run tutorial example.
Often run did not stop by pushing STOP button (mlogger stuck it, odbedit stop 
works)
After first START button pushed number of event taken by frontend equals mlogger 
events 
written. Next run (without mlogger restarting) mlogger double the number of 
events taken by 
frontend.(see attachment).Restarting mlogger fix this double counting.
What i've did wrong?

Suddenly the mhttpd odb inline editor is truncating the odb string entries to the actual length of the 
stored string value, this causes db_get_record() explode with "structure mismatch" errors. (Not my 
fault, You Honor! Honest!). For example, I see these errors from al_check() after changing 
"/programs/foo/start command" - suddenly it cannot get the program_info record.

What a mess.

Actually, this is not a new mess, midas was always been rather brittle with db_get_record()
and db_open_record(), always unhappy if something goes wrong in odb, like a lost
entry in equipment statistics or an extra variable in equipment common, etc.

To patch it all up, I added a function db_get_record1() which knows the structure of the data
and can call db_check_record() to fix the odb structure and make db_get_record() happy.
Many places in midas now use it, making odb structure mismatches "self healing" in a way.

But when looking at uses of db_get_record(), I notices that in many places it can be trivially
replaced by one or two db_get_value(). I did change this in a couple of places in mhttpd.
This way of coding is more robust against unexpected contents in odb and is easier
to maintain going forward, when new odb entries must be added for new functionality.

Most uses of db_get_record() are now converted to db_get_record1(), except where it is
used in together with of db_open_record(). (which uses db_get_record() internally).

To fix the db_open_record() uses, I considered adding db_open_record1() which would
also know the data structure and automatically repair any mismatch, but I think instead of that,
I will switch them to use db_watch() (in conjunction with manual db_get_record()/get_record1()
and plain db_get_value()).

When adding automatic repair mechanism like this, one should beware of "update wars",
where two midas programs built against slightly different versions of midas would
each try to change odb in it's way, in an endless loop. (yes, it did happen, more than once).
One solution to this is to assign an "owner" to each data structure, the "consumers"
of the data have to deal with anything missing or unexpected. If they use db_get_value()
it should all be happy. (if the owner has to be reassigned, back to the wars again, until
everything is rebuilt against the same version of midas).

P.S. In languages lacking reflection, like C and C++, it is impossible to trivially implement
a mapping from a data structure to an external entity, such as db_get_record() to map C struct
into ODB. Many attempts have been made, i.e. ROOT CINT, all of them brittle, hard
to maintain, generally unsatisfactory. Java was the first mainstream language
to have reflection. Modern languages, such as Go, have reflection from day 1. Of course
all scripting languages, perl, python, javascript, always had reflection. The C++ language
standard will get reflections some day. Today one can easily do reflection in C++ using the Clang
compiler, the main reason for ROOT v6 switching from CINT to Clang.

K.O.

> What a mess.

The mess with db_get_record() and db_open_record() is even deeper than I thought. There are several anomalies.

Records opened by db_open_record() are later accessed via db_get_record() which requires
that the odb structure and the C structure match exactly.

Of course anybody can modify anything in odb at any time, so there are protections against
modifying the odb structures "from under" db_open_record():

a) db_open_record(MODE_WRITE) makes the odb structure immutable by setting the "exclusive" flag. This works well. In the past 
there were problems with "exclusive mode" getting stuck behind dead clients, but these days it is efficiently cleaned and recovered 
by the odb validation code at the start of all midas programs.

b) db_create_record(), db_reorder_key() and db_delete_key() refuse to function on watched/hotlinked odb structures. One would 
think this is good, but there is a side-effect. If I run "odbedit watch /", all odb delete operations fail (including deletion of temporary 
items in /system/tmp).

c) db_create_key() and db_set_data()/db_set_value() do not have such protections, and they can (and do) add new odb entries and 
change size of existing entries (especially size of strings), and make db_get_record() fail. note that db_get_record() inside 
db_open_record() fails silently and odb hotlinks mysteriously stop working.

One could keep fixing this by adding protections against modification of hotlinked odb structures, but unfortunately, one cannot tell
db_watch() hotlinks from db_open_record() hotlinks. Only the latter ones require protection. db_watch() does not require such 
protections because it does not use db_get_record() internally, it leaves it to the user to sort out any mismatches.

Also it would be nice if "odbedit watch /" did not have the nasty side effect of making all odb unchangable (presently it only makes
things undeletable).

To sort it all out, I am moving in this direction:

1) replace all uses of db_get_record() with db_get_record1() which automatically cures any structure mismatch
2) replace all uses of db_open_record(MODE_READ) with db_watch() in conjunction with db_get_record1(). This is done in mfe.c 
and seems to work ok.
2a) automatic repair of structure mismatch is presently defeated by db_create_record() refusing to work on hotlinked odb entries.
3) with db_get_record() and db_open_record(MODE_READ) removed from use, turn off hotlink protection in item (b) above. This will 
fix problem (2a).
4) maybe replace db_open_record(MODE_WRITE) with explicit db_set_record(). I personally do not like it's "magical" operation, 
when in fact, it is just a short hand for "db_get_key/db_set_record" hidden inside db_send_changed_records().
4a) db_open_record(MODE_WRITE) works well enough right now, no need to touch it.


K.O.

> 2) replace all uses of db_open_record(MODE_READ) with db_watch() in conjunction with db_get_record1().

Done to all in-tree programs, except for mana.c (not using it), sequencer.cxx (cannot test it) and a few places where watching a TID_INT.

Nothing more needs to be done, other than turn off the check for hotlink in db_create_record() & co (removed #define CHECK_OPEN_RECORD in odb.c).

K.O.

$ grep db_open_record src/* | grep MODE_READ
src/lazylogger.cxx:   status = db_open_record(hDB, hKey, &run_state, sizeof(run_state), MODE_READ, NULL, NULL); // watch a TID_INT
src/mana.cxx:   db_open_record(hDB, hkey, NULL, 0, MODE_READ, banks_changed, NULL);
src/mana.cxx:   db_open_record(hDB, hkey, NULL, 0, MODE_READ, banks_changed, NULL);
src/mana.cxx:          db_open_record(hDB, hkey, &out_info, sizeof(out_info), MODE_READ, NULL, NULL);
src/mana.cxx:      db_open_record(hDB, hKey, ar_info, sizeof(AR_INFO), MODE_READ, update_request,
src/midas.c:   status = db_open_record(hDB, hKey, &_requested_transition, sizeof(INT), MODE_READ, NULL, NULL);
src/mlogger.cxx:      status = db_open_record(hDB, hKey, hist_log[index].buffer, size, MODE_READ, log_history, NULL);
src/mlogger.cxx:                     db_open_record(hDB, hVarKey, NULL, varkey.total_size, MODE_READ, log_system_history, (void *) (POINTER_T) index);
src/mlogger.cxx:               db_open_record(hDB, hHistKey, NULL, size, MODE_READ, log_system_history, (void *) (POINTER_T) index);
src/odbedit.cxx:            db_open_record(hDB, hKey, data, size, MODE_READ, key_update, NULL);
src/sequencer.cxx:   status = db_open_record(hDB, hKey, &seq, sizeof(seq), MODE_READ, NULL, NULL);
8s-macbook-pro:midas 8ss$ 

That all makes sense to me. 

Stefan

> > What a mess.
> 
> The mess with db_get_record() and db_open_record() is even deeper than I thought. There are several anomalies.
> 
> Records opened by db_open_record() are later accessed via db_get_record() which requires
> that the odb structure and the C structure match exactly.
> 
> Of course anybody can modify anything in odb at any time, so there are protections against
> modifying the odb structures "from under" db_open_record():
> 
> a) db_open_record(MODE_WRITE) makes the odb structure immutable by setting the "exclusive" flag. This works well. In the past 
> there were problems with "exclusive mode" getting stuck behind dead clients, but these days it is efficiently cleaned and recovered 
> by the odb validation code at the start of all midas programs.
> 
> b) db_create_record(), db_reorder_key() and db_delete_key() refuse to function on watched/hotlinked odb structures. One would 
> think this is good, but there is a side-effect. If I run "odbedit watch /", all odb delete operations fail (including deletion of temporary 
> items in /system/tmp).
> 
> c) db_create_key() and db_set_data()/db_set_value() do not have such protections, and they can (and do) add new odb entries and 
> change size of existing entries (especially size of strings), and make db_get_record() fail. note that db_get_record() inside 
> db_open_record() fails silently and odb hotlinks mysteriously stop working.
> 
> One could keep fixing this by adding protections against modification of hotlinked odb structures, but unfortunately, one cannot tell
> db_watch() hotlinks from db_open_record() hotlinks. Only the latter ones require protection. db_watch() does not require such 
> protections because it does not use db_get_record() internally, it leaves it to the user to sort out any mismatches.
> 
> Also it would be nice if "odbedit watch /" did not have the nasty side effect of making all odb unchangable (presently it only makes
> things undeletable).
> 
> To sort it all out, I am moving in this direction:
> 
> 1) replace all uses of db_get_record() with db_get_record1() which automatically cures any structure mismatch
> 2) replace all uses of db_open_record(MODE_READ) with db_watch() in conjunction with db_get_record1(). This is done in mfe.c 
> and seems to work ok.
> 2a) automatic repair of structure mismatch is presently defeated by db_create_record() refusing to work on hotlinked odb entries.
> 3) with db_get_record() and db_open_record(MODE_READ) removed from use, turn off hotlink protection in item (b) above. This will 
> fix problem (2a).
> 4) maybe replace db_open_record(MODE_WRITE) with explicit db_set_record(). I personally do not like it's "magical" operation, 
> when in fact, it is just a short hand for "db_get_key/db_set_record" hidden inside db_send_changed_records().
> 4a) db_open_record(MODE_WRITE) works well enough right now, no need to touch it.
> 
> 
> K.O.

> Done to all in-tree programs, except for mana.c (not using it), sequencer.cxx (cannot test it) and a few places where watching a TID_INT.
> Nothing more needs to be done, other than turn off the check for hotlink in db_create_record() & co (removed #define CHECK_OPEN_RECORD in odb.c).

Fixed a bug in mfe.c - it was overwriting odb /eq/xxx/common with default values. fixed now.

Running with CHECK_OPEN_RECORD seems to work okey so far. Will test some more before proposing to make it the default.

K.O.

Hi Midas forum,

I'm having a problem with odb hotlinks after increasing sub-directories in an 
odb. I now get the error code DB_NO_MEMORY after some db_open_record() calls. I 
tried 

1) increasing the parameter DEFAULT_ODB_SIZE in midas.h and make clean, make
but got the same error

2) increasing the parameter  MAX_OPEN_RECORDS in midas.h and make clean, make
but got fatal errors from odbedit and my midas FE and couldnt run anything

3) deleting my expts SHM files and starting odbedit with "odbedit -e SLAC -s 
0x1000000" to increse the odb size but got the same error?

4) I tried a different computer and got the same error code DB_NO_MEMORY

Maybe I running into some system limit that restricts the humber of open records? 
Or maybe I've not increased the correct midas parameter?

Best ,Tim.

oOne additional comment. I was able to trace the setting of the error code DB_NO_MEMORY 
to a call to the db_add_open_record() by mserver that is initiated during the start-up 
of my frontend via an RPC call. I checked with a debug printout that I have indeed 
reached the number of MAX_OPEN_RECORDS

> Hi Midas forum,
> 
> I'm having a problem with odb hotlinks after increasing sub-directories in an 
> odb. I now get the error code DB_NO_MEMORY after some db_open_record() calls. I 
> tried 
> 
> 1) increasing the parameter DEFAULT_ODB_SIZE in midas.h and make clean, make
> but got the same error
> 
> 2) increasing the parameter  MAX_OPEN_RECORDS in midas.h and make clean, make
> but got fatal errors from odbedit and my midas FE and couldnt run anything
> 
> 3) deleting my expts SHM files and starting odbedit with "odbedit -e SLAC -s 
> 0x1000000" to increse the odb size but got the same error?
> 
> 4) I tried a different computer and got the same error code DB_NO_MEMORY
> 
> Maybe I running into some system limit that restricts the humber of open records? 
> Or maybe I've not increased the correct midas parameter?
> 
> Best ,Tim.

Hi Tim,

I reproduced your problem and then managed to go through a procedure to increase the number of allowable open records.  The following is the procedure that I used 

1) Use odbedit to save current ODB

odbedit
save current_odb.odb

2) Stop all the running MIDAS processes, including mlogger and mserver using the web interface. Then stop mhttpd as well.

3) Remove your old ODB (we will recreate it after modifying MIDAS, using the backup you just made).

mv .ODB.SHM .ODB.SHM.20161104
rm /dev/shm/thomas_ODB_SHM

4) Make the following modifications to midas.  In this particular case I have increased the max number of open records from 256 to 1024.  You would need to change the constants if you want to change to other values

diff --git a/include/midas.h b/include/midas.h
index 02b30dd..33be7be 100644
--- a/include/midas.h
+++ b/include/midas.h
@@ -254,7 +254,7 @@ typedef std::vector<std::string> STRING_LIST;
-#define MAX_OPEN_RECORDS       256           /**< number of open DB records   */
+#define MAX_OPEN_RECORDS       1024           /**< number of open DB records   */
diff --git a/src/odb.c b/src/odb.c
index 47ace8f..ac1bef3 100755
--- a/src/odb.c
+++ b/src/odb.c
@@ -699,8 +699,8 @@ static void db_validate_sizes()
-   assert(sizeof(DATABASE_CLIENT) == 2112);
-   assert(sizeof(DATABASE_HEADER) == 135232);
+   assert(sizeof(DATABASE_CLIENT) == 8256);
+   assert(sizeof(DATABASE_HEADER) == 528448);

The calculation is as follows (in case you want a different number of open records): 
DATABASE_CLIENT = 64 + 8*MAX_OPEN_ERCORDS = 64 + 8*1024 = 8256
DATABASE_HEADER = 64 + 64*DATABASE_CLIENT = 64 + 64*8256 = 528448

5) Rebuild MIDAS

make clean; make

6) Create new ODB

odbedit -s 1000000

Change the size of the ODB to whatever you want.

7) reload your original ODB 

load  current_odb.odb

8) Rebuild your frontend against new MIDAS; then it should work and you should be able to produce more open records.

8.5*) Actually, I had a weird error where I needed to remove my .SYSTEM.SHM file as well when I first restarted my front-end.  Not sure if that was some unrelated error, but I mention it here for completeness.

This was a procedure based on something that originally was used for T2K (procedure by Renee Poutissou).  It is possible that not all steps are necessary and that there is a better way.  But this worked for me.

Also, any objections from other developers to tweaking the assert checks in odb.c so that the values are calculated automatically and MIDAS only needs to be touched in one place to modify the number of open records?

Let me know if it worked for you and I'll add these instructions to the Wiki.

Thomas


> oOne additional comment. I was able to trace the setting of the error code DB_NO_MEMORY 
> to a call to the db_add_open_record() by mserver that is initiated during the start-up 
> of my frontend via an RPC call. I checked with a debug printout that I have indeed 
> reached the number of MAX_OPEN_RECORDS
> 
> > Hi Midas forum,
> > 
> > I'm having a problem with odb hotlinks after increasing sub-directories in an 
> > odb. I now get the error code DB_NO_MEMORY after some db_open_record() calls. I 
> > tried 
> > 
> > 1) increasing the parameter DEFAULT_ODB_SIZE in midas.h and make clean, make
> > but got the same error
> > 
> > 2) increasing the parameter  MAX_OPEN_RECORDS in midas.h and make clean, make
> > but got fatal errors from odbedit and my midas FE and couldnt run anything
> > 
> > 3) deleting my expts SHM files and starting odbedit with "odbedit -e SLAC -s 
> > 0x1000000" to increse the odb size but got the same error?
> > 
> > 4) I tried a different computer and got the same error code DB_NO_MEMORY
> > 
> > Maybe I running into some system limit that restricts the humber of open records? 
> > Or maybe I've not increased the correct midas parameter?
> > 
> > Best ,Tim.

Hi Tim,

I reproduced your problem and then managed to go through a procedure to increase the number
of allowable open records.  The following is the procedure that I used 

1) Use odbedit to save current ODB

odbedit
save current_odb.odb

2) Stop all the running MIDAS processes, including mlogger and mserver using the web
interface. Then stop mhttpd as well.


3) Remove your old ODB (we will recreate it after modifying MIDAS, using the backup you just
made).

mv .ODB.SHM .ODB.SHM.20161104
rm /dev/shm/thomas_ODB_SHM

4) Make the following modifications to midas.  In this particular case I have increased the
max number of open records from 256 to 1024.  You would need to change the constants if you
want to change to other values

diff --git a/include/midas.h b/include/midas.h
index 02b30dd..33be7be 100644
--- a/include/midas.h
+++ b/include/midas.h
@@ -254,7 +254,7 @@ typedef std::vector<std::string> STRING_LIST;
-#define MAX_OPEN_RECORDS       256           /**< number of open DB records   */
+#define MAX_OPEN_RECORDS       1024           /**< number of open DB records   */
diff --git a/src/odb.c b/src/odb.c
index 47ace8f..ac1bef3 100755
--- a/src/odb.c
+++ b/src/odb.c
@@ -699,8 +699,8 @@ static void db_validate_sizes()
-   assert(sizeof(DATABASE_CLIENT) == 2112);
-   assert(sizeof(DATABASE_HEADER) == 135232);
+   assert(sizeof(DATABASE_CLIENT) == 8256);
+   assert(sizeof(DATABASE_HEADER) == 528448);

The calculation is as follows (in case you want a different number of open records): 
DATABASE_CLIENT = 64 + 8*MAX_OPEN_ERCORDS = 64 + 8*1024 = 8256
DATABASE_HEADER = 64 + 64*DATABASE_CLIENT = 64 + 64*8256 = 528448

5) Rebuild MIDAS

make clean; make

6) Create new ODB

odbedit -s 1000000

Change the size of the ODB to whatever you want.

7) reload your original ODB 

load  current_odb.odb

8) Rebuild your frontend against new MIDAS; then it should work and you should be able to
produce more open records.

8.5*) Actually, I had a weird error where I needed to remove my .SYSTEM.SHM file as well
when I first restarted my front-end.  Not sure if that was some unrelated error, but I
mention it here for completeness.

This was a procedure based on something that originally was used for T2K (procedure by Renee
Poutissou).  It is possible that not all steps are necessary and that there is a better way.
 But this worked for me.

Also, any objections from other developers to tweaking the assert checks in odb.c so that
the values are calculated automatically and MIDAS only needs to be touched in one place to
modify the number of open records?

Let me know if it worked for you and I'll add these instructions to the Wiki.

Thomas



> oOne additional comment. I was able to trace the setting of the error code DB_NO_MEMORY 
> to a call to the db_add_open_record() by mserver that is initiated during the start-up 
> of my frontend via an RPC call. I checked with a debug printout that I have indeed 
> reached the number of MAX_OPEN_RECORDS
> 
> > Hi Midas forum,
> > 
> > I'm having a problem with odb hotlinks after increasing sub-directories in an 
> > odb. I now get the error code DB_NO_MEMORY after some db_open_record() calls. I 
> > tried 
> > 
> > 1) increasing the parameter DEFAULT_ODB_SIZE in midas.h and make clean, make
> > but got the same error
> > 
> > 2) increasing the parameter  MAX_OPEN_RECORDS in midas.h and make clean, make
> > but got fatal errors from odbedit and my midas FE and couldnt run anything
> > 
> > 3) deleting my expts SHM files and starting odbedit with "odbedit -e SLAC -s 
> > 0x1000000" to increse the odb size but got the same error?
> > 
> > 4) I tried a different computer and got the same error code DB_NO_MEMORY
> > 
> > Maybe I running into some system limit that restricts the humber of open records? 
> > Or maybe I've not increased the correct midas parameter?
> > 
> > Best ,Tim.

The procedure I wrote seemed to work for Tim too, so I added a page to the wiki about it here:

https://midas.triumf.ca/MidasWiki/index.php/FAQ


> Hi Tim,
> 
> I reproduced your problem and then managed to go through a procedure to increase the number
> of allowable open records.  The following is the procedure that I used 
> 
> 1) Use odbedit to save current ODB
> 
> odbedit
> save current_odb.odb
> 
> 2) Stop all the running MIDAS processes, including mlogger and mserver using the web
> interface. Then stop mhttpd as well.
> 
> 
> 3) Remove your old ODB (we will recreate it after modifying MIDAS, using the backup you just
> made).
> 
> mv .ODB.SHM .ODB.SHM.20161104
> rm /dev/shm/thomas_ODB_SHM
> 
> 4) Make the following modifications to midas.  In this particular case I have increased the
> max number of open records from 256 to 1024.  You would need to change the constants if you
> want to change to other values
> 
> diff --git a/include/midas.h b/include/midas.h
> index 02b30dd..33be7be 100644
> --- a/include/midas.h
> +++ b/include/midas.h
> @@ -254,7 +254,7 @@ typedef std::vector<std::string> STRING_LIST;
> -#define MAX_OPEN_RECORDS       256           /**< number of open DB records   */
> +#define MAX_OPEN_RECORDS       1024           /**< number of open DB records   */
> diff --git a/src/odb.c b/src/odb.c
> index 47ace8f..ac1bef3 100755
> --- a/src/odb.c
> +++ b/src/odb.c
> @@ -699,8 +699,8 @@ static void db_validate_sizes()
> -   assert(sizeof(DATABASE_CLIENT) == 2112);
> -   assert(sizeof(DATABASE_HEADER) == 135232);
> +   assert(sizeof(DATABASE_CLIENT) == 8256);
> +   assert(sizeof(DATABASE_HEADER) == 528448);
> 
> The calculation is as follows (in case you want a different number of open records): 
> DATABASE_CLIENT = 64 + 8*MAX_OPEN_ERCORDS = 64 + 8*1024 = 8256
> DATABASE_HEADER = 64 + 64*DATABASE_CLIENT = 64 + 64*8256 = 528448
> 
> 5) Rebuild MIDAS
> 
> make clean; make
> 
> 6) Create new ODB
> 
> odbedit -s 1000000
> 
> Change the size of the ODB to whatever you want.
> 
> 7) reload your original ODB 
> 
> load  current_odb.odb
> 
> 8) Rebuild your frontend against new MIDAS; then it should work and you should be able to
> produce more open records.
> 
> 8.5*) Actually, I had a weird error where I needed to remove my .SYSTEM.SHM file as well
> when I first restarted my front-end.  Not sure if that was some unrelated error, but I
> mention it here for completeness.
> 
> This was a procedure based on something that originally was used for T2K (procedure by Renee
> Poutissou).  It is possible that not all steps are necessary and that there is a better way.
>  But this worked for me.
> 
> Also, any objections from other developers to tweaking the assert checks in odb.c so that
> the values are calculated automatically and MIDAS only needs to be touched in one place to
> modify the number of open records?
> 
> Let me know if it worked for you and I'll add these instructions to the Wiki.
> 
> Thomas
> 
> 
> 
> > oOne additional comment. I was able to trace the setting of the error code DB_NO_MEMORY 
> > to a call to the db_add_open_record() by mserver that is initiated during the start-up 
> > of my frontend via an RPC call. I checked with a debug printout that I have indeed 
> > reached the number of MAX_OPEN_RECORDS
> > 
> > > Hi Midas forum,
> > > 
> > > I'm having a problem with odb hotlinks after increasing sub-directories in an 
> > > odb. I now get the error code DB_NO_MEMORY after some db_open_record() calls. I 
> > > tried 
> > > 
> > > 1) increasing the parameter DEFAULT_ODB_SIZE in midas.h and make clean, make
> > > but got the same error
> > > 
> > > 2) increasing the parameter  MAX_OPEN_RECORDS in midas.h and make clean, make
> > > but got fatal errors from odbedit and my midas FE and couldnt run anything
> > > 
> > > 3) deleting my expts SHM files and starting odbedit with "odbedit -e SLAC -s 
> > > 0x1000000" to increse the odb size but got the same error?
> > > 
> > > 4) I tried a different computer and got the same error code DB_NO_MEMORY
> > > 
> > > Maybe I running into some system limit that restricts the humber of open records? 
> > > Or maybe I've not increased the correct midas parameter?
> > > 
> > > Best ,Tim.