> try to replicate the issue ...

I see ODB lock timeout (and abort() of everything) in the dsvslice test station. We have 
about 15-20 MIDAS clients connected.

I am pretty sure we have not seen this problem until recently (and I have not seen it 
personally for a very long time). There were no changes to the MIDAS ODB locking code in a 
long time.

I suspect a recent change in the linux kernel. But I am likely to be wrong.

I have 1000 core dumps from this crash of dsvslice, and among them should be the 1 thread
that has ODB locked. Wish me luck finding it. Worst case is to discover that ODB is locked 
but nobody is holding a lock ("missing unlock bug"). This is hard to debug, I would have add 
tracking of "who was the last one to lock it, who forgot to unlock it".

K.O.

there is a really good software tool developed by the Fermilab DAQ group, called TRACE - 

https://github.com/art-daq/trace ,

It could be useful for debugging cases like this one. In short, TRACE instruments the code 
with the printouts which could be selectively turned on and off without recompiling the executable. 

TRACE output could go to /dev/stdout (slow output) and/or to a circular buffer implemented via a shared 
memory segment (fast output). Sending unlimited output to the shared memory segment is extremely useful.

TRACE also allows to trigger on certain conditions, again, w/o recompiling the executable. 
For debugging cases like the one in question, that could turn out even more useful, 
however I didn't try the triggering functionality myself. 

-- regards, Pasha

In revision 4090 I implemented ODB links to individual array elements. Now you
can have for example:

Key name                        Type    #Val  Size  Last Opn Mode Value
---------------------------------------------------------------------------
array                           INT     10    4     2m   0   RWD
                                        [0]             0
                                        [1]             0
                                        [2]             123
                                        [3]             0
                                        [4]             0
                                        [5]             0
                                        [6]             0
                                        [7]             0
                                        [8]             0
                                        [9]             0
element2 -> /array[2]           INT     1     4     3m   0   RWD  123

In this case, the link "element2" points to the third element of "array", but is
treated like a single value. This links are very useful for example for the
"Edit on start" parameters, which can now point to individual array elements.
The same is true for the "Links BOR" when the logger writes to a MySQL database.

This modification required major modifications in the ODB. I have carefully
tested the example experiment from the distribution to verify that everything is
fine, but I'm not 100% sure that I covered all possible situations. So if you
update to revision 4090+ and you observe some strange behavior related to links
in the ODB, please report.

There are following two new functions related to this change: 

  db_get_link()
  db_get_link_data()

They are counterparts of db_get_key() and db_get_data(), respectively, but
without following links in the ODB. These functions are probably not of much use
outside odbedit and mhttpd, which are supposed to display links explicitly. Most
user applications want to follow links without even knowing that these are links.

To reproduce:
In the interactive ODB, click the 🔗 icon to create a link. Next to the target,  click the "..." button to open 
the key picker browser. Then try to close it by either:
- Selecting a key and clicking ok
- Clicking "cancel"
- Clicking the red circle at the top left

Expected result:
The key picker closes

Actual result:
The key picker does not close.

Depending on how you trying to close the picker, the error messages in the debug console differ slightly.

On the red circle:
Uncaught TypeError: dlg is null
    dlgClose http://localhost:8080/controls.js:791
    onclick http://localhost:8080/?cmd=ODB&odb_path=/Test:1

On "ok" or "cancel":
Uncaught TypeError: dlg is null
    dlgMessageDestroy http://localhost:8080/controls.js:828
    pickerButton http://localhost:8080/odbbrowser.js:453
    onclick http://localhost:8080/?cmd=ODB&odb_path=/Test:1


Another more minor visual problem is the edit-on-start dialog. There seems to be no upper bound to the 
size of the text box. In the attached screenshot, ShortString has a maximum length of 32 characters, 
LongString has 255. Both are empty at the time of the screenshot. Maybe, the size should be limited to a 
reasonable width.

> Actual result:
> The key picker does not close.

Thanks for reporting that bug. It has been fixed in the current commit (installed already on megon02)

Stefan

> Another more minor visual problem is the edit-on-start dialog. There seems to be no upper bound to the 
> size of the text box. In the attached screenshot, ShortString has a maximum length of 32 characters, 
> LongString has 255. Both are empty at the time of the screenshot. Maybe, the size should be limited to a 
> reasonable width.

I limited the input size now to (arbitrarily) 100 chars. The string can still be longer than 100 chars, and you start then scrolling inside the input box. Let me know if 
that's ok this way.

Stefan

> > Actual result:
> > The key picker does not close.
> 
> Thanks for reporting that bug. It has been fixed in the current commit (installed already on megon02)
>

Stefan, thank you for fixing both problems, I have seen them, too, but no time to deal with them.

K.O.

> > Another more minor visual problem is the edit-on-start dialog. There seems to be no upper bound to the 
> > size of the text box. In the attached screenshot, ShortString has a maximum length of 32 characters, 
> > LongString has 255. Both are empty at the time of the screenshot. Maybe, the size should be limited to a 
> > reasonable width.
> 
> I limited the input size now to (arbitrarily) 100 chars. The string can still be longer than 100 chars, and you start then scrolling inside the input box. Let me know if 
> that's ok this way.

I am moving the dragon experiment to the new midas and we see this problem on the begin-of-run page.

Old midas: no horizontal scroll bar, edit-on-start names, values and comments are all squeezed in into the visible frame.

New midas: page is very wide, values entry fields are very long and there is a horizontal scroll bar.

So something got broken in the htlm formatting or sizing. I should be able to spot the change by doing
a diff between old resources/start.html and the new one.

K.O.

Hello,

I have a question about length of the name of ODB key.
Is it possible to create an ODB key containing more than 32 characters?

Thanks.
Ruslan

No. But if you need more than 32 characters, you do something wrong. The 
information you want to put into the ODB key name should probably be stored in 
another string key or so.

Stefan


> Hello,
> 
> I have a question about length of the name of ODB key.
> Is it possible to create an ODB key containing more than 32 characters?
> 
> Thanks.
> Ruslan

Trying to play with midas file but I get error:

[Analyzer,ERROR] [odb.cxx:845:db_validate_name,ERROR] Invalid name "/Analyzer/Tests/low_sum/Rate [Hz]" passed to db_create_key_wlocked: should not contain "["

I'm not sure what sets the name so I'm not sure how to fix this.

Thanks

Dear expert,
      I'm developing a frontend and I'm getting this kind of error at each event:

10:14:56.564 2018/05/04 [Sample Frontend,ERROR] [odb.c:5911:db_set_data1,ERROR]
online database full

If I run the mem command in odbedit I get the result at the end of this post.

Notice that I need to use an event size which is significantly larger than the
default one. I don't know if it is relevant for this error. I have in the ODB:

  /Experiment/MAX_EVENT_SIZE = 900000000

and in the frontend code:

  /* maximum event size produced by this frontend */
  INT max_event_size = 300000000;

  /* maximum event size for fragmented events (EQ_FRAGMENTED) */
  INT max_event_size_frag = 5 * 1024 * 1024;

  /* buffer size to hold events */
  INT event_buffer_size = 600000000;

Events seem to be properly stored in the output files, but I'm afraid I could
get some other problem.

Thank you for your help,
         Francesco

-------------------------------------------------------------------------

Database header size is 0x21040, all following values are offset by this!
Key area  0x00000000 - 0x0007FFFF, size 524288 bytes
Data area 0x00080000 - 0x00100000, size 524288 bytes

Keylist:
--------
Free block at 0x00000B58, size 0x00000008, next 0x000053E0
Free block at 0x000053E0, size 0x00000008, next 0x00006560
Free block at 0x00006560, size 0x00079AA0, next 0x00000000

Free Key area: 498352 bytes out of 524288 bytes

Data:
-----
Free block at 0x000847F0, size 0x0007B810, next 0x00000000

Free Data area: 505872 bytes out of 524288 bytes

Free: 498352 (95.1%) keylist, 505872 (96.5%) data

Two options:

1) Do NOT send your events into the ODB. This is controlled via the flag RO_ODB in your frontend setting. For simple experiments with small events, it might make sense to copy each 
event into the ODB for debugging, but if you have large events, this does not make sense. Use the "mdump" utility to check your events instead.

2) Increase the size of the ODB. See the first FAQ here: https://midas.triumf.ca/MidasWiki/index.php/FAQ

Stefan


> Dear expert,
>       I'm developing a frontend and I'm getting this kind of error at each event:
> 
> 10:14:56.564 2018/05/04 [Sample Frontend,ERROR] [odb.c:5911:db_set_data1,ERROR]
> online database full
> 
> If I run the mem command in odbedit I get the result at the end of this post.
> 
> Notice that I need to use an event size which is significantly larger than the
> default one. I don't know if it is relevant for this error. I have in the ODB:
> 
>   /Experiment/MAX_EVENT_SIZE = 900000000
> 
> and in the frontend code:
> 
>   /* maximum event size produced by this frontend */
>   INT max_event_size = 300000000;
> 
>   /* maximum event size for fragmented events (EQ_FRAGMENTED) */
>   INT max_event_size_frag = 5 * 1024 * 1024;
> 
>   /* buffer size to hold events */
>   INT event_buffer_size = 600000000;
> 
> Events seem to be properly stored in the output files, but I'm afraid I could
> get some other problem.
> 
> Thank you for your help,
>          Francesco
> 
> -------------------------------------------------------------------------
> 
> Database header size is 0x21040, all following values are offset by this!
> Key area  0x00000000 - 0x0007FFFF, size 524288 bytes
> Data area 0x00080000 - 0x00100000, size 524288 bytes
> 
> Keylist:
> --------
> Free block at 0x00000B58, size 0x00000008, next 0x000053E0
> Free block at 0x000053E0, size 0x00000008, next 0x00006560
> Free block at 0x00006560, size 0x00079AA0, next 0x00000000
> 
> Free Key area: 498352 bytes out of 524288 bytes
> 
> Data:
> -----
> Free block at 0x000847F0, size 0x0007B810, next 0x00000000
> 
> Free Data area: 505872 bytes out of 524288 bytes
> 
> Free: 498352 (95.1%) keylist, 505872 (96.5%) data

Concurrence.

Normally, MIDAS data events are saved to ODB (via RO_ODB into /eq/xxx/variables) to make them go into the midas history (/eq/xxx/common/history > 0).

If you do not want events to go into the history, but still want them saved to ODB, it should work (as long as ODB itself
is big enough), but you may run into other problems, specifically ODB free space fragmentation, when no matter how big ODB is, there is never
enough continuous free space for saving a large event. If it happens you will also see random "odb full" errors.

K.O.



> Two options:
> 
> 1) Do NOT send your events into the ODB. This is controlled via the flag RO_ODB in your frontend setting. For simple experiments with small events, it might make sense to copy each 
> event into the ODB for debugging, but if you have large events, this does not make sense. Use the "mdump" utility to check your events instead.
> 
> 2) Increase the size of the ODB. See the first FAQ here: https://midas.triumf.ca/MidasWiki/index.php/FAQ
> 
> Stefan
> 
> 
> > Dear expert,
> >       I'm developing a frontend and I'm getting this kind of error at each event:
> > 
> > 10:14:56.564 2018/05/04 [Sample Frontend,ERROR] [odb.c:5911:db_set_data1,ERROR]
> > online database full
> > 
> > If I run the mem command in odbedit I get the result at the end of this post.
> > 
> > Notice that I need to use an event size which is significantly larger than the
> > default one. I don't know if it is relevant for this error. I have in the ODB:
> > 
> >   /Experiment/MAX_EVENT_SIZE = 900000000
> > 
> > and in the frontend code:
> > 
> >   /* maximum event size produced by this frontend */
> >   INT max_event_size = 300000000;
> > 
> >   /* maximum event size for fragmented events (EQ_FRAGMENTED) */
> >   INT max_event_size_frag = 5 * 1024 * 1024;
> > 
> >   /* buffer size to hold events */
> >   INT event_buffer_size = 600000000;
> > 
> > Events seem to be properly stored in the output files, but I'm afraid I could
> > get some other problem.
> > 
> > Thank you for your help,
> >          Francesco
> > 
> > -------------------------------------------------------------------------
> > 
> > Database header size is 0x21040, all following values are offset by this!
> > Key area  0x00000000 - 0x0007FFFF, size 524288 bytes
> > Data area 0x00080000 - 0x00100000, size 524288 bytes
> > 
> > Keylist:
> > --------
> > Free block at 0x00000B58, size 0x00000008, next 0x000053E0
> > Free block at 0x000053E0, size 0x00000008, next 0x00006560
> > Free block at 0x00006560, size 0x00079AA0, next 0x00000000
> > 
> > Free Key area: 498352 bytes out of 524288 bytes
> > 
> > Data:
> > -----
> > Free block at 0x000847F0, size 0x0007B810, next 0x00000000
> > 
> > Free Data area: 505872 bytes out of 524288 bytes
> > 
> > Free: 498352 (95.1%) keylist, 505872 (96.5%) data

MIDAS version:      2.1
GIT revision:       Tue Dec 31 17:40:14 2019 +0100 - midas-2019-09-i-1-gd93944ce-dirty on branch develop

/Logger/Channels/0/Settings

With the settings above the file last.json generated for a new run is empty and the events 0x8000 and 0x8001 are missing in the .mid file.

When setting "ODB dump format" to "xml", events 0x8000 and 0x8001 are included in the .mid file, however, the file last.xml is not created.

(Please post messages in "plain" mode, they are much easier to answer)

Thank you for reporting this problem. I will try to reproduce it.

In addition, I will say a few words about your version of midas:

> GIT revision: midas-2019-09-i-1-gd93944ce-dirty on branch develop

I recommend that for production systems one used the tagged release versions of midas. 
(i.e. see https://midas.triumf.ca/elog/Midas/1750).

(Your midas is "1 commit after the latest tag" - the "-1" in the git revision).

I apply bug fixes to both the release branch and the develop branch, but for you to get 
these fixes, on the develop branch you will also "get" all the unrelated changes that may 
come with new bugs. On the release branch, you will only get the bug fixes.

In your midas version it says "-dirty" which means that you have local modifications to the 
midas sources. With luck those changes are not related to the bug that you see. (but I 
cannot tell). You can do "git status" and "git diff" to see what the local changes are.

It is much better if bugs are reported against "clean" builds of MIDAS (no "-dirty").


K.O.


<p>&nbsp;</p>

<table align="center" cellspacing="1" style="border:1px solid #486090; width:98%">
	<tbody>
		<tr>
			<td style="background-color:#486090">Peter Kunz wrote:</td>
		</tr>
		<tr>
			<td style="background-color:#FFFFB0">
			<p>MIDAS version: &nbsp; &nbsp; &nbsp;2.1<br />
			GIT revision: &nbsp; &nbsp; &nbsp; Tue Dec 31 17:40:14 2019 +0100 - 
midas-2019-09-i-1-gd93944ce-dirty on branch develop</p>

			<p>/Logger/Channels/0/Settings</p>

			<table border="3" cellpadding="1" class="dialogTable">
				<tbody>
					<tr>
						<td>ODB dump</td>
						<td>y</td>
					</tr>
					<tr>
						<td>ODB dump format</td>
						<td>json</td>
					</tr>
				</tbody>
			</table>

			<p>With the settings above the file last.json generated for a new run is 
empty and the events 0x8000 and 0x8001 are missing in the .mid file.</p>

			<p>When setting &quot;ODB dump format&quot; to &quot;xml&quot;, events 
0x8000 and 0x8001 are included in the .mid file, however, the file last.xml is not created.
</p>

			<p>&nbsp;</p>

			<p>&nbsp;</p>

			<p>&nbsp;</p>
			</td>
		</tr>
	</tbody>
</table>

<p>&nbsp;</p>

Re: MIDAS versions

Thanks for pointing that out. I wasn't actually aware that there is a release branch and a development branch.
I was just following the installation instructions using

git clone https://bitbucket.org/tmidas/midas --recursive

Apparently that gave me the development branch. How can I get the release version?

(I think my version showed up a "dirty" because I played around with one of the examples. I didn't touch the actual source code.)

> (Please post messages in "plain" mode, they are much easier to answer)
> 
> Thank you for reporting this problem. I will try to reproduce it.
> 
> In addition, I will say a few words about your version of midas:
> 
> > GIT revision: midas-2019-09-i-1-gd93944ce-dirty on branch develop
> 
> I recommend that for production systems one used the tagged release versions of midas. 
> (i.e. see https://midas.triumf.ca/elog/Midas/1750).
> 
> (Your midas is "1 commit after the latest tag" - the "-1" in the git revision).
> 
> I apply bug fixes to both the release branch and the develop branch, but for you to get 
> these fixes, on the develop branch you will also "get" all the unrelated changes that may 
> come with new bugs. On the release branch, you will only get the bug fixes.
> 
> In your midas version it says "-dirty" which means that you have local modifications to the 
> midas sources. With luck those changes are not related to the bug that you see. (but I 
> cannot tell). You can do "git status" and "git diff" to see what the local changes are.
> 
> It is much better if bugs are reported against "clean" builds of MIDAS (no "-dirty").
> 
> 
> K.O.
> 
> 
> <p>&nbsp;</p>
> 
> <table align="center" cellspacing="1" style="border:1px solid #486090; width:98%">
> 	<tbody>
> 		<tr>
> 			<td style="background-color:#486090">Peter Kunz wrote:</td>
> 		</tr>
> 		<tr>
> 			<td style="background-color:#FFFFB0">
> 			<p>MIDAS version: &nbsp; &nbsp; &nbsp;2.1<br />
> 			GIT revision: &nbsp; &nbsp; &nbsp; Tue Dec 31 17:40:14 2019 +0100 - 
> midas-2019-09-i-1-gd93944ce-dirty on branch develop</p>
> 
> 			<p>/Logger/Channels/0/Settings</p>
> 
> 			<table border="3" cellpadding="1" class="dialogTable">
> 				<tbody>
> 					<tr>
> 						<td>ODB dump</td>
> 						<td>y</td>
> 					</tr>
> 					<tr>
> 						<td>ODB dump format</td>
> 						<td>json</td>
> 					</tr>
> 				</tbody>
> 			</table>
> 
> 			<p>With the settings above the file last.json generated for a new run is 
> empty and the events 0x8000 and 0x8001 are missing in the .mid file.</p>
> 
> 			<p>When setting &quot;ODB dump format&quot; to &quot;xml&quot;, events 
> 0x8000 and 0x8001 are included in the .mid file, however, the file last.xml is not created.
> </p>
> 
> 			<p>&nbsp;</p>
> 
> 			<p>&nbsp;</p>
> 
> 			<p>&nbsp;</p>
> 			</td>
> 		</tr>
> 	</tbody>
> </table>
> 
> <p>&nbsp;</p>

For using the release branch read the messages in this thread. Most of the time, the develop branch is fine, except when we are developing something 
new, and the only way to tell is to watch the git activity on bitbucket or see the release branch announcements I post on the midas forum.

https://midas.triumf.ca/elog/Midas/1706

K.O.


> Re: MIDAS versions
> 
> Thanks for pointing that out. I wasn't actually aware that there is a release branch and a development branch.
> I was just following the installation instructions using
> 
> git clone https://bitbucket.org/tmidas/midas --recursive
> 
> Apparently that gave me the development branch. How can I get the release version?
> 
> (I think my version showed up a "dirty" because I played around with one of the examples. I didn't touch the actual source code.)
> 
> > (Please post messages in "plain" mode, they are much easier to answer)
> > 
> > Thank you for reporting this problem. I will try to reproduce it.
> > 
> > In addition, I will say a few words about your version of midas:
> > 
> > > GIT revision: midas-2019-09-i-1-gd93944ce-dirty on branch develop
> > 
> > I recommend that for production systems one used the tagged release versions of midas. 
> > (i.e. see https://midas.triumf.ca/elog/Midas/1750).
> > 
> > (Your midas is "1 commit after the latest tag" - the "-1" in the git revision).
> > 
> > I apply bug fixes to both the release branch and the develop branch, but for you to get 
> > these fixes, on the develop branch you will also "get" all the unrelated changes that may 
> > come with new bugs. On the release branch, you will only get the bug fixes.
> > 
> > In your midas version it says "-dirty" which means that you have local modifications to the 
> > midas sources. With luck those changes are not related to the bug that you see. (but I 
> > cannot tell). You can do "git status" and "git diff" to see what the local changes are.
> > 
> > It is much better if bugs are reported against "clean" builds of MIDAS (no "-dirty").
> > 
> > 
> > K.O.
> > 
> > 
> > <p>&nbsp;</p>
> > 
> > <table align="center" cellspacing="1" style="border:1px solid #486090; width:98%">
> > 	<tbody>
> > 		<tr>
> > 			<td style="background-color:#486090">Peter Kunz wrote:</td>
> > 		</tr>
> > 		<tr>
> > 			<td style="background-color:#FFFFB0">
> > 			<p>MIDAS version: &nbsp; &nbsp; &nbsp;2.1<br />
> > 			GIT revision: &nbsp; &nbsp; &nbsp; Tue Dec 31 17:40:14 2019 +0100 - 
> > midas-2019-09-i-1-gd93944ce-dirty on branch develop</p>
> > 
> > 			<p>/Logger/Channels/0/Settings</p>
> > 
> > 			<table border="3" cellpadding="1" class="dialogTable">
> > 				<tbody>
> > 					<tr>
> > 						<td>ODB dump</td>
> > 						<td>y</td>
> > 					</tr>
> > 					<tr>
> > 						<td>ODB dump format</td>
> > 						<td>json</td>
> > 					</tr>
> > 				</tbody>
> > 			</table>
> > 
> > 			<p>With the settings above the file last.json generated for a new run is 
> > empty and the events 0x8000 and 0x8001 are missing in the .mid file.</p>
> > 
> > 			<p>When setting &quot;ODB dump format&quot; to &quot;xml&quot;, events 
> > 0x8000 and 0x8001 are included in the .mid file, however, the file last.xml is not created.
> > </p>
> > 
> > 			<p>&nbsp;</p>
> > 
> > 			<p>&nbsp;</p>
> > 
> > 			<p>&nbsp;</p>
> > 			</td>
> > 		</tr>
> > 	</tbody>
> > </table>
> > 
> > <p>&nbsp;</p>

Dear midas users,

I have changed the ODB dump format to XML. As you might know, the logger writes
a special begin-of-run event to the .mid file which includes an ASCII dump of
the ODB. The same at the end-of-run. To read these ODB dumps back in offline
analysis, this requires setting up a ODB just to read back these values. In
order to avoid this, we switched the format to XML instead of the old ASCII
format. That way ROME can read the ODB dump and extract individual values from
it without setting up a shared memory.

A similar thing has been made for the ODB dumps to separate .odb files, which
are controlled by "/Logger/ODB Dump" and "/Logger/ODB Dump file". If the dump
file has the extension .xml, the file is dumped in XML format as well.

All the XML functionality is implemented in the new mxml.c/h library, which has
been added to the distribution, and which can be used in other projects as well
(XML configuration of ROODY?). It has already been successfully implemented in
ROME, so ROME is no longer dependent on libxml.

- Stefan

> All the XML functionality is implemented in the new mxml.c/h library, which has
> been added to the distribution, and which can be used in other projects as well
> (XML configuration of ROODY?). It has already been successfully implemented in
> ROME, so ROME is no longer dependent on libxml.

Since mxml.c/h is used in several projects (midas, ROME, elog), I separated it's
CVS tree. So in order to compile midas from scratch, you have to check out midas
AND mxml like

cd ~
cvs -d :ext:cvs@midas.psi.ch:/usr/local/cvsroot checkout midas
cvs -d :ext:cvs@midas.psi.ch:/usr/local/cvsroot checkout mxml

cd midas
make

so the "mxml" tree is ABOVE the "midas" tree. The midas Makefile has been adjusted
accordingly. If you decide to put the mxml somwhere else, you have to change
MXML_DIR in the Makefile accordingly.

- Stefan