> Hi. Is it currently possible to automatically save the MIDAS ODB as a JSON file?
> I can do it manually in odbedit, but it looks like the only option for the
> automatic ODB save for each run is the standard .ODB format. Is there a way to
> change this?

I have the reverse question: Once we have a json file, can it be loaded again in the odb? We also use the odb as our analysis config. I don`t like .odb files in our repo, so we currently load settings in shell scripts (e.g. run dependent detector calibrations)

Is there a simple way from the javascript side to check if a fontend is running?
Currently one would need to go through the /System/Client list to find out if a
frontend/client is running. Wouldn't it be nice to have this centralized, either
in the mhttpd.cxx or mhttpd.js part? 

> Is there a simple way from the javascript side to check if a fontend is running?
> Currently one would need to go through the /System/Client list to find out if a
> frontend/client is running. Wouldn't it be nice to have this centralized, either
> in the mhttpd.cxx or mhttpd.js part? 

Hi,

I think that this option already exists with the cm_exist method for the mjsonrpc calls.  For instance, you can use a 
call like 

curl -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","id":null,"method":"cm_exist","params":
{"name":"Logger"}}' 'http://localhost:8081?mjsonrpc'

to get the status of the logger program.  There is a description of the cm_exist parameters on this page:

https://midas.triumf.ca/MidasWiki/index.php/Mjsonrpc

> > Is there a simple way from the javascript side to check if a fontend is running?
> > Currently one would need to go through the /System/Client list to find out if a
> > frontend/client is running. Wouldn't it be nice to have this centralized, either
> > in the mhttpd.cxx or mhttpd.js part? 
> 
> Hi,
> 
> I think that this option already exists with the cm_exist method for the mjsonrpc calls.  For instance, you can use a 
> call like 
> 
> curl -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","id":null,"method":"cm_exist","params":
> {"name":"Logger"}}' 'http://localhost:8081?mjsonrpc'
> 
> to get the status of the logger program.  There is a description of the cm_exist parameters on this page:
> 
> https://midas.triumf.ca/MidasWiki/index.php/Mjsonrpc

Thanks a lot for the info. I just simply missed it :-|

I started to use the new midas custom page features from 
https://midas.triumf.ca/MidasWiki/index.php/Custom_Page . I'd like to setup the 
editable odb values (.e.g <div name="modbvalue" data-odb-path="/Runinfo/Run 
number" data-odb-editable="1"                    
style="position:absolute;top:157px;left:288px;"></div>) from within javascript, 
which doesn`t seem to work. 

 Both 

document.getElementById("test").innerHTML = '<div data-odb-editable="1" data-
odb-path="/Runinfo/Run number" name="modbvalue" 
style="display:inline;position:absolute;top:157px;left:288px;"></div>'

or 

var elem = document.createElement("div");
	 //var id = "manifold0";
	    elem.setAttribute("name","modbvalue");
	 elem.setAttribute("data-odb-
path","/Equipment/Autofill/Variables/Output[6]");
	 elem.setAttribute("data-odb-editable","1");
	 elem.style="position:absolute;top:157px;left:288px;";
         document.getElementById("test").appendChild(elem);

fail on name="modbvalue" with the error 

mjsonrpc_error_alert: TypeError: Cannot set property 'innerHTML' of undefined


How should one do this? I don`t want hard code everything in the htlm body, as 
I have look up odb key indexes in the javascript code.

The function mhttpd_init() scans the custom page and installs handlers etc. for each modbxxx 
element. If you create an modbvalue dynamically in your code, this is probably done after you 
called mhttpd_init(), so the function has no chance to modify the dynamically created element.

To fix that, I separated mhttpd_init() into the old init function which installs the header and sidebar, 
and a function mhttpd_scan() which scans the custom page and processes all modbxxx elements. 
Next, I tapped the error you reported, and added an automatic call to mhttp_scan() in case that 
happens. I tried it on a test page and it worked for me. Please give it a try (commit 090394e8).

Stefan

> The function mhttpd_init() scans the custom page and installs handlers etc. for each modbxxx 
> element. If you create an modbvalue dynamically in your code, this is probably done after you 
> called mhttpd_init(), so the function has no chance to modify the dynamically created element.
> 
> To fix that, I separated mhttpd_init() into the old init function which installs the header and sidebar, 
> and a function mhttpd_scan() which scans the custom page and processes all modbxxx elements. 
> Next, I tapped the error you reported, and added an automatic call to mhttp_scan() in case that 
> happens. I tried it on a test page and it worked for me. Please give it a try (commit 090394e8).
> 
> Stefan

Also works for me

 thanks

> > The function mhttpd_init() scans the custom page and installs handlers etc. for each modbxxx 
> > element. If you create an modbvalue dynamically in your code, this is probably done after you 
> > called mhttpd_init(), so the function has no chance to modify the dynamically created element.
> > 
> > To fix that, I separated mhttpd_init() into the old init function which installs the header and sidebar, 
> > and a function mhttpd_scan() which scans the custom page and processes all modbxxx elements. 
> > Next, I tapped the error you reported, and added an automatic call to mhttp_scan() in case that 
> > happens. I tried it on a test page and it worked for me. Please give it a try (commit 090394e8).
> > 
> > Stefan
> 
> Also works for me
> 
>  thanks


This is more about aesthetic, but when I use the modbvalue div, it first only shows the odb value. However, 
after editing, the odb index gets added to the field, which is kinda ugly -> see attachments

Dear expert,
      I'm developing a frontend and I'm getting this kind of error at each event:

10:14:56.564 2018/05/04 [Sample Frontend,ERROR] [odb.c:5911:db_set_data1,ERROR]
online database full

If I run the mem command in odbedit I get the result at the end of this post.

Notice that I need to use an event size which is significantly larger than the
default one. I don't know if it is relevant for this error. I have in the ODB:

  /Experiment/MAX_EVENT_SIZE = 900000000

and in the frontend code:

  /* maximum event size produced by this frontend */
  INT max_event_size = 300000000;

  /* maximum event size for fragmented events (EQ_FRAGMENTED) */
  INT max_event_size_frag = 5 * 1024 * 1024;

  /* buffer size to hold events */
  INT event_buffer_size = 600000000;

Events seem to be properly stored in the output files, but I'm afraid I could
get some other problem.

Thank you for your help,
         Francesco

-------------------------------------------------------------------------

Database header size is 0x21040, all following values are offset by this!
Key area  0x00000000 - 0x0007FFFF, size 524288 bytes
Data area 0x00080000 - 0x00100000, size 524288 bytes

Keylist:
--------
Free block at 0x00000B58, size 0x00000008, next 0x000053E0
Free block at 0x000053E0, size 0x00000008, next 0x00006560
Free block at 0x00006560, size 0x00079AA0, next 0x00000000

Free Key area: 498352 bytes out of 524288 bytes

Data:
-----
Free block at 0x000847F0, size 0x0007B810, next 0x00000000

Free Data area: 505872 bytes out of 524288 bytes

Free: 498352 (95.1%) keylist, 505872 (96.5%) data

Two options:

1) Do NOT send your events into the ODB. This is controlled via the flag RO_ODB in your frontend setting. For simple experiments with small events, it might make sense to copy each 
event into the ODB for debugging, but if you have large events, this does not make sense. Use the "mdump" utility to check your events instead.

2) Increase the size of the ODB. See the first FAQ here: https://midas.triumf.ca/MidasWiki/index.php/FAQ

Stefan


> Dear expert,
>       I'm developing a frontend and I'm getting this kind of error at each event:
> 
> 10:14:56.564 2018/05/04 [Sample Frontend,ERROR] [odb.c:5911:db_set_data1,ERROR]
> online database full
> 
> If I run the mem command in odbedit I get the result at the end of this post.
> 
> Notice that I need to use an event size which is significantly larger than the
> default one. I don't know if it is relevant for this error. I have in the ODB:
> 
>   /Experiment/MAX_EVENT_SIZE = 900000000
> 
> and in the frontend code:
> 
>   /* maximum event size produced by this frontend */
>   INT max_event_size = 300000000;
> 
>   /* maximum event size for fragmented events (EQ_FRAGMENTED) */
>   INT max_event_size_frag = 5 * 1024 * 1024;
> 
>   /* buffer size to hold events */
>   INT event_buffer_size = 600000000;
> 
> Events seem to be properly stored in the output files, but I'm afraid I could
> get some other problem.
> 
> Thank you for your help,
>          Francesco
> 
> -------------------------------------------------------------------------
> 
> Database header size is 0x21040, all following values are offset by this!
> Key area  0x00000000 - 0x0007FFFF, size 524288 bytes
> Data area 0x00080000 - 0x00100000, size 524288 bytes
> 
> Keylist:
> --------
> Free block at 0x00000B58, size 0x00000008, next 0x000053E0
> Free block at 0x000053E0, size 0x00000008, next 0x00006560
> Free block at 0x00006560, size 0x00079AA0, next 0x00000000
> 
> Free Key area: 498352 bytes out of 524288 bytes
> 
> Data:
> -----
> Free block at 0x000847F0, size 0x0007B810, next 0x00000000
> 
> Free Data area: 505872 bytes out of 524288 bytes
> 
> Free: 498352 (95.1%) keylist, 505872 (96.5%) data

I am embedding histories into a custom page within an SVG,

<image x="21000" y="1000" width="6000" height="6000"
href="../HS/SampleCryo/SampleTemp.gif?width=230&scale=0.5h"/>

this works fine. However, I would like to update this regularly without
refreshing the full page via

<meta http-equiv="Refresh" content="60">

is there a good way to do that? By the way, the "Periodic update of parts of a
custom page" from the documentation does not seem to work here.

I am setting up a sqlite db to serve as a run database.

The easiest option is to use the history sqlite feature, and add run information 
as virtual history events

however:

Invalid tag 0 'Comment' in event 21 'Run Parameters': cannot do history for 
TID_STRING data, sorry!

I'd like to save e.g. the edit on start information , with shift crew checks. 
Would it be easy to allow for text, or is this inherent to the history system 
handling binary data?

I am implementing buttons to launch scripts from a custom page. 

The simple way works, i.e.

  <input type=submit name=customscript value="run_script">

But I want to stay on the page. Copying "Customscript button without a page 
reload" from https://midas.triumf.ca/MidasWiki/index.php/Custom_Page_Features 
yields the following error:

Uncaught ReferenceError: XMLHttpRequestGeneric is not defined
    at cs_button (Trend:165)
    at HTMLInputElement.onclick (Trend:90)  

I included  <script src="mhttpd.js"></script> and call mhttpd_init on page load.

So why can`t it run this ajax request?


Or is there a better way to launch a script without messing up the page

> Uncaught ReferenceError: XMLHttpRequestGeneric is not defined
>     at cs_button (Trend:165)
>     at HTMLInputElement.onclick (Trend:90)  

That code was not written by me, so I'm must guessing here.

Probably the XMLHttpRequestGeneric() is some function hiding browser specialities to create 
AJAX requests. These days most browser understand the standard request

XMLHttpRequest()

so why don't you try to just remove the "Generic"

Stefan

> > Uncaught ReferenceError: XMLHttpRequestGeneric is not defined
> >     at cs_button (Trend:165)
> >     at HTMLInputElement.onclick (Trend:90)  
> 
> That code was not written by me, so I'm must guessing here.
> 
> Probably the XMLHttpRequestGeneric() is some function hiding browser specialities to create 
> AJAX requests. These days most browser understand the standard request
> 
> XMLHttpRequest()
> 
> so why don't you try to just remove the "Generic"
> 
> Stefan

That removes the error, but script doesnt get called. It goes to the javascript function and 
callback, but nothing happens. 

When I change type=button to type=submit , the script gets called again, but with page refresh. 

We run as follows:

* sis3316 digitizers in a vme crate
* 1-2 midas events /s
* data rate at 20 MB/s

At a rate of 30 MB/s the daq crashed because the I think the mlogger can`t follow:

  * it runs at 100% cpu
  * memory usage of mlogger process goes from 2% to 15%
  * All other processes < 50 % cpu and < 20% RAM

Both the vme frontend and the mlogger crash about 2.5 minutes into a run. Both
the logger and vme fe spit out:
bm_validate_client_pointers: Assertion `pclient->read_pointer >= 0 &&
pclient->read_pointer <= pheader->size' failed.
Aborted

I first thought that writing-to-disk could be a bottle neck. But when I write to
an SSD, same thing.

Is there another bottleneck which keeps the mlogger busy?

> Invalid tag 0 'Comment' in event 21 'Run Parameters': cannot do history for 
> TID_STRING data, sorry!

The original MIDAS history API does not have provisions for storing TID_STRING data,
it is a very unfortunate limitation that has been with us for a very long time.

If I ever get around to rewrite the MIDAS history API, I will definitely add support for TID_STRING data.

But not today.

K.O.

P.S. Support for arbitrary binary blobs is also possible, but this will make the midas history
a kind of "a daq inside the daq" thing, probably we do not want to go this direction.

K.O.

Concurrence.

Normally, MIDAS data events are saved to ODB (via RO_ODB into /eq/xxx/variables) to make them go into the midas history (/eq/xxx/common/history > 0).

If you do not want events to go into the history, but still want them saved to ODB, it should work (as long as ODB itself
is big enough), but you may run into other problems, specifically ODB free space fragmentation, when no matter how big ODB is, there is never
enough continuous free space for saving a large event. If it happens you will also see random "odb full" errors.

K.O.



> Two options:
> 
> 1) Do NOT send your events into the ODB. This is controlled via the flag RO_ODB in your frontend setting. For simple experiments with small events, it might make sense to copy each 
> event into the ODB for debugging, but if you have large events, this does not make sense. Use the "mdump" utility to check your events instead.
> 
> 2) Increase the size of the ODB. See the first FAQ here: https://midas.triumf.ca/MidasWiki/index.php/FAQ
> 
> Stefan
> 
> 
> > Dear expert,
> >       I'm developing a frontend and I'm getting this kind of error at each event:
> > 
> > 10:14:56.564 2018/05/04 [Sample Frontend,ERROR] [odb.c:5911:db_set_data1,ERROR]
> > online database full
> > 
> > If I run the mem command in odbedit I get the result at the end of this post.
> > 
> > Notice that I need to use an event size which is significantly larger than the
> > default one. I don't know if it is relevant for this error. I have in the ODB:
> > 
> >   /Experiment/MAX_EVENT_SIZE = 900000000
> > 
> > and in the frontend code:
> > 
> >   /* maximum event size produced by this frontend */
> >   INT max_event_size = 300000000;
> > 
> >   /* maximum event size for fragmented events (EQ_FRAGMENTED) */
> >   INT max_event_size_frag = 5 * 1024 * 1024;
> > 
> >   /* buffer size to hold events */
> >   INT event_buffer_size = 600000000;
> > 
> > Events seem to be properly stored in the output files, but I'm afraid I could
> > get some other problem.
> > 
> > Thank you for your help,
> >          Francesco
> > 
> > -------------------------------------------------------------------------
> > 
> > Database header size is 0x21040, all following values are offset by this!
> > Key area  0x00000000 - 0x0007FFFF, size 524288 bytes
> > Data area 0x00080000 - 0x00100000, size 524288 bytes
> > 
> > Keylist:
> > --------
> > Free block at 0x00000B58, size 0x00000008, next 0x000053E0
> > Free block at 0x000053E0, size 0x00000008, next 0x00006560
> > Free block at 0x00006560, size 0x00079AA0, next 0x00000000
> > 
> > Free Key area: 498352 bytes out of 524288 bytes
> > 
> > Data:
> > -----
> > Free block at 0x000847F0, size 0x0007B810, next 0x00000000
> > 
> > Free Data area: 505872 bytes out of 524288 bytes
> > 
> > Free: 498352 (95.1%) keylist, 505872 (96.5%) data