> The function mhttpd_init() scans the custom page and installs handlers etc. for each modbxxx 
> element. If you create an modbvalue dynamically in your code, this is probably done after you 
> called mhttpd_init(), so the function has no chance to modify the dynamically created element.
> 
> To fix that, I separated mhttpd_init() into the old init function which installs the header and sidebar, 
> and a function mhttpd_scan() which scans the custom page and processes all modbxxx elements. 
> Next, I tapped the error you reported, and added an automatic call to mhttp_scan() in case that 
> happens. I tried it on a test page and it worked for me. Please give it a try (commit 090394e8).
> 
> Stefan

Also works for me

 thanks

I wanted to try to summarize the current situation with regards to the handling of strings through the MIDAS web interface.

During the last year, we started the switch-over to using the new mjson-rpc functions in the MIDAS webpages.  As part of this work, changes were made that allowed for 
resizing strings through the MIDAS web interface (specifically through the MJSON-RPC calls). This reflected desires from some users that strings could be allowed to 
grow larger than the default 256 size that is usually used for MIDAS strings (for instance, see [1]).  In this first set of changes the ODB strings would always be resized to 
the exact size of the string that it was set to.

The problem with this change was that it breaks the behaviour of db_get_record(), which typically expects strings to be a fixed length of 32 or 256 (for instance, see 
[2,3]).  Konstantin notes that we can work around this problem by using db_get_record1() function, which automatically resizes strings to expected values; this method 
has already been used in the MIDAS core code.  But the problem would still remain for some user code that uses db_get_record.

As a short-term compromise solution, Stefan implemented a change where the MJSON-RPC string setting will now expand the size of strings, but not truncate them; ie a 
string that is size 256 will stay 256 bytes, unless you set it to something larger.  So this should fix most cases of user code that call db_get_record() and hence expects 
fixed string lengths. Users who call that db_get_record with strings that exceed the expected length will still have problems; but now you will at least get an explicit 
MIDAS error message, rather than just silent string truncation (as was the case before).

In the longer run we still want to develop a new set of ODB methods that more naturally support C++-style variable-length strings. But that's a discussion for the longer 
term.

[1] https://midas.triumf.ca/elog/Midas/1150
[2] https://bitbucket.org/tmidas/midas/issues/121/odb-inline-editor-truncates-stings
[3] https://midas.triumf.ca/elog/Midas/1343

The function mhttpd_init() scans the custom page and installs handlers etc. for each modbxxx 
element. If you create an modbvalue dynamically in your code, this is probably done after you 
called mhttpd_init(), so the function has no chance to modify the dynamically created element.

To fix that, I separated mhttpd_init() into the old init function which installs the header and sidebar, 
and a function mhttpd_scan() which scans the custom page and processes all modbxxx elements. 
Next, I tapped the error you reported, and added an automatic call to mhttp_scan() in case that 
happens. I tried it on a test page and it worked for me. Please give it a try (commit 090394e8).

Stefan

I started to use the new midas custom page features from 
https://midas.triumf.ca/MidasWiki/index.php/Custom_Page . I'd like to setup the 
editable odb values (.e.g <div name="modbvalue" data-odb-path="/Runinfo/Run 
number" data-odb-editable="1"                    
style="position:absolute;top:157px;left:288px;"></div>) from within javascript, 
which doesn`t seem to work. 

 Both 

document.getElementById("test").innerHTML = '<div data-odb-editable="1" data-
odb-path="/Runinfo/Run number" name="modbvalue" 
style="display:inline;position:absolute;top:157px;left:288px;"></div>'

or 

var elem = document.createElement("div");
	 //var id = "manifold0";
	    elem.setAttribute("name","modbvalue");
	 elem.setAttribute("data-odb-
path","/Equipment/Autofill/Variables/Output[6]");
	 elem.setAttribute("data-odb-editable","1");
	 elem.style="position:absolute;top:157px;left:288px;";
         document.getElementById("test").appendChild(elem);

fail on name="modbvalue" with the error 

mjsonrpc_error_alert: TypeError: Cannot set property 'innerHTML' of undefined


How should one do this? I don`t want hard code everything in the htlm body, as 
I have look up odb key indexes in the javascript code.

> It might have worked some ~5 years ago, but it never really showed the target value of a link, just the 
> link itself. I reworked the code now to show both the link and the target of the link, so you can change 
> both in the mhttpd ODB page. Should be consistent now with odbedit. Have a look if it works for you.
> 
> Stefan

Thank you. That has solved the problem.
Suzannah 

It might have worked some ~5 years ago, but it never really showed the target value of a link, just the 
link itself. I reworked the code now to show both the link and the target of the link, so you can change 
both in the mhttpd ODB page. Should be consistent now with odbedit. Have a look if it works for you.

Stefan

> > > A feature I always missed (or just missed to find in the docu) is the following:
> > > 1) It would be nice to have a command in odbedit which allows to check how full
> > > the ODB currently is.
> > > 2) Even more important: I would like to have an ODB routine which allows me to
> > > check the fill level of the ODB, and/or a routine which tells me if I would
> > > create a structure of given size that it still fits in the current ODB or not. 
> > > The use case is that some clients create on the fly ODB entries and I would like
> > > to make sure before hand the ODB's remaining space in order not to crash things
> > > by overfilling the ODB. 
> > 
> > If you do "mem" in odbedit, you see the currently free areas, one for the keys themselves, one for 
> > the data of the keys. The corresponding C function is db_show_mem. At the moment it outputs the 
> > list of free blocks as a long ASCII string, but if necessary I can write a variant which returns the 
> > number of free bytes.
> > 
> > Stefan
> 
> Thanks for the info, and yes a variant of db_show_mem returning the number of free bytes would be just
> prefect!

I made you db_get_free_mem(HNDLE hDB, INT *key_size, INT *data_size)

The first return gets you the number of free bytes for the key area, the second one for the data area (values of keys).

Committed to develop

Stefan

> > A feature I always missed (or just missed to find in the docu) is the following:
> > 1) It would be nice to have a command in odbedit which allows to check how full
> > the ODB currently is.
> > 2) Even more important: I would like to have an ODB routine which allows me to
> > check the fill level of the ODB, and/or a routine which tells me if I would
> > create a structure of given size that it still fits in the current ODB or not. 
> > The use case is that some clients create on the fly ODB entries and I would like
> > to make sure before hand the ODB's remaining space in order not to crash things
> > by overfilling the ODB. 
> 
> If you do "mem" in odbedit, you see the currently free areas, one for the keys themselves, one for 
> the data of the keys. The corresponding C function is db_show_mem. At the moment it outputs the 
> list of free blocks as a long ASCII string, but if necessary I can write a variant which returns the 
> number of free bytes.
> 
> Stefan

Thanks for the info, and yes a variant of db_show_mem returning the number of free bytes would be just
prefect!

> A feature I always missed (or just missed to find in the docu) is the following:
> 1) It would be nice to have a command in odbedit which allows to check how full
> the ODB currently is.
> 2) Even more important: I would like to have an ODB routine which allows me to
> check the fill level of the ODB, and/or a routine which tells me if I would
> create a structure of given size that it still fits in the current ODB or not. 
> The use case is that some clients create on the fly ODB entries and I would like
> to make sure before hand the ODB's remaining space in order not to crash things
> by overfilling the ODB. 

If you do "mem" in odbedit, you see the currently free areas, one for the keys themselves, one for 
the data of the keys. The corresponding C function is db_show_mem. At the moment it outputs the 
list of free blocks as a long ASCII string, but if necessary I can write a variant which returns the 
number of free bytes.

Stefan

A feature I always missed (or just missed to find in the docu) is the following:
1) It would be nice to have a command in odbedit which allows to check how full
the ODB currently is.
2) Even more important: I would like to have an ODB routine which allows me to
check the fill level of the ODB, and/or a routine which tells me if I would
create a structure of given size that it still fits in the current ODB or not. 
The use case is that some clients create on the fly ODB entries and I would like
to make sure before hand the ODB's remaining space in order not to crash things
by overfilling the ODB. 

> > Is there a simple way from the javascript side to check if a fontend is running?
> > Currently one would need to go through the /System/Client list to find out if a
> > frontend/client is running. Wouldn't it be nice to have this centralized, either
> > in the mhttpd.cxx or mhttpd.js part? 
> 
> Hi,
> 
> I think that this option already exists with the cm_exist method for the mjsonrpc calls.  For instance, you can use a 
> call like 
> 
> curl -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","id":null,"method":"cm_exist","params":
> {"name":"Logger"}}' 'http://localhost:8081?mjsonrpc'
> 
> to get the status of the logger program.  There is a description of the cm_exist parameters on this page:
> 
> https://midas.triumf.ca/MidasWiki/index.php/Mjsonrpc

Thanks a lot for the info. I just simply missed it :-|

> Is there a simple way from the javascript side to check if a fontend is running?
> Currently one would need to go through the /System/Client list to find out if a
> frontend/client is running. Wouldn't it be nice to have this centralized, either
> in the mhttpd.cxx or mhttpd.js part? 

Hi,

I think that this option already exists with the cm_exist method for the mjsonrpc calls.  For instance, you can use a 
call like 

curl -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","id":null,"method":"cm_exist","params":
{"name":"Logger"}}' 'http://localhost:8081?mjsonrpc'

to get the status of the logger program.  There is a description of the cm_exist parameters on this page:

https://midas.triumf.ca/MidasWiki/index.php/Mjsonrpc

Is there a simple way from the javascript side to check if a fontend is running?
Currently one would need to go through the /System/Client list to find out if a
frontend/client is running. Wouldn't it be nice to have this centralized, either
in the mhttpd.cxx or mhttpd.js part? 

Further to my last message, I see that a midas version from 2013 does indeed display
links to arrays as I would expect (see attachment). Therefore the problem in later
versions  is a bug rather than a feature.


> A link to an array variable such as
> 
> [local:npet:Stopped]/>ls /rcparams/ControlVariables/
> TRFC:PB5 (V) -> /Equipment/Beamline/Variables/Demand[56]
>                                 17835
> 
> displays the whole Demand array on the mhttpd ODB page (see attachment)
> rather than just the one element Demand[56].
> This behaviour also occurs with older versions of mhttpd.
>  
> Not sure if it's a bug or a feature, but my suggestion is that it
> ought to display the one element only (as odbedit does) and not the whole array.
> 
> Suzannah  

A link to an array variable such as

[local:npet:Stopped]/>ls /rcparams/ControlVariables/
TRFC:PB5 (V) -> /Equipment/Beamline/Variables/Demand[56]
                                17835

displays the whole Demand array on the mhttpd ODB page (see attachment)
rather than just the one element Demand[56].
This behaviour also occurs with older versions of mhttpd.
 
Not sure if it's a bug or a feature, but my suggestion is that it
ought to display the one element only (as odbedit does) and not the whole array.

Suzannah  

Hi Amy,

Let me start by explaining the reasoning for the default behavior of db_get_values.  I think it was mentioned elsewhere, but is worth repeating.

The ODB is case-insensitive.  So the ODB key name /Equipment/dcrc01 is equivalent to /equipment/Dcrc01; you could rename the variable like that and your 
frontend programs would still work fine.  Javascript, of course, is case sensitive.  However, we want our default MIDAS webpages to work no matter what the 
capitalization is for a particular ODB; so, for instance, the main status.html page should work whether the ODB key is called /Runinfo or /rUnInFo, since both 
of these are equivalent from the point of the ODB (and the rest of MIDAS).

The solution was to have the db_get_values method convert all key names to lower case and consistently use the lower case spelling when writing the main 
MIDAS webpages; this makes us insensitive to ODB capitalization (and hence makes the MIDAS pages behaviour match the previous mhttpd behaviour).

That being said, I agree that it is sometimes counter-intuitive to use lower case key names with db_get_values, particularly if you are directly creating ODB 
keys and writing the javascript at the same time.  So we have added the option 'preserve_case' to db_get_values, which preserves the ODB key name 
capitalization (the default behaviour is still to make key names lower case).

This option should not be used for writing any standard MIDAS webpages (ie, webpages that will be used across multiple experiments), since standard MIDAS 
webpages should not break when ODB key name capitalization changes.  For the same reason you should use caution with this option for custom pages as 
well.

With regards to your second question: the db_save method is not deprecated and you could use that method instead.  The use-case for the db_save method 
is different; db_save is used to make dumps of the ODB.  In that case it seems best that key name capitalization is preserved.  Otherwise if you dumped your 
whole ODB and then reloaded it from the dump the new ODB would be different (in key capitalization) from the old ODB; different in a way that shouldn't 
matter but still probably not the behaviour that people expect.

Admittedly this means that the mjsonrpc API is not always intuitive; but I think is the best we can do, given the underlying case-insensitivity of the ODB.

Thomas


> It appears I needed to read the documentation more closely - the method db_save
> does respect key-name capitalization and solves my problem.
> 
> Is db_save considered a deprecated method?  If so, I'd reiterate my suggestion for
> a capitalize-preserve option for db_get_values.
> 
> Otherwise, I'll plan on using db_save.
> 
> > I'd like to use the mjsonrpc db_get_values method, but (as indicated in the
> > documentation) it returns all ODB keys as lowercase.
> > 
> > This breaks quite a lot of my code - it was written with the old AJAX commands,
> > and these did respect the capitalization of the ODB keys.
> > 
> > Would it be possible to add a capitalization-preserve option to db_get_values? 

Hearing no objections I changed the name of the program to msequencer.  Wiki
documentation updated.

> Hi Folks,
> 
> In last year's updates to MIDAS, the MIDAS sequencer has been broken out as a
> separate program (rather than running as part of mhttpd).  We hope that this
> change will make the sequencer operation more stable.
> 
> Before anyone gets too used to using the new sequencer program, I would like to
> rename it.  Currently the program is called 'sequencer'; I would like to rename
> it 'msequencer', to make it consistent with most other MIDAS programs.  If you
> object to making this change, please say so in the next two weeks.
> 
> Documentation on the MIDAS sequencer can be found on the wiki:
> 
> https://midas.triumf.ca/MidasWiki/index.php/Sequencer
> 
> Note that there are still some tweaks that need to be made to the sequencer
> webpage and mhttpd in order to handle this new sequencer program.
> 
> Cheers,
> Thomas