The type INT has been defined in 1989 when I for the first time sent data between a 16-bit MS-DOS computer and a 32-bit VAX computer (good old 
days!). At that time, uint32_t was not available at all. So much for the historical background.

I agree that switching from INT to int32_t is getting closer to standards and might help new people better understand things. This means however to 
touch all midas files and change about 5000 (!) locations:

BYTE -> uint8_t
WORD -> uint16_t
DWORD -> uint32_t
INT -> int32_t

Next we have the midas data types TID_xxx?

The nice thing now is that for example WORD and TID_WORD belong together and this is obvious. For uint16_t and TID_WORD is is not so obvious 
any more, so I guess we should rename TID_WORD to TID_UINT16_t. The same fore 

TID_BYTE -> TID_UINT8_T
TID_SBYTE -> TID_INT8_T
TID_WORD -> TID_UINT16_T
TID_DWORD -> TID_UINT32_T
TID_INT -> TID_INT32_T

But if we changer TID_XXX, the ASCII representations of the ODB break compatibility! Right now we have for example

[/Experiment]
midas http port = INT : 8080

which will become

[/Experiment]
midas http port = INT32_T : 8080

so one cannot load old ODB files any more!

With JSON encoding it's better because only the type number is stored, not the string. So INT -> 7 could stay, although in my opinion encoding the 
type in an integer number is not good for readability. Nobody knows what "7" means as a type. You always have to do a look-up in midas.c and count 
array indices manually.

I'm not sure how many experiments use the ASCII ODB format in one way or the other in some custom scripts. It might be that changing the format 
might have severe side effects for some experiments, so before we undertake this endeavor I would like to get some feedback here on the forum 
about people from other experiments and see what they think.

Stefan

> > if (sizeof(INT) != 4) then severe_error_and_stop_all_programs()
> 
> Quick reply.
> 
> Today, for fixed size data types one should use uint32_t & co, see
> stdint.h
> https://en.wikipedia.org/wiki/C_data_types#stdint.h
> https://en.wikipedia.org/wiki/C99 (scroll down and click to open "implementation -> compiler support"
> 
> The other popular convention is "u32" used by the Linux kernel, you will see it in the linux kernel drivers.
> 
> If I remember right, WORD and DWORD grow legs from the 16-bit Motorolla 68xxx processors,
> VxWorks and the VME bus. At some point the data buses were 16-bit wide and that we the WORD.
> 
> (I do not think UNIX ever used the WORD/DWORD names, i.e. MacOS has int32_t and u_int32_t).
> 
> K.O.

Currently one has to set the required flag in the ODB (e.g., /Programs/Logger/Required) to "y" for the program 
to appear on the "Programs page" and being able to start and stop the program easily.

However, if one wants to run with the "Prevent start on required progs" in /Experiment enabled, all the 
programs in the "Programs page" need to be running and one cannot have one of them stopped while still 
taking a run.

It would be nice to separate these two functionalities: Have a flag that makes the program appear on the 
"Programs page" and have a flag that controls the "Prevent start on required frogs" functionality.

> Currently one has to set the required flag in the ODB (e.g., /Programs/Logger/Required) to "y" for the program 
> to appear on the "Programs page" and being able to start and stop the program easily.
> 
> However, if one wants to run with the "Prevent start on required progs" in /Experiment enabled, all the 
> programs in the "Programs page" need to be running and one cannot have one of them stopped while still 
> taking a run.
> 
> It would be nice to separate these two functionalities: Have a flag that makes the program appear on the 
> "Programs page" and have a flag that controls the "Prevent start on required frogs" functionality.

I agree. All the programs should be always visible on the "programs" page, there should be /Programs/xxx/hidden to 
hide them, and /Programs/xxx/required should be used for "Prevent start on required progs".

K.O.

> > Currently one has to set the required flag in the ODB (e.g., /Programs/Logger/Required) to "y" for the program 
> > to appear on the "Programs page" and being able to start and stop the program easily.
> > 
> > However, if one wants to run with the "Prevent start on required progs" in /Experiment enabled, all the 
> > programs in the "Programs page" need to be running and one cannot have one of them stopped while still 
> > taking a run.
> > 
> > It would be nice to separate these two functionalities: Have a flag that makes the program appear on the 
> > "Programs page" and have a flag that controls the "Prevent start on required frogs" functionality.
> 
> I agree. All the programs should be always visible on the "programs" page, there should be /Programs/xxx/hidden to 
> hide them, and /Programs/xxx/required should be used for "Prevent start on required progs".

Konstantin, since you wrote the current "Programs" page, can you add that feature to the display (well, when you have time). I guess we 
event don't have to change the subdirectory structure (which might lead to incopatibilities), but just show a program if the "Start command" 
is non-null. If there is no start command, it does not make sense to start that program, so it can be hidden.

Stefan

I'd like to use the mjsonrpc db_get_values method, but (as indicated in the
documentation) it returns all ODB keys as lowercase.

This breaks quite a lot of my code - it was written with the old AJAX commands,
and these did respect the capitalization of the ODB keys.

Would it be possible to add a capitalization-preserve option to db_get_values? 

It appears I needed to read the documentation more closely - the method db_save
does respect key-name capitalization and solves my problem.

Is db_save considered a deprecated method?  If so, I'd reiterate my suggestion for
a capitalize-preserve option for db_get_values.

Otherwise, I'll plan on using db_save.

> I'd like to use the mjsonrpc db_get_values method, but (as indicated in the
> documentation) it returns all ODB keys as lowercase.
> 
> This breaks quite a lot of my code - it was written with the old AJAX commands,
> and these did respect the capitalization of the ODB keys.
> 
> Would it be possible to add a capitalization-preserve option to db_get_values? 

Hi Folks,

In last year's updates to MIDAS, the MIDAS sequencer has been broken out as a
separate program (rather than running as part of mhttpd).  We hope that this
change will make the sequencer operation more stable.

Before anyone gets too used to using the new sequencer program, I would like to
rename it.  Currently the program is called 'sequencer'; I would like to rename
it 'msequencer', to make it consistent with most other MIDAS programs.  If you
object to making this change, please say so in the next two weeks.

Documentation on the MIDAS sequencer can be found on the wiki:

https://midas.triumf.ca/MidasWiki/index.php/Sequencer

Note that there are still some tweaks that need to be made to the sequencer
webpage and mhttpd in order to handle this new sequencer program.

Cheers,
Thomas

Hearing no objections I changed the name of the program to msequencer.  Wiki
documentation updated.

> Hi Folks,
> 
> In last year's updates to MIDAS, the MIDAS sequencer has been broken out as a
> separate program (rather than running as part of mhttpd).  We hope that this
> change will make the sequencer operation more stable.
> 
> Before anyone gets too used to using the new sequencer program, I would like to
> rename it.  Currently the program is called 'sequencer'; I would like to rename
> it 'msequencer', to make it consistent with most other MIDAS programs.  If you
> object to making this change, please say so in the next two weeks.
> 
> Documentation on the MIDAS sequencer can be found on the wiki:
> 
> https://midas.triumf.ca/MidasWiki/index.php/Sequencer
> 
> Note that there are still some tweaks that need to be made to the sequencer
> webpage and mhttpd in order to handle this new sequencer program.
> 
> Cheers,
> Thomas

Hi Amy,

Let me start by explaining the reasoning for the default behavior of db_get_values.  I think it was mentioned elsewhere, but is worth repeating.

The ODB is case-insensitive.  So the ODB key name /Equipment/dcrc01 is equivalent to /equipment/Dcrc01; you could rename the variable like that and your 
frontend programs would still work fine.  Javascript, of course, is case sensitive.  However, we want our default MIDAS webpages to work no matter what the 
capitalization is for a particular ODB; so, for instance, the main status.html page should work whether the ODB key is called /Runinfo or /rUnInFo, since both 
of these are equivalent from the point of the ODB (and the rest of MIDAS).

The solution was to have the db_get_values method convert all key names to lower case and consistently use the lower case spelling when writing the main 
MIDAS webpages; this makes us insensitive to ODB capitalization (and hence makes the MIDAS pages behaviour match the previous mhttpd behaviour).

That being said, I agree that it is sometimes counter-intuitive to use lower case key names with db_get_values, particularly if you are directly creating ODB 
keys and writing the javascript at the same time.  So we have added the option 'preserve_case' to db_get_values, which preserves the ODB key name 
capitalization (the default behaviour is still to make key names lower case).

This option should not be used for writing any standard MIDAS webpages (ie, webpages that will be used across multiple experiments), since standard MIDAS 
webpages should not break when ODB key name capitalization changes.  For the same reason you should use caution with this option for custom pages as 
well.

With regards to your second question: the db_save method is not deprecated and you could use that method instead.  The use-case for the db_save method 
is different; db_save is used to make dumps of the ODB.  In that case it seems best that key name capitalization is preserved.  Otherwise if you dumped your 
whole ODB and then reloaded it from the dump the new ODB would be different (in key capitalization) from the old ODB; different in a way that shouldn't 
matter but still probably not the behaviour that people expect.

Admittedly this means that the mjsonrpc API is not always intuitive; but I think is the best we can do, given the underlying case-insensitivity of the ODB.

Thomas


> It appears I needed to read the documentation more closely - the method db_save
> does respect key-name capitalization and solves my problem.
> 
> Is db_save considered a deprecated method?  If so, I'd reiterate my suggestion for
> a capitalize-preserve option for db_get_values.
> 
> Otherwise, I'll plan on using db_save.
> 
> > I'd like to use the mjsonrpc db_get_values method, but (as indicated in the
> > documentation) it returns all ODB keys as lowercase.
> > 
> > This breaks quite a lot of my code - it was written with the old AJAX commands,
> > and these did respect the capitalization of the ODB keys.
> > 
> > Would it be possible to add a capitalization-preserve option to db_get_values? 

A link to an array variable such as

[local:npet:Stopped]/>ls /rcparams/ControlVariables/
TRFC:PB5 (V) -> /Equipment/Beamline/Variables/Demand[56]
                                17835

displays the whole Demand array on the mhttpd ODB page (see attachment)
rather than just the one element Demand[56].
This behaviour also occurs with older versions of mhttpd.
 
Not sure if it's a bug or a feature, but my suggestion is that it
ought to display the one element only (as odbedit does) and not the whole array.

Suzannah  

A feature I always missed (or just missed to find in the docu) is the following:
1) It would be nice to have a command in odbedit which allows to check how full
the ODB currently is.
2) Even more important: I would like to have an ODB routine which allows me to
check the fill level of the ODB, and/or a routine which tells me if I would
create a structure of given size that it still fits in the current ODB or not. 
The use case is that some clients create on the fly ODB entries and I would like
to make sure before hand the ODB's remaining space in order not to crash things
by overfilling the ODB. 

> A feature I always missed (or just missed to find in the docu) is the following:
> 1) It would be nice to have a command in odbedit which allows to check how full
> the ODB currently is.
> 2) Even more important: I would like to have an ODB routine which allows me to
> check the fill level of the ODB, and/or a routine which tells me if I would
> create a structure of given size that it still fits in the current ODB or not. 
> The use case is that some clients create on the fly ODB entries and I would like
> to make sure before hand the ODB's remaining space in order not to crash things
> by overfilling the ODB. 

If you do "mem" in odbedit, you see the currently free areas, one for the keys themselves, one for 
the data of the keys. The corresponding C function is db_show_mem. At the moment it outputs the 
list of free blocks as a long ASCII string, but if necessary I can write a variant which returns the 
number of free bytes.

Stefan

> > A feature I always missed (or just missed to find in the docu) is the following:
> > 1) It would be nice to have a command in odbedit which allows to check how full
> > the ODB currently is.
> > 2) Even more important: I would like to have an ODB routine which allows me to
> > check the fill level of the ODB, and/or a routine which tells me if I would
> > create a structure of given size that it still fits in the current ODB or not. 
> > The use case is that some clients create on the fly ODB entries and I would like
> > to make sure before hand the ODB's remaining space in order not to crash things
> > by overfilling the ODB. 
> 
> If you do "mem" in odbedit, you see the currently free areas, one for the keys themselves, one for 
> the data of the keys. The corresponding C function is db_show_mem. At the moment it outputs the 
> list of free blocks as a long ASCII string, but if necessary I can write a variant which returns the 
> number of free bytes.
> 
> Stefan

Thanks for the info, and yes a variant of db_show_mem returning the number of free bytes would be just
prefect!

> > > A feature I always missed (or just missed to find in the docu) is the following:
> > > 1) It would be nice to have a command in odbedit which allows to check how full
> > > the ODB currently is.
> > > 2) Even more important: I would like to have an ODB routine which allows me to
> > > check the fill level of the ODB, and/or a routine which tells me if I would
> > > create a structure of given size that it still fits in the current ODB or not. 
> > > The use case is that some clients create on the fly ODB entries and I would like
> > > to make sure before hand the ODB's remaining space in order not to crash things
> > > by overfilling the ODB. 
> > 
> > If you do "mem" in odbedit, you see the currently free areas, one for the keys themselves, one for 
> > the data of the keys. The corresponding C function is db_show_mem. At the moment it outputs the 
> > list of free blocks as a long ASCII string, but if necessary I can write a variant which returns the 
> > number of free bytes.
> > 
> > Stefan
> 
> Thanks for the info, and yes a variant of db_show_mem returning the number of free bytes would be just
> prefect!

I made you db_get_free_mem(HNDLE hDB, INT *key_size, INT *data_size)

The first return gets you the number of free bytes for the key area, the second one for the data area (values of keys).

Committed to develop

Stefan

I was planning to use the alarm system to display an information banner when a
certain valve is open, but I would like it to go away again when the valve is
closed.
Is there a way to achieve that? Maybe reset the alarm from an alarm script?
(Seems like a hack...)
Maybe this could be a useful feature, to be able to define an alarm class that
resets itself once the condition is no longer met?

If you run an external script anyway, you can also call "odbedit -c alarm" to
reset all alarms. Or you could try to set the "Triggered" entry of that certain
alarm to 0 (again, with odbedit), that could also work.

> If you run an external script anyway, you can also call "odbedit -c alarm" to
> reset all alarms. Or you could try to set the "Triggered" entry of that certain
> alarm to 0 (again, with odbedit), that could also work.

That would not really help, because you cannot trigger a script AFTER an alarm occurred. Having 
"self-resetting" alarms is actually not a bad idea. I could add a flag "Auto reset" which is false by 
default and can be set to true for this functionality. Will keep that in mind for the next 
development cycle.

Stefan

> I was planning to use the alarm system to display an information banner when a
> certain valve is open, but I would like it to go away again when the valve is
> closed.
> Is there a way to achieve that? Maybe reset the alarm from an alarm script?
> (Seems like a hack...)
> Maybe this could be a useful feature, to be able to define an alarm class that
> resets itself once the condition is no longer met?

Actually you can implement such a thing already now pretty quickly using custom javascript on 
the status page. Just read the valve state regularly from the ODB and dynamically modify the 
status page to show or hide a banner. Look how custom pages work in midas and try to apply 
this to the status page status.html which you find in the resources directory.

Stefan

> > If you run an external script anyway, you can also call "odbedit -c alarm" to
> > reset all alarms. Or you could try to set the "Triggered" entry of that certain
> > alarm to 0 (again, with odbedit), that could also work.
> 
> That would not really help, because you cannot trigger a script AFTER an alarm occurred. Having 
> "self-resetting" alarms is actually not a bad idea. I could add a flag "Auto reset" which is false by 
> default and can be set to true for this functionality. Will keep that in mind for the next 
> development cycle.
> 

I second, this is a good idea. Sometimes I want "sticky" alarms that stay on to indicate that a bad thing happened in the 
past, sometimes I want self-resetting alarms that go away when a bad thing turns back into a good thing.

When I do this in a frontend, I manually trigger the alarm and manually clear the alarm, i.e. you can see this
done in ~addaq/online/src/fectrl.cxx

Use al_trigger_alarm() and al_reset_alarm().

This can also be done through the json-rpc interface - both calls are available as rpc commands - and so easy to use 
from javascript. (but there is no simple unix command line tool to issue json-rpc requests. ouch. must write one now.)

K.O.