 22 Oct 2015, Konstantin Olchanski, Suggestion, Feature Request: MIDAS sequencer abort.
|
> it would be useful to have a kind of scripted "abort" for when something goes wrong ...
How about having the sequencer switching from the aborted sequence file to the special "abort" sequence file? That
should be simple to implement if it is not already there.
K.O. |
|
29 Oct 2015, Konstantin Olchanski, Info, midas JSON-RPC interface
|
>
> My implementation follows these internet standards:
>
> // https://tools.ietf.org/html/rfc4627 - JSON RFC
> // http://www.jsonrpc.org/specification - specification of JSON-RPC 2.0
> // http://www.simple-is-better.org/json-rpc/transport_http.html
>
> With following variances:
> - JSON encoding for NAN and Inf is Javascript-compatible strings "NaN", "Infinity" and "-Infinity"
> - HTTP GET is not supported (not recommended by standard)
> - batched JSON-RPC requests not supported yet
>
The last missing piece is now committed - the JSON-RPC interface is now self-documenting via an automatically
generated JSON Schema that lists all RPC methods with their parameters and return values. This documentation
schema is created from simple to use documentation code in each rpc server function, see mjsonrpc.cxx.
To kick the tires, checkout the feature/json_rpc branch, build mhttpd, setup the examples/javascript1 experiment,
run mhttpd in the terminal, from the "status" page, go to the "example" custom page, press "push me" in the mjsonrpc_db_get_values() section,
mhttpd will print the schema file on the terminal. Use any json schema visualization tool to look at it. In the future I hope
to link this schema to the midas "help" page.
The impatient can go directly here: (it is safe to press all buttons) (elog is making a dog's breakfast of my url)
http://ladd00.triumf.ca/~olchansk/test/docson/#../test.json
docson is here:
https://github.com/lbovet/docson
For more informantion about JSON Schema stuff, go here:
https://tools.ietf.org/html/draft-zyp-json-schema-04
http://spacetelescope.github.io/understanding-json-schema/
http://json-schema.org/
JSON Schema examples:
http://json-schema.org/examples.html
http://json-schema.org/example1.html
JSON Schema visualization: (schema file has to have a .json extension)
https://github.com/lbovet/docson
(there is also an interesting discussion on why there is no RFC for JSON schema - the draft expired several years ago)
K.O. |
|
29 Oct 2015, Konstantin Olchanski, Info, javascript docs, midas JSON-RPC interface
|
> JSON-RPC interface
For interfacing to MIDAS just from browser javascript, the user does not need to know anything about JSON-RPC, all the user-level mjsonrpc_xxx() functions provided by
mhttpd.js work the same as the old ODBxxx() functions.
As usual, the functions are documented using Doxygen, so here there is no difference between old and new interfaces.
To generate the documentation, run "make dox" (doxygen and graphviz "dot" packages should be installed). (it will take some time to generate everything), then open
html/index.html and navigate to "files" to "mhttpd.js" and you will see the list of all RPC funcrions (old functions are ODBxxx, new functions are mjsonrpc_xxx).
There was a possibility to implement the mjsonrpc javascript client interface as a javascript class, but older versions of doxygen seem to work incorrectly for such code making it
impossible to document the code. So it remains implemented as traditional functions with a few globals, but the design an implementation are done with a view to convert the
code to a javascript class some time in the future.
As ever, the examples/javascript1 experiment provides examples of using all available javascript functions supported by midas. (except for functions that are hard to example or
hard to document).
K.O. |
 29 Oct 2015, Konstantin Olchanski, Info, synchronous ajax deprecated
|
If using a synchronous AJAX call, such as "foo=ODBGet("/runinfo/state");", google chrome will prints this to the javascript console:
"Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience. For more help, check http://xhr.spec.whatwg.org/."
The referenced URL has this text:
"Synchronous XMLHttpRequest outside of workers is in the process of being removed from the web platform as it has detrimental effects to the end user's experience. (This is a long
process that takes many years.) Developers must not pass false for the async argument when the JavaScript global environment is a document environment. User agents are strongly
encouraged to warn about such usage in developer tools and may experiment with throwing an InvalidAccessError exception when it occurs."
Then jQuery say this: http://api.jquery.com/jquery.ajax/
"As of jQuery 1.8, the use of async: false with jqXHR ($.Deferred) is deprecated; you must use the success/error/complete callback options instead of the corresponding methods of the
jqXHR object such as jqXHR.done() or the deprecated jqXHR.success()."
This sounds rather severe but one must flow with the flow. Synchronous RPC is out, async is in.
Many of the old MIDAS AJAX functions are fully synchronous (i.e. "foo=ODBGet("/blah");"), some more recent ones support both sync and async use (i.e. ODBMCopy()).
All the newly added functions *must* by async-only. For example, all the new JSON-RPC functions are async-only and require the use of callbacks to get at the data.
Converting existing javascript custom pages from sync AJAX (hah! it's SJAX, not AJAX) will require some work, and one might as well start today.
Personally, I think this excessive use of callbacks for all javascript web page programming is an unnecessary PITA, but I also do understand the motivation
of people who write web browsers and javascript engines - removal of support for synchronous RPC makes many things much simpler -
and even small speedup of javascript execution and better browser efficiency is welcome improvements (but not free improvements - as old web pages need to be converted).
K.O. |
|
02 Nov 2015, Konstantin Olchanski, Info, synchronous ajax deprecated
|
> We're using mhttpd for calls that end up working better with asynchronous requests, and we've built up sort of a parallel, asynchronous library using javascript Promises.
>
> The Promises (which are in the ES6 spec) have worked incredibly well for building well-behaved, sequential calls to mhttpd. Personally, I also find their syntax much easier to wrap my
> head around, especially compared to callbacks.
>
Yes, the javascript wrappers for the json-rpc interface follow the Promise pattern - an RPC call is provided with two user functions,
one is called on success (and provides the rpc reply), the other on failure (and provides all rpc call information - the xhr object, any exception context, etc).
Use of the Promise class itself seems to be problematic - apparently it does not exist in google chrome 28 (the last version for RHEL/CentOS/SL6).
SL6 is still our main workhorse and it is good to have a choice of 2 browsers (old google chrome vs old firefox).
(All SL5 web browsers are already unusable with the modern web and current mhttpd.)
(Also the RPC calls have more than 1 item of data permitted by the javascript Promise class - of course it can be wrapped
be a container object - just an extra complication to document and to understand).
K.O. |
|
02 Nov 2015, Konstantin Olchanski, Info, midas JSON-RPC interface
|
> >
> > JSON-RPC My implementation follows these internet standards:
> >
> > // https://tools.ietf.org/html/rfc4627 - JSON RFC
> > // http://www.jsonrpc.org/specification - specification of JSON-RPC 2.0
> > // http://www.simple-is-better.org/json-rpc/transport_http.html
>
> JSON Schema
>
> https://github.com/lbovet/docson
> http://spacetelescope.github.io/understanding-json-schema/
> http://json-schema.org/
>
Without figuring out how to run docson one can see the JSON-RPC Schema linked from the mhttpd "Help" page
follow link "JSON RPC schema" -> "text format" you will see it pretty printed like this:
---------------------------------------------------------------------
Autogenerated schema for all MIDAS JSON-RPC methods
---------------------------------------------------------------------
cm_exist | calls MIDAS cm_exist() to check if given MIDAS program is running
| -----------------------------------------------------
| params | name | string | name of the program, corresponding to ODB /Programs/name
| | unique? | bool | bUnique argument to cm_exist()
| -----------------------------------------------------
| result | status | integer | return status of cm_exist()
---------------------------------------------------------------------
cm_shutdown | calls MIDAS cm_shutdown() to stop given MIDAS program
| -----------------------------------------------------
| params | name | string | name of the program, corresponding to ODB /Programs/name
| | unique? | bool | bUnique argument to cm_shutdown()
| -----------------------------------------------------
| result | status | integer | return status of cm_shutdown()
---------------------------------------------------------------------
db_copy | get copies of given ODB subtrees in the "save" json encoding
| -----------------------------------------------------
| params | paths[] | array of ODB subtree paths, see note on array indices
| | | array of | string
| -----------------------------------------------------
| result | data[] | copy of ODB data for each path
| | | array of | object
| | status[] | return status of db_copy_json() for each path
| | | array of | integer
| | last_written[] | last_written value of the ODB subtree for each path
| | | array of | number
---------------------------------------------------------------------
db_create | get copies of given ODB subtrees in the "save" json encoding
| -----------------------------------------------------
| params | array of ODB paths to be created
| | array of | arguments to db_create() and db_resize()
| | | path | string | ODB path
| | | type | integer | MIDAS TID_xxx type
| | | array_length? | integer | optional array length, default is 1
| | | string_length? | integer | for TID_STRING, optional string length, default is NAME_LENGTH
| -----------------------------------------------------
| result | status[] | return status of db_create() for each path
| | | array of | integer
---------------------------------------------------------------------
db_get_values | get values of ODB data from given subtrees
| -----------------------------------------------------
| params | paths[] | array of ODB subtree paths, see note on array indices
| | | array of | string
| -----------------------------------------------------
| result | data[] | values of ODB data for each path, all key names are in lower case, all symlinks are followed
| | | array of | any
| | status[] | return status of db_copy_json() for each path
| | | array of | integer
| | last_written[] | last_written value of the ODB subtree for each path
| | | array of | number
---------------------------------------------------------------------
db_paste | write data into ODB
| -----------------------------------------------------
| params | paths[] | array of ODB subtree paths, see note on array indices
| | | array of | string
| | values[] | data to be written using db_paste_json()
| | | array of | any
| -----------------------------------------------------
| result | status[] | return status of db_paste_json() for each path
| | | array of | integer
---------------------------------------------------------------------
get_debug | get current value of mjsonrpc_debug
| -----------------------------------------------------
| params | any | there are no input parameters
| -----------------------------------------------------
| result | integer | current value of mjsonrpc_debug
---------------------------------------------------------------------
get_schema | Get the MIDAS JSON-RPC schema JSON object
| -----------------------------------------------------
| params | any | there are no input parameters
| -----------------------------------------------------
| result | object | returns the MIDAS JSON-RPC schema JSON object
---------------------------------------------------------------------
null | RPC method always returns null
| -----------------------------------------------------
| params | any | method parameters are ignored
| -----------------------------------------------------
| result | null | always returns null
---------------------------------------------------------------------
set_debug | set new value of mjsonrpc_debug
| -----------------------------------------------------
| params | integer | new value of mjsonrpc_debug
| -----------------------------------------------------
| result | integer | new value of mjsonrpc_debug
---------------------------------------------------------------------
start_program | start MIDAS program defined in ODB /Programs/name
| -----------------------------------------------------
| params | name | string | name of the program, corresponding to ODB /Programs/name
| -----------------------------------------------------
| result | status | integer | return status of ss_system()
---------------------------------------------------------------------
user_example1 | any
---------------------------------------------------------------------
user_example2 | any
---------------------------------------------------------------------
user_example3 | any |
|
11 Nov 2015, Konstantin Olchanski, Info, merged: midas JSON-RPC interface
|
The JSON RPC branch has been merged into main MIDAS. Other than adding new functions, there are no changes to existing MIDAS functionality.
This is the current JSON RPC schema: (from the MIDAS Help page)
------------------------------------------------------------------------
Autogenerated schema for all MIDAS JSON-RPC methods
------------------------------------------------------------------------
cm_exist? | calls MIDAS cm_exist() to check if given MIDAS program is running
| -------------------------------------------------------
| params | name | string | name of the program, corresponding to ODB /Programs/name
| | unique? | bool | bUnique argument to cm_exist()
| -------------------------------------------------------
| result | status | integer | return status of cm_exist()
------------------------------------------------------------------------
cm_shutdown? | calls MIDAS cm_shutdown() to stop given MIDAS program
| -------------------------------------------------------
| params | name | string | name of the program, corresponding to ODB /Programs/name
| | unique? | bool | bUnique argument to cm_shutdown()
| -------------------------------------------------------
| result | status | integer | return status of cm_shutdown()
------------------------------------------------------------------------
db_copy? | get copies of given ODB subtrees in the "save" json encoding
| -------------------------------------------------------
| params | paths[] | array of ODB subtree paths, see note on array indices
| | | array of | string
| -------------------------------------------------------
| result | data[] | copy of ODB data for each path
| | | array of | object
| | status[] | return status of db_copy_json() for each path
| | | array of | integer
| | last_written[] | last_written value of the ODB subtree for each path
| | | array of | number
------------------------------------------------------------------------
db_create? | get copies of given ODB subtrees in the "save" json encoding
| -------------------------------------------------------
| params[] | array of ODB paths to be created
| | array of | arguments to db_create() and db_resize()
| | | path | string | ODB path
| | | type | integer | MIDAS TID_xxx type
| | | array_length? | integer | optional array length, default is 1
| | | string_length? | integer | for TID_STRING, optional string length, default is NAME_LENGTH
| -------------------------------------------------------
| result | status[] | return status of db_create() for each path
| | | array of | integer
------------------------------------------------------------------------
db_get_values? | get values of ODB data from given subtrees
| -------------------------------------------------------
| params | paths[] | array of ODB subtree paths, see note on array indices
| | | array of | string
| -------------------------------------------------------
| result | data[] | values of ODB data for each path, all key names are in lower case, all symlinks are followed
| | | array of | any
| | status[] | return status of db_copy_json() for each path
| | | array of | integer
| | last_written[] | last_written value of the ODB subtree for each path
| | | array of | number
------------------------------------------------------------------------
db_paste? | write data into ODB
| -------------------------------------------------------
| params | paths[] | array of ODB subtree paths, see note on array indices
| | | array of | string
| | values[] | data to be written using db_paste_json()
| | | array of | any
| -------------------------------------------------------
| result | status[] | return status of db_paste_json() for each path
| | | array of | integer
------------------------------------------------------------------------
get_debug? | get current value of mjsonrpc_debug
| -------------------------------------------------------
| params | any | there are no input parameters
| -------------------------------------------------------
| result | integer | current value of mjsonrpc_debug
------------------------------------------------------------------------
get_schema? | Get the MIDAS JSON-RPC schema JSON object
| -------------------------------------------------------
| params | any | there are no input parameters
| -------------------------------------------------------
| result | object | returns the MIDAS JSON-RPC schema JSON object
------------------------------------------------------------------------
null? | RPC method always returns null
| -------------------------------------------------------
| params | any | method parameters are ignored
| -------------------------------------------------------
| result | null | always returns null
------------------------------------------------------------------------
set_debug? | set new value of mjsonrpc_debug
| -------------------------------------------------------
| params | integer | new value of mjsonrpc_debug
| -------------------------------------------------------
| result | integer | new value of mjsonrpc_debug
------------------------------------------------------------------------
start_program? | start MIDAS program defined in ODB /Programs/name
| -------------------------------------------------------
| params | name | string | name of the program, corresponding to ODB /Programs/name
| -------------------------------------------------------
| result | status | integer | return status of ss_system()
------------------------------------------------------------------------
user_example1? | example of user defined RPC method that returns up to 3 results
| -------------------------------------------------------
| params | arg | string | example string argment
| | optional_arg? | integer | optional example integer argument
| -------------------------------------------------------
| result | string | string | returns the value of "arg" parameter
| | integer | integer | returns the value of "optional_arg" parameter
------------------------------------------------------------------------
user_example2? | example of user defined RPC method that returns more than 3 results
| -------------------------------------------------------
| params | arg | string | example string argment
| | optional_arg? | integer | optional example integer argument
| -------------------------------------------------------
| result | string1 | string | returns the value of "arg" parameter
| | string2 | string | returns "hello"
| | string3 | string | returns "world!"
| | value1 | integer | returns the value of "optional_arg" parameter
| | value2 | number | returns 3.14
------------------------------------------------------------------------
user_example3? | example of user defined RPC method that returns an error
| -------------------------------------------------------
| params | arg | integer | integer value, if zero, throws a JSON-RPC error
| -------------------------------------------------------
| result | status | integer | returns the value of "arg" parameter
K.O. |
|
11 Nov 2015, Konstantin Olchanski, Bug Report, jset/ODBSet using true/false for booleans
|
> > MIDAS does not seem to be consistent (or at least convenient) with how it
> > handles booleans in AJAX functions.
The JSON-RPC functions have been merged into main midas and you can now use the new function mjsonrpc_db_paste(paths, values, id,
callback, error_callback);
For example:
mjsonrpc_db_paste(["/foo","/bar","/baz"],[1,2,3]);
the target items should already exist (for this example, not in general).
All data is JSON encoded, success/failure is returned via callbacks.
K.O. |
|
17 Nov 2015, Konstantin Olchanski, Info, synchronous ajax deprecated
|
> > We're using mhttpd for calls that end up working better with asynchronous requests, and we've built up sort of a parallel, asynchronous library using javascript Promises.
I checked again on browser compatibility:
el6: firefox 38 - ok, google-chrome 27 - no
el7: firefox 38 - ok, google-chrome 46 - ok
ubuntu: firefox 42 - ok
mac os, windows - we say "latest firefox or google-chrome is required", then - ok
So we are probably okey with using javascript Promises with MIDAS...
I shall try to convert the json-rpc client library to promises, see how it shakes out.
K.O. |
|
18 Nov 2015, Konstantin Olchanski, Info, synchronous ajax deprecated
|
> > Why don't you post the functions here so that we can have a look?
> Here is (1) my promisified HTTP request function and (2) a function that uses the returned promises to build an asynchronous, sequential chain of requests to Midas.
>
> In addition to promisifying HTTP requests to Midas, I wanted the Promise.resolve from this function to always return valid JSON.
Thank you very much for posting this code. It is very similar to what I just wrote this morning for the JSON-RPC client library. In my case, the JSON-RPC responses
are much more regular so error handling is simple: a) check HTTP response 200, b) check JSON-RPC reply parses into valid JSON (catch exception), c) check JSON-RPC error status.
> I also wanted the promise to reject if the response from mhttpd indicated
> failure - so that we wouldn't have to rewrite this error checking throughout the code.
Right now the JSON-RPC client library does not check the return status of MIDAS calls themselves, i.e. ODBGet("/nonexistant") will go to Promise.resolve() with
the MIDAS db_find_key() status DB_NO_KEY instead of Promise.reject(). So some error handling in Promise.resolve() is still required.
I am thinking how to make these calls go to the error handler automatically, but there is no obvious solution for ODBMGet(["/runinfo", "/nonexistant"]) - the first path
is a success, the second is a failure, do I fail the entire transaction (i.e. with a JSON-RPC error)? Same for JSON-RPC batch transactions - if one transaction
in the batch fails, do I Promise.reject() all of them?
I guess I could "split hairs" and create a separate Promise for each "atomic" transaction, the Promise mechanism does seem to support that,
but this will create more complexity than I feel comfortable with.
Please take a look at the branch feature/js_promise - mjsonrpc_call() is Promisified (resources/mhttpd.js) and the db_copy() example is Promisified (examples/javascript1/example.html)
K.O. |
|
20 Nov 2015, Konstantin Olchanski, Info, documented, merged: midas JSON-RPC interface
|
> The JSON RPC branch has been merged into main MIDAS.
The interface is now mostly documented, go here: https://midas.triumf.ca/MidasWiki/index.php/Mjsonrpc
Documentation for individual javascript functions in mhttpd.js not merged into the MIDAS documentation yet, because the API is being converted to the Javascript Promise
pattern (git branch feature/js_promise).
The functions available from mhttpd.js are documented via doxygen, also linked from the mjsonrpc wiki page.
K.O. |
|
20 Nov 2015, Konstantin Olchanski, Info, midas wiki doxygen documentation links
|
I updated the links on the midas wiki to the doxygen-generated documentation for MIDAS that you
get after running "git clone midas; cd midas; make dox; firefox html/index.html".
Correct link is:
https://daq.triumf.ca/~daqweb/doc/midas-devel/html/
This takes you to a daily/nightly generated snapshot of the midas develop branch and the
generated documentation with full call graphs.
Previous links were deficient is different ways:
- referred to http://ladd00 instead of https://daq
- referred to wrong path ~daqweb/doc/midas instead of ~daqweb/doc/midas-devel
- referred to the obsolete doxygen generator in midas/doc/html instead of midas/html.
If wrong links are still present on the midas wiki, please let us know and we will fix them.
K.O. |
|
25 Nov 2015, Konstantin Olchanski, Forum, rpc_client_dispatch error
|
Not clear on what you are doing, so here is the brief description:
- you have two machines - say "midas" and "frontend"
- you run mttpd, mlogger, etc on "midas"
- you want to run some frontend on "frontend"
Do this:
- on "midas", open a new terminal, run "mserver -p 7071"
- on "frontend", open a new terminal, run "odbedit -h midas:7071"
If you did follow all the online instruction correctly, at this point, your odbedit on "frontend" would have
connected and all commands would work same as if run locally on "midas".
If I understand you correctly, you got this far.
Next do this:
- on "frontend", open a new terminal, run "your_frontend.exe -h midas:7071"
If all is good, the frontend would start, would connect to midas, you would see
it in odbedit "scl" and on the midas status page and you would be able to stop
it from the midas "programs" page. (this last bit is important).
I guess this is where things go wrong and you do not get anything working.
Do this:
a) cut and paste all the output from the terminal window where you are running the mserver (including the
command you used to start the mserver)
b) cut and paste all the output from the terminal window where you are starting the frontemd (again,
including the command you used to start the frontend)
c) cut and paste the contents of midas.log (in the experiment directory) from the time you started the
mserver until the very end.
Paste all this as reply to this message or email it to me at olchansk@triumf.ca
With this additional information we should be able to get you going (and hopefully improve the
documentation so the next person does not run into the same problem - whatever the problem turns out to
be).
K.O.
> I'm trying to set up an experiment with 2 frontends for the first time. When I
> start the remote frontend I get the following errors:
>
> first time (odd attempt):
>
> [MCS_Frontend_203,ERROR] [midas.c:9678:rpc_server_connect,ERROR] mserver
> subprocess could not be started (check path)
> [MCS_Frontend_203,ERROR] [mfe.c:2696:mainFE,ERROR] Cannot connect to experiment
> 'Default' on host 'ucntau-daq.lanl.gov', status 503
>
> second time (even attempt):
>
> MCS6A_frontend: src/midas.c:9085: rpc_client_dispatch: Assertion `n ==
> sizeof(NET_COMMAND_HEADER) + 4 * sizeof(INT)' failed.
>
> On the local host I'm running : mlogger, a frontend, an analzer, mhttp, and
> mserver. I followed the instructions for adding the remote computer to the
> RPC_ALLOWED list and I do see that the remote frontend was able to edit the
> local odb equipment list. At present I'm not running an event builder I just
> wanted to get the frontends connected to start. Do I need to have the mserver
> running on both computers? Any suggestions on where to start troubleshooting this?
>
> Thanks |
|
26 Nov 2015, Konstantin Olchanski, Info, browser compatibility test: synchronous ajax deprecated
|
> > I checked again on browser compatibility:
> >
> > el6: firefox 38 - ok, google-chrome 27 - no
> > el7: firefox 38 - ok, google-chrome 46 - ok
> > ubuntu: firefox 42 - ok
> >
> > mac os, windows - we say "latest firefox or google-chrome is required", then - ok
> >
> > So we are probably okey with using javascript Promises with MIDAS...
>
> [too bad about chrome on SL6] ... include a polyfill library like Lie (https://github.com/calvinmetcalf/lie).
Results of cross-browser testing.
MacOS 10.10.5:
google-chrome 46: Promise ok, Programs page ok, Overlay ok
firefox 42: Promise ok, Programs page ok, Overlay ok.
safari 9.0.1: Promise ok, Programs page ok, Overlay ok.
Linux SL6.7:
google-chrome 27: "Promise not defined"
firefox 38.4.0: Promise ok, Programs page ok, Overlay ok.
konqueror 4.3.4: no go "Can't find variable: JSON"
chromium/google-chrome 38: ok
Linux SL7.1:
google-crome 46: ok
firefox 38.4.0: ok
konqueror 4.10.5: no go, mhttpd.js parse error
Conclusion:
1) firefox is good everywhere
2) google-chrome is good on Mac, Windows and el7 Linux
3) chromium/google-chrome 38 is good on el6 Linux (SL6/CentOS6).
We are good to proceed with adopting the Promise API for MIDAS.
K.O. |
|
27 Nov 2015, Konstantin Olchanski, Info, synchronous ajax deprecated
|
> > I checked again on browser compatibility:
> >
> > el6: firefox 38 - ok, google-chrome 27 - no
>
> It looks like this does mean that people using RHEL6 won't have the option of chrome - can they update chrome?
>
It turns out that google-chrome 38 is available for RHEL6/SL6 via an old chromium build. Promises are supported (passes my tests).
See here:
http://www.if-not-true-then-false.com/2013/install-chromium-on-centos-red-hat-rhel
This is where I got the working chromium 38 (no explanation of why there are no newer builds)
http://people.centos.org/hughesjr/chromium/6/x86_64/RPMS/
There appear to be newer builds here: (but I will not test them)
http://install.linux.ncsu.edu/pub/yum/itecs/public/chromium-dev/rhel6/x86_64/
My SL6 google-chrome and chromium instructions:
https://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Install_Google_Chrome_web_browser_.2864-bit_SL6.29
K.O. |
|
27 Nov 2015, Konstantin Olchanski, Info, updated: note on midas history
|
(update: resolve all FIXMEs, document the breakup of "structured banks")
This note documents the workings of the midas history.
There is 2 separate history sections: equipment history and links history.
* is equipment history enabled?
For each equipment, history is controlled by the value of /eq/xxx/common/period:
0 = history disabled
1 = history is enabled
>1 = history is enabled, throttled down
The throttling is implemented in log_history()/watch_history() by this algorithm:
the very first history event is recorded, then all changed to the data are ignored until
"period" seconds has elapsed. Then the next history event will be recorded, and following
changes will be ignored until "period" second elapses, and so forth. Period value "1" has
special meaning - there is no throttling, all history events are logged.
If equipment history is enabled, history events are created by parsing the content of /eq/xxx/variables.
* what is history events?
A "history event" is a history atomic unit of data. Associated with each history event is a timestamp (unix time),
a name (limited to NAME_LENGTH in the old history) and a list of history tags that describe the individual data
values inside the history event.
When making history plots in mhttpd, for each curve on the plot, one selects a history event (from the list
of currently active events, recently active events or the list of all events that ever existed), then from the list of tags
inside the history event one selects the particular variable that will be plotted.
In the old MIDAS history, all history events are written into one history file (.hst file + optional .def and .idx event definition and time index files
which can be/are regenerated automatically from the .hst file). History events are identified by 16-bit history event IDs, the persistent mapping
from history event names and the 16-bit history event IDs is stored in ODB /History/Events. In addition the list of all known history event tags is
stored in ODB /History/Tags. For per-equipment history, the 16-bit history event ID is the value of ODB "/eq/xxx/common/event id".
In the SQL history (MySQL, SQLITE, etc), each history event is an SQL table. The history event tags are the SQL table columns.
In the new FILE history, each history event is written into a separate file, tag definition are recorded in text formal in the file header, history event
data is appended to the file in binary format (fixed record size). If the history event definition is changed, a new file will be started.
* how are history events constructed?
The mlogger creates history events in open_history() by parsing ODB /eq/xxx/variables. Each ODB entry under "variables" is referred to as a "variable".
Each variable can be a single ODB value, an array of ODB values, or a subdirectory (corresponding to TID_STRUCT structured data banks). As each variable
is processed, one or more tags are created to describe it. Single ODB values will generally produce a single tag, while arrays can produce
one single tag - describing the whole array - or multiple tags - one per array element - depending whether the array is "named" or not.
The code can generate two types of history:
- "per-equipment" history will have the tags for all variables concatenated together into one single history event
- "per-variable" history will have one history event defined for each variable. Inside could be one tag - for single odb values and unnamed arrays - or multiple tags - for named arrays and structured data
banks.
Per-equipment history is the original MIDAS history implementation.
Per-variable history was added to permit efficient data storage in SQL tables. It's initial implementation used 1 ODB hotlink for each variable and it was easy to exceed the maximum permitted number of
ODB hotlinks (db_open_record()).
To reduce consumption of hotlinks, db_watch() has been implemented and now per-variable history only uses 1 ODB hotlink per equipment.
With db_watch, per-equipment history is no longer available. per-variable history is the new default (and the only option).
* how are the history event tags constructed?
(quirk - single odb values are treated as arrays of length "1")
FIXME: single odb values should be treated as such, /eq/xxx/settings/names should not be applied
(quirk - "string" ODB entries are not permitted)
FIXME - single odb values of type TID_STRING should be possible with SQL, FILE and MIDAS history. arrays of strings is impossible "struct TAG" does not have a data field for string length - only n_data and
item length implied through it's TID.
History event tags are constructed in the mlogger add_equipment().
For variables of type TID_KEY (subdirectories, corresponding to TID_STRUCT structured banks), one tag is generated for each subdirectory entry. Tag names for /eq/xxx/var/aaa/bbb will be "aaa_bbb".
(with an underscore).
FIXME: subdirectory entries of type TID_KEY and TID_LINK should be explicitly forbidden.
FIXME: TID_KEY could be supported by replacing db_get_data() with db_get_record() in watch_history().
FIXME: TID_LINK could be supported by adding db_watch() on the link target.
For named arrays, individual tags are generated for each array element. Tag names are taken from the names array. For empty tag names (empty names array), tags are "aaa_0", "aaa_1", etc (for
/eq/xxx/var/aaa). For "single names" arrays, tag names have the variable name appended (with a space), for /eq/xxx/var/aaa and an empty names array, tags will be "aaa_0 aaa", "aaa_1 aaa", etc. For
populated names array, the tags will be "name0 aaa", "name1 aaa", etc.
For unnamed arrays and single odb variables (in ODB, single odb variables are arrays of length 1), a single tag is generated.
For TID_LINK variables what happens? FIXME!
FIXME: support TID_LINK variables by correctly parsing the link target and setting a db_watch() on the link target.
Named arrays have a "Names" entry in /eq/xxx/settings. For example, to add names to /eq/xxx/var/aaa, create a string array "/eq/xxx/settings/names aaa". The names array should be at least as long as
the corresponding data array. Individual entries in the names array can be left blank (tag names will be "aaa_0", "aaa_1", etc). Duplicate tag names are not permitted.
A single "Names" entry can be created to name all arrays in variables with the same names ("single names"). Create /eq/xxx/settings/names" and arrays /eq/xxx/var/aaa and /eq/xxx/var/bbb will have
history tags "name0 aaa", "name1 aaa", "name0 bbb", "name1 bbb", etc. If "names" are left blank, tag names will be "aaa_0 aaa", "aaa_1 aaa", "bbb_0 bbb", "bbb_1 bbb", etc.
In the mhttpd variables viewer, "single name" arrays are displayed in a 2D table.
* /history/links history
History events are created for each entry under /history/links.
Two types of links are permitted:
/history/links/aaa is a link to a subdirectory: db_watch() is setup to watch this subdirectory, tags are created for each subdirectory entry (1 tag per entry). There is no possibility for naming array elements, so 1 tag per array, regardless of the number of elements.
/history/links/bbb is a subdirectory with links to odb values: db_watch is setup to watch each link target, tags are created for each link (1 tag per link). tag name is the link name (NOT the target name). There is no possibility for naming array elements.
FIXME: Mixing links and subdirectories is not permitted, but could be done - additional db_watch() will need to be done on any links.
Update period history events created for /history/links is controlled by entries in "/history/links periods". Numeric values of periods are same as for equipment histories. Numeric value 0 disables the history for a particular event.
K.O. |
|
30 Nov 2015, Konstantin Olchanski, Release, Final MIDAS JSON-RPC API
|
The final bits of the JSON-RPC API to MIDAS are committed. The API uses the Javascript Promise mechanism (supported on all
supported platforms - MacOS, Windows, Linux Ubuntu, el5, el6, el7).
Simple example for pasting the current run number into an html element:
mjsonrpc_db_get_values(["/runinfo/run number"]).then(function(rpc) {
document.getElementById("run_number").innerHTML = rpc.response.data[0];
}).catch(function(error) {
mjsonrpc_error_alert(error);
});
The documentation for the JSON-RPC API, including special quirks in JSON encoding of ODB data is here:
https://midas.triumf.ca/MidasWiki/index.php/Mjsonrpc
Documentation (with examples) for the related Javascript functions in mhttpd.js is here (via Doxygen):
https://daq.triumf.ca/~daqweb/doc/midas-devel/html/group__mjsonrpc__js.html
Examples of using all mhttpd.js functions is in .../examples/javascript1/example.html
The experimental git branch feature/mhttpd_js implements the MIDAS "programs" page purely in html and javascript,
go there to see all this new JSON and RPC stuff in action. See .../resources/programs.html.
K.O. |
|
02 Dec 2015, Konstantin Olchanski, Release, Final MIDAS JSON-RPC API
|
> The final bits of the JSON-RPC API to MIDAS are committed.
Here is example conversion of the function "generate midas message" from old-style AJAX to JSON-RPC:
before (mhttpd.cxx):
/* process "jgenmsg" command */
if (equal_ustring(getparam("cmd"), "jgenmsg")) {
if (getparam("facility") && *getparam("facility"))
strlcpy(facility, getparam("facility"), sizeof(facility));
else
strlcpy(facility, "midas", sizeof(facility));
if (getparam("user") && *getparam("user"))
strlcpy(user, getparam("user"), sizeof(user));
else
strlcpy(user, "javascript_commands", sizeof(user));
if (getparam("type") && *getparam("type"))
type = atoi(getparam("type"));
else
type = MT_INFO;
if (getparam("msg") && *getparam("msg")) {
cm_msg1(type, __FILE__, __LINE__, facility, user, "%s", getparam("msg"));
}
show_text_header();
rsputs("Message successfully created\n");
return;
}
after: (mjsonrpc.cxx)
static MJsonNode* js_cm_msg1(const MJsonNode* params)
{
if (!params) {
MJSO *doc = MJSO::I();
doc->D("Generate a midas message using cm_msg1()");
doc->P("facility?", MJSON_STRING, "message facility, default is \"midas\"");
doc->P("user?", MJSON_STRING, "message user, default is \"javascript_commands\"");
doc->P("type?", MJSON_INT, "message type, MT_xxx from midas.h, default is MT_INFO");
doc->P("message", MJSON_STRING, "message text");
doc->R("status", MJSON_INT, "return status of cm_msg1()");
return doc;
}
MJsonNode* error = NULL;
const char* facility = mjsonrpc_get_param(params, "facility", &error)->GetString().c_str();
const char* user = mjsonrpc_get_param(params, "user", &error)->GetString().c_str();
int type = mjsonrpc_get_param(params, "type", &error)->GetInt();
const char* message = mjsonrpc_get_param(params, "message", &error)->GetString().c_str(); if (error) return error;
if (strlen(facility)<1)
facility = "midas";
if (strlen(user)<1)
user = "javascript_commands";
if (type == 0)
type = MT_INFO;
int status = cm_msg1(type, __FILE__, __LINE__, facility, user, "%s", message);
return mjsonrpc_make_result("status", MJsonNode::MakeInt(status));
}
With the corresponding javascript-side stabs:
before:
function ODBGenerateMsg(type,facility,user,msg)
{
var request = XMLHttpRequestGeneric();
var url = ODBUrlBase + '?cmd=jgenmsg';
url += '&type='+type;
url += '&facility='+facility;
url += '&user='+user;
url += '&msg=' + encodeURIComponent(msg);
request.open('GET', url, false);
request.send(null);
return request.responseText;
}
after:
function mjsonrpc_cm_msg(message, type, id) {
/// \ingroup mjsonrpc_js
/// Get values of ODB variables
///
/// RPC method: "cm_msg1"
///
/// \code
/// mjsonrpc_cm_msg("this is a new message").then(function(rpc) {
/// var req = rpc.request; // reference to the rpc request
/// var id = rpc.id; // rpc response id (should be same as req.id)
/// var status = rpc.result.status; // return status of MIDAS cm_msg1()
/// ...
/// }).catch(function(error) {
/// mjsonrpc_error_alert(error);
/// });
/// \endcode
/// @param[in] message Text of midas message (string)
/// @param[in] type optional message type, one of MT_xxx. Default is MT_INFO (integer)
/// @param[in] id optional request id (see JSON-RPC specs) (object)
/// @returns new Promise
///
var req = new Object();
req.message = message;
if (type)
req.type = type;
return mjsonrpc_call("cm_msg1", req, id);
}
K.O |
|
05 Jan 2016, Konstantin Olchanski, Suggestion, 64 bit bank type
|
> I've seen that a similar question has been asked in 2011 but I'll ask again in
> case there are any updates. Is there any way to write 64-bit data words to MIDAS
> banks (other than breaking them up in to two 32-bit words, such as 2 DWORDs)
> currently? And if not, is there any plan to introduce this feature in the future?
There is no "breaking them up" as such, you can treat a midas bank as a char* array
and store arbitrary data inside. In this sense, "there is no need" for a special 64-bit bank type.
For endian-ness conversion (if such things still matter, big-endian PPC CPUs still exist), single 64-bit
word converts the same as two 32-bit words, so here also "there is no need", once can use banks of
DWORD with equal effect.
The above applies equally to 64-bit integers and 64-bit double-precision IEEE-754 floating point
numbers.
But specifically for 64-bit values, such as float64, there is a big gotcha.
The MIDAS banks structure goes to great lengths to make sure each data type is correctly aligned,
and gets it exactly wrong for 64-bit quantities - all because the bank header is three 32-bit words.
bankhheader1
bh2
bh3
bankdata1 <--- misaligned
...
bankdataN
bh1
bh2
bh3
banddata1 <--- aligned
... etc
So we could introduce QWORD banks today, but inside the midas file, they will be misaligned defeating
the only purpose of adding them.
I guess the misalignement could be cured by adding dummy words, dummy banks, dummy bank
headers, etc.
I figure this problem dates all the way bank where alignement to 16-bits was just getting important.
Today, in the VME word, I have to align things on 128-bit boundaries (for 2eSST 2x2 DWORD transfers).
So back to your question, what advantage do you see in using a QWORD bank instead of putting the
same data in a DWORD bank?
K.O. |
|
28 Jan 2016, Konstantin Olchanski, Release, Final MIDAS JSON-RPC API
|
> > The final bits of the JSON-RPC API to MIDAS are committed.
JSON-RPC methods are now provided for all old ODBxxx() javascript functions, except ODBGetMsg().
The currently present RPC methods are sufficient to write the MIDAS "programs" and "alarms" pages
purely in HTML+Javascript (see the git branch feature/mhttpd_js). These pages can be served i.e. by apache httpd
with midas mhttpd only required to service the RPC requests.
Please see .../examples/javascript1/example.html on how to use the new RPC methods.
K.O.
P.S. Note how many examples use the generic mjsonrpc_call() because I did not write the corresponding
javascript functions - I wore out the cut-and-paste button on my keyboard. All are welcome to contribute
the missing functions, post them here or email them to me, I will commit them to midas git. |
|