Mjsonrpc

• Midas Doc
• Feature Listing
• Application Listing
• Online Database
• FAQ
• Index

MIDAS JSON-RPC interface

Links

• MIDAS Javascript library
• Custom Page
• Custom Page Features
• mhttpd

JSON general information

JSON is a lightweight data-interchange format usually associated with Javascript and web programming. It is a popular choice as replacement for older general purpose data formats such as XML. JSON is defined by RFC-7159 (read more at http://www.json.org/). When necessary, the overhead of text encoded JSON is reduced by compressing JSON documents (using gzip), or by using binary-encoded JSON.

JSON documents look like this:

{ "Runinfo" : { "State" : 1, "Online Mode" : 1, "Run number" : 13585 } }

Note that the JSON standard is incompatible with IEEE Standard for Floating-Point Arithmetic (IEEE 754), specifically, there is no standard way to encode the special numerical values +Infinity, -Infinity and NaN ("-0.0" is ok).

In MIDAS, JSON support is provided by an ODB data encoder (in odb.c), an ODB "JSON paste" decoder (in json_paste.cxx) and a general purpose JSON encoder/decoder (mjson.h, mjson.cxx, see https://daq.triumf.ca/~daqweb/doc/midas-devel/html/mjson_8h.html and https://daq.triumf.ca/~daqweb/doc/midas-devel/html/class_m_json_node.html).

The MIDAS implementation of JSON has following variances from the JSON standard:

• numerical values +Infinity, -Infinity and NaN are encoded and decoded as JSON strings "Infinity", "-Infinity" and "NaN". This is compatible with most in-browser JSON implementations. See also http://stackoverflow.com/questions/1423081/json-left-out-infinity-and-nan-json-status-in-ecmascript and similar.
• numerical values that are usually presented as hex numbers, such a DWORD values, are encoded as JSON strings, i.e. "0x55b961c8".

JSON encoding of ODB data

MIDAS has 3 way to encode ODB data into JSON:

• "save" format for saving all ODB data and metadata, ODB can be fully reloaded or restored from an ODB JSON save file. Links are preserved as links, upper and lower case in names of ODB keys is preserved.
• "db_values" format for exporting ODB data to web pages: links are followed to their final values, ODB key names are converted to lower case for use with case-sensitive languages such as Javascript.
• "list" format encodes a single ODB directory and returns the full information printed by the odbedit "ls -l" command.

For example:

JSON "save" format: db_copy_json_save()

The "save" format encodes all ODB data and metadata. ODB can be fully reloaded or restored from ODB JSON save files.

$ odbedit
odbedit> cd runinfo
[local:testexpt:S]/Runinfo>json
status: 1, json: {
  "State/key" : { "type" : 7, "access_mode" : 7, "last_written" : 1438212553 },
  "State" : 1,
  "Online Mode/key" : { "type" : 7, "access_mode" : 7, "last_written" : 1436830326 },
  "Online Mode" : 1,
  "Run number/key" : { "type" : 7, "access_mode" : 7, "last_written" : 1438212481 },
  "Run number" : 13585,
  "Transition in progress/key" : { "type" : 7, "access_mode" : 7, "last_written" : 1438212553 },
  "Transition in progress" : 0,
  "Start abort/key" : { "type" : 7, "access_mode" : 7, "last_written" : 1438212552 },
  "Start abort" : 0,
  "Requested transition/key" : { "type" : 7, "access_mode" : 7, "last_written" : 1436923816 },
  "Requested transition" : 0,
  "Start time/key" : { "type" : 12, "item_size" : 32, "access_mode" : 7, "last_written" : 1438212481 },
  "Start time" : "Wed Jul 29 16:28:01 2015",
  "Start time binary/key" : { "type" : 6, "access_mode" : 7, "last_written" : 1438212481 },
  "Start time binary" : "0x55b96181",
  "Stop time/key" : { "type" : 12, "item_size" : 32, "access_mode" : 7, "last_written" : 1438212552 },
  "Stop time" : "Wed Jul 29 16:29:12 2015",
  "Stop time binary/key" : { "type" : 6, "access_mode" : 7, "last_written" : 1438212552 },
  "Stop time binary" : "0x55b961c8"
}
[local:testexpt:S]/Runinfo>

JSON "db_values" format: db_copy_json_values()

The "db_values" format is intended for easy web page development - main thing is to return easy to parse ODB values with minimal metadata (only ODB last_written is provided).

Because most programming languages and most JSON parser (including Javascript) are case sensitive, but ODB key names are not, this JSON encoding normalizes ODB key names by converting them to lower-case. (Converting spaces to underscores is also a possibility).

This allows easy use of parsed JSON in Javascript, i.e. runinfo.state or runinfo["run number"] while avoiding case-matching getter functions (i.e. get_case_insensitive_property(runinfo, "state")).

Symlinks are followed to their final values and subdirectories are recursed.

$ odbedit
odbedit> cd runinfo
[local:testexpt:S]/Runinfo>jsvalues
status: 1, json: {
  "state/last_written" : 1438212553,
  "state" : 1,
  "online mode/last_written" : 1436830326,
  "online mode" : 1,
  "run number/last_written" : 1438212481,
  "run number" : 13585,
  "transition in progress/last_written" : 1438212553,
  "transition in progress" : 0,
  "start abort/last_written" : 1438212552,
  "start abort" : 0,
  "requested transition/last_written" : 1436923816,
  "requested transition" : 0,
  "start time/last_written" : 1438212481,
  "start time" : "Wed Jul 29 16:28:01 2015",
  "start time binary/last_written" : 1438212481,
  "start time binary" : "0x55b96181",
  "stop time/last_written" : 1438212552,
  "stop time" : "Wed Jul 29 16:29:12 2015",
  "stop time binary/last_written" : 1438212552,
  "stop time binary" : "0x55b961c8"
}
[local:testexpt:S]/Runinfo>

JSON "ls" format: db_copy_json_ls()

The "ls" format returns all the data printed by odbedit command "ls -l" and shown by the mhttpd ODB editor web page. It is intended for implementing web ODB editor functions.

Note how subdirectories are encoded as empty JSON objects "{}".

$ odbedit
odbedit> cd experiment
[local:testexpt:S]/Experiment>jsls
jsls "/Experiment", status: 1, json: {
  "Name/key" : { "type" : 12, "item_size" : 32, "access_mode" : 7, "last_written" : 1448038414 },
  "Name" : "testexpt",
  "Buffer sizes" : { },
  "edit on start" : { },
  "Security" : { },
  "Transition debug flag/key" : { "type" : 7, "access_mode" : 7, "last_written" : 1436830326 },
  "Transition debug flag" : 0,
  "Transition timeout/key" : { "type" : 7, "access_mode" : 7, "last_written" : 1436830326 },
  "Transition timeout" : 30000,
  ...

Access to ODB arrays

Some (not all) JSON-RPC methods allow access to individual array elements in ODB, i.e. /equipment/rpcexample/settings/a[10].

RPC methods that can do this are:

• db_copy
• db_get_values
• db_paste

Supported array index syntax

An array index syntax has been implemented to make it easier to write individual array elements (Table 1)

Table 1

Array Syntax	Array Elements	Number of elements/data values
a[1]	1	1
a[1,2,3]	1,2,3	3
a[1-3]	1,2,3	3
a[3-1]	3,2,1	3
a[1,2,3-5,6]	1,2,3,4,5,6	6
a[1-3,4-6,7-9]	1,2,3,4,5,6,7,8,9	9
a[3-0,6-4,9-7]	3,2,1,6,5,4,9,8,7	9
a[4,2,5-6,8]	4,2,5,6,8	5
a	0-n	n **

^** n=number of data values supplied - see #array name only

Writing an array

Because each array element is accessed individually, each array element value comes with a corresponding status value of odb db_get_data_index(), encoded as an array (i.e. rpc.result.status in example below). Note that the data to be written to the array elements must be enclosed by their own square brackets [] and the number of data values must match the number of array elements listed (except when the array name is used alone - see #note). Too few data values will throw an error, too many will be ignored. If the array in the ODB is not large enough for the number of elements and data to be written, it will be expanded automatically.

Writing to a variable and an array

The following example shows

• writing "9" to an integer variable called "test", and
• writing values to a number of elements of the array "array"

Both "test" and "array" are keys in the ODB subdirectory /equipment/rpcexample/settings/.

  mjsonrpc_db_paste(["/equipment/rpcexample/settings/test","/equipment/rpcexample/settings/array[3-1,4,5,8-10]"], [9,[1,2,3,4,5,8,9,10]]).then(function(rpc) {
       document.getElementById("wstatus").innerHTML ='Set Status='+ rpc.result.status
  }).catch(function(error) {
       mjsonrpc_error_alert(error);
    });

Assuming the array was filled with zeroes to start with, after writing to the array and reading it back, it would now contain the values

Array= 0,3,2,1,4,5,0,0,8,9,10,11

Set Status = 1,1,1,1,1,1,1,1,1 (rpc.result.status)

There are 10 elements in the rpc.result.status array, where 1=success. The first element is the status after writing "test", the next eight the status after writing each array element.

---

Array automatically expanded

If the array in the ODB contains 12 elements, writing

mjsonrpc_db_paste(["/equipment/rpcexample/settings/test","/equipment/rpcexample/settings/array[3-1,4,5,8-10,14]"], [9,[1,2,3,4,5,8,9,10,14] ]).then(function(rpc) { ...

will automatically expand the array to 15 elements.

---

Too few data values

Supplying too few data values, e.g. writing

mjsonrpc_db_paste(["/equipment/rpcexample/settings/array[4,5,8-10]"], [ [4,5,8,9] ]).then(function(rpc) { ...

should result in an error (bug - crashes program) because there aren't enough data words supplied (4) for the number of indices (5).

---

Array name only supplied

If you supply just the array name, then you do not need to supply the same number of elements as the array length. For example,

mjsonrpc_db_paste(["/equipment/rpcexample/settings/array"],[ [7,6,0] ]).then(function(rpc){ ...

will write the first three elements of the array to 7,6,0. Other array elements will be untouched. In this case, there is only one status value for the whole array.

Array= 7,6,0,1,4,5,0,0,8,9,10,11

Set Status = 1 (rpc.result.status)

---

Set all elements to one value

If you want to set the whole array to 1, you might write

var ones=new Array(10);
for (i=0; i<ones.length; i++)
  ones[i]=1;
mjsonrpc_db_paste(["/equipment/rpcexample/settings/array"], [ones]).then(function(rpc) { ...

Reading an array

In the same way, you can read all or part of an array.

var array = new Array();
mjsonrpc_db_get_values(["/Runinfo","/Experiment/name","/Equipment/rpcexample/Settings/array","/Equipment/rpcexample/Settings/array[3-6,11]"]).then(function(rpc) {
      document.getElementById("status").innerHTML = 'Status : '+ result.status  
      var runinfo= rpc.result.data[0] 
      var name =  rpc.result.data[1]
      array = rpc.result.data[2]   // whole array
      document.getElementById("array").innerHTML ='Array='+ array
      array=rpc.result.data[3]    // array indices
      document.getElementById("array_ele").innerHTML ='Array elements='+ array
}).catch(function(error) {
   mjsonrpc_error_alert(error);
});

This example gives

Array=1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,0

Array elements=4,5,6,7,12

Status : 1,1,1,1,1,1,1,1 (rpc.result.status)

Note that the "Array elements" line has read the contents of indices 3-6 and 11 as requested (5 indices in total). The status array contains 8 values. The last 5 values are the status from reading the 5 array elements.

JSON-RPC general information

• JSON-RPC standard: https://daq.triumf.ca/~daqweb/doc/midas-devel/doc/jsonrpc/JSON-RPC%202.0%20Specification.html
• JSON-RPC home page: http://www.jsonrpc.org/specification
• JSON-RPC via HTTP transport: https://daq.triumf.ca/~daqweb/doc/midas-devel/doc/jsonrpc/simple%20is%20better%20-%20JSON-RPC%202.0%20Transport_%20HTTP.html
• http://www.simple-is-better.org/json-rpc/index.html

Javascript client library

MIDAS provides a simple client library:

• documentation, see the mjsonrpc functions: https://daq.triumf.ca/~daqweb/doc/midas-devel/html/group__mjsonrpc__js.html
• source code, see the mjsonrpc functions: https://daq.triumf.ca/~daqweb/doc/midas-devel/resources/mhttpd.js

The MIDAS JSON RPC client library is based on the Javascript Promise pattern:

• http://www.html5rocks.com/en/tutorials/es6/promises/
• https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Promise

A simple example using the Promise API to display an ODB value on a web page:

mjsonrpc_db_get_values(["/runinfo/run number"]).then(function(rpc) {
   document.getElementById("run_number").innerHTML = rpc.response.data[0];
}).catch(function(error) {
   mjsonrpc_error_alert(error);
});

A partial list of available javasctipt functions. For the full list, and for full documentation, please go to the doxygen-generated documentation at https://daq.triumf.ca/~daqweb/doc/midas-devel/html/mhttpd_8js.html

• function mjsonrpc_set_url (url) - set the URL of the MIDAS JSON-RPC server, if different from web page URL. Cross-site access is fully supported (see CORS).
• function mjsonrpc_call (method, params, id) - call arbitrary RPC method
• function mjsonrpc_start_program (name, id, callback, error_callback)
• function mjsonrpc_stop_program (name, unique, id, callback, error_callback)
• function mjsonrpc_db_get_values (paths, id, callback, error_callback) - read ODB values
• function mjsonrpc_db_paste (paths, values, id, callback, error_callback) - write ODB values

Examples

Only most basic examples are given here. For more examples:

• look elsewhere on this page
• look at the example experiment examples/javascript1/example.html
• look at the doxygen-generated documentation for mjsonrpc_xxx functions
• look at the implementation of the functions
• look at the implementation of mhttpd internal web pages (functions mhttpd_xxx in mhttpd.js)

$ curl -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","id":null,"method":"db_get_values","params":{"paths":["/runinfo/run number"]}}' 'http://localhost:8080?mjsonrpc'
{"jsonrpc": "2.0","result":{"data":[324],"status":[1],"last_written":[1443570804]},"id":null}
$ 

Schema (List of all RPC methods)

The MIDAS JSON RPC API is documented via an automatically generated JSON Schema.

The JSON Schema ("describes your JSON data format") is not a well established standard (there is no final RFC) but seems to be in common use. Multiple 3rd party tools exist to visualize, explore and interface with JSON documents described by JSON Schemas. For more general information go here:

• http://json-schema.org/
• https://tools.ietf.org/html/draft-zyp-json-schema-04

The MIDAS JSON RPC Schema is automatically generated by mhttpd and is linked by the mhttpd "Help" page in JSON and in text format.

For reference, a recent copy of the JSON RPC API Schema is pasted here from the mhttpd "help" page. For final, complete and up-to-date schema, please get your own copy from the midas "help" page.

In this schema, "?" means an optional parameter, "[]" means an array parameter. (There is an artefact: all method names have question marks because they are optional as far as the schema generator is concerned).

MIDAS JSON RPC API Schema

------------------------------------------------------------------------
Autogenerated schema for all MIDAS JSON-RPC methods
------------------------------------------------------------------------
cm_exist?      | calls MIDAS cm_exist() to check if given MIDAS program is running
               | -------------------------------------------------------
               | params   | name           | string         | name of the program, corresponding to ODB /Programs/name
               |          | unique?        | bool           | bUnique argument to cm_exist()
               | -------------------------------------------------------
               | result   | status         | integer        | return status of cm_exist()
------------------------------------------------------------------------
cm_shutdown?   | calls MIDAS cm_shutdown() to stop given MIDAS program
               | -------------------------------------------------------
               | params   | name           | string         | name of the program, corresponding to ODB /Programs/name
               |          | unique?        | bool           | bUnique argument to cm_shutdown()
               | -------------------------------------------------------
               | result   | status         | integer        | return status of cm_shutdown()
------------------------------------------------------------------------
db_copy?       | get copies of given ODB subtrees in the "save" json encoding
               | -------------------------------------------------------
               | params   | paths[]        | array of ODB subtree paths, see note on array indices
               |          |                | array of       | string 
               | -------------------------------------------------------
               | result   | data[]         | copy of ODB data for each path
               |          |                | array of       | object 
               |          | status[]       | return status of db_copy_json() for each path
               |          |                | array of       | integer
               |          | last_written[] | last_written value of the ODB subtree for each path
               |          |                | array of       | number 
------------------------------------------------------------------------
db_create?     | get copies of given ODB subtrees in the "save" json encoding
               | -------------------------------------------------------
               | params[] | array of ODB paths to be created
               |          | array of       | arguments to db_create() and db_resize()
               |          |                | path           | string  | ODB path
               |          |                | type           | integer | MIDAS TID_xxx type
               |          |                | array_length?  | integer | optional array length, default is 1
               |          |                | string_length? | integer | for TID_STRING, optional string length, default is NAME_LENGTH
               | -------------------------------------------------------
               | result   | status[]       | return status of db_create() for each path
               |          |                | array of       | integer
------------------------------------------------------------------------
db_get_values? | get values of ODB data from given subtrees
               | -------------------------------------------------------
               | params   | paths[]        | array of ODB subtree paths, see note on array indices
               |          |                | array of       | string 
               | -------------------------------------------------------
               | result   | data[]         | values of ODB data for each path, all key names are in lower case, all symlinks are followed
               |          |                | array of       | any    
               |          | status[]       | return status of db_copy_json() for each path
               |          |                | array of       | integer
               |          | last_written[] | last_written value of the ODB subtree for each path
               |          |                | array of       | number 
------------------------------------------------------------------------
db_paste?      | write data into ODB
               | -------------------------------------------------------
               | params   | paths[]        | array of ODB subtree paths, see note on array indices
               |          |                | array of       | string 
               |          | values[]       | data to be written using db_paste_json()
               |          |                | array of       | any    
               | -------------------------------------------------------
               | result   | status[]       | return status of db_paste_json() for each path
               |          |                | array of       | integer
------------------------------------------------------------------------
get_debug?     | get current value of mjsonrpc_debug
               | -------------------------------------------------------
               | params   | any            | there are no input parameters
               | -------------------------------------------------------
               | result   | integer        | current value of mjsonrpc_debug
------------------------------------------------------------------------
get_schema?    | Get the MIDAS JSON-RPC schema JSON object
               | -------------------------------------------------------
               | params   | any            | there are no input parameters
               | -------------------------------------------------------
               | result   | object         | returns the MIDAS JSON-RPC schema JSON object
------------------------------------------------------------------------
null?          | RPC method always returns null
               | -------------------------------------------------------
               | params   | any            | method parameters are ignored
               | -------------------------------------------------------
               | result   | null           | always returns null
------------------------------------------------------------------------
set_debug?     | set new value of mjsonrpc_debug
               | -------------------------------------------------------
               | params   | integer        | new value of mjsonrpc_debug
               | -------------------------------------------------------
               | result   | integer        | new value of mjsonrpc_debug
------------------------------------------------------------------------
start_program? | start MIDAS program defined in ODB /Programs/name
               | -------------------------------------------------------
               | params   | name           | string         | name of the program, corresponding to ODB /Programs/name
               | -------------------------------------------------------
               | result   | status         | integer        | return status of ss_system()
------------------------------------------------------------------------
user_example1? | example of user defined RPC method that returns up to 3 results
               | -------------------------------------------------------
               | params   | arg            | string         | example string argment
               |          | optional_arg?  | integer        | optional example integer argument
               | -------------------------------------------------------
               | result   | string         | string         | returns the value of "arg" parameter
               |          | integer        | integer        | returns the value of "optional_arg" parameter
------------------------------------------------------------------------
user_example2? | example of user defined RPC method that returns more than 3 results
               | -------------------------------------------------------
               | params   | arg            | string         | example string argment
               |          | optional_arg?  | integer        | optional example integer argument
               | -------------------------------------------------------
               | result   | string1        | string         | returns the value of "arg" parameter
               |          | string2        | string         | returns "hello"
               |          | string3        | string         | returns "world!"
               |          | value1         | integer        | returns the value of "optional_arg" parameter
               |          | value2         | number         | returns 3.14
------------------------------------------------------------------------
user_example3? | example of user defined RPC method that returns an error
               | -------------------------------------------------------
               | params   | arg            | integer        | integer value, if zero, throws a JSON-RPC error
               | -------------------------------------------------------
               | result   | status         | integer        | returns the value of "arg" parameter