Remote Procedure Calls (RPC)

From MidasWiki
Revision as of 15:58, 26 April 2023 by Bsmith (talk | contribs)
Jump to navigation Jump to search


RPC in a nutshell

Midas uses a Remote Procedure Call (RPC) system to allow different midas clients to talk to each other directly. One client can issue a command to another, and the second client will perform some action and send a response back to the caller.

Internal usage in midas

Internally, midas uses the RPC system to handle run transitions, ODB hotlinks and more.

For example, if you start a run using the web interface, mhttpd (the webserver) will send an RPC_RC_TRANSITION message to clients that want to handle the begin-of-run transition. Each client responds with whether it's okay for the transition to proceed.

Any clients that are running on a remote server use RPC for all their communications with midas. The special mserver program (running on the main host) is responsible for handling these requests. For example, whereas a local client can read an ODB value directly from shared memory on the main host; a remote client instead issues an RPC call to mserver, which reads the value from shared memory and returns it to the caller.

At the very lowest level, each program checks if it has any outstanding RPC requests to handle as part of the cm_yield() function, which should be called periodically by every midas client.

Use in user code - JRPC

One of the RPC commands, RPC_JRPC, is very flexible and allows the user to easily integrate custom commands into their programs. There are interfaces in C++, python and javascript.

The calling code specifies:

  • The program it wants to talk to
  • The command it wants to issue (generally a short string)
  • Some arguments (either plain text or JSON)

The client then:

  • Decides what to do based on the command and arguments
  • Responds with a status code and text message (either plain text or JSON)

Some example uses of this system are:

  • Have a client turn some equipment off when the user clicks a button on a webpage.
  • Offload some complex calculations to another program (e.g. if your main client is in C++ but it's much easier to implement the calculations in python).
  • Implement a web-based plot display that gets data as JSON from a python/C++ client
  • Much much more...

Benefits vs ODB hotlinks

Often users implement inter-process communication using ODB hotlinks (where a client can say "tell me whenever /Path/to/something in the ODB changes"). Although this works for simple cases, the JRPC system is much more flexible and robust.

  • Much easier to specify arguments. Avoids race conditions if you were to use multiple ODB keys to tell the target what to do.
  • There is no "competition" for the ODB key(s). Multiple clients can call the same RPC without interfering with each other.
  • More flexible response format as you can return a (very) long string. Alternatives would be to put the response in an ODB value (limited maximum length) or in a file on disk (extra overhead, and possibly annoying to deal with stale files over NFS).

Registering your JRPC handler

If you want other midas clients to be able to talk to you, you need to register an RPC handler. You specify the RPC you're listening for (RPC_JRPC in this case) and a function that midas should call.

C++

The midas function to use is: 

# * id - RPC_JRPC in this case
# * func - Function to call (see below)
INT cm_register_function(INT id, INT(*func)(INT, void **))

The callback function (func in the above example) should look something like the below: 

# * index is the RPC id (RPC_JRPC in our case).
# * prpc_param is a list of pointers that we need to cast to the appropriate types:
#   - [0] is a C-string containing the "command"
#   - [1] is a C-string containing the "arguments"
#   - [2] is a C-string where you can write a response
#   - [3] is an integer stating the maximum response length the caller expects
# * The function should return SUCCESS if everything is okay.
INT my_rpc_callback(INT index, void *prpc_param[]) {
  const char* cmd  = CSTRING(0);
  const char* args = CSTRING(1);
  char* return_buf = CSTRING(2);
  int   return_max_length = CINT(3);

  if (strcmp(cmd, "add_one") == 0) {
     # Silly example that parses `args` as a number and returns x+1 in return_buf
     int val = atoi(args);
     val += 1;
     sprintf(return_buf, "%d", val);
  } else {
     # Up to you how to handle error conditions - report in return_buf or via return value.
     sprintf(return_buf, "err");
  }

  return SUCCESS;
}

...

cm_register_function(RPC_JRPC, my_rpc_callback);

C++ (TMFE)

In the TMFE framework, an RPC handler has already been registered.

You should implement the HandleRpc() function in your class that derives from TMFeEquipment: 

 TMFeResult HandleRpc(const char* cmd, const char* args, std::string& result) {
    if (strcmp(cmd, "add_one") == 0) {
       # Silly example that parses `args` as a number and returns x+1 as the result
       int val = atoi(args);
       val += 1;
       sprintf(return_buf, "%d", val);
    } else {
       # In TMFE, you must report errors via the result.
       result = "err";
    }
 
    return TMFeOk();
 }

Python

The python interface is very similar to the C++ interface for registering RPC callbacks.

In midas.client you call register_jrpc_callback(callback, return_success_even_on_failure).

The callback function should be defined as def rpc_handler(client, cmd, args, max_len), where the arguments are:

  • client (midas.client.MidasClient)
  • cmd (str) - The command user wants to execute
  • args (str) - Other arguments the user supplied
  • max_len (int) - The maximum string length the user accepts in the return value

The callback function should return either a tuple of (int, str) or just an int. The integer should be a status code from midas.status_codes. The string, if present, should be any text that should be returned to the caller.

The return_success_even_on_failure argument to register_jrpc_callback deals with a subtlety regarding mjsonrpc (the web interface for calling JRPC functions), as it does not return any message if the status code isn't "SUCCESS". This can be annoying if you want to show a specific error message to the user, and not have them trawl through the midas message log. If you set this parameter to False, then you get the "normal" behaviour, where the returned status code and result string are exactly what is returned from the callback function. If you set this parameter to True, then the status code will always be "SUCCESS", and the result string will be JSON-encoded text of the form {"code": 604, "msg": "Some error message"}.

Example code is: 

def my_rpc_callback(client, cmd, args, max_len):
   ret_int = midas.status_codes["SUCCESS"]
   ret_str = ""
   
   if cmd == "add_one":
       val = int(args)
       val = val + 1
       
       ret_str = str(val)
   else:
       ret_int = midas.status_codes["FE_ERR_DRIVER"]
       ret_str = "err"    
       
   return (ret_int, ret_str)

if __name__ == "__main__":
   client = midas.client.MidasClient("pytest")
   
   # Register our function.
   client.register_jrpc_callback(my_rpc_callback, True)
   
   # Spin forever. Program can be killed by Ctrl+C or
   # "Stop Program" through mhttpd.
   while True:
       client.communicate(10)

Calling JRPC on other clients

In all languages, you specify which program you want to talk to, a command and some arguments. You will then be returned a status code and a message string. As you are the person writing both sides of the JRPC communication, it is up to you to define what format the arguments and returned message take (e.g. plain text or JSON).

C++

In C++ you use cm_connect_client() and rpc_client_call() to call JRPC on another program. 

INT status = SUCCESS;

# First connect to the other client
HNDLE hConn = 0; 
status = cm_connect_client("other_program", &hConn);

if (status != SUCCESS) {return;}

# Then build arguments and call the RPC
std::string cmd = "add_one";
std::string args = "6";
char buf[1000];
int buf_length = 1000;

status = rpc_client_call(hConn, RPC_JRPC, cmd.c_str(), args.c_str(), buf, buf_length);

if (status != SUCCESS) {return;}

# 'buf' should contain "7" in our example.

Note that json.hpp is a very useful tool if you want to use JSON in your C++ code.

Python

In python you use the connect_to_other_client() and jrpc_client_call() functions in midas.client. 

client = midas.client.MidasClient("my_program")

# First connect to the other client
other = client.connect_to_other_client("other_program")

# Then build arguments and call the RPC
command = "add_one"
args = "6"
retval = client.jrpc_client_call(other, command, args)

# retval should now contain "7"
# An exception will be raised if the other program returned a non-SUCCESS status.

Javascript

In javascript you use the mjsonrpc_call function. 

# In JS, client to connect to is specified as a parameter
let params = Object();
params.client_name = "other_program";
params.cmd = "add_one";
params.args = "6";

# Call RPC asynchronously. Returns a Promise that you deal with using "then()"
mjsonrpc_call("jrpc", params).then(function(rpc) {
   if (rpc.result.status != 1) {
      // Other program reported a problem
   }

   // rpc.result.reply should contain "7"
}).catch(function(error) {
   mjsonrpc_error_alert(error);
});
Retrieved from "https://daq00.triumf.ca/MidasWiki/index.php?title=Remote_Procedure_Calls_(RPC)&oldid=3204"