Python

Since December 2019, midas has shipped with python wrappers allowing midas clients to be written in python.

The main features are:

• A file reader, which works with python 2.7 and python 3.x
• A client library and frontend framework which work with python 3.x only

To use the python libraries, you must have installed midas, and compiled midas using cmake. The python code is found at $MIDASSYS/python. You can view example scripts and frontends on BitBucket.

Midas file reader

midas/file_reader.py is a pure-python tool that lets you read midas files. It presents a very pythonic interface, allowing you to do things like

import midas.file_reader

my_file = midas.file_reader.MidasFile("/path/to/file.mid.lz4")

for event in my_file:
  event.dump()

To access information in the ODB dump, you can do something like

import midas.file_reader

my_file = midas.file_reader.MidasFile("/path/to/file.mid.lz4")
odb = my_file.get_bor_odb_dump().data

run_number = odb["Runinfo"]["Run number"]

For more details, see the documentation in $MIDASSYS/python/midas/file_reader.py and example code in $MIDASSYS/python/examples/file_reader.py.

• It works with python 2.7 and python 3.
• It does not require the rest of midas to be compiled.

Client and frontend libraries

• midas/client.py provides pythonic access to a midas experiment, by wrapping midas' C library. It provides nice ODB access through functions like odb_set and odb_get, and can also handle event buffers, callback functions and more.
• midas/frontend.py builds upon the client and provides a framework for writing midas frontends in python. It supports periodic and polled equipment, and can be controlled just like a C frontend.

Many users write scripts in python or bash that call the odbedit command-line tool or the mhttpd web server. We hope that the pythonic midas client tools will simplify and robustify such scripts. The python frontend framework may prove particularly beneficial for controlling devices that natively talk in JSON or other text-based protocols that are tedious to deal with in C. It may also reduce the development time required to write frontends that do not have strict performance requirements.

• They work with python 3 only.
• They require the rest of midas to be compiled, as they use the midas C library. For the correct libraries to be built, you must compile using CMake
• They are designed for ease of use, rather than optimal performance. If performance is critical for your application, you should write your code in C.

The python midas.client library provides support for:

• ODB access - creating/deleting/renaming keys, getting/setting/watching values, creating links
• run transitions - starting/stopping/pausing/resuming runs, and registering callback functions for each transition
• event buffers - connecting, sending events, receiving events
• remote procedure calls - connecting and talking to other midas clients
• alarms - creating and triggering alarms
• messages - sending and getting recent messages
• history system - reading data (since July 2023)

Documentation

Documentation of the tools is written as docstrings in the source code (in $MIDASSYS/python/midas). We do not currently have a web version of the documentation.

The documentation assumes a reasonable familiarity with core midas concepts. See the rest of this wiki if you're unfamiliar with a term or concept in the python documentation.

Installation

We highly recommend using pip and/or virtualenv to manage your python packages, but you can also just edit your paths if desired.

To install this package with pip:

pip install -e $MIDASSYS/python --user
# The -e flag makes this an "editable" install. If you upgrade midas in future,
# python will automatically see the new code without you having to re-install.
# The --user flag may or may not be needed, depending on your pip setup.

To "install" by editing environment variables:

export PYTHONPATH=$PYTHONPATH:$MIDASSYS/python

Examples

You can now use import midas and import midas.client etc in your python scripts.

$MIDASSYS/python/examples contains several examples of complete clients and frontends written in python.

midas.client ODB access

Here we show a very simple python script for interacting with the ODB. Note that we do not have any error-checking. If any call to the midas library fails, an exception will be raised.

import midas.client

"""
A simple example program that connects to a midas experiment,
reads an ODB value, then sets an ODB value.

Expected output is:

```
The experiment is currently stopped
The new value of /pyexample/eg_float is 5.670000
```
"""

if __name__ == "__main__":
    client = midas.client.MidasClient("pytest")

    # Read a value from the ODB. The return value is a normal python
    # type (an int in this case, but could also be a float, string, bool,
    # list or dict).
    state = client.odb_get("/Runinfo/State")

    if state == midas.STATE_RUNNING:
        print("The experiment is currently running")
    elif state == midas.STATE_PAUSED:
        print("The experiment is currently paused")
    elif state == midas.STATE_STOPPED:
        print("The experiment is currently stopped")
    else:
        print("The experiment is in an unexpected run state")


    # Update or create a directory in the ODB by passing a dict to `odb_set`
    client.odb_set("/pyexample", {"an_int": 1, "eg_float": 4.56})

    # Update a single value in the ODB
    client.odb_set("/pyexample/eg_float", 5.67)

    # Read the value back
    readback = client.odb_get("/pyexample/eg_float")

    print("The new value of /pyexample/eg_float is %f" % readback)

    # Delete the temporary directory we created
    client.odb_delete("/pyexample")

midas.frontend

Example of a basic midas frontend that has one periodic equipment. See $MIDASSYS/examples/multi_frontend.py for an example that uses more features (frontend index, polled equipment, ODB settings etc).

import midas
import midas.frontend
import midas.event

class MyPeriodicEquipment(midas.frontend.EquipmentBase):
    """
    We define an "equipment" for each logically distinct task that this frontend
    performs. For example, you may have one equipment for reading data from a
    device and sending it to a midas buffer, and another equipment that updates
    summary statistics every 10s.
    
    Each equipment class you define should inherit from 
    `midas.frontend.EquipmentBase`, and should define a `readout_func` function.
    If you're creating a "polled" equipment (rather than a periodic one), you
    should also define a `poll_func` function in addition to `readout_func`.
    """
    def __init__(self, client):
        # The name of our equipment. This name will be used on the midas status
        # page, and our info will appear in /Equipment/MyPeriodicEquipment in
        # the ODB.
        equip_name = "MyPeriodicEquipment"
        
        # Define the "common" settings of a frontend. These will appear in
        # /Equipment/MyPeriodicEquipment/Common. The values you set here are
        # only used the very first time this frontend/equipment runs; after 
        # that the ODB settings are used.
        default_common = midas.frontend.InitialEquipmentCommon()
        default_common.equip_type = midas.EQ_PERIODIC
        default_common.buffer_name = "SYSTEM"
        default_common.trigger_mask = 0
        default_common.event_id = 1
        default_common.period_ms = 100
        default_common.read_when = midas.RO_RUNNING
        default_common.log_history = 1
        
        # You MUST call midas.frontend.EquipmentBase.__init__ in your equipment's __init__ method!
        midas.frontend.EquipmentBase.__init__(self, client, equip_name, default_common)
        
        # You can set the status of the equipment (appears in the midas status page)
        self.set_status("Initialized")
        
    def readout_func(self):
        """
        For a periodic equipment, this function will be called periodically
        (every 100ms in this case). It should return either a `midas.event.Event`
        or None (if we shouldn't write an event).
        """
        
        # In this example, we just make a simple event with one bank.
        event = midas.event.Event()
        
        # Create a bank (called "MYBK") which in this case will store 8 ints.
        # data can be a list, a tuple or a numpy array.
        data = [1,2,3,4,5,6,7,8]
        event.create_bank("MYBK", midas.TID_INT, data)
        
        return event

class MyFrontend(midas.frontend.FrontendBase):
    """
    A frontend contains a collection of equipment.
    You can access self.client to access the ODB etc (see `midas.client.MidasClient`).
    """
    def __init__(self):
        # You must call __init__ from the base class.
        midas.frontend.FrontendBase.__init__(self, "myfe_name")
        
        # You can add equipment at any time before you call `run()`, but doing
        # it in __init__() seems logical.
        self.add_equipment(MyPeriodicEquipment(self.client))
        
    def begin_of_run(self, run_number):
        """
        This function will be called at the beginning of the run.
        You don't have to define it, but you probably should.
        You can access individual equipment classes through the `self.equipment`
        dict if needed.
        """
        self.set_all_equipment_status("Running", "greenLight")
        self.client.msg("Frontend has seen start of run number %d" % run_number)
        return midas.status_codes["SUCCESS"]
        
    def end_of_run(self, run_number):
        self.set_all_equipment_status("Finished", "greenLight")
        self.client.msg("Frontend has seen end of run number %d" % run_number)
        return midas.status_codes["SUCCESS"]
    
    def frontend_exit(self):
        """
        Most people won't need to define this function, but you can use
        it for final cleanup if needed.
        """
        print("Goodbye from user code!")
        
if __name__ == "__main__":
    # The main executable is very simple - just create the frontend object,
    # and call run() on it.
    with MyFrontend() as my_fe:
        my_fe.run()

Doing work when the program exits

Since April 2023, the python frontend framework has included the ability to ensure that work gets performed when the program exits (more specifically, just before we disconnect from midas). This may be useful, for example, to reset the state of some hardware before the program exits.

In the frontend framework, this is done by implementing the frontend_exit() function in your frontend class (see the example code above). If you're using a bare MidasClient you can instead call register_disconnect_callback().

To be certain that your function gets called, you should use the "context manager" style to create your client/frontend object.

# Context manager style - guaranteed that `frontend_exit()` will be called, even
# if there's an uncaught exception or exit(0) is called etc.
with MyFrontend() as my_fe:
    my_fe.run()


# Not using a context manager - `frontend_exit()` may NOT be called!
my_fe = MyFrontend()
my_fe.run()

The reason for this is that the best we can do in the latter approach is to hook up the midas disconnection logic to the destructor of the MidasClient object. But python does not guarantee that an object's __del__() function will be called in a timely manner (or even at all!). It depends on your python version, python implementation, and if you do any weird things in your class (e.g. circular references).

In the context manager approach, we can hook up the midas disconnection logic to the __exit__() function, which python DOES guarantee will be called.

Tests

$MIDASSYS/python/tests contains a variety of test scripts. If you wish, you can run them using

cd $MIDASSYS/python/tests
python -munittest

However, note that running these tests will interact with whichever midas experiment is currently active. They will edit the ODB and start and stop runs. If you are running these tests, we suggest doing so in a "dummy"/test experiment.