Python: Difference between revisions

From MidasWiki
Jump to navigation Jump to search
No edit summary
(The midas documentation has moved to https://midas.triumf.ca/docs/)
Tag: Replaced
 
Line 1: Line 1:
Since December 2019, midas has shipped with python wrappers allowing midas clients to be written in python.
The midas documentation has moved. The new documentation about the midas Python libraries is in:
 
* [https://midas.triumf.ca/docs/develop/api/python/ Midas Python API] (modules, clients, frontends, file reader)
The main features are:
* [https://midas.triumf.ca/docs/develop/features/core_system/writing_programs/#python-client Python client] in Hello world! Writing programs in midas (a first example)
* A file reader, which works with python 2.7 and python 3.x
* [https://midas.triumf.ca/docs/develop/getting_started/installation/#set-up-python Set up python] in Installation and environment (installation)
* A client library and frontend framework which work with python 3.x only
 
To use the python libraries, you must have installed midas, and compiled midas using cmake. The python code is found at <code>$MIDASSYS/python</code>. You can view [https://gitlab.triumf.ca/midas/midas/-/tree/develop/python/examples example scripts and frontends on GitLab].
 
== Midas file reader ==
 
<code>midas/file_reader.py</code> is a pure-python tool that lets you read midas files.
It presents a very pythonic interface, allowing you to do things like
 
import midas.file_reader
my_file = midas.file_reader.MidasFile("/path/to/file.mid.lz4")
for event in my_file:
  event.dump()
 
To access information in the ODB dump, you can do something like
 
import midas.file_reader
my_file = midas.file_reader.MidasFile("/path/to/file.mid.lz4")
odb = my_file.get_bor_odb_dump().data
run_number = odb["Runinfo"]["Run number"]
 
For more details, see the documentation in <code>$MIDASSYS/python/midas/file_reader.py</code> and example code in <code>$MIDASSYS/python/examples/file_reader.py</code>.
 
* It works with python 2.7 and python 3.
* It does not require the rest of midas to be compiled.
 
== Client and frontend libraries ==
 
* <code>midas/client.py</code> provides pythonic access to a midas experiment, by wrapping midas' C library. It provides nice ODB access through functions like <code>odb_set</code> and <code>odb_get</code>, and can also handle event buffers, callback functions and more.
* <code>midas/frontend.py</code> builds upon the client and provides a framework for writing midas frontends in python. It supports periodic and polled equipment, and can be controlled just like a C frontend.
 
Many users write scripts in python or bash that call the <code>odbedit</code> command-line tool or the mhttpd web server. We hope that the pythonic midas client tools will simplify and robustify such scripts. The python frontend framework may prove particularly beneficial for controlling devices that natively talk in JSON or other text-based protocols that are
tedious to deal with in C. It may also reduce the development time required to write frontends that do not have strict performance requirements.
 
* They work with python 3 only.
* They require the rest of midas to be compiled, as they use the midas C library. For the correct libraries to be built, you must compile using CMake
* They are designed for ease of use, rather than optimal performance. If performance is critical for your application, you should write your code in C.
 
The python <code>midas.client</code> library provides support for:
* ODB access - creating/deleting/renaming keys, getting/setting/watching values, creating links
* run transitions - starting/stopping/pausing/resuming runs, and registering callback functions for each transition
* event buffers - connecting, sending events, receiving events
* remote procedure calls - connecting and talking to other midas clients
* alarms - creating and triggering alarms
* messages - sending and getting recent messages
* history system - reading data (since July 2023)
 
== Documentation ==
 
Documentation of the tools is written as docstrings in the source code (in <code>$MIDASSYS/python/midas</code>). We do not currently have a web version of the documentation.
 
The documentation assumes a reasonable familiarity with core midas concepts. See the rest of this wiki if you're unfamiliar with a term or concept in the python documentation.
 
== Installation ==
 
We highly recommend using [https://pypi.org/project/pip/ pip] and/or [https://virtualenv.pypa.io/en/latest/ virtualenv] to manage your python packages, but you can also just edit your paths if desired.
 
To install this package with pip:
 
pip install -e $MIDASSYS/python --user
# The -e flag makes this an "editable" install. If you upgrade midas in future,
# python will automatically see the new code without you having to re-install.
# The --user flag may or may not be needed, depending on your pip setup.
 
The python code has some <b>optional dependencies</b> that enable extra functionality:
 
* <code>numpy</code> can be used to improve performance in midas python frontends that are running at high data rates
* <code>lz4</code> allows the midas file reader to parse .mid.lz4 files (.mid and .mid.gz files do not need any extra pacakges)
* <code>pyflakes</code> allows the PySequencer to validate the syntax of your script before it gets executed, catching many errors before they are encountered
 
As of September 2026, the optional dependencies will be installed automatically when you "pip install" the midas package.
 
 
If you don't want to use pip, a hacky way to "install" is by editing environment variables. This will not install any of the optional dependencies.
 
export PYTHONPATH=$PYTHONPATH:$MIDASSYS/python
 
== Examples ==
 
You can now use <code>import midas</code> and <code>import midas.client</code> etc in your python scripts.
 
[https://gitlab.triumf.ca/midas/midas/-/tree/develop/python/examples $MIDASSYS/python/examples] contains several examples of complete clients and frontends written in python.
 
=== midas.client ODB access ===
 
Here we show a very simple python script for interacting with the ODB. Note that we do not have any error-checking. If any call to the midas library fails, an exception will be raised.
 
<nowiki>import midas.client
 
"""
A simple example program that connects to a midas experiment,
reads an ODB value, then sets an ODB value.
 
Expected output is:
 
```
The experiment is currently stopped
The new value of /pyexample/eg_float is 5.670000
```
"""
 
if __name__ == "__main__":
    client = midas.client.MidasClient("pytest")
 
    # Read a value from the ODB. The return value is a normal python
    # type (an int in this case, but could also be a float, string, bool,
    # list or dict).
    state = client.odb_get("/Runinfo/State")
 
    if state == midas.STATE_RUNNING:
        print("The experiment is currently running")
    elif state == midas.STATE_PAUSED:
        print("The experiment is currently paused")
    elif state == midas.STATE_STOPPED:
        print("The experiment is currently stopped")
    else:
        print("The experiment is in an unexpected run state")
 
 
    # Update or create a directory in the ODB by passing a dict to `odb_set`
    client.odb_set("/pyexample", {"an_int": 1, "eg_float": 4.56})
 
    # Update a single value in the ODB
    client.odb_set("/pyexample/eg_float", 5.67)
 
    # Read the value back
    readback = client.odb_get("/pyexample/eg_float")
 
    print("The new value of /pyexample/eg_float is %f" % readback)
 
    # Delete the temporary directory we created
    client.odb_delete("/pyexample")
</nowiki>
 
=== midas.frontend ===
 
Example of a basic midas frontend that has one periodic equipment. See <code>$MIDASSYS/examples/multi_frontend.py</code> for an example that uses more features (frontend index, polled equipment, ODB settings etc).
 
<nowiki>import midas
import midas.frontend
import midas.event
 
class MyPeriodicEquipment(midas.frontend.EquipmentBase):
    """
    We define an "equipment" for each logically distinct task that this frontend
    performs. For example, you may have one equipment for reading data from a
    device and sending it to a midas buffer, and another equipment that updates
    summary statistics every 10s.
   
    Each equipment class you define should inherit from
    `midas.frontend.EquipmentBase`, and should define a `readout_func` function.
    If you're creating a "polled" equipment (rather than a periodic one), you
    should also define a `poll_func` function in addition to `readout_func`.
    """
    def __init__(self, client):
        # The name of our equipment. This name will be used on the midas status
        # page, and our info will appear in /Equipment/MyPeriodicEquipment in
        # the ODB.
        equip_name = "MyPeriodicEquipment"
       
        # Define the "common" settings of a frontend. These will appear in
        # /Equipment/MyPeriodicEquipment/Common. The values you set here are
        # only used the very first time this frontend/equipment runs; after
        # that the ODB settings are used.
        default_common = midas.frontend.InitialEquipmentCommon()
        default_common.equip_type = midas.EQ_PERIODIC
        default_common.buffer_name = "SYSTEM"
        default_common.trigger_mask = 0
        default_common.event_id = 1
        default_common.period_ms = 100
        default_common.read_when = midas.RO_RUNNING
        default_common.log_history = 1
       
        # You MUST call midas.frontend.EquipmentBase.__init__ in your equipment's __init__ method!
        midas.frontend.EquipmentBase.__init__(self, client, equip_name, default_common)
       
        # You can set the status of the equipment (appears in the midas status page)
        self.set_status("Initialized")
       
    def readout_func(self):
        """
        For a periodic equipment, this function will be called periodically
        (every 100ms in this case). It should return either a `midas.event.Event`
        or None (if we shouldn't write an event).
        """
       
        # In this example, we just make a simple event with one bank.
        event = midas.event.Event()
       
        # Create a bank (called "MYBK") which in this case will store 8 ints.
        # data can be a list, a tuple or a numpy array.
        data = [1,2,3,4,5,6,7,8]
        event.create_bank("MYBK", midas.TID_INT, data)
       
        return event
 
class MyFrontend(midas.frontend.FrontendBase):
    """
    A frontend contains a collection of equipment.
    You can access self.client to access the ODB etc (see `midas.client.MidasClient`).
    """
    def __init__(self):
        # You must call __init__ from the base class.
        midas.frontend.FrontendBase.__init__(self, "myfe_name")
       
        # You can add equipment at any time before you call `run()`, but doing
        # it in __init__() seems logical.
        self.add_equipment(MyPeriodicEquipment(self.client))
       
    def begin_of_run(self, run_number):
        """
        This function will be called at the beginning of the run.
        You don't have to define it, but you probably should.
        You can access individual equipment classes through the `self.equipment`
        dict if needed.
        """
        self.set_all_equipment_status("Running", "greenLight")
        self.client.msg("Frontend has seen start of run number %d" % run_number)
        return midas.status_codes["SUCCESS"]
       
    def end_of_run(self, run_number):
        self.set_all_equipment_status("Finished", "greenLight")
        self.client.msg("Frontend has seen end of run number %d" % run_number)
        return midas.status_codes["SUCCESS"]
   
    def frontend_exit(self):
        """
        Most people won't need to define this function, but you can use
        it for final cleanup if needed.
        """
        print("Goodbye from user code!")
       
if __name__ == "__main__":
    # The main executable is very simple - just create the frontend object,
    # and call run() on it.
    with MyFrontend() as my_fe:
        my_fe.run()
</nowiki>
 
=== Doing work when the program exits ===
 
Since April 2023, the python frontend framework has included the ability to ensure that work gets performed when the program exits (more specifically, just before we disconnect from midas). This may be useful, for example, to reset the state of some hardware before the program exits.
 
In the frontend framework, this is done by implementing the <code>frontend_exit()</code> function in your frontend class (see the example code above). If you're using a bare <code>MidasClient</code> you can instead call <code>register_disconnect_callback()</code>.
 
To be certain that your function gets called, you should use the "context manager" style to create your client/frontend object.
 
# Context manager style - guaranteed that `frontend_exit()` will be called, even
# if there's an uncaught exception or exit(0) is called etc.
with MyFrontend() as my_fe:
    my_fe.run()
# Not using a context manager - `frontend_exit()` may NOT be called!
my_fe = MyFrontend()
my_fe.run()
 
The reason for this is that the best we can do in the latter approach is to hook up the midas disconnection logic to the destructor of the <code>MidasClient</code> object. But python does not guarantee that an object's <code>__del__()</code> function will be called in a timely manner (or even at all!). It depends on your python version, python implementation, and if you do any weird things in your class (e.g. circular references).
 
In the context manager approach, we can hook up the midas disconnection logic to the <code>__exit__()</code> function, which python DOES guarantee will be called.
 
== Tests ==
 
<code>$MIDASSYS/python/tests</code> contains a variety of test scripts. If you wish, you can run them using
 
cd $MIDASSYS/python/tests
python -munittest
 
However, note that running these tests will interact with whichever midas experiment is currently active. They will edit the ODB and start and stop runs. If you are running these tests, we suggest doing so in a "dummy"/test experiment.

Latest revision as of 21:59, 30 October 2026

The midas documentation has moved. The new documentation about the midas Python libraries is in:

Retrieved from "https://daq00.triumf.ca/MidasWiki/index.php?title=Python&oldid=3645"