This page is a work in progress!

Links

Introduction

This page will document using object-oriented C++ to create a midas Frontend. For the classic C-style frontend see Frontend user code. For python frontends see Python.

Regardless of the framework you use, all midas frontends follow the same concept of "equipment" (which can be periodic or polled) that produce data in midas banks that can eventually get logged to file or the midas history system. If you don't need to produce data, you can write midas clients without using one of the frontend frameworks.

Key differences to the C-style frontend framework

In the C-style framework you must link against libmfe.a and libmidas.a. In this framework you should only link against libmidas.a.
In the C-style framework you do not write a main() function - you just implement functions (and variables) that the framework uses. In this framework you write your own main() function (generally just 2 lines of code).
In the C-style framework you populate nested structs with important information like the equipment name and type. In this framework you set member variables in classes instead.
In the C-style framework you write one function per frontend that will get called at the start of each run. In this framework you write one such function for each equipment in your frontend.

Examples

Examples of TMFE-based frontends can be found at

$MIDASSYS/progs/tmfe_example.cxx - minimal frontend with a periodic equipment
$MIDASSYS/progs/tmfe_example_frontend.cxx - the equivalent of $MIDASSYS/examples/experiment/frontend.cxx
$MIDASSYS/progs/tmfe_example_indexed.cxx - example of a frontend that handles the -i argument. If you pass -i 3 on the command line, the equipment will appear as example_03 and write data to BUF03
$MIDASSYS/progs/tmfe_example_multithread.cxx - example of a frontend that offloads some tasks to other threads. You can offload RPC communication (ODB updates, run transitions etc), periodic equipment checks, polled equipment checks into 3 separate threads if desired. The default is to run all checks in the main thread.
$MIDASSYS/progs/tmfe_example_everything.cxx - "kitchen sink" example that uses almost all features of the framework.
$MIDASSYS/progs/fetest.cxx - contains several periodic equipment and one that only reacts to RPC commands and never writes any data.

Frontend code

The main base classes you will need to derive from are:

TMFrontend
TMFeEquipment

Other classes you will interact with are:

TMFE - wrapper of some midas functions (cm_msg()/fMfe->Msg(), cm_yield()/fMfe->Yield() etc) that also contains some global state (fRunNumber etc).
TMFeResult - used by the framework to report success/failure of commands (rather than raw integer status codes used in the C-style framework). Note that the framework does NOT use exceptions to report errors.
MVOdb - simplified access to ODB that suits some use cases. You can still use the db_* family of C-style functions or the midas::odb JSON-like interface.

More details about these classes follow below.

Helper classes

Status and error-checking - TMFEResult/TMFeOk

Most functions in the TMFE framework return their status via a TMFEResult object. For example, the interface for handling a begin-of-run transition is:

  TMFeResult HandleBeginRun(int run_number);

When you implement this in your class that derives from TMFeEquipment, you must construct and return a TMFeResult object. Ways to do this are:

# Use the plain TMFeResult constructor. 
# TMFeResult(int code, const std::string& str);
# See $MIDASSYS/include/midas.h for details of pre-defined error codes (like FE_ERR_ODB is 602), but you may use any integer.
# Note that midas defines SUCCESS as 1!
return TMFeResult(FE_ERR_ODB, "Failed to read value from ODB");

# Use a shorthand to say that everything is okay.
return TMFeOk();

# Say there was a failure without specifying a specific midas status code (SUCCESS in midas is 1; this wrapper will set a code of 0)
return TMFeErrorMessage("Failed to read value from ODB");

# Apply some formatting to your error message saying which function had the issue.
# This example will result in a message of "Failed to read value from ODB, some_func() status 602"
return TMFeMidasError("Failed to read value from ODB", "some_func", FE_ERR_ODB);

Note that none of the above examples log the error message in the midas message log, they are simply used to provide more information about the error than just a status code. Call TMFE::Msg() (a wrapper around cm_msg()) if you want to log an error.

TMFE singleton

The TMFE class is implemented as a singleton. That means there is only one instance of it, and you can call its functions from anywhere in your code. It contains a mixture of global state and helper functions.

There are 3 ways to access the TMFE class:

# Access from anywhere
TMFE::Instance()->

# Access from within a class that derives from TMFeEquipment:
fMfe->

# Access from within a class that derives from TMFrontend:
fMfe->

Functions

The main functions that you will use are:

# Add a message to the midas message log (wrapper around cm_msg()).
# The simplest way to pass the first 3 parameters is via the MINFO or MERROR macros.
# If your compiler supports the __FUNCTION__ macro (which most do), you can use that to automatically generate the value for the 4th argument.
# The subsequent arguments behave like any call to printf/sprintf etc.
# E.g. fMfe->Msg(MINFO, __FUNCTION__, "6 x 3 = %d", 18);
void Msg(int message_type, const char *filename, int line, const char *routine, const char *format, ...);

# Start or stop a run from your code (wrapper around cm_transition()).
void StartRun();
void StopRun();

# Get the current UNIX time in seconds (to microsecond precision).
double GetTime();

# Raise/trigger an alarm (wrapper around al_trigger_alarm()). Uses the "internal" alarm type.
# Pre-defined alarm classes are "Warning" and "Alarm".
TMFeResult TriggerAlarm(const char* alarm_name, const char* message, const char* alarm_class);

# Cancel/reset an alarm
TMFeResult ResetAlarm(const char* alarm_name);

# Wait for up to N seconds for midas to tell us about any ODB updates, RPC calls etc (wrapper around cm_yield()).
# This is automatically called periodically by the frontend system, so you shouldn't need to call it yourself.
void Yield(double secs);

# Sleep the current thread for the specified number of seconds (very similar to ss_sleep()).
void Sleep(double secs);

Member variables

There are also a few member variable that may be of interest.

# This is set to true if the user passes -v as a command-line argument when running the frontend.
bool gfVerbose;

# Current run number.
int  fRunNumber;

# Whether run is running/paused (true) or stopped (false).
bool fStateRunning;

# ODB access via MVOdb.
MVOdb* fOdbRoot;

# ODB handle if using db_* C-style family of functions
HNDLE fDB;

MVOdb

The MVOdb class is yet another way to access the ODB. The mvodb package defines an API that can be used both by code that is reading from a live ODB (like this frontend) as well as from JSON/XML ODB dumps (when analysing midas files). It provides a simpler but more limited interface compared to the db_* family of C-style functions or the midas::odb JSON-like interface.

Access

You can access the ODB in a few ways:

# Access to the top-level (from within classes that derive from TMFeEquipment or TMFrontend).
fMfe->fOdbRoot->    # /

# Access to equipment-specific ODB directories (from within classes that derive from TMFeEquipment):
fOdbEq->            # /Equipment/<xxx>
fOdbEqCommon->      # /Equipment/<xxx>/Common
fOdbEqSettings->    # /Equipment/<xxx>/Settings
fOdbEqVariables->   # /Equipment/<xxx>/Variables
fOdbEqStatistics->  # /Equipment/<xxx>/Statistics

Moving to / creating ODB directories

You can create your own object that points to a specific directory using the ChDir() function. Note that this returns a new object rather than editing the state of the object you pass in!

MVOdb* Chdir(const char* subdirname, bool create = false, MVOdbError* error = NULL);

# E.g.
MVOdb* my_dir = fMfe->fOdbRoot->ChDir("/Path/to/my/dir");
# my_dir will access entries in /Path/to/my/dir
# fMfe->fOdbRoot still accesses /

If you wish to create a subdirectory, pass true as the second argument to ChDir():

# Will create /Path/to/my/dir if it doesn't exist.
MVOdb* my_dir = fMfe->fOdbRoot->ChDir("/Path/to/my/dir", true);

# Will return NULL if /Path/to/my/dir doesn't exist (this is the default behaviour).
MVOdb* my_dir = fMfe->fOdbRoot->ChDir("/Path/to/my/dir", false);

Reading/writing ODB entries

To read/write ODB elements use the R and W families of functions:

 # E.g. for reading a single boolean value
 void RB(const char* varname, bool   *value, bool create = false, MVOdbError* error = NULL);
 
 # E.g. for reading a boolean array
 void RBA(const char* varname, std::vector<bool>   *value, bool create = false, int create_size = 0, MVOdbError* error = NULL);
 
 # E.g. for reading a single element in a boolean array
 void RBAI(const char* varname, int index, bool   *value, MVOdbError* error = NULL);
 
 # E.g. for writing a single boolean value
 void WB(const char* varname, bool   v, MVOdbError* error = NULL);
 
 # E.g. for writing a  boolean array
 void WBA(const char* varname, const std::vector<bool>&   v, MVOdbError* error = NULL);
 
 # E.g. for writing a single element in a boolean array
 void WBAI(const char* varname, int index, bool   v, MVOdbError* error = NULL);

There are similar functions for the following data types. Note that the functions do NOT do type conversions for you - you must know the type that your ODB data is stored as! If you call RD() for data that is stored as an integer, you will get an error (more on errors later).

data        | midas       | read  | read  | read   | write | write | write  |
type        | type        | 1 val | array | elem   | 1 val | array | elem   |
------------+-------------+-------+-------+--------+-------+-------+--------+
bool        | TID_BOOL    | RB    | RBA   | RBAI   | WB    | WBA   | WBAI   |
int         | TID_INT     | RI    | RIA   | RIAI   | WI    | WIA   | WIAI   |
double      | TID_DOUBLE  | RD    | RDA   | RDAI   | WD    | WDA   | WDAI   |
float       | TID_FLOAT   | RF    | RFA   | RFAI   | WF    | WFA   | WFAI   |
std::string | TID_STRING  | RS    | RSA   | RSAI   | WS    | WSA   | WSAI   |
uint16_t    | TID_WORD    | RU16  | RU16A | RU16AI | WU16  | WU16A | WU16AI |
uint32_t    | TID_DWORD   | RU32  | RU32A | RU32AI | WU32  | WU32A | WU32AI |

If your data is stored as a type not supported by MVOdb, you will need to use the C-style functions or JSON-like interface mentioned previously. You can use fMfe->fDB as the ODB handle. For example:

 uint8_t data = 0;
 INT size = sizeof(data);
 INT status = db_get_value(fMfe->fDB, 0, "/Path/to/some/uint8", &data, &size, TID_UINT8, false);
 
 if (status == SUCCESS) { 
    # data is valid
 }

Deleting ODB entries

You can delete ODB keys using the Delete function

void Delete(const char* odbname, MVOdbError* error = NULL);

# E.g. 
fMfe->fOdbRoot->Delete("/Path/to/my/dir");

Error-checking

MVOdb uses the MVOdbError object to report errors. As you will see from the function signatures, you need to create this object yourself, pass a pointer to that object to the function you're calling, then check the status after the call...

MVOdbError err;
bool create_dir = false;
MVOdb* my_dir = fMfe->fOdbRoot->ChDir("/Path/to/my/dir", create_dir, &err);

if (my_dir == nullptr) {
   # Directory doesn't exist
} else if (err.fError) {
   # Some other problem. See err.fErrorString and err.fStatus.
} else {
   # Directory exists! Try to read an entry...
   bool create_key = false;
   bool value = false;
   my_dir->RB("my_boolean", &value, create_key, &err);
 
   if (err.fError) {
      # Some problem reading the value. See err.fErrorString and err.fStatus.
   } else {
      `value` now contains the ODB value!
   }
}

Code you need to implement

Equipment - derive from TMFeEquipment

In midas parlance, an "equipment" is a piece of code that generates events. Often each "equipment" talks to a piece of hardware, but you can also have purely software-based equipment. You need to create at least one "equipment" in your frontend. Like in the C-style framework, your can create periodic or polled equipment (periodic ones generate events at a set frequency, while the framework repeatedly asks a polled equipment if it has any events to send). You may even want to have multiple equipment for a single piece of hardware (e.g. a polled equipment that sends the main data stream, plus a periodic equipment that sends some summary statistics every 5 seconds).

Each equipment is defined by a separate class that inherits from TMFeEquipment.

You may implement the following functions for each equipment:

 # Constructor must match this interface
 MyEquip(const char* eqname, const char* eqfilename);
 
 # Initialisation - handle any arguments that appeared after "--" on the command-line.
 TMFeResult HandleInit(const std::vector<std::string>& args);
 
 # Print any usage information to screen (e.g. if you handle any arguments after "--" on the command-line).
 void HandleUsage();
 
 # Run transitions
 TMFeResult HandleBeginRun(int run_number);
 TMFeResult HandleEndRun(int run_number);
 TMFeResult HandlePauseRun(int run_number);
 TMFeResult HandleResumeRun(int run_number);
 TMFeResult HandleStartAbortRun(int run_number);
 
 # JRPC handler
 TMFeResult HandleRpc(const char* cmd, const char* args, std::string& result);
 
 # For periodic frontends - compose event then call EqSendEvent().
 void HandlePeriodic();
 
 # For polled frontends - is there an event to send?
 bool HandlePoll();
 
 # For polled frontends - compose event then call EqSendEvent().
 void HandlePollRead();

There are other functions defined in TMFeEquipment that you may want to call, but do not need to implement:

 # Event creation
 TMFeResult ComposeEvent(char* pevent, size_t size) const;
 TMFeResult BkInit(char* pevent, size_t size) const;
 void*      BkOpen(char* pevent, const char* bank_name, int bank_type) const;
 TMFeResult BkClose(char* pevent, void* ptr) const;
 
 # Event dispatch
 TMFeResult EqSendEvent(const char* event, bool write_to_odb=true);
 
 # Equipment status
 TMFeResult EqSetStatus(char const* eq_status, char const* eq_color);

Constructor and configuration

Your class constructor should call the TMFeEquipment constructor.

  MyEquip(const char* eqname, const char* eqfilename) : TMFeEquipment(eqname, eqfilename) { }

Within your constructor you may optionally set the default values for parameters that define the equipment behaviour. These are stored in the ODB in /Equipment/<equip_name>/Common - see equipment list parameters for a full list of the ODB meanings.

Below are the parameters you may set, their default values, and their ODB locations.

  # Special! Not stored in the ODB!
  # Defines whether the values in the constructor take precedence over the values in the ODB.
  # But fEqConfEnabled and fEqConfEventLimit are always read from the ODB regardless of this setting.....
  bool        fEqConfReadConfigFromOdb = true;
  
  # Equivalent of setting Common/Type to EQ_POLLED or EQ_PERIODIC in the ODB.
  # TMFE ignores the ODB value, even in fEqConfReadConfigFromOdb is true!
  bool        fEqConfEnablePeriodic = true;
  bool        fEqConfEnablePoll     = false;
  
  # Common/Enabled
  bool        fEqConfEnabled        = true;
  
  # Common/EventID
  uint16_t    fEqConfEventID        = 1;
  
  # Common/TriggerMask
  uint16_t    fEqConfTriggerMask    = 0;
  
  # Common/Buffer
  std::string fEqConfBuffer         = "SYSTEM";
  
  # Common/ReadOn
  # Note, as of April 2023 TMFE does not support all values here! It "simplifies"/mangles 
  # the configuration into:
  #   fEqConfReadOnlyWhenRunning = !(fEqConfReadOn & (RO_PAUSED|RO_STOPPED));
  #   fEqConfWriteEventsToOdb    = (fEqConfReadOn & RO_ODB);
  # i.e. RO_RUNNING|RO_PAUSED will implicitly mean events will be written all the time,
  # even when the run is stopped......
  int         fEqConfReadOn         = 0;
  
  # Common/Period
  int         fEqConfPeriodMilliSec = 1000;
  
  # Common/Event limit
  double      fEqConfEventLimit     = 0;
  
  # Common/Log History
  int         fEqConfLogHistory     = 0;
  
  # Common/Hidden
  bool        fEqConfHidden         = false;
  
  # Common/Write cache size
  int         fEqConfWriteCacheSize = 10000000;

You can of course define and initialise your own member variables in the equipment constructor.

Initialisation and handling arguments

The TMFE framework allows the user to specify arbitrary arguments on the command-line. Any arguments after -- are passed to all the equipments in the frontend. The arguments are passed as simple vector of strings to your equipment's HandleInit() function.

 TMFeResult HandleInit(const std::vector<std::string>& args);

For example, if the user runs my_fe.exe -v -- --some-arg 6 --other-arg, then the framework will handle the -v argument (setting TMFE::gfVerbose to true, and your HandleInit() function will be given {"--some-arg", "6", "--other-arg"}. If your HandleInit() function returns an error, then the program will quit.

Another function related to this is the HandleUsage() function, which should print to screen any usage information (e.g. about --some-arg and --other-arg in the above example.

 void HandleUsage();

The HandleUsage() function will be called if the user passes an invalid argument on the command-line or if the argument --help is present.

The framework recognises the following arguments (before any --:

  --help                  print help message
  -v                      print debug output to screen
  -h hostname[:tcpport]   connect to MIDAS mserver on given host and tcp port number
  -e exptname             connect to given MIDAS experiment
  -i NNN                  set frontend index number (if running multiple instances of the same frontend)
  -D                      become a daemon
  -O                      become a daemon but keep stdout for saving in a log file: frontend -O >file.log 2>&1

Run transitions

These functions will be called for your frontend when the run starts/stops etc. Return TMFeOK() if you're happy for the transition to proceed, or an error if not.

 TMFeResult HandleBeginRun(int run_number);
 TMFeResult HandleEndRun(int run_number);
 TMFeResult HandlePauseRun(int run_number);
 TMFeResult HandleResumeRun(int run_number);
 TMFeResult HandleStartAbortRun(int run_number);

To change the order in which your equipment is handled in the transition sequence, you can call:

 fMfe->SetTransitionSequenceStart(int seqno);
 fMfe->SetTransitionSequenceStop(int seqno);
 fMfe->SetTransitionSequencePause(int seqno);
 fMfe->SetTransitionSequenceResume(int seqno);
 fMfe->SetTransitionSequenceStartAbort(int seqno);

To register multiple transition callbacks (e.g. to do a bit of work early in the transition sequence, and some more later in the sequence) you will need to call cm_register_transition() yourself (no TMFE wrapper). Do NOT deregister the functions that TMFE has already registered!

Remote procedure calls

Midas allows programs to talk to each other via RPC. The most common user-facing approach is the "RPC_JRPC" call, which can be integrated with the midas webserver and custom webpages, and also has a wrapper in the Python library.

When another program calls JRPC, they say which client they want to talk to, specify a "command" (a short text string), and give some "arguments" (a text or JSON string). These will then be passed to your equipment's HandleRpc() function.

 # JRPC handler
 TMFeResult HandleRpc(const char* cmd, const char* args, std::string& result);

The string you put in result will be returned to the calling program. As of April 2023, the TMFeResult you return is ignored.

If you have multiple equipment in your frontend, HandleRpc() will be called on each in turn until one of them returns a non-empty result string. Take care over your command naming to avoid colissions!

An example function might be:

 TMFeResult HandleRpc(const char* cmd, const char* args, std::string& result) {
    if (strcmp(cmd, "my_command") == 0) {
      # Do some work.
      # Could return some data as JSON, or a status string, or...
      result = "OK";
    }
 
    return TMFeOk();
 }

Creating events

You need to call EqSendEvent() to write a data event. For periodic equipment you should do this in HandlePeriodic(); and for polled equipment you should do this in HandlePollRead(). For polled equipment you should also implement HandlePoll() to state whether you have any data to send.

# Implement this for a periodic equipment
void HandlePeriodic();

# Implement these for a polled equipment 
bool HandlePoll();
void HandlePollRead();

To actually create/send events in HandlePeriodic()/HandlePollRead(), use the following functions:

 TMFeResult ComposeEvent(char* pevent, size_t size) const;
 TMFeResult BkInit(char* pevent, size_t size) const;
 void*      BkOpen(char* pevent, const char* bank_name, int bank_type) const;
 TMFeResult BkClose(char* pevent, void* ptr) const;
 
 TMFeResult EqSendEvent(const char* event, bool write_to_odb=true);

For example, this code will create an event with 2 banks (TDBL and TINT), each containing a few values:

  void HandlePeriodic() {
     # buf must be large enough for all your data plus some header words!
     char buf[1024];
  
     # Initialise the event
     ComposeEvent(buf, sizeof(buf));
     BkInit(buf, sizeof(buf));      
  
     # Create and populate the first bank (with values 1.23/3.14
     double* ptr_d = (double*)BkOpen(buf, "TDBL", TID_DOUBLE);
     *ptr_d++ = 1.23;
     *ptr_d++ = 3.14;
     BkClose(buf, ptr_d);      
  
     # Create and populate another bank
     int32_t* ptr_i = (int32_t*)BkOpen(buf, "TINT", TID_INT32);
     *ptr_i++ = 5;
     *ptr_i++ = 8;
     *ptr_i++ = 12;
     BkClose(buf, ptr_i);
  
     # Send the event to the midas buffer specified by fEqConfBuffer
     EqSendEvent(buf);
  
     # The midas buffer now contains an event with 2 banks. TDBL (with data [1.23,3.14]) and TINT (with data [5,8,12]).
  }

Equipment status

You may call EqSetStatus() at any time to update the status of your equipment in the ODB. The status of your equipment is shown on the main midas status page.

 TMFeResult EqSetStatus(char const* eq_status, char const* eq_color);

The status string should be fairly short, to avoid creating a messy layout on the webpage. You can specify any CSS color as the color, but midas also has some pre-defined colors if you want to match the webpage theme: --var(mgreen), --var(myellow), --var(mred), --var(mblue), --var(morange), --var(mgray).

# Set a happy message
EqSetStatus("All systems go!", "--var(mgreen)");

# Set a warning message
EqSetStatus("Trouble brewing...", "--var(myellow)");

Frontend - derive from TMFrontend

The main purpose of your frontend class is to instantiate your equipment classes.

  virtual TMFeResult HandleArguments(const std::vector<std::string>& args)     { return TMFeOk(); };
  virtual void       HandleUsage()                                             { };
  virtual TMFeResult HandleFrontendInit(const std::vector<std::string>& args)  { return TMFeOk(); };
  virtual TMFeResult HandleFrontendReady(const std::vector<std::string>& args) { return TMFeOk(); };
  virtual void       HandleFrontendExit()                                      { };

  void       FeSetName(const char* program_name);
  TMFeResult FeAddEquipment(TMFeEquipment* eq);

  TMFE* fMfe = NULL;
  int  fFeIndex = 0; //< frontend index

Frontend index - running multiple copies of your code

If you have 3 digitizers in your experiment, you may wish to run 3 copies of your frontend program, each with slightly different settings. You can achieve this by specifying the -i flag on the command-line, which will appear in your frontend as fFeIndex - the frontend index. TMFE, the C-style framework and the python framework all support this behaviour.

You can get the framework to create a unique name for your frontend (based on the frontend index) by specifying %02d as part of your frontend/equipment names.

  MyFrontend() {
     FeSetName("my_frontend_%02d");
     FeAddEquipment(new MyEquipment("my_equip_%02d", __FILE__));
  }

If you run with -i 4, for example, then the frontend will appear in midas as my_frontend_04 and the equipment as my_equip_04 (with ODB settings in /Equipment/my_equip_04).

main()

Your main() should be very simple - just create an instance of your frontend class and call FeMain(). That function will handle parsing all the command-line arguments for you.

 int main(int argc, char* argv[]) {
    MyFrontend fe;
    return fe.FeMain(argc, argv);
 }

Compilation

TODO

Frontend user code (object oriented - TMFE)

Contents

Links

Introduction

Key differences to the C-style frontend framework

Examples

Frontend code

Helper classes

Status and error-checking - TMFEResult/TMFeOk

TMFE singleton

Functions

Member variables

MVOdb

Access

Moving to / creating ODB directories

Reading/writing ODB entries

Deleting ODB entries

Error-checking

Code you need to implement

Equipment - derive from TMFeEquipment

Constructor and configuration

Initialisation and handling arguments

Run transitions

Remote procedure calls

Creating events

Equipment status

Frontend - derive from TMFrontend

Frontend index - running multiple copies of your code

main()

Compilation

Navigation menu

Frontend user code (object oriented - TMFE)

Links

Introduction

Key differences to the C-style frontend framework

Examples

Frontend code

Helper classes

Status and error-checking - TMFEResult/TMFeOk

TMFE singleton

Functions

Member variables

MVOdb

Access

Moving to / creating ODB directories

Reading/writing ODB entries

Deleting ODB entries

Error-checking

Code you need to implement

Equipment - derive from TMFeEquipment

Constructor and configuration

Initialisation and handling arguments

Run transitions

Remote procedure calls

Creating events

Equipment status

Frontend - derive from TMFrontend

Frontend index - running multiple copies of your code

main()

Compilation

Navigation menu

Search