Back Midas Rome Roody Rootana
  Midas DAQ System, Page 114 of 146  Not logged in ELOG logo
ID Date Author Topic Subjectdown
  957   11 Feb 2014 Stefan RittForumHuge events (>10MB) every second or so
> I'm looking into using MIDAS for an experiment that creates one large event
> (20MB or more) every second.
> 
> Q1: It looks like I should use EQ_FRAGMENTED. Has this feature been in use
> recently? Is it known to work/not work?
> 
> More specifically, the computer should initiate a 1 second data taking, start to
> such the data out of the electronics (which may take a while), change some
> experimental parameters, and start over. 
> 
> Q2: What's the best way to do this? EQ_PERIODIC? 
> I cannot guarantee that the time required to read the hardware has an upper bound.
> In a standalone-prog I would simply use a big loop and let the machine execute
> it as fast as it can: 1.1s, 1.5s, 1.1s, 1.3s, 2.5s, ..... depending on the HW
> deadtimes.
> Will this work with EQ_PERIODIC?
> 
> (Sorry for these maybe stupid questions, but I have so far only used MIDAS for
> externally generated events, with <32kB event size).
> 
> 
> Thanks a lot,
> 
> Randolf

Hi Randolf,

EQ_FRAGMENTED is kind of historically, when computers had a few MB of memory and you have to play special tricks to get large data buffers through. Today I 
would just use EQ_PERIODIC and increase the midas maximal event size to your needs. For details look here:

https://midas.triumf.ca/MidasWiki/index.php/Event_Buffer

The front-end scheduler is asynchronous, which means that your readout is called when the given period (1 second) is elapsed. If the readout takes longer 
than 1s, the schedule will (hopefully) call your readout immediately after the event has been sent. So you get automatically your maximal data rate. At MEG, we 
use 2 MB events with 10 Hz, so a 20 MB/sec data rate should not be a problem on decent computers.

Best,
Stefan
  962   18 Feb 2014 Konstantin OlchanskiForumHuge events (>10MB) every second or so
> I'm looking into using MIDAS for an experiment that creates one large event
> (20MB or more) every second.

Hi, there - 20 Mbyte event at 1/sec is not so large these days. (Well, depending on your hardware).

Using typical 1-2 year old PC hardware, 20 M/sec to local disk should work right away. Sending data from a 
remote front end (through the mserver), or writing to a remote disk (NFS, etc) - will of course requre a GigE 
network connection.

By default, MIDAS is configured for using about 1-2 Mbyte events, so for your case, you will need to:

- increase the event size limits in your frontend,
- increase /Experiment/MAX_EVENT_SIZE in ODB
- increase the size of the SYSTEM event buffer (/Experiment/Buffer sizes/SYSTEM in ODB)

I generally recommend sizing the SYSTEM event buffer to hold a few seconds worth of data (ot 
accommodate any delays in writing to local disk - competing  reads, internal delays of the disks, etc).

So for 20 M/s, the SYSTEM buffer size should be about 40-60 Mbytes.

For your case, you also want to buffer 3-5-10 events, so the SYSTEM buffer size would be between 100 and 
200 MBytes.

Assuming you have between 8-16-32 GBytes of RAM, this should not be a problem.

One the other hand, if you are running on a low-power ("green") ARM system with 1 Gbyte of RAM and a 
1GHz CPU, you should be able to handle the data rate of 20 Mbytes/sec, as long as your network and 
storage can handle it - I see GigE ethernet running at about 30-40 Mbytes/sec, so you should be okey,
but local storage to SD flash is only about 10 Mbytes/sec - too slow. You can try USB-attached HDD or SSD, 
this should run at up to 30-40 Mbytes/sec. I would expect no problems with this rate from MIDAS, as long 
as you can fit into your 1 GByte of RAM - obviously your SYSTEM buffer will have to be a little smaller than 
on a full-featured PC.

More information on MIDAS event size limits is here (as already reported by Stefan)
https://midas.triumf.ca/MidasWiki/index.php/Event_Buffer

Let us know how it works out for you.

K.O.
  975   01 Mar 2014 Randolf PohlForumHuge events (>10MB) every second or so
Works, and here is how I did it. The attached example is based on the standard MIDAS
example in "src/midas/examples/experiment". 

My somewhat unsorted notes, haven't really tweaked the numbers. But it WORKS.

(1) mlogger writes "last.xml" (hard-coded!) which takes an awful amount of time
    as it writes the complete ODB containing the 10MB bank!
    just outcomment 
       // odb_save("last.xml");
    in mlogger.cxx, function 
    INT tr_start(INT run_number, char *error)
    (line ~3870 in mlogger rev. 5377, .cxx-file included)

(2) frontend.c:
     * the most important declarations are

/* BIG_DATA_BYTES is the data in 1 bank
   BIG_EVENT_SIZE is the event size. It's a bit larger than the bank size
                  because MIDAS needs to add some header bytes, I think
 */

#define BIG_DATA_BYTES  (10*1024*1024)   // 10 MB
#define BIG_EVENT_SIZE  (BIG_DATA_BYTES + 100)

/* maximum event size produced by this frontend */
INT max_event_size = BIG_EVENT_SIZE;

/* maximum event size for fragmented events (EQ_FRAGMENTED) */
INT max_event_size_frag = 5 * BIG_EVENT_SIZE;

/* buffer size to hold 10 events */
INT event_buffer_size = 10 * BIG_EVENT_SIZE;


     * bk_init() can hold at most 32kByte size events! Use bk_init32() instead.

     * complete frontend.c is attached

(3) in an xterm do
    # . setup.sh
    # odbedit -s 41943040
          (first invocation of odbedit must create large enough odb,
           otherwise you'll get "odb full" errors)
(4) odbedit> load big.odb      
    (attached). Essentials are:

    /Experiment/MAX_EVENT_SIZE = 20971520
    /Experiment/Buffer sizes/SYSTEM = 41943040   <- at least 2 events!

    To avoid excessive latecies when starting/stopping a run, do
    /Logger/ODB Dump = no 
    /Logger/Channels/0/Settings/ODB Dump = no 
     
    and create an Equipment Tree to make the mlogger happy

(5) a few more xterms, always ". setup.sh":
    # mlogger_patched (see (1))
    # ./frontend  (attched)

(6) in your odbedit (4) say "start". You should fill your disk rather quickly.
Attachment 1: big_event.tgz
  976   01 Mar 2014 Stefan RittForumHuge events (>10MB) every second or so
> Works, and here is how I did it. The attached example is based on the standard MIDAS
> example in "src/midas/examples/experiment". 

If you have such huge events, it does not make sense to put them into the ODB. The size needs to be increased (as 
you realized correctly) and the run stop takes long if you write last.xml. So just remove the RO_ODB flag in the 
frontend program and you won't have these problems.

/Stefan
  697   08 Apr 2010 Exaos LeeForumHow to stop a run with a timer?
I want to let the run stop and start periodically. But I looked through the ODB
and didn't find anything may help. I also checked the FAQ online and didn't find
answer either. Who can help me? Thank you.
  698   22 Apr 2010 Jimmy NgaiForumHow to stop a run with a timer?
Hi Exaos,

This may help: https://ladd00.triumf.ca/elog/Midas/645

You need to set the following keys:
/Logger/Run duration
/Logger/Auto restart
/Logger/Auto restart delay

Regards,
Jimmy


> I want to let the run stop and start periodically. But I looked through the ODB
> and didn't find anything may help. I also checked the FAQ online and didn't find
> answer either. Who can help me? Thank you.
  2885   05 Nov 2024 Jack CarltonForumHow to properly write a client listens for events on a given buffer?
If there's some template for writing a client to access event data, that would be 
very useful (and you can probably just ignore the context I gave below in that 
case).


Some context:

Quite a while ago, I wrote the attached "data pipeline" client whose job was to 
listen for events, copy their data, and pipe them to a python script. I believe I 
just stole bits and pieces from mdump.cxx to accomplish this. Later I wrote the 
attached wrapper class "MidasConnector.cpp" and a main.cpp to generalize
data_pipeline.cxx a bit. There were a lot of iterations to the code where I had the 
below problems; so don't take the logic in the attached code as the exact code that 
caused the issues below.

However, I'm unable to resolve a couple issues:

1. If a timeout is set, everything will work until that timeout is reached. Then 
regardless of what kind of logic I tried to implement (retry receiving event, 
disconnect and reconnect client, etc.) the client would refuse to receive more data.

2. When I ctrl-C main, it hangs; this is expected because it's stuck in a while 
loop. But because I can't set a timeout I have to ctrl-C twice; this would 
occasionally corrupt the ODB which was not ideal. I was able to get around this with 
some impractical solution involving ncurses I believe.


Thanks,
Jack
Attachment 1: data_pipeline_(2).cxx
#include "midas.h"
#include "msystem.h"
#include "mrpc.h"
#include "mdsupport.h"
#include <iostream>
#include <unistd.h>
#include <stdio.h>  // Added for popen
#include <stdlib.h> // Added for malloc and free

INT hBufEvent;

void process_event(EVENT_HEADER *pheader) {
    printf("Received event #%d\n", pheader->serial_number);
    printf("Event ID: %d\n", pheader->event_id);
    printf("Data Size: %d bytes\n", pheader->data_size);
    printf("Timestamp: %d\n", pheader->time_stamp);
    printf("Trigger mask: %d\n", pheader->trigger_mask);

    // Print a marker to indicate the start of serialized data
    printf("EVENT_DATA_START\n");

    // Serialize and print the event data
    int* eventData = (int*)((char*)pheader + sizeof(EVENT_HEADER));
    int numIntegers = (pheader->data_size - sizeof(EVENT_HEADER)) / sizeof(int);
    for (int i = 0; i < 8; ++i) {
        printf("%d ", eventData[i]);
    }
    printf("\n");

    // Process the event here
}

int main() {
    HNDLE hDB, hKey;
    char host_name[HOST_NAME_LENGTH], expt_name[NAME_LENGTH], str[80];
    char buf_name[32] = EVENT_BUFFER_NAME, rep_file[128];
    unsigned int status, start_time, stop_time;
    INT ch, request_id, size, get_flag, action, single, i;

    // Define the maximum event size you expect to receive
    INT max_event_size = 4000;

    // Allocate memory for storing event data dynamically
    void* event_data = malloc(max_event_size);

    printf("1\n");
    /* Get if existing the pre-defined experiment */
    cm_get_environment(host_name, sizeof(host_name), expt_name, sizeof(expt_name));
    // Print host_name
    printf("host_name = %s\n", host_name);

    // Print expt_name
    printf("expt_name = %s\n", expt_name);

    printf("2\n");
    /* connect to the experiment */
    status = cm_connect_experiment(host_name, expt_name, "data_pipeline", 0);
    if (status != CM_SUCCESS) {
        return 1;
    }
    printf("3\n");

    status = bm_open_buffer(buf_name, DEFAULT_BUFFER_SIZE, &hBufEvent);
    if (status != BM_SUCCESS && status != BM_CREATED) {
        cm_msg(MERROR, "data_pipeline", "Cannot open buffer \"%s\", bm_open_buffer() status %d", buf_name, status);
        return 1;
    }
    printf("4\n");
    /* set the buffer cache size if requested */
    bm_set_cache_size(hBufEvent, 100000, 0);
    printf("5\n");

    /* place a request for a specific event id */
    status = bm_request_event(hBufEvent, EVENTID_ALL, TRIGGER_ALL, GET_ALL, &request_id, NULL); // Use NULL as the callback routine
    printf("6\n");
    printf("status = %d\n",status);
    // Open a pipe to a Python script for data transfer
    FILE* pipe = popen("python3 data_pipeline.py", "w");
    if (pipe == NULL) {
        perror("popen");
        return 1;
    }

    // Enter the event processing loop
    while (1) {
        // Use the address of max_event_size in bm_receive_event
        status = bm_receive_event(hBufEvent, event_data, &max_event_size, BM_WAIT); // Wait for new data indefinitely

        if (status == BM_SUCCESS) {
            //process_event((EVENT_HEADER*)((char*)event_data + sizeof(EVENT_HEADER)));
            // Send the event data to the Python script via the pipe
            fprintf(pipe, "EVENT_DATA_START\n");
            int* eventData = (int*)((char*)event_data + sizeof(EVENT_HEADER));
            int numIntegers = (max_event_size - sizeof(EVENT_HEADER)) / sizeof(int);
            for (int i = 4; i < 12; ++i) {
                fprintf(pipe, "%d ", eventData[i]);
            }
            fprintf(pipe, "\n");
            fflush(pipe); // Flush the buffer to ensure data is sent immediately
        } else {
            printf("Error receiving event: %d\n", status);
            break; // Exit the loop if an error occurs
        }
    }

    // Close the pipe
    pclose(pipe);

    // Free the dynamically allocated memory
    free(event_data);

    cm_disconnect_experiment();

    printf("7\n");
    return 1;
}
Attachment 2: MidasConnector.cpp
#include "MidasConnector.h"

MidasConnector::MidasConnector(const char* clientName) {
    // Initialize client name
    strncpy(client_name_, clientName, NAME_LENGTH);

    // Get host name and experiment name from environment
    cm_get_environment(host_name_, sizeof(host_name_), experiment_name_, sizeof(experiment_name_));

    // Initialize other private variables if needed
    event_id = EVENTID_ALL;  // Initialize with default value
    trigger_mask = TRIGGER_ALL;  // Initialize with default value
    sampling_type = GET_ALL;  // Initialize with default value (renamed from get_flags)
    buffer_size = DEFAULT_BUFFER_SIZE;  // Initialize with default value
    timeout_millis = BM_WAIT;
    strncpy(buffer_name, EVENT_BUFFER_NAME, sizeof(buffer_name));  // Initialize with default value
}

// Getters for the private variables
short MidasConnector::getEventId() const {
    return event_id;
}

short MidasConnector::getTriggerMask() const {
    return trigger_mask;
}

int MidasConnector::getSamplingType() const {
    return sampling_type;  
}

int MidasConnector::getBufferSize() const {
    return buffer_size;
}

const char* MidasConnector::getBufferName() const {
    return buffer_name;
}

int MidasConnector::getTimeout() const {
    return timeout_millis;
}

HNDLE MidasConnector::getEventBufferHandle() const {
    return hBufEvent;
}

// Setters for the private variables
void MidasConnector::setEventId(short eventId) {
    event_id = eventId;
}

void MidasConnector::setTriggerMask(short triggerMask) {
    trigger_mask = triggerMask;
}

void MidasConnector::setSamplingType(int samplingType) {
    sampling_type = samplingType;  
}

void MidasConnector::setBufferSize(int bufferSize) {
    buffer_size = bufferSize;
}

void MidasConnector::setBufferName(const char* bufferName) {
    strncpy(buffer_name, bufferName, sizeof(buffer_name));
}

void MidasConnector::setTimeout(int timeoutMillis) {
    timeout_millis = timeoutMillis;
}

void MidasConnector::setEventBufferHandle(HNDLE eventBufferHandle) {
    hBufEvent = eventBufferHandle;
}


bool MidasConnector::ConnectToExperiment() {
    // Connect to the experiment
    int status = cm_connect_experiment(host_name_, experiment_name_, client_name_, NULL);
    if (status != CM_SUCCESS) {
        // Handle connection error
        return false;
    }
    return true;
}

void MidasConnector::DisconnectFromExperiment() {
    // Disconnect from the experiment
    cm_disconnect_experiment();
}

bool MidasConnector::OpenEventBuffer() {
    int status = bm_open_buffer(buffer_name, buffer_size, &hBufEvent);
    if (status != BM_SUCCESS && status != BM_CREATED) {
        cm_msg(MERROR, client_name_, "Cannot open buffer \"%s\", bm_open_buffer() status %d", buffer_name, status);
        return false;
    }
    return true;
}

bool MidasConnector::SetCacheSize(int cacheSize) {
    bm_set_cache_size(hBufEvent, cacheSize, 0);
    return true;
}

bool MidasConnector::RequestEvent() {
    int request_id;
    int status = bm_request_event(hBufEvent, event_id, trigger_mask, sampling_type, &request_id, NULL);
    return status == BM_SUCCESS;
}

bool MidasConnector::ReceiveEvent(void* eventBuffer, int& maxEventSize) {
    int status = bm_receive_event(hBufEvent, eventBuffer, &maxEventSize, timeout_millis);
    return status == BM_SUCCESS;
}

Attachment 3: main.cpp
#include "event_processor/EventProcessor.h"
#include "data_transmitter/DataTransmitter.h"
#include "midas_connector/MidasConnector.h"
#include "json.hpp"
#include <fstream>

INT hBufEvent1;
INT hBufEvent2;

// Function to initialize MIDAS and open an event buffer
bool initializeMidas(MidasConnector& midasConnector, const nlohmann::json& config) {
    // Set the MidasConnector properties based on the config
    midasConnector.setEventId(config["eventId"].get<short>());
    midasConnector.setTriggerMask(config["triggerMask"].get<short>());
    midasConnector.setSamplingType(config["samplingType"].get<int>());
    midasConnector.setBufferSize(config["bufferSize"].get<int>());
    midasConnector.setBufferName(config["bufferName"].get<std::string>().c_str());
    midasConnector.setBufferSize(config["bufferSize"].get<int>());

    // Call the ConnectToExperiment method
    if (!midasConnector.ConnectToExperiment()) {
        return false;
    }

    // Call the OpenEventBuffer method
    if (!midasConnector.OpenEventBuffer()) {
        return false;
    }

    // Set the buffer cache size if requested
    midasConnector.SetCacheSize(config["cacheSize"].get<int>());

    // Place a request for a specific event id
    if (!midasConnector.RequestEvent()) {
        return false;
    }

    return true;
}

int main() {
    // Read configuration from a JSON file
    nlohmann::json config;
    std::ifstream configFile("config.json");
    configFile >> config;
    configFile.close();

    // Initialize MidasConnector and connect to the MIDAS experiment
    MidasConnector midasConnector(config["clientName"].get<std::string>().c_str());
    if (!initializeMidas(midasConnector, config)) {
        printf("Error: Failed to initialize MIDAS.\n");
        return 1;
    }

    // Read the maximum event size from the JSON configuration
    INT max_event_size = config["maxEventSize"].get<int>();

    // Allocate memory for storing event data dynamically
    void* event_data = malloc(max_event_size);

    // Initialize EventProcessor with detector mapping file and verbosity flag
    EventProcessor eventProcessor(config["detectorMappingFile"].get<std::string>(), config["verbose"].get<bool>());

    // Initialize DataTransmitter with the ZeroMQ address
    DataTransmitter dataPublisher(config["zmqAddress"].get<std::string>());

    // Connect to the ZeroMQ server
    if (!dataPublisher.bind()) {
        // Handle connection error
        printf("Error: Failed to bind to port %s.\n", config["zmqAddress"].get<std::string>().c_str());
        return 1;
    } else {
        printf("Connected to the ZeroMQ server.\n");
    }


    // Event processing loop
    while (true) {

        midasConnector.ReceiveEvent(event_data, max_event_size);

        //Prcoess data once we have it
        eventProcessor.processEvent(event_data, max_event_size);

        // Serialize the event data with EventProcessor and store it in serializedData
        std::string serializedData = eventProcessor.getSerializedData();

        // Send the serialized data to the ZeroMQ server with DataTransmitter
        if (!dataPublisher.publish(serializedData)) {
            // Handle send error
            printf("Error: Failed to send serialized data.\n");
        }
    }

    // Cleanup and finalize your application
    midasConnector.DisconnectFromExperiment(); // Disconnect from the MIDAS experiment

    return 0;
}
  2886   05 Nov 2024 Maia Henriksson-WardForumHow to properly write a client listens for events on a given buffer?
> If there's some template for writing a client to access event data, that would be 
> very useful (and you can probably just ignore the context I gave below in that 
> case).
> 
> 
> Some context:
> 
> Quite a while ago, I wrote the attached "data pipeline" client whose job was to 
> listen for events, copy their data, and pipe them to a python script. I believe I 
> just stole bits and pieces from mdump.cxx to accomplish this. Later I wrote the 
> attached wrapper class "MidasConnector.cpp" and a main.cpp to generalize
> data_pipeline.cxx a bit. There were a lot of iterations to the code where I had the 
> below problems; so don't take the logic in the attached code as the exact code that 
> caused the issues below.
> 
> However, I'm unable to resolve a couple issues:
> 
> 1. If a timeout is set, everything will work until that timeout is reached. Then 
> regardless of what kind of logic I tried to implement (retry receiving event, 
> disconnect and reconnect client, etc.) the client would refuse to receive more data.
> 
> 2. When I ctrl-C main, it hangs; this is expected because it's stuck in a while 
> loop. But because I can't set a timeout I have to ctrl-C twice; this would 
> occasionally corrupt the ODB which was not ideal. I was able to get around this with 
> some impractical solution involving ncurses I believe.
> 
> 
> Thanks,
> Jack

midas/examples/lowlevel/consume.cxx might be what you're looking for, but I think all 
you're missing is a call to cm_yield() in your loop, so your midas client doesn't get 
killed when the timeout is reached (and also so you can act on shutdown requests from 
midas)

Something like 
      int status = cm_yield(100);
      if (status == SS_ABORT || status == RPC_SHUTDOWN)
         break;

There might be a recommended way to handle the ctrl-c and disconnect from the ODB, but 
off the top of my head I don't remember it. 

Also check out Ben's new(ish) python library, midas/python/examples/event_receiver.py 
might be a much easier solution. And you can use the context manager, which will take 
care of safely disconnecting from midas after you ctrl-C.
  609   03 Aug 2009 Exaos LeeForumHow to distinguish the status and value returned from "mvme_read_value(...)"
The definition of mvme_read_value is as the following:
unsigned int EXPRT mvme_read_value(MVME_INTERFACE * vme, mvme_addr_t vme_addr);

Read single data from VME bus. Useful for register access. See example in
mvme_open()

Parameters:
    	*vme 	  VME structure
    	vme_addr  source address (VME location).

Returns:
    MVME_SUCCESS


Question: How to distinguish the status and value returned? Should the definition be something like
int EXPRT mvme_read_value(MVME_INTERFACE *mvme, mvme_addr_t vme_addr, unsigned int *var);
  610   03 Aug 2009 Stefan RittForumHow to distinguish the status and value returned from "mvme_read_value(...)"

Exaos Lee wrote:
The definition of mvme_read_value is as the following:
unsigned int EXPRT mvme_read_value(MVME_INTERFACE * vme, mvme_addr_t vme_addr);

Read single data from VME bus. Useful for register access. See example in
mvme_open()

Parameters:
    	*vme 	  VME structure
    	vme_addr  source address (VME location).

Returns:
    MVME_SUCCESS


Question: How to distinguish the status and value returned? Should the definition be something like
int EXPRT mvme_read_value(MVME_INTERFACE *mvme, mvme_addr_t vme_addr, unsigned int *var);


This function is a shortcut when you want something like
  printf("%d\n", mvme_read_value(...)); 

and you know that the status is ok. Without this function, you would need to define a variable
  unsigned long d;
  mvme_read_value(..., &d);
  printf("%d\n", d);

so the above function is just a handy shortcut. If you want to see the status however, you can call the "normal" function mvme_read:
  status = mvme_read(..., &d, adr, 4); 
  611   03 Aug 2009 Konstantin OlchanskiForumHow to distinguish the status and value returned from "mvme_read_value(...)"
> uint32_t mvme_read_value(MVME_INTERFACE * vme, mvme_addr_t vme_addr);
> Question: How to distinguish the status and value returned?

On VME interfaces using the Universe and tsi148 PCI-VME bridges, your question has no meaning.

The VME address space is directly mapped into the PCI address space, then mmap()ed into your 
program address space. Internally mvme_read_value() is "return *(uint32_t*)(mmap_base + vme_addr);" 
and there is no such thing as "status".

Physically on the VME bus, for single-word VME cycles (mvme_read_value), there are only 2 error 
conditions, an AS or a DS timeout, and these bridges return the bit pattern 0xFFFFFFFF for either error, 
the same as the traditional VME bus always worked (i.e. before PCI, before ISA, back when the VME bus 
*was* the main CPU-memory-IO bus).

So the answer to your question is "yes". If mvme_read_value() returned 0xFFFFFFFF, there was a VME 
bus timeout because the board you are trying to address a) is not installed, b) was unplugged, c) does 
not decode the address you tried to access (maybe you used the wrong AM code or the wrong data 
width).

With Universe and tsi148 PCI-VME bridges, the mvme_read() call runs the DMA interface that can issue 
block transfer cycles on the VME bus. These DMA interfaces have interesting error handling, but 
basically, they only tell you the estimated VME address at which the AS or DS timeout or BERR has 
occurred. For sane VME boards, DMA errors mean very basic breakage of the VME crate and VME board.

With non-directly attached VME interfaces, i.e. the SIS3100, you can also have communication errors. I 
do not know how those are reported by the SIS3100 Linux drivers, and I do not know how the MIDAS 
driver reports them. But I do know that if you see those errors, your interface is very broken and VME 
bus errors are the least of your worries.

P.S. There also exist PCI bus errors, they also return the bit pattern 0xFFFFFFFF and mean basic 
breakage inside your computer. PCI-PCI and PCI-host bridges have special registers you can read to 
find out the exact cause of the error ("your computer is broken").

K.O.
  1764   13 Jan 2020 Konstantin OlchanskiInfoHow to convert C midas frontends to C++, CAEN libraries
Big thanks to Peter Kunz - specifically when using the CAEN libraries:

>
> After upgrading to the lastes MIDAS version I got the DAQ frontend of my application running by
> changing all compiler directives from cc to g++ and using
>
> #include "mfe.h"
>
> extern HNDLE hDB
> 
>  extern "C" { 
>  #include <CAENComm.h>  
>  }
>
> With these changes everything seems to work fine.
>


K.O.



> > To convert a MIDAS frontend to C++ follow this checklist:
> 
> Pierre A.-A. reminded me that include files for CAEN libraries have  to
> use "extern C" brackets:
> 
> some 3rd party libraries (CAEN, etc) are written in C (or require C linkage),
> if their include files are not C++ compatible (do not have "extern C" brackets
> for all exported symbols), the experiment frontend code must say something like this:
> 
> extern "C" {
> #include "3rd-party-c-library.h"
> }
> 
> Note: "#ifdef cplusplus" is not needed because we already know we are C++, not C.
> 
> K.O.
  1633   25 Jul 2019 Frederik WautersForumHow to convert C midas frontends to C++ (my problem solved)
Ok, so the detail that I missed was that the dummy functions

INT poll_event(INT source[], INT count, BOOL test)
{
   return 1;
};

as shown in some of the older examples, work when you set

INT poll_event(INT source, INT count, BOOL test)
{
   return 1;
};


as a side comment, not all drivers are c++ compatible yet (e.g. mscbvr), so changes needed are small 
  1637   30 Jul 2019 Stefan RittForumHow to convert C midas frontends to C++ (my problem solved)
> as a side comment, not all drivers are c++ compatible yet (e.g. mscbvr), so changes needed are small 

Right. We recently switched the whole midas to c++, but we could not cover all drivers. Most of them just need some type 
casting to compile under c++. I got already patches from several people which I'm happy to merge in. If you got mscbhvr 
or any other driver compile under c++, please send me the diff.

Stefan
  1639   01 Aug 2019 Stefan RittForumHow to convert C midas frontends to C++ (my problem solved)
> 
> Ok, so the detail that I missed was that the dummy functions
> 
> INT poll_event(INT source[], INT count, BOOL test)
> {
>    return 1;
> };
> 
> as shown in some of the older examples, work when you set
> 
> INT poll_event(INT source, INT count, BOOL test)
> {
>    return 1;
> };

If you would have read elog:1526 more carefully (point a4) you would have saved yourself a lot of time.

Stefan
  1526   30 Apr 2019 Konstantin OlchanskiInfoHow to convert C midas frontends to C++
To convert a MIDAS frontend to C++ follow this checklist:

a) add #include "mfe.h" after include of midas.h and fix all compilation errors.

NOTE: there should be no "extern C"  brackets around MIDAS include files.

NOTE: Expect to see following problems:

a1) duplicate or mismatched declarations of functions defined in mfe.h
a2) frontend_name and frontend_file_name should be "const char*" instead of "char*"
a3) duplicate "HNDLE hDB" collision with hDB from mfe.c - not sure why it worked before, either use HNDLE hDB from mfe.h or use "extern HNDLE hDB".
a4) poll_event() and interrupt_configure() have "source" as "int[]" instead of "int" (why did this work before?)
a5) use of "extern int frontend_index" instead of get_frontend_index() from mfe.h
a6) bk_create() last argument needs to be cast to (void**)
a7) "bool debug" collides with "debug" from mfe.h (why did this work before?)

b) remove no longer needed "extern C" brackets around mfe related code. Ideally there should be no "extern C" brackets anywhere.

c) in the Makefile, change CC=gcc to CC=g++ for compiling and linking everything as C++

c1) fix all compilation problems. most valid C code will compile as valid C++, but there is some known trouble:
- return value of malloc() & co needs to be cast to the correct data type: "char* s = (char*)malloc(...)"
- some C++ compilers complain about mismatch between signed and unsigned values

If you need help with converting your frontend from C to C++, I will be most happy
to assist you - post your compiler error messages to this forum or email them to me privately.

Good luck,
K.O.
  1542   05 Jun 2019 Konstantin OlchanskiInfoHow to convert C midas frontends to C++
> To convert a MIDAS frontend to C++ follow this checklist:

Pierre A.-A. reminded me that include files for CAEN libraries have  to
use "extern C" brackets:

some 3rd party libraries (CAEN, etc) are written in C (or require C linkage),
if their include files are not C++ compatible (do not have "extern C" brackets
for all exported symbols), the experiment frontend code must say something like this:

extern "C" {
#include "3rd-party-c-library.h"
}

Note: "#ifdef cplusplus" is not needed because we already know we are C++, not C.

K.O.
  1626   23 Jul 2019 Frederik WautersForumHow to convert C midas frontends to C++
I am moving our fe code to c++ midas with cmake. I did encounter your a) problems.

How do I solve mismatched declarations in the mfe (or other places in the midas code)? It is having issues with the midas defined BOOL/... types. This 
is what I get for a minimal scfe:

[ 12%] Building CXX object CMakeFiles/sc_fe_mini.dir/sc_fe_mini.cpp.o
[ 25%] Building CXX object CMakeFiles/sc_fe_mini.dir/home/frederik/packages/midas/drivers/class/hv.cxx.o
[ 37%] Building CXX object CMakeFiles/sc_fe_mini.dir/home/frederik/packages/midas/drivers/class/multi.cxx.o
[ 50%] Building CXX object CMakeFiles/sc_fe_mini.dir/home/frederik/packages/midas/drivers/device/nulldev.cxx.o
[ 62%] Building CXX object CMakeFiles/sc_fe_mini.dir/home/frederik/packages/midas/drivers/bus/null.cxx.o
[ 75%] Building CXX object CMakeFiles/sc_fe_mini.dir/home/frederik/packages/midas/drivers/device/mscbdev.cxx.o
[ 87%] Building CXX object CMakeFiles/sc_fe_mini.dir/home/frederik/packages/midas/mscb/src/mscb.cxx.o
[100%] Linking CXX executable sc_fe_mini
/home/frederik/packages/midas/build/libmfe.a(mfe.cxx.o): In function `_readout_thread':
/home/frederik/packages/midas/src/mfe.cxx:1271: undefined reference to `poll_event(int, int, unsigned int)'
/home/frederik/packages/midas/build/libmfe.a(mfe.cxx.o): In function `check_polled_events':
/home/frederik/packages/midas/src/mfe.cxx:1601: undefined reference to `poll_event(int, int, unsigned int)'
/home/frederik/packages/midas/src/mfe.cxx:1643: undefined reference to `poll_event(int, int, unsigned int)'
/home/frederik/packages/midas/build/libmfe.a(mfe.cxx.o): In function `readout_enable(unsigned int)':
/home/frederik/packages/midas/src/mfe.cxx:1158: undefined reference to `interrupt_configure(int, int, long)'
/home/frederik/packages/midas/src/mfe.cxx:1156: undefined reference to `interrupt_configure(int, int, long)'
/home/frederik/packages/midas/build/libmfe.a(mfe.cxx.o): In function `initialize_equipment':
/home/frederik/packages/midas/src/mfe.cxx:614: undefined reference to `interrupt_configure(int, int, long)'
/home/frederik/packages/midas/src/mfe.cxx:649: undefined reference to `poll_event(int, int, unsigned int)'
/home/frederik/packages/midas/build/libmfe.a(mfe.cxx.o): In function `scheduler':
/home/frederik/packages/midas/src/mfe.cxx:1890: undefined reference to `poll_event(int, int, unsigned int)'
/home/frederik/packages/midas/src/mfe.cxx:1932: undefined reference to `poll_event(int, int, unsigned int)'
/home/frederik/packages/midas/build/libmfe.a(mfe.cxx.o): In function `main':
/home/frederik/packages/midas/src/mfe.cxx:2701: undefined reference to `interrupt_configure(int, int, long)'
/home/frederik/packages/midas/src/mfe.cxx:2702: undefined reference to `interrupt_configure(int, int, long)'
collect2: error: ld returned 1 exit status
make[2]: *** [sc_fe_mini] Error 1
make[1]: *** [CMakeFiles/sc_fe_mini.dir/all] Error 2
make: *** [all] Error 2


This is my cmakelists for my user code:

#
# cmake for the muX software
#
cmake_minimum_required(VERSION 3.3)

project(muX)

#
# find installations
#
set(MIDAS_DIR $ENV{MIDASSYS})
message("MIDAS dir: " ${MIDAS_DIR})

#
# set directories
#
set(MIDASBUILD_DIR ${MIDAS_DIR}/build)
set(MIDASINCLUDE_DIR ${MIDAS_DIR}/include)
set(MXML_DIR ${MIDAS_DIR}/mxml)
set(MSCB_DIR ${MIDAS_DIR}/mscb)
set(DRV_DIR ${MIDAS_DIR}/drivers)


#
# drivers, libs
#
set(DRIVERS
    ${MIDAS_DIR}/drivers/class/hv
    ${MIDAS_DIR}/drivers/class/multi
    ${MIDAS_DIR}/drivers/device/nulldev
    ${MIDAS_DIR}/drivers/bus/null
)
set(MIDASLIB ${MIDASBUILD_DIR}/libmidas.a)
set(FELIB ${MIDASBUILD_DIR}/libmfe.a)

#
# sc_fe
#
add_executable(sc_fe_mini
        sc_fe_mini.cpp
        ${DRIVERS}
        ${MIDAS_DIR}/drivers/device/mscbdev
        ${MIDAS_DIR}/mscb/src/mscb)

target_include_directories(sc_fe_mini PRIVATE ${DRV_DIR} ${MIDAS_DIR}/mscb/include ${MIDAS_DIR}/include)
target_link_libraries(sc_fe_mini ${LIBS} ${MIDASLIB} ${FELIB} rt pthread util)



I seem to be able to compile the current midas distributions, including the scfe frontend



> To convert a MIDAS frontend to C++ follow this checklist:
> 
> a) add #include "mfe.h" after include of midas.h and fix all compilation errors.
> 
> NOTE: there should be no "extern C"  brackets around MIDAS include files.
> 
> NOTE: Expect to see following problems:
> 
> a1) duplicate or mismatched declarations of functions defined in mfe.h
> a2) frontend_name and frontend_file_name should be "const char*" instead of "char*"
> a3) duplicate "HNDLE hDB" collision with hDB from mfe.c - not sure why it worked before, either use HNDLE hDB from mfe.h or use "extern HNDLE hDB".
> a4) poll_event() and interrupt_configure() have "source" as "int[]" instead of "int" (why did this work before?)
> a5) use of "extern int frontend_index" instead of get_frontend_index() from mfe.h
> a6) bk_create() last argument needs to be cast to (void**)
> a7) "bool debug" collides with "debug" from mfe.h (why did this work before?)
> 
> b) remove no longer needed "extern C" brackets around mfe related code. Ideally there should be no "extern C" brackets anywhere.
> 
> c) in the Makefile, change CC=gcc to CC=g++ for compiling and linking everything as C++
> 
> c1) fix all compilation problems. most valid C code will compile as valid C++, but there is some known trouble:
> - return value of malloc() & co needs to be cast to the correct data type: "char* s = (char*)malloc(...)"
> - some C++ compilers complain about mismatch between signed and unsigned values
> 
> If you need help with converting your frontend from C to C++, I will be most happy
> to assist you - post your compiler error messages to this forum or email them to me privately.
> 
> Good luck,
> K.O.
  1627   23 Jul 2019 Stefan RittForumHow to convert C midas frontends to C++
Did you include mfe.h as written in elog:1526 ?

Stefan
  1628   23 Jul 2019 Frederik WautersForumHow to convert C midas frontends to C++
> Did you include mfe.h as written in elog:1526 ?
> 
> Stefan


Yes I did

this is my include 

#include <stdio.h>
#include <string.h>
#include <assert.h>
#include <math.h>
#include <pthread.h>


#include "midas.h"
#include "mscb.h"
#include "multi.h"
#include "mscbdev.h"
#include "mfe.h"

(I attach my dummy fe)

What confuses me is that I can compile examples/experiment/ if I copy that to a 
fresh dir.

I also copied the CMakeLists from the example:

#
# cmake for the muX software
#
cmake_minimum_required(VERSION 3.3)

project(muX)

#
# find midas installation, from CMakeLists in examples/experiment
#
set(MIDAS_DIR $ENV{MIDASSYS})
message("MIDAS dir: " ${MIDAS_DIR})
if (NOT EXISTS $ENV{MIDASSYS})
  message(FATAL_ERROR "Environment variable $MIDASSYS not defined, aborting.")
endif()

set(INC_PATH ${MIDAS_DIR}/include ${MIDAS_DIR}/mxml ${MIDAS_DIR}/mscb/include 
${MIDAS_DIR}/drivers/class ${MIDAS_DIR}/drivers/device)
link_directories($ENV{MIDASSYS}/lib)

# enable certain compile warnings
add_compile_options(-Wall -Wformat=2 -Wno-format-nonliteral -Wno-strict-
aliasing -Wuninitialized -Wno-unused-function)

set(LIBS -lpthread -lutil -lrt) 


add_executable(sc_fe_mini sc_fe_mini.cpp)
target_include_directories(sc_fe_mini PRIVATE ${INC_PATH})
target_link_libraries(sc_fe_mini mfe midas ${LIBS})
Attachment 1: sc_fe_mini.cpp
/********************************************************************\
 
 Name:         sc_fe.c
 Created by:   Stefan Ritt
 		Frederik Wauters
 		Andreas Knecht
 
 Contents:     Slow control frontend for the muX  experiment
 
 $Id: sc_fe.c 21520 2014-11-03 12:03:56Z ritt $
 
 \********************************************************************/

#include <stdio.h>
#include <string.h>
#include <assert.h>
#include <math.h>
#include <pthread.h>


#include "midas.h"
#include "mscb.h"
#include "multi.h"
#include "mscbdev.h"
#include "mfe.h"

/*-- Globals -------------------------------------------------------*/

/* The frontend name (client name) as seen by other MIDAS clients   */
const char *frontend_name = "SC Frontend";
/* The frontend file name, don't change it */
const char *frontend_file_name = __FILE__;

/* frontend_loop is called periodically if this variable is TRUE    */
BOOL frontend_call_loop = TRUE;

/* a frontend status page is displayed with this frequency in ms    */
INT display_period = 0;//1000;

/* maximum event size produced by this frontend */
INT max_event_size = 5*100000;

/* maximum event size for fragmented events (EQ_FRAGMENTED) */
INT max_event_size_frag = 5 * 1024 * 1024;

/* buffer size to hold events */
INT event_buffer_size = 2 *5* 100000;




/*-- Equipment list ------------------------------------------------*/

/* device driver list */


// New HV power supply with "many" channels
DEVICE_DRIVER hv_driver[] = {
    {"64xHV-PSI", mscbdev, 0, NULL, DF_MULTITHREAD},
    {""}
};


/*DEVICE_DRIVER cfd_driver[] = {
    {"CFD-950", mscbdev, 0, NULL, DF_INPUT | DF_MULTITHREAD},
    {""}
};

DEVICE_DRIVER pressure_driver[] = {
    {"PFEIFFER", mscbdev, 0, NULL, DF_INPUT | DF_MULTITHREAD},
    {""}
};

DEVICE_DRIVER multi_driver[] = {
    {"ADC", mscbdev, 0, NULL, DF_INPUT | DF_MULTITHREAD},
    {""}
};

DEVICE_DRIVER autofill_A_driver[] = {
    {"Input", mscbdev, 0, NULL, DF_INPUT | DF_MULTITHREAD},
    {"Output", mscbdev, 0, NULL, DF_OUTPUT | DF_MULTITHREAD},
    {""}
};

DEVICE_DRIVER autofill_B_driver[] = {
    {"Input", mscbdev, 0, NULL, DF_INPUT | DF_MULTITHREAD},
    {"Output", mscbdev, 0, NULL, DF_OUTPUT | DF_MULTITHREAD},
    {""}
};

// HV device with hardware ramp and 1 channel per node
DEVICE_DRIVER poshv_driver[] = {
   {"PMT", mscbhvr, 0, NULL, DF_HW_RAMP | DF_PRIO_DEVICE | DF_MULTITHREAD},
   {""}
};*/

EQUIPMENT equipment[] = {
    
    

    
    {"SiPM_HV",                       /* equipment name */
        {10, 0,                     /* event ID, trigger mask */
            "SYSTEM",                  /* event buffer */
            EQ_SLOW,                   /* equipment type */
            0,                         /* event source */
            "MIDAS",                   /* format */
            TRUE,                      /* enabled */
            RO_ALWAYS,
            60000,                     /* produce event every 60 sec */
            1000,                      /* read one event every second */
            0,                         /* number of sub events */
            1,                        /* log history every 10 seconds event */
            "", "", ""} ,
        cd_multi_read,                 /* readout routine */
        cd_multi,                      /* class driver main routine */
        hv_driver,                  /* device driver list */
        NULL,                       /* init string */
    },
    
    
    //    {"CFD",                     /* equipment name */
    //    {11, 0,                     /* event ID, trigger mask */
    //     "SYSTEM",                  /* event buffer */
    //     EQ_SLOW,                   /* equipment type */
    //     0,                         /* event source */
    //     "MIDAS",                   /* format */
    //     TRUE,                      /* enabled */
    //     RO_ALWAYS,
    //     60000,                     /* read every 60 sec */
    //     1000,                      /* read one event every second */
    //     0,                         /* number of sub events */
    //     1,                         /* log history every second */
    //     "", "", ""} ,
    //    cd_multi_read,              /* readout routine */
    //    cd_multi,                   /* class driver main routine */
    //    cfd_driver,                 /* device driver list */
    //    NULL,                       /* init string */
    //    },
    
    //   {"Pressures",                /* equipment name */
    //    {12, 0,                     /* event ID, trigger mask */
    //     "SYSTEM",                  /* event buffer */
    //     EQ_SLOW,                   /* equipment type */
    //     0,                         /* event source */
    //     "MIDAS",                   /* format */
    //     TRUE,                      /* enabled */
    //     RO_ALWAYS,
    //     60000,                     /* read every 60 sec */
    //     1000,                      /* read one event every second */
    //     0,                         /* number of sub events */
    //     1,                         /* log history every second */
    //    "", "", ""} ,
    //    cd_multi_read,              /* readout routine */
    //    cd_multi,                   /* class driver main routine */
    //    pressure_driver,            /* device driver list */
    //    NULL,                    
    
    {""}
};

/*-- Dummy routines ------------------------------------------------*/

INT poll_event(INT source[], INT count, BOOL test)
{
    return 1;
};
INT interrupt_configure(INT cmd, INT source[], PTYPE adr)
{
    return 1;
};

/*-- Function to define MSCB variables in a convenient way ---------*/

void mscb_define(const char *submaster, const char *equipment, const char *devname, DEVICE_DRIVER *driver, 
                 int address, unsigned char var_index, const char *name, double threshold)
{
   int i, dev_index, chn_index, chn_total;
   char str[256];
   float f_threshold;
   HNDLE hDB;

   cm_get_experiment_database(&hDB, NULL);

   if (submaster && submaster[0]) {
      sprintf(str, "/Equipment/%s/Settings/Devices/%s/Device", equipment, devname);
      db_set_value(hDB, 0, str, submaster, 32, 1, TID_STRING);
      sprintf(str, "/Equipment/%s/Settings/Devices/%s/Pwd", equipment, devname);
      db_set_value(hDB, 0, str, "meg", 32, 1, TID_STRING);
   }

   /* find device in device driver */
   for (dev_index=0 ; driver[dev_index].name[0] ; dev_index++)
      if (equal_ustring(driver[dev_index].name, devname))
         break;

   if (!driver[dev_index].name[0]) {
      cm_msg(MERROR, "mscb_define", "Device \"%s\" not present in device driver list", devname);
      return;
   }

   /* count total number of channels */
   for (i=chn_total=0 ; i<=dev_index ; i++)
      chn_total += driver[i].channels;

   chn_index = driver[dev_index].channels;
   sprintf(str, "/Equipment/%s/Settings/Devices/%s/MSCB Address", equipment, devname);
   db_set_value_index(hDB, 0, str, &address, sizeof(int), chn_index, TID_INT, TRUE);
   sprintf(str, "/Equipment/%s/Settings/Devices/%s/MSCB Index", equipment, devname);
   db_set_value_index(hDB, 0, str, &var_index, sizeof(char), chn_index, TID_BYTE, TRUE);

   if (threshold != -1) {
     sprintf(str, "/Equipment/%s/Settings/Update Threshold", equipment);
     f_threshold = (float) threshold;
     db_set_value_index(hDB, 0, str, &f_threshold, sizeof(float), chn_total, TID_FLOAT, TRUE);
   }

   if (name && name[0]) {
      sprintf(str, "/Equipment/%s/Settings/Names Input", equipment);
      db_set_value_index(hDB, 0, str, name, 32, chn_total, TID_STRING, TRUE);
   }

   /* increment number of channels for this driver */
   driver[dev_index].channels++;
}

void scfe_error(const char *error)
{
    char str[256];
    
    strlcpy(str, error, sizeof(str));
    cm_msg(MERROR, "scfe_error", "%s", str);
    al_trigger_alarm("MSCB", str, "MSCB Alarm", "Communication Problem", AT_INTERNAL);
}

/*-- Frontend Init -------------------------------------------------*/

INT frontend_init()
{
    HNDLE hDB;
    //int i, im, iv;
    //char str[80];
    
    cm_get_experiment_database(&hDB, NULL);
    
    /* set error dispatcher for alarm functionality */
    mfe_set_error(scfe_error);
    
    /* set maximal retry count */
    mscb_set_max_retry(100);
    

    
    /* HV */
    // New SiPM power supply with many channels
    mscb_define("mscb267.psi.ch", "SiPM_HV", "64xHV-PSI", hv_driver, 1, 0, "Temperature 1", 0.1);
        
    return CM_SUCCESS;
}

/*-- Frontend Exit -------------------------------------------------*/

INT frontend_exit()
{
    return CM_SUCCESS;
}

/*-- Begin of Run --------------------------------------------------*/

INT begin_of_run(INT run_number, char *error)
{
    return CM_SUCCESS;
}

/*-- End of Run ----------------------------------------------------*/

INT end_of_run(INT run_number, char *error)
{
    return CM_SUCCESS;
}

/*-- Pause Run -----------------------------------------------------*/

INT pause_run(INT run_number, char *error)
{
    return CM_SUCCESS;
}

/*-- Resuem Run ----------------------------------------------------*/

INT resume_run(INT run_number, char *error)
{
    return CM_SUCCESS;
}

/*-- Frontend Loop -------------------------------------------------*/

INT frontend_loop()
{
... 6 more lines ...
ELOG V3.1.4-2e1708b5