Back Midas Rome Roody Rootana
  Midas DAQ System  Not logged in ELOG logo
Entry  24 Jun 2011, Exaos Lee, Suggestion, Build MIDAS debian packages using autoconf/automake. daq-midas_deb.tar.gzmdaq.py
    Reply  27 Jun 2011, Konstantin Olchanski, Suggestion, Build MIDAS debian packages using autoconf/automake. 
Message ID: 768     Entry time: 24 Jun 2011     Reply to this: 769
Author: Exaos Lee 
Topic: Suggestion 
Subject: Build MIDAS debian packages using autoconf/automake. 
Here is my story. I deployed several Debian Linux boxes as the DAQ systems in our lab. But I feel it's boring to build and install midas and its related softwares (such as root) on each box. So I need a local debian software repository and put midas and its related packages in it. First of all, I need a midas debian package. After a week's study and searching, I finally finished the job. Hope you feel it useful.

All the work is attached as "daq-midas_deb.tar.gz". The detail is followed. I also created several debian packages. But it's too large to be uploaded. I havn't my own site accessible from internet. So, if you need the debian packages, please give me an accessible ftp or other similar service, then I can upload them to you.

First, I use autoconf/automake to rewrite the building system of MIDAS. You can check it this way:
1. Untar daq-midas_deb.tar.gz somewhere, assumming ~/Temp.
2. cd ~/Temp/daq-midas
3. svn co -r 5065 svn+ssh://svn@savannah.psi.ch/repos/meg/midas/trunk midas
4. svn co -r 68 svn+ssh://svn@savannah.psi.ch/repos/meg/mxml/trunk mxml
5. cp -rvp debian/autoconf/* ./
6. ./configure --help
7. ./configure <--options>
8. make && make install

Then, I created the debian packages based on the new building files. You need to install root-system package from http://lcg-heppkg.web.cern.ch/lcg-heppkg/debian/. You can build debs this way:
1. untar daq-midas_deb.tar.gz somewhere, assuming ~/Temp.
2. cd ~/Temp/daq-midas
3. svn co -r 5065 svn+ssh://svn@savannah.psi.ch/repos/meg/midas/trunk midas
4. svn co -r 68 svn+ssh://svn@savannah.psi.ch/repos/meg/mxml/trunk mxml
5. dpkg-buildpackage -b -us -uc

I split the package into serverals parts:
  • daq-midas-doc -- The documents and references
  • daq-midas-root -- the midas runtime library and utilities built with root
  • daq-midas-noroot -- the midas runtime library and utilities built without root
  • daq-midas-dev-root -- the midas devel files (headers, objects, drivers, examples) built with root
  • daq-midas-dev-noroot -- the midas devel files (headers, objects, drivers, examples) built without root

Here are the installation:
  • executalbes -- /usr/lib/daq-midas/bin
  • library and objs -- /usr/lib/daq-midas/lib
  • headers -- /usr/lib/daq-midas/include
  • sources and drivers -- /usr/share/daq-midas/
  • docs and examples -- /usr/share/doc/daq-midas
  • mdaq-config -- /usr/bin/mdaq-config

I add an auto-generated shell script -- mdaq-config. It behaves just like "root-config". You can get midas build flags and link flags this way:
gcc `mdaq-config --cflags` -c -o myfe.o myfe.c
gcc `mdaq-config --libs` -o myfe myfe.o `mdaq-config --libdir`/mfe.o

Bugs and suggestions are welcomed.

P.S. Based on debian packages, I am planing to write another script, "mdaq.py":
  • each midas experiment will be configured in a file named "mdaq.yaml"
  • mdaq.py reads the configure file and prepare the daq environment, just like "examples/experiment/start_daq.sh"
  • mdaq.py will handle "start/stop/restart/info" about the daq codes.
The attached "mdaq.py" is the old one.
Attachment 1: daq-midas_deb.tar.gz  233 kB
Attachment 2: mdaq.py  8 kB  | Hide | Hide all
#!/usr/bin/env python
'''
   Python script to handle daq sessions using MIDAS/PSI.
   Auther: Exaos Lee <Exaos DOT Lee AT gmail DOT com>
   Date:   2009-07-15
'''

import os, os.path
import sys, signal, time
from commands     import getoutput
from ConfigParser import ConfigParser
from pprint       import pprint

class mdaq:
    _codes_ = { 'MSERVER':"MIDAS Server", 'MHTTPD':"MIDAS HTTPD Server",
                'ODBEDIT':"ODB Editor",   'LOGGER':"Data Logger",
                'PROG_FE':"Frontend",   'PROG_ANA':"Analyzer" }
    _env_ = dict()

    def __init__(self,**kws):
        env = dict()
        if kws.has_key('fname') and kws['fname'] != "":
            env = self.parse_conf(kws['fname'])
        self.apply_env(env)

    def parse_conf(self, fname):
        if not os.path.isfile(fname):
            print("**WARNING** File \"%s\" does not exist!"%fname)
            return None

        conf = ConfigParser()
        conf.read(fname)
        env={'CONF_FILE':fname }
        fcpath = os.path.dirname(fname)

        sec = 'EXP_INFO' ## Parsing Experiment Information
        env['EXP_INFO'] = [ self.safe_parse(conf,sec,'Name'),
                            self.safe_parse(conf,sec,'Info'), ]
        msg = self.safe_parse(conf,sec,'DAQ_DIR')
        if msg != "" and os.path.isdir(msg):
            dpath = msg
            if not os.path.isabs(msg):
                dpath = os.path.join(os.path.abspath(fcpath),msg)
            env['DAQ_DIR'] = os.path.normpath(dpath)

        ## Parsing MIDAS configuration
        env['MIDASSYS'] = ["MIDAS System Path",
                           self.safe_parse(conf,"MIDAS","MIDASSYS")]
        if env['MIDASSYS'][1]=="": env['MIDASSYS'][1] = self.get_default_msys()

        ## Parsing Info of programs
        for key in self._codes_.keys():
            code = self.parse_prog_opt(conf, key, fpath=fcpath)
            if code:
                env[key] = [self._codes_[key], ]
                env[key].extend(code)

        port = self.safe_parse(conf, "MHTTPD","PORT")
        if port != "":
            if not env.has_key('MHTTPD'):
                env['MHTTPD'] = ["MIDAS HTTP Server",
                                 os.path.join(env['MIDASSYS'][1],"bin/mhttpd"),
                                 "-h localhost -D"]
            if "-p %s"%port not in env['MHTTPD'][2]:
                env['MHTTPD'][2] = env['MHTTPD'][2] + " -p %s"%port

        return env

    def parse_prog_opt(self, cnf, sec, fpath="./"):
        name = self.safe_parse(cnf, sec, "NAME")
        path = self.safe_parse(cnf, sec, "PATH")
        fn = os.path.join(path,name)
        if name == "": fn = ""
        elif not os.path.isabs(fn):
            fn = os.path.normpath(os.path.join(os.path.abspath(fpath),fn))
        else:
            fn = os.path.normpath(fn)
        opts = self.safe_parse(cnf, sec, "Option")
        if opts != "" or name != "":
            return [fn, opts]

        return None
 
    def safe_parse(self, cnf, sec, opt):
        try:    return cnf.get(sec,opt)
        except: return ""

    def get_default_msys(self):
        if os.environ.has_key('MIDASSYS'): return os.environ['MIDASSYS']
        return "/opt/MIDAS.PSI/Version/Current"

    def apply_env(self, daq_env=dict()):
        env = daq_env
        if not env:
            print("** WARNING** No environment defined! Using the default values. ****")
            env = {'MIDASSYS':['MIDAS System Path', ""], 'CONF_FILE':"", }
        if not env.has_key('MIDASSYS'): env['MIDASSYS'] = ['MIDAS System Path',""]
        if env['MIDASSYS'][1]=="": env['MIDASSYS'][1] = self.get_default_msys()
        def_env = {
            'MSERVER' :["MIDAS Server",
                        os.path.join(env['MIDASSYS'][1],"bin/mserver"),
                        "-D", True],
            'MHTTPD'  :["MIDAS HTTP Server",
                        os.path.join(env['MIDASSYS'][1],"bin/mhttpd"),
                        "-h localhost -D -p 8080", True],
            'ODBEDIT' :["ODB Editor",
                        os.path.join(env['MIDASSYS'][1],"bin/odbedit"),
                        "-c clean", False],
            'LOGGER'  :["Data Logger",
                        os.path.join(env['MIDASSYS'][1],"bin/mlogger"),
                        "", True],
            'PROG_FE' :["Frontend","", "", True],
            'PROG_ANA':["Analyzer","", "", True],
            'EXP_INFO':["test","The test experiment"],
            'DAQ_DIR':os.path.join(os.environ['HOME'],"online/test") }

        for key in def_env.keys():
            if not env.has_key(key): env[key] = def_env[key]

        for key in self._codes_.keys():
            if env[key][1] == "": env[key][1] = def_env[key][1]
            if env[key][2] == "": env[key][2] = def_env[key][2]

        if not os.path.isdir(env['DAQ_DIR']):
            try: os.makedirs(env['DAQ_DIR'])
            except: print("ERROR to make file")

        ## Asign default program options
        exp_opt = "-e %s"%env['EXP_INFO'][0]
        for key in ['LOGGER', 'PROG_FE', 'PROG_ANA', "MHTTPD"]:
            if exp_opt not in env[key][2]:
                env[key][2] = exp_opt + ' ' + env[key][2]
            if "-D" not in env[key][2]:
                env[key][2] = env[key][2] + ' -D'
        if exp_opt not in env['ODBEDIT'][2]:
            env['ODBEDIT'][2] = env['ODBEDIT'][2] + ' ' + exp_opt

        self._env_ = env

    def print_conf(self):
        print "\n================= DAQ Config ====================="
        for key in self._env_.keys():
            print key,": ",
            pprint(self._env_[key])

    def info(self): self.print_conf()

    ## running programs
    def getpids(self, pname):
        return [int(k) for k in getoutput("pidof %s"%pname).strip().split()]

    def kill_prog(self,pname):
        for i in self.getpids(pname): os.kill(i,signal.SIGTERM)

    def run_prog(self, prog):
        if len(prog)>=4 and not prog[3]: return

        if len(prog)>=3 and prog[1]:
            print("Starting %s: %s"%(prog[0],prog[1]))
            os.system("%s %s"%(prog[1],prog[2]))
        else:
            print("%s: no executable exist!"%prog[0])

    def get_status(self,pfile):
        msg = "Not running!"
        pids = self.getpids( os.path.basename(pfile) )
        if pids: return 'PIDs = ' + str(pids)
        return msg

    def echo_status(self,pname,pfile):
        print("%s:\n   %s\n   "%(pname,pfile)
              + self.get_status(pfile))

    ## Session manage: start, stop, status, restart

    def status(self):
        print( "\n===== Session status for experiment \"%s\" =====\n" %
               self._env_['EXP_INFO'][0] )
        for code in ['MSERVER','MHTTPD','LOGGER','PROG_FE','PROG_ANA']:
            self.echo_status(self._env_[code][0], self._env_[code][1])

    def start(self):
        ## Running mserver & mhttpd
        ms_pids = self.getpids(os.path.basename(self._env_['MSERVER'][1]))
        if not ms_pids:
            self.run_prog(self._env_['MSERVER'])
            print("Killing %s ...\n"%self._env_['MHTTPD'][0])
            self.kill_prog(self._env_['MHTTPD'][1])
        if not self.getpids(os.path.basename(self._env_['MHTTPD'][1])):
            self.run_prog(self._env_['MHTTPD'])

        ## Running FE and Analyzer
        old_path = os.getcwd()
        os.chdir( self._env_['DAQ_DIR'] )

        # Init ODB
        print("Initializing ODB for experiment %s ..." % self._env_['EXP_INFO'][0])
        self.run_prog(self._env_['ODBEDIT'])
        time.sleep(1)

        # First stop, then start ...
        self.stop()
        time.sleep(1)

        # Start FE & ANA
        self.run_prog(self._env_['PROG_FE'])
        self.run_prog(self._env_['PROG_ANA'])
        time.sleep(1)

        # Start Logger
        self.run_prog(self._env_['LOGGER'])

        os.chdir(old_path)

    def stop(self):
        for key in ["LOGGER", "PROG_FE", "PROG_ANA"]:
            if self._env_[key][1]:
                print("Killing threads:\n  %s"%self._env_[key][1])
                self.kill_prog(self._env_[key][1])

    def stop_all(self):
        self.stop()
        for key in ['MHTTPD', 'MSERVER']:
            if self._env_[key][1]:
                print("Killing threads:\n  %s"%self._env_[key][1])
                self.kill_prog(self._env_[key][1])

    def restart(self): self.start()


if __name__=='__main__':
    from optparse import OptionParser

    usage = "usage: %prog [options] <command>"
    parser = OptionParser(usage=usage, version="%prog 0.1")
    parser.add_option("-c","--config", metavar="filename",
                      help="Read config from FILENAME")
    cmds = ["start","stop","stop_all","restart","status","info"]

    (opts,args) = parser.parse_args( sys.argv[1:] )
    if not args or len(args)>2:
         parser.print_help()
         print "\nCommands:\n\t",
         pprint(cmds)
         exit()

    fcnf = "daq.conf"
    if opts.config: fcnf = opts.config

    if args[0] in cmds:
        daq = mdaq(fname=fcnf)
        exec( "daq.%s()"%args[0] )
    else:
        parser.print_help()
        print "\nCommands:\n\t",
        pprint(cmds)
ELOG V3.1.4-2e1708b5