| ID |
Date |
Author |
Topic |
Subject |
|
788
|
25 Apr 2012 |
Konstantin Olchanski | Bug Report | Build error with mlogger: invalid conversion from ‘void*’ to ‘gzFile’ | Stefan's fix is incomplete - the "gzFile" cast is needed for all calls to zlib, not just those that some version
of GCC happens to complain about. Fixed.
svn rev 5286.
BTW, I read the midas elog via email and if you post html or elcode messages, I receive complete
gibberish. For prompt service, please select message type "plain". (yes, you cannot use fancy colours and
blinking text, but better than me not reading your stuff at all).
BTW2, for easier reading, please include error messages as plain text in your message. As opposed to
compressed attachements.
K.O. |
|
789
|
27 Apr 2012 |
Stefan Ritt | Bug Report | Build error with mlogger: invalid conversion from ‘void*’ to ‘gzFile’ |
| KO wrote: | BTW, I read the midas elog via email and if you post html or elcode messages, I receive complete
gibberish. For prompt service, please select message type "plain". (yes, you cannot use fancy colours and
blinking text, but better than me not reading your stuff at all).
BTW2, for easier reading, please include error messages as plain text in your message. As opposed to
compressed attachements.
K.O.
|
BTW3, if you use a real email program you don't get glibberish. I know some people prefer good-old-text-only pine, but I'm sure you do not use the ascii-only browser lynx to browse the internet, right? So if you browse the web in graphics, why not read your email in graphics as well. Better change yourself than the whole rest of the world  |
|
311
|
16 Oct 2006 |
Exaos Lee | Bug Fix | Build error with mana.c while using CERNLIB, svn 3366 | If you use CERNLIB to build hmana.o, you may encounter the following error:
src/mana.c: In function ‘write_event_hbook’:
src/mana.c:2881: error: invalid assignment
or somthing like this:
src/mana.c: In function ‘write_event_hbook’:
src/mana.c:2881: warning: target of assignment not really an lvalue; this will be a hard error in the future
So I checked the mana.c and found these lines
2880 /* shift data pointer to next item */
2881 (char *) pdata += key.item_size * key.num_values;
should be changed to
2880 /* shift data pointer to next item */
2881 pdata += key.item_size * key.num_values * sizeof(char) ;
|
|
314
|
16 Oct 2006 |
Stefan Ritt | Bug Fix | Build error with mana.c while using CERNLIB, svn 3366 | | Committed, thanks. |
|
667
|
02 Nov 2009 |
Exaos Lee | Bug Fix | Build error due to missing header | I encountered a build error as "sort undefined...". It is caused by missing C++ header <algorithm> in which "sort" is defined. It can be fixed as the attachment.
Environment:
G++: 4.3.4
Platform: Debian Linux testing
> I committed an updated lazylogger with updated documentation. The new version supports subruns and
> can save to external storage arbitrary files (i.e. odb dump files). It also moves most book keeping out of
> odb to permit handling more files on bigger storage disks.
>
> Example lazylogger scripts for castor (CERN) and dcache (TRIUMF) are in the directory "utils".
>
> The lazylogger documentation was updated to remove obsolete information and to describe the new
> functions. As usual "make dox; cd doxfiles/html; firefox index.html" or see my copy at:
>
> http://ladd00.triumf.ca/~olchansk/midas/Utilities.html#lazylogger_task
>
> svn rev 4615, 4616.
> K.O. |
| Attachment 1: lazylogger.diff
|
diff --git a/src/lazylogger.c b/src/lazylogger.c
index 34be17a..a7ba4a3 100644
--- a/src/lazylogger.c
+++ b/src/lazylogger.c
@@ -17,6 +17,7 @@ $Id$
#include <vector>
#include <string>
+#include <algorithm>
#define NOTHING_TODO 0
#define FORCE_EXIT 1
|
|
768
|
24 Jun 2011 |
Exaos Lee | Suggestion | Build MIDAS debian packages using autoconf/automake. | Here is my story. I deployed several Debian Linux boxes as the DAQ systems in our lab. But I feel it's boring to build and install midas and its related softwares (such as root) on each box. So I need a local debian software repository and put midas and its related packages in it. First of all, I need a midas debian package. After a week's study and searching, I finally finished the job. Hope you feel it useful.
All the work is attached as "daq-midas_deb.tar.gz". The detail is followed. I also created several debian packages. But it's too large to be uploaded. I havn't my own site accessible from internet. So, if you need the debian packages, please give me an accessible ftp or other similar service, then I can upload them to you.
First, I use autoconf/automake to rewrite the building system of MIDAS. You can check it this way:
1. Untar daq-midas_deb.tar.gz somewhere, assumming ~/Temp.
2. cd ~/Temp/daq-midas
3. svn co -r 5065 svn+ssh://svn@savannah.psi.ch/repos/meg/midas/trunk midas
4. svn co -r 68 svn+ssh://svn@savannah.psi.ch/repos/meg/mxml/trunk mxml
5. cp -rvp debian/autoconf/* ./
6. ./configure --help
7. ./configure <--options>
8. make && make install
Then, I created the debian packages based on the new building files. You need to install root-system package from http://lcg-heppkg.web.cern.ch/lcg-heppkg/debian/. You can build debs this way:
1. untar daq-midas_deb.tar.gz somewhere, assuming ~/Temp.
2. cd ~/Temp/daq-midas
3. svn co -r 5065 svn+ssh://svn@savannah.psi.ch/repos/meg/midas/trunk midas
4. svn co -r 68 svn+ssh://svn@savannah.psi.ch/repos/meg/mxml/trunk mxml
5. dpkg-buildpackage -b -us -uc
I split the package into serverals parts:
- daq-midas-doc -- The documents and references
- daq-midas-root -- the midas runtime library and utilities built with root
- daq-midas-noroot -- the midas runtime library and utilities built without root
- daq-midas-dev-root -- the midas devel files (headers, objects, drivers, examples) built with root
- daq-midas-dev-noroot -- the midas devel files (headers, objects, drivers, examples) built without root
Here are the installation:
- executalbes -- /usr/lib/daq-midas/bin
- library and objs -- /usr/lib/daq-midas/lib
- headers -- /usr/lib/daq-midas/include
- sources and drivers -- /usr/share/daq-midas/
- docs and examples -- /usr/share/doc/daq-midas
- mdaq-config -- /usr/bin/mdaq-config
I add an auto-generated shell script -- mdaq-config. It behaves just like "root-config". You can get midas build flags and link flags this way:
gcc `mdaq-config --cflags` -c -o myfe.o myfe.c
gcc `mdaq-config --libs` -o myfe myfe.o `mdaq-config --libdir`/mfe.o
Bugs and suggestions are welcomed.
P.S. Based on debian packages, I am planing to write another script, "mdaq.py":
- each midas experiment will be configured in a file named "mdaq.yaml"
- mdaq.py reads the configure file and prepare the daq environment, just like "examples/experiment/start_daq.sh"
- mdaq.py will handle "start/stop/restart/info" about the daq codes.
The attached "mdaq.py" is the old one. |
| Attachment 1: daq-midas_deb.tar.gz
|
| Attachment 2: mdaq.py
|
#!/usr/bin/env python
'''
Python script to handle daq sessions using MIDAS/PSI.
Auther: Exaos Lee <Exaos DOT Lee AT gmail DOT com>
Date: 2009-07-15
'''
import os, os.path
import sys, signal, time
from commands import getoutput
from ConfigParser import ConfigParser
from pprint import pprint
class mdaq:
_codes_ = { 'MSERVER':"MIDAS Server", 'MHTTPD':"MIDAS HTTPD Server",
'ODBEDIT':"ODB Editor", 'LOGGER':"Data Logger",
'PROG_FE':"Frontend", 'PROG_ANA':"Analyzer" }
_env_ = dict()
def __init__(self,**kws):
env = dict()
if kws.has_key('fname') and kws['fname'] != "":
env = self.parse_conf(kws['fname'])
self.apply_env(env)
def parse_conf(self, fname):
if not os.path.isfile(fname):
print("**WARNING** File \"%s\" does not exist!"%fname)
return None
conf = ConfigParser()
conf.read(fname)
env={'CONF_FILE':fname }
fcpath = os.path.dirname(fname)
sec = 'EXP_INFO' ## Parsing Experiment Information
env['EXP_INFO'] = [ self.safe_parse(conf,sec,'Name'),
self.safe_parse(conf,sec,'Info'), ]
msg = self.safe_parse(conf,sec,'DAQ_DIR')
if msg != "" and os.path.isdir(msg):
dpath = msg
if not os.path.isabs(msg):
dpath = os.path.join(os.path.abspath(fcpath),msg)
env['DAQ_DIR'] = os.path.normpath(dpath)
## Parsing MIDAS configuration
env['MIDASSYS'] = ["MIDAS System Path",
self.safe_parse(conf,"MIDAS","MIDASSYS")]
if env['MIDASSYS'][1]=="": env['MIDASSYS'][1] = self.get_default_msys()
## Parsing Info of programs
for key in self._codes_.keys():
code = self.parse_prog_opt(conf, key, fpath=fcpath)
if code:
env[key] = [self._codes_[key], ]
env[key].extend(code)
port = self.safe_parse(conf, "MHTTPD","PORT")
if port != "":
if not env.has_key('MHTTPD'):
env['MHTTPD'] = ["MIDAS HTTP Server",
os.path.join(env['MIDASSYS'][1],"bin/mhttpd"),
"-h localhost -D"]
if "-p %s"%port not in env['MHTTPD'][2]:
env['MHTTPD'][2] = env['MHTTPD'][2] + " -p %s"%port
return env
def parse_prog_opt(self, cnf, sec, fpath="./"):
name = self.safe_parse(cnf, sec, "NAME")
path = self.safe_parse(cnf, sec, "PATH")
fn = os.path.join(path,name)
if name == "": fn = ""
elif not os.path.isabs(fn):
fn = os.path.normpath(os.path.join(os.path.abspath(fpath),fn))
else:
fn = os.path.normpath(fn)
opts = self.safe_parse(cnf, sec, "Option")
if opts != "" or name != "":
return [fn, opts]
return None
def safe_parse(self, cnf, sec, opt):
try: return cnf.get(sec,opt)
except: return ""
def get_default_msys(self):
if os.environ.has_key('MIDASSYS'): return os.environ['MIDASSYS']
return "/opt/MIDAS.PSI/Version/Current"
def apply_env(self, daq_env=dict()):
env = daq_env
if not env:
print("** WARNING** No environment defined! Using the default values. ****")
env = {'MIDASSYS':['MIDAS System Path', ""], 'CONF_FILE':"", }
if not env.has_key('MIDASSYS'): env['MIDASSYS'] = ['MIDAS System Path',""]
if env['MIDASSYS'][1]=="": env['MIDASSYS'][1] = self.get_default_msys()
def_env = {
'MSERVER' :["MIDAS Server",
os.path.join(env['MIDASSYS'][1],"bin/mserver"),
"-D", True],
'MHTTPD' :["MIDAS HTTP Server",
os.path.join(env['MIDASSYS'][1],"bin/mhttpd"),
"-h localhost -D -p 8080", True],
'ODBEDIT' :["ODB Editor",
os.path.join(env['MIDASSYS'][1],"bin/odbedit"),
"-c clean", False],
'LOGGER' :["Data Logger",
os.path.join(env['MIDASSYS'][1],"bin/mlogger"),
"", True],
'PROG_FE' :["Frontend","", "", True],
'PROG_ANA':["Analyzer","", "", True],
'EXP_INFO':["test","The test experiment"],
'DAQ_DIR':os.path.join(os.environ['HOME'],"online/test") }
for key in def_env.keys():
if not env.has_key(key): env[key] = def_env[key]
for key in self._codes_.keys():
if env[key][1] == "": env[key][1] = def_env[key][1]
if env[key][2] == "": env[key][2] = def_env[key][2]
if not os.path.isdir(env['DAQ_DIR']):
try: os.makedirs(env['DAQ_DIR'])
except: print("ERROR to make file")
## Asign default program options
exp_opt = "-e %s"%env['EXP_INFO'][0]
for key in ['LOGGER', 'PROG_FE', 'PROG_ANA', "MHTTPD"]:
if exp_opt not in env[key][2]:
env[key][2] = exp_opt + ' ' + env[key][2]
if "-D" not in env[key][2]:
env[key][2] = env[key][2] + ' -D'
if exp_opt not in env['ODBEDIT'][2]:
env['ODBEDIT'][2] = env['ODBEDIT'][2] + ' ' + exp_opt
self._env_ = env
def print_conf(self):
print "\n================= DAQ Config ====================="
for key in self._env_.keys():
print key,": ",
pprint(self._env_[key])
def info(self): self.print_conf()
## running programs
def getpids(self, pname):
return [int(k) for k in getoutput("pidof %s"%pname).strip().split()]
def kill_prog(self,pname):
for i in self.getpids(pname): os.kill(i,signal.SIGTERM)
def run_prog(self, prog):
if len(prog)>=4 and not prog[3]: return
if len(prog)>=3 and prog[1]:
print("Starting %s: %s"%(prog[0],prog[1]))
os.system("%s %s"%(prog[1],prog[2]))
else:
print("%s: no executable exist!"%prog[0])
def get_status(self,pfile):
msg = "Not running!"
pids = self.getpids( os.path.basename(pfile) )
if pids: return 'PIDs = ' + str(pids)
return msg
def echo_status(self,pname,pfile):
print("%s:\n %s\n "%(pname,pfile)
+ self.get_status(pfile))
## Session manage: start, stop, status, restart
def status(self):
print( "\n===== Session status for experiment \"%s\" =====\n" %
self._env_['EXP_INFO'][0] )
for code in ['MSERVER','MHTTPD','LOGGER','PROG_FE','PROG_ANA']:
self.echo_status(self._env_[code][0], self._env_[code][1])
def start(self):
## Running mserver & mhttpd
ms_pids = self.getpids(os.path.basename(self._env_['MSERVER'][1]))
if not ms_pids:
self.run_prog(self._env_['MSERVER'])
print("Killing %s ...\n"%self._env_['MHTTPD'][0])
self.kill_prog(self._env_['MHTTPD'][1])
if not self.getpids(os.path.basename(self._env_['MHTTPD'][1])):
self.run_prog(self._env_['MHTTPD'])
## Running FE and Analyzer
old_path = os.getcwd()
os.chdir( self._env_['DAQ_DIR'] )
# Init ODB
print("Initializing ODB for experiment %s ..." % self._env_['EXP_INFO'][0])
self.run_prog(self._env_['ODBEDIT'])
time.sleep(1)
# First stop, then start ...
self.stop()
time.sleep(1)
# Start FE & ANA
self.run_prog(self._env_['PROG_FE'])
self.run_prog(self._env_['PROG_ANA'])
time.sleep(1)
# Start Logger
self.run_prog(self._env_['LOGGER'])
os.chdir(old_path)
def stop(self):
for key in ["LOGGER", "PROG_FE", "PROG_ANA"]:
if self._env_[key][1]:
print("Killing threads:\n %s"%self._env_[key][1])
self.kill_prog(self._env_[key][1])
def stop_all(self):
self.stop()
for key in ['MHTTPD', 'MSERVER']:
if self._env_[key][1]:
print("Killing threads:\n %s"%self._env_[key][1])
self.kill_prog(self._env_[key][1])
def restart(self): self.start()
if __name__=='__main__':
from optparse import OptionParser
usage = "usage: %prog [options] <command>"
parser = OptionParser(usage=usage, version="%prog 0.1")
parser.add_option("-c","--config", metavar="filename",
help="Read config from FILENAME")
cmds = ["start","stop","stop_all","restart","status","info"]
(opts,args) = parser.parse_args( sys.argv[1:] )
if not args or len(args)>2:
parser.print_help()
print "\nCommands:\n\t",
pprint(cmds)
exit()
fcnf = "daq.conf"
if opts.config: fcnf = opts.config
if args[0] in cmds:
daq = mdaq(fname=fcnf)
exec( "daq.%s()"%args[0] )
else:
parser.print_help()
print "\nCommands:\n\t",
pprint(cmds)
|
|
769
|
27 Jun 2011 |
Konstantin Olchanski | Suggestion | Build MIDAS debian packages using autoconf/automake. | > I deployed several Debian Linux boxes as the DAQ systems in our lab. But I
feel it's boring to build and install midas and its related softwares (such as
root) on each box.
Our solution at TRIUMF is to install such packages on a shared NFS filesystem
visible to all client computers. This works well for ROOT and but MIDAS we found
it nearly impossible to keep MIDAS versions in sync between different projects
and expiments, so each experiment uses it's own copy of MIDAS, usually located
in the experiment home directory ($HOME/packages/midas). Because we often need
to make local modifications to MIDAS sources (Makefile, etc), we do not
"install" MIDAS into non-user-writable /usr/local & etc.
> I use autoconf/automake
The promise (premise) of autoconf/automake is to "hide" system dependencies. The
scripts are supposed to automatically probe the build environment and construct
an appropriate Makefile.
In practice, the autotool scripts always have bugs and incorrect assumptions
about the build environment and only work well for a few standardized systems
(RHEL and Debian derivatives) where the differences are so trivial that
autotools is an overkill and a normal Makefile is adequate for the job.
In my experience, as soon as I try to build an autotool-ized package on anything
that does not look like RHEL or Debian, autotool scripts explode and have to be
debugged and kludged by hand. Anybody who has ever done that would agree with me
that one would rather hack the ugliest Makefile than any of the autotool
generated gibberish.
And of course autotools have never handled cross-compilation in any reasonable
way. Since we do cross-compile MIDAS (for VxWorks and embedded Linux, see "make
crosscompile") a Makefile is required and it so happens that the same Makefile
also works for normal Linux and MacOS, thank you very much.
> Here are the installation:
> [*] executalbes -- /usr/lib/daq-midas/bin
> [*] library and objs -- /usr/lib/daq-midas/lib
Is this in violation of the LSB (or LFS)? I though they mandate that files
controlled by package manager should be /usr/bin/odbedit, /usr/lib64/libmidas.a,
etc (/usr/bin/midas/odbedit no permitted).
> gcc `mdaq-config --cflags` -c -o myfe.o myfe.c
Please check if your config scripts correctly handle the "-m32" and "-m64" flags
- we frequently cross-compile 32-bit MIDAS executables on 64-bit machines.
K.O. |
|
2172
|
24 May 2021 |
Mathieu Guigue | Bug Report | Bug "is of type" | Hi,
I am running a simple FE executable that is supposed to define a PRAW DWORD bank.
The issue is that, right after the start of the run, the logger crashes without messages.
Then the FE reports this error, which is rather confusing.
```
12:59:29.140 2021/05/24 [feTestDatastruct,ERROR] [odb.cxx:6986:db_set_data1,ERROR] "/Equipment/Trigger/Variables/PRAW" is of type UINT32, not UINT32
``` |
|
2199
|
02 Jun 2021 |
Konstantin Olchanski | Bug Report | Bug "is of type" | > Hi,
>
> I am running a simple FE executable that is supposed to define a PRAW DWORD bank.
> The issue is that, right after the start of the run, the logger crashes without messages.
> Then the FE reports this error, which is rather confusing.
> ```
> 12:59:29.140 2021/05/24 [feTestDatastruct,ERROR] [odb.cxx:6986:db_set_data1,ERROR] "/Equipment/Trigger/Variables/PRAW" is of type UINT32, not UINT32
> ```
I think this is fixed in latest midas. There was a typo in this message, the same tid was printed twice,
with result you report "mismatch UINT32 and UINT32", instead of "mismatch of UINT32 vs what is actually there".
This fixes the message, after that you have to manually fix the mismatch in the data type in ODB (delete old one, I guess).
K.O. |
|
1456
|
02 Mar 2019 |
Pintaudi Giorgio | Forum | Best MIDAS branch/version for "production" | Hello!
My name is Giorgio Pintaudi and I am a Ph.D. student at Yokohama National
University (Japan). I also happen to be a T2K collaborator.
I am currently developing the DAQ software for a T2K near detector called WAGASCI
(different from ND280) and we recently decided to adopt MIDAS as a user
interface.
Now I am using the "develop" branch of the MIDAS BitBucket repository: I merge
the remote repository every now and then with my local copy and that is fine ...
but on the 25th of April our experiment is officially starting and I was
wondering which version of MIDAS should I use for "production".
So my question is:
For a running experiment that needs software stability what branch of the MIDAS
repository is better suited? The master branch or the develop branch? Moreover,
what point in time do you think is more stable?
Best regards
Giorgio |
|
1462
|
04 Mar 2019 |
Konstantin Olchanski | Forum | Best MIDAS branch/version for "production" | Hi, Giorgio - you are asking excellent questions. I will try to answer them, but as ever, there are no
easy answers.
In general, the top of the midas "develop" branch is "the best midas there is".
So for a new experiment, it is a reasonable place to start. Of course you can see
that there is quite a bit of commit activity going on, however, most substantial
changes are done on separate branches, where we try the new code, debug
it and test it. Only when the new code is "ready", we commit it to the "develop"
branch. Then, most often, we find some more last minute post-merge bugs,
and fix them right then and there on the head of the "develop" branch. Eventually
the dust settles and we have stable code that stays stable for a long time.
For example, right now we are waiting for the dust to settle on the change
of the MIDAS URL scheme, which was necessitated by needs of several experiments
that have more-complex-then-usual https proxy configurations. Unusual today,
but more common as we move forward, I think.
So if the head of the "develop" branch does not work for you, we encourage
that you file a bug report (here on this forum or on the bitbucket issue tracker).
While we try to sort out your problem, you can fall back to a previous version
of midas:
a) go back to one of the older "midas-YYYY-MM-X" tags
b) go back to one of the release candidate branches "feature/midas-YYYY-MM".
But if you have an already running system and you already have a working
instance of MIDAS, you do not have to update it to the latest version
unless you need some newest feature or you suffer from a bug
that has been fixed in a newer version.
In general, I find that it is fairly safe to update a working instance of MIDAS to
the latest code. But do keep your old working copy, if there is trouble, you can
always go back to something that works.
Now to your questions:
> For a running experiment that needs software stability what branch of the MIDAS
> repository is better suited?
easy to answer. the most stable is the version you are using right now (doh!). each time
you upgrade, there is a risk that something will go wrong.
if you start from scratch, use the head of the "develop" branch (the latest and greatest),
if you run into trouble, report the trouble and update to a newer version with your
trouble fixed, or go back in time to previous tags and release candidate branches (as I described
above).
> The master branch or the develop branch?
the "develop" branch.
> Moreover, what point in time do you think is more stable?
I find that it is impossible to have a stable-stable-stable version of midas
because the rest of the world flows forward in time. Old versions of midas
stop compiling because of OS and compiler changes; they stop running
because of OS or web browser or hardware changes. Then somebody
always asks for new features and new or old bugs surface constantly.
But we try to mark some "good" spots using git tags and release branches,
however the more far you go back in time, the more likely the code will
not even compile anymore.
K.O.
> Hello!
> My name is Giorgio Pintaudi and I am a Ph.D. student at Yokohama National
> University (Japan). I also happen to be a T2K collaborator.
> I am currently developing the DAQ software for a T2K near detector called WAGASCI
> (different from ND280) and we recently decided to adopt MIDAS as a user
> interface.
> Now I am using the "develop" branch of the MIDAS BitBucket repository: I merge
> the remote repository every now and then with my local copy and that is fine ...
> but on the 25th of April our experiment is officially starting and I was
> wondering which version of MIDAS should I use for "production".
> So my question is:
> For a running experiment that needs software stability what branch of the MIDAS
> repository is better suited? The master branch or the develop branch? Moreover,
> what point in time do you think is more stable?
> Best regards
> Giorgio |
|
1463
|
04 Mar 2019 |
Pintaudi Giorgio | Forum | Best MIDAS branch/version for "production" | Hi Konstantin,
thank you for the very in-depth answer. Right now I am using the latest version of MIDAS (the head of the
develop branch) and I have not noticed any bugs or problems in the MIDAS code, except the ones that I
myself put in the part of the code that I am developing of course.
So I will do as you suggest and continue to use the head of the develop branch.
Thank you again for taking the time to answer all my questions!
PS other than the code peculiar to our experiment, I have made a little modification to the MIDAS install
Makefile (I noticed that there is an "install" target but not an "uninstall" target so I wrote it).
If you are interested, I could make a merge request on BitBucket, just let me know.
Bests
Giorgio
> Hi, Giorgio - you are asking excellent questions. I will try to answer them, but as ever, there are no
> easy answers.
>
> In general, the top of the midas "develop" branch is "the best midas there is".
>
> So for a new experiment, it is a reasonable place to start. Of course you can see
> that there is quite a bit of commit activity going on, however, most substantial
> changes are done on separate branches, where we try the new code, debug
> it and test it. Only when the new code is "ready", we commit it to the "develop"
> branch. Then, most often, we find some more last minute post-merge bugs,
> and fix them right then and there on the head of the "develop" branch. Eventually
> the dust settles and we have stable code that stays stable for a long time.
>
> For example, right now we are waiting for the dust to settle on the change
> of the MIDAS URL scheme, which was necessitated by needs of several experiments
> that have more-complex-then-usual https proxy configurations. Unusual today,
> but more common as we move forward, I think.
>
> So if the head of the "develop" branch does not work for you, we encourage
> that you file a bug report (here on this forum or on the bitbucket issue tracker).
>
> While we try to sort out your problem, you can fall back to a previous version
> of midas:
>
> a) go back to one of the older "midas-YYYY-MM-X" tags
> b) go back to one of the release candidate branches "feature/midas-YYYY-MM".
>
> But if you have an already running system and you already have a working
> instance of MIDAS, you do not have to update it to the latest version
> unless you need some newest feature or you suffer from a bug
> that has been fixed in a newer version.
>
> In general, I find that it is fairly safe to update a working instance of MIDAS to
> the latest code. But do keep your old working copy, if there is trouble, you can
> always go back to something that works.
>
> Now to your questions:
>
> > For a running experiment that needs software stability what branch of the MIDAS
> > repository is better suited?
>
> easy to answer. the most stable is the version you are using right now (doh!). each time
> you upgrade, there is a risk that something will go wrong.
>
> if you start from scratch, use the head of the "develop" branch (the latest and greatest),
> if you run into trouble, report the trouble and update to a newer version with your
> trouble fixed, or go back in time to previous tags and release candidate branches (as I described
> above).
>
> > The master branch or the develop branch?
>
> the "develop" branch.
>
> > Moreover, what point in time do you think is more stable?
>
> I find that it is impossible to have a stable-stable-stable version of midas
> because the rest of the world flows forward in time. Old versions of midas
> stop compiling because of OS and compiler changes; they stop running
> because of OS or web browser or hardware changes. Then somebody
> always asks for new features and new or old bugs surface constantly.
>
> But we try to mark some "good" spots using git tags and release branches,
> however the more far you go back in time, the more likely the code will
> not even compile anymore.
>
>
> K.O.
>
>
>
> > Hello!
> > My name is Giorgio Pintaudi and I am a Ph.D. student at Yokohama National
> > University (Japan). I also happen to be a T2K collaborator.
> > I am currently developing the DAQ software for a T2K near detector called WAGASCI
> > (different from ND280) and we recently decided to adopt MIDAS as a user
> > interface.
> > Now I am using the "develop" branch of the MIDAS BitBucket repository: I merge
> > the remote repository every now and then with my local copy and that is fine ...
> > but on the 25th of April our experiment is officially starting and I was
> > wondering which version of MIDAS should I use for "production".
> > So my question is:
> > For a running experiment that needs software stability what branch of the MIDAS
> > repository is better suited? The master branch or the develop branch? Moreover,
> > what point in time do you think is more stable?
> > Best regards
> > Giorgio |
|
1466
|
05 Mar 2019 |
Konstantin Olchanski | Forum | Best MIDAS branch/version for "production" | >
> PS other than the code peculiar to our experiment, I have made a little modification to the MIDAS install
> Makefile (I noticed that there is an "install" target but not an "uninstall" target so I wrote it).
> If you are interested, I could make a merge request on BitBucket, just let me know.
>
Hmm... for most experiments, we do not "install" midas. I should probably remove the "install" target from the Makefile.
K.O.
> Bests
> Giorgio
>
> > Hi, Giorgio - you are asking excellent questions. I will try to answer them, but as ever, there are no
> > easy answers.
> >
> > In general, the top of the midas "develop" branch is "the best midas there is".
> >
> > So for a new experiment, it is a reasonable place to start. Of course you can see
> > that there is quite a bit of commit activity going on, however, most substantial
> > changes are done on separate branches, where we try the new code, debug
> > it and test it. Only when the new code is "ready", we commit it to the "develop"
> > branch. Then, most often, we find some more last minute post-merge bugs,
> > and fix them right then and there on the head of the "develop" branch. Eventually
> > the dust settles and we have stable code that stays stable for a long time.
> >
> > For example, right now we are waiting for the dust to settle on the change
> > of the MIDAS URL scheme, which was necessitated by needs of several experiments
> > that have more-complex-then-usual https proxy configurations. Unusual today,
> > but more common as we move forward, I think.
> >
> > So if the head of the "develop" branch does not work for you, we encourage
> > that you file a bug report (here on this forum or on the bitbucket issue tracker).
> >
> > While we try to sort out your problem, you can fall back to a previous version
> > of midas:
> >
> > a) go back to one of the older "midas-YYYY-MM-X" tags
> > b) go back to one of the release candidate branches "feature/midas-YYYY-MM".
> >
> > But if you have an already running system and you already have a working
> > instance of MIDAS, you do not have to update it to the latest version
> > unless you need some newest feature or you suffer from a bug
> > that has been fixed in a newer version.
> >
> > In general, I find that it is fairly safe to update a working instance of MIDAS to
> > the latest code. But do keep your old working copy, if there is trouble, you can
> > always go back to something that works.
> >
> > Now to your questions:
> >
> > > For a running experiment that needs software stability what branch of the MIDAS
> > > repository is better suited?
> >
> > easy to answer. the most stable is the version you are using right now (doh!). each time
> > you upgrade, there is a risk that something will go wrong.
> >
> > if you start from scratch, use the head of the "develop" branch (the latest and greatest),
> > if you run into trouble, report the trouble and update to a newer version with your
> > trouble fixed, or go back in time to previous tags and release candidate branches (as I described
> > above).
> >
> > > The master branch or the develop branch?
> >
> > the "develop" branch.
> >
> > > Moreover, what point in time do you think is more stable?
> >
> > I find that it is impossible to have a stable-stable-stable version of midas
> > because the rest of the world flows forward in time. Old versions of midas
> > stop compiling because of OS and compiler changes; they stop running
> > because of OS or web browser or hardware changes. Then somebody
> > always asks for new features and new or old bugs surface constantly.
> >
> > But we try to mark some "good" spots using git tags and release branches,
> > however the more far you go back in time, the more likely the code will
> > not even compile anymore.
> >
> >
> > K.O.
> >
> >
> >
> > > Hello!
> > > My name is Giorgio Pintaudi and I am a Ph.D. student at Yokohama National
> > > University (Japan). I also happen to be a T2K collaborator.
> > > I am currently developing the DAQ software for a T2K near detector called WAGASCI
> > > (different from ND280) and we recently decided to adopt MIDAS as a user
> > > interface.
> > > Now I am using the "develop" branch of the MIDAS BitBucket repository: I merge
> > > the remote repository every now and then with my local copy and that is fine ...
> > > but on the 25th of April our experiment is officially starting and I was
> > > wondering which version of MIDAS should I use for "production".
> > > So my question is:
> > > For a running experiment that needs software stability what branch of the MIDAS
> > > repository is better suited? The master branch or the develop branch? Moreover,
> > > what point in time do you think is more stable?
> > > Best regards
> > > Giorgio |
|
1471
|
05 Mar 2019 |
Stefan Ritt | Forum | Best MIDAS branch/version for "production" | > Hmm... for most experiments, we do not "install" midas. I should probably remove the "install" target from the Makefile.
... and change the documentation accordingly (Suzannah!?). Installing midas these days does not really make sense, since normally only one
users uses it on a given machine.
Stefan |
|
Draft
|
06 Mar 2019 |
Pintaudi Giorgio | Forum | Best MIDAS branch/version for "production" | > > Hmm... for most experiments, we do not "install" midas. I should probably remove the "install" target from the Makefile.
>
>
> ... and change the documentation accordingly (Suzannah!?). Installing midas these days does not really make sense, since normally only one
> users uses it on a given machine.
>
> Stefan
I understand. Anyway, I preferred to install MIDAS in the /opt/midas folder to be consistent with the other programs (Pyrame and Calicoes from LLR) that we are using for our experiment (they are all installed in the /opt folder). Moreover, I am using Linux systemd to enable mhttpd on startup (and other handy features like auto-restart after a crash). And quite unfortunately CentOS doesn't support to start systemd units as a non-root user.
So in my particular case, perhaps it made some sense to install MIDAS in a folder other than the source code folder.
Giorgio |
|
1479
|
06 Mar 2019 |
Pintaudi Giorgio | Forum | Best MIDAS branch/version for "production" | > > Hmm... for most experiments, we do not "install" midas. I should probably remove the "install" target from the Makefile.
>
>
> ... and change the documentation accordingly (Suzannah!?). Installing midas these days does not really make sense, since normally only one
> users uses it on a given machine.
>
> Stefan
I understand. Anyway, I preferred to install MIDAS in the /opt/midas folder to remain consistent with the other programs that we are using for
our experiment (Pyrame and Calicoes from LLR). I am also using Linux systemd to enable mhttpd on startup (and other handy features like auto-
restart after a crash) and unfortunally CentOS doesn't support to enable systemd units as a non-root user.
So in my particular case, perhaps it made some sense to install MIDAS in a folder other than the source code folder
Giorgio |
|
1480
|
06 Mar 2019 |
Konstantin Olchanski | Forum | Best MIDAS branch/version for "production" | > > > Hmm... for most experiments, we do not "install" midas. I should probably remove the "install" target from the Makefile.
>
> install MIDAS in the /opt/midas folder to remain consistent with the other programs that we are using for
> our experiment (Pyrame and Calicoes from LLR).
I see. Would this work as well? Instead of "make install" do this:
su - root
cd /opt
git pull midas
cd midas
make
add /opt/midas/linux/bin to your PATH. (is it time to get rid of the "linux" part from the default build path?!?)
> I am also using Linux systemd to enable mhttpd on startup
We use cron @reboot to start mhttpd.
But. CentOS7 systemd cron unit file has a bug and they run @reboot cron jobs before NIS and autofs is ready, see
http://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Enable_crontab_.40reboot_for_MIDAS_.28CentOS7.29
Can you post your systemd unit file to this elog, others may find it useful.
K.O. |
|
1483
|
06 Mar 2019 |
Pintaudi Giorgio | Forum | Best MIDAS branch/version for "production" | > I see. Would this work as well? Instead of "make install" do this:
> su - root
> cd /opt
> git pull midas
> cd midas
> make
> add /opt/midas/linux/bin to your PATH. (is it time to get rid of the "linux" part from the default build path?!?)
Got it. I will do that in the future.
> Can you post your systemd unit file to this elog, others may find it useful.
[Unit]
Description=MIDAS data acquisition system
After=network.target
StartLimitIntervalSec=0
[Service]
Type=simple
Restart=always
RestartSec=3
User=neo
ExecStart=/opt/midas/bin/mhttpd -e WAGASCI --http 8081 --https 8444
Environment="MIDASSYS=/opt/midas" "MIDAS_EXPTAB=/home/neo/Code/WAGASCI/MIDAS/online/exptab" "MIDAS_EXPT_NAME=WAGASCI"
"SVN_EDITOR=emacs -nw" "GIT_EDITOR=emacs -nw"
PassEnvironment=MIDASSYS MIDAS_EXPTAB MIDAS_EXPT_NAME SVN_EDITOR GIT_EDITOR
[Install]
WantedBy=multi-user.target |
|
451
|
02 Mar 2008 |
Exaos Lee | Suggestion | Bash Script for handling an experiment code | I rearanged the files in "examples/experiment" as the attached "mtest_exp.zip". I re-write the start/stop script as the attached "daq.sh". The script "daq.sh" can be re-used for many experiments. The user only needs to provide an script "daq_env.sh" as the following containing the settings for the experiment environment.
#!/bin/sh
[ ! "$MIDASSYS" ] && MIDASSYS=/opt/MIDAS.PSI/Version/Current
[ ! "$HTTPPORT" ] && HTTPPORT=8080
[ ! "$SRVHOST" ] && SRVHOST=localhost
LOGGER=${MIDASSYS}/bin/mlogger
EXPPATH=/home/das/online/test
CODEPATH=${EXPPATH}/code
LOGGER=${MIDASSYS}/bin/mlogger
PROG_FE=${CODEPATH}/frontend
PROG_ANA=${CODEPATH}/analyzer
if [ ! "$MIDAS_EXPTAB" ]; then
MIDAS_DIR=${EXPPATH}
else
MIDAS_EXPT_NAME="test"
fi
I hope this can be helpful.  There seem to be some problems such as:
1. When several experiments are defined, the $LOGGER may be not the one used for this exp.
2. The "pidof" may be not in some platforms, so this script is limited.
Hope anybody can help me to improve it for general purpose. All my best! |
| Attachment 1: daq.sh
|
#!/bin/sh
DAQ_SETENV=./daq_env.sh
[ -f "${DAQ_SETENV}" ] && source ${DAQ_SETENV}
### Start ....
start()
{
#### Running mserver
pidmsrv=`pidof mserver`
if [ ! "$pidmsrv" ]; then
echo Starting mserver ...
$MIDASSYS/bin/mserver -D -m
pidmhttpd=`pidof mhttpd`
[ "$pidmhttpd" ] && kill -9 $pidmhttpd
echo Starting mhttpd ...
$MIDASSYS/bin/mhttpd -h localhost -D -p $HTTPPORT
fi
#### Change work directory
OLD_DIR=`pwd`
cd ${EXPPATH}
#### Running FE and Analyzer ...
# Options for multi experiments running ...
[ "$MIDAS_EXPTAB" ] && OPTIONS="-e $MIDAS_EXPT_NAME"
$MIDASSYS/bin/odbedit ${OPTIONS} -c clean
sleep 2
echo Starting frontend session: ${PROG_FE} ...
${PROG_FE} ${OPTIONS} -D
echo Starting analyzer session: ${PROG_ANA} ...
${PROG_ANA} ${OPTIONS} -D
sleep 1
echo Starting logger utility: mlogger ...
${LOGGER} ${OPTIONS} -D
cat <<EOF
Please point your web browser to http://localhost:$HTTPPORT
Or run: firefox http://localhost:$HTTPPORT
To look at live histograms, run: roody -Hlocalhost
EOF
#### Resume old directory
cd ${OLD_DIR}
}
### Stop ....
stop()
{
#### Kill Running sessions
pidlog=`pidof $(basename ${LOGGER})`
pidfe=` pidof $(basename ${PROG_FE})`
pidana=`pidof $(basename ${PROG_ANA})`
[ "$pidlog" ] && kill $pidlog
[ "$pidfe" ] && kill $pidfe
[ "$pidana" ] && kill $pidana
}
### Status ....
status()
{
echo "MIDAS Server (mserver): "
pid=`pidof mserver`
echo_status $pid
echo "MIDAS HTTPD Server (mhttpd): "
pid=`pidof mhttpd`
echo_status $pid
echo "Logger (${LOGGER}): "
pid=`pidof $(basename ${LOGGER})`
echo_status $pid
echo "Frontend Code ($PROG_FE): "
pid=`pidof $(basename ${PROG_FE})`
echo_status $pid
echo "Analyzer Code ($PROG_ANA): "
pid=`pidof $(basename ${PROG_ANA})`
echo_status $pid
}
echo_status()
{
if [ "$1" ]; then
echo -e "\t Running as $1."
else
echo -e "\t Not Running!"
fi
}
### Usage ....
usage()
{
echo "Usage: $0 [<start>|<stop>|<restart>|<status>]"
}
###################### Main Procedure ###########################
case "${1:-''}" in
"start")
start
;;
"stop")
stop
;;
"restart")
stop
start
;;
"status")
status
;;
*)
usage
;;
esac
######################### The END #############################
|
| Attachment 2: mtest_exp.zip
|
|
941
|
28 Nov 2013 |
Konstantin Olchanski | Info | Audit of fixed size arrays | In one of the experiments, we hit a long time bug in mdump - there was an array of 32 equipments and if
there were more than 32 entries under /equipment, it would overrun and corrupt memory. Somehow this
only showed up after mdump was switched to c++. The solution was to use std::vector instead of fixed
size array.
Just in case, I checked other midas programs for fixed size arrays (other than fixed size strings) and found
none. (in midas.c, there is a fixed size array of TR_FIFO[10], but code inspection shows that it cannot
overrun).
I used this script. It can be modified to also identify any strange sized string arrays.
K.O.
#!/usr/bin/perl -w
while (1) {
my $in = <STDIN>;
last unless $in;
#print $in;
$in =~ s/^\s+//;
next if $in =~ /^char/;
next if $in =~ /^static char/;
my $a = $in =~ /(.*)[(\d+)\]/;
next unless $a;
my $a1 = $1;
my $a2 = $2;
next if $a2 == 0;
next if $a2 == 1;
next if $a2 == 2;
next if $a2 == 3;
#print "[$a] [$a1] [$a2]\n";
print "-> $a1[$a2]\n";
}
# end |
|