Back Midas Rome Roody Rootana
  Midas DAQ System, Page 41 of 122  Not logged in ELOG logo
ID Date Author Topic Subjectup
  1093   14 Aug 2015 Konstantin OlchanskiInfoMerged - improved midas network security
> [local:Online:S]RPC hosts>set "Allowed hosts[1]" "host.psi.ch"
> [ODBEdit,INFO] Reloading RPC hosts access control list via hotlink callback

Yes, this debug message can be removed after (say) one or two weeks. I find it useful at the moment.

> The trick not to have to "guess" a remote name is quite useful. I'm happy it even made it into the documentation.

Yes, this came from CERN/ALPHA felabview where we did not have a full list of labview machines that were sending us labview data.

> There is however an
> important shortcoming in the documentation. The old documentation had a "quick start" section, which allowed people quickly to
> set-up and run a midas system. This is still missing. And it should now contain a link to the "Security" page, so that people can set-up
> quickly remote programs.

Yes, Suzannah was just asking me about the same - we will need to review the quick start guide and the documentation about the mserver and midas rpc.

K.O.
  1189   08 Aug 2016 Konstantin OlchanskiReleaseMerged - new pure html web pages: programs and alarms.
The code for the new pure html and javascript web pages was merged into main midas.

In this release, the "programs" and "alarms" pages are implemented as html files, see 
resources/programs.html and alarms.html.

Eventually we hope to implement all midas web pages in html, so this is just a start.

If you see problems with the new html code, you can revert to the old mhttpd-generated web 
pages by removing the files programs.html and alarms.html.

The new code for starting and stopping runs (start.html and transition.html) is also merged, but not 
yet enabled, pending a few more tests.

K.O.
  846   14 Dec 2012 Vinzenz BildsteinSuggestionMidas + Elog with SSL
I've been trying to set up midas to create an automatic elog entry at the end of
each run and I've run into a problem. I've setup an elog on our server which
uses SSL and it seems that the melog provided by midas to create logbook entries
doesn't know any SSL.

My solution to this was to copy the crypt.c from the elog package to the
computer running midas and changed melog.c and the makefile to use SSL if a flag
-s is used. Does this seem like a sensible solution or did I oversee the obvious
and/or right way to do this?
  847   14 Dec 2012 Stefan RittSuggestionMidas + Elog with SSL
> I've been trying to set up midas to create an automatic elog entry at the end of
> each run and I've run into a problem. I've setup an elog on our server which
> uses SSL and it seems that the melog provided by midas to create logbook entries
> doesn't know any SSL.
> 
> My solution to this was to copy the crypt.c from the elog package to the
> computer running midas and changed melog.c and the makefile to use SSL if a flag
> -s is used. Does this seem like a sensible solution or did I oversee the obvious
> and/or right way to do this?

Indeed melog.c is an old version of the elog.c utility in the elog package, which has not been maintained since a 
long time. Can't you just use the recent elog.c utility from the elog package?
  848   17 Dec 2012 Vinzenz BildsteinSuggestionMidas + Elog with SSL
> > I've been trying to set up midas to create an automatic elog entry at the end of
> > each run and I've run into a problem. I've setup an elog on our server which
> > uses SSL and it seems that the melog provided by midas to create logbook entries
> > doesn't know any SSL.
> > 
> > My solution to this was to copy the crypt.c from the elog package to the
> > computer running midas and changed melog.c and the makefile to use SSL if a flag
> > -s is used. Does this seem like a sensible solution or did I oversee the obvious
> > and/or right way to do this?
> 
> Indeed melog.c is an old version of the elog.c utility in the elog package, which has not been maintained since a 
> long time. Can't you just use the recent elog.c utility from the elog package?

Well, that's essentially what I did, I just didn't want to install the whole elog package on the midas server. Whether
the utility is called elog or melog doesn't really matter. I just wanted to make sure that this is the right way to do
it. 

Thanks!
  266   08 Jun 2006 Konstantin OlchanskiBug ReportMidas does not build on Fedora 5
Fresh svn checkout of MIDAS does not build on Fedora 5, I get this error:

cc -c -g -O2 -Wall -Wuninitialized -Iinclude -Idrivers -I../mxml -Llinux/lib
-DINCLUDE_FTPLIB   -D_LARGEFILE64_SOURCE -DHAVE_ROOT -pthread
-I/triumfcs/trshare/olchansk/root/root_v5.10.00_SL40/include -m32 -DOS_LINUX
-fPIC -Wno-unused-function -o linux/lib/odb.o src/odb.c
src/odb.c: In function 'db_open_database':
src/odb.c:805: warning: dereferencing type-punned pointer will break
strict-aliasing rules
src/odb.c: In function 'db_lock_database':
src/odb.c:1350: warning: dereferencing type-punned pointer will break
strict-aliasing rules
cc: Internal error: Segmentation fault (program cc1)
Please submit a full bug report.
See <URL:http://bugzilla.redhat.com/bugzilla> for instructions.
make: *** [linux/lib/odb.o] Error 1

If I compile odb.c without "-O2", the rest of MIDAS builds without any more errors.

The observed warnings are (I do not know what they mean):
warning: dereferencing type-punned pointer will break strict-aliasing rules
warning: missing sentinel in function call (Cannot do without sentinels, eh?)
warning: pointer targets in passing argument 3 of 'getsockname' differ in signedness
warning: non-local variable '<anonymous struct> out_info' uses anonymous type

The "invalid lvalue" errors seem to have been successfully vanquished.

K.O.
  665   20 Oct 2009 Peter SimpsonForumMidas in linux
Hi,
I'm new to both Linux and Midas and having trouble installing the programme - 
the install file suggeats that I should have a directory:

midas/unix

but this doesn't appear.

Also, when running the "make" for the library (step 3 of the installation), I 
don't see the directory "zlib.a", but there is a "libz.a" and I note the 
original .tar file has no libz.a file in it. Is this a typo on the 
installation?
The terminal window at this point displays:

undefined reference to `errno'
make: ***[example] Error 1

Further help will no doubt be required as I've used windows throughout my 
research and now looking to learn how to use linux. Any help greatly 
appreciated. Thanks!
  324   22 Jan 2007 Carl MetelkoForumMidas on a x86_64
Hi,
   has anyone managed to get midas to work on a x86_64 processor. I followed the
instructions for the 64-bit opteron but i am getting runtime error when trying
the examples.

 When running example/basic/odb_test I getting errors like

[odb.c:6818:db_get_record] struct size mismatch for "/Alarms/Alarms/Demo ODB"
(464 instead of 452)
[odb.c:6818:db_get_record] struct size mismatch for "/Alarms/Alarms/Demo ODB"
(464 instead of 452)
[midas.c:16576:al_check] Cannot get alarm record

Any ideas what is wrong?
  325   22 Jan 2007 Konstantin OlchanskiForumMidas on a x86_64
>    has anyone managed to get midas to work on a x86_64 processor. I followed the
> instructions for the 64-bit opteron but i am getting runtime error when trying
> the examples.


We run 64-bit MIDAS on RHEL4 with 64-bit ROOT and everything generally works,
except for compatibility problems with 32-bit MIDAS.

Everything should work if you ensure that on your 64-bit machine everything is
compiled 64-bit (including the mserver - we always forget to install the correct version
to /usr/local/bin). 32-bit MIDAS programs running on other machine
can talk to 64-bit MIDAS via the mserver.

The big problem is that 64-bit and 32-bit ODB turned out to be incompatible - several data
fields have different sizes - and we did not decide yet how to fix this. Any fix will involve
breaking the binary ODB for one of the two platforms (we could break both, just to be fair, heh!)

>  When running example/basic/odb_test I getting errors like
> [odb.c:6818:db_get_record] struct size mismatch for "/Alarms/Alarms/Demo ODB" (464 instead of 452)

Yes, data size mismatch errors indicates that you mixed 32-bit and 64-bit MIDAS. Recompile everyting
as 64-bit, remove all the dot-ODB files, remove all the shared memory segments (ipcrm),
then everything should work.

K.O.
  331   26 Jan 2007 Carl MetelkoForumMidas on a x86_64
I upgraded from 1.9.5 to the latest on SVN an it works fine
  395   12 Jul 2007 Konstantin OlchanskiForumMidas on a x86_64 - incompatible with x86_32
> We run 64-bit MIDAS on RHEL4 with 64-bit ROOT and everything generally works,
> except for compatibility problems with 32-bit MIDAS.
> 
> The big problem is that 64-bit and 32-bit ODB turned out to be incompatible ...

I have now identified 3 data structures that change size when compiled with "-m64":

EVENT_REQUEST: stores a pointer to a function. Pointer size is 4 bytes with -m32 and 8 bytes with -m64.
This structure is part of an array inside BUFFER_HEADER, resulting in a sizable size mismatch between 32
bit and 64 bit shared memory data buffers.

The fix is simple: the function pointer is not used anywhere. Replace is with a "DWORD unused_filler"
makes -m32 and -m64 data buffers compatible. (But breaks compatibility with previous -m64 compiled midas).

CHN_SETTINGS and CHN_STATISTICS: apparently, -m32 and -m64 GCC has different packing rules and in -m64
mode, 4 bytes of padding are added to these data structures. Size size mismatch appears to be benign,
but will result in "size mismatch" complaints from ODB.

The fix is simple: adding "__attribute__ ((__packed__))" to the definition of the data structure makes
-m64 identical to -m32.

The "svn diff" of changes involved is attached below.

The biggest problem here is that making 32-bit ODB and 64-bit ODB compatible requires breaking one or
the other (My proposed changes break the 64-bit version. Alternatively, one could add explicit padding
to these data structures and break the 32-bit ODB).

I think it is important to make 32-bit and 64-bit code compatible: at TRIUMF we have to use a mixed
environment because out latest host computers all run 64-bit Linux while all our VME processors and all
older machines can only run 32-bit code; this incompatibility causes us weekly headaches.

Any thoughts?

K.O.

(this output of svn diff is doctored for clarity)

ladd00:midas$ svn diff
Index: include/midas.h
===================================================================
--- include/midas.h     (revision 3744)
+++ include/midas.h     (working copy)
-   void (*dispatch) (HNDLE, HNDLE, EVENT_HEADER *, void *);
+   INT unused; // was void (*dispatch) (HNDLE, HNDLE, EVENT_HEADER *, void *);
 } EVENT_REQUEST;
 
--- include/msystem.h   (revision 3744)
+++ include/msystem.h   (working copy)

+#define PACKED __attribute__ ((__packed__))  <--- this goes into midas.h inside the #ifdef "we use GCC"
 
-typedef struct {
+typedef struct PACKED { ... CHN_SETTINGS
 
-typedef struct {
+typedef struct PACKED { ... CHN_STATISTICS
  396   13 Jul 2007 Stefan RittForumMidas on a x86_64 - incompatible with x86_32
> The biggest problem here is that making 32-bit ODB and 64-bit ODB compatible requires breaking one or
> the other (My proposed changes break the 64-bit version. Alternatively, one could add explicit padding
> to these data structures and break the 32-bit ODB).
> 
> I think it is important to make 32-bit and 64-bit code compatible: at TRIUMF we have to use a mixed
> environment because out latest host computers all run 64-bit Linux while all our VME processors and all
> older machines can only run 32-bit code; this incompatibility causes us weekly headaches.
> 
> Any thoughts?

I agree to make 32-bit and 64-bit compatible. In the long run, everything will be 64-bit, so I would suggest
in breaking the 32-bit ODB, add some padding there where needed, probably with some conditional compiling.
This ensures to keep the native 64-bit packing, which probably will be somehow optimized for 64-bit
architectures and therefore might be a bit faster in the long run, when most systems are 64-bit. After this
has been implemented and well tested, I would go with an official announcement of the 32-bit break in the ODB,
and release a new version, so people can update from a TAR file if necessary. Existing ODB's can be converted
to the new format by exporting them in XML form and importing them again after the upgrade.
  399   12 Aug 2007 Konstantin OlchanskiForumMidas on a x86_64 - incompatible with x86_32
> I agree to make 32-bit and 64-bit compatible. In the long run, everything will be 64-bit, so I would suggest
> in breaking the 32-bit ODB, add some padding there where needed, probably with some conditional compiling.

I now have the patches to implement this. Changes turned out to be minimal:

1) midas.h: remove unused field "dispatch" from EVENT_REQUEST and bump DATABASE_VERSION from 2 to 3
2) msystem.h: add 32-bit padding to CHN_STATISTICS and CHN_SETTINGS

(Pedantic note: the C/C++ languages permit compilers to arbitrary pad data members inside structures and one is
not supposed to rely on the specific layout of "struct"s, they could changing from day to day depending on
compiler vendor, version, 32/64 bit, optimization level, etc. This is quite silly, but I guess it was the only way
"they" could agree on a standard)

In practice, compilers are will behaved and one can follow simple rules and stay out of trouble.
1) if all data members are of the same size -> no padding
2) do not use "double" (64-bit) and "short" (16-bit), make all char[] arrays divisible by 4 -> size of everything
is 32-bit, see rule 1
3) if you have to use "short", they have to come in pairs to keep everything else aligned to 32-bit
4) if you have to use "double" (or uint64_t), keep them aligned to 64-bit, i.e. struct { int a,b,c; double x;} is
*bad* (4-byte padding may be added between c and x). struct { int a,b,c,d; double x; } is good.

Below are is "svn diff include/midas.h include/msystem.h". These changes have been tested on SL4 32-bit and
64-bit, SL5 32/64, F7 32/64 and SL4/ICC (Intel compiler) 32 bit and 64 bit.

The testing was done by adding checks on sizes of all struct's kept on ODB, i.e.
   assert(sizeof(CHN_SETTINGS        ) ==    640); // ODB v3 with padding
   assert(sizeof(CHN_STATISTICS      ) ==     32); // ODB v3 with padding
   ... etc ...

K.O.

ladd03:midas$ svn diff include/midas.h include/msystem.h
Index: include/midas.h
===================================================================
--- include/midas.h     (revision 3798)
+++ include/midas.h     (working copy)
@@ -38,7 +38,7 @@
  *  @{  */

 /* has to be changed whenever binary ODB format changes */
-#define DATABASE_VERSION 2
+#define DATABASE_VERSION 3

 /* MIDAS version number which will be incremented for every release */
 #define MIDAS_VERSION "2.0.0"
@@ -810,8 +810,6 @@
    short int event_id;           /**< event ID                        */
    short int trigger_mask;       /**< trigger mask                    */
    INT sampling_type;            /**< GET_ALL, GET_SOME, GET_FARM     */
-                                 /**< dispatch function */
-   void (*dispatch) (HNDLE, HNDLE, EVENT_HEADER *, void *);
 } EVENT_REQUEST;

 typedef struct {
Index: include/msystem.h
===================================================================
--- include/msystem.h   (revision 3798)
+++ include/msystem.h   (working copy)
@@ -454,6 +454,7 @@
    INT event_id;
    INT trigger_mask;
    DWORD event_limit;
+   INT pad; // FIXME 64-bit "double" should be 64-bit aligned
    double byte_limit;
    double tape_capacity;
    char subdir_format[32];
@@ -465,6 +466,7 @@
    double bytes_written;
    double bytes_written_total;
    INT files_written;
+   INT pad; // FIXME pad data structure to be 64-bit aligned
 } CHN_STATISTICS;

 typedef struct {
ladd03:midas$
  400   20 Aug 2007 Konstantin OlchanskiForumMidas on a x86_64 - incompatible with x86_32
> > I agree to make 32-bit and 64-bit compatible. In the long run, everything will be 64-bit, so I would suggest
> > in breaking the 32-bit ODB, add some padding there where needed, probably with some conditional compiling.
> 
> I now have the patches to implement this. Changes turned out to be minimal:
> 
> 1) midas.h: remove unused field "dispatch" from EVENT_REQUEST and bump DATABASE_VERSION from 2 to 3
> 2) msystem.h: add 32-bit padding to CHN_STATISTICS and CHN_SETTINGS

The padding of CHN_STATISTICS and CHN_SETTINGS is not working right - somehow mhttpd and mlogger keep recreating the
data in ODB and erasing the padding fields. I am looking into this.

K.O.
  1050   05 May 2015 Pierre-Andre AmaudruzForumMidas seminar
Dear Midas users, 

As part of our commitment to  Midas improvements, this year Dr. Stefan Ritt is coming to Vancouver 
BC, CANADA for his biennial visit from the end of June to mid-July 2015.

A Data acquisition system now a days is expected to do more than just collect data, it has become an 
integrated process with various types of data source for monitoring, control, storage and analysis, 
as well as data visualization using modern techniques.
MIDAS stands for "Maximum Integration Data Acquisition System". It is interesting to think that this 
name was given 20 years ago when none of the interconnectability was available at today's level. 

So in order to keep MIDAS current with new technology and provide a better DAQ tool, we plan to 
discuss topics that would address integration in a larger format, the goal being to provide to the 
users a more robust and "simple" way of doing their work. We will also be working on improvements 
and the addition of new features.

Towards the end of Stefan's visit, we will have a "Midas seminar" with a few presentations related 
to specific experiments managed by Triumf. Each talk will bring a different aspect of the DAQ that 
Midas had to deal with. This will potentially be a good starting point for further discussions.

We will broadcast this seminar. Webcast information will be provided in a later message, preliminary 
date: 13 or 14 July. I would encourage you to participate in this event, if not in person, at least 
virtually. It is a good time for you to send to Stefan (midas@psi.ch) or myself (midas@triumf.ca), 
questions, requests, wishes, issues that you experience, general comment that has been on the back 
of your mind but you didn't manage to submit to us. This would help us to better understand how 
Midas is used, where it is used, and what can be addressed to better serve your needs as a user. 

Let us know how Midas is helping you, we would really appreciate it. Let us also know if you are 
thinking of attending this virtual seminar. 

If you happen to be in the Vancouver vicinity around the end of June, you are most welcome to join 
us at Triumf. The Midas team will take the time to chat about Data Acquisition and perhaps the 
benefit of our west coast weather!

Best Regards, Pierre-André Amaudruz
  1063   03 Jun 2015 Pierre-Andre AmaudruzForumMidas seminar
Dear Midas users, 

As promise, the first Midas seminar is happening.

Time    : July 15th 2015 from 12:15 to 16:00 PST.
Location: ISAC-II conference room at Triumf, Vancouver BC. Canada

The program is under construction, but it will consist of talks covering 
particular Midas implementation in different experiments such as SuperCDMS, DEAP, 
GRIFFIN, MEG-2.

Webcast information will be provided in early July, including link to the 
presentations.

If you're planning to attend this seminar remotely, please drop a quick note to 
me for a head count.

The Midas team is looking forward hearing from you.

Best Regards, Pierre-André Amaudruz
  1067   08 Jul 2015 Pierre-Andre AmaudruzForumMidas seminar
Dear Midas users,

For the upcoming "Midas Seminar" on the July 15th, you can find the necessary 
information here: https://indico.psi.ch/conferenceDisplay.py?confId=3793

The talks will be available for download prior the beginning of the seminar.


Cheers, PAA
  2348   23 Feb 2022 Stefan RittInfoMidas slow control event generation switched to 32-bit banks
The midas slow control system class drivers automatically read their equipment and generate events containing midas banks. So far these have been 16-bit banks using bk_init(). But now more and more experiments use large amount of channels, so the 16-bit address space is exceeded. Until last week, there was even no check that this happens, leading to unpredictable crashes.

Therefore I switched the bank generation in the drivers generic.cxx, hv.cxx and multi.cxx to 32-bit banks via bk_init32(). This should be in principle transparent, since the midas bank functions automatically detect the bank type during reading. But I thought I let everybody know just in case.

Stefan
  293   12 Aug 2006 Pierre-André AmaudruzReleaseMidas updates
Midas development:

Over the last 2 weeks (Jul26-Aug09), Stefan Ritt has been at Triumf for the "becoming" traditional Midas development 'brainstorming/hackathon' (every second year).

A list with action items has been setup combining the known problems and the wish list from several Midas users.
The online documentation has been updated to reflect the modifications.

Not all the points have been covered, as more points were added daily but the main issues that have been dealt or at least discussed are:

  • ODB over Frontend precedence.
    When starting a FE client, the equipment settings are taken from the ODB if this equipment already existed. This meant the ODB has precedence over the EQUIPEMENT structure and whatever change you apply to the C-Structure, it will NOT be taken in consideration until you clean (remove) the equipment tree in ODB.

  • Revived 64 bit support. This was required as more OS are already supporting such architecture. Originally Midas did support Alpha/OSF/1 which operated on 64 bit machine. This new code has been tested on SL4.2 with Dual-Core 64-bit AMD Opterons.

  • Multi-threading in Slow Control equipments.
    Check entry 289 in Midas Elog from Stefan.

  • mhttpd using external Elog.
    The standalone ELOG package can be coupled to an existing experiment and therefore supersede the internal elog functionality from mhttpd.
    This requires a particular configuration which is described in the documentation.

  • MySQL test in mlogger
    A reminder that mlogger can generate entries in a MySQL database as long as the pre-compilation flag -HAVE_MYSQL is enabled during system built. The access and form filling is then defined from the ODB under Logger/SQL once the logger is running, see documentation.

  • Directory destination for midas.log and odb dump files
    It is now possible to specify an individual directory to the default midas.log file as well as to the "ODB Dump file" destination. If either of these fields contains a preceding directory, it will take the string as an absolute path to the file.

  • User defined "event Data buffer size" (ODB)
    The event buffer size has been until now defined at the system level in midas.h. It is now possible to optimize the memory allocation specific to the event buffer with an entry in the ODB under /experiment, see documentation.

  • History group display
    It is now possible to display an individual group of history plots. No documentation on that topics as it should be self explanatory.

  • History export option
    From the History web page, it is possible to export to a ASCII .csv file the history content. This file can later be imported into excel for example. No documentation on that topics as it should be self explanatory.

  • Multiple "minor" corrections:
    - Alarm reset for multiple experiment (return directly to the experiment).
    - mdump -b option bug fixed.
    - Alarm evaluation function fixed.
    - mlogger/SQL boolean handling fixed.
    - bm_get_buffer_level() was returning a wrong value which has been fixed now.
    - Event buffer bug traced and exterminated (Thanks to Konstantin).
  1064   09 Jun 2015 Michael McEvoyForumMidas-MSCB SCS2000 integration
I am using the MSCB SCS2000 to monitor slow control variables (temperatures, voltages, etc). I am trying to 
get it set up at fermilab as a test stand in the MC1 building and was wondering if anyone has integrated 
Midas with a MSCB SCS2000 before. We have two systems at fermilab, one system that is currently running 
in the g-2 experimental hall, but running an out of date version of midas. The second test stand I am 
setting up is working with the current version of midas. I believe we will easily be able to figure out the 
external probes for temperatures and voltages just fine. But the MSCB SCS2000 box itself has 1 
temperature value, 1 current value, and 5 voltages internally that we also need to monitor. If I use the msc 
command I can read back the external values through the daughter cards I have installed on the SCS2000 
box but has no way of reading back the internal values that I need. I also have been looking through the 
MIDAS files trying to find a possible way to read these out to no avail.

If anyone has any ideas or has had previous work with the SCS2000 and knows how to read back the 
internal values please let me know. 

Thanks,

Michael McEvoy
NIU Graduate Student
ELOG V3.1.4-2e1708b5