ID |
Date |
Author |
Topic |
Subject |
259
|
25 May 2006 |
Stefan Ritt | Bug Fix | Fixed compiler warnings with gcc 3.4.4 | I fixed a couple of compiler warning which came up with the new gcc 3.4.4. Seems like the compiler gets more and more picky. There a still warning left in ybos.c and in mcnaf.c, which I leave to the original author  |
267
|
09 Jun 2006 |
Stefan Ritt | Bug Fix | fix compilation of musbstd.h, add it back to libmidas | > I fixed the compilation of musbstd.h (it required -DHAVE_LIBUSB on Linux, but
> nothing knew about defining it) and put musbstd.o back into libmidas (USB
> support should be part of the standard base midas library). K.O.
I'm not so sure about that. One could consider musbstd.o as a driver, and the
philosophy used for midas programs is that drivers get added explicitly when
compiling a frontend. We do not put mvmestd.c and mcstd.c into libmidas since for
different interfaces a different driver might be required. If we at some point use
an usb library different than libusb.a, we would have to compile different
libmidas for these different drivers.
I know it's convenient to have things in libmidas and not having to specify it
expliceitely for each frontend, but it is then somehow inconsistent with drivers
for vme and camac. So please reconsider this again.
- Stefan |
268
|
13 Jun 2006 |
Stefan Ritt | Info | ZLIB dependency modified | Due to recent problems with the ROME analyzer having zlib.h both in the
system and in the midas tree it has been decided to change the zlib policy in midas. By default, zlib support is not included in the midas analyzer. If one want it (but I guess only very few experiments need that), one can do a
make NEED_ZLIB=1
to compile zlib support into mana.c
Under linux (&Co), the zlib is these days normally pre-installed. The header file will therefor be taken from /usr/include and the library from /usr/lib/libz.a. Under Windows, the zlib is still included in the distribution, and has to be manually added to the Visual C++ project file. |
269
|
13 Jun 2006 |
Stefan Ritt | Info | Scheduler changed for slow control equipment | The schedule in mfe.c is used both for "normal" front-ends and for "slow-control" front-ends. Unfortunately it was only optimized for the first class. This lead to the fact that the slow control equipment was read out at different speed depending if the run is started or not. Furthermore, the maximum readout speed was somehow limited. This has been changed in the current version of mfe.c (SVN revision 3146). There are now two ways to control the readout speed of slow control equipment:
1) The "event limit" in the equipment list can be used as minimum time between readouts. I'm not happy about the "mis-use" of this variable, but it has been there since the beginning. If I would change it now, all front-ends on this world would have to be changed, which I maybe not a good idea. If this event limit is set to let's say 10, then the slow control equipment is read out with a maximum speed of 1/10ms = 100Hz. That means up to 100 variables (not complete equipments) are read out per second. If an equipment has 200 variables, each variable is then read out every two seconds of course. This number can be used to limit the readout speed differently for different equipments. Like one might want to read a sensitive pressure as often as possible, but some beamline magnet values only once every minute.
2) By default, the scheduler runs now at "full speed" when slow control equipment is present, resulting in a 100% CPU usage. To avoid this, following code can be added into the frontend_loop function:
BOOL frontend_call_loop = TRUE;
INT frontend_loop()
{
/* don't eat up all CPU time */
return cm_yield(10);
}
This limits the readout speed of all slow control equipment again to 100Hz, but avoids the 100% CPU usage. On most operating systems, the minimum time is 10ms as shown above, since this is the basic time slice of a process.
The readout scheme of slow control equipment will be re-visited this summer, when multi-threaded slow control front-ends will be implemented. |
272
|
23 Jul 2006 |
Stefan Ritt | Forum | File output for histories |
Art Olin wrote: | Basically we need the output from the mhist code. The most convenient, and possibly easiest implementation would be to select required data (ID, variable, time range) in the midas history display, click a button requesting file output and input a file name. One might also want to specify the interval time required. |
So what is wrong with using mhist directly? I understand that some users used to point and click might have hard time to start a command line utility, but I'm sure that I teach anybody to use mhist much faster compared to the time I would have to spend on implementing such a feature in the web interface. Well, I'll keep it in mind, but it has low priority.
Art Olin wrote: | A related nice feature would be like the root "view event status" , where text at the bottom of the history would display the position of the cursor in the history chart coordinates. Probably more work and less important to us. |
Well if you teach me how to do this I'm happy to implement it. We are in a browser, and the history plot is a dump GIF image, while the ROOT windows is a native application. One would have to use some fance Javascript to implement such a thing, but I have no clue of how to do that.
- Stefan |
276
|
24 Jul 2006 |
Stefan Ritt | Bug Report | Elog attachments |
Art Olin wrote: | Hi. When I attach the file below, Mix+Positronorig.xlx to an elog, and then open it or download it to disk, the file, 060... is severely truncated.
-rw-r--r-- 1 alpha users 17408 Jul 24 11:25 Mix+Positronorig.xls
-rw-r--r-- 1 alpha users 1 Jul 24 11:04 060724_100544_Mix+Positron Cabling 20060723.xls
It's something to do with long filenames or special characters in filenames. Worked OK when I renamed the original file to M1.xls. |
You should not use "+" in a file name for elog. |
279
|
27 Jul 2006 |
Stefan Ritt | Bug Report | MIDAS revision 3184 bombs on FC5 |
Shawn Bishop wrote: | include/musbstd.h:29:17: error: usb.h: No such file or directory |
This indicates that you are missing libusb. If you can find a RPM for libusb, that will solve your problem. But anyhow we should modify the makefile such that it does not try to compile the USB drivers if libusb is missing on a system. |
285
|
03 Aug 2006 |
Stefan Ritt | Bug Report | MIDAS packaged examples: compilation bug? |
Shawn Bishop wrote: | Anyone have an idea what's going on here? |
The Makefile contained the outdated target fal, which is a combined frontend/analyzer/logger. You don't need that, so I removed it from the makefile. Now it should compile fine. |
289
|
07 Aug 2006 |
Stefan Ritt | Info | New multi-threaded midas slow control system | Multi-threaded slow control system
The Midas slow control system has been modified to support multi-threaded slow control front-ends. Each device gets it's own thread in the front-end, which has several advantages:
- the communication of all devices runs in parallel and therefor is much faster
- slow devices cannot block any more the front-end. Response times to run transitions etc. become therefore much faster.
This modification requires some minor modifications in the existing class and device drivers.
Dropping of CMD_xxx_ALL commands
The slow control commands CMD_SET_ALL, CMD_GET_ALL, CMD_SET_CURRENT_LIMIT_ALL, CMD_GET_CURRENT_LIMIT_ALL, etc. have been dropped. They were there to accomodate some slow devices, which sometimes works a bit faster if all channels are set or read at once. Since the inter-thread communication scheme implemented now does only allow passing one channel at a time, the "ALL" functions cannot be supported any more. On the other hand this is not such an issue any more, since slow devices are handled now in parallel, speeding up things considereably.
The command have been removed from midas.h and from all device and class drivers coming with the midas distribution. If you have your own drivers, just delete the sections wich use these commands.
Calling the device driver inside the class driver
The device drivers have now to be called differently in the class driver. The reason for that is that in a multi-threaded front-end, there is only one central device driver dispatcher, which communicates with the individual device driver threads. The device drivers do not need to be modified, but all existing class drivers need modification, if they are going to be run in a multi-threaded front-end. Old class drivers which are not used in a multi-threaded front-end do not to be modified.
Following modifications are necessary:
- Remove following line:
#define DRIVER(_i) ...
- Find all lines containing
DRIVER(i)(CMD_xxx, info->dd_info[i], ...)
and replace them with
device_driver(info->driver[i], CMD_xxx, ...)
note that info->dd_info[i] is not passed any more. Instead, you pass info->driver[i]. Pleae note that the arguments passed after CMD_xxx are not checked by the compiler, since they are a variable argument list. Any error there will not produce a compiler warning, but will just crash the front-end.
- Find the line with
status = pequipment->driver[i].dd(CMD_INIT, hKey, &pequipment->driver[i].dd_info,
pequipment->driver[i].channels,
pequipment->driver[i].flags,
pequipment->driver[i].bd);
and replace it with
status = device_driver(&pequipment->driver[i], CMD_INIT, hKey);
- Find the line with
pequipment->driver[i].dd(CMD_EXIT, pequipment->driver[i].dd_info);
and replace it with
device_driver(&pequipment->driver[i], CMD_EXIT);
- Find following lines
hv_info->driver[i] = pequipment->driver[index].dd;
hv_info->dd_info[i] = pequipment->driver[index].dd_info;
hv_info->channel_offset[i] = offset;
hv_info->flags[i] = pequipment->driver[index].flags;
and replace them with
hv_info->driver[i] = &pequipment->driver[index];
hv_info->channel_offset[i] = offset;
The class drivers multi.c and generic.c can be used as a reference for these modifications.
Implementing CMD_STOP command
For multithread-enabled device drivers it is necessary to support the CMD_STOP command, which is needed to stop all device threads before the actual device gets closed. Following code is necessary:
INT cd_xxx(INT cmd, EQUIPMENT * pequipment)
{
INT i, status;
switch (cmd) {
case CMD_INIT:
...
case CMD_STOP:
for (i = 0; pequipment->driver[i].dd != NULL &&
pequipment->driver[i].flags & DF_MULTITHREAD ; i++)
status = device_driver(&pequipment->driver[i], CMD_STOP);
break;
case CMD_IDLE:
...
return status;
}
Enabling multi-thread support
To turn on multi-thread support for a device, the flag DF_MULTITHREAD must be used in the front-end user code device driver list, such as
DEVICE_DRIVER multi_driver[] = {
{"Input", nulldev, 2, null, DF_INPUT | DF_MULTITHREAD},
{"Output", nulldev, 2, null, DF_OUTPUT | DF_MULTITHREAD},
{""}
}; |
295
|
17 Aug 2006 |
Stefan Ritt | Bug Report | "double" values are truncated | > The mhttpd ODB displays and mhist truncate values of "float" and "double"
> floating point variables to 6 digits. In reality, "float" has 7 significant
> digits and "double" has 16. I recommend that db_sprintf() in odb.c be changed to
> read this:
>
> case TID_FLOAT:
> sprintf(string, "%.7g", *(((float *) data) + index));
> break;
> case TID_DOUBLE:
> sprintf(string, "%.16g", *(((double *) data) + index));
> break;
>
> K.O.
I had there
case TID_FLOAT:
if (ss_isnan(*(((float *) data) + index)))
sprintf(string, "NAN");
else
sprintf(string, "%g", *(((float *) data) + index));
break;
case TID_DOUBLE:
if (ss_isnan(*(((double *) data) + index)))
sprintf(string, "NAN");
else
sprintf(string, "%lg", *(((double *) data) + index));
break;
so I assumed that "%g" takes care of the maximal resolution. But apparently it does
not. So I changed it as you proposed. |
302
|
20 Sep 2006 |
Stefan Ritt | Suggestion | Increase of maximum event size | Dear midas users,
The current event size in midas is limited to 512k (MAX_EVENT_SIZE in midas.h). This is mainly due to old (pre 2.2) linux kernels which had only a very limited shared memory pool. These days this limit has increased considerably and I question if we should increase the default event size and to which size we should increase it.
The drawback of a larger event size is that the SYSTEM event buffer has to hold at least two events, and when the last midas program is stopped or started, this buffer has to be written to or read from the .SYSTEM.SHM file, which slows down the start/stop of the program. But writing/reading a few MB is fast these days anyhow so this again might now be a big problem. So what do you think how big we should make the default max event size?
- Stefan |
303
|
20 Sep 2006 |
Stefan Ritt | Suggestion | Increase of maximum event size | Since nobody complained so far, I increased MAX_EVENT_SIZE to 2MB. If anybody has problems with this setting, please report. Note that after updating to SVN revision 3327 it will be necessary to recompile all midas programs and to delete any old SYSTEM.SHM or .SYSTEM.SHM. I added some code which should check for inconsistent SYSTEM.SHM sizes, but I'm not sure if it works everywhere. |
306
|
24 Sep 2006 |
Stefan Ritt | Bug Report | mhttpd elog corruption via double-edit |
K.O. wrote: | Aparently the mhttpd elog will corrupt the elog files if two (or more\?) elog entries are being edited at the same time. K.O. |
That's strange. Since mhttpd is single threaded, there should not be any multi-thread/process conflict there, since the elog files cannot be written simultaneously from two different browser sessions. If entries are edited at the same time, they get then submitted one after the other. Of course it is possible to edit the same entry, in which case the second submission "wins", overwriting the first one without notification. Withing the standalone elog server there is the option to lock entries ("use lock = 1") to prevent this, but this feature is not present in the mhttpd elog. |
309
|
28 Sep 2006 |
Stefan Ritt | Suggestion | Increase of maximum event size |
K.O. wrote: | Now, we have per-buffer tunable size (see message
https://ladd00.triumf.ca/elog/Midas/283) and in the long run, I would prefer the
compiled-in limit to go away: already all memory is allocated dynamically and
the MAX_EVENT_SIZE is only useful as kind of a sanity check against frontend
misconfiguration or against malformed events.
If MAX_EVENT_SIZE goes away, the maximum event size becomes limited by the
largest SysV shared memory segment permitted by Linux (via sysctl kernel.shmmax).
To go beyound the limit on SysV shared memories, on can use mmap() based shared
memory: this is limited by available RAM+swap (and disk space for the
.SYSTEM.SHM file). Current MIDAS system.c has an experimental implementation of
mmap() shared memory, but AFAIK it has not been used in any production system, yet. |
MAX_EVENT_SIZE is also used for the RPC layer, since the receiving buffer must hold at
least one event. It is right that this can and should be made dynamically. Concerning
the shared memory there is the problem that it cannot be increased when any program is
running and attached to the shared memory, so it can only be defined at startup of the
first program creating the shared memory.
The sanity check in the frontend is done against max_event_size defined in frontend.c which can be smaller than MAX_EVENT_SIZE (some front-ends have limited memory).
So I agree that this issue may need revision, maybe something for me next visit  |
310
|
28 Sep 2006 |
Stefan Ritt | Bug Report | mhttpd elog corruption via double-edit | > I do not know how to "properly" fix this bug without changing the indexing
> scheme to something similar to what is used by elogd- message numbers instead of
> file indices. In the existing scheme, message editing also breaks URLs shown in
> the email notifications (they contain file indices that point to the wrong
> places after messages are moved around by editing) and "reply threading" links.
Well, the development of elogd with it's message numbers was actually stimulated by
the problem you mentioned. After that all those problems went away. Another
incarnation of that problem is if you edit an mhttpd log file manually. Afterwards
the file offsets are different and the system gets corrupted. To fix this properly,
one would have to backport the el_xxx functions from elogd to mhttpd, or, even
simpler, remove the elog functionality in mhttpd and "force" everybody to use elogd
(after doing elconv to convert the files into the new format). |
313
|
16 Oct 2006 |
Stefan Ritt | Bug Fix | "make install" error on MacOS 10.4.7, svn 3366 | > While executing "make install" under MacOS 10.4.7, you may encounter errors about "dio". It is the
> problem of "Makefile". I did some change to it and attach the diff file here.
I committed your patch. Thank you. |
314
|
16 Oct 2006 |
Stefan Ritt | Bug Fix | Build error with mana.c while using CERNLIB, svn 3366 | Committed, thanks. |
318
|
08 Jan 2007 |
Stefan Ritt | Suggestion | Access to out_info from mana.c | I changed out_info into a global structure definition ANA_OUTPUT_INFO and put it into
midas.h, so it can be accessed easily from the user analyzer source code.
> Would it be relevant to transform out_info into a *non-static* variable of a type
> defined by a *named* struct?
> Currently, programs that try to access out_info cannot do it anymore; and they
> typically copy the struct definition from mana.c, which is not robust against future
> changes in mana.c.
>
> If mana.c could be changed in the way described above, that would be great .
> Otherwise, is it safe to patch it myself for local use? or is there a better way of
> accessing out_info from mana.c?
>
> As always, any help would be much appreciated :)
>
> EOL
>
> > Hello,
> >
> > Is it possible to access out_info (defined in mana.c) from another program?
> >
> > In fact, out_info is now defined as an (anonymous) "static struct" in mana.c,
> > which it seems to me precludes any direct use in another program. Is there an
> > indirect way of getting ahold of out_info? or of the information it contains?
> >
> > out_info used to be defined as a *non-static* struct, and the code I'm currently
> > modifying used to compile seamlessly: it now stops the compilation during
> > linking time, as out_info is now static and the program I have to compile
> > contains an "extern struct {} out_info".
> >
> > Any help would be much appreciated! I searched in vain in this forum for
> > details about out_info and I really need to access the information it contains!
> >
> > EOL (a pure MIDAS novice) |
320
|
11 Jan 2007 |
Stefan Ritt | Forum | Shared memory problems | > Hello,
>
> Just did a fresh install of MIDAS from the SVN repository under CentOS and
> everything compiles fine, but when I go to run the frontend (using dio), I get
> the following error message:
>
> Connect to experiment ...[odb.c:868:db_open_database] Different database format:
> Shared memory is 14, program is 2
> [midas.c:1763:cm_connect_experiment1] cannot open database
>
>
> Any ideas on what the problem could be, or how to fix it?
You have an old .ODB.SHM from a previous version in your directoy (note the '.' in
front, so you need a 'ls -alg' to see it). Delete that file and try again. |
322
|
11 Jan 2007 |
Stefan Ritt | Forum | Shared memory problems | That sounds like you mix versions: You have an old executable (maybe your mlogger) which
has been linked against the old midas version, but you create the ODB with the new
odbedit or frontend. The new version complains if it finds an ODB from a previous version
(the error you reported first), but an old program does not have that version check, so
it finds a different binary ODB structure and crashes.
> Thanks for your help. I tried again and it got me back to the initial problem I had.
> The frontend will start, and the analyzer starts (complains about there not being a
> last.root, but other than that it's fine), and then when starting mlogger, I get:
>
> [odb.c:860:db_validate_db] Warning: database corruption, first_free_key 0x0001A4
> 04
> [odb.c:3666:db_get_key] invalid key handle
> [midas.c:1970:cm_check_client] cannot delete client info
> [odb.c:3666:db_get_key] invalid key handle
> [midas.c:1970:cm_check_client] cannot delete client info
> [odb.c:3666:db_get_key] invalid key handle
>
>
> And it continues to shoot out error messages about invalid key handles until I kill
> it. Then trying to start the frontend again fails until I remove the .ODB.SHM file.
> Any other ideas?
>
> > > Hello,
> > >
> > > Just did a fresh install of MIDAS from the SVN repository under CentOS and
> > > everything compiles fine, but when I go to run the frontend (using dio), I get
> > > the following error message:
> > >
> > > Connect to experiment ...[odb.c:868:db_open_database] Different database format:
> > > Shared memory is 14, program is 2
> > > [midas.c:1763:cm_connect_experiment1] cannot open database
> > >
> > >
> > > Any ideas on what the problem could be, or how to fix it?
> >
> > You have an old .ODB.SHM from a previous version in your directoy (note the '.' in
> > front, so you need a 'ls -alg' to see it). Delete that file and try again. |
|