ID |
Date |
Author |
Topic |
Subject |
518
|
28 Oct 2008 |
Stefan Ritt | Bug Report | strange output from "odbedit cleanup" | > When I run odbedit remotely (odbedit -h ladd09), the "cleanup" command unexpectedly produces the
> output of the "sor" command (sure enough, there is a call to db_get_open_records() there), but when I run
> it locally, I do not get this output (but db_get_open_records() is still called). Strange. K.O.
The db_get_open_records() call was by mistake there, I removed it. What remains is that the notification
message if a client is removed from the ODB goes through the system messages. When running locally, odbedit
echoes it's own messages, but when running remotely, this is not the case. So the messages can be seen by
everybody else (plus it ends up in the message file), but not by the remote odbedit where the cleanup is
started. The quick fix for that is to say "old" in odbedit which shows the last few lines of the message
file, so one can see any successful cleanup. |
519
|
28 Oct 2008 |
Stefan Ritt | Info | mscb timeouts and retries | > A new set of functions was added to mscb.h to adjust mscb timeouts and retries to better match specific
> applications:
>
> + int EXPRT mscb_get_max_retry();
> + int EXPRT mscb_set_max_retry(int max_retry);
> + int EXPRT mscb_get_usb_timeout();
> + int EXPRT mscb_set_usb_timeout(int timeout);
> + int EXPRT mscb_get_eth_max_retry();
> + int EXPRT mscb_set_eth_max_retry(int eth_max_retry);
In the spirit of this, a variable retry scheme has been implemented in the mscbdev.c device driver. At the
MEG experiment, we have one mscb device which is pretty slow, while the others are fast. Therefore it is
necessary to have a per-device max retry count which can be different for different submasters. I moved
therefore the max_eth_retry variable into the mscb_fd structure and adjusted a few functions accordingly. I
did not bother with the other timeouts and retries, since I don't need this for the moment, but it would be
nice if they would be handled in the same way. Then I added code into mscbdev.c to read the retry variable
form the ODB under /Equipment/<name>/Settings/Device/<Name>/Retries. The default is 10, but it can be
changed and becomes valid after the program has been restarted. |
520
|
29 Oct 2008 |
Stefan Ritt | Bug Report | custom web pages: customscript buttons and start/stop buttons generate errors | To fix this problem, do the following:
- Update to the current SVN revision 4368 of mhttpd.c
- Add following tag into your custom page:
<input type=hidden name="redir" value="name">
where "name" is the name of your custom page which follows the CS/ in the URL. Like
if you have a custom page which you access through httpd://localhost/CS/junk then the
tag would be
<input type=hidden name="redir" value="junk">
The "redir" parameter is now evaluated inside mhttpd and brings you back to the proper
custom page. You can also define another custom page as the target, if that makes
sense in your application.
Pierre: Would be nice to document this somewhere more officially. |
525
|
09 Nov 2008 |
Stefan Ritt | Bug Report | bool values in "/custom/images/my_image.gif/labels/src" seem to lose their format string | > Not sure if this is a bug or a feature:
>
> Writing a boolean label on an image seems to produce rather strange behaviour.
>
> For example,
> odb>ls /Equipment/gas/settings/my_bool -lt
> Key name Type #Val Size Last Opn Mode Value
> ---------------------------------------------------------------------------
> my_bool BOOL 1 4 14m 0 RWD y
>
>
> odb>cd /custom/images/my_image.gif/labels
> odb>ls
> Src /Equipment/gas/settings/my_bool
> Format val: %d (bool)
> Font Medium
> X 10
> Y 10
> Align 0
> FGColor FFFFFF
> BGColor FF8800
>
> Instead of the expected string "val: y (bool)", only the value of the key
> appears, i.e. "y".
> The behaviour is the same whether I use %d, %u, %s, %c etc as the format character.
That has been fixed in rev. 4379 |
526
|
09 Nov 2008 |
Stefan Ritt | Suggestion | <odb ... edit=1> buttons and javascript | > When writing custom webpages, it would be nice to be able to write code such as
>
> <odb src="/Equipment/TITAN_ACQ/ppg cycle/trans3/time offset (ms)" edit=1>
>
> from Javascript, e.g.
> <script type="text/javascript">
> if ( flag != 3)
> document.write('<odb src="/Equipment/TITAN_ACQ/ppg cycle/trans3/time offset
> (ms)" edit=1>ms');
> else
> document.write('<odb src="/Equipment/TITAN_ACQ/ppg cycle/trans4/time offset
> (ms)" edit=1>ms');
> </script>
>
> This is not translated correctly by mhttpd; the final quote and bracket get
> stripped off, and it gives Javascript error
>
> Error: unterminated string literal
> Source File: http://titan04:8089/CS/ppg_cycle?cmd=Edit&index=11
> Line: 477, Column: 18
> Source Code:
> document.write('<input type=text size=10 maxlength=80 name=value value="1">
>
> I can get round this by using an input box and a combination of ODBGet and
> ODBSet, but it would be easier if the edit=1 form above worked correctly, or
> there was a command like ODBSet that would accept input from the user.
>
> Thanks.
>
> would be nice is there was a command such as ODBGet or ODBSet that would work
> with javascript to
Actually that won't work, even if I would fix it. The <odb> tag is evaluated on the
server side (mhttpd), where is gets replaced by the actual ODB value. But if you
use JavaScript to generate the <odb> tag dynamically, this only happens on the
client side, so the server has no chance to substitute them. So you have to go with
ODBGet's I'm afraid. Nevertheless, I changed the code such that any ODB tags inside
a JavaScript is not interpreted by mhttpd. |
527
|
09 Nov 2008 |
Stefan Ritt | Bug Report | custom web pages: customscript buttons and start/stop buttons generate errors | > Thanks Stefan.
> Your fix works nicely with the start/stop buttons not returning to the same or to a
> different web page.
>
> However, it does not seem to have fixed the problem with the Customscript button. It does
> not seem to pick up the redirect, nor do the Pause/Resume buttons (which are programmed to
> appear when the run starts).
That has been fixed in rev. 4377 |
529
|
20 Nov 2008 |
Stefan Ritt | Info | Recommended platform for running MIDAS | > Dear All,
>
> Is there any recommended platforms for running MIDAS? Have anyone encountered
> problems when running MIDAS on Scientific Linux?
>
> Thanks.
>
> Jimmy
I run MIDAS on scientific Linux 5.1 without any problem. |
531
|
26 Nov 2008 |
Stefan Ritt | Info | Send email alert in alarm system | > We have a temperature/humidity sensor in MIDAS now and will add a liquid level
> sensor to MIDAS soon. We want the operators to get alerted ASAP when the
> laboratory environment or the liquid level reached some critical levels. Can
> MIDAS send email alerts or SMS alerts to cell phones when the alarms are
> triggered? If yes, how can I config it?
Sure that's possible, that's why MIDAS contains an alarm system. To use it, define
an ODB alarm on your liquid level, like
/Alarms/Alarms/Liquid Level
Active y
Triggered 0 (0x0)
Type 3 (0x3)
Check interval 60 (0x3C)
Checked last 1227690148 (0x492D10A4)
Time triggered first (empty)
Time triggered last (empty)
Condition /Equipment/Environment/Variables/Input[0] < 10
Alarm Class Level Alarm
Alarm Message Liquid Level is only %s
The Condition if course might be different in your case, just select the correct
variable from your equipment. In this case, the alarm triggers an alarm of class
"Level Alarm". Now you define this alarm class:
/Alarms/Classes/Level Alarm
Write system message y
Write Elog message n
System message interval 600 (0x258)
System message last 0 (0x0)
Execute command /home/midas/level_alarm '%s'
Execute interval 1800 (0x708)
Execute last 0 (0x0)
Stop run n
Display BGColor red
Display FGColor black
The key here is to call a script "level_alarm", which can send emails. Use
something like:
#/bin/csh
echo $1 | mail -s \"Level Alarm\" your.name@domain.edu
odbedit -c 'msg 2 level_alarm \"Alarm was sent to your.name@domain.edu\"'
The second command just generates a midas system message for confirmation. Most
cell phones (depends on the provider) have an email address. If you send an email
there, it gets translated into a SMS message.
The script file above can of course be more complicated. We use a perl script
which parses an address list, so everyone can register by adding his/her email
address to that list. The script collects also some other slow control variables
(like pressure, temperature) and combines this into the SMS message.
For very sensitive systems, having an alarm via SMS is not everything, since the
alarm system could be down (computer crash or whatever). In this case we use
'negative alarms' or however you might call it. The system sends every 30 minutes
an SMS with the current levels etc. If the SMS is missing for some time, it might
be an indication that something in the midas system is wrong and one can go there
and investigate. |
536
|
01 Dec 2008 |
Stefan Ritt | Bug Fix | Fix ss_file_size() on 32-bit Linux | > I also changed ss_file_size(), ss_disk_size() and ss_disk_free() to return -1 if
> the system call returns an error. I also added a test program
> utils/test_ss_file_size.c.
The test program gave under 64-bit SL5:
For [(null)], file size: -1, disk size: -0.001, disk free -0.001
sh: -c: line 0: syntax error near unexpected token `('
sh: -c: line 0: `/bin/ls -ld (null)'
sh: -c: line 0: syntax error near unexpected token `('
sh: -c: line 0: `/bin/df -k (null)'
Anyhow I guess that this test program just accidentally slipped into the repository.
Test programs for the developers should not be in the repository since they are of
not much use for the average user. If I would have added every test I made as an
individual test program, we would by now have tons of test programs making the whole
distribution pretty bulky, which nobody would know how to use now. So I removed the
test program again. If people do not agree, I suggest to make a central "main" test
program which combines all tests. I know there are also some C structure alignment
tests etc., which then could all be combined into a single, well documented, test
program. |
538
|
02 Dec 2008 |
Stefan Ritt | Bug Fix | Fix ss_file_size() on 32-bit Linux | > I now fixed this problem by using the stat64() system call for "#ifdef OS_LINUX".
That does not work if _LARGEFILE64_SOURCE is not defined. In that case, the compiler
complains that stat64 is undefined. Since many Makefiles for front-ends out there do
not have _LARGEFILE64_SOURCE defined, I changed system.c so that stat64 is only used
if that flag is defined:
#ifdef _LARGEFILE64_SOURE
struct stat64 stat_buf;
int status;
/* allocate buffer with file size */
status = stat64(path, &stat_buf);
if (status != 0)
return -1;
return (double) stat_buf.st_size;
#else
... |
540
|
02 Dec 2008 |
Stefan Ritt | Bug Fix | Fix ss_file_size() on 32-bit Linux |
K.O. wrote: | This does not work (observe the typoe in the #ifdef). |
Sorry for that, I fixed and committed it.
K.O. wrote: | But you cannot know this because you already deleted the test program I wrote and committed to svn exactly to detect and prevent this kind of breakage (+ plus to give the Solaris, BSD and other wierdo users some way to check that ss_file_size() works on their systems).. |
Well, you figured it out even without the test program in the distribution! But I'm sure no other user would have known how to use your test program to diagnose this problem. So 99% of the users would scratch their head about this undocumented program and get confused. I believe we two are responsible that the midas kernel functions work correctly and the average user should not have to bother with it. I agree that it's handy for you to have this little test program in the distribution, so you can run it everywhere you install midas. But for me it would be handy to have files with, let's say, nature's constants, particle decay life times, list of ASCII codes, and so on. But it would clutter up the distribution and the disadvantage of annoying users would be bigger than my personal benefit, so I don't do it.
If you absolutely want to keep a certain test functionality, you can add it into a "central" test program, write some help and documentation for it, educate users how to use it and how to report any errors back to you. Maybe some printout like "all tests ok" and some specific comment if a test fails would be helpful for the normal user. This test program could then also contain other tests like C structure alignment (which sometimes is a problem), some mutex tests and whatever we collected along the road. An alternative would be to add this into a "test" command inside odbedit. |
542
|
14 Dec 2008 |
Stefan Ritt | Info | Custom page which executes custom function | > How can I add a button at the top of the "Status" webpage which will show a
> page similar to the "CNAF" one after I click on it? and how can I make a
> custom page similar to "CNAF" which allow me to call some custom funtions? I
> want to make a page which is particularly for doing calibration.
The CNAF page calls directly functions through the RPC layer of midas, which is
not possible from custom pages. All you can do is to execute a scrip on the
server side, which then causes some action. For details please consult the
documentation. |
545
|
22 Dec 2008 |
Stefan Ritt | Bug Report | Overflow on "cm_msg" command generates segfault | > The following error has been reported to me by T2K colleagues:
>
> When using "odbedit -c "msg my_message", the following behavior
> has been observed depending on the length "n" of the message.
>
> 1) n < 100 All is well
> 2) 100 <= n < 245 Log not written but exit code = 0
> 3) 245 <= n < 280 Error: "Experiment not defined" and exit code = 1
> 4) 280 <= n Error: "Cannot connect to remote host" and exit code = 1
>
> Also, when logging from compiled C code - when messages reach some magic length
> the MIDAS client sending them segfaults.
>
> Please fix
Uhhh, who wants this long messages? You should consider to split this into several
smaller messages. Anyhow, having the above behavior is not good, so I fixed it in
SVN revision 4422. I increased the maximum length to 1000 characters. Above that,
the message gets truncated. If you need even more, we can make it a #define.
The second problem you describe (logging from compiled C code) I could not
reproduce, so maybe it was related to the first one. Please try again and report
if it persists. |
549
|
13 Jan 2009 |
Stefan Ritt | Forum | mlogger problem | > Hi,
>
> I am running Scientific Linux with kernel 2.6.9-34.EL and I have
> glibc-2.3.4-2.25. When I run mlogger, I receive the error:
>
> *** glibc detected *** free(): invalid pointer: 0x0073e93e ***
> Aborted
>
> Any ideas?
Not much. Try to clean up the ODB (delete the .ODB.SHM file, remove all shared
memory via ipcrm) and run again. I run under kernel 2.6.18 and glibc 2.5 and this
problem does not occur. If you cannot fix it, try to run mlogger inside gdb and
make a stack trace to see who called the free(). |
551
|
13 Jan 2009 |
Stefan Ritt | Forum | mlogger problem | > Sorry for being vague. I cleaned up the ODB, but it doesn't seem to be the
> problem. Here is a sample run of mlogger and gdb:
Thanks for the info, that explained the problem. It is related to the lines
rargv[rargc] = (char *)malloc(3);
rargv[rargc++] = "-b";
where one first allocates some memory (3 bytes), but then overwrites the pointer with
another pointer to some static memory ("-b"). The following
free(rargv[1]);
then tries to free the static memory which fails.
The problem was already fixed some time ago, so please update your version from the SVN
revision (see https://midas.psi.ch/download.html for details). |
552
|
13 Jan 2009 |
Stefan Ritt | Info | Custom page which executes custom function | The UDP connection you mention is only used locally for inter-process communication. When I implemented that, I
made extensive tests and found that there is never a packet being dropped. This happens for UDP only if the packet
goes over a physical network. Maybe this is different in modern Linux versions, so one should double check this
again.
For remote hot-link notification, the notification is sent over the TCP link, so it should not be lost either. But
your second point is correct. The hot-link mechanism was developed to change parameters in front-end programs for
example. So by design it is guaranteed that if you change a value in the ODB, any client hot-linked to that will
see the change (sooner or later). If there are many changes in short intervals (or the callback function on the
remote client takes long time), only the last change is guaranteed to arrive. Therefore, as you correctly state,
the hot-link mechanism is not a save replacement for the RPC layer (That's why the RPC layer is there after all). |
553
|
14 Jan 2009 |
Stefan Ritt | Info | odb "hot link" magic explored |
KO wrote: | note 1: I do not completely understand the ss_suspend_xxx() stuff. The best I can tell is it creates a number of udp sockets bound to the local host and at least one udp rpc receive socket ultimately connected to the cm_dispatch_rpc() function. |
The ss_suspend_xxx() stuff is indeed the most complicated thing in midas an I have to remind myself always
on how this works. So let me try again:
The basic idea is that for a high performance system, you cannot do the inter-process communication via
polling. That would waste CPU time. Inter-process communication is necessary for for buffer manager
(producer notifies consumer when new events are there), for the RPC mechanism (odbedit tells mlogger to
start a run) or for ODB hot-links. To avoid polling, the inter-process communication works with sockets (UDP
and TCP). This allows to use the select() call, which suspends the calling process until some socket
receives data or a pre-defined time-out expires. This is the only portable method I found which works under
unix and windows (signals are only poorly supported under windows).
So after creating all sockets, ss_suspend() does a select() on these sockets:
_suspend_struct[idx].listen_socket | Server side for any new RPC connection (each client is also a RPC server which gets contacted directly during run transitions for example
|
_suspend_struct[idx].server_acception.recv_sock | Receive socket (TCP) for any active RPC connection
|
_suspend_struct[idx].server_acception.event_sock | Receive socket (TCP) for bare events (bypassing RPC layer for performance reasons)
|
_suspend_struct[idx].server_connection->recv_sock | Outgoing TCP connection to mserver. Used for example for hot-link notifications from mserver
|
_suspend_struct[idx].ipc_recv_socket | UDP socket for inter-process notification
|
For each socket there is a dispatch function, which gets called if that socket receives some data. Hope this sheds some light on the guts of that. |
556
|
20 Jan 2009 |
Stefan Ritt | Info | Subrun scheme implemented | A new "subrun" scheme has been implemented in mlogger to split a big data file into several individual data files. This feature might be helpful if a data file from a single run gets too large (>4 GB for example) and if shorter runs are not wanted for efficiency reasons. The scheme works as follows:
Each subrun will contain an ODB dump if this is turned on via /Channels/x/Settings/ODB dump. The stopping of the "main" run (after four subruns in the above example) can be done in the usual way (event limit in the front-end, manually through odbedit, etc.).
The code has been tested in two test environments, but not yet in a real experiment. So please test it before going into production. The modification in mlogger requires SVN revision 4440 of mlogger.c and 4441 of odb.c.
Please note that the lazylogger cannot be used with this scheme at the moment since it does not recognize the subruns. That will be fixed in a future version and announced in this forum.
- Stefan |
559
|
25 Jan 2009 |
Stefan Ritt | Info | Subrun scheme implemented |
Renee Poutissou wrote: | I have tested the new subrun functionality a bit more and I have two observations. First, it seems to work on a basic level, i.e. subruns are created, which are equal in size. However, I can't relate their size to the byte limit set in the ODB. |
What you describe is expected. The logger process maintains a write cache, which is 32 kB under linux and 1 MB under Windows. The size is controlled through the constant TAPE_BUFFER_SIZE defined in midas.h. The reason for this buffer is to optimize writes to disks and tapes and has been carefully optimized to give maximum performance. It means however that data gets written only in 32 kB chunks to disk. That's the reason why your run size is 32kB plus a few bytes. You can change this by modifying TAPE_BUFFER_SIZE, but be aware that this will then slow down your logging of data. |
564
|
24 Mar 2009 |
Stefan Ritt | Forum | Analyzer gets killed cm_watchdog | Hi,
your log script sound to me like the analyzer either got into an infinite loop or
did a segment violation and just died. I would recommend to run the analyzer from
inside the debugger. When you then get the segment violation, you can inspect the
stack trace and see where the bad things happen. Since the analyzer works nicely in
other experiment, I expect that your problem is related to the user code. Maybe it
happens at the end of the run, but there is a timeout before the crashed process
gets cleaned from the ODB, that's why you might think that it happens "between"
runs.
Best regards,
Stefan
>
> Hello Midas experts:
>
> We have setup a DAQ using MIDAS to readout two ADCs in the crate.
> We are running into problem of analyzer getting killed between
> runs. Sometimes it would crash after a few runs and sometimes it
> would go on for many many runs before analyzer gets killed. It always
> occurred between runs not when we are taking data. Any suggestions
> on what we could try? The error message from the midas.log file is
> appended below.
>
> Thanks,
>
> Dawei
>
> Wed Mar 4 11:53:11 2009 [Analyzer,ERROR] [midas.c:1739:,ERROR]
> cm_disconnect_experiment not called at end of program
> Wed Mar 4 11:53:22 2009 [mhttpd,INFO] Client 'Analyzer' on buffer 'SYSMSG'
> removed by cm_watchdog (idle 10.7s,TO 10s)
> Wed Mar 4 11:53:22 2009 [mhttpd,INFO] Client 'Analyzer' (PID 1) on buffer 'ODB'
> removed by cm_watchdog (idle 10.7s,TO 10s)
> Wed Mar 4 11:53:22 2009 [AL Experiment Frontend,INFO] Client 'Analyzer' on
> buffer 'SYSTEM' removed by cm_watchdog (idle 10.9s,TO 10s)
> Wed Mar 4 11:53:29 2009 [AL Experiment Frontend,TALK] starting new run
> Wed Mar 4 11:53:29 2009 [AL Experiment Frontend,ERROR]
> [midas.c:8264:rpc_client_check,ERROR] Connection broken to "Analyzer" on host
> tsunami |
|