int mvme_open(MVME_INTERFACE **vme, int idx)

   /* open VME */
   sprintf(str, "/dev/sis1100_%02dremote", idx);
   (*vme)->handle = open(str, O_RDWR, 0);
   if ((*vme)->handle < 0)
      return MVME_NO_INTERFACE;
   }

> - cm_transition() timeouts now settable from ODB (/experiment/transition timeout, transition connect timeout). Rev 4479

transition connect timeout was actually only half of that specified because of an error in computing timeout arguments to the select() system 
call in recv_string() in system.c. This is now fixed.

rev 4488
K.O.

> - cm_transition() timeouts now settable from ODB (/experiment/transition timeout, transition connect timeout). Rev 4479

transition connect timeout was actually only half of that specified because of an error in computing timeout arguments to the select() system 
call in recv_string() in system.c. This is now fixed.

rev 4488
K.O.

We have seen on several daq systems this problem: we start a run and observe that the number of 
events written by mlogger to the output file is double the number of events actually collected. Upon 
inspection of the output file, we see that every event is written twice. Restarting the run usually fixes 
this problem.

We now traced this to an error in mlogger.c. If we start a run and the run transition fails in some 
frontend,  mlogger does not disconnect from the SYSTEM buffer (it does not know the transition failed 
and the run did not really start). The SYSTEM buffer connection and the associated event request 
remain active. Then we start the next run and mlogger connects to the SYSTEM buffer again, creates a 
second (third, etc) event request. Eventually mlogger reaches the maximum permitted number of event 
requests and no more runs can be started unless mlogger is restarted.

If at some point a run actually starts successfully, there are multiple event requests present from 
mlogger and theoretically, each event should be written to the output file many times. This was a 
puzzle until we got a good laugh from looking at mlogger.c::receive_event() callback - in retrospect it 
is obvious why events are only written in duplicate.

Then, after the run is ended, mlogger disconnects from the SYSTEM buffer, all multiple event requests 
are automatically deleted and the problem is not present during the next run.

I am not yet sure how to best fix this, but I see that other midas programs (i.e. mevb) suffer form the 
same problem - multiple connections to the event buffer - in presence of failed run starts. I think we 
have seen "event duplication" from mevb, as well.

K.O.

When using "/bin/bash" shell, if I exit odbedit (and other midas programs) using ctrl-C, the terminal 
enters a funny state, "echo" is turned off (I cannot see what I type), "delete" key does not work (echoes 
^H instead).

This problem does not happen if I exit using the "exit" command or if I use the "/bin/tcsh" shell.

When this happens, the terminal can be restored to close to normal state using "stty sane", and "stty 
erase ^H".

The terminal is set into this funny state by system.c::getchar() and normal settings are never restored 
unless the midas program calls getchar(1) at the end. If the program does not finish normally, original 
terminal settings are never restored and the terminal is left in a funny state.

It is not clear why the problem does not happen with /bin/tcsh - perhaps they restore sane terminal 
settings automatically for us.
K.O.

Documentation for writing midas history data to SQL (mysql) is now documented in midas doxygen files 
(make dox; firefox doxfiles/html/index.html). The corresponding logger and mhttpd code has been 
committed for some time now and it is used in production environment by the t2k/nd280 slow controls 
daq system at TRIUMF.

svn rev 4487
K.O.

Fixed problems with mlogger starting and stopping runs.

Basic difficulty was with the mlogger using ASYNC transitions, which did not implement proper 
transition sequencing according to transition sequence numbers. Basically all clients were called at the 
same time, regardless of how long they took to process the transitions.

Switching from ASYNC to SYNC transitions introduces a deadlock between mlogger (not reading data 
from SYSTEM buffer while inside cm_transition) and any program trying to write into the SYSTEM buffer 
(buffer is full, does not listen for transition requests while waiting for mlogger which tries to call it's 
transition handler).

Then we invented the mtransition helper program. In the original implemtation for t2k it was spawned 
directly from the mlogger to stop the run (avoiding the deadlock). Then cm_transition(DETACHED) was 
introduced, but the mlogger start/stop/restart run logic became broken. One problem was with when 
auto restart delay is zero, mtransition tries to restart the run before previous run is stopped (instead, 
mlogger should restart the run from it's tr_stop() handler). Another problem was with the auto restart 
delay counting from the time when we start stopping the run - because stopping the run can take an 
unpredictable time, depending on when various frontends have to do - it is impossible to have a 
predictable delay between runs (again this is fixed by restarting the run from mlogger.c::tr_stop()).

All this has been straightened out by svn revision 4484. Basically the old run stop/restart logic was 
restored in mlogger.c, using cm_transition(DETACH) to avoid the deadlocks.

To remind all, these are the present controls for transitions initiated by mlogger:

/experiment/transition debug flag - set to "2" to capture transition sequences into midas.log
/experiment/transition timeout and transition connect timeout - one can change default timeouts as 
needed to accommodate non cooperative frontends.
/logger/async transitions - do not use mtransition - do ASYNC transitions, as before.
/logger/auto restart delay - delay between stopping the run (mlogger.c::tr_stop) and starting the next 
run.

svn rev 4484
K.O.

mhttpd did not like it when the equipment "Names" arrays had different length compared to the 
corresponding "Variables" arrays. These limitations are now removed.
svn rev 4469
K.O.

mhttpd function for starting and stopping runs now uses cm_transition(DETACH) which spawns an 
external helper program called mtransition to handle the transition sequencing. This helps with the old 
problem of looking at a blank screen for a long time if some frontends take a long time to process run 
transitions. Now mhttpd returns right back and shows start "starting run", "stopping run", etc as 
appropriate.

svn rev 4484 (some bits of this feature are present in rev 4473 and later).
K.O.

P.S. In one of our experiments, I sometimes see mhttpd getting "stuck" when starting or stopping a run 
using this feature. strace shows it is stuck in repeated calls to wait(), but I am unable to reproduce this 
problem in a test system and it happens only sometimes in the experiment. When it does, mhttpd has to 
be restarted. Replacing system("mtransition ...") with ss_sysem("mtransition ...") seems to fix this problem, 
but there are downsides to this (mtransition debug output vanishes) so I am not committing this yet.
K.O.

When using remote midas clients with mserver, you may have noticed the zero-size .RPC.SHM files 
these clients create in the directory where you run them. These files are associated with the semaphore 
created by the midas rpc layer (rpc_call) to synchronize rpc calls between multiple threads. This 
semaphore is always created, even for single-threaded midas applications. Also normally midas 
semaphore files are created in the midas experiment directory specified in exptab (same place as 
.ODB.SHM), but for remote clients, we do not know that location until we start making rpc calls, so the 
semaphore file is created in the current directory (and it is on a remote machine anyway, so this 
location may not be visible locally).

There are 2 problems with these semaphores:
1) in multiple experiments, we have observed the RPC.SHM semaphore stuck in a locked state, 
requiring manual cleanup (ipcrm -s xxx). So far, I have failed to duplicate this lockup using test 
programs and test experiments. The code appears to be coded correctly to automatically unlock the 
semaphore when the program exits or is killed.
2) RPC.SHM is created as a global shared semaphore so it synchronizes rpc calls not just for all threads 
inside one application, but across all threads in all applications (excessive locking - separate 
applications are connected to separate mservers and do not need this locking); but only for applications 
that run from the same current directory - RPC.SHM files in different directories are "connected" to 
different semaphores.

To try to fix this, I implemented "private semaphores" in system.c and made rpc_call() use them.

This introduced a major bug - a semaphore leak - quickly using up all sysv semaphores (see sysctl 
kernel.sem).

The code was now reverted back to using RPC.SHM as described above.

The "bad" svn revisions start with rev 4472, the problem is fixed in rev 4480.

If you use remote midas clients and have one of these bad revisions, either update midas.c to rev 4480 
or apply this patch to midas.c::rpc_call():
ss_mutex_create("", &_mutex_rpc);
should read
ss_mutex_create("RPC", &_mutex_rpc);

Apologies for any inconvenience caused by this problem
K.O.

(catching up on recent changes from t2k and pienu)

Various timeout problems fixed:
- cm_transition() timeouts now settable from ODB (/experiment/transition timeout, transition connect 
timeout). Rev 4479
- rpc_client_call() timeout did not work because of bad select() and alarm() interaction. Rev 4479
- implement rpc connect timeout (was hardwired 10 sec) via rpc_{set,get}_option(-2, RPC_OTIMEOUT). Rev 
4478
- ss_mutex_wait_for() timeout only worked if 1Hz alarm() interrupts are present. Now I use semtimedop() 
and timeout should always work. Rev 4472

K.O.

> I have one more question. I use <odb src="odb field" edit=1> to display an 
> editable ODB value, but how can I show this value in hexadecimal?

Again with JavaScript:

  var v = ODBGet('/some/path&format=%X');

this will retrieve /some/path and format it in hexadecimal. Then you can set a table 
cell with "v" as I wrote in the last reply. If you want to change this value 
however, you need to encode this yourself in JavaScript.

- Stefan

> 1) I display the status of the run with <odb src="/Runinfo/State">, but it 
> returns numbers which is not user friendly. How can I make something 
> like "Running" with green background and "Stopped" with red background in the 
> default status page?

Sorry my late reply, I was really busy. You need JavaScript to perform such a 
task. See the attached example.

> 2) When I click either Start/Stop/Pause/Resume, it can performs the right 
> things, but afterward it jumps to the page "http://domain.name:8081/CS/" 
> which shows "Invalid custom page: NULL path". How can I make it returns 
> to the correct page "http://domain.name:8081/CS/Control%20panel"?

You add a hidden redirect statement:

  <input type=hidden name=redir value="CS/Control panel">

Best regards,

  Stefan

Dear All,

I have one more question. I use <odb src="odb field" edit=1> to display an 
editable ODB value, but how can I show this value in hexadecimal?

Thanks.

Best Regards,
Jimmy


> Dear All,
> 
> I have created a custom page (please see the attachment) and imported into 
> MIDAS with key name "Control panel&" (without the ""). I have the following 
> two questions:
> 
> 1) I display the status of the run with <odb src="/Runinfo/State">, but it 
> returns numbers which is not user friendly. How can I make something 
> like "Running" with green background and "Stopped" with red background in the 
> default status page?
> 
> 2) When I click either Start/Stop/Pause/Resume, it can performs the right 
> things, but afterward it jumps to the page "http://domain.name:8081/CS/" which 
> shows "Invalid custom page: NULL path". How can I make it returns to the 
> correct page "http://domain.name:8081/CS/Control%20panel"?
> 
> Thank you for your attention.
> 
> Best Regards,
> Jimmy

Dear All,

I have created a custom page (please see the attachment) and imported into 
MIDAS with key name "Control panel&" (without the ""). I have the following 
two questions:

1) I display the status of the run with <odb src="/Runinfo/State">, but it 
returns numbers which is not user friendly. How can I make something 
like "Running" with green background and "Stopped" with red background in the 
default status page?

2) When I click either Start/Stop/Pause/Resume, it can performs the right 
things, but afterward it jumps to the page "http://domain.name:8081/CS/" which 
shows "Invalid custom page: NULL path". How can I make it returns to the 
correct page "http://domain.name:8081/CS/Control%20panel"?

Thank you for your attention.

Best Regards,
Jimmy

Hi,

your log script sound to me like the analyzer either got into an infinite loop or 
did a segment violation and just died. I would recommend to run the analyzer from 
inside the debugger. When you then get the segment violation, you can inspect the 
stack trace and see where the bad things happen. Since the analyzer works nicely in 
other experiment, I expect that your problem is related to the user code. Maybe it 
happens at the end of the run, but there is a timeout before the crashed process 
gets cleaned from the ODB, that's why you might think that it happens "between" 
runs.

Best regards,

  Stefan

> 
> Hello Midas experts:
> 
> We have setup a DAQ using MIDAS to readout two ADCs in the crate.
> We are running into problem of analyzer getting killed between 
> runs.  Sometimes it would crash after a few runs and sometimes it 
> would go on for many many runs before analyzer gets killed.  It always 
> occurred between runs not when we are taking data.  Any suggestions 
> on what we could try?  The error message from the midas.log file is 
> appended below.
> 
> Thanks,
> 
> Dawei
> 
> Wed Mar  4 11:53:11 2009 [Analyzer,ERROR] [midas.c:1739:,ERROR]
> cm_disconnect_experiment not called at end of program
> Wed Mar  4 11:53:22 2009 [mhttpd,INFO] Client 'Analyzer' on buffer 'SYSMSG'
> removed by cm_watchdog (idle 10.7s,TO 10s)
> Wed Mar  4 11:53:22 2009 [mhttpd,INFO] Client 'Analyzer' (PID 1) on buffer 'ODB'
> removed by cm_watchdog (idle 10.7s,TO 10s)
> Wed Mar  4 11:53:22 2009 [AL Experiment Frontend,INFO] Client 'Analyzer' on
> buffer 'SYSTEM' removed by cm_watchdog (idle 10.9s,TO 10s)
> Wed Mar  4 11:53:29 2009 [AL Experiment Frontend,TALK] starting new run
> Wed Mar  4 11:53:29 2009 [AL Experiment Frontend,ERROR]
> [midas.c:8264:rpc_client_check,ERROR] Connection broken to "Analyzer" on host
> tsunami

Hello Midas experts:

We have setup a DAQ using MIDAS to readout two ADCs in the crate.
We are running into problem of analyzer getting killed between 
runs.  Sometimes it would crash after a few runs and sometimes it 
would go on for many many runs before analyzer gets killed.  It always 
occurred between runs not when we are taking data.  Any suggestions 
on what we could try?  The error message from the midas.log file is 
appended below.

Thanks,

Dawei

Wed Mar  4 11:53:11 2009 [Analyzer,ERROR] [midas.c:1739:,ERROR]
cm_disconnect_experiment not called at end of program
Wed Mar  4 11:53:22 2009 [mhttpd,INFO] Client 'Analyzer' on buffer 'SYSMSG'
removed by cm_watchdog (idle 10.7s,TO 10s)
Wed Mar  4 11:53:22 2009 [mhttpd,INFO] Client 'Analyzer' (PID 1) on buffer 'ODB'
removed by cm_watchdog (idle 10.7s,TO 10s)
Wed Mar  4 11:53:22 2009 [AL Experiment Frontend,INFO] Client 'Analyzer' on
buffer 'SYSTEM' removed by cm_watchdog (idle 10.9s,TO 10s)
Wed Mar  4 11:53:29 2009 [AL Experiment Frontend,TALK] starting new run
Wed Mar  4 11:53:29 2009 [AL Experiment Frontend,ERROR]
[midas.c:8264:rpc_client_check,ERROR] Connection broken to "Analyzer" on host
tsunami

> mhttpd and mlogger have been updated with potentially troublesome changes.
> These new features are now available:
> - a "feature complete" implementation of "history in an SQL database".

The mlogger SQL history driver has been updated with improvements that make this new system usable in 
production environment: the silly "create all tables on startup, every time, even if they already exist" is fixed,
mlogger survives restarts of mysqld and checks that existing sql columns have data types compatible with the 
data we are trying to write.

There are still a few trouble spots remaining. For example, in mapping midas names into sql names (sql names 
have more restrictions on permitted characters) and in reverse mapping of sql data types to midas data types. 
To properly solve this, I may have to save the midas names and data types into an additional index table.

Included is the mh2sql utility for importing existing history files into an SQL database (in the same way as if 
they were written into the database by mlogger).

The mhttpd side of this system still needs polishing, but should be already fully functional.

A preliminary version of documentation for this new SQL history system is here. After additional review and 
editing it will be committed to the midas midox documentation. Included are full instructions on enabling 
writing of midas history into a MySQL database.
http://ladd00.triumf.ca/~olchansk/midas/Internal.html#History_sql_internal

svn revision 4452
K.O.

At present the only documentation on the Javascript library is in this elog
e.g. Message 496 31 Jul 08

The Javascript library which you can view
http://<your mhttpd host>/mhttpd.js
now supports ODBEdit as well as ODBGet and ODBSet
 
I advise you get the latest version of mhttpd.c so you can use ODBEdit which changes
the ODB value directly via ODBSet.

You use it like this:
document.write('<a href="#" onclick="ODBEdit(/Equipment/test/Variables/Demand[0])">');
document.write('<odb src="/Equipment/test/Variables/Demand[0]">');
document.write('</a>');

You can also use HTML to edit the variables, but the advantage of Javascript is that
you can use variable ODB paths, so it is more powerful.

Here is an example of using a form on a custom page to edit a variable (in the
example, the run number) using Javascript (ODBEdit) and HTML. 

To try this example, in ODB, create key (STRING)
/custom/try& 
 and set it to "/home/user/try.html"

where the path of the example code on the disk is  /home/user/try.html

This will put an alias link on the Main Status page called "try" which you click on
to see the custom page.

Code of try.html:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 TRANSITIONAL//EN">
<html><head>
<title> ODBEdit test</title>
<script src="/js/mhttpd.js" type="text/javascript"></script>

<script type="text/javascript">
var my_action = '"/CS/try&"'
var rn
var path

document.write('</head><body>')
document.write('<form method="get" name="form2" action='+my_action+'> ') 
document.write('<input name="exp" value="'+my_expt+'" type="hidden">');

document.write('Using Javascript and ODBEdit:
')
path='/runinfo/run number' 
rn = ODBGet(path)
document.write('Run Number: '+rn+'
')
document.write('Edit Run Number:')
document.write('<a href="#" onclick="ODBEdit(path)" >')
document.write(rn)
document.write('</a>');
document.write('
') ;
</script>


Using HTML :


Using edit=2 ...  Run Number:
<odb src="/runinfo/run number" edit=2>


Using edit=1 ...  Run Number:
<odb src="/runinfo/run number" edit=1>


</form>
</html>


Note the "edit=2" feature is handy so that you can use Javascript or HTML on your
page and the user sees no difference.


> Hi, I am looking for a way to mutate ODB values from a custom page. I have been
> using the edit attribute for the 'odb' tag, but for some things it would be nice
> if a form can handle the change. I have seen references to ODBSet on the forums,
> but I haven't been able to find documentation on it. Is there an available
> Javascript library for Midas and/or are there more tags than I am aware of (I am
> only aware of the 'odb' tag)?

Hi, I am looking for a way to mutate ODB values from a custom page. I have been
using the edit attribute for the 'odb' tag, but for some things it would be nice
if a form can handle the change. I have seen references to ODBSet on the forums,
but I haven't been able to find documentation on it. Is there an available
Javascript library for Midas and/or are there more tags than I am aware of (I am
only aware of the 'odb' tag)?