> Does the sequencer survive a crash or a restart of mhttpd?

Yes. Of course runs will not be started/stopped when mhttpd is not running, but when you restart it gracefully continues where it stopped, since all variables such as event count or current line number of 
the sequence are store in the ODB.

/Stefan

> "programs auto start" : works but has strange interactions and side effects, do not use it.
> "programs auto stop" : works, can be used to stop programs at the end of run (but what for?)
> "programs auto restart" : works, seems to work correctly, can be used to auto restart mlogger, frontends, 

auto start and auto stop have been requested by PAA loooong time ago. Maybe he remembers if/where this has been used at all. I never used it. So if 
this is the case for others, we can easily change it and won't break anything. Like auto start can be executed before the run transition happens, check 
for a previous version of the program, and only continue when the program is actually running. Should be only a few lines of code. Auto restart is used 
successfully here at PSI, for example for the lazy logger.

/Stefan

Seems to me a problem with the ODBC library, so maybe Konstantin can comment.

/Stefan

The warnings with the set but unused variables are real. While John O'Donnell proposed:

==========

somewhere I long the way I found an include file to help remove this kind of message.  try something like:

#include "use.h"
int foo () { return 3; }
int main () {
 { USED int i=foo(); }
 return 0;
}

with -Wall, and you will see the unused messages are gone.

==========

I would rather go and remove the unused variables to clean up the code a bit. Unfortunately my gcc version does 
not yet bark on that. So once I get a new version and I got plenty of spare time (....) I will consider removing all 
these variables.

/Stefan

> got around to look at compile messages on ubuntu: in addition to "variable 'error' set but not used" we have these:
> 
> warning: ignoring return value of 'ssize_t write(int, const void*, size_t)'
> warning: ignoring return value of 'ssize_t read(int, void*, size_t)'
> warning: ignoring return value of 'int setuid(__uid_t)'
> and a few more of similar

Arghh, now it is getting even more picky. I can understand the "variable xyz set but not used" and I'm willing to remove all the variables. But checking the 
return value from every function? Well, if the disk gets full, our code will silently ignore this for write(), so maybe it's not a bad idea to add a few checks. Also 
for the read(), there could be some problem, where an explicit cm_msg() in case of an error would help.

> And I think that works just fine for frontends directly connected to the shared memory, one call to 
> bm_flush_buffer() should be sufficient.

That's correct. What you want is once per second or so for polled events, and once per periodic event (which anyhow will typically come only every 10 seconds or so). If there are 3 calls 
per event, this is certainly too much.


> But for remote fronends connected through the mserver, it turns out there is a race condition between 
> sending the event data on one tcp connection and sending the bm_flush_cache() rpc request on another 
> tcp connection.
> 
> ...
> 
> One solution to this would be to implement periodic bm_flush_buffer() in the mserver, making all calls to 
> bm_flush_buffer() in mfe.c unnecessary (unless it's a direct connection to shared memory).
> 
> Another solution could be to send events with a special flag telling the mserver to "flush the buffer right 
> away".

That's a very good and useful observation. I never really thought about that. 

Looking at your proposed solutions, I prefer the second one. mserver is just an interface for RPC calls, it should not do anything "by itself". This was a strategic decision at the beginning. 
So sending a flag to punch through the cache on mserver seems to me has less side effects. Will just break binary compatibility :-)

/Stefan

Hello,

I'm co-organizing the upcoming Real Time Conference, which covers also the field of data acquisition, so it might be interesting for people working 
with MIDAS. If you have something to report, you could also consider to send an abstract to this conference. It will be located in Nara, Japan. The conference
site is now open at http://rt2014.rcnp.osaka-u.ac.jp/

Best regards,
Stefan Ritt

> The capability to generate doxygen documentation of MIDAS was restored.
> 
> Use "make dox" and "make cleandox",
> find generated documentation in ./html,
> look at it via "firefox html/index.html".
> 
> The documentation is not generated by default - it takes quite a long time to build all the call graphs.
> 
> And the call graphs is what makes this documentation useful - without some visual graphical 
> representation it is quite difficult to understand some parts of MIDAS. Both caller and callee graphs are 
> generated.
> 
> Note that doxygen documentation for the javascript functions in mhttpd.js is also generated, making a 
> handy reference in addition to the full documentation on the MIDAS Wiki.
> 
> K.O.

To generate the files, you need doxygen installed which not everybody has. Is there a web site where one can see the generated graphs?

/Stefan

Since several projects at PSI need MSCB but not MIDAS, I decided to separate the two repositories. So if you 
need MIDAS with MSCB support inside mhttpd, you have to clone MIDAS, MXML and MSCB from bitbucket 
(or the local clone at TRIUMF) as described in

https://midas.triumf.ca/MidasWiki/index.php/Main_Page#Download

I tried to fix all Makefiles to link to the new locations, but I'm not sure if I got all. So if something does not 
compile please let me know.

-Stefan

> The MIDAS Web password function is broken - with the web password enabled, I am not prompted for a 
> password when editing ODB. The password still partially works - I am prompted for the web password 
> when starting a run. K.O.
> 
> P.S. https://midas.triumf.ca/MidasWiki/index.php/Security says "web password" needed for "write access", 
> but does not specify if this includes editing odb. (I would think so, and I think I remember that it used to).

Didn't we agree to put those issues into the bitbucket issue tracker?

This functionality got broken when implementing the new inline edit functionality. Actually one has to "manually" check for the password. The old way 
was that there web page asking for the web password, but if we do ODBSet via Ajax there is nobody who could fill out that form. So I added a 
"manual" checking into ODBCheckWebPassword(). This will not work for custom pages, but they have their own way to define passwords.

/Stefan

> If you follow the MIDAS documentation for setting up password protection, you will get strange messages:

This is interesting. When I used it last time (some years ago...) it worked fine. I did not touch this, and now it's broken. Must be related to some modifications of the semaphore system. 
Well, anyhow, the problem seems to me the db_close_all_databses() and the re-opening of the ODB. Apparently the db_close_database() call does not clean up the semaphores properly. 
Actually there is absolutely no need to close and re-open the ODB upon a wrong password, so I just removed that code and now it works again.

/Stefan

> I'm looking into using MIDAS for an experiment that creates one large event
> (20MB or more) every second.
> 
> Q1: It looks like I should use EQ_FRAGMENTED. Has this feature been in use
> recently? Is it known to work/not work?
> 
> More specifically, the computer should initiate a 1 second data taking, start to
> such the data out of the electronics (which may take a while), change some
> experimental parameters, and start over. 
> 
> Q2: What's the best way to do this? EQ_PERIODIC? 
> I cannot guarantee that the time required to read the hardware has an upper bound.
> In a standalone-prog I would simply use a big loop and let the machine execute
> it as fast as it can: 1.1s, 1.5s, 1.1s, 1.3s, 2.5s, ..... depending on the HW
> deadtimes.
> Will this work with EQ_PERIODIC?
> 
> (Sorry for these maybe stupid questions, but I have so far only used MIDAS for
> externally generated events, with <32kB event size).
> 
> 
> Thanks a lot,
> 
> Randolf

Hi Randolf,

EQ_FRAGMENTED is kind of historically, when computers had a few MB of memory and you have to play special tricks to get large data buffers through. Today I 
would just use EQ_PERIODIC and increase the midas maximal event size to your needs. For details look here:

https://midas.triumf.ca/MidasWiki/index.php/Event_Buffer

The front-end scheduler is asynchronous, which means that your readout is called when the given period (1 second) is elapsed. If the readout takes longer 
than 1s, the schedule will (hopefully) call your readout immediately after the event has been sent. So you get automatically your maximal data rate. At MEG, we 
use 2 MB events with 10 Hz, so a 20 MB/sec data rate should not be a problem on decent computers.

Best,
Stefan

As reported in the bug tracker, the proposed header does not work if no specific (= different from the default 60 sec.) update period is specified, 
since then no cookie is present. Here is the updated code which works for all cases:



<div id="footerDiv" class="footerDiv">
<div style="display:inline; float:left;">MIDAS experiment "Test"</div>
<div id="refr" style="display:inline; float:right;"></div>
</div>
<script type="text/javascript">
var r = document.getElementById('refr');
var now = new Date();
var refr;
if (document.cookie.search('midas_refr') == -1)
   refr = 60;
else {
   var c = document.cookie.split('midas_refr=');
   refr = c.pop().split(';').shift();
}
r.innerHTML = now.toString() + '&nbsp;&nbsp;&nbsp;' + 'Refr:' + refr;
</script>



/Stefan

> On most Linux systems, doxygen is easy to install. Red Hat instructions are here: 
> http://www.triumf.info/wiki/DAQwiki/index.php/SLinstall#Install_packages_needed_for_QUARTUS.2C_ROOT.2C_EPICS_and_MIDAS_DAQ
> 
> On MacOS, doxygen is easy to install via macports: sudo port install doxygen
> 
> > Is there a web site where one can see the generated graphs?
> 
> http://ladd00.triumf.ca/~olchansk/midas/index.html
> 
> there is no cron job to update this, but I may update it infrequently.
> 
> K.O.

Great, thanks a lot!

-Stefan

> I am not sure what to do with the javascript snippet 

Just read elog:908, it tells you to put this into a file, name it header.html for example, and put into the ODB:

/Custom/Header [string32] = header.html

make sure that you put the file into the directory indicated by /Custom/Path.

Cheers,
Stefan

 > If I am not mistaken, the mhttpd.css is hard coded (path/name) into the mhttpd.

I agree that this should be removed, Unfortunately I'm away right now, so I will fix it next week. Also will put in 
Andreas' diffs.

/Stefan

> Works, and here is how I did it. The attached example is based on the standard MIDAS
> example in "src/midas/examples/experiment". 

If you have such huge events, it does not make sense to put them into the ODB. The size needs to be increased (as 
you realized correctly) and the run stop takes long if you write last.xml. So just remove the RO_ODB flag in the 
frontend program and you won't have these problems.

/Stefan

I put mhttpd.css and mhttpd.js into the ODB, so every experiment can change it. I put also Andreas' modifications of the CSS file for the runlog table and 
committed the changes.

/Stefan