Back Midas Rome Roody Rootana
  Midas DAQ System, Page 23 of 152  Not logged in ELOG logo
New entries since:Wed Dec 31 16:00:00 1969
ID Date Authordown Topic Subject
  724   23 Sep 2010 Stefan RittInfoExample javascript midas page
> We had javascript ODBGet() and ODBSet() functions for some time now, permitting 
implementation of 
> "page-reload-free" "self-updating" web pages. I finally got around to put all 
the javascript bits together 
> to actually implement such a page.

Unfortunately the page has tons of JavaScript errors, probably happened during 
copy-and-paste to elog. Note that 
such files are better places as attachment. I attached a screen dump from the JS 
debugger inside Chrome which I 
use normally to debug JS.
Attachment 1: js_error.png
js_error.png
  725   23 Sep 2010 Stefan RittInfoAnother example of a JavaScript midas page
Please find attached another example of a JavaScript (JS) page using the 
ODBGet/Set functions. 

In contrast to the previous posting, the page is not constructed via the 
document.writeln() function, but written directly in HTML and modified through the 
"innerHTML = ..." functionality. 

It is a control page for our beamline, which gets updated in the background. In 
addition, the user can set the beamline to three predefined settings which are 
stored in an array at the top of the page. As an little extra there is a progress 
bar, which is updated locally via JS since changing the beamline takes a while. 
The progress bar is implemented as a table with variable width, and dynamically 
updated by the JS program. The second attachment is a screen dump from such a 
switching process. Since only values in the ODB are changed, you can try it 
yourself without actually modifying a PSI beam line ;-)
Attachment 1: beamline.html
Attachment 2: beamline.png
beamline.png
  727   24 Sep 2010 Stefan RittInfoExample javascript midas page
> The attached errors all seem to be from cut-and-paste line breaks in the long "document.writeln()" statements. 
> When the page runs, there are no errors from Firefox and Safari.

Then it would be good if you re-submit the file as an attachment so that other people can use it.

> This example uses "document.writeln()" because the number of PostAmp devices displayed in the table is not 
> known in advance and is potentially read from ODB at page load time.

This was not a criticism but just to show that there are different ways of constructing such a page, depending on the 
needs. So people have the choice. Anyhow I think it's very good to have some working examples for people to start 
with.
  732   17 Nov 2010 Stefan RittBug Reportmhttpd "edit on start" breakage
> very recent mhttpd mangles spaces in URL encoding-decoding and I cannot create or delete entries in for 
> example "/experiment/edit on start". For example attempt to delete "/experiment/Pedestals Run" 
> produces:
> <h1>Cannot find key Experiment/edit%20on%20start/Pedestals run</h1>
> (notice "%20" instead of spaces. further navigation sometimes replaces the "%" sign with "%25" making it 
> even more mangled)
> 
> this used to work. looks like a call to URL unmangling went missing somewhere.
> K.O.

Thanks for reporting. Fixed in SVN revision 4882. Actually I outcommented the fix some time ago and forgot to 
put it back. Now I hope that this does not blow anything else...

- Stefan
  733   15 Dec 2010 Stefan RittInfoNew source file structure of MSCB tree
A long planned modification of the source file structure of the MSCB subsystem has been implemented. This is however only for those people who do actively participate in micro controller programming with MSCB. The idea behind this is tha the central include file mscbemb.h had a section for each new project. So whenever a new project was added, this file had to be modified which is clumsy and hard to maintain. Therefore I took the project specific sections out of this file and put it into a config.h file, which is separate for each project (very similar to VxWorks). So the folder tree now looks like this:
midas\mscb\embedded
  \include                <- place for framework include file mscbemb.h
  \lib                    <- precompiled TCP/IP library for SCS-260 submaster
  \src                    <- framework sources mscbmain.c and mscbutil.c
  \<project1>             <- separate folder for project1
      config.h            <- config file for project1
  \<project2>             <- separate folder for project2
      config.h            <- config file for project2
  ...
  \experiment
     \<experiment1>
     \<experiment2>

So each project has it's own config.h, which is included from the central mscbemb.h and can be used to enable certain features of the framework without having to change the framework itself. The "projectx" folders contain devices which are used across several experiments and sometimes also between institutes (PSI and TRIUMF). If you make a device which is only used in a specific experiment, this should go under \experiment with the name of the device or the experiment as a subdirectory. I encourage everybody to submit even specific projects to the subversion system since they can sometimes be useful for others to look at some example code.

A few other things have to be changed in order to adapt to the new structure:

  • The framework files mscbmain.c mscbutil.c and mscbemb.h have moved and therefore they have to be re-added to the projects in the Keil MicroVision Development Environment.
  • The name of the device should not be defined under compiler settings (Project Options/C51/Preprocessor Symbols), but put directly into the config.h file associated with the project.
  • The include paths in the compile have to be changed and point to \midas\mscb\embedded\include
  • The file config.h has to be copied from a similar project and adjusted to fit the new project.

I did remove all project specific sections from mscbemb.h in the current SVN version, so certain projects (FDB_008 at TRIUMF, CRATE_MONITOR and PT100X8 at PSI) have to retrieve the settings (like LED ports etc.) from the old mscbemb.h and put it into the config.h file.

Furthermore there is a new STARTUP_VDDMON.A51 file in the src directory which should be added to each project. This was recommended by the micro controller manufacturer and fixes cases where the EEPROM contents of the CPU gets lost from time to time during power up.

The last thing is that PSI switched to MicroVision 4 as the development environment, so I added new project files (*.uvproj and *.uvopt instead *.Uv2), but I left the old ones there in case someone still has the uV2 environment. They are however not maintained any more.

If there is any problem with the new structure or you have some comments, please don't hesitate to contact me.


- Stefan
  752   17 Feb 2011 Stefan RittBug ReportProblems with midas history SVN 4936
> uhm, mine might be completely unrelated to this, but it just so happened that the rev.
> 4936 was one that was used in a recent experiment, in which there was complaints about
> the responsiveness of the history plots. The history plots would take up to 30 seconds
> to respond, if the run was about 30-40 minutes old. When the run is about  < 10 minutes
> old , the history plot was responsive to within 1-2 seconds.

Ah, that rings a bell. How big are your history files on disk? How much RAM do you have?

What I see in our experiment is that linux buffers everything I write to disk in a cache 
located in RAM. But this cache is limited, so after a certain time it's overwritten. Now 
this is handled by the OS, the application (mlogger in this case) has no influence on it. 
Let's say you write 5 MB/minute of history, and your cache is 50 MB large. Then after 10 
minutes you can still read the history data from the RAM cache which is ~10x faster than 
your disk. But your older history data (30-40 min) is flushed out of the cache and has to be 
re-read from disk. A typical symptom of this is that the first time you display this it 
takes maybe 30 seconds, but if you do a "reload" of your page it goes much faster. In that 
case the contents is cached again in RAM. If you observe this, you can almost be certain 
that you see th "too small RAM cache" problem. In that case just add RAM and things should 
run better (I use 16 GByte in my machine).

Best regards,

  Stefan
  755   30 Mar 2011 Stefan RittForumHow large does "bank32" support?
> Reading an FADC buffer often needs large buffer size, especially while several
> FADCs work together. I want to know how large a bank32 can support.

The "32" in bank32 means 32-bits, so the bank holds 2^32=4 GBytes, hope that is enough for your ADC. 
The "normal" bank has only a 16-bit header, so it can hold only 64 kBytes. But for small banks, the overhead 
is therefore smaller.
  759   10 May 2011 Stefan RittForumsimple example frontend for V1720

Jianglai Liu wrote:
Hi,

Who has a good example of a frontend program using CAEN V1718 VME-USB bridge and
V1720 FADC? I am trying to set up the DAQ for such a simple system.

I put together a frontend which talks to the VME. However it gets stuck at
"Calibrating" in initialize_equipment().

I'd appreciate some help!

Thanks,
Jianglai


During "Calibrating", the framework calls your poll_event() routine. You code there accesses for the first time the VME crate and probably gets stuck.
  766   20 Jun 2011 Stefan RittInfoJavascript ODB interface revised
The Javascript interface to the ODB has been revised. This extends the capabilities of custom web pages requesting data from the ODB. By grouping several request together, the number of round-trips is minimized and the response time is reduced. Following functions are new or extended:

  • ODBGet(path[, format]): This functions works now also with subdirectories in the ODB. The command ODBGet('/Runinfo') returns for example:
    1
    1
    1024
    0
    0
    0
    Mon Jun 20 09:40:14 2011
    1308588014
    Mon Jun 20 09:40:46 2011
    1308588046
    

  • ODBGetRecord(path), ODBExtractRecord(key): While ODBGet can be used for subdirectories, an easier way is to use ODBGetRecord and ODBExtractRecord. The first function retrieves the subtree (record), while the second one can be used to extract individual items. Here is an example:
    result = ODBGetRecord('/Runinfo');
    run_number = ODBExtractRecord(result, 'Run number');
    start_time = ODBExtractRecord(result, 'Start time');
    

  • ODBMGet(paths[, callback, formats]): This function ("Multi-Get") can be used to obtain ODB values from different paths in one call. The ODB paths have to be supplied in an array, the result is again an array. An optional callback routine might be supplied for asynchronous operation. Optional formats might be supplied if the resulting number should be formatted in a specific way. Here is an example:
       var req = new Array();
       req[0] = "/Runinfo/Run number";
       req[1] = "/Equipment/Trigger/Statistics/Events sent";
       var result = ODBMGet(req);
       run_number = result[0];
       events_sent = result[1];
    

The new functions are implemented in mhttpd revision 5075.
  767   21 Jun 2011 Stefan RittInfoNew MIDAS sequencer
A new sequencer for starting and stopping runs has been implemented. Although it is till kind of in a preliminary phase, it is usable, so I would like to share the syntax with you.

The sequencer runs inside mhttpd, and creates a new ODB subdirectory "/Sequencer". There is a new button on the status page called "Sequencer". In can run scripts in XML format, which reside on the server (where mhttpd is running). The sequencer is stateless, that means even if mhttpd is stopped and restarted, it resumes operation from where it has been stopped. Following statements are implemented:

  • <Comment>comment</Comment>
    a comment for this XML file, for information only

  • <ODBSet path="path">value</ODBSet>
    to set a value in the ODB

  • <ODBInc path="path">delta</ODBInc>
    to increment a value in the ODB

  • <RunDescription>Description</RunDescription>
    a run description which is stored under /Experiment/Run Parameters/Run Description.

  • <Transition>Start | Stop</Transition>
    to start or stop a run

  • <Loop n="n"> ... </Loop>
    to execute a loop n times. For infinite loops, "infinit" can be specified as n

  • <Wait for="events | ODBvalue | seconds" [path="ODB path"]>x</Wait>
    wait until a number of events is acquired (testing /Equipment/Trigger/Statistics/Events sent), or until a value in the ODB exceeds x, or wait for x seconds.

  • <Script [loop_counter="1"]>Script</Script>
    to call a script on the server side. Optionally, the loop counter(s) are passed to the script

Attached is a simple script which can be used as a starting point.
Attachment 1: seqtest.xml
<?xml version="1.0" encoding="ISO-8859-1"?>
<RunSequence xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="">
<Comment>This is the XML test file</Comment>

<ODBSet path="/Experiment/Run Parameters/Comment">Test comment</ODBSet>
<RunDescription>Test Run</RunDescription>

<!-- do 10 runs -->
<Loop n="10">
   <!-- increment high voltage and wait for some time to settle -->
   <ODBInc path="/Equipment/HV/Variables/Demand[0]">10</ODBInc>
   <Wait for="seconds">10</Wait>

   <!-- start a run, acquire 3000 events -->
   <Transition>Start</Transition>
   <Wait for="events">3000</Wait>
   <Transition>Stop</Transition> -->
</Loop>
</RunSequence>
  782   19 Dec 2011 Stefan RittBug Reportbk_delete uses memcpy instead of memmove
> > In midas.c, the bk_delete function removes a bank by decrementing the total
> > event size and then copying the remaining banks into the location of the first
> > using memcpy from string.h.
> 
> 
> I confirm the documented difference between memcpy() and memmove() and I confirm the 
> questionable use of memcpy() in bk_delete(). I think it should be memmove(). I made it so in my copy 
> of midas, so this change will not be lost.
> 
> But I am not sure how to test it - I do not think I ever used bk_delete(). I will probably ponder upon 
> this and do a blind commit.
> 
> 
> K.O.

It cannot hurt to use memmove(), so please go ahead to commit the changes.

- Stefan
  783   30 Jan 2012 Stefan RittInfoIEEE Real Time 2012 Call for Abstracts
Hello,

I'm co-organizing the upcoming Real Time Conference, which covers also the field of data acquisition, so it might be interesting for people working 
with MIDAS. If you have something to report, you could also consider to send an abstract to this conference.  It will be nicely located in Berkeley, 
California. We plan excursions to San Francisco and to Napa Valley.

Best regards,
Stefan Ritt

---------------------------

18th Real Time Conference
June 11 – 15, 2012
Berkeley, CA

We invite you to the Hotel Shattuck Plaza in downtown Berkeley, California for
the 2012 Real-Time Conference (RT2012).   It will take place Monday, June 11
through Friday, June 15, 2012, with optional pre-conference tutorials Saturday
and Sunday, June 9-10.

Like the previous editions, RT2012 will be a multidisciplinary conference
devoted to the latest developments on realtime techniques in the fields of
plasma and nuclear fusion, particle physics, nuclear physics and astrophysics,
space science, accelerators, medical physics, nuclear power instrumentation and
other radiation instrumentation.

Abstract submission is open as of 18 January (deadline 2 March). Please visit

http://www.npss-confs.org/rtc/welcome.asp?flag=44675.77&Retry=1 to submit an

abstract.

Call for Abstracts 

RT 2012 is an interdisciplinary conference on realtime data acquisition and
computing applications in the physical sciences. These applications include:

* High energy physics 
* Nuclear physics 
* Astrophysics and astroparticle physics 
* Nuclear fusion 
* Medical physics 
* Space instrumentation 
* Nuclear power instrumentation 
* Realtime security and safety 
* General Radiation Instrumentation 

Specific topics include (but are certainly not limited to) the list shown below.
We welcome correspondence to see how your research fits our venue.   

Key Dates

* Abstract submission opened:  January 18, 2012 
* Abstract deadline:  March 2, 2012 
* Program available: April 2 

Suggested Topics

* Realtime system architectures 
* Intelligent signal processing 
* Programmable devices 
* Fast data transfer links and networks 
* Trigger systems 
* Data acquisition 
* Processing farms 
* Control, monitoring, and test systems 
* Upgrades 
* Emerging realtime technologies 
* New standards 
* Realtime safety and security 
* Feedback on experiences 

Contact Information

If you have a question or wish to opt in for occasional e-mail updates about
RT2012, send us a message at RT2012@lbl.gov. To view full conference
information, visit http://rt2012.lbl.gov/index.html
  785   01 Mar 2012 Stefan RittBug ReportProblem with semaphores
> Anyhow, if anybody can suggest such an advanced locking library it would be great. Will save me the 
> effort of writing one.

Hi Konstantin,

yes there is a good way, which I used during development of the buffer manager function. Put in each sm_xxx function a cm_msg(M_DEBUG, ...) to 
generate a debug system message. They go only into the SYSMSG ring buffer and thus are light weight and don't influence the timing much. You can 
keep odbedit open to see these messages, but there is also another way. You can write a little program which dumps the whole SYSMSG buffer, which 
you can call when the lock happens. You then look "backwards" in time and get all messages stored there, depending of the size of the SYSMSG buffer of 
course. Of course this only works if the lock does not happen on the SYSMSB buffer itself. In that case you have to produce M_LOG messages which are 
written to the logging file. This will influence the timing slightly (the file might grow rapidly) but you are independent of semaphores.

The interesting thing is that in the MEG experiment (9 Front-ends, Event Builder, Logger, Lazylogger, ....) we run for months without any lock up. So I 
might suspect it's caused in your case from a program only you are using.

Best regards,
Stefan
  787   19 Apr 2012 Stefan RittBug ReportBuild error with mlogger: invalid conversion from ‘void*’ to ‘gzFile’

Exaos Lee wrote:
I tried to build MIDAS under ArchLinux, failed on errors as following:
src/mlogger.cxx: In function ‘INT midas_flush_buffer(LOG_CHN*)’:
src/mlogger.cxx:1011:54: error: invalid conversion from ‘void*’ to ‘gzFile’ [-fpermissive]
In file included from src/mlogger.cxx:33:0:
/usr/include/zlib.h:1318:21: error:   initializing argument 1 of ‘int gzwrite(gzFile, voidpc, unsigned int)’ [-fpermissive]
src/mlogger.cxx: In function ‘INT midas_log_open(LOG_CHN*, INT)’:
src/mlogger.cxx:1200:79: error: invalid conversion from ‘void*’ to ‘gzFile’ [-fpermissive]
In file included from src/mlogger.cxx:33:0:
Please refer to attachment elog:786/1 for detail. There are also many warnings listed.

This error can be supressed by adding -fpermissive to CXXFLAGS. But the error message is correct."gzFile" is not equal to "void *"! C allows implicit casts between void* and any pointer type, C++ doesn't allow that. It's better to fix this error. A quick fix would be adding explicit casts. But I'm not sure what is the proper way to fix this.


Ah, dumb gcc gets pickier and pickier. I added a case (gzFile)log_chn->gzfile which fixes the error. I cannot put gzFile already into the header file since the zlib header is included after the midas header, otherwise we get some other problems. The SVN version with the fix is 5275.
  789   27 Apr 2012 Stefan RittBug ReportBuild error with mlogger: invalid conversion from ‘void*’ to ‘gzFile’

KO wrote:
BTW, I read the midas elog via email and if you post html or elcode messages, I receive complete
gibberish. For prompt service, please select message type "plain". (yes, you cannot use fancy colours and
blinking text, but better than me not reading your stuff at all).

BTW2, for easier reading, please include error messages as plain text in your message. As opposed to
compressed attachements.

K.O.


BTW3, if you use a real email program you don't get glibberish. I know some people prefer good-old-text-only pine, but I'm sure you do not use the ascii-only browser lynx to browse the internet, right? So if you browse the web in graphics, why not read your email in graphics as well. Better change yourself than the whole rest of the world Wink
  799   14 Jun 2012 Stefan RittBug ReportCannot start/stop run through mhttpd
> I found the problem only appears when I run mhttpd in scripts, whether bash or python.
> And I'm quite sure that the MIDAS environments (e.g. PATH, MIDAS_EXPTAB, MIDASSYS, etc.)
> are set in such scripts. If I start mhttpd in an xterm with or without "-D", it works
> fine. So, what's the difference between invoking mhttpd directly and through a script?

When you start it with "-D", then mhttpd become a daemon. According to linux rules, it has to "cd /", so it lives in the 
root directory, in order not to block any NFS mount/unmount. If something with the path is not correct then, mhttpd 
cannot find mtransition then. Once I fixed that problem my moving mtransition to /usr/bin.

Stefan
  807   21 Jun 2012 Stefan RittInfomidas vme benchmarks
Just for completeness: Attached is the VME transfer speed I get with the SIS3100/SIS1100 interface using 
2eVME transfer. This curve can be explained exactly with an overhead of 125 us per DMA transfer and a 
continuous link speed of 83 MB/sec.
Attachment 1: Screen_Shot_2012-06-21_at_10.14.09_.png
Screen_Shot_2012-06-21_at_10.14.09_.png
  808   21 Jun 2012 Stefan RittBug ReportCannot start/stop run through mhttpd
> I agree. Somehow mhttpd cannot run mtransition. I am not super happy with this dependance on user $PATH settings and the inability to capture error messages 
> from attempts to start mtransition. I am now thinking in the direction of running mtransition code by forking. But remember that mlogger and the event builder also
> have to use mtransition to stop runs (otherwise they can dead-lock). So an mhttpd-only solution is not good enough...

The way to go is to make cm_transition multi-threaded. Like on thread for each client to be contacted. This way the transition can go in parallel when there are many frontend computers for example, which will speed up 
transitions significantly. In addition, cm_transition should execute a callback whenever a client succeeded or failed, so to give immediate feedback to the user. I think of something like implementing WebSockets in mhttpd for that (http://en.wikipedia.org/wiki/WebSocket).

I have this in mind since many years, but did not have time to implement it yet. Maybe on my next visit to TRIUMF?

Stefan
  810   22 Jun 2012 Stefan RittInfomidas vme benchmarks
> > Just for completeness: Attached is the VME transfer speed I get with the SIS3100/SIS1100 interface using 
> > 2eVME transfer. This curve can be explained exactly with an overhead of 125 us per DMA transfer and a 
> > continuous link speed of 83 MB/sec.
> 
> What VME module is on the other end?
> 
> K.O.

The PSI-built DRS4 board, where we implemented the 2eVME protocol in the Virtex II FPGA. The same speed can be obtained with the commercial 
VME memory module CI-VME64 from Chrislin Industries (see http://www.controlled.com/vme/chinp1.html).

Stefan
  814   25 Jun 2012 Stefan RittInfomidas vme benchmarks
> P.S. Observe the ever present unexplained event rate fluctuations between 130-140 event/sec.

An important aspect of optimizing your system is to keep the network traffic under control. I use GBit Ethernet between FE and BE, and make sure the switch 
can accomodate all accumulated network traffic through its backplane. This way I do not have any TCP retransmits which kill you. Like if a single low-level 
ethernet packet is lost due to collision, the TCP stack retransmits it. Depending on the local settings, this can be after a timeout of one (!) second, which 
punches already a hole in your data rate. On the MSCB system actually I use UDP packets, where I schedule the retransmit myself. For a LAN, 10-100ms timeout 
is there enough. The one second is optimized for a WAN (like between two continents) where this is fine, but it is not what you want on a LAN system. Also 
make sure that the outgoing traffic (lazylogger) uses a different network card than the incoming traffic. I found that this also helps a lot.

- Stefan
ELOG V3.1.4-2e1708b5