These issues have been addressed by Stefan during his visit at Triumf last month.
The latest git has those fixes.

> While I know some of those limits/problems have been already been reported from
> DEAP (and maybe corrected in the last version), I am recording them here:
> 
> Bugs (not working as it should): 
> - "SCRIPT" does not seem to take the parameters into account

Fixed

> - The operators for WAIT are incorrectly set:
> the default ">=" and ">" are correct, but "<=", "<", "==" and "!=" are all using
> ">=" for the test. 

Fixed

> 
> Possible improvements:
> - in LOOP, how can I get the index of the LOOP? I used an extra variable that I
> increment, but it there a better way?

See LOOP doc
 LOOP cnt, 10
   ODBGET /foo/bflag, bb 
   IF $bb==1 THEN
     SET cnt, 10
   ELSE 
     ... 

> - PARAM is giving "string" (or a bool) whose size is set by the user input. The
> side effect is that if I am making a loop starting at "1", the incrementation
> will loop at "9" -> "1". If I start at "01", the incrementation will give "2.",
> "3.",... "9.", "10"... The later is probably what most people would use.

Fixed

> - ODBGet (and ODBSet?) does seem to be able to take a variable as a path... I
> was trying to use an array whose index would be incremented.

To be checked.

> Dear all,
> 
> I'm using MIDAS system and CAEN V1721 to digitize the waveform from photomultipliers ( 
> and the link bridge to PC is V2718 ). I use BLTRead to read data of the digitizer, but 
> I found that if the event counting rate is high ( about 100KB/s ), the communication 
> of V1721 and PC would be suspended randomly, and I get an error code of -2. Could you 
> give me some suggestion? Thanks a lot.

Hi, 

Can you provide the BLTread call fragment code and the PC /var/log/messages at the time of 
the hang up.
What is needed to restart the daq?

PAA

Hi,
We do generate a .json odb at the end of run in order to extract some of its info for our CouchDB.
This is done using the "/program/Execute on stop run" script command. This method decouples the necessity 
to describe completely the info extraction within the ODB/Logger/"CouchDB" and provides possibly better 
flexibility. But including a CouchDB support in the logger as well (like SQL) would be nice too.

Pierre-André


> > Hi. Is it currently possible to automatically save the MIDAS ODB as a JSON file?
> > I can do it manually in odbedit, but it looks like the only option for the
> > automatic ODB save for each run is the standard .ODB format. Is there a way to
> > change this?
> 
> You mean you like an ODB dump at the end of every run in JSON format?
> 
> Sure this can be implemented. But I wonder for what purpose you need that. Can you elaborate a 
> bit, maybe it's a useful feature also other people should be aware of. I'm also thinking if we should 
> offer a CouchDB interface, so ODB data is written directly to that database.
> 
> Stefan

> > Hello, 
> > I'm trying to make Wiener PCIVME link work with MIDAS. 
> > In documentation/VME dirvers/ it's saying: "wevmemm.c PCI/VME Wiener board
> > supported. (see Wiener PCI)".
> > Provided link is dead. Does anyone have that file? I would appreciate very very
> > much if someone could send it to me.
> > 
> > Thank you and best regards, 
> > L.P.
> 
> Hi, I am not familiar with this module, I am pretty sure I have never seen one.
> I do not see any code for it in the midas distribution.
> I do not see any reference to it on the wiener web site (http://www.wiener-d.com/)
> 
> For obsolete modules, they direct us to http://file.wiener-d.com/ which is dead.
> 
> The next best step is to contact Wiener customer support. They usually reply very quickly.
> 
> If you have no luck getting answer directly from Wiener, you can ask me to contact them through
> our sales representative. He is always super very helpful.
> 
> K.O.

Hi, 
I'm having a meeting with Wiener during this weekend, I'll try to dig some info about that module. But I do recall that we had this interface and provided a driver for it.
More later... 
PAA

> > Hello, 
> > I'm trying to make Wiener PCIVME link work with MIDAS. 
> > In documentation/VME dirvers/ it's saying: "wevmemm.c PCI/VME Wiener 
board
> > supported. (see Wiener PCI)".
> > Provided link is dead. Does anyone have that file? I would appreciate 
very very
> > much if someone could send it to me.
> > 
> > Thank you and best regards, 
> > L.P.
> 
> Hi, I am not familiar with this module, I am pretty sure I have never 
seen one.
> I do not see any code for it in the midas distribution.
> I do not see any reference to it on the wiener web site 
(http://www.wiener-d.com/)
> 
> For obsolete modules, they direct us to http://file.wiener-d.com/ which 
is dead.
> 
> The next best step is to contact Wiener customer support. They usually 
reply very quickly.
> 
> If you have no luck getting answer directly from Wiener, you can ask me 
to contact them through
> our sales representative. He is always super very helpful.
> 
> K.O.

Hi, I do recall that we had this interface a while ago. 
I'll be meeting with Wiener during the weekend and will post my findings 
later. 
PAA 

> Hi,
> 
> We've been trying to run Fetest in the attempt of plotting the sine wave data on
> the history page on the web server. However each time we've tried running a new
> plot we have come across the error of 'no data' from the variables. In the
> status page we are clearly obtaining data from the frontend and it is updating
> the variable as expected in SLOW.
> 
> When setting up MIDAS we managed to produce a history plot from Fetest but are
> unable to do so any longer. We did have a go at modifying the Fetest code but
> created a backup before doing so and are now running the original backup.
> 
> What could be causing the Fetest data not to be showing in the history plot?

Is the logger running? (this application is handling the history data).
If yes: Did you change the variable names? If yes: restart the logger.

> > > Dear all
> > > 
> > > We just started our beam time at ILL and just found yesterday that for certain 
> > > settings of our detector the data is not saved into the .mid files. Running "mdump 
> > > -l 10" online we see the data coming in as they should. Nevertheless, if we run 
> > > "mdump -x runXXXXXX.mid" offline, the data file has no events and the banks are 
> > > missing. Any ideas where the data could go lost?
> > > 
> > > Thanks in advance,
> > > Ivo
> > 
> > Have you checked 
> > 
> > /Logger/Channels/0/Settings/Event ID = -1
> > /Logger/Channels/0/Settings/Trigger mask = -1
> > 
> > If these settings are not -1, they filter the data stream for certain events and trigger 
> > masks.
> > 
> > Stefan
> 
> Good morning Stefan
> 
> Both set to -1. We only have one logging channel. If we run a sequence with a few runs and the 
> same settings, sometimes data is in the .mid file and sometimes it is not.
> 
> Best,
> Ivo 


Hi,

If the online mdump is correct (by default using the -1, -1 filter), the data are in the main SYSTEM buffer. 
Similar to the dump -
The fact that the analyzer doesn't see the banks would indicate a buffer handling issue as mentioned by Stefan.
To confirm, I would check at the end of a run that the sum of the equipment "events sent" matches the logger "Events written".

> Have you tried longer files? Maybe a few 100 MB or so. Maybe a buffer is not flushed correctly at the end of a run.

Hi,

If the online mdump is correct (by default using the -1, -1 filter), the data are in the main SYSTEM buffer. 

by the way, similar to the dump - mdump -x file.mid -m raw -d x will show you the events.

The fact that the analyzer doesn't see the banks would indicate a buffer handling issue as mentioned by Stefan.
To confirm, I would check at the end of a run that the sum of the equipment "events sent" matches the logger "Events written".

Hi,

You're building under Linux like. You want to define the LINUX and skip the VARIANT_BOOL all together.
PAA

> Hi everyone,
> 
> I have been building an experiment using the v1718 CAEN interface to talk to my modules and I am using the CAENVMElib Linux Library (2.50). I've managed to deal with data type issues by including additional libraries to my driver code but there is one type error 
that persists:
> 
> 
> In file included from /usr/include/CAENVMElib.h:27:0,
>              	from include/v1718.h:25,
>              	from v1718.c:26:
> /usr/include/CAENVMEtypes.h:323:9: error: unknown type name ‘VARIANT_BOOL’
>      	CAEN_BOOL cvDS0;  	/* Data Strobe 0 signal                     	*/
> 
> 
> The header file used to defined the CAEN types (CAENVMEtypes.h) defines 'CAEN_BOOL' like this:
> 
> 
> #ifdef LINUX
> #define CAEN_BYTE   	unsigned char
> #define CAEN_BOOL   	int
> #else
> #define CAEN_BYTE   	byte
> #define CAEN_BOOL   	VARIANT_BOOL
> #endif
> 
> 
> Has anyone ever ran into that problem when setting up an experiment using the CAEN standard?
> 
> Thanks for your help.
> 
> Isaac

Indeed this "mirror" of the ODB in settings option can cause frustration in 
particular when we think the ODB is empty but is not.
In the other hand, over time the settings are adjusted to a particular 
configuration or touched or not by the individual run preset parameters. Later, if 
a bug or code correction requires multiple restart of the fe, for every start of 
the application, you loose the latest configuration. This can be frustrating as 
well until you force a post-setting or report the specifics parameters in the fe 
code.
BTW I believe, we originally went for the ODB priority for that specific reason.
 
I would be in favour for having a general flag (FALSE) in /experiment which would 
define this global behaviour.  
PAA

> Today I addressed a topic which bugged me since long time. The ODB contains 
> settings under /Equipment/<name>/Common which are a "mirror" of the equipment[] 
> setting in a frontend (using the mfe.cxx framework). If the "Common" entry in 
> the ODB is not present (fresh experiment), the equipment[] settings from the 
> frontend are copied to the ODB. But if it exists, it takes precedence over the 
> equipment[] entries, which is wrong in my opinion. Like if you change some 
> settings in equipment[] (like the logging period of the history), then recompile 
> and restart the frontend, the old values in the ODB are kept and your 
> modification in the frontend code has no effect.
> 
> Starting on commit c3017c6c on Nov. 17th 2020 I reversed the precedence: Now, on 
> each start of the frontend program, the values from equipment[] are written to 
> the ODB. They are still "live". If one changes them when the frontend is 
> running, that change takes effect immediately. But on the next restart of the 
> frontend, the old values from equipment[] is put back there.
> 
> I fell too many times into this trap, and I hope the modification helps 
> everybody. If there are however experiments which rely on the fact that the 
> common settings in the ODB are NOT overwritten by the frontend, please let me 
> know and I can put a flag "EQUIPMENT_FE_PRECEDENCE = FALSE" somewhere to restore 
> the old behaviour.
> 
> Stefan

> Hi all,
> 
> I'm currently trying to see if I can speed up polling in a frontend I'm testing. 
> Currently it seems like I can't get 'lam's to happen faster than 120 times/second. 
> There must be a way to make this faster. From what I understand, changing the poll 
> time (500ms by default) won't affect the frequency of polling just the 'lam' 
> period.
> 
> Any suggestions?
> 
> Thanks for your help!
> 
> Isaac

Hi,

How many equipment do you have and of what type?
What is the measured readout time of your equipment?

As you mentioned the polling time define the maximum time you spend in the in polling call before checking other equipment and system activities. But as soon as you get a LAM during the polling loop, the event is readout. The readout time of this equipment is obviously to be considered as well.

In case you have multiple equipment, the readout time of the other equipment is to be taken in account as you wont return to your polling prior the completion of them.

> > The new history as well as the clock in the web page header show the local time 
> > of the user's computer running the browser.
> > Would it be possible to make it either always use the time zone of the Midas 
> > server, or make it selectable from the config page?
> > It's not ideal trying to relate error messages from the midas.log to history 
> > plots if the time stamps don't match.
> 
> I implemented a new row in the config page to select the time zone. 
> 
> "Local": Time zone where the browser runs
> "Server": Time zone where the midas server runs (you have to update mhttpd for that)
> "UTC+X": Any other time zone
> 
> The setting affects both the status header and the history display.
> 
> I spent quite some time with "named" time zones like "PST" "EST" "CEST", but the 
> support for that is not that great in JavaScript, so I decided to go with simple 
> UTC+X. Hope that's ok.
> 
> Please give it a try and let me know if it's working for you.
> 
> Best,
> Stefan

Hi Stefan,

This is great, the UTC+x is perfect, thank you.
PAA

While I know some of those limits/problems have been already been reported from
DEAP (and maybe corrected in the last version), I am recording them here:

Bugs (not working as it should): 
- "SCRIPT" does not seem to take the parameters into account
- The operators for WAIT are incorrectly set:
the default ">=" and ">" are correct, but "<=", "<", "==" and "!=" are all using
">=" for the test. 

Possible improvements:
- in LOOP, how can I get the index of the LOOP? I used an extra variable that I
increment, but it there a better way?
- PARAM is giving "string" (or a bool) whose size is set by the user input. The
side effect is that if I am making a loop starting at "1", the incrementation
will loop at "9" -> "1". If I start at "01", the incrementation will give "2.",
"3.",... "9.", "10"... The later is probably what most people would use.
- ODBGet (and ODBSet?) does seem to be able to take a variable as a path... I
was trying to use an array whose index would be incremented.

For an unknown reason, Logger died few days ago while writing the history. The
file mhf_1489577446_20170315_system.dat was created, but was empty.

When trying to restart Logger, I would get a seg fault without any special error
message.

I tracked the issue to the "read_file_schema" function in history_schema.cxx

* L4731, a pointer to HsFileSchema *s is declared.
* L4747, We enter a while(1) loop.
* L4749, get char on the filename.
In our case, the file was empty, so the variable "b" gets NULL and the loop breaks.

Problem: the memory allocation for "s" is later in the loop, L4768.
Upon exiting the loop, L4854, we try to access record_size on a NULL pointer ==>
SegFault. 

It would be nice to at least have a message before breaking the loop...

> Hi everyone, 
> 
> We have recently been experiencing a lot of lag with our midas control webpage,
> which is making it very frustrating to use. Has anyone experienced this, and do
> you have any advice to speed it up? Are there particular web browsers that work
> better than others, or certain settings that can make respond more quickly?
> 
> Thanks!
> Wes

We saw this happening as well. In our case, we could track this down to mhttpd
taking a lot of CPU. A kill/restart of mhttpd is usually doing the trick (without
disturbing data taking). We did not find an obvious reason for this happening.

> Dear experts,
>               is there any way to launch an executable script on the host computer from the MIDAS 
> sequencer? If not, is there any interest to develop such a feature?
> 
> Thank you,
>          Francesco

The SCRIPT command will do that (on the machine running MIDAS). I know it works with either python or
bash scripts. I tried without success to pass the parameters and I went around by setting ODB entries
prior to running the script and then access to them within the script.

> 
> .... some ODB settings ....
>      TRANSITION START
>      WAIT events 100
>      TRANSITION STOP
> I have two questions: 
> 
> - for the long run, I want to write on disk only a maximum number of events. I think I can suppress 
> the event polling in the frontend, with an ODB query of the number of collected events. I'm 
> wondering if there is a smarter way to do that. It is also ok if the run is stopped after a maximum 
> number of events, but the subsequent short run should still start exactly after 1h from the previous 
> short run. 

I don't know about a way to give you an exact number of events (maybe /Logger/Run duration). 

I personally use 
    WAIT ODBValue,"/Equipment/DTM/Statistics/Events sent",>,100

Where DTM is the frontend of my trigger. Because of the lag in the run stop, the run will always exceed by few
seconds*rates.

Hope it helps

Context: out of the box  MIDAS (using cmake) on OSX Mojave. 

Running with mongoose/opensslm installation following instruction here:
https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux

mhttpd crashing when midas webpage opened with Safari (12.1.2). Usually when opening the "chat" tab but sometimes also with the "message" tab.
mhttpd(11109,0x70000827a000) malloc: *** error for object 0x7f8669501ef0: pointer being freed was not allocated
mhttpd(11109,0x70000827a000) malloc: *** set a breakpoint in malloc_error_break to debug

No crash if using firefox (70.0.1 (64-bit))

It is reproducible alright.
Here are the core dump and the backtrace (I think  the former is more informative).



> > Context: out of the box  MIDAS (using cmake) on OSX Mojave. 
> > 
> > Running with mongoose/opensslm installation following instruction here:
> > https://midas.triumf.ca/MidasWiki/index.php/Quickstart_Linux
> > 
> > mhttpd crashing when midas webpage opened with Safari (12.1.2). Usually when opening the "chat" tab but sometimes also with the "message" tab.
> > mhttpd(11109,0x70000827a000) malloc: *** error for object 0x7f8669501ef0: pointer being freed was not allocated
> > mhttpd(11109,0x70000827a000) malloc: *** set a breakpoint in malloc_error_break to debug
> > 
> > No crash if using firefox (70.0.1 (64-bit))
> 
> I think we also have reports of mhttpd crash on macos with safari from the Dragon experiment,
> but cannot reproduce the problem.
> 
> If you can reproduce this, can you capture the crash stack trace?
> 
> One way to do this is to enable core dumps in odb "/expt/enable core dumps" set to "y", restart mhttpd,
> wait for the crash. I think macos writes core dumps into /cores/... Or you can run mhttpd inside lldb
> and wait for the crash. the lldb command to show the stack trace is "bt", but you may need
> to switch to different threads to see which one actually crashed. I forget what the command
> for that is.
> 
> BTW, the mhttpd networking code has not changed in a long time, but an update
> of mongoose web server library is overdue (to fix a memory leak, at least).
> 
> K.O.

> - On the other hand, if we have to break compatibility, now is maybe a good time since most accelerators worldwide are off. But before doing so, I would like to get feedback from the main experiments 
> around the world (MEG, T2K, g-2, DEAP besides ALPHA).

Hello Stefan,
For what is worth, DEAP will not be impacted: as we have been taking data around the clock for the last few years, we froze the code running on the computers. We may have some window of opportunity for upgrade in few months but such a move has not been discussed yet.

Best regards,
Pierre