Anothe idea which comes to my mind, we could make it kind of object oriented, like

typedef struct {
  int handle;
  int crate;
  int amod;
  int fifo_mode;
  ...
} MVME_INTERFACE;

main()
{
  MVME_INERFACE *vme;

  vme = mvme_init(); // allocated and fills MVME_INTERFACE structure

  mvme_set_crate(vme, crate_no);
  mvme_set_address_mode(vme, A24);
  ...

  mvme_read(vme, vme_address, dest_addr, num_bytes);
  mvme_exit(vme);    // frees memory allocated in mvme_init()
}

------------------------------------------

This way we would only have one structure containing all required parameters, and get/set
functions for it, like the OO textbooks propose it. This would actually make it thread
save. The "vme" pointer from above still has to be passed around to subroutines, but a
single pointer is better than lots of handles.

> Good that you brought up the MIDAS VME API again, since this is still not complete, but
> has to be completed soon.

Right, but I can only complete the parts that I thought of and for which I already have
code. This leaves out support for DMA (read: any block transfers) and interrupts.

> Let me summarize the goals:
> - have a single set of functions which can be used with all VME CPUs/Interfaces at our
> institutes. Using this technique, one can change the interface or CPU and still keep
> the same frontend source code. This was already successfully done with the MIDAS CAMAC
> standard (as defined in mcstd.h)

Well, all interfaces are different and no amount of software will make them look all the
same. I am now facing this problem with the Wiener CCUSB CAMAC-USB2 interface. I can
implement all of mcstd.h, but the interface is intended to be used by downloading it with a
CAMAC readout program and mcstd.h knows nothing about that.

> - base any ADC/TDC driver we write on that API, so these modules can be used with any
> CPU/Interface without changing the driver

Right. Most useful.

> - have a simple and easy to understand set of functions

Right.

> - "cover" any specialities from the drivers, like memory mapping.

Exactly. We are facing a tricky task of inventing one API for two completely different
modes of operation- purely memory mapped access on UniverseII based hardware and message
passing access for the SIS3100 and VMUSB (Wiener VME-USB2).

> So one should only use explicity vme_read/vme_write functions.

Rightey-ho. The fly in the ointement is that all VME ADC and TDC drivers in TRIUMF are
written assuming memory mapped access, and I will not convert them to vme_read/vme_write
overnight (think of testing).

> So I would propose the following:
> 
>   mvme_set_crate()   each funciton has a _get_ partner, like mvme_get_crate()
>   mvme_set_address_mode()
>   mvme_set_amod()
>   mvme_set_blocktransfer()
>   mvme_set_fifomode()             // speciality of the SIS3100 interface, write a
>                                   // block of data to the same address
>   ...
> 
>   mvme_read(vme_address, dest_addr, num_bytes);
>   mvme_write(src_addr, vme_address, num_bytes);

This is compatible with what we do now and I will look into implementing this for
VMIC/Linux and MVME/VxWorks interfaces.

> Now I would like to hear anybody's comments. If we agree on this method, we have to
> define a complete set of functions mvme_set_xxx.

We currently require only single-word transfers so we can concentrate on mvme_set_xxx for
block-transfers later.

> If we get a new interface in the
> future which has new functionality (like 2eVME block transfers), we have to change the
> API each time (while with the ioctl() we only would have to add one parameter).

This amounts to the same thing: add a new function or add a new ioctl() call.

> maybe we can make a more generic mvme_set_vme_mode(mode), where mode could be fifomode,
> 2eVME mode, chained block transfer mode and so on.

This is a can of worms and I would rather postpone discussion of block transfers. To give
you a taste: UniverseII does not have a "fifo mode"- it *always* increments the vme address
 (silly). A fifo mode can be emulated using chained transfers (read 256 bytes from
addresses A through A+256, then read 256 more from address A, etc.), but the present VMIC
VME library does not support chained transfers. On VxWorks, we do not even have a driver
for the DMA engine, so not block transfers there at all.

I will now think about and post an updated proposal for mvmestd.h

K.O.

P.S. There is a proposal for musbstd.h heading your way, too.

> Right, but I can only complete the parts that I thought of and for which I already have
> code. This leaves out support for DMA (read: any block transfers) and interrupts.

DMA should be simple. We have a dma_flag in the MVME_INTERFACE structure, which only needs to
be set with mvme_set_dma_mode(...). The mvme_read/write subroutine then checks this flag and
calls the appropriate routine from the native API. About interrupts I haven't thought so much.
Does TRIUMF use interrupts anywhere? Or are all midas frontends in polled mode?

> Well, all interfaces are different and no amount of software will make them look all the
> same. 
>
> > - "cover" any specialities from the drivers, like memory mapping.
> 
> Exactly. We are facing a tricky task of inventing one API for two completely different
> modes of operation- purely memory mapped access on UniverseII based hardware and message
> passing access for the SIS3100 and VMUSB (Wiener VME-USB2).

Not all the same, but some common denominator. The memory mapped architecture can probably be
hidden in an API. So if one calls mvme_read/write, the routine checks if that region is already
mapped, and maps it if necessary. Then all you need is a proper offset and a memcpy(). Checking
about mapping causes some overhead. You have to check a hash table or a linked list which takes
time. But I think (see previous message) that this overhead should be small compared with the
IO operation. 

> I am now facing this problem with the Wiener CCUSB CAMAC-USB2 interface. I can
> implement all of mcstd.h, but the interface is intended to be used by downloading it with a
> CAMAC readout program and mcstd.h knows nothing about that.

Downloading a program you probably cannot cover with a common API, you are right. The problem
with USB is that you can only make ~1000 transfers per second, even with 2.0. So if you want
more, you need the old list concept.
> > So one should only use explicity vme_read/vme_write functions.
> 
> Rightey-ho. The fly in the ointement is that all VME ADC and TDC drivers in TRIUMF are
> written assuming memory mapped access, and I will not convert them to vme_read/vme_write
> overnight (think of testing).

You don't have to. This question only comes up if you (have to) use a non-memory mapped
interface. You can then either write then two separate drivers, or one driver and two MVME APIs.

> > Now I would like to hear anybody's comments. If we agree on this method, we have to
> > define a complete set of functions mvme_set_xxx.
> 
> We currently require only single-word transfers so we can concentrate on mvme_set_xxx for
> block-transfers later.

I need block transfers end of this month, so we should it include it in our current discussion.
The problem is that I use our (own) DRS2 waveform digitizing board, where each board produces
70kB of data per event. In non-DMA mode, the transfer would take forever.

> > maybe we can make a more generic mvme_set_vme_mode(mode), where mode could be fifomode,
> > 2eVME mode, chained block transfer mode and so on.
> 
> This is a can of worms and I would rather postpone discussion of block transfers. To give
> you a taste: UniverseII does not have a "fifo mode"- it *always* increments the vme address
>  (silly). A fifo mode can be emulated using chained transfers (read 256 bytes from
> addresses A through A+256, then read 256 more from address A, etc.), but the present VMIC
> VME library does not support chained transfers. On VxWorks, we do not even have a driver
> for the DMA engine, so not block transfers there at all.

If a native API does not support block transfer, the MVME driver should just ignore the DMA
setting. A ADC driver might then run slower, but still run. 

> I will now think about and post an updated proposal for mvmestd.h

Please also consider elog:221, I guess this is a cleaner and more flexible way of implementing
any MXXX standard.

- Stefan

I manage to access some vme modules with the older vmicvme interface and seemed
confused with the
new interface as the sample code provided does not have a specific test sample.
The test code
provided in the earlier version for accessing  V792 32ch. QDC was quite handy,
how can I apply it
for the new interface?  

> I manage to access some vme modules with the older vmicvme interface and seemed
> confused with the
> new interface as the sample code provided does not have a specific test sample.
> The test code
> provided in the earlier version for accessing  V792 32ch. QDC was quite handy,
> how can I apply it
> for the new interface?  

Hello Ian,

I'm in the process of updating the V1190B, V792 and other to the new mvmestd.
These drivers will soon be committed to the repository.

Cheers, Pierre-André 

I started to use the new midas custom page features from 
https://midas.triumf.ca/MidasWiki/index.php/Custom_Page . I'd like to setup the 
editable odb values (.e.g <div name="modbvalue" data-odb-path="/Runinfo/Run 
number" data-odb-editable="1"                    
style="position:absolute;top:157px;left:288px;"></div>) from within javascript, 
which doesn`t seem to work. 

 Both 

document.getElementById("test").innerHTML = '<div data-odb-editable="1" data-
odb-path="/Runinfo/Run number" name="modbvalue" 
style="display:inline;position:absolute;top:157px;left:288px;"></div>'

or 

var elem = document.createElement("div");
	 //var id = "manifold0";
	    elem.setAttribute("name","modbvalue");
	 elem.setAttribute("data-odb-
path","/Equipment/Autofill/Variables/Output[6]");
	 elem.setAttribute("data-odb-editable","1");
	 elem.style="position:absolute;top:157px;left:288px;";
         document.getElementById("test").appendChild(elem);

fail on name="modbvalue" with the error 

mjsonrpc_error_alert: TypeError: Cannot set property 'innerHTML' of undefined


How should one do this? I don`t want hard code everything in the htlm body, as 
I have look up odb key indexes in the javascript code.

The function mhttpd_init() scans the custom page and installs handlers etc. for each modbxxx 
element. If you create an modbvalue dynamically in your code, this is probably done after you 
called mhttpd_init(), so the function has no chance to modify the dynamically created element.

To fix that, I separated mhttpd_init() into the old init function which installs the header and sidebar, 
and a function mhttpd_scan() which scans the custom page and processes all modbxxx elements. 
Next, I tapped the error you reported, and added an automatic call to mhttp_scan() in case that 
happens. I tried it on a test page and it worked for me. Please give it a try (commit 090394e8).

Stefan

> The function mhttpd_init() scans the custom page and installs handlers etc. for each modbxxx 
> element. If you create an modbvalue dynamically in your code, this is probably done after you 
> called mhttpd_init(), so the function has no chance to modify the dynamically created element.
> 
> To fix that, I separated mhttpd_init() into the old init function which installs the header and sidebar, 
> and a function mhttpd_scan() which scans the custom page and processes all modbxxx elements. 
> Next, I tapped the error you reported, and added an automatic call to mhttp_scan() in case that 
> happens. I tried it on a test page and it worked for me. Please give it a try (commit 090394e8).
> 
> Stefan

Also works for me

 thanks

> > The function mhttpd_init() scans the custom page and installs handlers etc. for each modbxxx 
> > element. If you create an modbvalue dynamically in your code, this is probably done after you 
> > called mhttpd_init(), so the function has no chance to modify the dynamically created element.
> > 
> > To fix that, I separated mhttpd_init() into the old init function which installs the header and sidebar, 
> > and a function mhttpd_scan() which scans the custom page and processes all modbxxx elements. 
> > Next, I tapped the error you reported, and added an automatic call to mhttp_scan() in case that 
> > happens. I tried it on a test page and it worked for me. Please give it a try (commit 090394e8).
> > 
> > Stefan
> 
> Also works for me
> 
>  thanks


This is more about aesthetic, but when I use the modbvalue div, it first only shows the odb value. However, 
after editing, the odb index gets added to the field, which is kinda ugly -> see attachments

I received my new 2020 mac book air, so between Stefan and myself, MacOS support for 
MIDAS is assured for 5 more years at the least. K.O.

I committed an updated lazylogger with updated documentation. The new version supports subruns and 
can save to external storage arbitrary files (i.e. odb dump files). It also moves most book keeping out of 
odb to permit handling more files on bigger storage disks.

Example lazylogger scripts for castor (CERN) and dcache (TRIUMF) are in the directory "utils".

The lazylogger documentation was updated to remove obsolete information and to describe the new 
functions. As usual "make dox; cd doxfiles/html; firefox index.html" or see my copy at:

http://ladd00.triumf.ca/~olchansk/midas/Utilities.html#lazylogger_task

svn rev 4615, 4616.
K.O.

the new history panel editor has been activated. it is meant to work the same as 
the old editor, with some improvements to the history variables selection page. 
this new version is written in html+javascript and it will be easier to improve, 
update and maintain compared to the old version written in c++. the old history 
panel editor is still usable and accessible by pressing the "edit in old editor" 
button. please report any problem, quirks and improvements in this thread or in 
the bitbucket bug reports. K.O.

the code for writing midas history to mysql has been updated to work against 
MYSQL 8.0.23 (CERN ALPHA-2):

- as ever mysql reports inconsistent data types (I create column with type 
"integer", mysql reports it has type "int" and so forth), the special kludge to 
take care of this had to be tweaked.

- this caused some columns to be marked "inactive" and the code to "reactivate" 
them was missing (fixed)

- binary history event data size was computed incorrectly for events with 
"inactive" columns (fixed) and caused assert() failure and mlogger crash.

- mysql read of column definitions for history event "system" (as in 
/history/links/system) bombed because of incorrect quoting (worked before, why? 
why bombed now?). this caused duplicate columns to be created in mysql table 
"system" and mlogger bomb-out with complaint about "duplicated columns" 
(actually the error message was missing, so it was a silent bomb-out). quoting 
fixed, missing error message fixed, but cleanup of duplicate columns has to be 
done by hand. in case of alpha-2 the fix was to remove the unused 
/history/links/system).

if you are using mysql history please update or patch src/history_schema.cxx.

commit 9d17d2fef233cf457121ca7c2a283c4c76ed33bc

K.O.

Hey,

while trying to compile Midas under openSUSE 15.0 with mysql support, I was
running into the problem that somehow the mysql header file my_global.h is not
included in the packages. This might be a bug that concerns the suse developers
more, but is it actually needed? Compilation worked fine with the include line
commented out.

If it's not needed, I would like to suggest to remove line 735 of
src/history_schema.cxx (where it's included)

Cheers
Lukas

mysql  Ver 15.1 Distrib 10.2.22-MariaDB, for Linux (x86_64) using  EditLine wrapper
Also, all mariadb development packages are installed

Confirmed. my_global.h is removed in MySQL 8.0 (gives a compile error) and deprecated in 
MariaDB 10.2 (gives a #warning).

I removed include of my_global.h, it is not needed on el6, el7 and ubuntu.

Also added explicit support for mariadb via mariadb_config if it exists.

Note that the cmake build does not actually enable mysql, sqlite and odbc - it detects them, but 
does not do anything about it. We will fix this shortly.

K.O.




> Hey,
> 
> while trying to compile Midas under openSUSE 15.0 with mysql support, I was
> running into the problem that somehow the mysql header file my_global.h is not
> included in the packages. This might be a bug that concerns the suse developers
> more, but is it actually needed? Compilation worked fine with the include line
> commented out.
> 
> If it's not needed, I would like to suggest to remove line 735 of
> src/history_schema.cxx (where it's included)
> 
> Cheers
> Lukas
> 
> mysql  Ver 15.1 Distrib 10.2.22-MariaDB, for Linux (x86_64) using  EditLine wrapper
> Also, all mariadb development packages are installed

Thanks!

I have a web page and I try to use modbvalue, but nothing happens. The best I can tell, I follow the documentation 
(https://midas.triumf.ca/MidasWiki/index.php/Custom_Page#modbvalue).

<td id=setv0><div class="modbvalue" data-odb-path="/Equipment/CAEN_hvps01/Settings/VSET[0]" data-odb-editable="1">(ch0)</div></td>

I suppose I could add debug logging to the javascript framework for modbvalue to find out why it is not seeing
or how it is not liking my web page.

But how would a non-expert user (or an expert user in a hurry) would debug this?

Should the modbvalue framework log more error messages to the javascrpt console ("I am ignoring your modbvalue entry because...")?

Should it have a debug mode where it reports to the javascript console all the tags it scanned, all the tags it found, etc
to give me some clue why it does not find my modbvalue tag?

Right now I am not even sure if this framework is activated, perhaps I did something wrong in how I load the page
and the modbvalue framework is not loaded. The documentation gives some magic incantations but does not explain
where and how this framework is loaded and activated. (But I do not see any differences between my page and
the example in the documentation. Except that I do not load control.js, I do not need all the thermometer bars, etc.
If I do load it, still my modbvalue does not work).

K.O.

Can you post your complete page here so that I can have a look?

Stefan

> > > the midas release 2019-03 is ready for general use.
> A bug fix update for midas-2019-03:
> odbedit "ver" should report: Thu Jun 6 18:02:14 2019 -0700 - midas-2019-03-h on branch feature/midas-2019-03

For building this release of MIDAS, please use mxml branch feature/midas-2019-03, tag mxml-2019-03-a:

cd .../mxml
git fetch
git checkout feature/midas-2019-03

Going forward, I will try to remember to tag the mxml version that corresponds to specific midas versions.

K.O.

the mxml library was updated to make it thread-safe.
https://bitbucket.org/tmidas/mxml/src/master/

I also take an opportunity to remind all to update your copy to the latest version
as I just stumbled on old bug that I fixed 1 year ago (crash of mlogger)
but forgot to update all and every of my copies of mxml.

I also looked at the xml encoder and I see that it has several places where it may
truncate the data, but none of these places can cause truncation of ODB data
because the fixed-size internal buffers are big enough to hold the longest
values sent by the odb xml encoder.

K.O.