Back Midas Rome Roody Rootana
  Midas DAQ System, Page 118 of 121  Not logged in ELOG logo
ID Date Author Topic Subjectup
  251   27 Mar 2006 Sergio BallestreroInfosvn@savannah.psi.ch down ?
> I just tried now and it seemed to work fine. Do you still have the problem?
> 
> - Stefan

 The problem was still there this morning, shortly after seeing your mail, but seems
to be fixed now.
 BTW, which is the best way to submit patches ? I have a version of khyt1331 for Linux
kernel 2.6 (we are running Scientific Linux 4.1), and a few smaller things, mostly in
the examples. 

 Thanks, Sergio
  1815   10 Feb 2020 Konstantin OlchanskiSuggestionswitch midas to c++ threads?
Hi, Stefan & co - now that midas is c++11 and c++11 comes with a threads library, should we 
switch midas to use the c++11 threads instead of pthreads? (Of course on Linux c++11 
threads are a layer on top of pthreads, the best I know).

This should remove the dependency on pthreads.h and use a more native implementation of 
threads on MacOS and Windows. (again, the best I can tell).

Of course this depends on c++11 threads having all the functions we need. Specifically, "lock 
with timeout" is useful to deal with "gah! everything stopped! what do I do!", a problem 
bedeviling midas in the early days (and still happens today!). Current midas kills everything 
after 5 minutes of deadlock - then the user knows how to restart everything and the developer 
has core dumps to look at. (to see which program/thread holds the lock and would not give it 
up).

Any thoughts on this?

K.O.
  1817   11 Feb 2020 Stefan RittSuggestionswitch midas to c++ threads?
I'm thinking of this already since some time, and it was part of my motivation switching to C++11.
I was delighted to see that what we do in system.c (encapsulate system functions such as threads
and shared memory) is now done natively in C++11, and it's done by experts and not amateurs like us.

I see that with std::timed_mutex and try_lock_for we have the desired timeout function. Pity that
C++11 does not contain inter-process communication like semaphores, so there we still have to use
our old functions. 

But for threads switching to std::thread, I'm all in.

Stefan

> Hi, Stefan & co - now that midas is c++11 and c++11 comes with a threads library, should we 
> switch midas to use the c++11 threads instead of pthreads? (Of course on Linux c++11 
> threads are a layer on top of pthreads, the best I know).
> 
> This should remove the dependency on pthreads.h and use a more native implementation of 
> threads on MacOS and Windows. (again, the best I can tell).
> 
> Of course this depends on c++11 threads having all the functions we need. Specifically, "lock 
> with timeout" is useful to deal with "gah! everything stopped! what do I do!", a problem 
> bedeviling midas in the early days (and still happens today!). Current midas kills everything 
> after 5 minutes of deadlock - then the user knows how to restart everything and the developer 
> has core dumps to look at. (to see which program/thread holds the lock and would not give it 
> up).
> 
> Any thoughts on this?
> 
> K.O.
  1819   11 Feb 2020 Berta BeltranSuggestionswitch midas to c++ threads?
> Hi, Stefan & co - now that midas is c++11 and c++11 comes with a threads library, should we 
> switch midas to use the c++11 threads instead of pthreads? (Of course on Linux c++11 
> threads are a layer on top of pthreads, the best I know).
> 
> This should remove the dependency on pthreads.h and use a more native implementation of 
> threads on MacOS and Windows. (again, the best I can tell).
> 
> Of course this depends on c++11 threads having all the functions we need. Specifically, "lock 
> with timeout" is useful to deal with "gah! everything stopped! what do I do!", a problem 
> bedeviling midas in the early days (and still happens today!). Current midas kills everything 
> after 5 minutes of deadlock - then the user knows how to restart everything and the developer 
> has core dumps to look at. (to see which program/thread holds the lock and would not give it 
> up).
> 
> Any thoughts on this?
> 
> K.O.

Hi, I just wanted to say that I have seen this post and maybe that is the solution to the pthreads compiler problem in OS 10.15, but 
of course I am a total amateur in here. Thanks for thinking about this and I will wait and hold to see what gets decided. Thanks

Berta  
  1519   15 Apr 2019 Konstantin OlchanskiInfoswitch of MIDAS to C++
For a long time now we have keep the core of midas (odb.c, midas.c, etc) compatible with plain C and by default 
we have built the MIDAS library using the plain C compiler. Over time, we have switched most MIDAS programs 
(mhttpd, mlogger, mdump, odbedit, etc) to C++ (with happy results). (and for a long time now, all of MIDAS 
could be build as C++, even if the default build remained plain C).

The main reason for keeping the core of MIDAS as C has been to allow writing MIDAS frontends in C - for 
example, in environments with no C++ compilers or no C++ runtime (VxWorks) or where C++ had too much 
overhead (small memory machines, etc).

Today, all concerns against using C++ seem to have receded into the past. C++ compilers are now always 
available, even for small embedded systems. C++ overheads are now well understood and one can easily write 
C++ code that is as efficient as C for using limited CPU and memory resources. (While at the same time, today's 
embedded systems tend to have more CPU and RAM than "big" MIDAS DAQ machines had in the past - 1GHz 
CPU, 1GB RAM is pretty typical for embedded ARM).

As examples of small hardware where MIDAS frontends written in C++ worked just fine, consider the T2K ND280 
FGD data collector running on XILINX FPGA with a 300MHz PowerPC and 128 Mbytes of RAM (standard Linux 
kernel) and the GRIFFIN Clock distribution module control running on a Microsemi FPGA with a 300MHz ARM 
CPU (ucLinux without an MMU). More typical Cyclone-5 ARM SoCs with 1GB RAM and 1GHz CPU run standard 
Linux (CentOS7) and can build MIDAS natively (no need for cross-compiling).

With the removal of the requirement to make it possible to write MIDAS frontends in C, we can switch the MIDAS 
default build to C++ and start using C++ features in the MIDAS API (std::string, std::vector, etc).

Next to consider is "which C++ should we use?".

K.O.
  1531   22 May 2019 Konstantin OlchanskiInfoswitch of MIDAS to C++
> switch MIDAS to C++

switch to C++ will proceed as follows:

- create a new branch off develop (feature/switch_to_cxx)
- remove all extern "C", ifdef c++, etc
- switch Makefile from gcc to g++
- test
- merge into develop
- before merge, tag the last "C" midas
- cut a new release branch (tentatively feature/midas-2019-06)

the last recommended "pre-C++" midas will remain the midas-2019-03 release (where we can retroactively apply bug fixes, as I just did a few minutes ago).

K.O.
  1521   15 Apr 2019 Konstantin OlchanskiInfoswitch of MIDAS to C++, how much C++?
> >
> > With the removal of the requirement to make it possible to write MIDAS frontends in C, we can switch the MIDAS 
> > default build to C++ and start using C++ features in the MIDAS API (std::string, std::vector, etc).
> > 

C++ is a big animal. Obviously we want to use std::string, std::vector and similar improvements over plain C (we already use "//" for comments).

But in keeping with the Camel's nose fable (https://en.wikipedia.org/wiki/Camel%27s_nose), there are some parts of C++ we definitely do not want to use in MIDAS. Even the C++ FAQ talks 
about "evil features", see https://isocpp.org/wiki/faq/big-picture#use-evil-things-sometimes

Here is my list of things to use and to avoid. Comments on this are very welcome - as everybody's experience with C++ is different (and everybody's experience is very valuable and very 
welcome).

- std::string, std:vector, etc are in. I am already using them in the MIDAS API (midas.h)
- extern "C" is out, everything has to be C++, will remove "extern "C"" from all midas header files.
- exceptions are out, see https://stackoverflow.com/questions/1736146/why-is-exception-handling-bad
- std::thread and std::mutex are in, at least for writing new frontends, but see discussion of "cannot use c++11". (maybe replace ss_mutex_xxx() with out own std::mutex look-alike).
- heavy use of templates and heavy use of argument overloading is out - just by looking at the code, impossible to tell what function will be called
- "auto" is on probation. I need to know if "auto v=f()" is an integer or a double when I write "auto w=v/2" or "auto w=v/2.0". see 
https://softwareengineering.stackexchange.com/questions/180216/does-auto-make-c-code-harder-to-understand
- unreadable gibberish is out (lambdas, etc)
- C-style malloc()/free() is in. C++ new and delete are okey, but "delete[]" confuses me.
- C-style printf() is in. C++ cout and "<<" gunk provide no way to easily format the output for easy reading.

K.O.
  1522   16 Apr 2019 Pintaudi GiorgioInfoswitch of MIDAS to C++, how much C++?
Dear Konstantin,

even if I am still quite young and have only limited experience (but not null), I would like to give my two cents. I have reflected a bit about the C++ issue, also because I am developing a 
brand new MIDAS interface for the WAGASCI-T2K experiment, and I feel that the future of MIDAS could influence the future of our DAQ system, too. I'll start from the conclusions: I completely 
agree with you on a practical level, even if I kind of disagree on an "ethical" level.

What you propose in essence is to migrate the MIDAS core from pure C to a version of C with some fancy C++ features. Let's say a kind of C+ with only one plus. Theoretically speaking, even if 
on the surface C and C++ are very similar, they are completely different languages and require different mindsets (and I am sure that everyone is aware of it). This is the reason why even if I 
would have preferred to develop the MIDAS frontend for our experiment in C++, I have chosen to stick to pure C because I feel that MIDAS is still very C-like in its architecture (or from what 
I can see from the documentation). So I wanted to "keep on track" for better internal coherence. What I mean is that, if someone told me to port a C project of mine to C++, I would end up 
rewriting it almost completely, instead of just modifying it (I really don't know how much of the MIDAS core has been written with C++ in mind, so if a large part of it is already C++-like, 
please ignore my comment above).

Anyway, on a practical level, I completely agree with your approach, because I imagine that a complete rewrite of MIDAS is off the table but, at the same time, some new C++ features like 
better string and vector handling are very tempting to use. Moreover, in general, physicists are more familiar with the C syntax than with the C++ one (but thanks to ROOT that is changing). As 
for the use of MIDAS in embedded devices, I have no experience so I refrain from judging. So, in the particular case of MIDAS, what you propose is probably the best and only option.

As far as the C++ standard to adopt, I would say that the C++11 standard is the best fit for the T2K experiment since the official OS for T2K is CentOS7 and, out of the box, it supports C++11 
only. Anyway, I acknowledge that there are many other experiments and requirements. For the records, I do development on Ubuntu 18.04.

Best regards
Giorgio
  1523   17 Apr 2019 John M O'DonnellInfoswitch of MIDAS to C++, how much C++?
some semi-random thoughts:

no templates strictly means you can't use std::string, std::vector etc.

printf is in any case part of C++ (#include <cstdio>), but std::ostreams can be faster (for std::cout, endl line causes buffer flushing, whereas "\n" does not flush the buffer but printf
always flushes the buffer), and formatting is possible (though very long winded).  printf does not allow to print things other than simple data, e.g. BANK_HEADER* bh; printf( "%?", *bh);

I've been writing all our DAQ code in C++ for a while now.

> > >
> > > With the removal of the requirement to make it possible to write MIDAS frontends in C, we can switch the MIDAS 
> > > default build to C++ and start using C++ features in the MIDAS API (std::string, std::vector, etc).
> > > 
> 
> C++ is a big animal. Obviously we want to use std::string, std::vector and similar improvements over plain C (we already use "//" for comments).
> 
> But in keeping with the Camel's nose fable (https://en.wikipedia.org/wiki/Camel%27s_nose), there are some parts of C++ we definitely do not want to use in MIDAS. Even the C++ FAQ talks 
> about "evil features", see https://isocpp.org/wiki/faq/big-picture#use-evil-things-sometimes
> 
> Here is my list of things to use and to avoid. Comments on this are very welcome - as everybody's experience with C++ is different (and everybody's experience is very valuable and very 
> welcome).
> 
> - std::string, std:vector, etc are in. I am already using them in the MIDAS API (midas.h)
> - extern "C" is out, everything has to be C++, will remove "extern "C"" from all midas header files.
> - exceptions are out, see https://stackoverflow.com/questions/1736146/why-is-exception-handling-bad
> - std::thread and std::mutex are in, at least for writing new frontends, but see discussion of "cannot use c++11". (maybe replace ss_mutex_xxx() with out own std::mutex look-alike).
> - heavy use of templates and heavy use of argument overloading is out - just by looking at the code, impossible to tell what function will be called
> - "auto" is on probation. I need to know if "auto v=f()" is an integer or a double when I write "auto w=v/2" or "auto w=v/2.0". see 
> https://softwareengineering.stackexchange.com/questions/180216/does-auto-make-c-code-harder-to-understand
> - unreadable gibberish is out (lambdas, etc)
> - C-style malloc()/free() is in. C++ new and delete are okey, but "delete[]" confuses me.
> - C-style printf() is in. C++ cout and "<<" gunk provide no way to easily format the output for easy reading.
> 
> K.O.
  1524   22 Apr 2019 Pintaudi GiorgioInfoswitch of MIDAS to C++, how much C++?
Dear Konstantin and others,
our recent discussion stimulated my curiosity and I wrote a small frontend for the trigger board of our experiment in C++.
The underlying hardware details are not relevant here. I would just like to briefly report and discuss what I found out.

I have written all the frontend files (but the bus driver) in C++11:
  • my_frontend.cpp
  • driver/class/my_class_driver.cpp
  • driver/device/my_device_driver.cpp

All went quite smoothly, but I feel that the overall structure is still very C-like (that may be a good thing or a bad thing depending on the point of view).
As far as I know, the MIDAS frontend mfe.c has still only the C version (I couldn't find any mfe.cxx). This means that all the points of contact between the MIDAS frontend code and the user
frontend code must be C compatible (no C++ features or name mangling). To accomplish this I needed to slightly modify the midas.h header file like this:
@@ -1141,7 +1141,13 @@ typedef struct eqpmnt {
+#ifdef __cplusplus
+extern "C" {
+#endif
 INT device_driver(DEVICE_DRIVER *device_driver, INT cmd, ...);
+#ifdef __cplusplus
+}
+#endif

I also tested the new strcomb1 function and it seems to work OK.

I have attached a source file to show how I implemented the device driver in C++. The code is not meant to be compilable: it is just to show how I implemented it. This is the most C++-like syntax that I could come out with. Feel free to comment it and if you think that it could be improved let me know.

Best Regards
Giorgio
  1525   23 Apr 2019 Konstantin OlchanskiInfoswitch of MIDAS to C++, how much C++?
> Dear Konstantin and others, our recent discussion stimulated my curiosity and I wrote a small frontend for the trigger board of our 
experiment in C++.

Yay!

> my_frontend.cpp

In MIDAS we are using .cxx, not .cpp, per ROOT coding convention https://root.cern.ch/coding-conventions

> the overall structure is still very C-like

this is object-oriented programming done in C. (actually C++ looks exactly the same if you look behind the curtain)

right now we do not hope to rewrite the slow control class driver framework in C++, but if somebody does it,
we should be happy to add it to midas.

for the mfe.c framework, I have a new C++ class based frontend framework in development (and already in use
in the ALPHA-g experiment at CERN). There is a number of lose ends to polish befire I can add it to midas.
And as usual the last 10% of the work consume 90% of the time.

> the MIDAS frontend mfe.c has still only the C version (I couldn't find any mfe.cxx). 
> This means that all the points of contact between the MIDAS frontend code and the user frontend code must be C compatible
> (no C++ features or name mangling).

this will change with the switch to C++, mfe.c will become mfe.cxx and I shall add the required definitions to mfe.h (or midas.h, TBD)

> To accomplish this I needed to slightly modify the midas.h header file like this:
> +#ifdef __cplusplus
> +extern "C" {
> +#endif
>  INT device_driver(DEVICE_DRIVER *device_driver, INT cmd, ...);

I intend for all "extern "C"" to go away, everything will use the C++ linkage (and name mangling). This will break existing frontends
and I will need to write clear instructions on converting them to the new scheme.

> I also tested the new strcomb1 function and it seems to work OK.

good.

> I have attached a source file to show how I implemented the device driver in C++

Yup, looks familiar, I have a couple of C++ frontends written like this, too.

K.O.
  1520   15 Apr 2019 Konstantin OlchanskiInfoswitch of MIDAS to C++, which C++?
>
> With the removal of the requirement to make it possible to write MIDAS frontends in C, we can switch the MIDAS 
> default build to C++ and start using C++ features in the MIDAS API (std::string, std::vector, etc).
> 

Consider the most basic C++ construct, std::string, and observe how many member functions are annotated "c++11", "c++17", etc:
https://en.cppreference.com/w/cpp/string/basic_string

For MIDAS this means that we cannot target "a" C++ or "the" C++, we have to chose between C++ "before C++11", C++11, C++17 
(plus the incoming c++20).

For example, the ROOT 6 package requires C++11 *and* g++ >= 4.8.

Now consider the platforms we use at TRIUMF:

- Linux RHEL/SL/CentOS6 - gcc 4.4.7, no C++11.
- Linux RHEL/SL/CentOS7 - gcc 4.8.5, full C++11, no C++14, no C++17
- Ubuntu 18.04.2 LTS - gcc 7.3.0, full C++11, full C++14, "experimental" C++17.
- MacOS 10.13 - llvm 10.0.0 (clang-1000.11.45.5), full C++11, full C++14, full C++17.

(see here for GCC C++ support: https://gcc.gnu.org/projects/cxx-status.html)
(see here for LLVM clang c++ support: https://clang.llvm.org/cxx_status.html)

As is easy to see from the std::string reference how C++17 has a large number of very useful new features.

Alas, at TRIUMF we still run MIDAS on many SL6 machines where C++11 and C++17 is not normally available. I estimate another 1-2 
years before all our SL6 machines are upgraded to RHEL/SL/CentOS7 (or Ubuntu LTS).

This means we cannot use C++11 and C++17 in MIDAS yet. We are stuck with pre-C++11 for now.

Remarks:
- there will be trouble right away as both Stefan and myself do MIDAS development on MacOS where full C++17 is available and is 
tempting to use. (as they say, watch this space)
- it is possible to install a newer C++ compiler into RHEL/SL/CentOS 6 and 7 systems, but we are loath to require this (same as we 
are loath to require cmake for building MIDAS) - the "I" in MIDAS means integrated, meaning "does not require installing 100 
additional packages before one can use it".
- the MS Windows situation is unclear, but since one has to install the C++ compiler as an additional package anyway, I do not see 
any problem with requiring C++17 support, with a choice of MS compilers, GCC and LLVM. I doubt we will support anything older 
than Windows 10.

K.O.
  1528   11 May 2019 Konstantin OlchanskiInfoswitch of MIDAS to C++, which C++?
> [which c++]
> 
> - Linux RHEL/SL/CentOS6 - gcc 4.4.7, no C++11.
> - Linux RHEL/SL/CentOS7 - gcc 4.8.5, full C++11, no C++14, no C++17
>

The construct I now always use:

class X {
int a = 0; // do not leave data members uninitialized, see "Non-static data member initializers", N2756 and N2628
}

is only available starting from gcc 4.7, see https://gcc.gnu.org/projects/cxx-status.html

Another nail into the coffin of "pre c++11" c++ and el < el7.

Hmm...

K.O.
  1494   14 Mar 2019 Konstantin OlchanskiInfoswitch to json odb dump format
Regarding odb dumps saved into the midas output files, there are several
requests to make it possible to save odb in the json format.

Since 2019-02-12 commits
https://bitbucket.org/tmidas/midas/commits/ccaeeef4d6a1f0d91a67f2dc0a68105b7f6b77ae
and
https://bitbucket.org/tmidas/midas/commits/b3251587a68ef577919c81a221559e9af4dc8ae6

The format of the odb dump is specified by ODB
/Logger/Channels/0/Settings/ODB dump format (default is "json", was hardcoded as "xml").

The filename of the odb dump file saved at the end of run is specified by ODB
/Logger/ODB Last Dump File (default is "last.json", was hardcoded as "last.xml").

In addition, the following defaults have been changed, enabling LZ4 compession and CRC32C checksums by default:

/Logger/ODB dump file (new default is "run%05d.json", was "run%05d.odb")
/Logger/Channels/0/Settings/
Output "FILE"
Compress "lz4"
Checksum "CRC32C".

This brings mlogger default settings closer to what one would expect in the 21st century - "free" compression (LZ4)
and output file data integrity protected by checksums (CRC32C). These defaults are "free" in the sense that turning
them off would not noticeably improve the system performance. (Some users may want to enable better compression,
such as BZIP2 or PBZIP2 and better checksums, such as SHA256 or SHA512 - both choices that have significant CPU-use cost).

The default ODB dump format is now consistently "json" (vs the previous mix of XML and ODB formats).

Why "json" as the default? There has to be some default. The old ODB dump format is right out. Compared to XML,
JSON dump file size is smaller and the encoder is faster (important for reducing the time to start a run - where ODB dump is done twice). Tools for parsing 
JSON data are more widely available and "JSON has won in the marketplace". See also https://www.w3schools.com/js/js_json_xml.asp

Those who want to continue to use XML ODB dumps can trivially continue to do so by changing the settings listed above.

In the rootana package, we have the XmlOdb class for parsing XML ODB dumps, but no matching JsonOdb class for parsing
JSON ODB data. This reminds me of difficulties working with XML data. I originally wrote the XmlOdb class using
the "libxml2" package. After discovering that many Linux distributions do not install this package by default, I switched
to the DOM and XML parsers from ROOT (since rootana is not very useful without ROOT anyway). Only to discover that
some binary ROOT distributions distributed by CERN exclude the XML parser from their default build.

I do intend to write the JsonOdb class for rootana, and I will probably do it using the mjson.{h,cxx} parser that I wrote
for MIDAS. (lack of the JsonOdb class is the reason I did not switch MIDAS ODB dumps to JSON much earlier
than now. the camel back was broken by the extra slow run start time in the ALPHA-g DAQ system).

K.O.
  1128   29 Oct 2015 Konstantin OlchanskiInfosynchronous ajax deprecated
If using a synchronous AJAX call, such as "foo=ODBGet("/runinfo/state");", google chrome will prints this to the javascript console:

"Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience. For more help, check http://xhr.spec.whatwg.org/."

The referenced URL has this text:

"Synchronous XMLHttpRequest outside of workers is in the process of being removed from the web platform as it has detrimental effects to the end user's experience. (This is a long 
process that takes many years.) Developers must not pass false for the async argument when the JavaScript global environment is a document environment. User agents are strongly 
encouraged to warn about such usage in developer tools and may experiment with throwing an InvalidAccessError exception when it occurs."

Then jQuery say this: http://api.jquery.com/jquery.ajax/

"As of jQuery 1.8, the use of async: false with jqXHR ($.Deferred) is deprecated; you must use the success/error/complete callback options instead of the corresponding methods of the 
jqXHR object such as jqXHR.done() or the deprecated jqXHR.success()."

This sounds rather severe but one must flow with the flow. Synchronous RPC is out, async is in.

Many of the old MIDAS AJAX functions are fully synchronous (i.e. "foo=ODBGet("/blah");"), some more recent ones support both sync and async use (i.e. ODBMCopy()).

All the newly added functions *must* by async-only. For example, all the new JSON-RPC functions are async-only and require the use of callbacks to get at the data.

Converting existing javascript custom pages from sync AJAX (hah! it's SJAX, not AJAX) will require some work, and one might as well start today.

Personally, I think this excessive use of callbacks for all javascript web page programming is an unnecessary PITA, but I also do understand the motivation
of people who write web browsers and javascript engines - removal of support for synchronous RPC makes many things much simpler -
and even small speedup of javascript execution and better browser efficiency is welcome improvements (but not free improvements - as old web pages need to be converted).

K.O.
  1129   29 Oct 2015 Amy RobertsInfosynchronous ajax deprecated
We're using mhttpd for calls that end up working better with asynchronous requests, and we've built up sort of a parallel, asynchronous library using javascript Promises.

The Promises (which are in the ES6 spec) have worked incredibly well for building well-behaved, sequential calls to mhttpd.  Personally, I also find their syntax much easier to wrap my
head around, especially compared to callbacks.

I'd be happy to add these functions to midas.js if there's general interest. 

> If using a synchronous AJAX call, such as "foo=ODBGet("/runinfo/state");", google chrome will prints this to the javascript console:
> 
> "Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience. For more help, check http://xhr.spec.whatwg.org/."
> 
> The referenced URL has this text:
> 
> "Synchronous XMLHttpRequest outside of workers is in the process of being removed from the web platform as it has detrimental effects to the end user's experience. (This is a long 
> process that takes many years.) Developers must not pass false for the async argument when the JavaScript global environment is a document environment. User agents are strongly 
> encouraged to warn about such usage in developer tools and may experiment with throwing an InvalidAccessError exception when it occurs."
> 
> Then jQuery say this: http://api.jquery.com/jquery.ajax/
> 
> "As of jQuery 1.8, the use of async: false with jqXHR ($.Deferred) is deprecated; you must use the success/error/complete callback options instead of the corresponding methods of the 
> jqXHR object such as jqXHR.done() or the deprecated jqXHR.success()."
> 
> This sounds rather severe but one must flow with the flow. Synchronous RPC is out, async is in.
> 
> Many of the old MIDAS AJAX functions are fully synchronous (i.e. "foo=ODBGet("/blah");"), some more recent ones support both sync and async use (i.e. ODBMCopy()).
> 
> All the newly added functions *must* by async-only. For example, all the new JSON-RPC functions are async-only and require the use of callbacks to get at the data.
> 
> Converting existing javascript custom pages from sync AJAX (hah! it's SJAX, not AJAX) will require some work, and one might as well start today.
> 
> Personally, I think this excessive use of callbacks for all javascript web page programming is an unnecessary PITA, but I also do understand the motivation
> of people who write web browsers and javascript engines - removal of support for synchronous RPC makes many things much simpler -
> and even small speedup of javascript execution and better browser efficiency is welcome improvements (but not free improvements - as old web pages need to be converted).
> 
> K.O.
  1130   30 Oct 2015 Stefan RittInfosynchronous ajax deprecated
> We're using mhttpd for calls that end up working better with asynchronous requests, and we've built up sort of a parallel, asynchronous library using javascript Promises.
> 
> The Promises (which are in the ES6 spec) have worked incredibly well for building well-behaved, sequential calls to mhttpd.  Personally, I also find their syntax much easier to wrap my
> head around, especially compared to callbacks.
> 
> I'd be happy to add these functions to midas.js if there's general interest. 

Why don't you post the functions here so that we can have a look? They don't have to be incorporated into mhttpd.js necessarily, but could live in a separate file, the people can choose which one to use.

Stefan
  1131   02 Nov 2015 Konstantin OlchanskiInfosynchronous ajax deprecated
> We're using mhttpd for calls that end up working better with asynchronous requests, and we've built up sort of a parallel, asynchronous library using javascript Promises.
> 
> The Promises (which are in the ES6 spec) have worked incredibly well for building well-behaved, sequential calls to mhttpd.  Personally, I also find their syntax much easier to wrap my
> head around, especially compared to callbacks.
> 

Yes, the javascript wrappers for the json-rpc interface follow the Promise pattern - an RPC call is provided with two user functions,
one is called on success (and provides the rpc reply), the other on failure (and provides all rpc call information - the xhr object, any exception context, etc).

Use of the Promise class itself seems to be problematic - apparently it does not exist in google chrome 28 (the last version for RHEL/CentOS/SL6).

SL6 is still our main workhorse and it is good to have a choice of 2 browsers (old google chrome vs old firefox).

(All SL5 web browsers are already unusable with the modern web and current mhttpd.)

(Also the RPC calls have more than 1 item of data permitted by the javascript Promise class - of course it can be wrapped
be a container object - just an extra complication to document and to understand).

K.O.
  1136   17 Nov 2015 Konstantin OlchanskiInfosynchronous ajax deprecated
> > We're using mhttpd for calls that end up working better with asynchronous requests, and we've built up sort of a parallel, asynchronous library using javascript Promises.

I checked again on browser compatibility:

el6: firefox 38 - ok, google-chrome 27 - no
el7: firefox 38 - ok, google-chrome 46 - ok
ubuntu: firefox 42 - ok

mac os, windows - we say "latest firefox or google-chrome is required", then - ok

So we are probably okey with using javascript Promises with MIDAS...

I shall try to convert the json-rpc client library to promises, see how it shakes out.

K.O.
  1137   18 Nov 2015 Amy RobertsInfosynchronous ajax deprecated
> Why don't you post the functions here so that we can have a look? 

Here is (1) my promisified HTTP request function and (2) a function that uses the returned promises to build an asynchronous, sequential chain of requests to Midas.

Note that if something seems ugly, it's likely because I didn't take the time to clean it up, and not because it particularly *needs* to be ugly.

###### promisified HTTP request ######
In addition to promisifying HTTP requests to Midas, I wanted the Promise.resolve from this function to always return valid JSON.  I also wanted the promise to reject if the response from mhttpd indicated
failure - so that we wouldn't have to rewrite this error checking throughout the code.  The function is so long becuase we make many different calls to mhttpd, and most of them need custom error checking
and, if successful, response packaging.

// begin cdms.daq.utilities.get()
cdms.daq.utilities.get = function(url) {
  return new Promise(function(resolve, reject) {
    // XHR request
    var req = new XMLHttpRequest();
    req.open('GET',url);
 
    req.onload = function() {
      //console.log('done with ', url);
      if(req.status == 200 || req.status == 302) {
        // 'http://dcrc01.triumf.ca:8081/?cmd=jcopy&odb=CustomScript/nonexistent&encoding=json'
        // 'http://dcrc01.triumf.ca:8081/?cmd=jkey&odb=CustomScript/nonexistent'
        // both return <DB_NO_KEY>
        if(/DB_NO_KEY/gi.test(req.response)) {
          reject(req.response);
      
        // 'http://dcrc01.triumf.ca:8081/?cmd=jkey&odb=CustomScript/nonexistent&encoding=json' 
        // returns {"/error": 312}
        } else if(/error.+:\s*\d+/.test(req.response)) {
          reject('<DB_NO_KEY>');
 
        // attempting to start (or stop) a run when a run is already started (or stopped) with
        // 'http://dcrc01.triumf.ca:8081/?cmd=stop' 
        // returns ...<title>MIDAS error</title></head><body><H1>Run is not running</H1>...
        } else if(/MIDAS error/i.test(req.response)) {
          var error_str = /<H1>(.*)<\/H1>/i.exec(req.response);
          reject(error_str[0]);
 
        // 'http://dcrc01.triumf.ca:8081/?cmd=jcreate&odb=/test/foo2&type=7&encoding=json'
        // returns either 1 (creation successful) or 311 (already exists)
        } else if(/jcreate/i.test(url) && (req.response=='1' || req.response=='311')) {
          resolve({'success': true});
 
        // 'http://dcrc01.triumf.ca:8081/?cmd=jset&odb=/test/foo2&val=9&encoding=json'
        // returns OK
        } else if(/jset/i.test(url) && (req.response=='OK')) {
          resolve({'success': true});
 
        // http://dcrc01.triumf.ca:8081/SEQ/?cmd=Load+Script
        // returns an html page showing all available sequencer files
        } else if(/SEQ.*cmd=Load.+Script/i.test(url)) {
          console.log('match Seq load script command url');
          fileList = req.response.match(/\w+.msl/gi);
          resolve({'success': true, 'files': fileList});
 
        // http://dcrc01.triumf.ca:8081/SEQ/?cmd=jmsg&n=10
        // returns 10 most recent messages
        } else if(/.*cmd=jmsg/i.test(url)) {
          console.log('match command url to get messages from mlogger');
          //console.log(req.response);
          resolve({'success': true, 'messages': req.response});
 
        // http://dcrc01.triumf.ca:8081/SEQ/?fs=testflash.msl&dir=&cmd=Load
        // returns <html>redir</html>
        } else if(/SEQ.+fs=\w+.msl&.*cmd=Load/i.test(url)) {
          resolve({'success': true});
 
        // http://dcrc01.triumf.ca:8081/SEQ/?cmd=Start+Script&params=1
        // returns a status of 302 but no response
        } else if(/SEQ.+\?cmd=Start.+Script/i.test(url)) {
          resolve({'success': true});
 
        // http://dcrc01.triumf.ca:8081/?customscript=XYZ&redir=.
        // returns a status of 302 but no response
        // although the redir causes a network call to
        // (in this case) the Midas home page,
        // and that seems to be the req.response
        } else if(/redir/i.test(url)) {
          resolve({'success': true});
 
        // all other responses should be valid JSON
        } else {
          try {
            var json_obj = JSON.parse(req.response);
 
            if(json_obj && typeof json_obj === "object" && json_obj !== null){
              resolve(json_obj);
            }
          } catch(err) {
            console.log('url is ',url);
            console.log('response is ',req.response);
 
            alert(err);
            reject(err);
          }
      }
    } 
    };
     
    req.onerror = function() {
      reject(Error("Network Error"));
    };
 
    req.send();
    //console.log('request to ', url);
  });
}; // end cdms.daq.utilities.get()


###### using promisified HTTP request ######
This is an excerpt that attempts to 
(1) run a script on the DAQ computer, producing a sequencer file
(2) check that the script is completed
(3) load the sequencer file
(4) run the sequencer

Failures on any step jump to the catch, which prints the error on screen.

These are HTTP calls, and given my buggy network should be asynchronous.  At the same time, each of these steps should happen only after its predecessor is completed.  To force sequential execution,
functions within a .then() clause return a Promise object. 

// set the argument string for {{scriptname}}
// then run {{scriptname}}
// check the message log until the success of the script is verified
// check that the sequencer sees the expected file (not implemented!)
// then load the resulting sequencer file
// then start the sequencer
cdms.daq.utilities.get(url).then(function() {
  // call customscript
  var cmd_str = '?customscript=' + scriptName_str + '&redir=.';
  return cdms.daq.utilities.get(baseURL_str + cmd_str);
}).then(function() {
  return checkScriptLoop(uid);
}).then(function() {
  var load_str = baseURL_str + '/SEQ/?cmd=Load+Script';
  console.log('execute promise for url ', load_str);
  return cdms.daq.utilities.get(load_str);
}).then(function() {
  var load_str = '/SEQ/?fs=' + seqFile_str + '&dir=&cmd=Load';
  console.log('execute promise for url ', load_str);
  return cdms.daq.utilities.get(baseURL_str + load_str);
}).then(function() {
  var startSeq_str = '/SEQ/?cmd=Start+Script&params=1';
  console.log('execute promise for url ', startSeq_str);
  return cdms.daq.utilities.get(baseURL_str + startSeq_str);
}).catch(function(error) {
  alert(error);
}).then(function() {
  window.frames['flash-frame'].location = baseURL_str + '/SEQ/';
});
ELOG V3.1.4-2e1708b5