Changelog

The midas git repository uses tags to denote specific releases. Tags are of the format midas-YEAR-MONTH-SUFFIX, e.g. midas-2019-06-b. This page details the major changes in each release, and highlights how to update client code to adapt to any changes that are not backwards-compatible.

2020-12

Release midas-2020-12-a

Relevant elog entry - 2089 (midas-2020-12-a)

Improvements

• New ODB variable /Experiment/Enable sound can be used to globally prevent mhttpd from playing sounds.
• Lazylogger now supports writing data over SFTP.
• modbvalue elements on custom pages now support an onload() callback as well as onchange(). Most elements now also support a data-validate callback.
• Custom pages can now tie a select drop-down box to an ODB value using modbselect.
• Ability to choose whether the code or the current ODB values take precendence for the "Common" settings of an equipment when starting a frontend. See elog thread 2014 for more details, and the "Upgrade guide" below for instructions.
• Minor improvements to mdump program - support for 64-bit data types and ability to load larger events if needed.
• Minor improvements to History plots and Buffers webpage.
• Various bug fixes

Upgrade guide

Updating midas

cd $MIDASSYS
git pull
git checkout midas-2020-12-a
git submodule update
cd build
make install

Updating experiment frontends

Any frontends that are written in the MFE framework need to have an extra variable declared. This dictates whether the "Common" settings for an equipment get overwritten by the defaults specified in the equipment listing part of the code when the frontend is started. The old behaviour was that ODB values are never overwritten by the values in the code.

To keep the old behaviour, add this line to your frontend code (somewhere near the equipment listing, in the global scope):

BOOL equipment_common_overwrite = false;

If you do not add a line like this, your compiler will complain that the variable is undefined.

2020-08

Release midas-2020-08-a

Relevant elog entry - 1987 (midas-2020-08-a)

Improvements

• C++ ODB interface (think of it like a "magic" std::map that syncs changes with the ODB)
• Image history for logging webcam images and displaying timelapses
• Much improved history plots
• Sequencer page is now javascript-based and more responsive
• UTF-8 clean ODB (complains if any TID_STRING is invalid UTF-8)
• mhttpd updated to mongoose 6.16 with much improved mulththreading
• mhttpd updated to use MBEDTLS in preference to problematic OpenSSL
• MidasConfig.cmake contributed by Mathieu Guigue
• Various bug fixes

Upgrade guide

cd $MIDASSYS
git pull
git checkout midas-2020-08-a
git submodule update
cd build
make install

An ODB key has been deprecated / replaced, and you will be warned about this when you restart mlogger:

• /Logger/Message file has been replaced by /Logger/Message dir and /Logger/Message file date format. If your "Message file" used to just say "midas.log", then you can just delete the "Message file" key and leave the new entries blank. If you had a more customised value, then read the linked documentation to see how to use the new keys.

2020-03

Release midas-2020-03-a

Relevant elog entry - 1854 (midas-2020-03-a)

Improvements

• Python library wrapping the C++ library, with framework for writing clients and frontends.
• New ODB tree /Webserver that gathers all mhttpd settings in one place, including the new ability for mhttpd to act as a proxy to other web servers
• New javascript-based sequencer page (via the "NewSequencer" link in the mhttpd menu)
• Various bug fixes

Upgrade guide

cd $MIDASSYS
git pull
git checkout midas-2020-08-a
git submodule update
cd build
make install

Then, restart mhttpd and configure the new /Webserver ODB directory to your liking.

2019-09

Releases midas-2019-09-a through midas-2019-09-i

Relevant elog entries - 1706 (midas-2019-09), 1747 (midas-2019-09-e), 1749 (midas-2019-09-g) and 1750 (midas-2019-09-i)

Improvements

• New javascript-based history plots (old image-based plots are available via the "OldHistory" link in the mhttpd menu)
• Reduced memory and CPU usage of mhttpd web pages
• Latest version of mxml
• Support for MySQL 8+, which removed the my_global.h header
• Various bug fixes

Bug fixes

• 187 - drivers/bus/rs232.cxx does not compile after transition to C++
• 186 - drivers/bus/tcpip.cxx does not compile after transition to C++
• 130 - Mysql::Prepare() doesn't recover from MySQL connection going away
• 188 - Shouldn't memset structs containing std::string

Upgrade guide

If updating from a version before 2019-06, take special note of the upgrade instructions for that version - you will have to update your client code to adapt to C++.

# Grab the new code
cd $MIDASSYS
git checkout develop
git pull
git checkout midas-2019-09-a
git submodule update
git pull
# Finally, update mxml by updating the submodules
git submodule update --recursive
cd build 
cmake ..
make
make install

2019-06

Releases midas-2019-06-a, midas-2019-06-b.

Relevant elog entries - 1564 (midas-2019-06 with cmake and c++) and 1526 (How to convert C midas frontends to C++).

Improvements

• Migration from C to C++. You will have to make changes to your clients (see upgrade guide below).
• Ability to compile using cmake/cmake3 as well as make. To use cmake, you can either build manually with mkdir build; cd build; cmake ..; make; make install, or use the handy shortcut make cmake3. Note that the location where libraries and executables are built has changed - the OS-specific subdirectories (e.g. /linux/lib) has been replaced by a common /lib and /bin.
• mxml and mscb are now included as git submodules. See the "upgrade guide" instructions for how to checkout the latest version of these modules.

Bug fixes

• 183 - mhttpd could write garbage to midas.log, which would break webpages that try to show messages
• 181 - race condition between msequencer and mhttpd could make sequencer webpage buttons useless
• 171 - make RPC calls more thread-safe
• 180 - addition of strcomb1 which is thread-safe; strcomb will be deprecated
• 179 - don't try to send an ODB dump to mlogger's ROOT output, as it would crash mlogger
• 184 - don't trap the user in an error loop if elog doesn't exist
• Minor fixes and stability improvements

Known issues

• cmake/cmake3 - ZLIB support is not detected, so gzipped files cannot be written by the logger. Will be fixed in the next release, or you can update midas to a commit after August 2.
• mxml can segfault due to a double free. Will be fixed in the next release, or you can update mxml to commit f6fc49d: cd mxml; git checkout f6fc49d; cd .. and re-compile midas.

Upgrade guide

Updating midas

# Grab the new code
cd $MIDASSYS
git checkout develop
git pull
git checkout midas-2019-06-b
git submodule update
git pull
git submodule update --init # this will checkout correct versions of mxml and mscb

# Tidy up the old build - be sure to delete the deprecated linux directory!
make clean
make cclean
rm -rf linux/bin
rm -rf linux/lib
rmdir linux

# Build the new midas
mkdir build 
cd build 
cmake ..
make
make install

If you have a script that sets up environment variables, you should change PATH from $MIDASSYS/linux/bin to $MIDASSYS/bin.

You should then restart mserver, mlogger, mhttpd, and any other midas programs that you run.

Cleanup old packages that are now submodules

As mxml and mscb are included as submodules now, you can remove the external packages that were downloaded previously (assuming they aren't used by other experiments on the same machine). E.g.

rm -r $HOME/packages/mxml # new location $MIDASSYS/mxml
rm -r $HOME/packages/mscb # new location $MIDASSYS/mscb

Update experiment frontends

The migration from C to C++ is one of the biggest user-facing changes in midas for a long time. Unfortunately it requires manual work from experimenters to update their client code:

1. Update your Makefile
   1. Update library search path for code that links against libmidas or mfe.o ($MIDASSYS/linux/lib becomes $MIDASSYS/lib)
   2. If you reference mxml in your Makefile, change the include path to $MIDASSYS/mxml
   3. If you explicitly have the compiler as gcc, change it to g++
2. Update frontend code to use mfe.h and build as C++
   1. Add #include "mfe.h" after including midas.h
   2. Remove extern C brackets around mfe-related code. Ideally there should be no extern C brackets anywhere.
   3. Ensure that frontend_name and frontend_file_name are const char* rather than char*.
   4. If you define your own global HNDLE hDB, change it to extern HNDLE hDB to pick up the one provided by mfe.
   5. Ensure that poll_event and interrupt_configure() use INT rather than INT[] for the source argument.
   6. If you use extern int frontend_index, change it to use the get_frontend_index() function from mfe.h instead.
   7. Ensure that the last argument to bk_create is cast to (void**)
3. Try to compile, and fix any more compilation errors. Examples may include:
   1. Duplicate or mismatched declarations of functions defined in mfe.h (fix the mismatched declarations in your code)
   2. bool debug colliding with declaration in mfe.h (suggest to rename the variable in the client)
   3. Return value of malloc() etc needs to be cast to the correct data type (e.g. char* s = (char*)malloc(...))

An example diff of a frontend is:

  #include "midas.h"
+ #include "mfe.h"
  #include "msystem.h"
  #include "utils1.h"

- #ifdef __cplusplus
- extern "C" {
- #endif

- char *frontend_name = FE_NAME;
+ const char *frontend_name = FE_NAME;

- HNDLE hDB;
+ extern HNDLE hDB;

- #ifdef __cplusplus
- }
- #endif

- extern "C" int frontend_index;

  INT frontend_init() {
-   printf("We are running as frontend index %d", frontend_index);
+   printf("We are running as frontend index %d", get_frontend_index());
    return SUCCESS;
  }

- extern "C" INT interrupt_configure(INT cmd, INT[] source, PTYPE adr) {
+ INT interrupt_configure(INT cmd, INT source, PTYPE adr) {
   return 0;
  }

2019-05

Releases midas-2019-05-cxx and midas-2019-05-before-cmake were development tags that are not intended to be used more widely.

2019-03

Releases midas-2019-03-f through midas-2019-03-h.

Relevant elog entries: 1513 (midas-2019-03-f), 1530 (midas-2019-03-g), 1543 (midas-2019-03-h).

Improvements

• ODB and event buffer access is now fully thread-safe
• Stability improvements

Bug fixes

• 99 - Protect against recursive calls to db_lock_database() that could cause corruption
• 173 - Fix redirection from old URL scheme to new scheme
• 167 - Avoid RPC timeouts for clients that have already been closed
• 115 - Better error messages if events are too big

Upgrade guide

cd $MIDASSYS
git pull
git checkout midas-2019-03-h
make clean
make

2019-02

Releases midas-2019-02-a and midas-2019-02-b.

Improvements

• Format of ODB dumps in midas files can now be specified using new ODB key /Logger/Channels/<N>/Settings/ODB dump format. Note that the default dump format is now json; if your analysis tools parse the ODB dump at the start/end of your midas files, you may want to change the setting to xml.
• Location of most recent end-of-run ODB dump can be set using new ODB key /Logger/ODB Last Dump File.
• New mhttpd_exec_script() javascript function to execute a custom script.
• Rationalised and more secure URL scheme for webpages served by mhttpd
• Conversion of many midas pages to more responsive design
• Stability and usability improvements

Bug fixes

• 58 - Reduce data rate of test programs
• 153 - Simplified creation of webpage aliases in the /Alias tree
• 136 - Fix bug in db_validate_open_records() and 'sor' command in odbedit tht could cause an abort

Upgrade guide

cd $MIDASSYS
git pull
git checkout midas-2019-02-b
make clean
make

2018-12

Release midas-2018-12-a.

2017-10

Releases midas-2017-10-a.

2017-07

Releases midas-2017-07-a through midas-2017-07-c.