| ID |
Date |
Author |
Topic |
Subject |
|
3148
|
27 Nov 2025 |
Stefan Ritt | Bug Report | Error(?) in custom page documentation |
Indeed a bug. Fixed in commit
https://bitbucket.org/tmidas/midas/commits/5c1133df073f493d74d1fc4c03fbcfe80a3edae4
Stefan |
|
3149
|
27 Nov 2025 |
Stefan Ritt | Suggestion | Improve process for adding new variables that can be shown in history plots |
> I assume that mlogger rescanning ODB is somewhat intensive process; and that's why we don't want rescanning to
> happen every time the ODB is changed?
A rescan maybe takes some tens of milliseconds. Something you can do on every run, but not on every ODB change (like writing to the slow control values).
We would need a somehow more clever code which keeps a copy of the variable names for each equipment. If the names change or the array size changes,
the scan can be triggered.
> Stopping/restarting mlogger is okay. But would it be better to have some alternate way to force mlogger to
> rescan the ODB? Like an odbedit command like 'mlogger_rescan'; or some magic ODB key to force the rescan. I
> guess neither of these options is really any easier for the developer. It just seems awkward to need to restart
> mlogger for this.
Indeed. But whatever "new" we design for the scan will users complain "last week it was enough to restart the logger, now what do I have to do". So nothing
is perfect. But having a button in the ODB editor like "Rebuild history database" might look more elegant. One issue is that it needs special treatment, since
the logger (in the Mu3e experiment) needs >10s for the scan, so a simple rpc call will timeout.
Let's see what KO has to say on this.
Best,
Stefan |
|
3157
|
27 Nov 2025 |
Stefan Ritt | Suggestion | Improve process for adding new variables that can be shown in history plots |
> 1) history is independent from "runs", we see a change, we apply it (even if it takes 10 sec or 2 minutes).
>
> 2) "nothing should change during a run", we must process all changes before we start a run (starting a run takes forever),
> and we must ignore changes during a run (i.e. updated frontend starts to write new data to history). (this is why
> the trick to "start a new run twice" used to work).
"nothing should change during a run" violates the action when a user adds a new variable during a run. So if the user does that, they don't
care if things change during a run. Then we can also modify the history DB during the run. Note that some MIDAS installations are purely
slow control (kind of a replacement of LabView, have no runs at all). In those installations runs do not make sense at all, so keeping the
history independent of runs makes sense to me.
> It is "free" to rescan ODB every 10 second or so. Then we can output a midas message "please restart the logger",
> and set an ODB flag, then when user opens the history panel editor, it will see this flag
> and tell the user "please restart the logger to see the latest changes in history". It can even list
> the specific changes, if we want ot be verbose about it.
Sounds good to me.
> I say, let's take the low road for now and see if it's good enough:
>
> a) have the history system report any changes in midas.log - "history event added", "new history variable added" (or "renamed"),
> this will let user see that their changes to the equipment frontend "took" and flag any accidental/unwanted changes.
>
> b) have mlogger periodically scan ODB and set a "please restart me" flag. observe this flag in the history editor
> and tell the user "please restart the logger to see latest changes in the history".
Actually you don't have to actively "scan" the ODB. You have hotlinks to the logger anyway from the equipment variables. All we need
in addition is a hotline to the settings array in the ODB. The logger receives the hotline update, checks if the names changed or got
extended, then flags this as a change.
Stefan |
|
3158
|
27 Nov 2025 |
Stefan Ritt | Suggestion | mvodb WS and family type matching |
> 2) "advanced" c++ code:
>
> void foo(const std::string& xxx) { ... };
> int main() { foo("bar"); }
>
> copy-created 2nd string is avoided, but string object to hold "bar" is still must be
> made, 1 malloc(), 1 memcpy().
Are you sure about this? I always thought that foo only receives a pointer to xxx which it puts on the stack, so
no additional malloc/free is involved.
Have a look here: https://en.cppreference.com/w/cpp/language/reference.html
It says "References are not objects; they do not necessarily occupy storage".
Stefan |
|
3167
|
03 Dec 2025 |
Stefan Ritt | Suggestion | Improve process for adding new variables that can be shown in history plots |
> Now, mlogger just silently continues not writing to history. There is no ongoing error message, there is no
> ongoing alarm, only sign of trouble is empty history plots and history files not growing.
I would recommend to use an "internal alarm". This is not an "ODB alarm" where values are compared limits, but it is directly triggered by C code. To do
so, call
if (disk_full)
al_trigger-alarm("Disk full", "Disk full, no history will be written", "Alarm", "Disk full", AT_INTERNAL);
This will cause an alarm to show up prominently on the status page and beep every few minutes.
Stefan |
|
3176
|
09 Dec 2025 |
Stefan Ritt | Bug Report | odbxx memory leak with JSON ODB dump |
Thanks for reporting this. It was caused by a
MJsonNode* node = MJsonNode::Parse(str.c_str());
not followed by a
delete node;
I added that now in odb::odb_from_json_string(). Can you try again?
Stefan |
|
3180
|
10 Dec 2025 |
Stefan Ritt | Bug Report | odbxx memory leak with JSON ODB dump |
> BTW, it looks like we are parsing the whole JSON ODB dump (200k+) on every odbxx access. Can you parse it just once?
You are right. I changed the code so that the dump is only parsed once. Please give it a try.
Stefan |
|
3184
|
11 Dec 2025 |
Stefan Ritt | Bug Report | odbxx memory leak with JSON ODB dump |
> Confirmed fixed, thanks! There are 2 small changes I made in odbxx.h, please pull.
There was one missing enable_jsroot in manalyzer, please pull yourself.
Stefan |
|
3187
|
04 Jan 2026 |
Stefan Ritt | Info | Ad-hoc history plots of slow control equipment |
After popular demand and during some quite holidays I implemented ad-hoc history plots. To enable this
for a certain equipment, put
/Equipment/<name>/Settings/History buttons = true
into the ODB. You will then see a graph button after each variable. Pressing this button reveals the history
for this variable, see attachment.
Stefan |
| Attachment 1: Screenshot_2026-01-04_at_14.41.55.png
|
|
|
3188
|
09 Jan 2026 |
Stefan Ritt | Forum | MIDAS installation |
Since we have no many RaspberryPi based control systems running at our lab with midas, I want to
streamline the midas installation, such that a non-expert can install it on these devices.
First, midas has to be cloned under "midas" in the user's home directory with
git clone https://bitbucket.org/tmidas/midas.git --recurse-submodules
For simplicity, this puts midas right into /home/<user>/midas, and not into any "packages" subdirectory
which I believe is not necessary.
Then I wrote a setup script midas/midas_setup.sh which does the following:
- Add midas environment variables to .bashrc / .zschenv depending on the shell being used
- Compile and install midas to midas/bin
- Load an initial ODB which allows insecure http access to port 8081
- Install mhttpd as a system service and start it via systemctl
Since I'm not a linux system expert, the current file might be a bit clumsy. I know that automatic shell
detection can be made much more elaborate, but I wanted a script which can easy be understood even by
non-experts and adapted slightly if needed.
If you know about shell scripts and linux administration, please have a quick look at the attached script
and give me any feedback.
Stefan |
| Attachment 1: midas_setup.sh
|
#!/bin/sh
#
# Change environment
#
f="$HOME/.bashrc"
[ -f "$HOME/.zshenv" ] && f="$HOME/.zshenv"
grep -q '>>> MIDAS >>>' "$f" 2>/dev/null || cat >>"$f" <<'EOF'
# >>> MIDAS >>>
export PATH="$HOME/midas/bin:$PATH"
export MIDASSYS="$HOME/midas"
export MIDAS_DIR="$HOME/online"
export MIDAS_EXPT_NAME="Online"
# <<< MIDAS <<<
EOF
. "$f"
printf '\033[1;33m★\033[0m %s\n' "Environment variables written to $f"
#
# Compile MIDAS
#
printf '\033[1;33m★\033[0m %s\n' "Compiling and installing MIDAS"
mkdir -p "$HOME/midas/build"
cd "$HOME/midas/build"
cmake ..
make install
cd "$HOME/midas"
#
# Load initial ODB, opens port 8081 for mhttpd
#
printf '\033[1;33m★\033[0m %s\n' "Loading initial ODB"
mkdir -p "$HOME/online"
odbedit -c "load midas_setup.odb" > /dev/null
#
# Installing mhttpd servcice
#
printf '\033[1;33m★\033[0m %s\n' "Installing mhttpd service"
sudo cp mhttpd.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable mhttpd
sudo systemctl start mhttpd
printf '\033[1;33m★\033[0m %s\n' "Finished MIDAS setup"
|
|
3190
|
13 Jan 2026 |
Stefan Ritt | Forum | MIDAS installation |
Thanks for your feedback. I reworked the installation script, and now also called it "install.sh" since it includes also the git clone. I modeled it after
homebrew a mit (https://brew.sh). This means you can now run the script on a prison linux system with:
/bin/bash -c "$(curl -sS https://bitbucket.org/tmidas/midas/raw/HEAD/install.sh)"
It contains three defaults for MIDASSYS, MIDAS_DIR and MIDAS_EXPT_NAME, but when you run in, you can overwrite these
defaults interactively. The script creates all directories, clones midas, compiles and installs it, installs and runs mhttpd as a system
service, then starts the logger and the example frontend. I also added your PYTHONPATH variable. The RC file is now automatically
detected.
Yes one could add more config files, but I want to have this basic install as simple as possible. If more things are needed, they
should be added as separate scripts or .ODB files.
Please have a look and let me know what you think about. I tested it on a RaspberryPi, but not yet on other systems.
Stefan |
| Attachment 1: install.sh
|
#!/bin/sh
#
# This is a MIDAS install script which installes MIDAS and sets up
# a proper environment for a simple experiment
#
# On a new system you can execut it with
#
# /bin/bash -c "$(curl -sS https://bitbucket.org/tmidas/midas/raw/HEAD/install.sh)"
#
#
# Default directories
#
DEFAULT_MIDASSYS="$HOME/midas"
DEFAULT_MIDAS_DIR="$HOME/online"
DEFAULT_MIDAS_EXPT_NAME="Online"
#
# Query directories
#
# ---- MIDASSYS ----
printf "Clone MIDAS into directory [%s]: " "$DEFAULT_MIDASSYS"
read USER_MIDASSYS
# If user entered something, it must start with a /
if [ -n "$USER_MIDASSYS" ] && [ "${USER_MIDASSYS#/}" = "$USER_MIDASSYS" ]; then
echo "Error: directory must be an absolute path starting with '/'"
exit 1
fi
# Use default if user input is emtpy
MIDASSYS=${USER_MIDASSYS:-$DEFAULT_MIDASSYS}
# ---- MIDAS_DIR ----
printf "MIDAS experiment directory [%s]: " "$DEFAULT_MIDAS_DIR"
read USER_MIDAS_DIR
# If user entered something, it must start with a /
if [ -n "$USER_MIDAS_DIR" ] && [ "${USER_MIDAS_DIR#/}" = "$USER_MIDASDIR" ]; then
echo "Error: directory must be an absolute path starting with '/'"
exit 1
fi
# Use default if user input is emtpy
MIDAS_DIR=${USER_MIDAS_DIR:-$DEFAULT_MIDAS_DIR}
# ---- MIDAS_EXPT_NAME ----
printf "MIDAS experiment name [%s]: " "$DEFAULT_MIDAS_EXPT_NAME"
read USER_MIDAS_EXPT_NAME
# Use default if user input is emtpy
MIDAS_EXPT_NAME=${USER_MIDAS_EXPT_NAME:-$DEFAULT_MIDAS_EXPT_NAME}
echo "\n---------------------------------------------------------------"
printf '\033[1;33m*\033[0m %s\n' "MIDAS system directory : $MIDASSYS"
printf '\033[1;33m*\033[0m %s\n' "MIDAS experiment directory : $MIDAS_DIR"
printf '\033[1;33m*\033[0m %s\n' "MIDAS experiment name : $MIDAS_EXPT_NAME"
#
# Change environment
#
detect_rc_file() {
# $SHELL is usually reliable
case "$(basename "$SHELL")" in
bash)
[ -f "$HOME/.bashrc" ] && echo "$HOME/.bashrc" || echo "$HOME/.bash_profile"
;;
zsh)
# zsh always reads .zshenv
echo "$HOME/.zshenv"
;;
fish)
# fish is not POSIX, but handle gracefully
echo "$HOME/.config/fish/config.fish"
;;
*)
# fallback for sh, dash, etc.
echo "$HOME/.profile"
;;
esac
}
RC_FILE=$(detect_rc_file)
grep -q '>>> MIDAS >>>' "$RC_FILE" 2>/dev/null || cat >>"$RC_FILE" <<'EOF'
# >>> MIDAS >>>
export PATH="__MIDASSYS__/bin:$PATH"
export MIDASSYS="__MIDASSYS__"
export MIDAS_DIR="__MIDAS_DIR__"
export MIDAS_EXPT_NAME="__MIDAS_EXPT_NAME__"
export PYTHONPATH="$PYTHONPATH:__MIDASSYS__/python"
# <<< MIDAS <<<
EOF
# substitue placeholders
sed -i.bak \
-e "s|__MIDASSYS__|$MIDASSYS|g" \
-e "s|__MIDAS_DIR__|$MIDAS_DIR|g" \
-e "s|__MIDAS_EXPT_NAME__|$MIDAS_EXPT_NAME|g" \
"$RC_FILE"
# source environment file
. "$RC_FILE"
printf '\033[1;33m*\033[0m %s\n' "Environment variables written to : $RC_FILE"
echo "---------------------------------------------------------------\n"
printf '\033[1;33m*\033[0m %s\n\n' "Cloning and compiling MIDAS..."
# create experiment directory
mkdir -p "$MIDAS_DIR"
# clone MIDAS
git clone https://bitbucket.org/tmidas/midas.git "$MIDASSYS" --recurse-submodules
# compile MIDAS
mkdir -p "$MIDASSYS/build"
cd "$MIDASSYS/build"
cmake .. -DCMAKE_BUILD_TYPE=RelWithDebInfo
make install
#
# Load initial ODB, opens port 8081 for mhttpd
#
printf '\n\033[1;33m*\033[0m %s\n' "Loading initial ODB"
cd "$MIDASSYS"
odbedit -c "load install.odb" > /dev/null
# start example frontend and logger
printf '\033[1;33m*\033[0m %s\n' "Starting frontend and logger"
$MIDASSYS/build/examples/experiment/frontend -D 1>/dev/null
$MIDASSYS/bin/mlogger -D 1>/dev/null
#
# Installing mhttpd servcice
#
printf '\033[1;33m*\033[0m %s\n' "Installing mhttpd service"
sudo cp mhttpd.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable mhttpd
printf '\033[1;33m*\033[0m %s\n' "Starting mhttpd service"
sudo systemctl start mhttpd
printf '\033[1;33m*\033[0m %s\n' "Finished MIDAS setup"
|
|
3191
|
13 Jan 2026 |
Stefan Ritt | Forum | MIDAS installation |
I put the documentation under
https://daq00.triumf.ca/MidasWiki/index.php/Install_Script
Would be good if anybody could check that.
Stefan |
|
3193
|
14 Jan 2026 |
Stefan Ritt | Bug Report | DEBUG messages not showing and related |
MT_DEBUG messages are there for debugging, not logging. They only go into the SYSMSG buffer and NOT to the log file. If you want anything logged, just use MT_INFO.
Not sure if that's missing in the documentation. Anyhow, there are my original ideas (from 1995 ;-) )
MT_ERROR
Error message, to be displayed in red
MT_INFO
Info or status message
MT_DEBUG
Only sent to SYSMSG buffer, not to midas.log file. Handy if you produce lots of message and don't want to flood the message file. Plus it does not change the timing of your app, since the SYSMSG buffer is much faster than writing
to a file.
MT_USER
Message generated interactively by a user, like in the chat window or via the odbedit "msg" command
MT_LOG
Messages with are only logged but not put into the SYSMSG buffer
MT_TALK
Messages which should go through the speech synthesis in the browser and are "spoken"
MT_CALL
Message which would be forwarded to the user via a messaging app (historically this was an actual analog telephone call via a modem ;-) )
If that is missing in the documentation, please feel free to copy/paste it to the appropriate place.
Stefan |
|
3195
|
20 Jan 2026 |
Stefan Ritt | Info | New tabbed custom pages |
Tabbed custom pages have been implemented in MIDAS. Below you see and example. The documentation
is here:
https://daq00.triumf.ca/MidasWiki/index.php/Custom_Page#Tabbed_Pages
Stefan |
| Attachment 1: tabbed_page.png
|
|
|
3197
|
23 Jan 2026 |
Stefan Ritt | Info | Homebrew support for midas |
Hi Mathieu,
thanks for your contribution. Have you looked at the install.sh script I developed last week:
https://daq00.triumf.ca/MidasWiki/index.php/Install_Script
which basically does the same, plus it modifies the environment and installs mhttpd as a service.
Actually I modeled the installation after the way Homebrew is installed in the first place (using curl).
I wonder if the two things can kind of be integrated. Would be great to get with brew always the newest midas version, and it would also
check and modify the environment.
If you tell me exactly what is wrong MidasConfig.cmake.in I'm happy to fix it.
Best,
Stefan |
|
3199
|
26 Jan 2026 |
Stefan Ritt | Info | Homebrew support for midas |
> Actually, these two approaches are slightly different I guess:
> - the installation script you are linking manages the
> installation and the subsequent steps, but doesn't manage the dependencies: for instance on my machine, it didn't find root and so manalyzer
> is built without root support.
> Maybe this is just something to adapt?
Yes indeed. From your perspective, you probably always want ROOT with MIDAS. But at PSI here we have several installation where we do not
need ROOT. These are mainly beamline control PCs which just connect to EPICS or pump station controls replacing Labview installations. All
graphics there is handled with the new mplot graphs which is better in some case.
I therefore added a check into install.sh which tells you explicitly if ROOT is found and included or not. Then it's up to the user to choose to
install ROOT or not.
> Brew on the other hand manages root and so knows how to link these two
> together.
If you really need it, yes.
> - The nice thing I like about brew is that one can "ship bottles" aka compiled version of the code; it is great and fast for
> deployment and avoid compilation issues.
> - I like that your setup does deploy and launch all the necessary executables ! I know brew can do
> this too via brew services (see an example here: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/r/rabbitmq.rb#L83 ), maybe worth
> investigating...?
Indeed this is an advantage of brew, and I wholeheartedly support it therefore. If you decide to support this for the midas
community, I would like you to document it at
https://daq00.triumf.ca/MidasWiki/index.php/Installation
Please talk to Ben <bsmith@triumf.ca> who manages the documentation and can give you write access there. The downside is that you will
then become the supporter for the brew and all user requests will be forwarded to you as long as you are willing to maintain the package ;-)
> - Brew relies on code tagging to better manage the bottles, so that it uses the tag to get a well-defined version of the
> code and give a name to the version.
> I had to implement my own tags e.g. midas-mod-2025-12-a to get a release.
> I am not sure how to do in the
> case of midas where the tags are not that frequent...
Yes we always struggle with the tagging (what is a "release", when should we release, ...). Maybe it's the simplest if we tag once per month
blindly with midas-2026-02a or so. In the past KO took care of the tagging, he should reply here with his thoughts.
> Thank you for the feedback, I will make the modifications (aka naming my formula
> ``midas-mod'') so that it doesn't collide with a future official midas one.
Nope. The idea is that YOU do the future official midas realize from now on ;-)
> Concerning the MidasConfig.cmake issue, this is what I need ...
Let's take this offline not to spam others.
Best,
Stefan |
|
3120
|
19 Nov 2025 |
Stefan Mathis | Forum | Control external process from inside MIDAS |
Dear all,
I want to control (start / stop / monitor its stdout and stderr) an external process (systemd / EPICS IOC shell script) from within MIDAS.
In order to make this as convenient as possible for the user, I want the process to behave just like any other MIDAS client:
- I can start it from the ODB as a program
- The process gets regularly polled from MIDAS to see whether it is still running
- I can stop the process from the ODB like any other program
- Optional, but highly appreciated: Its stdout and stderr should be a MIDAS message.
Did anyone already solve a similar problem?
Best regards
Stefan |
|
3122
|
20 Nov 2025 |
Stefan Mathis | Forum | Control external process from inside MIDAS |
Thanks a lot,
Nick. Regarding the messages: Zaher showed me that it is possible to simply place a custom log file generated by the systemd next to midas.log - then it shows up next to the "midas" tab in "Messages".
One follow-up question: Is it possible to use the systemctl status for the "Running on host" column? Or does this even happen automatically?
Best regards
Stefan
> Hi,
>
> what you describe is exactly how I normally run mhttpd, mlogger, mserver and some other
> custom frontend programs. Eg:
>
> [local:T2KGSC:Running]/>ls /programs/Logger/
> Required y
> Watchdog timeout 100000
> Check interval 180000
> Start command systemctl --user start mlogger
> Auto start n
> Auto stop n
> Auto restart n
> Alarm class AlarmNotify
> First failed 0
>
> The only exception is your last point about stdout and stderr
> being midas messages. I use journalctl to see these.
>
> Cheers,
>
> Nick.
>
> > I want to control (start / stop / monitor its stdout and stderr) an external process (systemd / EPICS IOC shell script) from within MIDAS.
> >
> > In order to make this as convenient as possible for the user, I want the process to behave just like any other MIDAS client:
> > - I can start it from the ODB as a program
> > - The process gets regularly polled from MIDAS to see whether it is still running
> > - I can stop the process from the ODB like any other program
> > - Optional, but highly appreciated: Its stdout and stderr should be a MIDAS message.
> >
> > Did anyone already solve a similar problem?
> >
> > Best regards
> > Stefan |
|
3129
|
20 Nov 2025 |
Stefan Mathis | Forum | Control external process from inside MIDAS |
Hi,
unfortunately I don't have a documentation link to the feature, I just know that it works on my machine ;-) The general idea is that you place a custom whatever.log file in Logger/Data Dir (where midas.log is stored). Then, in the Messages page, there will be a "midas" tab and a "whatever" tab - the latter showing the content of whatever.log. One problem here is that timestamping does not work automatically - you have to prepend every line with the same Hours:Minutes:Seconds.Milliseconds Year/Month/Day format that midas.log is using.
So you have a custom Programs page which does systemctl status on your systemd? Does the status then transfer over automatically to the Status page? Is there an example how to write such a custom page?
Best regards
Stefan
> Hi,
>
> > Nick. Regarding the messages: Zaher showed me that it is possible to simply place
> > a custom log file generated by the systemd next to midas.log - then it shows up
> > next to the "midas" tab in "Messages".
>
> Interesting. I'm not familiar with that feature. Do you have link to documentation?
>
> > One follow-up question: Is it possible to use the systemctl status for the
> > "Running on host" column? Or does this even happen automatically?
>
> On the programs page that column is populated by the odb key /System/Clients/<PID>/Host
> so no. However, there is nothing stopping you from writing your own version of
> programs.html to show whatever you want. For example I have a custom programs
> page the includes columns to enable/disable and to reset watchdog alarms.
>
> Cheers,
>
> Nick. |
|
2004
|
13 Oct 2020 |
Soichiro Kuribayashi | Info | About remote control of front end part of MIDAS on chip |
Hello!
My name is Soichiro Kuribayashi and I am a Ph.D. student at Kyoto University.
I'm a T2K collaborator and working for Super FGD which is new detector in ND280.
I'm a beginner of MIDAS and I've just started to develop the DAQ software with
MIDAS for Super FGD.
For the DAQ of Super FGD, we will run remotely front end part of MIDAS on ZYNQ
which is system on chip.
For this remote control of front end part with mserver, we have to mount home
directory of DAQ PC(Cent OS8) on that of Linux on ZYNQ.
So I wonder if we should use NFS(Network file system) + NIS(Network information
service) + autofs for the mounting. Is it correct?
If you have any information or any suggestion for the remote control on chip,
please let me know.
Best regards,
Soichiro |