Indeed a bug. Fixed in commit

https://bitbucket.org/tmidas/midas/commits/5c1133df073f493d74d1fc4c03fbcfe80a3edae4

Stefan

> I assume that mlogger rescanning ODB is somewhat intensive process; and that's why we don't want rescanning to 
> happen every time the ODB is changed?

A rescan maybe takes some tens of milliseconds. Something you can do on every run, but not on every ODB change (like writing to the slow control values). 
We would need a somehow more clever code which keeps a copy of the variable names for each equipment. If the names change or the array size changes, 
the scan can be triggered.


> Stopping/restarting mlogger is okay.  But would it be better to have some alternate way to force mlogger to 
> rescan the ODB?  Like an odbedit command like 'mlogger_rescan'; or some magic ODB key to force the rescan.  I 
> guess neither of these options is really any easier for the developer.  It just seems awkward to need to restart 
> mlogger for this.

Indeed. But whatever "new" we design for the scan will users complain "last week it was enough to restart the logger, now what do I have to do". So nothing 
is perfect. But having a button in the ODB editor like "Rebuild history database" might look more elegant. One issue is that it needs special treatment, since 
the logger (in the Mu3e experiment) needs >10s for the scan, so a simple rpc call will timeout.

Let's see what KO has to say on this.

Best,
Stefan

> 1) history is independent from "runs", we see a change, we apply it (even if it takes 10 sec or 2 minutes).
> 
> 2) "nothing should change during a run", we must process all changes before we start a run (starting a run takes forever),
>    and we must ignore changes during a run (i.e. updated frontend starts to write new data to history). (this is why
>    the trick to "start a new run twice" used to work).

"nothing should change during a run" violates the action when a user adds a new variable during a run. So if the user does that, they don't
care if things change during a run. Then we can also modify the history DB during the run. Note that some MIDAS installations are purely
slow control (kind of a replacement of LabView, have no runs at all). In those installations runs do not make sense at all, so keeping the
history independent of runs makes sense to me.

> It is "free" to rescan ODB every 10 second or so. Then we can output a midas message "please restart the logger",
> and set an ODB flag, then when user opens the history panel editor, it will see this flag
> and tell the user "please restart the logger to see the latest changes in history". It can even list
> the specific changes, if we want ot be verbose about it.

Sounds good to me.

> I say, let's take the low road for now and see if it's good enough:
> 
> a) have the history system report any changes in midas.log - "history event added", "new history variable added" (or "renamed"),
>    this will let user see that their changes to the equipment frontend "took" and flag any accidental/unwanted changes.
> 
> b) have mlogger periodically scan ODB and set a "please restart me" flag. observe this flag in the history editor
>    and tell the user "please restart the logger to see latest changes in the history".

Actually you don't have to actively "scan" the ODB. You have hotlinks to the logger anyway from the equipment variables. All we need 
in addition is a hotline to the settings array in the ODB. The logger receives the hotline update, checks if the names changed or got
extended, then flags this as a change.

Stefan

> 2) "advanced" c++ code:
> 
> void foo(const std::string& xxx) { ... };
> int main() { foo("bar"); }
> 
> copy-created 2nd string is avoided, but string object to hold "bar" is still must be 
> made, 1 malloc(), 1 memcpy().

Are you sure about this? I always thought that foo only receives a pointer to xxx which it puts on the stack, so
no additional malloc/free is involved.

Have a look here: https://en.cppreference.com/w/cpp/language/reference.html

It says "References are not objects; they do not necessarily occupy storage".

Stefan

> Now, mlogger just silently continues not writing to history. There is no ongoing error message, there is no 
> ongoing alarm, only sign of trouble is empty history plots and history files not growing.

I would recommend to use an "internal alarm". This is not an "ODB alarm" where values are compared limits, but it is directly triggered by C code. To do 
so, call 

  if (disk_full)
     al_trigger-alarm("Disk full", "Disk full, no history will be written", "Alarm", "Disk full", AT_INTERNAL);

This will cause an alarm to show up prominently on the status page and beep every few minutes.

Stefan

Thanks for reporting this. It was caused by a

  MJsonNode* node = MJsonNode::Parse(str.c_str());

not followed by a 

  delete node;

I added that now in odb::odb_from_json_string(). Can you try again?

Stefan

> BTW, it looks like we are parsing the whole JSON ODB dump (200k+) on every odbxx access. Can you parse it just once?

You are right. I changed the code so that the dump is only parsed once. Please give it a try.

Stefan

> Confirmed fixed, thanks! There are 2 small changes I made in odbxx.h, please pull.

There was one missing enable_jsroot in manalyzer, please pull yourself.

Stefan

After popular demand and during some quite holidays I implemented ad-hoc history plots. To enable this 
for a certain equipment, put 

  /Equipment/<name>/Settings/History buttons = true

into the ODB. You will then see a graph button after each variable. Pressing this button reveals the history 
for this variable, see attachment.

Stefan

Since we have no many RaspberryPi based control systems running at our lab with midas, I want to 
streamline the midas installation, such that a non-expert can install it on these devices. 

First, midas has to be cloned under "midas" in the user's home directory with

git clone https://bitbucket.org/tmidas/midas.git --recurse-submodules

For simplicity, this puts midas right into /home/<user>/midas, and not into any "packages" subdirectory 
which I believe is not necessary.

Then I wrote a setup script midas/midas_setup.sh which does the following:

- Add midas environment variables to .bashrc / .zschenv depending on the shell being used

- Compile and install midas to midas/bin

- Load an initial ODB which allows insecure http access to port 8081

- Install mhttpd as a system service and start it via systemctl

Since I'm not a linux system expert, the current file might be a bit clumsy. I know that automatic shell 
detection can be made much more elaborate, but I wanted a script which can easy be understood even by 
non-experts and adapted slightly if needed.

If you know about shell scripts and linux administration, please have a quick look at the attached script 
and give me any feedback.

Stefan

Thanks for your feedback. I reworked the installation script, and now also called it "install.sh" since it includes also the git clone. I modeled it after 
homebrew a mit (https://brew.sh). This means you can now run the script on a prison linux system with:

  /bin/bash -c "$(curl -sS https://bitbucket.org/tmidas/midas/raw/HEAD/install.sh)"

It contains three defaults for MIDASSYS, MIDAS_DIR and MIDAS_EXPT_NAME, but when you run in, you can overwrite these
defaults interactively. The script creates all directories, clones midas, compiles and installs it, installs and runs mhttpd as a system
service, then starts the logger and the example frontend. I also added your PYTHONPATH variable. The RC file is now automatically
detected.

Yes one could add more config files, but I want to have this basic install as simple as possible. If more things are needed, they
should be added as separate scripts or .ODB files.

Please have a look and let me know what you think about. I tested it on a RaspberryPi, but not yet on other systems.

Stefan

I put the documentation under

  https://daq00.triumf.ca/MidasWiki/index.php/Install_Script

Would be good if anybody could check that.

Stefan

MT_DEBUG messages are there for debugging, not logging. They only go into the SYSMSG buffer and NOT to the log file. If you want anything logged, just use MT_INFO.

Not sure if that's missing in the documentation. Anyhow, there are my original ideas (from 1995 ;-) )

MT_ERROR  
Error message, to be displayed in red

MT_INFO    
 Info or status message

MT_DEBUG 
Only sent to SYSMSG buffer, not to midas.log file. Handy if you produce lots of message and don't want to flood the message file. Plus it does not change the timing of your app, since the SYSMSG buffer is much faster than writing 
to a file.

MT_USER
Message generated interactively by a user, like in the chat window or via the odbedit "msg" command

MT_LOG
Messages with are only logged but not put into the SYSMSG buffer

MT_TALK
Messages which should go through the speech synthesis in the browser and are "spoken"

MT_CALL
Message which would be forwarded to the user via a messaging app (historically this was an actual analog telephone call via a modem ;-) )

If that is missing in the documentation, please feel free to copy/paste it to the appropriate place.


Stefan

Dear all,

I want to control (start / stop / monitor its stdout and stderr) an external process (systemd / EPICS IOC shell script) from within MIDAS.

In order to make this as convenient as possible for the user, I want the process to behave just like any other MIDAS client:
- I can start it from the ODB as a program
- The process gets regularly polled from MIDAS to see whether it is still running
- I can stop the process from the ODB like any other program
- Optional, but highly appreciated: Its stdout and stderr should be a MIDAS message.

Did anyone already solve a similar problem?

Best regards
Stefan

Thanks a lot,

Nick. Regarding the messages: Zaher showed me that it is possible to simply place a custom log file generated by the systemd next to midas.log - then it shows up next to the "midas" tab in "Messages".

One follow-up question: Is it possible to use the systemctl status for the "Running on host" column? Or does this even happen automatically?

Best regards
Stefan

> Hi,
> 
> what you describe is exactly how I normally run mhttpd, mlogger, mserver and some other
> custom frontend programs. Eg:
> 
> [local:T2KGSC:Running]/>ls /programs/Logger/
> Required                        y
> Watchdog timeout                100000
> Check interval                  180000
> Start command                   systemctl --user start mlogger
> Auto start                      n
> Auto stop                       n
> Auto restart                    n
> Alarm class                     AlarmNotify
> First failed                    0
> 
> The only exception is your last point about stdout and stderr
> being midas messages. I use journalctl to see these.
> 
> Cheers,
> 
> Nick.
> 
> > I want to control (start / stop / monitor its stdout and stderr) an external process (systemd / EPICS IOC shell script) from within MIDAS.
> > 
> > In order to make this as convenient as possible for the user, I want the process to behave just like any other MIDAS client:
> > - I can start it from the ODB as a program
> > - The process gets regularly polled from MIDAS to see whether it is still running
> > - I can stop the process from the ODB like any other program
> > - Optional, but highly appreciated: Its stdout and stderr should be a MIDAS message.
> > 
> > Did anyone already solve a similar problem?
> > 
> > Best regards
> > Stefan

Hi,

unfortunately I don't have a documentation link to the feature, I just know that it works on my machine ;-) The general idea is that you place a custom whatever.log file in Logger/Data Dir (where midas.log is stored). Then, in the Messages page, there will be a "midas" tab and a "whatever" tab - the latter showing the content of whatever.log. One problem here is that timestamping does not work automatically - you have to prepend every line with the same Hours:Minutes:Seconds.Milliseconds Year/Month/Day format that midas.log is using.

So you have a custom Programs page which does systemctl status on your systemd? Does the status then transfer over automatically to the Status page? Is there an example how to write such a custom page?

Best regards
Stefan

> Hi,
>  
> > Nick. Regarding the messages: Zaher showed me that it is possible to simply place
> > a custom log file generated by the systemd next to midas.log - then it shows up
> > next to the "midas" tab in "Messages".
> 
> Interesting. I'm not familiar with that feature. Do you have link to documentation?
> 
> > One follow-up question: Is it possible to use the systemctl status for the
> > "Running on host" column? Or does this even happen automatically?
> 
> On the programs page that column is populated by the odb key /System/Clients/<PID>/Host
> so no. However, there is nothing stopping you from writing your own version of
> programs.html to show whatever you want. For example I have a custom programs
> page the includes columns to enable/disable and to reset watchdog alarms.
> 
> Cheers,
> 
> Nick.

Hello!

My name is Soichiro Kuribayashi and I am a Ph.D. student at Kyoto University. 
I'm a T2K collaborator and working for Super FGD which is new detector in ND280.

I'm a beginner of MIDAS and I've just started to develop the DAQ software with 
MIDAS for Super FGD.
For the DAQ of Super FGD, we will run remotely front end part of MIDAS on ZYNQ 
which is system on chip.

For this remote control of front end part with mserver, we have to mount home 
directory of DAQ PC(Cent OS8) on that of Linux on ZYNQ.
So I wonder if we should use NFS(Network file system) + NIS(Network information 
service) + autofs for the mounting. Is it correct?

If you have any information or any suggestion for the remote control on chip, 
please let me know.

Best regards,
Soichiro 

Dear Konstantin,

Thank you very much for your reply and detailed information.
I would appreciate if you could help us.

> I can also suggest that on your frontend SOC/FPGA machine, you boot linux
> using the "nfs-root" method. This way, the local flash memory only
> contains a boot loader (and maybe the linux kernel image, depending on
> bootloader limitations). The rest of the linux rootfs can be on your
> central development machine. This way management of flash cards,
> confusion with different contents of local flash and need to make backups
> of frontend machines is much reduced.

As you said, we can run complete Linux (Ubuntu 16) on ZYNQ and I'm using common NFS 
system now. However, I didn't know "nfs-root" method which you mentioned and this method 
seems to be reasonable way to just share linux rootfs.
First of all, I will try this method for simpler system.

> If you use a fast SSD and ZFS with deduplication, you will also have good
> performance gain (NFS over 1gige network to server with fast SSD works
> so much better compared to the very slow SD/MMC/NAND flash).
>
> I can point you to some of my documentation how we do this.

I'm concerned about such performance and I have checked the performance with common NFS 
over gige network and my DAQ PC roughly(data transfer rate ~ O(10) MByte/sec). However, I 
didn't know the ZFS and also how we can have performance gain with a fast SSD and ZFS.
Please let me know your documentation how to do it if possible.

> I would say you are on a good track. For early development on just one board,
> pretty much any way you do it will work, but once you start scaling up
> beyound 3-4-5 frontends, you will start seeing benefits from common NFS-mounted
> home directories, NFS-root booted linux, etc.

I'm developing with just one board and common NFS-mounted now. I'm looking forward to 
seeing such benefits when I will use multiple frontends.
 
> And of course you may want to study the existing ND280/FGD DAQ. I hope you
> have access to the running system at Jparc. If not, I have a copy of
> pretty much everything (except for running hardware, it is stored in the basement, 
> dead) and I can give you access.

I don't have access to the system at Jparc, but Nick has told us where FGD DAQ code is.
Is bellow URL everything of code of FGD DAQ?
https://git.t2k.org/hastings/fgddaq/-/tree/master

Best regards,
Soichiro

Dear Stefan,

Thank you very much for your help.

I have already contacted someone who has used ZYNQ in that order and It's working fine for now.
But, I'll let you know if something goes wrong.

Best regards,
Soichiro