Tonight we got another error of that type after the update:

04:17 - [mhttpd,ERROR] [history_schema.cxx:2913:FileHistory::read_data,ERROR] Cannot read 
'/data2/history/mhf_1692128214_20230815_gassystem.dat', read() errno 2 (No such file or directory)

This morning I looked at the file, and it was there:

[meg@megon02 history]$ ls -alg mhf_1692128214_20230815_gassystem.dat
-rw-rw-r--. 1 meg 4663228 Aug 17 08:50 mhf_1692128214_20230815_gassystem.dat
[meg@megon02 history]$


Stefan

> > This feature was asked by some people ...
> 
> "show notifications" popups are strongly associated with disreputable web sites (presumably to 
> push spam), it was surprising to see it from midas.
> 
> K.O.

I agree. But unlike emails (where you get lots of spam as well), you can nicely blacklist/whitelist 
desktop notifications. I suppress all of them except the one for MIDAS. This allows me to watch our 
experiment without staring on the web page all the time.

The main question here is maybe if the desktop notification should be on or off by default (for a 
fresh browser). While you always can change that via the mhttpd "Config" page, the default value is 
chosen by the system. I thought I put it to "on" so people can experience it, and then turn it off if 
they don't like. Having them off by default, most people never would notice this possibility. But I'm 
open to a discussion here.

Stefan

Confirmed. The error message is wrong. It is printed after a short read(), but short read() does not 
set errno, and errno reported by the error message is from some previous syscall. Corrected error 
message is already committed. K.O.


> Tonight we got another error of that type after the update:
> 
> 04:17 - [mhttpd,ERROR] [history_schema.cxx:2913:FileHistory::read_data,ERROR] Cannot read 
> '/data2/history/mhf_1692128214_20230815_gassystem.dat', read() errno 2 (No such file or directory)
> 
> This morning I looked at the file, and it was there:
> 
> [meg@megon02 history]$ ls -alg mhf_1692128214_20230815_gassystem.dat
> -rw-rw-r--. 1 meg 4663228 Aug 17 08:50 mhf_1692128214_20230815_gassystem.dat
> [meg@megon02 history]$
> 
> 
> Stefan

> > > Our default configuration of apache httpd logs every request.
> > > MIDAS custom web pages can easily make a huge number of RPC calls creating a 
> > > huge log file and filling system disk to 100% capacity.
> added "daily" to /etc/logrotate.d/httpd, default was "weekly", not often enough.

this should fix it good, make /var/log bigger:

[root@mpmt-test ~]# df
Filesystem      1K-blocks       Used  Available Use% Mounted on
/dev/sdc2        52403200   52296356     106844 100% /
[root@mpmt-test ~]# 
[root@mpmt-test ~]# xfs_growfs /
data blocks changed from 13107200 to 106367750
[root@mpmt-test ~]# 
[root@mpmt-test ~]# df
Filesystem      1K-blocks       Used  Available Use% Mounted on
/dev/sdc2       425445400   52300264  373145136  13% /

K.O.

Still get the same error with the latest version:

3:28 [mhttpd,ERROR] [history_schema.cxx:2913:FileHistory::read_data,ERROR] Cannot read 
'/data2/history/mhf_1692391703_20230818_hv_tc.dat', read() errno 2 (No such file or directory)

Stefan

The epics frontend overwrites the "start command" odb after each start:

  // set start command in ODB
   midas::odb efe("/Programs/EPICS Frontend");
   std::string p(__FILE__);
   std::string s("build/epics_fe");
   auto i = p.find("epics_fe.cxx");
   p.replace(i, s.length(), s);
   p = p.substr(0, i + s.length());
   efe["Start command"].set_string_size(p, 256);

this should be set such that it only writes when the key is not there. It causes the following issue: on a pc with multiple experiments defined, you need to start the fe's with a "-e <name>" flag. 

Thanks for reporting this problem. It has been fixed today, so the start command is only written if it's emtpy.

Stefan

Hi,

I was playing with MIDAS devel branch and I realized that
if I set an ODB INT32 key to a value using new ODB web interface 
it is reported in parenthesis always as (0xFFFFFFFF);

I tested with different browser and result is the same while this 
never happens in OldODB web interface...

Cheers,
Gennaro

Thanks for reporting this bug, I fixed it in the last commit.

Best,
Stefan

Hi,

the method get_eor_odb_dump() of midas.file_reader does not contain an
initial jump_to_start() and this is a problem if the following access
pattern is used:

---

mfile = midas.file_reader.MidasFile("run00008.mid.lz4")
begin_odb = mfile.get_bor_odb_dump().data

# loop on data events
...

end_odb = mfile.get_eor_odb_dump().data

---

in this case the script ends with a RuntimeError (Unable to find EOR event) and
force user to do a manual mfile.jump_to_start() before mfile.get_eor_odb_dump();

Thanks,
Gennaro

> Still get the same error with the latest version:
> 3:28 [mhttpd,ERROR] [history_schema.cxx:2913:FileHistory::read_data,ERROR] Cannot read 
> '/data2/history/mhf_1692391703_20230818_hv_tc.dat', read() errno 2 (No such file or directory)

I figured it out. I claim defense of temporary insanity and old age senility.

1) I added the "short read" check in one place, missed the second place
2) writes of history were meant to be atomic, and they are atomic in my head, but not in the midas 
code:

history_schema.cxx:HsFileSchema::write_event()
...
   status = write(s->writer_fd, &t, 4);
   if (status != 4) {
      cm_msg(MERROR, "FileHistory::write_event", "Cannot write to \'%s\', write(timestamp) errno 
%d (%s)", s->file_name.c_str(), errno, strerror(errno));
      return HS_FILE_ERROR;
   }

   status = write(s->writer_fd, data, expected_size);
   if (status != expected_size) {
      cm_msg(MERROR, "FileHistory::write_event", "Cannot write to \'%s\', write(%d) errno %d 
(%s)", s->file_name.c_str(), data_size, errno, strerror(errno));
      return HS_FILE_ERROR;
   }
...

that's not atomic, that's two separate writes. history reader hits the history file between the 
two writes and gets a short read of 4 bytes timestamp instead of full record size. that's the 
error message reported by mhttpd.

two fixes forthcoming:
a) check for short read in the 2nd place that I missed
b) two write() are replaced by 2 memcpy() to a preallocated buffer and 1 write()

Overall, I am pretty happy that this is the only bug in the FILE history code found in N years, 
and it does not even cause data corruption...

K.O.

> two fixes forthcoming:
> a) check for short read in the 2nd place that I missed
> b) two write() are replaced by 2 memcpy() to a preallocated buffer and 1 write()

commit 713ec4a583365d57ffcd700ceeb09dcc14518295

K.O.

Thanks for the bug report Gennaro! 

I've fixed the code so that we'll now find the end-of-run ODB dump even if the user is already at the end of the file when they call get_eor_odb_dump().

Ben

Hello,

Since commit bc227a8a34def271a598c0200ca30d73223c3373 I've been getting the compilation error below (on a Raspberry Pi 3 Model B Plus Rev 1.3).

The fix is obvious from the reported error, but I am wondering whether this should be fixed in the main git??

Thanks,

Zaher

> git pull
> git submodule update

confirmed. just run into this myself. I think "make" should warn about out of 
date git modules. Also check that the build git version is tagged with "-dirty".

K.O.

> > git pull
> > git submodule update
> 
> confirmed. just run into this myself. I think "make" should warn about out of 
> date git modules. Also check that the build git version is tagged with "-dirty".
> 
> K.O.

The submodule business becomes kind of annoying. I updated the documentation at

https://daq00.triumf.ca/MidasWiki/index.php/Quickstart_Linux#MIDAS_Package_Installatio
n

to tell people to use 

1) "git clone ... --recurse-submodules" during the first clone
2) "git submodule update --init --recursive" in case they forgot 1)
3) "git pull --recurse-submodules" for each update or to use
4) "git config submodule.recurse true" to make the --recurse-submodules the default

I use 4) since a while and it works nicely, so one does not have to remember to pull 
recursively each time.

Stefan

Hi,

I like the new eqtable, but stumbled over some issues.

1) In the attached snapshot you see that the values shown from our vacuum Pirani and Penning cells are all zero, which of course is not true.
It would be nice to have under the equipment settings some formatting options, like the possibility to add units.

2) If one of the number evaluates to Infinity, the table is not shown properly anymore.

Best,
  Andreas

> 1) In the attached snapshot you see that the values shown from our vacuum Pirani and Penning cells are all zero, which of course is not true.
> It would be nice to have under the equipment settings some formatting options, like the possibility to add units.

You have a 

  /Equipment/LEMVAC/Settings/Format Input

array where you can specify the format for every value. Default is "%f2" for two digits after the period. For vacuum levels you might want to 
consider "%e3" which give you exponential format with three significant digits. The "format" setting is described at

  https://daq00.triumf.ca/MidasWiki/index.php//Equipment_ODB_tree#Format_%3Cvariable%3E

and the details are at

  https://daq00.triumf.ca/MidasWiki/index.php/Custom_Page#Formatting

The was a bug with the format handling, so please pull the current develop branch.


> 2) If one of the number evaluates to Infinity, the table is not shown properly anymore.

I fixed that as well in the current version.

Best,
Stefan

Great! This is it.
Sorry that I missed it in the docu.

Best,
  Andreas