Thanks for fixing these long lasting bugs. The code is much cleaner now, please
commit it.

We switched from the history files from MIDAS to FILE, so we have *.dat files now (per variable), instead of the old *.hst. 

How shoul
d one now extract data from these data files? With the old *,hst files I can e.g. mhdump -E 102 231010.hst 

but with the new *.dat files I get

...2023/history$ mhdump -E 0 -T "Run number" mhf_1697445335_20231016_run_transitions.dat | head -n 15
event name: [Run transitions], time [1697445335]
tag: tag: /DWORD 1 4 /timestamp
tag: tag: UINT32 1 4 State
tag: tag: UINT32 1 4 Run number
record size: 12, data offset: 1024
record 0, time 1697557722, incr 112387
record 1, time 1697557783, incr 61
record 2, time 1697557804, incr 21
record 3, time 1697557834, incr 30
record 4, time 1697557888, incr 54
record 5, time 1697558318, incr 430
record 6, time 1697558323, incr 5
record 7, time 1697558659, incr 336
record 8, time 1697558668, incr 9
record 9, time 1697558753, incr 85

not very intelligible

Yes, I can do csv export on the webpage. But it would be nice to be able to extract from just the files. Also, the webpage export only saves the data shown ( range limited and/or downsampled)

$ cat mhf_1697445335_20231016_run_transitions.dat
event name: [Run transitions], time [1697445335]
tag: tag: /DWORD 1 4 /timestamp
tag: tag: UINT32 1 4 State
tag: tag: UINT32 1 4 Run number
record size: 12, data offset: 1024
...

data is in fixed-length record format. from the file header, you read "record size" is 12 and data starts at offset 1024.

the 12 bytes of the data record are described by the tags:
4 bytes of timestamp (DWORD, unix time)
4 bytes of State (UINT32)
4 bytes of "Run number" (UINT32)

endianess is "local endian", which means "little endian" as we have no big-endian hardware anymore to test endian conversions.

file format is designed for reading using read() or mmap().

and you are right mhdump, does not work on these files, I guess I can write another utility that does what I just described and spews the numbers to stdout.

K.O.

> $ cat mhf_1697445335_20231016_run_transitions.dat
> event name: [Run transitions], time [1697445335]
> tag: tag: /DWORD 1 4 /timestamp
> tag: tag: UINT32 1 4 State
> tag: tag: UINT32 1 4 Run number
> record size: 12, data offset: 1024
> ...
> 
> data is in fixed-length record format. from the file header, you read "record size" is 12 and data starts at offset 1024.
> 
> the 12 bytes of the data record are described by the tags:
> 4 bytes of timestamp (DWORD, unix time)
> 4 bytes of State (UINT32)
> 4 bytes of "Run number" (UINT32)
> 
> endianess is "local endian", which means "little endian" as we have no big-endian hardware anymore to test endian conversions.
> 
> file format is designed for reading using read() or mmap().
> 
> and you are right mhdump, does not work on these files, I guess I can write another utility that does what I just described and spews the numbers to stdout.
> 
> K.O.


Thanks for the answer. As this FILE system is advertised as the new default (eog:2617), this format does merit some more WIKI info.

> The JSON RPC branch has been merged into main MIDAS.

The interface is now mostly documented, go here: https://midas.triumf.ca/MidasWiki/index.php/Mjsonrpc

Documentation for individual javascript functions in mhttpd.js not merged into the MIDAS documentation yet, because the API is being converted to the Javascript Promise 
pattern (git branch feature/js_promise).

The functions available from mhttpd.js are documented via doxygen, also linked from the mjsonrpc wiki page.

K.O.

Dear MIDAS Experts,

I'm having an odd problem when trying to display an integer stored in ODB on a custom 
web page:  the hex specifier, "%x", displays integers as if it were "%d" .

- attachment 1 shows the layout and the contents of the ODB sub-tree in question
- attachment 2 shows the web page as it is displayed
- attachment 3 shows the snippet of html/js producing the web page

I bet I'm missing smth trivial - an advice is greatly appreciated! 

Also, is there an equivalent of a "0x%04x" specifier to have the output formatted 
into a fixed length string ?  

-- thanks, regards, Pasha  

Thanks for reporting that bug. I fixed it and committed the change to the develop branch.

Stefan

Hi Stefan (and Ben),

thanks for reacting so promptly - your commits on Bitbucket fixed the problem.

For those of us who knows little about how the web browsers work: 

- picking up the fix required flushing the cache of the MIDAS client web browser - apparently the web browser 
  I'm using - Firefox 115.6 - cached the old version of midas.js but wouldn't report it cached and wouldn't load 
  the updated file on its own.

-- thanks again, regards, Pasha

> odb.c rev 4622 fixes ODB corruption by db_connect_database() if client_name is
> too long. Also fixed is potential ODB corruption by too long key names in
> db_create_key(). Problem kindly reported by Tim Nichols of T2K/ND280 experiment.


Related bug fix - db_connect_database() should not permit client names that contain
the slash (/) character. Names like "aaa/bbb" create entries /Programs/aaa/bbb (aaa
is a subdirectory) and names like "../aaa" create entries in the ODB root directory.

svn rev 4623.
K.O.

Hi

I had a problem with the compilation of midas after an OS update to the recent version of OpenSuse tumbleweed. The function stime() in system.cxx:3196 is no longer available. 

In the documentation it is also marked as deprecated with the suggestion to use clock_settime instead:
https://man7.org/linux/man-pages/man2/stime.2.html

replacing system.cxx:3196 with the clock_settime - method in system.cxx:3200 - 3204 also for OS_UNIX seems to solve the problem, but i'm not sure if this will cause problems on older OS's.

Martin

The function stime() has been replaced by clock_settime() on Feb. 2020:

https://bitbucket.org/tmidas/midas/commits/c732120e7c68bbcdbbc6236c1fe894c401d9bbbd

Please always pull before submitting bug reports.

Best,
Stefan

I encountered many warning while building MIDAS (svn r4556). Please see the
attached log file. Most of them are caused by type conversion from string to
"char*".
Though I can ignore all the warning without any problem, I still hate to see
them. :-)

> I encountered many warning while building MIDAS (svn r4556). Please see the
> attached log file. Most of them are caused by type conversion from string to
> "char*".
> Though I can ignore all the warning without any problem, I still hate to see
> them. :-)

There is no "type conversions". The compiler is whining about code like this:

/* data type names */
static char *tid_name[] = {
   "NULL",
   "BYTE",
   ...

I guess we should keep the compiler happy and make them "static const char*".

BTW, my compiler is SL5.2 gcc-4.1.2 and it does not complain. What's your compiler?

K.O.

> There is no "type conversions". The compiler is whining about code like this:
> 
> /* data type names */
> static char *tid_name[] = {
>    "NULL",
>    "BYTE",
>    ...
> 
> I guess we should keep the compiler happy and make them "static const char*".
> 
> BTW, my compiler is SL5.2 gcc-4.1.2 and it does not complain. What's your compiler?
> 
> K.O.

Using built-in specs.
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 4.3.4-2' --with-
bugurl=file:///usr/share/doc/gcc-4.3/README.Bugs --enable-
languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --enable-shared --enable-multiarch -
-enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib --without-included-
gettext --enable-threads=posix --enable-nls --with-gxx-include-dir=/usr/include/c++/4.3 
--program-suffix=-4.3 --enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc --
enable-mpfr --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --
host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.3.4 (Debian 4.3.4-2)

Hi,

I have noticed a problem with deleting a key to an array in odb, then
recreating the record as in the code below. The record is recreated
successfully, but when viewing it with mhttpd, a spurious blank line
(coloured orange) is visible, followed by the rest of the data as normal.

This blank line causes trouble with experim.h because it
produces an empty structure e.g. :

#define CYCLE_SCALERS_SETTINGS_DEFINED

typedef struct {
  struct {
  } ;
  char      names[60][32];
} CYCLE_SCALERS_SETTINGS;

rather than :

#define CYCLE_SCALERS_SETTINGS_DEFINED

typedef struct {
  char      names[60][32];
} CYCLE_SCALERS_SETTINGS;


This empty structure causes a compilation error when rebuilding clients that
use experim.h

SD



 CYCLE_SCALERS_TYPE1_SETTINGS_STR(type1_str);
 CYCLE_SCALERS_TYPE2_SETTINGS_STR(type2_str);

Both type1_str and type2_str have been defined as in
experim.h
i.e.
#define CYCLE_SCALERS_TYPE1_SETTINGS_STR(_name) char *_name[] = {\
"[.]",\
"Names = STRING[60] :",\
"[32] Back%BSeg00",\
"[32] Back%BSeg01",\
 ........
 ........
"[32] General%NeutBm Cycle Sum",\
"[32] General%NeutBm Cycle Asym",\
"",\
NULL }

#define CYCLE_SCALERS_TYPE2_SETTINGS_STR(_name) char *_name[] = {\
"[.]",\
"Names = STRING[60] :",\
"[32] Back%BSeg00",\
"[32] Back%BSeg01",\
...........
............
"[32] General%B/F Cumul -",\
"[32] General%Asym Cumul -",\
"",\
NULL }

  if (db_find_key(hDB, 0, "/Equipment/Cycle_scalers/Settings/",&hKey) ==
DB_SUCCESS)
    db_delete_key(hDB,hKey,FALSE);
          
  if (  strncmp(fs.input.experiment_name,"1",1) == 0) {
      exp_mode = 1; /* Imusr type - scans */
      status =
db_create_record(hDB,0,"/Equipment/Cycle_scalers/Settings/",strcomb(type1_str));
    }
  else {
    exp_mode = 2; /* TDmusr types - noscans */
    status =
db_create_record(hDB,0,"/Equipment/Cycle_scalers/Settings/",strcomb(type2_str));
  }

> I have noticed a problem with deleting a key to an array in odb, then
> recreating the record as in the code below. The record is recreated
> successfully, but when viewing it with mhttpd, a spurious blank line
> (coloured orange) is visible, followed by the rest of the data as normal.
> 
> db_create_record(hDB,0,"/Equipment/Cycle_scalers/Settings/",strcomb(type1_str));
>     }
>   else {
>     exp_mode = 2; /* TDmusr types - noscans */
>     status =
> db_create_record(hDB,0,"/Equipment/Cycle_scalers/Settings/",strcomb(type2_str));
>   }

The first problem is that the db_create_record has a trailing "/" in the key name 
after Settings. This causes the (empty) subsirectory which causes your trouble. 
Simple removing it fixes the problem. I agree that this is not obvious, so I 
added some code in db_create_record() which removes such a trailing slash if 
present. New version under CVS.

Second, the db_create_record() call is deprecated. You should use the new 
function db_check_record() instead, and remove your db_delete_key(). This avoids 
possible ODB trouble since the structure is not re-created each time, but only 
when necessary.

- Stefan

When using the sequencer to start and stop runs which use a deferred transition,
the sequencer fails with a "Cannot stop run: ..." error.

Checking for the "CM_DEFERRED_TRANSITION" case in the first-pass block of the
'Stop' code in sequencer.cxx is one way to solve the problem - though there may
well be better solutions. 

My edited portion of sequencer.cxx is below.  Is this an acceptable solution
that could be introduced to the master branch?

} else if (equal_ustring(mxml_get_value(pn), "Stop")) {
    if (!seq.transition_request) {
       seq.transition_request = TRUE;
       size = sizeof(state);
       db_get_value(hDB, 0, "/Runinfo/State", &state, &size, TID_INT, FALSE);
       if (state != STATE_STOPPED) {
          status = cm_transition(TR_STOP, 0, str, sizeof(str), TR_MTHREAD |
TR_SYNC, TRUE);
          if (status == CM_DEFERRED_TRANSITION) {
             // do nothing
          } else if (status != CM_SUCCESS) {
             sprintf(str, "Cannot stop run: %s", str);
             seq_error(str);
          }
       }
    } else {
       // Wait until transition has finished
       size = sizeof(state);
       db_get_value(hDB, 0, "/Runinfo/State", &state, &size, TID_INT, FALSE);
       if (state == STATE_STOPPED) {
          seq.transition_request = FALSE;

          if (seq.stop_after_run) {
             seq.stop_after_run = FALSE;
             seq.running = FALSE;
             seq.finished = TRUE;
             cm_msg(MTALK, "sequencer", "Sequencer is finished.");
          } else
             seq.current_line_number++;

          db_set_record(hDB, hKeySeq, &seq, sizeof(seq), 0);
       } else {
         // do nothing
       }
    }
}

Hi,

I'm facing some issues with 'stop' deferred transition and I suspect of
a MIDAS bug regarding this...

to reproduce the issue I use the 'deferredfe' MIDAS example (develop branch), 
changing only the equipment name from 'Deferred' to 'Deferred%02d' in order
to be able to run multiple 'deferredfe' instances;

I run *three* 'deferredfe' frontends using:

./deferredfe -i 0
./deferredfe -i 2
./deferredfe -i 3

Everything goes fine on MIDAS web page and 'deferredfe' frontends are initialized
and ready to run; issues occour after 'start' when I stop the frontends: sometimes
at first shot and sometimes at next 'start'/'stop' the deferred 'stop' transition
seems to be handled in wrong way... and often one frontend goes in 'segmentation fault'

The odd thing is when I run *two* instances: in this case no issues are reported...

Thanks in advance,
Gennaro

Deferred transitions were only implemented with a single instance of a program deferring the 
transition. To have several instances, MIDAS probably needs to be extended. Certainly this 
was never tested, so it's not a surprise that we get a segmentation fault.

Stefan

Hi Stefan,

so if I have two different frontends (feov1725 and feodt5751) connected on the same 'mserver'
I'm in the same situation ?

Cheers,
Gennaro

> Deferred transitions were only implemented with a single instance of a program deferring the 
> transition. To have several instances, MIDAS probably needs to be extended. Certainly this 
> was never tested, so it's not a surprise that we get a segmentation fault.
> 
> Stefan