One addition I would be in favour of is to remove the "Order" and replace it with drag&drop handles, because this is what people are more 
used to today. Only the old guys like us remember the /etc/init.d/xx_yy scheme where one uses an integer number in the file name to 
determine an order. 

See for example: https://jsbin.com/hijetos/edit?js,output

But instead of relying on a foreign library, I would rather implement that myself, since I need the same thing later for the to-be-
implemented ODB editor (next year? next lockdown?)

Stefan

> --- Makefile
> < 	ld -shared -o $@ $^ $(LIBS) -lc
> ---
> > 	$(CXX) -shared -o $@ $^ $(LIBS) -lc

Will do. We also have a long standing request to change shared library name from lidmidas.so to libmidas-shared.so.

Different name for the .so file will permit us to build the shared library by default, but still link all MIDAS executables
with the static libmidas.a.

(there is no benefit from linking MIDAS executables - mlogger, mhttpd, etc - with the shared library,
and there is a significant cost in confusion from version skew between the executables and shared
libraries - I have had enough midnight calls "why did odbedit stop working? Oh, who changed LD_LIBRARY_PATH
and why is it now binding against this obsolete libmidas.so left over from 2 years ago?").

For user applications you can do whatever, but for MIDAS core applications I strongly suggest that they
be linked to the midas static library.

K.O.

> > > 	$(CXX) -shared -o $@ $^ $(LIBS) -lc

Done.
MIDAS shared library renamed from libmidas.so to libmidas-shared.so and always build on Linux (.so) and MacOS (.dylib and .so bundle).

Users who wish to link their applications to the shared library should change their Makefile from "gcc ... -lmidas" to "gcc ... -lmidas-shared".

MIDAS core applications (mhttpd, mlogger, etc) are linked to the static library to permit multiple versions of midas to be used
at the same time (LD_LIBRARY_PATH should point to the shared library for *which* midas?!?), avoid problems with wrong setting
of LD_LIBRARY_PATH and to avoid problems with version skew (which we found unavoidable once a midas daq is used for a year or
more - main reason we gave up on using the midas shared library in the first place).

svn rev 4594
K.O.

> While reviving midas on midtig01 after it was not used for a while ... 
> [local:tigress:S]/>scl -w
> Name                Host                Timeout    Last called
> mhttpd              midtig01.triumf.ca  10000      -2037131082

These clients cannot be deleted. I tried:
1) shutdown from mhttpd "programs" page -> "cannot shutdown client"
2) "sh mhttpd" from odbedit -> 
   [midas.c:5298:cm_shutdown] cannot connect to client mhttpd on host
   midtig01.triumf.ca, port 32853
   Client mhttpd not active
3) in odbedit: "cd /system/clients; rm xxxx"
   refuses to delete the key

Lacking any better ideas, I deleted them via brain surgery on the odb file:
1) stop everything
2) ipcrm the SYSV shared memory segment
3) odbedit -> save xxx.odb
4) xemacs xxx.odb, delete offending odb entries
5) rm .ODB.SHM
6) odbedit -> load xxx.odb
7) voila, bad clients gone, gone, gone.

K.O.

> 1) shutdown from mhttpd "programs" page -> "cannot shutdown client"
> 2) "sh mhttpd" from odbedit -> 
>    [midas.c:5298:cm_shutdown] cannot connect to client mhttpd on host
>    midtig01.triumf.ca, port 32853
>    Client mhttpd not active
> 3) in odbedit: "cd /system/clients; rm xxxx"
>    refuses to delete the key

Have you tried a "cleanup" in ODBEdit?

The "last_activity" is a 32-bit int, filled with milliseconds. So indeed it 
wraps around after about one month. So if a all clients are stopped 
simultaneously the hard way (such that nobody's watchdog can clean any other 
client from the ODB), like with a power off, and you start the thing one 
month later, there might be a problem. I never tried that before. So next 
time to a cleanup. If that does not help, we should change last_activity 
from INT to DWORD. This way it's alway positive and the wraparound does not 
hurt.

> > 1) shutdown from mhttpd "programs" page -> "cannot shutdown client"
> Have you tried a "cleanup" in ODBEdit?

Nope. Will try next time...

> The "last_activity" is a 32-bit int, filled with milliseconds. So indeed it 
> wraps around after about one month.... change last_activity 
> from INT to DWORD. This way it's alway positive and the wraparound does not 
> hurt.

INT == "int", wraparound in 1 month
DWORD == "unsigned int", wraparound in 2 months

should we make it the 64-bit "long long" (or C98's "int64_t")?

K.O.

> INT == "int", wraparound in 1 month
> DWORD == "unsigned int", wraparound in 2 months
> 
> should we make it the 64-bit "long long" (or C98's "int64_t")?

Won't work on all supported compilers. The point is that DWORD wraps around in 
2 months, but the difference of two DWORDs is alywas positive, never negative 
like you had it. We only have to distinguish if the difference of the current 
time (im ms) minus the last_activity of a client is larget than the timeout, 
typically 10 seconds or so. If you have a wraparound on 32-bit DWORD, the 
difference is still ok. Like

current "time" : 0x0000 0100
last_activity:   0xFFFF FF00

then current_time - last_activity = 0x00000100 - 0xFFFFFF00 = 0x00000200 if 
calculated with 32-bit values.

Indeed the ODB command "cleanup" really works. I have used it several
times with the TWIST DAQ and regularly with the BNMR/MUSR setups where
we have these stubborn clients (ie feepics) that do not want to shutdown
cleanly.  
But there is one problem with "cleanup". It has a hardwired timeout of
2 seconds.  This is a problem for tasks like lazylogger which set a timeout
of 60 seconds when moving the tape. So BEWARE, if you issue the "cleanup"
command, it might kill some clients who have setup their timeout to longer
than 2 seconds. 

I have asked Stefan to change this before. He said that, to be effective,
the timeout value used for "cleanup" has to be rather short. 
One possibility, would be to allow for a user entered "cleanup" timeout.
The default could stay at 2 seconds. 




> > Have you tried a "cleanup" in ODBEdit?
> 
> Nope. Will try next time...
> 

> But there is one problem with "cleanup". It has a hardwired timeout of
> 2 seconds.  This is a problem for tasks like lazylogger which set a timeout
> of 60 seconds when moving the tape. So BEWARE, if you issue the "cleanup"
> command, it might kill some clients who have setup their timeout to longer
> than 2 seconds. 
> 
> I have asked Stefan to change this before. He said that, to be effective,
> the timeout value used for "cleanup" has to be rather short. 
> One possibility, would be to allow for a user entered "cleanup" timeout.
> The default could stay at 2 seconds. 

I changed the behaviour of cleanup by adding an extra parameter 
ignore_timeout to cm_cleanup(). Now, in ODBEdit, a "clanup" obeys the 
timeout set by the clients. The problem with that is if the logger crashes 
for example, and it's timeout is set o 5 min., it cannot be clean-up'ed any 
more for the next five minutes, and therefor not be restarted wasting 
precious beam time. That's why I hard-wired originally the "cleanup" timout 
to 2 sec. Now I added a flag "-f" to the ODBEdit cleanup command which works 
in the old fashion with a 2 sec. timeout. So a "cleanup" alone won't kill a 
looger which currently rewinds a tape or so, but a "cleanup -f" does.

I also changed internal timeouts from INT to DWORD, which should fix the 
problem Konstantin reported recently (re-starting an experiment after 
several weeks). New changes are commited, but I only did basic tests. So 
please try the new code and tell me if there is any problem.

- Stefan

the day has arrived when I cannot git push to bitbucket. cloud computing rules!

I have never seen this error before and I do not think we have any hooks installed,
so it must be some bitbucket stuff. their status page says some kind of maintenance
is happening, but the promised error message is "repository is read only" or something 
similar.

I hope this clears out automatically. I am updating all the cmake crud and I have no idea 
which changes I already pushed and which I did not, so no idea if anything will work for 
people who pull from midas until this problem is cleared out.

daq00:mvodb$ git push
X11 forwarding request failed on channel 0
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Delta compression using up to 12 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (2/2), 247 bytes | 247.00 KiB/s, done.
Total 2 (delta 1), reused 0 (delta 0)
remote: null value in column "attempts" violates not-null constraint
remote: DETAIL:  Failing row contains (13586899, 2021-07-10 01:13:28.812076+00, 1970-01-01 
00:00:00+00, 1970-01-01 00:00:00+00, 65975727, null).
To bitbucket.org:tmidas/mvodb.git
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'git@bitbucket.org:tmidas/mvodb.git'
daq00:mvodb$

K.O.

The default el6 (RHEL/SL/CentOS-6) compiler is gcc-4.4.7, it does not support c++11, not even a little bit.

Do this to install newer c++ compilers and build MIDAS with c++11:

ssh root@sl6machine
# yum install centos-release-scl-rh
# yum install devtoolset-8
# yum install cmake3
# scl -l
devtoolset-8
...

$ ssh user@sl6machine
$ scl enable devtoolset-8 bash
$ gcc -v
COLLECT_LTO_WRAPPER=/opt/rh/devtoolset-8/root/usr/libexec/gcc/x86_64-redhat-linux/8/lto-wrapper
gcc version 8.3.1 20190311 (Red Hat 8.3.1-3) (GCC) 
$ cd git/midas
$ make cclean
$ make cmake3
$ ls -l bin/odbedit

K.O.

> The default el6 (RHEL/SL/CentOS-6) compiler is gcc-4.4.7, it does not support c++11, not even a little bit.

The previously posted instructions are incomplete - one cannot cross-compile 32-bit executables (i.e. for running on 32-bit VME 
processors) because 64-bit packages are missing 4 files for the 32-bit C++ standard library (libstdc++_nonshared.a).

After a bit of searching I found the missing files, i.e. here:
https://copr-be.cloud.fedoraproject.org/results/mayeut/devtoolset-8/epel-6-i386/01045166-devtoolset-8-gcc/

There are 2 options:

a) install the 32-bit development package:
rpm -vh --install https://ladd00.triumf.ca/~olchansk/devtoolset-8/devtoolset-8-libstdc++-devel-8.3.1-3.1.el6.i686.rpm

b) install just the 4 missing files from here:
https://ladd00.triumf.ca/~olchansk/devtoolset-8/i686-redhat-linux/8/
into
/opt/rh/devtoolset-8/root/usr/lib/gcc/i686-redhat-linux/8/

After doing this, "make linux32" builds. (requires latest midas-2019-09 for minor Makefile fixes)

K.O.


> 
> Do this to install newer c++ compilers and build MIDAS with c++11:
> 
> ssh root@sl6machine
> # yum install centos-release-scl-rh
> # yum install devtoolset-8
> # yum install cmake3
> # scl -l
> devtoolset-8
> ...
> 
> $ ssh user@sl6machine
> $ scl enable devtoolset-8 bash
> $ gcc -v
> COLLECT_LTO_WRAPPER=/opt/rh/devtoolset-8/root/usr/libexec/gcc/x86_64-redhat-linux/8/lto-wrapper
> gcc version 8.3.1 20190311 (Red Hat 8.3.1-3) (GCC) 
> $ cd git/midas
> $ make cclean
> $ make cmake3
> $ ls -l bin/odbedit
> 
> K.O.

bug in init
-----------

I used the lv.c class driver, combined with a custom device driver, to control 
our Keithley2611B source meter. This to set negative voltage on Si detectors.

In the 'init' routing, the class driver sets the hv:

  hv_info->demand_mirror[i] = MIN(hv_info->demand[i], hv_info->voltage_limit[i]);

This fails for negative voltage, as it sets the (negative) voltage limit, instead 
of the demand voltage. A simple 'fabs' solves this.

suggestion for 'idle'
---------------------

I let the device do the ramping, not the driver. This also means I have to reset 
the state of the device (current limit) after ramping. The easiest way to to 
this, is using CMD_IDLE of the device driver. This is currently not done in the 
hv.c class driver.

Hi, Frederick, this is my personal opinion on the slow controls hv classes, I have
used them a couple of times and I found them full of little buglets like this,
plus some incomplete functions, plus some missing features, plus it is all
written in C trying to do object oriented programming. On the balance my opinion
is that it is less work to write a high voltage control program in C++ from scratch
using the regular midas frontend infrastructure compared to having to understand
the hv class driver, write the missing bits, fix the little buglets, debug
the crashes in the C string handling, and what not. (For example I had to debug
mysterious failures to pass float and double values through the C stdarg interface,
there are more fun things to do out there).

K.O.

> bug in init
> -----------
> 
> I used the lv.c class driver, combined with a custom device driver, to control 
> our Keithley2611B source meter. This to set negative voltage on Si detectors.
> 
> In the 'init' routing, the class driver sets the hv:
> 
>   hv_info->demand_mirror[i] = MIN(hv_info->demand[i], hv_info->voltage_limit[i]);
> 
> This fails for negative voltage, as it sets the (negative) voltage limit, instead 
> of the demand voltage. A simple 'fabs' solves this.
> 
> suggestion for 'idle'
> ---------------------
> 
> I let the device do the ramping, not the driver. This also means I have to reset 
> the state of the device (current limit) after ramping. The easiest way to to 
> this, is using CMD_IDLE of the device driver. This is currently not done in the 
> hv.c class driver.

> bug in init
> -----------
> 
> I used the lv.c class driver, combined with a custom device driver, to control 
> our Keithley2611B source meter. This to set negative voltage on Si detectors.
> 
> In the 'init' routing, the class driver sets the hv:
> 
>   hv_info->demand_mirror[i] = MIN(hv_info->demand[i], hv_info->voltage_limit[i]);
> 
> This fails for negative voltage, as it sets the (negative) voltage limit, instead 
> of the demand voltage. A simple 'fabs' solves this.
> 
> suggestion for 'idle'
> ---------------------
> 
> I let the device do the ramping, not the driver. This also means I have to reset 
> the state of the device (current limit) after ramping. The easiest way to to 
> this, is using CMD_IDLE of the device driver. This is currently not done in the 
> hv.c class driver.

I can't find the line you quote in the class driver. Why don't you make a git pull request
and I will approve it.

The original idea behind the hv driver is that all voltages in the ODB and the class driver are
positive. If you have a negative power supply, then the voltage is inverted at the device
driver level. That's why you have MIN and MAX in the class driver.

Stefan

> 
> The original idea behind the hv driver is that all voltages in the ODB and the class driver are
> positive. If you have a negative power supply, then the voltage is inverted at the device
> driver level. That's why you have MIN and MAX in the class driver.
> 

This rings a bell. I used the hv class driver to write a frontend for the L1440 mainframe (negative voltage),
on ODB it will be positive values, when writing to the device I had to add a minus sign,
and when reading back they came back negative and I had to add an fabs() in the comparison
between readback and demand.

Persons with bipolar power supplies need not apply.

K.O.

first-console$ python evprint.py receive
second-console$ python evsend.py 123
[first console]
id=2007 mask=2007 serial=2007 time=1169372833 len=3 '123'

$ python evprint.py --log --path 127.0.0.1/online receive"
cm_connect_experiment('127.0.0.1', 'online', 'evprint.py', None)
bm_open_buffer('SYSTEM', 1048576, &c_long(2)) -> BM_CREATED
bm_request_event(2, -1, -1, 2, &c_long(0), None)
... wait for a couple of seconds ...
[midas.c:9348:rpc_call] rpc timeout, routine = "bm_receive_event"
[system.c:3570:send_tcp] send(socket=0,size=8) returned -1, errno: 88 (Socket 
operation on non-socket)
[midas.c:9326:rpc_call] send_tcp() failed

bm_receive_event(2, ...) -> RPC_TIMEOUT

bm_remove_event_request(2, 0) -> BM_INVALID_HANDLE
bm_close_buffer(2) -> BM_INVALID_HANDLE
cm_disconnect_experiment()

   1: python evprint.py receive
   2: python evsend.py 123
   1: id=2007 mask=2007 serial=2007 time=1169372833 len=3 '123'

   1: python evprint.py --path 127.0.0.1/ dispatch
   2: python evsend.py --path 127.0.0.1/ 123
   1: id=2007 mask=2007 serial=2007 time=1169373366 len=3 '123'

   1: python evprint.py dispatch
   2: python evsend.py --path 127.0.0.1/ 123
   1: (nothing printed)

   1: python evprint.py --path 127.0.0.1/ dispatch
   2: python evsend.py 123
   1: (nothing printed)

D:\denis\cmd\midas\current\06jan21-export\midas\NT\bin>produce.exe
ID of event to produce: 1
Host to connect: 127.0.0.1
Event size: 10
Level:   0.0 %, Rate: 0.64 MB/sec
flush
Level:   0.0 %, Rate: 0.64 MB/sec
Level:   0.0 %, Rate: 0.63 MB/sec
Level:   0.0 %, Rate: 0.64 MB/sec
Level:   0.0 %, Rate: 0.61 MB/sec
Level:   0.0 %, Rate: 0.62 MB/sec
Level:   0.0 %, Rate: 0.62 MB/sec
Level:   0.0 %, Rate: 0.64 MB/sec
Level:   0.0 %, Rate: 0.63 MB/sec
Level:   0.0 %, Rate: 0.63 MB/sec
Level:   0.0 %, Rate: 0.64 MB/sec
flush
Level:   0.0 %, Rate: 0.62 MB/sec

## Now I've started consume in the other console ##

[system.c:3570:send_tcp] send(socket=1900,size=8136) returned -1, errno: 0 (No error)
send_tcp() returned -1
[midas.c:9669:rpc_send_event] send_tcp() failed
rpc_send_event returned error 503, event_size 10

D:\denis\cmd\midas\current\06jan21-export\midas\NT\bin>consume.exe
ID of event to request: 1
Host to connect:
Get all events (0/1): 1
Receive via callback ([y]/n):
Level:   0.0 %, Rate: 0.00 MB/sec, ser mismatches: 0
Level:   0.0 %, Rate: 0.00 MB/sec, ser mismatches: 0
Level:   0.0 %, Rate: 0.00 MB/sec, ser mismatches: 0
Received break. Aborting...

D:\denis\cmd\midas\current\06jan21-export\midas\NT\bin\mserver.exe started interactively
[midas.c:2315:bm_validate_client_index] Invalid client index 0 in buffer 'SYSTEM'.
Client name 'Power Consumer', pid 1964 should be 3216

D:\denis\cmd\midas\current\06jan21-export\midas\NT\bin>produce.exe
ID of event to produce: 1
Host to connect:
Event size: 10
Client 'Producer' (PID 2584) on 'ODB' removed by cm_watchdog (idle 144.1s,TO 10s)
Level:   0.0 %, Rate: 3.20 MB/sec
flush
Level:   0.0 %, Rate: 3.20 MB/sec
Level:   0.0 %, Rate: 3.11 MB/sec
Level:   0.0 %, Rate: 3.13 MB/sec
Level:   0.0 %, Rate: 3.06 MB/sec
Level:   0.0 %, Rate: 3.20 MB/sec
Level:   0.0 %, Rate: 2.96 MB/sec
Level:   0.0 %, Rate: 3.11 MB/sec
Level:   0.0 %, Rate: 3.18 MB/sec
Level:   0.0 %, Rate: 3.13 MB/sec
Level:   0.0 %, Rate: 3.17 MB/sec
flush
Level:   0.0 %, Rate: 3.19 MB/sec
Level:   0.0 %, Rate: 3.08 MB/sec
Level:   0.0 %, Rate: 3.06 MB/sec

## Now I've started consume ##

[midas.c:2315:bm_validate_client_index] Invalid client index 0 in buffer 'SYSTEM'. Client name '', pid 0 should be 760

D:\denis\cmd\midas\current\06jan21-export\midas\NT\bin>consume.exe
ID of event to request: 1
Host to connect: 127.0.0.1
Get all events (0/1): 1
Receive via callback ([y]/n):
Level:   0.0 %, Rate: 0.00 MB/sec, ser mismatches: 0
Level:   0.0 %, Rate: 0.00 MB/sec, ser mismatches: 0
Received break. Aborting...
Level:   0.0 %, Rate: 0.00 MB/sec, ser mismatches: 0

D:\denis\cmd\midas\current\06jan21-export\midas\NT\bin>produce.exe
ID of event to produce: 1
Host to connect: 127.0.0.1
Event size: 10
Level:   0.0 %, Rate: 0.65 MB/sec
flush
Level:   0.0 %, Rate: 0.66 MB/sec
Level:   0.0 %, Rate: 0.65 MB/sec
Level:   0.0 %, Rate: 0.60 MB/sec
Level:   0.0 %, Rate: 0.64 MB/sec
Level:   0.0 %, Rate: 0.63 MB/sec
Level:   0.0 %, Rate: 0.61 MB/sec
Level:   0.0 %, Rate: 0.63 MB/sec
Level:   0.0 %, Rate: 0.65 MB/sec
Level:   0.0 %, Rate: 0.65 MB/sec
Level:   0.0 %, Rate: 0.67 MB/sec
flush
Level:   0.0 %, Rate: 0.66 MB/sec
Level:   0.0 %, Rate: 0.65 MB/sec
Level:   0.0 %, Rate: 0.65 MB/sec
Level:   0.0 %, Rate: 0.66 MB/sec
Level:   0.0 %, Rate: 0.66 MB/sec
Level:   0.0 %, Rate: 0.65 MB/sec
Level:   0.0 %, Rate: 0.66 MB/sec
Level:   0.0 %, Rate: 0.66 MB/sec
Level:   0.0 %, Rate: 0.66 MB/sec
Level:  66.8 %, Rate: 0.66 MB/sec
flush
Level:   0.0 %, Rate: 0.00 MB/sec
Level:  66.8 %, Rate: 0.31 MB/sec
Level:  57.2 %, Rate: 0.15 MB/sec
Level:  57.3 %, Rate: 0.14 MB/sec
Level:  57.3 %, Rate: 0.15 MB/sec
Level:  57.3 %, Rate: 0.14 MB/sec
Level:  57.3 %, Rate: 0.14 MB/sec
Level:  57.3 %, Rate: 0.14 MB/sec
Received break. Aborting...
Received 2nd break. Hard abort.
[midas.c:1581:] cm_disconnect_experiment not called at end of program

D:\denis\cmd\midas\current\06jan21-export\midas\NT\bin>consume.exe
ID of event to request: 1
Host to connect: 127.0.0.1
Get all events (0/1): 1
Receive via callback ([y]/n):
[consume.c:73:process_event] Serial number mismatch: Ser: 1397076, OldSer: 0, ID: 1, size: 10
Level:  37.1 %, Rate: 0.00 MB/sec, ser mismatches: 1
Level:   0.0 %, Rate: 0.15 MB/sec, ser mismatches: 1
Level:  95.4 %, Rate: 0.08 MB/sec, ser mismatches: 1
Level:  66.8 %, Rate: 0.14 MB/sec, ser mismatches: 1
Level:  66.8 %, Rate: 0.12 MB/sec, ser mismatches: 1
Level:  76.3 %, Rate: 0.12 MB/sec, ser mismatches: 1
Level:  95.4 %, Rate: 0.11 MB/sec, ser mismatches: 1
Level:  57.3 %, Rate: 0.15 MB/sec, ser mismatches: 1
Level:  66.8 %, Rate: 0.11 MB/sec, ser mismatches: 1
Level:  85.9 %, Rate: 0.11 MB/sec, ser mismatches: 1
Level:  95.5 %, Rate: 0.12 MB/sec, ser mismatches: 1
Level:  57.4 %, Rate: 0.15 MB/sec, ser mismatches: 1
Level:   9.7 %, Rate: 0.15 MB/sec, ser mismatches: 1
[Producer] [midas.c:1581:] cm_disconnect_experiment not called at end of program
Level:   0.0 %, Rate: 0.03 MB/sec, ser mismatches: 1
Level:   0.0 %, Rate: 0.00 MB/sec, ser mismatches: 1
Received break. Aborting...