bug in init
-----------

I used the lv.c class driver, combined with a custom device driver, to control 
our Keithley2611B source meter. This to set negative voltage on Si detectors.

In the 'init' routing, the class driver sets the hv:

  hv_info->demand_mirror[i] = MIN(hv_info->demand[i], hv_info->voltage_limit[i]);

This fails for negative voltage, as it sets the (negative) voltage limit, instead 
of the demand voltage. A simple 'fabs' solves this.

suggestion for 'idle'
---------------------

I let the device do the ramping, not the driver. This also means I have to reset 
the state of the device (current limit) after ramping. The easiest way to to 
this, is using CMD_IDLE of the device driver. This is currently not done in the 
hv.c class driver.

Hi, Frederick, this is my personal opinion on the slow controls hv classes, I have
used them a couple of times and I found them full of little buglets like this,
plus some incomplete functions, plus some missing features, plus it is all
written in C trying to do object oriented programming. On the balance my opinion
is that it is less work to write a high voltage control program in C++ from scratch
using the regular midas frontend infrastructure compared to having to understand
the hv class driver, write the missing bits, fix the little buglets, debug
the crashes in the C string handling, and what not. (For example I had to debug
mysterious failures to pass float and double values through the C stdarg interface,
there are more fun things to do out there).

K.O.

> bug in init
> -----------
> 
> I used the lv.c class driver, combined with a custom device driver, to control 
> our Keithley2611B source meter. This to set negative voltage on Si detectors.
> 
> In the 'init' routing, the class driver sets the hv:
> 
>   hv_info->demand_mirror[i] = MIN(hv_info->demand[i], hv_info->voltage_limit[i]);
> 
> This fails for negative voltage, as it sets the (negative) voltage limit, instead 
> of the demand voltage. A simple 'fabs' solves this.
> 
> suggestion for 'idle'
> ---------------------
> 
> I let the device do the ramping, not the driver. This also means I have to reset 
> the state of the device (current limit) after ramping. The easiest way to to 
> this, is using CMD_IDLE of the device driver. This is currently not done in the 
> hv.c class driver.

> bug in init
> -----------
> 
> I used the lv.c class driver, combined with a custom device driver, to control 
> our Keithley2611B source meter. This to set negative voltage on Si detectors.
> 
> In the 'init' routing, the class driver sets the hv:
> 
>   hv_info->demand_mirror[i] = MIN(hv_info->demand[i], hv_info->voltage_limit[i]);
> 
> This fails for negative voltage, as it sets the (negative) voltage limit, instead 
> of the demand voltage. A simple 'fabs' solves this.
> 
> suggestion for 'idle'
> ---------------------
> 
> I let the device do the ramping, not the driver. This also means I have to reset 
> the state of the device (current limit) after ramping. The easiest way to to 
> this, is using CMD_IDLE of the device driver. This is currently not done in the 
> hv.c class driver.

I can't find the line you quote in the class driver. Why don't you make a git pull request
and I will approve it.

The original idea behind the hv driver is that all voltages in the ODB and the class driver are
positive. If you have a negative power supply, then the voltage is inverted at the device
driver level. That's why you have MIN and MAX in the class driver.

Stefan

> 
> The original idea behind the hv driver is that all voltages in the ODB and the class driver are
> positive. If you have a negative power supply, then the voltage is inverted at the device
> driver level. That's why you have MIN and MAX in the class driver.
> 

This rings a bell. I used the hv class driver to write a frontend for the L1440 mainframe (negative voltage),
on ODB it will be positive values, when writing to the device I had to add a minus sign,
and when reading back they came back negative and I had to add an fabs() in the comparison
between readback and demand.

Persons with bipolar power supplies need not apply.

K.O.

first-console$ python evprint.py receive
second-console$ python evsend.py 123
[first console]
id=2007 mask=2007 serial=2007 time=1169372833 len=3 '123'

$ python evprint.py --log --path 127.0.0.1/online receive"
cm_connect_experiment('127.0.0.1', 'online', 'evprint.py', None)
bm_open_buffer('SYSTEM', 1048576, &c_long(2)) -> BM_CREATED
bm_request_event(2, -1, -1, 2, &c_long(0), None)
... wait for a couple of seconds ...
[midas.c:9348:rpc_call] rpc timeout, routine = "bm_receive_event"
[system.c:3570:send_tcp] send(socket=0,size=8) returned -1, errno: 88 (Socket 
operation on non-socket)
[midas.c:9326:rpc_call] send_tcp() failed

bm_receive_event(2, ...) -> RPC_TIMEOUT

bm_remove_event_request(2, 0) -> BM_INVALID_HANDLE
bm_close_buffer(2) -> BM_INVALID_HANDLE
cm_disconnect_experiment()

   1: python evprint.py receive
   2: python evsend.py 123
   1: id=2007 mask=2007 serial=2007 time=1169372833 len=3 '123'

   1: python evprint.py --path 127.0.0.1/ dispatch
   2: python evsend.py --path 127.0.0.1/ 123
   1: id=2007 mask=2007 serial=2007 time=1169373366 len=3 '123'

   1: python evprint.py dispatch
   2: python evsend.py --path 127.0.0.1/ 123
   1: (nothing printed)

   1: python evprint.py --path 127.0.0.1/ dispatch
   2: python evsend.py 123
   1: (nothing printed)

D:\denis\cmd\midas\current\06jan21-export\midas\NT\bin>produce.exe
ID of event to produce: 1
Host to connect: 127.0.0.1
Event size: 10
Level:   0.0 %, Rate: 0.64 MB/sec
flush
Level:   0.0 %, Rate: 0.64 MB/sec
Level:   0.0 %, Rate: 0.63 MB/sec
Level:   0.0 %, Rate: 0.64 MB/sec
Level:   0.0 %, Rate: 0.61 MB/sec
Level:   0.0 %, Rate: 0.62 MB/sec
Level:   0.0 %, Rate: 0.62 MB/sec
Level:   0.0 %, Rate: 0.64 MB/sec
Level:   0.0 %, Rate: 0.63 MB/sec
Level:   0.0 %, Rate: 0.63 MB/sec
Level:   0.0 %, Rate: 0.64 MB/sec
flush
Level:   0.0 %, Rate: 0.62 MB/sec

## Now I've started consume in the other console ##

[system.c:3570:send_tcp] send(socket=1900,size=8136) returned -1, errno: 0 (No error)
send_tcp() returned -1
[midas.c:9669:rpc_send_event] send_tcp() failed
rpc_send_event returned error 503, event_size 10

D:\denis\cmd\midas\current\06jan21-export\midas\NT\bin>consume.exe
ID of event to request: 1
Host to connect:
Get all events (0/1): 1
Receive via callback ([y]/n):
Level:   0.0 %, Rate: 0.00 MB/sec, ser mismatches: 0
Level:   0.0 %, Rate: 0.00 MB/sec, ser mismatches: 0
Level:   0.0 %, Rate: 0.00 MB/sec, ser mismatches: 0
Received break. Aborting...

D:\denis\cmd\midas\current\06jan21-export\midas\NT\bin\mserver.exe started interactively
[midas.c:2315:bm_validate_client_index] Invalid client index 0 in buffer 'SYSTEM'.
Client name 'Power Consumer', pid 1964 should be 3216

D:\denis\cmd\midas\current\06jan21-export\midas\NT\bin>produce.exe
ID of event to produce: 1
Host to connect:
Event size: 10
Client 'Producer' (PID 2584) on 'ODB' removed by cm_watchdog (idle 144.1s,TO 10s)
Level:   0.0 %, Rate: 3.20 MB/sec
flush
Level:   0.0 %, Rate: 3.20 MB/sec
Level:   0.0 %, Rate: 3.11 MB/sec
Level:   0.0 %, Rate: 3.13 MB/sec
Level:   0.0 %, Rate: 3.06 MB/sec
Level:   0.0 %, Rate: 3.20 MB/sec
Level:   0.0 %, Rate: 2.96 MB/sec
Level:   0.0 %, Rate: 3.11 MB/sec
Level:   0.0 %, Rate: 3.18 MB/sec
Level:   0.0 %, Rate: 3.13 MB/sec
Level:   0.0 %, Rate: 3.17 MB/sec
flush
Level:   0.0 %, Rate: 3.19 MB/sec
Level:   0.0 %, Rate: 3.08 MB/sec
Level:   0.0 %, Rate: 3.06 MB/sec

## Now I've started consume ##

[midas.c:2315:bm_validate_client_index] Invalid client index 0 in buffer 'SYSTEM'. Client name '', pid 0 should be 760

D:\denis\cmd\midas\current\06jan21-export\midas\NT\bin>consume.exe
ID of event to request: 1
Host to connect: 127.0.0.1
Get all events (0/1): 1
Receive via callback ([y]/n):
Level:   0.0 %, Rate: 0.00 MB/sec, ser mismatches: 0
Level:   0.0 %, Rate: 0.00 MB/sec, ser mismatches: 0
Received break. Aborting...
Level:   0.0 %, Rate: 0.00 MB/sec, ser mismatches: 0

D:\denis\cmd\midas\current\06jan21-export\midas\NT\bin>produce.exe
ID of event to produce: 1
Host to connect: 127.0.0.1
Event size: 10
Level:   0.0 %, Rate: 0.65 MB/sec
flush
Level:   0.0 %, Rate: 0.66 MB/sec
Level:   0.0 %, Rate: 0.65 MB/sec
Level:   0.0 %, Rate: 0.60 MB/sec
Level:   0.0 %, Rate: 0.64 MB/sec
Level:   0.0 %, Rate: 0.63 MB/sec
Level:   0.0 %, Rate: 0.61 MB/sec
Level:   0.0 %, Rate: 0.63 MB/sec
Level:   0.0 %, Rate: 0.65 MB/sec
Level:   0.0 %, Rate: 0.65 MB/sec
Level:   0.0 %, Rate: 0.67 MB/sec
flush
Level:   0.0 %, Rate: 0.66 MB/sec
Level:   0.0 %, Rate: 0.65 MB/sec
Level:   0.0 %, Rate: 0.65 MB/sec
Level:   0.0 %, Rate: 0.66 MB/sec
Level:   0.0 %, Rate: 0.66 MB/sec
Level:   0.0 %, Rate: 0.65 MB/sec
Level:   0.0 %, Rate: 0.66 MB/sec
Level:   0.0 %, Rate: 0.66 MB/sec
Level:   0.0 %, Rate: 0.66 MB/sec
Level:  66.8 %, Rate: 0.66 MB/sec
flush
Level:   0.0 %, Rate: 0.00 MB/sec
Level:  66.8 %, Rate: 0.31 MB/sec
Level:  57.2 %, Rate: 0.15 MB/sec
Level:  57.3 %, Rate: 0.14 MB/sec
Level:  57.3 %, Rate: 0.15 MB/sec
Level:  57.3 %, Rate: 0.14 MB/sec
Level:  57.3 %, Rate: 0.14 MB/sec
Level:  57.3 %, Rate: 0.14 MB/sec
Received break. Aborting...
Received 2nd break. Hard abort.
[midas.c:1581:] cm_disconnect_experiment not called at end of program

D:\denis\cmd\midas\current\06jan21-export\midas\NT\bin>consume.exe
ID of event to request: 1
Host to connect: 127.0.0.1
Get all events (0/1): 1
Receive via callback ([y]/n):
[consume.c:73:process_event] Serial number mismatch: Ser: 1397076, OldSer: 0, ID: 1, size: 10
Level:  37.1 %, Rate: 0.00 MB/sec, ser mismatches: 1
Level:   0.0 %, Rate: 0.15 MB/sec, ser mismatches: 1
Level:  95.4 %, Rate: 0.08 MB/sec, ser mismatches: 1
Level:  66.8 %, Rate: 0.14 MB/sec, ser mismatches: 1
Level:  66.8 %, Rate: 0.12 MB/sec, ser mismatches: 1
Level:  76.3 %, Rate: 0.12 MB/sec, ser mismatches: 1
Level:  95.4 %, Rate: 0.11 MB/sec, ser mismatches: 1
Level:  57.3 %, Rate: 0.15 MB/sec, ser mismatches: 1
Level:  66.8 %, Rate: 0.11 MB/sec, ser mismatches: 1
Level:  85.9 %, Rate: 0.11 MB/sec, ser mismatches: 1
Level:  95.5 %, Rate: 0.12 MB/sec, ser mismatches: 1
Level:  57.4 %, Rate: 0.15 MB/sec, ser mismatches: 1
Level:   9.7 %, Rate: 0.15 MB/sec, ser mismatches: 1
[Producer] [midas.c:1581:] cm_disconnect_experiment not called at end of program
Level:   0.0 %, Rate: 0.03 MB/sec, ser mismatches: 1
Level:   0.0 %, Rate: 0.00 MB/sec, ser mismatches: 1
Received break. Aborting...

> > I checked again on browser compatibility:
> > 
> > el6: firefox 38 - ok, google-chrome 27 - no
> > el7: firefox 38 - ok, google-chrome 46 - ok
> > ubuntu: firefox 42 - ok
> > 
> > mac os, windows - we say "latest firefox or google-chrome is required", then - ok
> > 
> > So we are probably okey with using javascript Promises with MIDAS...
> 
> [too bad about chrome on SL6] ... include a polyfill library like Lie (https://github.com/calvinmetcalf/lie).

Results of cross-browser testing.

MacOS 10.10.5:

google-chrome 46: Promise ok, Programs page ok, Overlay ok
firefox 42: Promise ok, Programs page ok, Overlay ok.
safari 9.0.1: Promise ok, Programs page ok, Overlay ok.

Linux SL6.7:

google-chrome 27: "Promise not defined"
firefox 38.4.0: Promise ok, Programs page ok, Overlay ok.
konqueror 4.3.4: no go "Can't find variable: JSON"
chromium/google-chrome 38: ok

Linux SL7.1:

google-crome 46: ok
firefox 38.4.0: ok
konqueror 4.10.5: no go, mhttpd.js parse error

Conclusion:

1) firefox is good everywhere
2) google-chrome is good on Mac, Windows and el7 Linux
3) chromium/google-chrome 38 is good on el6 Linux (SL6/CentOS6).

We are good to proceed with adopting the Promise API for MIDAS.

K.O.

midas web pages that use overlays (dlgPanel, etc) are currently broken - if 
overlay does not fit in the visible window, it's bottom is truncated and control 
buttons like "create" and "cancel" are not visible, not clickable, page does not 
work.

when these pages were originally written, I am pretty sure these overlays were 
scrollable and this problem did not exist. I think that was broken recently, 
maybe withint the last year or so.

specific examples:

a) odb editor:

- open odb editor,
- click on "create odb link"
- click on "link target ...", a dialog overlay opens with a list of odb keys in 
the current directory
- select a directory with a large number of entries (i.e. "/Programs")
- alternatively, make browser window smaller
- observe the "ok" and "cancel" buttons are not visible, cannot be clicked
- definitely, there used be a scroll bar and one could scroll down to see these 
buttons.

b) history planel editor:

- open history plot,
- click on "configure this plot" icon,
- history editor opens,
- click "add active variables"
- select active event that has many variables
- observe that the list is cut off at the bottom, the very last variables are 
not visible
- alternatively, make the browser window smaller

I wrote this page and at the time this problem did not exist, there was a scroll 
bar and one could scroll up and down the list even if there were really many 
variables there.

Maybe this breakage is not from us, I see similar problems on other sites, so 
maybe browser behaviour changed recentlyshly.

I think Stefan write the dlgPanel code originally? I am not very familiar with 
it and I do not know if anybody changed it recently?

K.O.

I confirm the problem is there (at least under MacOSX Safari) and I will take care of it.

Stefan

This problem was introduced by ZS in March 2023 with these commits:

https://bitbucket.org/tmidas/midas/commits/25b13f875ff1f7e2f4e987273c81d6356dd2ff53
https://bitbucket.org/tmidas/midas/commits/2a9e902e07156e12edecb5c2257e4dbd944f8377

by setting

 d.style.position = "fixed";

which prevents the scrolling. I have no idea why this change was made, so it should be fixed by the original 
author.

Stefan

Sorry about that. I could not figure out what was the reason for doing this. This was during the time I was working on the file_picker. I removed these lines and see no effect on the file_picker. I'll continue checking it affect anything else.

Zaher

> This problem was introduced by ZS in March 2023 with these commits:
> 
> https://bitbucket.org/tmidas/midas/commits/25b13f875ff1f7e2f4e987273c81d6356dd2ff53
> https://bitbucket.org/tmidas/midas/commits/2a9e902e07156e12edecb5c2257e4dbd944f8377
> 
> by setting
> 
>  d.style.position = "fixed";
> 
> which prevents the scrolling. I have no idea why this change was made, so it should be fixed by the original 
> author.
> 
> Stefan

> Sorry about that. I could not figure out what was the reason for doing this. This was during the time I was working on the file_picker. I removed these lines and see no effect on the file_picker. I'll continue checking it affect anything else.

I confirm reported problem seems to be fixed in commit:
https://bitbucket.org/tmidas/midas/commits/7f2690b478d6dfb16b48fc98955093e6369b04c1

Big thanks to Stefan and Zaher for figuring it out quickly.

K.O.

Not sure if this is a bug or a feature:

Writing a boolean label on an image seems to produce rather strange behaviour.

For example,
odb>ls /Equipment/gas/settings/my_bool -lt
Key name                        Type    #Val  Size  Last Opn Mode Value
---------------------------------------------------------------------------
my_bool                         BOOL    1     4     14m  0   RWD  y


odb>cd /custom/images/my_image.gif/labels
odb>ls
Src                             /Equipment/gas/settings/my_bool
Format                          val: %d (bool)
Font                            Medium
X                               10
Y                               10
Align                           0
FGColor                         FFFFFF
BGColor                         FF8800

Instead of the expected string "val: y (bool)", only the value of the key
appears, i.e. "y". 
The behaviour is the same whether I use %d, %u, %s, %c etc as the format character. 

> Not sure if this is a bug or a feature:
> 
> Writing a boolean label on an image seems to produce rather strange behaviour.
> 
> For example,
> odb>ls /Equipment/gas/settings/my_bool -lt
> Key name                        Type    #Val  Size  Last Opn Mode Value
> ---------------------------------------------------------------------------
> my_bool                         BOOL    1     4     14m  0   RWD  y
> 
> 
> odb>cd /custom/images/my_image.gif/labels
> odb>ls
> Src                             /Equipment/gas/settings/my_bool
> Format                          val: %d (bool)
> Font                            Medium
> X                               10
> Y                               10
> Align                           0
> FGColor                         FFFFFF
> BGColor                         FF8800
> 
> Instead of the expected string "val: y (bool)", only the value of the key
> appears, i.e. "y". 
> The behaviour is the same whether I use %d, %u, %s, %c etc as the format character. 

That has been fixed in rev. 4379

When mserver receives events from remote client, writes them into a data buffer and this data buffer 
becomes 100% full, we see mserver go into 100% consumption.

It turns out this happens because bm_wait_for_free_space() never sleeps, instead, it busy-loops waiting 
for free space. bm_wait_for_free_space() does call ss_suspend(), but ss_suspend() does not sleep 
because there is pending data in the event network connection and it want to process it.

Best solution I have is to use silly "if (ss_suspend()!=SS_TIMEOUT) sleep(1);"

Also read this explanation: (bm_cleanup is needed to detect that the client holding the buffer at 100% 
full (a stuck or dead GET_ALL reader, mevb in our case), has been killed off and we can continue as 
usual)

       /* signal other clients wait mode */
       pheader->client[bm_validate_client_index(pbuf)].write_wait = requested_space;
 
+      bm_cleanup("bm_wait_for_free_space", ss_millitime(), FALSE);
+
       status = ss_suspend(1000, MSG_BM);
 
+      /* make sure we do sleep in this loop:
+       * if we are the mserver receiving data on the event
+       * socket and the data buffer is full, ss_suspend() will
+       * never sleep: it will detect data on the event channel,
+       * call rpc_server_receive() (recursively, we already *are* in
+       * rpc_server_receive()) and return without sleeping. Result
+       * is a busy loop waiting for free space in data buffer */
+      if (status != SS_TIMEOUT)
+         sleep(1);
+
       /* validate client index: we could have been removed from the buffer */
       pheader->client[bm_validate_client_index(pbuf)].write_wait = 0;

K.O.

Hi all

There is a bug report in the ROME repository which says bm_receive_event timeouts.

https://bitbucket.org/muegamma/rome3/issues/8/rome-with-midas-produces-timeout-after

Does anybody have any ideas what could causing the problem ?

Ryu

> There is a bug report in the ROME repository which says bm_receive_event timeouts.
> https://bitbucket.org/muegamma/rome3/issues/8/rome-with-midas-produces-timeout-after
> Does anybody have any ideas what could causing the problem ?

There could be a problem with causing bm_receive_event() to wait for an event for a time longer than 
the rpc timeout. This rings a very small bell for me. But I do not remember the details.

As I now go through the midas event buffer code, I will check that bm_receive_event() connected 
through the mserver has correctly working timeouts.

Thank you for reminding me about this difficulty.

K.O.