mhttpd.c svn rev 4823 implements a modified history planel editor. all previous functions should work 
as before (minus new bugs).

New experimental functions added:

a) there is a new column "Order" containing numbers 10, 20, 30, etc. If you change "30" to (say) "15" 
and press "refresh", the history variables will be reordered according to the new values. If you change 
(say) "10" to "" (empty) or "0" and press "refresh", this variable will be deleted. (But there is a UI wart - if 
you accidentally change the order value to something non-numeric (i.e. "aaa1" or " 1" (leading space) 
and press "enter", the variable will be immediately deleted from odb - "enter" works as "refresh + save" 
- should probably work as "refresh" requiring explicit press on the "save" button).

b) there is a new button "List all" to list all existing variables - next to each variable is a checkbox - 
select any checkboxes and press "add selected" to add selected variables to the history plot. You may 
find this function useful (or not), depending on how many variables you have in your history. For 
T2K/ND280 this is still not good enough (there are still too many variables) and I want to change this to 
a 3 level (equipment, history event, history tag) expandable/collapsable tree (or whatever is simplest to 
implement) - to permit the user to quickly zoom on the interesting variables.

I may still tweak with the UI of these new functions, but the basic functionality (reorder+delete and 
selection of multiple variables from a list) seems to be solid. Comments and suggestions on how to 
make it work the best for your experiment are very welcome.

K.O.

> It looks like some example drivers in .../drivers/class want to link against YBOS libraries. 
> This fails because ybos.o is missing from the MIDAS library.

I fixed the class drivers in meantime (SVN 4814).

There is however another problem: The lazylogger needs YBOS support compiled in if the FTP transfer mode is used. 
At PSI we are stuck at the moment to FTP, so we still need YBOS there (although none of the data is in YBOS format). 
Maybe there is a chance that this will be fixed some time and we can get rid of YBOS.

> As of svn rev 4794, midas builds, runs and should be fully usable on MacOS 10.6.4. Previous revisions did 
> not compile due to assorted Linuxisms and did not run because of a sizeof() problem in ss_gettid(). Also 
> one of the system header files (mtio.h?) present in MacOS 10.5 vanished from 10.6.

It turns out that on MacOS 10.6 the default maximum SYSV shared memory size is about 2 Mbytes, too small even for the default MIDAS SYSTEM 
event buffer.

Svn revision 4807 implements POSIX shared memory, which does not seem to have such a small size limit and makes it the default on MacOS.

This update fixes the last issue that I am aware of for running MIDAS on MacOS.

svn rev 4807
K.O.

As of svn rev 4807, src/system.c has an experimental implementation of POSIX shared memory. It is 
similar to the already existing implementation of MMAP shared memory, but uses POSIX shm_open() 
instead of directly mmapping the .xxx.SHM file.

There are several benefits to using POSIX shared memory:
1) on MacOS, the (unchangable?) maximum SYSV shared memory is about 2 Mbytes, too small for most 
MIDAS experiments. POSIX shared memory does not seem to have such a limit;
2) on Linux, when using SYSV shared memory, the .xxx.SHM files are tied to the shared memory keys 
using ftok(). If the .xxx.SHM files are located on an NFS-mounted filesystem, ftok() has been observed 
to malfunction and return the same key for different shared memory buffers, causing mighty confusing 
behaviour of MIDAS. (while "man ftok" discusses a theoretical possibility for such collisions, I have 
observed ftok() collisions first hand on a running experiment and it cost us several hours to understand 
why all the events go into the wrong event buffers). The present POSIX shared memory implementation 
does not have such a problem.

This implementation has received limited testing on Linux and MacOS, and it is now the default shared 
memory implementation on MacOS. Linux continues to use SYSV shared memory (ipcs & co). Windows 
uses it's own implementation of shared memory (same as mmap, the best I can tell).

svn 4807
K.O.

> As of svn rev 4800, YBOS support was made optional, disabled by default. (But note that ybos.c is still used 
> by mdump). See HAVE_YBOS in the Makefile.


It looks like some example drivers in .../drivers/class want to link against YBOS libraries. This fails because ybos.o is missing from the MIDAS library.

After discussions with SR and PAA, we think YBOS support can be removed or made optional, but there are too many of these drivers for me to fix 
them all right now in five minutes. Please accept my apology and use these workarounds:

If you get linker errors because of missing YBOS functions:

1) enable YBOS suport in the Makefile (uncomment HAVE_YBOS=1), or
2) "#ifdef HAVE_YBOS" all places that call YBOS functions

Solution (2) is preferable as it permits us to eventually remove YBOS completely. If you fix files from MIDAS svn, please do send me patches or diffs (or 
post them here).


K.O.

As of svn rev 4800, YBOS support was made optional, disabled by default. (But note that ybos.c is still used 
by mdump). See HAVE_YBOS in the Makefile.
K.O.

As of svn rev 4794, midas builds, runs and should be fully usable on MacOS 10.6.4. Previous revisions did 
not compile due to assorted Linuxisms and did not run because of a sizeof() problem in ss_gettid(). Also 
one of the system header files (mtio.h?) present in MacOS 10.5 vanished from 10.6.

Please continue reporting all problems with midas on macos to this forum.
K.O.

> > The way connections work under Midas is there is a callback scheme. The client starts 
> > mserver on the back-end, then the back-end connects back to the front-end on three 
> > different ports. These ports are assigned dynamically by the operating system and are 
> > typically in the range 40000-60000. So you also have to allow the reverse connection on 
> > your firewalls.
> 
> It works now after allowing ports 40000-60000 in the front-end computer. Thanks!


Yes, right. Midas networking does not like firewalls.

In the nutshell, TCP connections on all TCP ports have to be open between all computers
running MIDAS. I think in practice it is not a problem: you only ever have a finite (a small
integer) number of computers running MIDAS and you can be added them as exceptions to the
firewall rules. These exceptions should not create any security problem because you still have
the MIDAS computers firewalled from the outside world and one hopes that they will not be
attacking each other.

P.S. Permitting ports 40000-60000 is not good enough. TCP ports are allocated to TCP
connections semi-randomly from a 16-bit address space (0..65535) and your system will bomb
whenever port numbers like 39999 or 60001 get used.


K.O.

> The way connections work under Midas is there is a callback scheme. The client starts 
> mserver on the back-end, then the back-end connects back to the front-end on three 
> different ports. These ports are assigned dynamically by the operating system and are 
> typically in the range 40000-60000. So you also have to allow the reverse connection on 
> your firewalls.

It works now after allowing ports 40000-60000 in the front-end computer. Thanks!

Best Regards,
Jimmy

> > Hi, there. I have not recently run mserver through inetd, and we usually do not do
> > that at TRIUMF. We do this:
> > 
> > a) on the main computer: start mserver: "mserver -p 7070 -D" (note - use non-default
> > port - can use different ports for different experiments)
> > b) on remote computer: "odbedit -h main:7070" ("main" is the hostname of your main
> > computer). Use same "-h" switch for all other programs, including the frontends.
> > 
> > This works well when all computers are on the same network, but if you have some
> > midas clients running on private networks you may get into trouble when they try to
> > connect to each other and fail because network routing is funny.
> 
> Hi K.O.,
> 
> Thanks for your reply. I have tried your way but I got the same error: 
> 
> [midas.c:8623:rpc_server_connect,ERROR] mserver subprocess could not be started 
> (check path)
> 
> My front-end and back-end computers are on the same network connected by a router. I 
> have allowed port 7070 in the firewall and done the port forwarding in the router (for 
> connecting from outside the network). From the error message it seems that some 
> processes can not be started automatically. Could it be related to some security 
> settings such as the SELinux?

The way connections work under Midas is there is a callback scheme. The client starts 
mserver on the back-end, then the back-end connects back to the front-end on three 
different ports. These ports are assigned dynamically by the operating system and are 
typically in the range 40000-60000. So you also have to allow the reverse connection on 
your firewalls.

> Hi, there. I have not recently run mserver through inetd, and we usually do not do
> that at TRIUMF. We do this:
> 
> a) on the main computer: start mserver: "mserver -p 7070 -D" (note - use non-default
> port - can use different ports for different experiments)
> b) on remote computer: "odbedit -h main:7070" ("main" is the hostname of your main
> computer). Use same "-h" switch for all other programs, including the frontends.
> 
> This works well when all computers are on the same network, but if you have some
> midas clients running on private networks you may get into trouble when they try to
> connect to each other and fail because network routing is funny.

Hi K.O.,

Thanks for your reply. I have tried your way but I got the same error: 

[midas.c:8623:rpc_server_connect,ERROR] mserver subprocess could not be started 
(check path)

My front-end and back-end computers are on the same network connected by a router. I 
have allowed port 7070 in the firewall and done the port forwarding in the router (for 
connecting from outside the network). From the error message it seems that some 
processes can not be started automatically. Could it be related to some security 
settings such as the SELinux?

Best Regards,
Jimmy

> This is my first time running an experiment on separate computers. I followed 
> the documentation (https://midas.psi.ch/htmldoc/quickstart.html) to setup the 
> files:
> /etc/services
> /etc/xinetd.d/midas


Hi, there. I have not recently run mserver through inetd, and we usually do not do
that at TRIUMF. We do this:

a) on the main computer: start mserver: "mserver -p 7070 -D" (note - use non-default
port - can use different ports for different experiments)
b) on remote computer: "odbedit -h main:7070" ("main" is the hostname of your main
computer). Use same "-h" switch for all other programs, including the frontends.

This works well when all computers are on the same network, but if you have some
midas clients running on private networks you may get into trouble when they try to
connect to each other and fail because network routing is funny.


K.O.

Dear All,

This is my first time running an experiment on separate computers. I followed 
the documentation (https://midas.psi.ch/htmldoc/quickstart.html) to setup the 
files:
/etc/services
/etc/xinetd.d/midas
/etc/ld.so.conf
/etc/exptab

but when I started the frontend program in the front-end computer I got the 
following error (computerB is my back-end): 

[midas.c:8623:rpc_server_connect,ERROR] mserver subprocess could not be started 
(check path)
[mfe.c:2573:mainFE,ERROR] Cannot connect to experiment '' on host 'computerB', 
status 503

In both front-end and back-end computers only a file '.SYSMSG.SHM' was created 
after the attempt. If I start the frontend program somewhere in the back-end 
computer by connecting to 'localhost', seven .SHM files are created in the 
experiment directory together with a .RPC.SHM in the directory where I run the 
frontend program.

Is that I misconfigure something? I cannot find a solution...

Thanks.

Best Regards,
Jimmy

> - does your feTCPPacketReceiver die during the start-of-run? Maybe you do some segfault 
> int he begin-of-run routine. Can you STOP a run?
when start a run, it bring the mtransition process and I guess the server try to talk to the
client, then it fails and the frontend application get killed since not response.

>> When you try to stop your fe from 
> odbedit with  # odbedit -c "shutdown feTCPPacketReceiver"

it gets
[midas.c:8423:rpc_client_connect,ERROR] timeout on receive remote computer info: 
[midas.c:4880:cm_shutdown,ERROR] Cannot connect to client
"feTPCPacketReceiver" on host 'tpcdaq0', port 35865
[midas.c:4883:cm_shutdown,ERROR] Killing and Deleting client
'feTPCPacketReceiver' pid 27250
Client feTPCPacketReceiver not active


what does this error mean? :
11:03:42 [Logger,ERROR] [system.c:563:ss_shm_close,ERROR]
shmctl(shmid=7274511,IPC_RMID) failed, errno 1 (Operation not permitted)


thanks
hai

p.s. that code runs fine on my laptop with ubuntu 9, so that also be possible that somewhere
my configuration not right to cause problem

> I use fedora 12 and midas 4680. there is problem to start run when the frontend
> application runs fine. 

I don't know exactly what is wrong, but I would check following things:

- does your feTCPPacketReceiver die during the start-of-run? Maybe you do some segfault 
int he begin-of-run routine. Can you STOP a run?

- is there any network problem due to your two cards? When you try to stop your fe from 
odbedit with

# odbedit -c "shutdown feTCPPacketReceiver"

do you then get the same error? The shutdown functionality uses the same RPC channel as 
the start/stop run. Some people had firewall problems, on both sides (host AND client), 
so make sure all firewalls are disabled.

- if you disable one network card, do you still get the same problem?

Dear experts,

I use fedora 12 and midas 4680. there is problem to start run when the frontend
application runs fine. 


# odbedit -c start


Starting run #18
[midas.c:8423:rpc_client_connect,ERROR] timeout on receive remote computer info: 
[midas.c:3659:cm_transition,ERROR] cannot connect to client
"feTPCPacketReceiver" on host tpcdaq0, port 36663, status 503
[midas.c:8423:rpc_client_connect,ERROR] timeout on receive remote computer info: 
[midas.c:4880:cm_shutdown,ERROR] Cannot connect to client 'frontend' on host
'hostname', port 36663
[midas.c:4883:cm_shutdown,ERROR] Killing and Deleting client
'feTPCPacketReceiver' pid 24516
[midas.c:3857:cm_transition,ERROR] Could not start a run: cm_transition() status
503, message 'Cannot connect to client 'frontend''
Run #18 start aborted
Error: Cannot connect to client 'frontend'

11:03:42 [Logger,INFO] Deleting previous file "/home/daq/Run/online/run00018.mid"

11:03:42 [Logger,INFO] Client 'feTPCPacketReceiver' on buffer 'SYSMSG' removed
by cm_watchdog because client pid 24516 does not exist

11:03:42 [Logger,ERROR] [system.c:563:ss_shm_close,ERROR]
shmctl(shmid=7274511,IPC_RMID) failed, errno 1 (Operation not permitted)

11:03:42 [ODBEdit,INFO] Run #18 start aborted
==========================================================================

there are several ethernet cards on the host machine. eth0 connect the host
machine to the gateway machine and the front end application listen to eth1 for
the incoming data packets:

eth0      Link encap:Ethernet  HWaddr xx:xx:xx:xx:xx:xx  
          inet addr:10.0.1.1  Bcast:10.0.1.63  Mask:255.255.255.0
          inet6 addr: fe80::f6ce:46ff:fe99:709b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:470870 errors:0 dropped:0 overruns:0 frame:0
          TX packets:515987 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:345000246 (329.0 MiB)  TX bytes:377269124 (359.7 MiB)
          Interrupt:17 

eth1      Link encap:Ethernet  HWaddr xx:xx:xx:xx:xx:xx   
          inet addr:10.0.1.2  Bcast:10.255.255.255  Mask:255.0.0.0
          inet6 addr: fe80::226:55ff:fed6:56a9/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:15 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:1836 (1.7 KiB)
          Memory:ec180000-ec1a0000 


thanks for hints

> do: svn co svn+ssh://svn@savannah.psi.ch/afs/psi.ch/project/meg/svn/midas/trunk
> midas
> shows: ssh: connect to host savannah.psi.ch port 22: Connection timed out
> svn: Connection closed unexpectedly

sorry, my side network problem.
N.

do: svn co svn+ssh://svn@savannah.psi.ch/afs/psi.ch/project/meg/svn/midas/trunk
midas
shows: ssh: connect to host savannah.psi.ch port 22: Connection timed out
svn: Connection closed unexpectedly

Dear All,

After clicking the "Start" button, there is a page for the operator to change some 
ODB values. I have created "/Experiment/Edit on start" and added some links there. 
If the link is pointed to a boolean type key, a check box will appear in the 
"Start" page, which is great. But how about if I want to have some radio buttons 
or pull-down menus for the operator to select among different calibration sources 
or running modes?

Thanks,

Jimmy

Hi Exaos,

This may help: https://ladd00.triumf.ca/elog/Midas/645

You need to set the following keys:
/Logger/Run duration
/Logger/Auto restart
/Logger/Auto restart delay

Regards,
Jimmy


> I want to let the run stop and start periodically. But I looked through the ODB
> and didn't find anything may help. I also checked the FAQ online and didn't find
> answer either. Who can help me? Thank you.