ID |
Date |
Author |
Topic |
Subject |
402
|
22 Aug 2007 |
Konstantin Olchanski | Bug Fix | commit latest ccusb.c CAMAC-USB driver | > > I commited the latest driver for the Wiener CCUSB USB-CAMAC driver. It
> > implements all functions from mcstd.h and has been tested to be plug-compatible
> > with at least one of our CAMAC frontends. K.O.
Well, it took almost a year to finish an updated driver, which has now been
commited to MIDAS SVN (see http://savannah.psi.ch/viewcvs/trunk/drivers/camac/ccusb/?root=midas).
This supports ccusb firmware release 0x402. With earlier firmware, simple CAMAC operations should work,
but to use the readout list feature one has to have the latest main firmware (0x402 as of today) and the latest CPLD
firmware.
The driver kit includes:
- the "ccusb" driver which implements the MIDAS mcstd.h CAMAC interface;
- test_ccusb to probe the interface and generally make the lights flash;
- ccusb_flash for updating the ccusb main firmware (assembled from bits and pieces found on the CCUSB driver CD);
- feccusb, an example midas frontend, which uses the ccusb readout list feature and has extensive error handling,
should be good enough for production use (unlike the Wiener libxxusb drivers, which lack basic error handling).
- analyzer.cxx, an rootana-based example on how to decode the ccusb data;
- README file with release notes.
If you use this driver, please drop me an email (even if it works perfectly for you, hah!) - the ccusb device is very
nice but can be hard to use and I would like to hear about problems other people have with it.
Today's version of the README files is attached below:
MIDAS driver for the Wiener/JTec CC-USB CAMAC-USB interface.
Date: 22-AUG-2007/KO
Note 1: The CC-USB interface comes with a CD which contains manuals,
firmware files, Windows and Linux software. The Wiener/JTec driver
is called "libxxusb". These MIDAS/musbstd drivers were written before
libxxusb bacame available and do not use libxxusb.
This driver implements the MIDAS CAMAC interafce "mcstd.h" using
the MIDAS USB interface musbstd.h.
Note 2: There exist many revisions of CCUSB firmware. Basic CAMAC
access works in all of them, but the "readout list" feature seems
to be only functional with firmware revision 0x402 or older and
with CPLD revisions CC_atmmgr_101406.jed, CC_datamgr_021905.jed,
CC_lammgr_brdcst_041906.jed or older.
To upgrade the main CCUSB firmware, follow instructions from
the CCUSB manual. On Linux, one can use the ccusb_flash
program included with these MIDAS drivers. It is a copy
of ccusb_flash from the Wiener CD, with all the pieces
assembled into one place and with a working Makefile. (I am too
lazy to add the flashing bits to the ccusb.c driver).
To upgrade the CPLD firmware, one needs a Xilinx JTag programmer
cable (we use a "parallel port to JTag" cable provided by Wiener),
and the Xilinx software (on Linux, we use Xilinx91i). For successful
upgrade, follow instructions from Xilinx and Wiener.
Note 3: Before starting to use the CCUSB interface, one should obtain
the latest version of the CCUSB manual and firmware by downloading
the latest version the CCUSB driver CD from the Wiener web
site (registration required)
Note 4: The example CCUSB frontend assumes this hardware configuration:
LeCroy 2249A 12 channel ADC in slot 20, Kinetic Systems 3615 6 channel
scaler in slot 12. NIM trigger input connected to CCUSB input "I1"
firing at 10-100 Hz. Without the external trigger CCUSB will not
generate any data and the frontend will only give "data timeout"
errors. With the trigger, the LED on the scaler should flash at 1 Hz
and the LEDs on the CCUSB should flash at the trigger rate.
Note 5: The CCUSB interface does not reliably power up in some CAMAC
crates (this has something to do with the sequence in which
different voltages start at different times with different CAMAC
power supplies). Some newer CCUSB modules may have this
problem fixed in the hardware and in the CPLD firmware. For modules
exhibiting this problem (i.e. no USB communication after power up),
try to cycle the power several time, or implement the "hardware reset
switch" (ask Wiener).
Note 6: The CCUSB firmware is very fickle and would crash if you look
at it the wrong way. This MIDAS driver tries to avoid all known crashers
and together with the example frontend, can recover from some
of them. Other crashes cannot be recovered from other than by
a hardware reset or power cycle.
//end |
403
|
29 Aug 2007 |
Konstantin Olchanski | Forum | ODBv3, second try - Midas on a x86_64 - incompatible with x86_32 | > > > I agree to make 32-bit and 64-bit compatible. In the long run, everything will be 64-bit, so I would suggest
> > > in breaking the 32-bit ODB, add some padding there where needed, probably with some conditional compiling.
> > 1) midas.h: remove unused field "dispatch" from EVENT_REQUEST and bump DATABASE_VERSION from 2 to 3
> > 2) msystem.h: add 32-bit padding to CHN_STATISTICS and CHN_SETTINGS
I am now trying a different solution of to fixing the issue of CHN_STATISTICS and CHN_SETTINGS changing size.
1) midas.h: (same as before) remove unused field "dispatch" from EVENT_REQUEST and bump DATABASE_VERSION from 2 to 3
2) msystem.h: in CHN_STATISTICS and CHN_SETTINGS change type of "event_limit" and "files_written" from int to "double".
Below are the latest ODBv3 meta patches:
ladd03:midas$ svn diff
Index: include/midas.h
===================================================================
--- include/midas.h (revision 3844)
+++ include/midas.h (working copy)
/* has to be changed whenever binary ODB format changes */
-#define DATABASE_VERSION 2
+#define DATABASE_VERSION 3
.........
short int trigger_mask; /**< trigger mask */
INT sampling_type; /**< GET_ALL, GET_SOME, GET_FARM */
- /**< dispatch function */
- void (*dispatch) (HNDLE, HNDLE, EVENT_HEADER *, void *);
} EVENT_REQUEST;
Index: include/msystem.h
===================================================================
--- include/msystem.h (revision 3845)
+++ include/msystem.h (working copy)
-"Event limit = DWORD : 0",\
+"Event limit = DOUBLE : 0",\
..................
-"Files written = INT : 0",\
+"Files written = DOUBLE : 0",\
..................
- DWORD event_limit;
+ double event_limit;
..................
- INT files_written;
+ double files_written;
K.O. |
404
|
29 Aug 2007 |
Konstantin Olchanski | Info | Added data compression to mlogger | I now commited the changes to mlogger (mlogger.c, msystem.h) implementing data
compression using zlib (svn revision 3845)
To enable compression, observe that mlogger is compiled with -DHAVE_ZLIB (see
the Makefile), in "/Logger/Channels/NNN/Settings", set "compression" to "1" and
the filename to "run%05d.mid.gz" (note the suffix ".gz").
In the Makefile, I only enabled HAVE_ZLIB for Linux, as that is the only
platform I tested. If somebody can test compression on Windows, please do and
let us know.
My ROOT analyzer (rootana) package can read compressed MIDAS files directly and
if one wants to add this capability to other MIDAS-related packages, one is
welcome to use my TMidasFile.cxx as an example
(http://ladd00.triumf.ca/viewcvs/rootana/trunk/TMidasFile.cxx?view=markup).
K.O. |
407
|
02 Oct 2007 |
Konstantin Olchanski | Info | ROODY, ROOTANA updates | The ROODY online histogram viewer and the ROOTANA midas analyzer toolkit have been updated to work
with ROOT version 5.16 and tested on Linux (SL4.4) and MacOS (10.4.10/PPC).
This update includes the library called "TNetDirectory" for access to remote ROOT objects. This library is
still under development, but is complete enough for use with ROODY. To try it, please specify -P9091 in
rootana and -Plocalhost:9091 in ROODY.
K.O. |
416
|
20 Nov 2007 |
Konstantin Olchanski | Info | mhdump: a standalone MIDAS history dump utility | > > I hope people find this program useful. If you have any feedback (patches, bug
> > reports, requests for improvements), please post them as replies to this forum
> > message.
>
> I wouldn't mind putting this into the midas distribution. Put it under utils/, add
> an entry to the Makefile, and fix that warning:
>
>
> mhdump.cxx: In function `int readHstFile(FILE*)':
> mhdump.cxx:161: warning: comparison between signed and unsigned integer expressions
Done and done.
The program mhdump, a standalone decoder for midas history files, is now in midas svn.
K.O. |
417
|
21 Nov 2007 |
Konstantin Olchanski | Forum | ODBv3, second try - Midas on a x86_64 - incompatible with x86_32 |
These changes to make 32-bit and 64-bit ODB binary compatible with each other are now commited to midas svn, revision 4080.
Starting with this revision, ODB version changes from 2 to 3, breaking binary compatibility with previous releases.
Before upgrading to this revision, save your ODB as an XML file, *and* try to reload it, to catch any potential problems with parsing of the XML file.
Part of this commit are checks for sizes of important midas data structures stored in ODB shared memory - if the compiled size does not match the expected
value, binary compatibility is broken and the program will abort - to avoid further corruption of ODB shared memory. This feature is only enabled on Linux and
it is expected to trigger only on compiler malfunctions (generates wrong data size) and on accidental or intentional changes to important data structures in
midas, to warn the user that they broke ODB binary compatibility.
K.O.
> > > > I agree to make 32-bit and 64-bit compatible. In the long run, everything will be 64-bit, so I would suggest
> > > > in breaking the 32-bit ODB, add some padding there where needed, probably with some conditional compiling.
> > > 1) midas.h: remove unused field "dispatch" from EVENT_REQUEST and bump DATABASE_VERSION from 2 to 3
> > > 2) msystem.h: add 32-bit padding to CHN_STATISTICS and CHN_SETTINGS
>
> I am now trying a different solution of to fixing the issue of CHN_STATISTICS and CHN_SETTINGS changing size.
>
> 1) midas.h: (same as before) remove unused field "dispatch" from EVENT_REQUEST and bump DATABASE_VERSION from 2 to 3
> 2) msystem.h: in CHN_STATISTICS and CHN_SETTINGS change type of "event_limit" and "files_written" from int to "double".
>
> Below are the latest ODBv3 meta patches:
>
> ladd03:midas$ svn diff
> Index: include/midas.h
> ===================================================================
> --- include/midas.h (revision 3844)
> +++ include/midas.h (working copy)
> /* has to be changed whenever binary ODB format changes */
> -#define DATABASE_VERSION 2
> +#define DATABASE_VERSION 3
> .........
> short int trigger_mask; /**< trigger mask */
> INT sampling_type; /**< GET_ALL, GET_SOME, GET_FARM */
> - /**< dispatch function */
> - void (*dispatch) (HNDLE, HNDLE, EVENT_HEADER *, void *);
> } EVENT_REQUEST;
>
> Index: include/msystem.h
> ===================================================================
> --- include/msystem.h (revision 3845)
> +++ include/msystem.h (working copy)
> -"Event limit = DWORD : 0",\
> +"Event limit = DOUBLE : 0",\
> ..................
> -"Files written = INT : 0",\
> +"Files written = DOUBLE : 0",\
> ..................
> - DWORD event_limit;
> + double event_limit;
> ..................
> - INT files_written;
> + double files_written;
>
> K.O. |
428
|
13 Feb 2008 |
Konstantin Olchanski | Info | Implementation of relative paths in mhttpd | > A major change was made to mhttpd, changing all internal URLs to relative paths.
> This allows proxy access to mhttpd via an apache server for example, which might
> be needed to securely access an experiment from outside the lab through a
> firewall.
It is good to see improvements to the MIDAS URLs. We have been successfully running
mhttpd behind an apache SSL/HTTPS proxy without these changes, but our case was very
limited to one experiment, one mhttpd behind one proxy. I hope to test these changes
in the near future at CERN, I guess we will hear if things broke. I am especialloy
worried about the function for "split mhttpd history generator" via "/History/URL".
I remember it was hard to get it right and I hope if this function did not survive
this update, it will be easy to resurrect.
K.O. |
429
|
13 Feb 2008 |
Konstantin Olchanski | Info | Roll-back for history sytem added | > The midas history system always had the problem that the database can get
> corrupted if the disk gets full where the history records (*.hst & *.idx) are
> stored.
Stefan - big thanks for fixing this problem - it is one of those cases "how come I
did not think of do it!".
This change should fix the last remaining problem with history at CERN - we seem to
be unable to avoid running out of disk space once in a while (run away scripts, fat
fingers, etc) and history got corrupted every time.
But to make things more interesting we had another history outage this week - we
happen to write history files to an NFS server (not recommened! do not do this!) and
when the NFS server had a glitch, history files got corrupted - because during the
glitch NFS was not available, I think this roll-back feature would not have helped.
Anyhow, I now have a patch to allow hs_read() to "skip the bad spots" in the history
files. (hs_gen_index() also needs a patch).
In the nutshell, if invalid history data is detected, the code continues to read the
data one byte at a time, looking for valid event_id markers (etc).
The code looks sane by inspection, and if nobody objects, I would like to commit it
in the next few days.
Here is the diff against src/history.c rev 4114
Index: history.c
===================================================================
--- history.c (revision 4118)
+++ history.c (working copy)
@@ -129,6 +129,7 @@
HIST_RECORD rec;
INDEX_RECORD irec;
DEF_RECORD def_rec;
+ int recovering = 0;
printf("Recovering index files...\n");
@@ -171,7 +172,7 @@
/* skip tags */
lseek(fh, rec.data_size, SEEK_CUR);
- } else {
+ } else if (rec.record_type == RT_DATA) {
/* write index record */
irec.event_id = rec.event_id;
irec.time = rec.time;
@@ -180,6 +181,15 @@
/* skip data */
lseek(fh, rec.data_size, SEEK_CUR);
+ } else {
+
+ if (!recovering)
+ cm_msg(MERROR, "hs_gen_index", "broken history file %d, trying to
recover", (int)ltime);
+
+ recovering = 1;
+ lseek(fh, -sizeof(rec)+1, SEEK_CUR);
+
+ continue;
}
} while (TRUE);
@@ -220,6 +230,7 @@
time_t lt;
int fh, fhd, fhi;
struct tm *tms;
+ int idxsize = 0;
if (*ltime == 0)
*ltime = ss_time();
@@ -250,12 +261,15 @@
hs_open_file(*ltime, "idf", O_RDONLY, &fhd);
hs_open_file(*ltime, "idx", O_RDONLY, &fhi);
+ if (fhi >= 0)
+ idxsize = lseek(fhi, 0, SEEK_END);
+
close(fh);
close(fhd);
close(fhi);
/* generate them if not */
- if (fhd < 0 || fhi < 0)
+ if (fhd < 0 || fhi < 0 || idxsize == 0)
hs_gen_index(*ltime);
return HS_SUCCESS;
@@ -1480,12 +1494,33 @@
i = -1;
M_FREE(cache);
cache = NULL;
- } else
+ } else {
+
+ try_again:
+
i = sizeof(irec);
-
- if (cp < cache_size) {
memcpy(&irec, cache + cp, sizeof(irec));
cp += sizeof(irec);
+
+ /* if history file is broken ... */
+ if (irec.time < last_irec_time) {
+ //printf("time %d -> %d, cache_size %d, cp %d\n", last_irec_time, irec.time,
cache_size, cp);
+
+ //printf("Seeking next record...\n");
+
+ while (cp < cache_size)
+ {
+ DWORD* evidp = (DWORD*)(cache + cp);
+ if (*evidp == event_id) {
+ //printf("Found at cp %d\n", cp);
+ goto try_again;
+ }
+
+ cp++;
+ }
+
+ i = -1;
+ }
}
} else
i = read(fhi, (char *) &irec, sizeof(irec));
K.O. |
430
|
13 Feb 2008 |
Konstantin Olchanski | Info | mhttpd history display updates | I now merged almost all the mhttpd changes from CERN AD-5/ALPHA. Only the code
for mhttpd HTTP:// access control list remains unmerged.
Changes to the history display code merged from ALPHA:
- add option to show latest values of history variables: "show values of
variables" check box on the history config panel
- add custom labels for each variables: instead of midas variable name, history
plots would show the text entered into the "label" text area
- show history errors on the plot: before, if one out of 10 history variables
could not be plotted, nothing was shown at all, now all variables are \
show, those that could not be read with hs_read() show the error code
- the selection of which variables to plot is alphanumerically sorted (adc11 >
adc9) [this code is not active for standard midas because mlogger sup\
port has not yet been committed]
- the selection of which variables to plot shows the last variable selected, not
the first one - useful when entering variables from a long list [th\
is code is not active for standard midas because mlogger support has not yet
been committed]
These changes have been extensively tested since last Summer at the AD-5 ALPHA
expt at CERN.
I could only do minimal testing for this merged code, so if there are any errors,
they would most likely be merge errors. This new code will be heavily used at
TRIUMF,
so if any errors got any, we hope to flush them out quickly.
As noted, mlogger support for some of the mhttpd functions is not in standard
midas yet. It will be committed shortly.
K.O. |
433
|
18 Feb 2008 |
Konstantin Olchanski | Bug Report | mhttpd safari 3.0.4 redirect problem | I now encountered a new problem with mhttpd - I connect using the Safari 3.0.4 browser, go to the
"Programs" page, press the button "Start feplc" (or any other "start" button) and instead of starting this
program, I get an error in the browser, funny entries in ODB in "/Programs", corrupted ODB and a spew
of messages in the midas log file about the mess. ODB has to be reloaded from backup to recover.
Investigation shows that the culprit is odd bahaviour of the "redirect()" function:
/* start command */
if (*getparam("Start")) {
/* for NT: close reply socket before starting subprocess */
- redirect2("?cmd=programs");
+ redirect2("/?cmd=programs");
The version without "/" makes Safari explode - it appends the "?cmd..." stuff to the existing URL, which
already has the "?cmd..." tags, making a mess.
Firefox accepts either version.
ODB corruption happens here:
sprintf(str, "/Programs/%s/Start command", name);
- db_get_value(hDB, 0, str, command, &size, TID_STRING, TRUE);
+ db_get_value(hDB, 0, str, command, &size, TID_STRING, FALSE);
if (command[0]) {
ss_system(command);
It looks like db_get_value() would corrupt ODB if given funny "str". When Safari explodes,
funny strings are generated.
The simple fix is to replace "TRUE" with "FALSE", then at least db_get_value() does not try to make bogus
entries in ODB.
The "Stop" command has the same problem, but does not currupt ODB - there is no db_get_value() in
that code path.
I am reporting this "fresh" as I made one of our daq systems work again.
I did not investigate the history of changes to this "redirect" command (perhaps it was broken in the
recent reorganisation of midas urls?), what versions of Safari work or not.
K.O. |
434
|
18 Feb 2008 |
Konstantin Olchanski | Bug Report | potential memory corruption in odb,c:extract_key() | It looks like ODB function extract_key() will overwrite the array pointed to by "key_name" if given an odb
path with very long names (as seems to happen when redirection explodes in the Safari web browser, via
db_get_value(TRUE) via mhttpd "start program" button). All callers of this function seem to provide 256
byte strings, so the problem would not show up in normal use - only when abnormal odb paths are being
parsed. Proposed solution is to add a "length" argument to this function. (Actually ODB path elements
should be restricted to NAME_LENGTH (32 bytes), right?). K.O. |
443
|
21 Feb 2008 |
Konstantin Olchanski | Info | mhttpd history display updates | > You misspelled one ODB entry:
> Line 9014:
> sprintf(str, "/History/Display/%s/Label", path);
>
> Line 9028:
> sprintf(str, "/History/Display/%s/Labels", path);
> ---^
>
> I wonder how you could have tested that code for 1/2 year without noticing this error.
> I fixed and committed it.
It turns out that the program was tested as originally committed. With the above
modification, it corrupts ODB - originally, it used the wrong array element size to create
the wrong array. Corrected, it creates the right array with the wrong size, then
subsequent db_set_data_index() happily corrupts ODB.
Fix for mhttpd committed as svn revision 4128.
Fix for ODB corruption committed at svn revision 4129 (also fixes extract_key())
K.O. |
444
|
21 Feb 2008 |
Konstantin Olchanski | Bug Report | potential memory corruption in odb,c:extract_key() | > It looks like ODB function extract_key() will overwrite the array pointed to by "key_name" if given an odb
> path with very long names (as seems to happen when redirection explodes in the Safari web browser, via
> db_get_value(TRUE) via mhttpd "start program" button). All callers of this function seem to provide 256
> byte strings, so the problem would not show up in normal use - only when abnormal odb paths are being
> parsed. Proposed solution is to add a "length" argument to this function. (Actually ODB path elements
> should be restricted to NAME_LENGTH (32 bytes), right?). K.O.
This is fixed in svn revision 4129.
K.O. |
446
|
27 Feb 2008 |
Konstantin Olchanski | Bug Report | NEED_SHLIB=1 is broken | --- Makefile (revision 4129)
+++ Makefile (working copy)
- $(LIB_DIR)/mxml.o $(LIB_DIR)/cnaf_callback.o \
+ $(LIB_DIR)/mxml.o \
> i.e. remove cnaf_callback.o which causes the link errors.
Hi, Denis - I confirm that cnaf_callback.c is only used by MIDAS frontends that implement CAMAC
functions and that it should not required for building the MIDAS library. I am now looking at removing
it from libmidas.
> I propose that libmidas.so is built by default, so when something breaks it won't go unnoticed
We have been through this before and decided that shared libraries are bad and we do not want to use
them. The option for building libmidas.so was preserved, though.
Not to refight old wars, on reason against using shared libraries was version skew - one could never be
sure what version of midas is being used - depending on the PATH, LD_LIBRARY_PATH, rpath settings,
etc. There were other reasons, perhaps practical, perhaps with the mserver.
The main problem with "just build it", is that then the rest of midas will link against it bringing back all
the problems we solved by going away from using shared libraries.
So back to your proposal about building libmidas.so - can you look and see if you can do the Python
bindings with a statically linked midas library?
I know it is possible with Perl bindings - perl creates it's own shared library containing perl api glue
linked against a foreign static library libfoo.a , so in theory, the shared library is not needed.
But perhaps, Python do things differently...
K.O. |
447
|
27 Feb 2008 |
Konstantin Olchanski | Info | CAMAC register_cnaf_callback() - removed from libmidas | > > Affected files:
> > Makefile (add cnaf_callback.o)
> That's a good idea.
> To make things a bit easier, I modified the midas\examples\experiment\fronted.c to
> contain this call, so people should be guided by that. I also added cnaf_callback.c
> to the Makefile of the example frontend.
A request was made to remove cnaf_callback.o from libmidas as it creates a unwanted dependency on the CAMAC
hardware driver when libmidas.so is used in programs that do not use CAMAC.
After looking around, it appears that removing cnaf_callback.o from libmidas would not break anything critical,
other than CAMAC frontends that would fail to link with an obvious and easy to fix error.
I am leaving cnaf_callback.o in the Makefile - so it will be built and placed in linux/lib/cnaf_callback.o for anybody
who wants to use it.
svn revision 4130.
K.O. |
448
|
27 Feb 2008 |
Konstantin Olchanski | Bug Report | mhttpd: cannot attach history to elog | From "history" pages, the "create elog" button stopped working - it takes us to the elog entry form, but
then, the "submit" button does not create any elog entries, instead dumps us into an invalid history
display. This is using the internal elog.
This change in mhttpd.c::show_elog_new() makes it work again:
- ("<body><form method=\"POST\" action=\"./\" enctype=\"multipart/form-data\">\n");
+ ("<body><form method=\"POST\" action=\"/EL/\" enctype=\"multipart/form-data\">\n");
Problem and fix confirmed with Linux/firefox and MacOS/firefox and Safari.
K.O. |
449
|
28 Feb 2008 |
Konstantin Olchanski | Bug Report | mhttpd: cannot attach history to elog | > From "history" pages, the "create elog" button stopped working - it takes us to the elog entry form, but
> then, the "submit" button does not create any elog entries, instead dumps us into an invalid history
> display. This is using the internal elog.
>
> This change in mhttpd.c::show_elog_new() makes it work again:
> - ("<body><form method=\"POST\" action=\"./\" enctype=\"multipart/form-data\">\n");
> + ("<body><form method=\"POST\" action=\"/EL/\" enctype=\"multipart/form-data\">\n");
This was a problem with relative URLs and it is now fixed. Svn revision 4131, fixes: delete elog, make elog from odb, make elog from history.
K.O. |
454
|
07 Mar 2008 |
Konstantin Olchanski | Suggestion | RFC- ACLs for midas rpc, mserver, mhttpd access | The mhttpd host-based access control list as used by ALPHA at CERN is now committed to
SVN (revision 4135).
When accepting connection from a remote host, the remote IP address is converted to a
hostname using gethostbyaddr(). If ODB directory "/experiment/security/mhttpd hosts",
exists, access is permitted if there is an entry for the this hostname. "localhost" is
always permitted.
In other words:
1) To enable the mhttpd access control list, create an ODB directory
"/experiment/security/mhttpd hosts".
2) From this moment, only access from "localhost" is permitted.
3) All connections from remote hosts are rejected with an error written into the midas
log file: Rejecting http connection from 'ladd05.triumf.ca'.
4) To permit access from remote hosts, take the hostname from this error message and
create an entry in "mhttpd hosts": odbedit -> cd "/Experiment/Security/mhttpd hosts" ->
create INT ladd05.triumf.ca
The idea behind this is that mhttpd is running behind an SSL proxy (or an SSH tunnel)
and only accepts connections from this proxy and perhaps from selected machines in the
experiment counting room.
P.S. I considered using tcp_wrappers, but this package does not seem to contain any
simple-to-use function "bool areTheyPermitted(const char* remoteHostname);".
P.P.S. The ODB path name is in variance from Stefan's email. I committed this code
before rereading it, please let me know if I should change the ODB paths.
P.P.P.S. I will now proceed with implementing similar code for the mserver/midas rpc.
Again, the use case is very simple: all machines permitted access to the mserver are
known in advance and can be listed in the access list. All unknown machines should be
rejected.
K.O. |
459
|
10 Mar 2008 |
Konstantin Olchanski | Suggestion | New Makefile for building MIDAS | > I rewrote the Makefile for MIDAS in order to make it tidy.
Not that the current Makefile is too pretty (I have seen worse), but it works and it is fairly compact for a project of
this complexity, it handles a large number of operating systems and build options very efficiently.
I think you found that out with your rewriting exercise - your version of the Makefile contains all the same code,
just rearranged to suite your taste, with existing bugs preserved and new bugs added.
> I tested it on my box and it works here.
As they say, the devil is in the details. I notice some subtle changes in your Makefile that make me go "what?":
1) the command for building the midas shared library used to be "ld -shared", in your version, "-shared" is gone.
But check with the GCC manual, today's recommended command is probably "gcc -shared".
2) mhdump is now linked with ROOT, but I wrote it recently enough to remember that it does not use ROOT
3) hand-crafted dependancies have been replaced with generic "almost every .o depends on every .h", which is
incorrect. The "almost every .o" part bothers me.
4) "make clean" runs "rm -rf" - plain scary.
5) "$(shell ...)" is overused
I think by the end all these little details are sorted out and all the quirks are put back in, your Makefile will look no
better than the current Makefile.
> 2. The file is less than 400 lines now. The original one is more than 500 lines.
It looks like your savings came from removing comments, removing hand-crafted dependancy lists and replacing
fairly verbose "make install" targets (which we do not use anyway) with your own much simpler scripts.
All the juicy bits needed to actually build all the code appear to take about as much space as before.
Also the original mistake of recompiling programs when they only need relinking was not fixed. (For example,
when libmidas is updated, to update mhttpd, the current Makefile needlessly recompiles mhttpd.c. Better use
would be to compile mhttpd.c into mhttpd.o, then only a relink is needed).
> I tried to learn "autoconf" and "automake" in order to make building MIDAS more
> compatible for various platforms. But I havn't enough time now. Hope somebody
> can help it. The attached file is original named "Makefile.in" for using "autoconf".
Most experience with autoconf/automake is all negative. The promise was "never debug your Makefile ever
again!", delivered was "debug the configure script instead!". In practice, with autoconf/automake, you try to run
configure, kludge it until it stops crashing, then tweak the incomprehensible Makefiles it produces until the code
compiles.
K.O. |
460
|
10 Mar 2008 |
Konstantin Olchanski | Suggestion | RFC- ACLs for midas rpc, mserver, mhttpd access | > While your "positive list" will certainly work, it is much more inflexible than a more
> general hosts.allow/hosts.deny with wildcards. Assume some experiment decides it wants to
> be controlled from all inside CERN. With hosts.allow/deny you could do
I was going to bring this up later, but since mhttpd does not pass security audits, I believe
the only way it should be run in the modern computing environement is behind
a password-protected SSL proxy. In this case, the allow/deny list is very simple: deny all,
allow localhost (assuming httpd runs on the same host as mhttpd).
Speaking about CERN, "deny all; allow *.cern.ch" is the "default" setting, enforced by the CERN firewall. Our problem is with
random "*.cern.ch" computers poking at our DAQ and crashing the mserver. Plus we do not want our competition to access our
DAQ system, so "allow *.cern.ch" is a no go.
But since hosts.allow/hosts.deny is a superset of what I want, and since we can reuse existing code from elogd, I guess I have
no ground to object your suggestion.
I will do the mserver/mrpc this way, then retrofit it into mhttpd. (But have to commit mlogger history changes first!!!).
K.O. |
|