| ID |
Date |
Author |
Topic |
Subject |
|
449
|
28 Feb 2008 |
Konstantin Olchanski | Bug Report | mhttpd: cannot attach history to elog | > From "history" pages, the "create elog" button stopped working - it takes us to the elog entry form, but
> then, the "submit" button does not create any elog entries, instead dumps us into an invalid history
> display. This is using the internal elog.
>
> This change in mhttpd.c::show_elog_new() makes it work again:
> - ("<body><form method=\"POST\" action=\"./\" enctype=\"multipart/form-data\">\n");
> + ("<body><form method=\"POST\" action=\"/EL/\" enctype=\"multipart/form-data\">\n");
This was a problem with relative URLs and it is now fixed. Svn revision 4131, fixes: delete elog, make elog from odb, make elog from history.
K.O. |
|
454
|
07 Mar 2008 |
Konstantin Olchanski | Suggestion | RFC- ACLs for midas rpc, mserver, mhttpd access | The mhttpd host-based access control list as used by ALPHA at CERN is now committed to
SVN (revision 4135).
When accepting connection from a remote host, the remote IP address is converted to a
hostname using gethostbyaddr(). If ODB directory "/experiment/security/mhttpd hosts",
exists, access is permitted if there is an entry for the this hostname. "localhost" is
always permitted.
In other words:
1) To enable the mhttpd access control list, create an ODB directory
"/experiment/security/mhttpd hosts".
2) From this moment, only access from "localhost" is permitted.
3) All connections from remote hosts are rejected with an error written into the midas
log file: Rejecting http connection from 'ladd05.triumf.ca'.
4) To permit access from remote hosts, take the hostname from this error message and
create an entry in "mhttpd hosts": odbedit -> cd "/Experiment/Security/mhttpd hosts" ->
create INT ladd05.triumf.ca
The idea behind this is that mhttpd is running behind an SSL proxy (or an SSH tunnel)
and only accepts connections from this proxy and perhaps from selected machines in the
experiment counting room.
P.S. I considered using tcp_wrappers, but this package does not seem to contain any
simple-to-use function "bool areTheyPermitted(const char* remoteHostname);".
P.P.S. The ODB path name is in variance from Stefan's email. I committed this code
before rereading it, please let me know if I should change the ODB paths.
P.P.P.S. I will now proceed with implementing similar code for the mserver/midas rpc.
Again, the use case is very simple: all machines permitted access to the mserver are
known in advance and can be listed in the access list. All unknown machines should be
rejected.
K.O. |
|
459
|
10 Mar 2008 |
Konstantin Olchanski | Suggestion | New Makefile for building MIDAS | > I rewrote the Makefile for MIDAS in order to make it tidy.
Not that the current Makefile is too pretty (I have seen worse), but it works and it is fairly compact for a project of
this complexity, it handles a large number of operating systems and build options very efficiently.
I think you found that out with your rewriting exercise - your version of the Makefile contains all the same code,
just rearranged to suite your taste, with existing bugs preserved and new bugs added.
> I tested it on my box and it works here.
As they say, the devil is in the details. I notice some subtle changes in your Makefile that make me go "what?":
1) the command for building the midas shared library used to be "ld -shared", in your version, "-shared" is gone.
But check with the GCC manual, today's recommended command is probably "gcc -shared".
2) mhdump is now linked with ROOT, but I wrote it recently enough to remember that it does not use ROOT
3) hand-crafted dependancies have been replaced with generic "almost every .o depends on every .h", which is
incorrect. The "almost every .o" part bothers me.
4) "make clean" runs "rm -rf" - plain scary.
5) "$(shell ...)" is overused
I think by the end all these little details are sorted out and all the quirks are put back in, your Makefile will look no
better than the current Makefile.
> 2. The file is less than 400 lines now. The original one is more than 500 lines.
It looks like your savings came from removing comments, removing hand-crafted dependancy lists and replacing
fairly verbose "make install" targets (which we do not use anyway) with your own much simpler scripts.
All the juicy bits needed to actually build all the code appear to take about as much space as before.
Also the original mistake of recompiling programs when they only need relinking was not fixed. (For example,
when libmidas is updated, to update mhttpd, the current Makefile needlessly recompiles mhttpd.c. Better use
would be to compile mhttpd.c into mhttpd.o, then only a relink is needed).
> I tried to learn "autoconf" and "automake" in order to make building MIDAS more
> compatible for various platforms. But I havn't enough time now. Hope somebody
> can help it. The attached file is original named "Makefile.in" for using "autoconf".
Most experience with autoconf/automake is all negative. The promise was "never debug your Makefile ever
again!", delivered was "debug the configure script instead!". In practice, with autoconf/automake, you try to run
configure, kludge it until it stops crashing, then tweak the incomprehensible Makefiles it produces until the code
compiles.
K.O. |
|
460
|
10 Mar 2008 |
Konstantin Olchanski | Suggestion | RFC- ACLs for midas rpc, mserver, mhttpd access | > While your "positive list" will certainly work, it is much more inflexible than a more
> general hosts.allow/hosts.deny with wildcards. Assume some experiment decides it wants to
> be controlled from all inside CERN. With hosts.allow/deny you could do
I was going to bring this up later, but since mhttpd does not pass security audits, I believe
the only way it should be run in the modern computing environement is behind
a password-protected SSL proxy. In this case, the allow/deny list is very simple: deny all,
allow localhost (assuming httpd runs on the same host as mhttpd).
Speaking about CERN, "deny all; allow *.cern.ch" is the "default" setting, enforced by the CERN firewall. Our problem is with
random "*.cern.ch" computers poking at our DAQ and crashing the mserver. Plus we do not want our competition to access our
DAQ system, so "allow *.cern.ch" is a no go.
But since hosts.allow/hosts.deny is a superset of what I want, and since we can reuse existing code from elogd, I guess I have
no ground to object your suggestion.
I will do the mserver/mrpc this way, then retrofit it into mhttpd. (But have to commit mlogger history changes first!!!).
K.O. |
|
470
|
12 Mar 2008 |
Konstantin Olchanski | Suggestion | New Makefile for building MIDAS | > > Most experience with autoconf/automake is all negative. The promise was "never debug your Makefile ever
> > again!", delivered was "debug the configure script instead!".
>
> I admit that the new one is fit to my flavor. For a common user, I think, a simple procedure of configure/make/install
> is better than changing the Makefile manually because many users are lack of knowledges about Makefile. That's why
> I want to learn autotools.
The reality is that you will never deliver a Makefile/Configure script that works for everybody in every case - users will always have a need to tweak the build
process to suit their needs. In this situation, "Makefile" is a much better language and "make" is a much better tool for users to deal with - much simpler, better
documented and better understood compared to autotools (*nobody* understands autotools; also compare the size of the midas Makefile with the size of a
typical configure script).
Anyhow, we will be cross-compiling midas to run on a PowerPC processor inside a Virtex4 FPGA. Go handle that with configure scripts.
K.O. |
|
471
|
23 Mar 2008 |
Konstantin Olchanski | Info | Per-variable history implementation in the mlogger | The changes to mlogger implementing per-variable history have been committed to
svn. Revision 4145.
The rationale for these changes is roughly described in
https://ladd00.triumf.ca/elog/Midas/347
The main user-visible effect is reduction of data volume written to history
files and better integration with the history plot system in mhttpd.
The new functionality is disabled by default, pending review by Stefan (Except
for /history/tags stuff, which will be created by mlogger and used by mhttpd).
To enable it, set "/equipment/xxx/Common/PerVariableHistory" to 1 (type TID_INT).
In the "per-variable" mode, each entry in /equipment/xxx/variables is assigned
it's own event id and creates it's own events in the history file. In the
"classical" (or per-equipment) mode, all variables are assigned the same event
id (equal to the equipment id) and are written to disk at the same time.
In other words, in per-equipment mode, if there are 100 variables and 1 of them
is updated, all 100 numbers are written to disk. In per-variable mode, only the
one updated variable is written out.
The one point for review in this implementation is the assignment of event id's.
Committed code uses the formula "1000*eq_id + n" (i.e. variables in equipment id
2 get 2001, 2002, etc..., equipment id 3 get 3001, 3002, ...). This formula
works for most experiments, but as I understand is no good for some experiments
at PSI. Other than inventing a better formula that would work for everybody in
every case, one can also assign event id's manually by creating appropriate
entries in "/history/events".
This code has been used at CERN for running ALPHA since last Summer and it will
be used extensively at TRIUMF for T2K/ND280 slow controls. Per-variable history
is also required for the pending implementation of "history logged directly to
an SQL database", to be used at T2K/ND280.
If history (ahem) is any guide, we will now have a brief period of fixing merge
errors and "works for me" mistakes.
K.O. |
|
472
|
23 Mar 2008 |
Konstantin Olchanski | Info | Per-variable history implementation in the mlogger | > The changes to mlogger implementing per-variable history have been committed to
> svn. Revision 4145.
To make code changes more clear, the commit was done in 3 stages:
revision 4142+4143 are minor fixes, refactoring (switch the code to use helper
functions) and implementation of history for structured banks
revision 4144 implements the per-variable history
revision 4145 is minor cleanup.
K.O. |
|
473
|
23 Mar 2008 |
Konstantin Olchanski | Info | History SQL database poll: MySQL, PgSQL, ODBC? | I would like to hear from potential users on which SQL database would be
preferable for storage of MIDAS history data.
My current preference is to use the ODBC interface, leaving the choice of
database engine to the user. While ODBC is not pretty, it appears to be adequate
for the job, permits "funny" databases (i.e. flat files) and I already have
prototype implementations for reading (mhttpd) and writing (mhdump/mlogger)
history data using ODBC.
In practice, MySQL and PgSQL are the main two viable choices for using with the
MIDAS history system. We tested both (no change in code - just tell ODBC which
driver to use) and both provide comparable performance and disk space use. We
were glad to see that the disk space use by both SQL databases is very
efficient, only slightly worse than uncompressed MIDAS history files.
At TRIUMF, for T2K/ND280, we now decided to use MySQL - it provides a better
match to MIDAS data types (has 1-byte and 2-byte integers, etc) and appears to
have working database replication (required for our use).
With mlogger already including support for MySQL, and MySQL being a better match
for MIDAS data, this gives them a slight edge and I think it would be reasonable
choice to only implement support for MySQL.
So I see 3 alternatives:
1) use ODBC (my preference)
2) use MySQL exclusively
3) implement a "midas odbc layer" supporting either MySQL or PgSQL.
Before jumping either way, I would like to hear from you folks.
K.O. |
|
475
|
02 Apr 2008 |
Konstantin Olchanski | Info | add "const" attributes to db_xxx() functions | Now that we use more and more C++, lack of "const" attribute on most midas functions is causing some
problems. I am now ready to commit changes to midas.h and odb.c that add the const attributes to ODB
access functions db_xxx(), i.e.
INT db_rename_key(HNDLE hDB, HNDLE hKey, char *name)
becomes
INT db_rename_key(HNDLE hDB, HNDLE hKey, const char *name)
If we proceed with this conversion, and it does not cause major havoc, I can continue and "const"ify the
rest of midas.h. I note that the mxml functions appear to already have the correct "const" declarations.
P.S. Adding the "const" attribute caught a few places where we were modifying a "char*" string passed by
the caller. This is undesirable if we are passed a string literal, i.e. db_rename_key(...,"foo"), and it is a
complete disaster in conjunction with C++ strings, i.e. db_rename_key(...,foo.c_str())
K.O. |
|
476
|
02 Apr 2008 |
Konstantin Olchanski | Info | add "const" attributes to db_xxx() functions | Now that we use more and more C++, lack of "const" attribute on most midas functions is causing some
problems. I am now ready to commit changes to midas.h and odb.c that add the const attributes to ODB
access functions db_xxx(), i.e.
INT db_rename_key(HNDLE hDB, HNDLE hKey, char *name)
becomes
INT db_rename_key(HNDLE hDB, HNDLE hKey, const char *name)
If we proceed with this conversion, and it does not cause major havoc, I can continue and "const"ify the
rest of midas.h. I note that the mxml functions appear to already have the correct "const" declarations.
P.S. Adding the "const" attribute caught a few places where we were modifying a "char*" string passed by
the caller. This is undesirable if we are passed a string literal, i.e. db_rename_key(...,"foo"), and it is a
complete disaster in conjunction with C++ strings, i.e. db_rename_key(...,foo.c_str())
K.O. |
|
478
|
03 Apr 2008 |
Konstantin Olchanski | Info | add "const" attributes to db_xxx() functions | > > I am now ready to commit changes to midas.h and odb.c that add the const attributes to ODB
> > access functions db_xxx(), i.e.
> > INT db_rename_key(HNDLE hDB, HNDLE hKey, char *name)
> > becomes
> > INT db_rename_key(HNDLE hDB, HNDLE hKey, const char *name)
>
> I fully approve your idea.
Committed revision 4172.
K.O. |
|
479
|
30 Apr 2008 |
Konstantin Olchanski | Info | triumf elog updated to elog-2.7.3-1.i386.rpm | FYI - in conjunction with replacement of ladd00.triumf.ca, this MIDAS ELOG has been updated to the latest
version 2.7.3-2058. Please report any problems or anomalies. K.O. |
|
480
|
20 May 2008 |
Konstantin Olchanski | Bug Report | pending problems and fixes from triumf | Here is the list of known problems I am aware of and of fixes not yet committed
to midas svn: |
|
481
|
20 May 2008 |
Konstantin Olchanski | Bug Report | pending problems and fixes from triumf | Here is the list of known problems I am aware of and of fixes not yet committed
to midas svn:
1) added variable /equiment/foo/common/PerVariableHistory breaks stuff (mostly
mhttpd). It is not clear how this problem escaped my pre-commit checks. This
per-equipment variable enables the per-variable history for the given equipment.
Local consensus is that this variable should not be in "common" and should not
be in "settings". Probably in "/history"? Or have only one variable to enable
this for all equipments at once (like we do in ALPHA).
2) writing compressed midas files (foo.mid.gz) crashes the mlogger when file
size reaches 2 GBytes. This problem could be new in SL5.1.
3) when a midas client becomes unresponsive, runs cannot be stopped using the
"stop" button in mhttpd. This is because cm_transition() loops over all attached
clients, but never removes clients that are known to be dead. Proposed fix is to
call cm_check_client() for each client before calling their rpc transition handler.
4) the discussed before fix for reading broken history files (skip bad data).
5) mhttpd history "export" button needs to be fixed (by request from ALPHA). At
present it either does not return all exiting data or crashes mhttpd. (no fix)
6) mhttpd ODB editor in "set value" page, the "cancel" button is broken (needs
to be corrected for "relative URL"). (no fix)
7) mhttpd needs AJAX-style methods for reading and writing ODB. (no fix)
K.O. |
|
482
|
28 May 2008 |
Konstantin Olchanski | Info | Roll-back for history sytem added | > > But to make things more interesting we had another history outage this week...
> > Anyhow, I now have a patch to allow hs_read() to "skip the bad spots" in history files.
>
> [Stefan suggested]
>
> if ((irec.time - last_irec_time) > 3600*24)
Yes, your stronger check works quite nicely. The whole patch is now committed into SVN,
revision 4202.
This is how it all works:
0) teach hs_gen_index() to skip over bad data. This is important because hs_read() only
looks at data records listed in the index file: if bad data is omitted from the index,
hs_read() will never see it and we do not need to worry about it in hs_read().
0a) because hs_gen_index() does not check validity of time stamps, we still need to check
them in hs_read().
1) in hs_read(), if we detect bad data (invalid headers, bad time stamps, etc), we
regenerate the index files - this removes a while class of bad data. We also look at time
stamps carefully and ignore records where time goes backwards (usually bad data) and ignore
records with time in the future beyound the end of the current history file (each history
file only contains 24*60*60 seconds = 1 day's worth of data).
While certainly not bullet-proof, these changes should make it easier to deal with
corruption of history files.
K.O. |
|
483
|
28 May 2008 |
Konstantin Olchanski | Bug Report | pending problems and fixes from triumf | > Here is the list of known problems I am aware of and of fixes not yet committed
> to midas svn:
>
> 1) added variable /equiment/foo/common/PerVariableHistory
corrected in svn revision 4203, read
http://savannah.psi.ch/viewcvs/trunk/src/mlogger.c?root=midas&rev=4203&sortby=rev&view=log
> 2) writing compressed midas files (foo.mid.gz) crashes the mlogger when file
> size reaches 2 GBytes. This problem could be new in SL5.1.
(no change)
> 3) when a midas client becomes unresponsive, runs cannot be stopped using the
> "stop" button in mhttpd. This is because cm_transition() loops over all attached
> clients, but never removes clients that are known to be dead. Proposed fix is to
> call cm_check_client() for each client before calling their rpc transition handler.
Fixed in SVN revision 4198, read
http://savannah.psi.ch/viewcvs/trunk/src/midas.c?root=midas&rev=4201&sortby=rev&view=log
> 4) the discussed before fix for reading broken history files (skip bad data).
Fixed in SVN revision 4202, read https://ladd00.triumf.ca/elog/Midas/482
> 5) mhttpd history "export" button needs to be fixed (by request from ALPHA). At
> present it either does not return all exiting data or crashes mhttpd. (no fix)
(no change)
> 6) mhttpd ODB editor in "set value" page, the "cancel" button is broken (needs
> to be corrected for "relative URL").
Apply this patch to src/mhttpd.c
@@ -11156,10 +11190,7 @@
sprintf(str, "SC/%s/%s", eq_name, group);
redirect(str);
} else {
- strlcpy(str, path, sizeof(str));
- if (strrchr(str, '/'))
- strlcpy(str, strrchr(str, '/')+1, sizeof(str));
- redirect(str);
+ redirect("./");
}
> 7) mhttpd needs AJAX-style methods for reading and writing ODB. (no fix)
(no change)
K.O. |
|
484
|
29 May 2008 |
Konstantin Olchanski | Bug Report | pending problems and fixes from triumf | > > Here is the list of known problems I am aware of and of fixes not yet committed to midas svn:
> > 1) added variable /equiment/foo/common/PerVariableHistory
>
> corrected in svn revision 4203, read
> http://savannah.psi.ch/viewcvs/trunk/src/mlogger.c?root=midas&rev=4203&sortby=rev&view=log
Was still broken - all should work in revision 4207.
> > 2) writing compressed midas files (foo.mid.gz) crashes the mlogger when file
> > size reaches 2 GBytes. This problem could be new in SL5.1.
It turns out that on SL5 and SL5.1 (and others?) the 32-bit version of ZLIB opens the
compressed output file without the O_LARGEFILE flag, this limits the file size to 2 GB.
Fixed by opening the file ourselves, then attach compression stream using gzdopen().
Revision 4207. (Not tested on Windows - may be broken!)
> > 5) mhttpd history "export" button needs to be fixed (by request from ALPHA). At
> > present it either does not return all exiting data or crashes mhttpd. (no fix)
>
> (no change)
>
> > 6) mhttpd ODB editor in "set value" page, the "cancel" button is broken (needs
> > to be corrected for "relative URL").
>
> Apply this patch to src/mhttpd.c
>
> @@ -11156,10 +11190,7 @@
> sprintf(str, "SC/%s/%s", eq_name, group);
> redirect(str);
> } else {
> - strlcpy(str, path, sizeof(str));
> - if (strrchr(str, '/'))
> - strlcpy(str, strrchr(str, '/')+1, sizeof(str));
> - redirect(str);
> + redirect("./");
> }
>
> > 7) mhttpd needs AJAX-style methods for reading and writing ODB. (no fix)
>
> (no change)
>
> K.O. |
|
488
|
11 Jun 2008 |
Konstantin Olchanski | Suggestion | mlogger is flooding the message queue | > The current versions of mlogger SVN 4215 is flooding our message system with
> stuff like
>
> > Tue Jun 10 16:42:01 2008 [Logger,INFO] Configured history with 22 events
> > Tue Jun 10 16:42:14 2008 [Logger,INFO] Configured history with 22 events
> > Tue Jun 10 16:42:26 2008 [Logger,INFO] Configured history with 22 events
>
> This is fatal to us and blowing up the midas.log like hell. I would prefer if
> one could flag these kind of messages (ODB /Logger/..), i.e. enable and disable
> it. At the moment I have to comment it out in the source code since we cannot
> work with it.
I just sent the attached message to Stefan - please read it.
Before we take any action, we need to understand why history is being
reconfigured every 10 seconds (according to your logfile snippet).
Are you starting a new run every 10 seconds?
If that is what you do and that is your intent, I guess it is atypical usage of
MIDAS and the message from the mlogger is offensive and should be removed/disabled.
If something else is going on, we need to understand it before we sweep trouble
under the carpet by disabling this message.
K.O.
Stefan - there is more bad news - the message is produced when the history
is being reconfigured. This only is supposed to happen when the mlogger
starts or at the begin of run.
So these messages are just a tip of an iceberg of some other trouble.
The logic of when history is reconfigured I did not change. So likely
the trouble existed before, but you did not know about it.
We can kill the message, but why is the history being reconfigured
at a rate that "floods the log file"? That cannot possibly be good.
K.O. |
|
490
|
16 Jun 2008 |
Konstantin Olchanski | Suggestion | mlogger is flooding the message queue | > The current versions of mlogger SVN 4215 is flooding our message system with
> stuff like
>
> > Tue Jun 10 16:42:01 2008 [Logger,INFO] Configured history with 22 events
> > Tue Jun 10 16:42:14 2008 [Logger,INFO] Configured history with 22 events
> > Tue Jun 10 16:42:26 2008 [Logger,INFO] Configured history with 22 events
Problem confirmed on the M11 DAQ system at TRIUMF. We definitely do nothing funny
there, so what is going on? Will investigate.
K.O. |
|
497
|
28 Aug 2008 |
Konstantin Olchanski | Info | triumf/t2k midas updates | Following changes to midas produced from the TRIUMF T2K project have been
committed to svn:
1) cm_shutdown() will now SIGKILL clients that cannot be stopped via normal
means. Previously cm_shutdown() would print a message to the effect "please kill
this client yourself manually". The user action in this case (assuming they did
not issue cm_shutdown() by mistake) has been to find out the client pid using
"ps", kill -KILL it, then "odbedit clean". cm_shutdown() now performs all this
automatically.
2) rpc_send_event() did not correctly detect loss of connection to the remote
mserver (i.e. in case it was killed by cm_shutdown() above). Now, correct error
handling is in place and the remote frontend should gracefully shutdown if
mserver connection is lost. (However I observe that some of my remote frontends
fail to exit unless I do "exit(1);" from my frontend_exit() function.
3) mhttpd bug fixed: when editing odb entries, the "cancel" button did not work
correctly.
4) lazylogger "script" backup type is now fully tested and documented. Example
scripts for writing to dcache are available by request.
5) mlogger and mhttpd changes for writing history data to an sql database are
mostly completed and will be committed after some more debugging. (If you are
interested in details, please contact me directly).
6) (committed some time ago) Makefile changes for cross-compiling midas are now
in: "make linux32", "make linux64", "make crosscompile".
K.O. |
|