ID |
Date |
Author |
Topic |
Subject |
1111
|
11 Sep 2015 |
Konstantin Olchanski | Info | mhttpd/SSL error message on MacOS | > On my macbook (OS X 10.10.3) I get this error message when starting mhttpd with mongoose-SSL:
>
> [mhttpd,ERROR] [mhttpd.cxx:17092:mongoose,ERROR] mongoose web server error: set_ssl_option:
> openssl "modern cryptography" ECDH ciphers not available
>
It means what it says - "modern cryptography" is not available (in google-chrome terms), different browsers report this
differently, same (apple safari) do not seem to care.
In practice if ECDH ciphers are not available, the https connection uses "obsolete cryptography" and (depending) it
probably not actually secure (might even be using RC4 ciphers).
The reason you get this error is the obsolete OpenSSL library shipped with MacOS (all version). (same on SL4 and SL5).
Reasonably up-to-date OpenSSL library that has ECDH support can be installed using MacPorts, this step should be
added to the MIDAS documentation.
>
> mhttpd seems to start fine anyway and safari connects to the secure midas page without complaining
> about the SSL (it complains about the certificate of course). So maybe this error message is
> relatively harmless?
>
Some browsers do not care about the quality of the connection - google-chrome seems to be the most conservative
and flags anything that is less than "most state of the art encryption".
Some browsers seem to be happy even if the connection is SSLv2 with RC4 encryption, even though it is not secure at
all by current thinking.
Is that harmless? (browser says "secure" when it is not?)
> I don't get this error message with Scientific Linux 6.7.
el6 has a reasonably recent OpenSSL library which supports "modern cryptography".
The best guide to this is to run the SSLlabs scanner and read through it's report.
K.O.
P.S. All this said, I hope my rationale to switching away from OpenSSL makes a bit more sense. If we use something
like PolarSSL, at least we get the same behaviour on all OSes.
K.O. |
1110
|
11 Sep 2015 |
Konstantin Olchanski | Info | mhttpd HTTPS/SSL server updated | >
> Thanks for the detailed explanation. I agree with your recommendations. I was mostly interested in having both options treated equally in the documentation.
>
I did not review the documentation yet, so it is most likely completely wrong.
But in the nutshell, we should document 2 configurations:
1) standalone mhttpd - with built-in https and password protection
2) mhttpd behind a password-protected https proxy (apache httpd) - mhttpd will have https and built-in passwords disabled, http access restricted to localhost (or the host of the httpd, if they are not the same - as at CERN/ALPHA).
K.O. |
1109
|
09 Sep 2015 |
Thomas Lindner | Info | Documentation regarding specifying custom pages | Hi,
We have recently been changing the code in mhttpd that maps custom web pages and resources to
particular files on the server file system. Ie, changing the code that uses the ODB keys in /Custom to
map a web address like
http://myhost:8081/CS/MyCustomPage
to some file like
/home/user/resource/mypage.html
This mapping gets complicated when you use the /Custom/Path key to specify a location for web
resources like images. We have tried to summarize how the current system works on the wiki
https://midas.triumf.ca/MidasWiki/index.php//Custom_ODB_tree
Please provide any suggestions on how either the documentation or the actual algorithm can be
improved. |
1108
|
09 Sep 2015 |
Thomas Lindner | Info | mhttpd/SSL error message on MacOS | On my macbook (OS X 10.10.3) I get this error message when starting mhttpd with mongoose-SSL:
[mhttpd,ERROR] [mhttpd.cxx:17092:mongoose,ERROR] mongoose web server error: set_ssl_option:
openssl "modern cryptography" ECDH ciphers not available
mhttpd seems to start fine anyway and safari connects to the secure midas page without complaining
about the SSL (it complains about the certificate of course). So maybe this error message is
relatively harmless?
I don't get this error message with Scientific Linux 6.7. |
1107
|
09 Sep 2015 |
Thomas Lindner | Info | mhttpd HTTPS/SSL server updated | > >
> > I find that I don't understand this recommendation to use secure mongoose
> > instead of putting mhttpd behind an apache proxy.
> >
>
> This is a very valid question.
>
> I think for a small operation that does not require root access to the host computer, mhttpd+mongoose is a good light weight solution.
> ...
> So, which one to use?
>
> - for maximum security, use httpd apache (but remember to restrict access to mhttpd web port to be "only from the proxy")
> - for light-weight cases, or when root access is not available use built-in https in mhttpd.
>
> The midas wiki documentation should probably be updated to explain all of this.
Thanks for the detailed explanation. I agree with your recommendations. I was mostly interested in having both options treated equally in the documentation.
My only small complaint is that since the default mhttpd comes with mongoose security turned on, you need to explicitly disable the mhttpd+mongoose security first before you can start setting up apache. I guess that the motivation is
that we should force people to disable security, rather than hoping that they will enable it. That's a convincing argument; so all I really need is that this procedure be well documented. |
1106
|
02 Sep 2015 |
Konstantin Olchanski | Info | mlogger history changes | The git branch feature/logger_db_watch is getting ready for merging into main midas.
The main change in the logger is the switch from db_open_record() to db_watch() as the
method of listening to ODB variables. The new db_watch() function makes it cheap (in the
number of hotlinks used) to implement "per-variable" history as the new default. In the new
code, the old "per-equipment" history is no longer available.
In other words:
old per-equipment history: 1 hotlink per equipment
old per-variable history: 1 hotlink per "ls" entry in /eq/xxx/variables (big experiments can
easily exceed the maximum number of hotlinks!)
new (per-variable) history: (back to) 1 hotlink per equipment
Notable changes from old history:
- works as described in my recent notes - the new code will complain about all incorrect use
of history - where the old code sometimes silently malfunctioned (i.e. symlinks in unexpected
places) or bombed (i.e. infinite loop reloading the history).
- all references to "PerVariableHistory" in ODB are removed (this is the new default)
- the "structured bank" records (subdirectories under variables, as in /eq/xxx/var/struct/value)
are now broken up into individual items. This change is forced by the difference between
db_open_record() and db_watch() for structured banks written using db_set_record(). The old
per-variable history kept these items together in one event.
This change is also inline with Stefan's suggestion that all compound items, including arrays,
should be broken up into separate history events. Keeping with this suggestion, right now
only arrays are not broken up - because of limitations in the history storage level. As history
storage is improved, arrays will also be broken up into individual elements.
The new code is functionally complete and all are welcome to try it (but beware as it may eat
your odb or your history storage - make a backup!).
git checkout feature/logger_db_watch
K.O. |
1105
|
01 Sep 2015 |
Konstantin Olchanski | Info | note on midas history | Sorting |
1104
|
31 Aug 2015 |
Konstantin Olchanski | Info | mhttpd HTTPS/SSL server updated | Configuration of web server completely changed (merge of branch feature/mongoose-config2). Hopefully for the last time.
mhttpd is now controlled by these ODB variables:
/experiment/
.../midas http port -> 8080 (0x1F90)
.../midas https port -> 8443 (0x20FB)
.../http redirect to https -> "y"
the names are self obvious (hopefully)
access control is done by the "-a" command line arguments
and by the access control list in ODB, which works the same way
as the RPC ports access control list. An empty list means free access
from everywhere:
/experiment/security/mhttpd hosts/allowed hosts
the access control list is watched by httpd, there is no need to restarted it after updating the list.
after changing the port number settings, mhttpd should be restarted.
other web control options to mhttpd are:
daq06:midas$ ./linux/bin/mhttpd -h
usage: ./linux/bin/mhttpd [-h Hostname[:port]] [-e Experiment] [-v] [-D] [-a Hostname]
-a only allow access for specific host(s), several [-a Hostname] statements might be given (default list is ODB "/Experiment/security/mhttpd hosts/allowed hosts")
--http port - bind to specified HTTP port (default is ODB "/Experiment/midas http port")
--https port - bind to specified HTTP port (default is ODB "/Experiment/midas https port")
--nomg use the old mhttpd web server
--oldserver [port] - use the old web server on given port
--nooldserver - do not use the old mhttpd web server
To run mhttpd "the old way" (mhttpd -D -p 8080), say "mhttpd --oldserver 8080 --nomg".
The normal way to run mhttpd is: "mhttpd -v" to get debug information and "mhttpd -D" to run in the background.
K.O. |
1103
|
27 Aug 2015 |
Konstantin Olchanski | Info | mhttpd HTTPS/SSL server updated | >
> I find that I don't understand this recommendation to use secure mongoose
> instead of putting mhttpd behind an apache proxy.
>
This is a very valid question.
I think for a small operation that does not require root access to the host computer, mhttpd+mongoose is a good light weight solution.
For a more elaborate setup with private networks, etc, apache https proxy is probably better - for big experiments, resources like webcams,
ganglia, couchdb, etc also need password protection and apache https is the one stone to kill all birds. (one bird to kill all pigs).
Which one is easier to setup?
mhttpd+mongoose I tried to make simple - you have to create a password file and (optional) a properly signed https certificate.
apache httpd is fairly straightforward if you follow well written instructions (such as we provide for using it with midas). but you do need root access
and you do have to edit a good number of config files.
Which one is secure?
By one definition - will it pass muster with central IT - only apache httpd is secure.
At CERN all we have to say "we use password protected apache httpd HTTPS proxy" and they say "ok!".
If we were to say "we use custom web server based on some strange version of mongoose, customized", they will probably raise some eyebrows.
And keeping central IT happy is important if you want holes in their firewall for off-site access to MIDAS.
Now, which one is secure?
The default distribution of apache httpd in SL6 is insecure. period. some steps to secure it are non-controversial - disable SSLv2, SSLv3, disable RC4 ciphers. This is not enough to pass
muster with the SSLlabs scanner. One should also disable some obsolete and known-weak ciphers. This will disable some old web browsers and is a more controversial step. (see the SSLlabs
reports).
The default distribution of mhttpd+mongoose passes muster with SSLlabs and on the strength of that I deem it "secure out of the box". One can suggest alternative security tools and one
can/should run the SSLlabs scanner against mhttpd after each update, report problems as a bug to midas.
Now, which one is *secure*?
Both apache httpd and mongoose are based on OpenSSL which has been recently demonstrated to be severely insecure. (look into the OpenBSD fork of OpenSSL).
There are alternative HTTPS libraries, such as PolarSSL, which are intended for embedding into other applications and devices - such as into mhttpd or into MSCB-ethernet boxes -
and I hope mongoose/fossa will make the switch by the end of the year. (a compatibility layer for using mongoose with PolarSSL already exists).
So, which one to use?
- for maximum security, use httpd apache (but remember to restrict access to mhttpd web port to be "only from the proxy")
- for light-weight cases, or when root access is not available use built-in https in mhttpd.
The midas wiki documentation should probably be updated to explain all of this.
K.O. |
1102
|
27 Aug 2015 |
Konstantin Olchanski | Info | mhttpd HTTPS/SSL server updated | Stefan identified a serious multi-thread locking bug in mhttpd that affects the operation of the sequencer (a race condition between db_set_record() and
db_get_record() inside the hotlink code). This is now fixed. If you use the sequencer, please update mhttpd.cxx to the latest (or to this) version.
https://bitbucket.org/tmidas/midas/commits/9d79218a125a4427d0cc2f2b5e4e56d585655c88
K.O. |
1101
|
24 Aug 2015 |
Konstantin Olchanski | Info | note on midas history | >
> * |
1100
|
21 Aug 2015 |
Thomas Lindner | Info | mhttpd HTTPS/SSL server updated | >
> I recommend that you use "mhttpd --mg" as the alternative for running "mhttpd -p" behind an apache
> proxy. Using "mhttpd -p" (no HTTPS/SSL) on an internet-connected machine is insecure and should not be
> done. (private network such as 192.168.x.y addresses is okey for now, I guess).
Finally reading through your documentation in detail [1,2]. I find that I don't understand this recommendation to use secure mongoose
instead of putting mhttpd behind an apache proxy. I think that it is nice to have secure mhttpd with mongoose as an option, but your
documentation seems to imply that mhttpd-mongoose is much better than mhttpd-behind-apache and that the latter solution is strongly
deprecated.
Perhaps I am not understanding the benefits of the new system. In reference [2] you say "If this is not possible, somewhat better security
for HTTP is gained by using a password protected SSL (https) proxy." This seems to imply that the security of mhttpd-mongoose is better
than the security of mhttpd-behind-apache. Is that correct? I thought that they provided similar security (assuming you follow
recommended configurations for APACHE).
Setting up apache is trivial and it seems that mhttpd-behind-apache has other advantages, like being able to put other web resources
(ganglia, cameras, elog, etc) behind the same secure server. Also you can start to build complicated custom pages that are served directly
from apache and just use MIDAS AJAX calls. I was imagining slowly moving away from using mhttpd at all and just having html/js/css
resources served up by apache.
So, unless I'm missing something, at this point I would continue to recommend people use mhttpd-behind-apache and I'd suggest this be
presented as an equally valid option in the documentation.
[1] https://midas.triumf.ca/MidasWiki/index.php/Mhttpd
[2] https://midas.triumf.ca/MidasWiki/index.php/Setup_MIDAS_experiment#Install_SSL_proxy |
1099
|
21 Aug 2015 |
Stefan Ritt | Bug Report | MIDAS message page auto-size (horizontally) is annoying | > New version of MIDAS has a feature where it seems to automatically resize the
> message page horizontally in order to fix each MIDAS message into one line.
> Some of my MIDAS messages (in particular error messages, where I need details)
> are very long. The result is that the MIDAS page automatically becomes very
> wide and I have to scroll a lot left/right in order to read my messages. This
> is annoying.
>
> I would vote to roll-back this new feature.
Issues should be reported in the bitbucket issue tracker. I moved this issue over there.
/Stefan |
1098
|
20 Aug 2015 |
Thomas Lindner | Bug Report | MIDAS message page auto-size (horizontally) is annoying | New version of MIDAS has a feature where it seems to automatically resize the
message page horizontally in order to fix each MIDAS message into one line.
Some of my MIDAS messages (in particular error messages, where I need details)
are very long. The result is that the MIDAS page automatically becomes very
wide and I have to scroll a lot left/right in order to read my messages. This
is annoying.
I would vote to roll-back this new feature. |
1097
|
20 Aug 2015 |
Stefan Ritt | Bug Report | Sequencer limits | > > - ODBGet (and ODBSet?) does seem to be able to take a variable as a path... I
> > was trying to use an array whose index would be incremented.
>
> To be checked.
It does not take a variable as a path, but as an index. So you can do
LOOP i, 5
WAIT seconds, 3
ODBSET /System/Tmp/Test[$i], $i
ENDLOOP
And you will get
/System/Tmp/Test
[1] 1
[2] 2
[3] 3
[4] 4
[5] 5
/Stefan |
1096
|
19 Aug 2015 |
Konstantin Olchanski | Bug Report | Sequencer limits | >
> See LOOP doc
> LOOP cnt, 10
> ODBGET /foo/bflag, bb
> IF $bb==1 THEN
> SET cnt, 10
> ELSE
> ...
>
Looks like we have PE |
1095
|
19 Aug 2015 |
Pierre-Andre Amaudruz | Bug Report | Sequencer limits | These issues have been addressed by Stefan during his visit at Triumf last month.
The latest git has those fixes.
> While I know some of those limits/problems have been already been reported from
> DEAP (and maybe corrected in the last version), I am recording them here:
>
> Bugs (not working as it should):
> - "SCRIPT" does not seem to take the parameters into account
Fixed
> - The operators for WAIT are incorrectly set:
> the default ">=" and ">" are correct, but "<=", "<", "==" and "!=" are all using
> ">=" for the test.
Fixed
>
> Possible improvements:
> - in LOOP, how can I get the index of the LOOP? I used an extra variable that I
> increment, but it there a better way?
See LOOP doc
LOOP cnt, 10
ODBGET /foo/bflag, bb
IF $bb==1 THEN
SET cnt, 10
ELSE
...
> - PARAM is giving "string" (or a bool) whose size is set by the user input. The
> side effect is that if I am making a loop starting at "1", the incrementation
> will loop at "9" -> "1". If I start at "01", the incrementation will give "2.",
> "3.",... "9.", "10"... The later is probably what most people would use.
Fixed
> - ODBGet (and ODBSet?) does seem to be able to take a variable as a path... I
> was trying to use an array whose index would be incremented.
To be checked. |
1094
|
19 Aug 2015 |
Pierre Gorel | Bug Report | Sequencer limits | While I know some of those limits/problems have been already been reported from
DEAP (and maybe corrected in the last version), I am recording them here:
Bugs (not working as it should):
- "SCRIPT" does not seem to take the parameters into account
- The operators for WAIT are incorrectly set:
the default ">=" and ">" are correct, but "<=", "<", "==" and "!=" are all using
">=" for the test.
Possible improvements:
- in LOOP, how can I get the index of the LOOP? I used an extra variable that I
increment, but it there a better way?
- PARAM is giving "string" (or a bool) whose size is set by the user input. The
side effect is that if I am making a loop starting at "1", the incrementation
will loop at "9" -> "1". If I start at "01", the incrementation will give "2.",
"3.",... "9.", "10"... The later is probably what most people would use.
- ODBGet (and ODBSet?) does seem to be able to take a variable as a path... I
was trying to use an array whose index would be incremented. |
1093
|
14 Aug 2015 |
Konstantin Olchanski | Info | Merged - improved midas network security | > [local:Online:S]RPC hosts>set "Allowed hosts[1]" "host.psi.ch"
> [ODBEdit,INFO] Reloading RPC hosts access control list via hotlink callback
Yes, this debug message can be removed after (say) one or two weeks. I find it useful at the moment.
> The trick not to have to "guess" a remote name is quite useful. I'm happy it even made it into the documentation.
Yes, this came from CERN/ALPHA felabview where we did not have a full list of labview machines that were sending us labview data.
> There is however an
> important shortcoming in the documentation. The old documentation had a "quick start" section, which allowed people quickly to
> set-up and run a midas system. This is still missing. And it should now contain a link to the "Security" page, so that people can set-up
> quickly remote programs.
Yes, Suzannah was just asking me about the same - we will need to review the quick start guide and the documentation about the mserver and midas rpc.
K.O. |
1092
|
14 Aug 2015 |
Stefan Ritt | Info | Merged - improved midas network security | I tested the new scheme and am quite happy with. Just a minor thing. When I change the ACL, I get messages from all attached programs, like:
[local:Online:S]RPC hosts>set "Allowed hosts[1]" "host.psi.ch"
[ODBEdit,INFO] Reloading RPC hosts access control list via hotlink callback
09:05:11 [mserver,INFO] Reloading RPC hosts access control list via hotlink callback
09:05:11 [ODBEdit,INFO] Reloading RPC hosts access control list via hotlink callback
[ODBEdit,INFO] Reloading RPC hosts access control list via hotlink callback
[ODBEdit,INFO] Reloading RPC hosts access control list via hotlink callback
09:05:11 [mserver,INFO] Reloading RPC hosts access control list via hotlink callback
09:05:11 [ODBEdit,INFO] Reloading RPC hosts access control list via hotlink callback
09:05:11 [mserver,INFO] Reloading RPC hosts access control list via hotlink callback
09:05:11 [ODBEdit,INFO] Reloading RPC hosts access control list via hotlink callback
While this is good for debugging, I would remove it not to confuse the average user.
The trick not to have to "guess" a remote name is quite useful. I'm happy it even made it into the documentation. There is however an
important shortcoming in the documentation. The old documentation had a "quick start" section, which allowed people quickly to
set-up and run a midas system. This is still missing. And it should now contain a link to the "Security" page, so that people can set-up
quickly remote programs. |
|