Author: michael+wmbugs
Description:
Forward Secrecy capable ciphers are not currently available on wikipedia.org. The only ciphers available on wikipedia.org are:
^ Original assertion no long true, but this ticket is now tracking progress on all of our many other sites, aside from the primary wikis.
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| Change ru.wikinews.org to HTTPS only. | operations/puppet | production | +4 -0 |
For the potential impact on HTTPS clients Chris Steipp told me on IRC he looked into what I assume is EventLogging data and later told me that Oliver had done some analysis work on that. I wanted to ask Oliver if he could publish his queries (or SQL and R code or whatever he used), but haven't yet done so (feel free to do that). The idea was also to compare before and after deployment. It would be interesting if we could publish an aggregated and anonymized analysis of the before and after comparison.
(In reply to Jan Zerebecki from comment #11)
It would be interesting if we could publish an aggregated
and anonymized analysis of the before and after comparison.
You know about:
don't you?
None of those on gdash differentiate between HTTP and HTTPS. I do not have full graphite access, so the ability to create something that might help may exist.
wikitech.wikimedia.org also doesn't support Forward Secrecy.
More importantly, SSL Labs says Wikitech server is "vulnerable to the OpenSSL CCS vulnerability (CVE-2014-0224) and exploitable".
(In reply to chmarkine from comment #15)
wikitech.wikimedia.org also doesn't support Forward Secrecy.
More importantly, SSL Labs says Wikitech server is "vulnerable to the
OpenSSL CCS vulnerability (CVE-2014-0224) and exploitable".
F to A- now
Yes and there are more sites that still lack forward secrecy. Now that there is an acceptable configuration with FS we can just apply that one to them. Some like wikitech and gerrit can probably use one that is less backwards compatible (like no SSL3, disable RC4, difficult: disable non-fs ciphers).
I agree with Jan. I think disabling SSL3 and non-fs ciphers is feasible, because only IE 6-8 on XP do not support any FS ciphers, only IE 6 does not support TLS 1.0 or higher, and even IE 7 on Vista supports ECDHE.
Also ticket.wikimedia.org does not support PFS. So all together:
https://www.ssllabs.com/ssltest/analyze.html?d=ticket.wikimedia.org
I just find more and more sites with no FS:
Again, graphite.wikimedia.org, gdash.wikimedia.org and dumps.wikimedia.org are "vulnerable to the OpenSSL CCS vulnerability (CVE-2014-0224) and exploitable".
lists.wikimedia.org is "vulnerable to the OpenSSL CCS vulnerability (CVE-2014-0224), but probably not exploitable", and lists.wikimedia.org does not support TLS 1.1 and TLS 1.2.
[1] https://www.ssllabs.com/ssltest/analyze.html?d=graphite.wikimedia.org (F)
[2] https://www.ssllabs.com/ssltest/analyze.html?d=gdash.wikimedia.org (F)
[3] https://www.ssllabs.com/ssltest/analyze.html?d=dumps.wikimedia.org (F)
[4] https://www.ssllabs.com/ssltest/analyze.html?d=lists.wikimedia.org (B)
meanwhile dumps and lists have been fixed it seems
dumps.wikimedia.org
Experimental: This server is not vulnerable to the OpenSSL CCS vulnerability (CVE-2014-0224).
lists.wikimedia.org
Experimental: This server is not vulnerable to the OpenSSL CCS vulnerability (CVE-2014-0224).
It's a bit unpractical to have one comment for each domain. Jan and chmarkine, it would be IMHO more useful if you resurrected https://wikitech.wikimedia.org/wiki/Httpsless_domains to make a table of which domains have https but lack PFS.
Change 144731 had a related patch set uploaded by Dzahn:
update SSL cipher list for gerrit to support PFS
Change 144734 had a related patch set uploaded by Dzahn:
update SSL cipher list for OTRS to support PFS
Change 144736 had a related patch set uploaded by Dzahn:
update SSL cipher list on wikitech to support PFS
all services behind the misc. varnish cluster should be fixed now. they were lacking an nginx restart on cp1043/cp1044, which i did now
this should have fixed all these:
doc
git
gdash
graphite
parsoid-tests
performance
integration
releases
legalpad
logstash
scholarships
(In reply to Nemo from comment #22)
It's a bit unpractical to have one comment for each domain. Jan and
chmarkine, it would be IMHO more useful if you resurrected
https://wikitech.wikimedia.org/wiki/Httpsless_domains to make a table of
which domains have https but lack PFS.
I made such a list: https://wikitech.wikimedia.org/wiki/User:Chmarkine/HTTPS
It summarizes support status for Forward Secrecy and HSTS. It also shows protocol versions, whether HTTP redirects to HTTPS, links to SSL Labs and SSL Labs grades.
It is an incomplete list. Please feel free to update it or move it to main namespace, if you want!
chmarkine: very nice list, thanks!
I just wanted to add that even though i have those (partly pending) patches to enable it on gerrit,wikitech,otrs ..it will not actually work before Apache is also a 2.4 version. But do you agree i should merge already anyways,based on it being an improvement anyways? Then it would just automatically be supported as soon as Apache will be upgraded.
(In reply to Daniel Zahn from comment #30)
chmarkine: very nice list, thanks!
I just wanted to add that even though i have those (partly pending) patches
to enable it on gerrit,wikitech,otrs ..it will not actually work before
Apache is also a 2.4 version. But do you agree i should merge already
anyways,based on it being an improvement anyways? Then it would just
automatically be supported as soon as Apache will be upgraded.
I agree! I think we should definitely merge them.
Change 146510 had a related patch set uploaded by Chmarkine:
update SSL ciphers for contacts.wm.org to support PFS
Change 147110 had a related patch set uploaded by Chmarkine:
update SSL ciphers for Ganglia to support PFS
Change 147123 had a related patch set uploaded by Chmarkine:
update SSL ciphers for noc.wikimedia.org to support PFS
Why does ganglia still get a B from Qualys SSL Labs after the change, while others are fine?
Change 147123 merged by Dzahn:
update SSL ciphers for noc.wikimedia.org to support PFS
It is B for ganglia because that old of an libssl and apache do not support newer TLS versions. ganglia / nickel.wikimedia.org is still on Ubuntu Lucid.
Change 147185 had a related patch set uploaded by JanZerebecki:
racktables - update SSL cipher list
Change 147196 had a related patch set uploaded by JanZerebecki:
smokeping - update SSL cipher list
Change 147199 had a related patch set uploaded by JanZerebecki:
etherpad - update SSL cipher list
Change 147207 had a related patch set uploaded by JanZerebecki:
icinga - update SSL cipher list
Change 147208 had a related patch set uploaded by JanZerebecki:
generic_vhost (webserver) - update SSL ciphers
Change 147214 had a related patch set uploaded by JanZerebecki:
metrics - update SSL cipher list
Change 147715 had a related patch set uploaded by Chmarkine:
rt -- update cipher suite list to support PFS
Change 147739 had a related patch set uploaded by Chmarkine:
blog -- update cipher suite list to support PFS
Change 147740 had a related patch set uploaded by Chmarkine:
ishmael -- update cipher suite list to support PFS
Change 147739 abandoned by Chmarkine:
blog -- update cipher suite list to support PFS
Change 148618 had a related patch set uploaded by Chmarkine:
tendril -- update cipher suite list to support PFS
Change 148624 had a related patch set uploaded by Chmarkine:
planet -- update cipher suite list to support PFS
Change 148631 had a related patch set uploaded by Chmarkine:
svn -- update cipher suite list to support PFS
Change 149267 had a related patch set uploaded by Chmarkine:
icinga-admin -- update cipher suite list to support PFS
Change 149267 merged by Dzahn:
icinga-admin -- update cipher suite list to support PFS
I just found that https://payments.wikimedia.org is still using the old cipher suite list:
TLS_RSA_WITH_AES_128_GCM_SHA256
TLS_RSA_WITH_RC4_128_SHA
TLS_RSA_WITH_RC4_128_MD5
TLS_RSA_WITH_3DES_EDE_CBC_SHA
TLS_RSA_WITH_AES_128_CBC_SHA
TLS_RSA_WITH_AES_256_CBC_SHA
https://www.ssllabs.com/ssltest/analyze.html?d=payments.wikimedia.org
Change 178676 had a related patch set uploaded (by JanZerebecki):
Change ru.wikinews.org to HTTPS only.
Change 178676 abandoned by JanZerebecki:
Change ru.wikinews.org to HTTPS only.
Reason:
Was done elsewhere.
All Gerrit patchsets linked in this ticket are merged or abandoned.
What's left to do here?
A few servers still do not provide forward secrecy because they need to be upgraded to apache 2.4 to support that. See: https://wikitech.wikimedia.org/wiki/HTTPS/domains
Also we still offer non-FS ciphers, but maybe disabling non-forward-secrecy should be a different task.
Guys, while it is true that apache 2.2 doesn’t support ECHD, it supports DHE just fine. And DHE IS a PFS. I know that it is a bit slower, but does it really matter for these web-servers?
We are currently using this:
'compat' => 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-E CDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:AES128- GCM-SHA256:AES256-GCM-SHA384:AES128:AES256:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!DH',
whether it's Apache 2.2 or 2.4
T83471 - seems like a duplicate of this but was resolved and not visibally public, fixed that
T91504 - OTRS SSL config (this is ticket.wikimedia.org in the linked list above)
T86655 - re: SVN (will probably be rejected in favor of decom)
T82698 - blocker for lists.wm
RT - will probably be rejected in favor of decom? needs ticket though?
T94585 - wikitech-static, needs distro upgrade, ticket created
that leaves from the list: gerrit? librenms? icinga?
In this case server load is the least of the concerns. Enabling non-EC DHE ciphers may be acceptable. It is necessary if we want to disable no-FS ciphers, which is desirable. Work on this in general is not blocked by the upgrade.
But there is a big downside to enabling non-EC DHE ciphers on older apaches: they only support something like 1k bit DH parameters. Which means one would decrease key size. I suggest to not enable non-EC DHE on older apaches, but to upgrade them.
Agreed on all points. Now that the primary production sites are up to speed, we're paying a lot more attention to these issues there as well, and trying to set ever-evolving standards that the other/misc sites can use to follow the primary sites' lead. The ciphersuite lists were recently updated and refactored a few times and there are now 3 options: strong, compat, and compat-dhe. You can read the details in the source file itself: https://github.com/wikimedia/operations-puppet/blob/production/modules/wmflib/lib/puppet/parser/functions/ssl_ciphersuite.rb . In the medium term, we're planning to switch to compat-dhe for the primary prod sites and then slowly remove most (eventually, all) of the non-PFS options from compat-dhe, and then deprecate the use of compat entirely, even for misc sites. There's a lot of auditing and checking that has to happen along the way (and notification of breakage for small minority client platforms!), but we'll eventually get there.
The TL;DR as it applies to random other/misc apache/nginx instances is:
There are more. The strong version doesn't allow TLS 1.0, so IE < 11, Android < 4.4, Java < 8, OpenSSL < 1.0, Safari (OS X) < 7 are inaccessible.[1]
Good point, I wasn't thinking about the -TLSv1.0 part when I wrote that last night (or the other big ticket description). Perhaps we need an intermediate option that's slightly less strong than strong, and just more about ECDHE ciphersuites only.
IE can handle TSL1.2 with IE8+, just not by default-config (you can enable it in the security-config). While that is of course no way for the ordinary users of Wikipedia, in controlled places like the OTRS it would be no problem to tell the few agents that uses such an old IE to enable TSL1.2 in my eyes.
Apparently back on May 28, Ubuntu released an update to their apache2.2 package (2.2.22-1ubuntu1.9) which backported ECC/ECDH/ECDHE support from upstream. Just by upgrading this package (which we should do anyways, as it's a sec update), we'll gain PFS for modern clients with a lot of these misc services. I'll see if I can get the update applied everywhere applicable.
Done, which should get us ECDHE-based Forward Secrecy on any remaining precise+apache2.2 hosts. Tested a few in ssllabs and they look fine. sodium (lists.wikimedia.org) should be the only apache left that doesn't do FS now.
I think this task can finally be closed as resolved, as there're no more domains that lack FS. (T91504 is now about DNSSEC.)