BetterTLS
A Name Constraints test suite for HTTPS clients, created by Netflix
Written by Ian Haken
At Netflix we run a microservices architecture that has hundreds of independent applications running throughout our ecosystem. One of our goals, in the interest of implementing security in depth, is to have end-to-end encrypted, authenticated communication between all of our services wherever possible, regardless of whether or not it travels over the public internet. Most of the time, this means using TLS, an industry standard implemented in dozens of languages. However, this means that every application in our environment needs a TLS certificate.
Bootstrapping the identity of our applications is a problem we have solved, but most of our applications are resolved using internal names or are directly referenced by their IP (which lives in a private IP space). Public Certificate Authorities (CAs) are specifically restricted from issuing certificates of this type (See section 7.1.4.2.1 of the CA/B baseline requirements), so it made sense to use an internal CA for this purpose. As we convert applications to use TLS (e.g., by using HTTPS instead of HTTP) it was reasonably straightforward to configure them to use a truststore which includes this internal CA. However, the question remained of what to do about users accessing their services using a browser. Our internal CA isn’t trusted by browsers out-of-the-box, so what should we do?
The most obvious answer is straightforward: “add the CA to browsers’ truststores.” But we were hesitant about this solution. By forcing our users to trust a private CA, they must take on faith that this CA is only used to mint certificates for internal services and is not being used to man-in-the-middle traffic to external services (such as banks, social media sites, etc). Even if our users do take on faith our good behavior, the impact of a compromise to our infrastructure becomes significant; not only could an attacker compromise our internal traffic channels, but all of our employees are suddenly at risk, even when they’re at home.
Fortunately, the often underutilized Name Constraints extension provides us a solution to both of these concerns.
The Name Constraints Extension
One powerful (but often neglected) feature of the TLS specification is the Name Constraints extension. This is an extension that can be put on CA certificates which whitelists and/or blacklists the domains and IPs for which that CA or any sub-CAs are allowed to create certificates for. For example, suppose you trust the Acme Corp Root CA, which delegates to various other sub-CAs that ultimately sign certificates for websites. They may have a certificate hierarchy that looks like this:
Now suppose that Beta Corp and Acme Corp become partners and need to start trusting each other’s services. Similar to Acme Corp, Beta Corp has a root CA that has signed certificates for all of its services. Therefore, services inside Acme Corp need to trust the Beta Corp root CA. Rather than update every service in Acme Corp to include the new root CA in its truststore, a simpler solution is for Acme Corp to cross-certify with Beta Corp so that the Beta Corp root CA has a certificate signed by the the Acme Root CA. For users inside Acme Corp their trust hierarchy now looks like this.
However, this has the undesirable side effect of exposing users inside of Acme Corp to the risk of a security incident inside Beta Corp. If a Beta Corp CA is misused or compromised, it could issue certificates for any domain, including those of Acme Corp.
This is where the Name Constraints extension can play a role. When Acme Corp signs the Beta Corp root CA certificate, it can include an extension in the certificate which declares that it should only be trusted to issue certificates under the “betacorp.com” domain. This way Acme Corp users would not trust mis-issued certificates for the “acmecorp.com” domain from CAs under the Beta Corp root CA.
This example demonstrates how Name Constraints can be useful in the context of CA cross-certification, but it also applies to our original problem of inserting an internal CA into browsers’ trust stores. By minting the root CA with Name Constraints, we can limit what websites could be verified using that trust root, even if the CA or any of its intermediaries were misused.
At least, that’s how Name Constraints should work.
The Trouble with Name Constraints
The Name Constraints extension lives on the certificate of a CA but can’t actually constrain what a bad actor does with that CA’s private key (much less control what a subordinate CA issues), so even with the extension present there is nothing to stop the bad actor from signing a certificate which violates the constraint. Therefore, it is up to the TLS client to verify that all constraints are satisfied whenever the client verifies a certificate chain.
This means that for the Name Constraints extension to be useful, HTTPS clients (and browsers in particular) must enforce the constraints properly.
Before relying on this solution to protect our users, we wanted to make sure browsers were really implementing Name Constraints verification and doing so correctly. The initial results were promising: each of the browsers we tested (Chrome, Firefox, Edge, and Safari) all gave verification exceptions when browsing to a site where a CA signed a certificate in violation of the constraints.
However, as we extended our test suite beyond basic tests we rapidly began to lose confidence. We created a battery of test certificates which moved the subject name between the certificate’s subject common name and Subject Alternate Name extension, which mixed the use of Name Constraint whitelisting and blacklisting, and which used both DNS names and IP names in the constraint. The result was that every browser (except for Firefox, which showed a 100% pass rate) and every HTTPS client (such as Java, Node.JS, and Python) allowed some sort of Name Constraint bypass.
Introducing BetterTLS
In order to raise awareness around the issues we discovered and encourage TLS implementers to correct them, and to allow them to include some of these tests in their own test suite, we are open sourcing the test suite we created and making it available online. Inspired by badssl.com, we created bettertls.com with the hope that the tests we add to this site can help improve the resiliency of TLS implementations.
Before we made bettertls.com public, we reached out to many of the affected vendors and are happy to say that we received a number of positive responses. We’d particularly like to thank Ryan Sleevi and Adam Langley from Google who were extremely responsive and immediately took actions to remediate some of the discovered issues and incorporate some of these test certificates into their own test suite. We have also received confirmation from Oracle that they will be addressing the results of this test suite in Java in an upcoming security release.
The source for bettertls.com is available on github, and we welcome suggestions, improvements, corrections, and additional tests!
