Closed Bug 1294150 Opened 9 years ago Closed 9 years ago

Move the CA certificates list out of NSS into a separate repository

Categories

(NSS :: Libraries, defect)

Product:
NSS
Component:
Libraries
Version:
3.26
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED INCOMPLETE

People

(Reporter: KaiE, Assigned: KaiE)

References

Details

Attachments

(5 files)

part1-nssckbi.h.patch
1.09 KB, patch
Details | Diff | Splinter Review
part3-certdata.txt.patch
1.17 KB, patch
Details | Diff | Splinter Review
part4-empty-certdata.txt.patch
3.95 KB, patch
Details | Diff | Splinter Review
part5-makefile-manifest.patch
1.77 KB, patch
Details | Diff | Splinter Review
combined-steps-1-to-5.patch
11.66 KB, patch
Details | Diff | Splinter Review
I suggest to move the CA certificates list out of NSS into a separate repository. It will allow consumers to use arbitrary combinations of the NSS library and the CA certificates list, without having to understand the internals of NSS. In the past, in order to pick up code fixes for security reasons, we occassionally had to update the NSS code in a stable Firefox branch to a newer version of NSS. However, for various reasons, we didn't want to update the CAs on that branch to a newer snapshot. As a workaround, we had to release additional NSS branch releases that contained the code fixes, but kept the CA list stable. On other occassions, we wanted to backport a CA update to an older branch of Firefox, because it contained new blacklisted CAs, but didn't want to update the NSS code. By separating NSS from the CA list, we will be able to declare release versions and release tags individually, and consumers, such as Firefox branches, can only update the part they need. Another scenario are CA removals, which need to be postponed to a future date. Currently, those CA removals need to be landed early into a Firefox development branch, because changes to Firefox during the beta testing phase are discouraged. However, releasing them early as part of an NSS release, has the undesirable consequence, that consumers other than Firefox, who might be interested to run the most recent NSS code as soon as possible, can get the undesirable side effect of removing CAs too early, while they are still needed. The split will give us the ability to handle those situations more flexibly.
See Also: → 1290999
Depends on: 1294016
I would like to register my strong opposition to this. "Without having to understand the internals" of NSS should be a non-goal, as explained at https://wiki.mozilla.org/CA:FAQ#Can_I_use_Mozilla.27s_set_of_CA_certificates.3F Further, the CA database is tightly coupled with implementation details of NSS. Implementation of constraints on particular CAs, for example, the whitelisting of some certificates (e.g. CNNIC), or other policies that may apply or be applied in the future. The CA database is tightly coupled to NSS, in implementation and behaviour, and separating it out sends the wrong message to both NSS developers and potential consumers. While I'm sensitive to and appreciate the overhead that is involved with an NSS release, the issues it purports to solve will still exist in another form (e.g. the application of constraints), while the damage will be done.
On Bug 1294016, you position justification as the very thing that we SHOULD NOT do - which is mix versions of NSS and the CA certificate database. There was significant discussion when setting up https://wiki.mozilla.org/CA:FAQ#Can_I_use_Mozilla.27s_set_of_CA_certificates.3F about the risks that such decisions can pose. Judging from the empirical evidence of how distros are packaging NSS as well, I'm convinced this will make users less secure on Linux in particular, and have detrimental effects on the ecosystem as a whole.
As of today, in order to build NSS, it's already required to retrieve source code from two separate repositories: - https://hg.mozilla.org/projects/nspr - https://hg.mozilla.org/projects/nss The build environment assumes the following directory layout: main-directory |- nspr |- nss When built in the simple configuration, files in directory nss will access files from ../nspr I'd like to suggest we add a new repo: - https://hg.mozilla.org/projects/ca-certificates which I've already requested in bug 1294016, and suggest that we assume anyone building nss needs the following directory layout: main-directory |- nspr |- ca-certificates |- nss and nss will directly access ../ca-certificates
Right, I'm aware of what your suggestion was, I'm stating that I believe it's bad for users and security to separate out the ca-certificates from NSS. Though I appreciate your comments in Comment #3 to suggest they'll still be a part of NSS, it's very clear from your Comment #1 and your Bug 1294016 that your larger justification is two anti-patterns: 1) Making it easier for non-NSS consumers to consume the database, when the decisions and policies are intrinsically tied to the behaviour of NSS/Firefox 2) Making it easier for NSS consumers to mix/match the database version, when we really want to ensure that the version being deployed by users is the current/secure set of roots, and we constantly see distros not taking ca-certificate updates unless they're part of an NSS security fix release. The trade-off we're making here is whether it's better to cost the NSS maintainers - of which, I readily admit, you bear the brunt of this with relation to root cert updates every quarter - time in figuring out the exact sequence of landing, versus the ecosystem costs for both NSS and non-NSS consumers. From my experiences the past year with a variety of open-source and Google projects, I'm absolutely convinced that this proposal will make it more error prone and dangerous for consumers, and as such, we should bear the cost.
Several distributions already package the ca-certificates list separately. In order to simplify consumption of NSPR and NSS for those that don't, we already provide combined archives on the NSS source code distribution site, where we group an NSS release with the recommended NSPR release. I suggest that we expand that. In the future, instead of shipping nss-with-nspr, I suggest we ship combined nss-with-ca-and-nspr archives, that bundles the version of CA list, which we recommended for major consumption at the time of the release of the respective NSS release.
Kai, I'm not sure why we're talking past each other, but it doesn't seem you're replying to my concerns, or perhaps you've misunderstood. I'm aware that distros package the ca-certificates list separately. That's BAD. That's the anti-pattern. That's the thing that should not (generally) be done - https://wiki.mozilla.org/CA:FAQ#Can_I_use_Mozilla.27s_set_of_CA_certificates.3F The rest of your Comment #5 seems to be replying to a separate concern, which is not the set I raised, so I don't feel as if responding to it would be useful.
Correct, I haven't yet replied to Ryan's comments.
I'd like to proceed with attaching and describing a potential implementation, because I have already worked on it. Step 1: Remove version numbers from nssckbi.h and replace them with symbols that will be dynamically replaced at build time. See part1-nssckbi.h.patch Step 2: Rename nssckbi.h to nssckbi.template Step 3: Embed version number information into the certdata.txt file See part3-certdata.txt.patch Step 4: Because the intention is to move file certdata.txt to a separate repository, a developer might want to decide to build NSS without it, by explicitly defining symbol NSS_EMPTY_BUILTINS. (For example, Fedora and RHEL offers a replacement library for libnssckbi.so, implemented using p11-kit-trust, which uses a different source, provided by the ca-certificates package, and extendable by the local system administrator. The NSS libnssckbi.so library is still shipped, but it's automatically obsoleted using the update-alternatives system, so having the module provided by NSS to be empty could be fine.) To make that possible, a new file empty-certdata.txt is added to NSS. See part4-empty-certdata.txt.patch Step 5: If certdata.txt is available, we want to dynamically extract the version information, and combine it with nssckbi.template to create nssckbi.h This requires to change the build rules in Makefile and manifest.mn. Explanation for sed 's/^# *MOZILLA_CERTDATA_MAJOR_VERSION *\(.*\) *.*/s#CERTDATA_VERSION_MAJOR#\1#g/' The first part searches for the version numbers. The replacement text is a new sed command. For that new sed command, I use '#' as the separator character, to avoid having to escape /, which is hopefully easier to read. See part5-makefile-manifest.patch In addition to the individual patches, which are intended to simplify review, I'm attaching a combined patch, which can be applied to NSS immediately. It allows to test this new code. Later - Step 6 When we are ready to move certdata.txt to the new separate repository, we will adjust the published NSS build instructions, and change Makefile again, to access certdata.txt from an external parent directory (see the comment in the Makefile in part5). When we land an updated NSS snapshot into the Firefox repository, we will import a snapshot the the ca-certificates repo into the Firefox repository, too.
This patch combines all the individual steps I've described in my previous comment. It already contains preparations for step 6, which would replace NSS_CERTDATA_TXT = certdata.txt with NSS_CERTDATA_TXT = $(CORE_DEPTH)/../ca-certificates/certdata.txt in the Makefile.
Assignee: nobody → kaie
With the attached potential implementation, consumers who download NSS without any ca-certificates library, will fail to get a build. They will either need to get the ca-certificates and install them in the correct place for building, or they must define symbol NSS_EMPTY_BUILTINS. Note I've haven't yet tested the use of NSS_EMPTY_BUILTINS, that's a todo, should we decide to proceed with this approach.
Let's discuss this in the NSS call tomorrow. Thanks.
(In reply to Ryan Sleevi from comment #1) > Further, the CA database is tightly coupled with implementation details of > NSS. Implementation of constraints on particular CAs, for example, the > whitelisting of some certificates (e.g. CNNIC), or other policies that may > apply or be applied in the future. I cannot find any mention of the string CNNIC inside NSS, except in certdata.txt Are you referring to certverifier/CNNICHashWhitelist.inc ? That's outside of NSS, and in Firefox application code.
(In reply to Kai Engert (:kaie) from comment #16) > (In reply to Ryan Sleevi from comment #1) > > Further, the CA database is tightly coupled with implementation details of > > NSS. Implementation of constraints on particular CAs, for example, the > > whitelisting of some certificates (e.g. CNNIC), or other policies that may > > apply or be applied in the future. > > I cannot find any mention of the string CNNIC inside NSS, except in > certdata.txt > > Are you referring to certverifier/CNNICHashWhitelist.inc ? > That's outside of NSS, and in Firefox application code. I'm not entirely sure the point you're trying to make, but it may have been that mine was not as clear as it could have been, so I'm sorry for that. The point I was making is that the decision to trust some CAs - such as CNNIC or ANSSI - is entirely dependent upon what the application code supports, generally NSS, but specifically, Firefox. That is, one should not simply examine certdata.txt (as many distros and maintainers were and are doing) to determine if CA X is trustworthy, but look at how the applications (NSS and Firefox) actually behave. Which is the point: The decisions in nssckbi are based upon what the NSS code supports and, now that Mozilla has moved more out into mozilla::pkix/PSM, what Firefox supports. You should not separate the two, and if you do, you should be aware of that coupling. The proposal to split these out is, in part, intentionally motivated by a desire to remove the above statement. But that's not desirable. Cert Version Y is making implicit and explicit decisions based on what the (bundled) version of Firefox & NSS are going to do. So you can't just assume version Y will work with old-Firefox or old-NSS, or afford the same security. Similarly, Cert Version X (the older version) will generally be 'less secure' than Cert Version Y - there will be CAs that shouldn't still be in still trusted, or there won't be new CAs needed for transitions. So you wouldn't want to mix Cert Version X with new-NSS or new-Firefox. That is, your explicit stated goal - allowing ca-certificates (implicitly, nssckbi) - to be independently versioned from Firefox/NSS - is a thing specifically advised against because it creates security and interoperability issues. I realize Red Hat has taken a different perspective of this security stance (e.g. including unaudited 1024-bit certificates because you were using them with not-NSS, particularly OpenSSL, and they didn't support the audited 2048-bit paths), but I'm arguing that's damage that should be kept localized to Red Hat. That was also reflected in the previous decision - which was that Red Hat would add those 1024-bit certs back, rather than having upstream do it (as was requested). Further, while Red Hat may be adept and capable of ensuring a relative synchronicity in the versions (although historically we know this not to be true), many other bundlers are not, which is why the FAQ was added to begin with, to try to mitigate the harm. The proposal - to split out the CA certificates - is to add legitimacy to the perception that the certificates are independently maintainable, rather than a core and inseparable part of NSS (now, Firefox). These decisions affect not just the direct NSS developers, but affect how the project is perceived, used, and consumed, and we have ample evidence that going down the path suggested causes developers to make poor or suboptimal solutions.
(In reply to Ryan Sleevi from comment #17) > > I cannot find any mention of the string CNNIC inside NSS, except in > > certdata.txt > > > > Are you referring to certverifier/CNNICHashWhitelist.inc ? > > That's outside of NSS, and in Firefox application code. > > I'm not entirely sure the point you're trying to make My point is: You have earlier argued (I'm trying to summarize in my own words) that: - splitting NSS and the CA list is a bad idea - it's a bad idea, because the CA list logically belongs to constraints that have been implemented in code, and shouldn't be separated The above made sense, if as of today, both logic and CA were indeed tightly coupled to each other. But apparently they aren't. Apparently the Firefox application developers have already made the (IMHO unfortunate) decision to move required logic into application code, instead of adding it at the more general NSS library level. So, given that NSS+CA-Certificates are already being distributed independently of the related code in Firefox, it seems to me that the strictly bundled state, which you're trying to conserve, is already nonexistent? If the strict coupling is already nonexistent, I'm not convinced that conserving strict coupling can be used as an argument against the proposed split. The NSS team would continue to make recommendations which combinations of NSS library and CA list are recommended, by demonstrating which combinations are being used on Firefox release branches, and by making announcements on appropriate announcement channels, and by publishing combined source archives.
While Red Hat does not preserve the strict state, Mozilla does. So it seems you're arguing that because Red Hat breaks this bundling, it's OK for others to do so as well?
(In reply to Ryan Sleevi from comment #19) > While Red Hat does not preserve the strict state, Mozilla does. Mozilla does already publish the CA list plus the NSS code as separately downloadable software, that excludes the Firefox code, that you argue is required for handling the CA list correctly.
Resolving this as incomplete for now, as based on discussions, it's currently unclear if this will be done.
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → INCOMPLETE

Instead of parsing certdata.txt, we recommend that people use the links provided on the Common CA Database (CCADB) Resources tab, which provide lists of root certificates that are being curated for the purposes of Code Signing, Email (S/MIME), and Server Authentication (SSL/TLS).

Hi Kathleen,
While it's a good start, it's not a complete solution. CA-certificates needs more data than just the list of certs. It needs, at the very least, all the data that NSS expects or firefox on Linux platforms will fail. (most notable, distrust after date).

For now I'm probably going to keep certdata.txt and pull in codesigning and S/MIME to adjust my certdata.txt.

Of course, NSS should continue to directly use certdata.txt.

Folks other than NSS who would like to use this root store should read this:
https://blog.mozilla.org/security/2021/05/10/beware-of-applications-misusing-root-stores/

I think I should explain. On Linux, the cacertificate package replaces nssckbi to create a single certificate store. everything (well in theory) uses that single certificate store, including NSS, so whatever we use as a source for cacertficates, it needs to have all the data that NSS uses. So when I mean 'I'll probably keep certdata.txt and pull in codesigning and S/MIME' I mean the certdata.txt that I use to source the ca-certificates package, which will continue to massage that certdata.txt to produce all the lists that cacertificates produce.

bob

Having read the blog, I still need to keep to my plan. ca-certificates is maintained by NSS developers, since it's sourced from the NSS root store. We already know that some certs are in the cert store a disabled (since I created the distrust attribute for NSS). certdata.txt isn't just parsed, it's used to create a PKCS #11 module that mimics nssckbi.so. Some other libraries have switched to using it. This pkcs #11 module needs all of the NSS bits, so it needs to be built from certdata.txt. It also includes system included certs which may not be in certdata.txt. From this, the NSS bits are used to decide purpose, and appropriate lists are created for less intelligent applications (lists for s/mime, tls, and object signing, and specific lists for openssl, java, etc.). The idea is we have a centrally managed place for root certificates. Users/Sysadmins can then add locally trusted ca's or remove ones that they don't want to trust.

Since mozilla dropped code signing, we verified that no one was using code signing in RHEL 8, so we still shipeed codesigning list, but it was empty (previous to RHEL8, we kept the old mozilla code signing). It was news to me that Microsoft started using codesigning on Linux, which meant they were abusing the list. There is a codesigning list which was empty, so I guess they just decided to use a tls list (which is not for that purpose).

Anyway I'm going to pick up the code signing CA's and add them back to our source of certdata.txt with the appropriate codesigning bit set on (merging with any existing ca's already there). Once that happens, the code signing list will be populated, and Microsoft can use that list).

Upshot: what the blog post envisioned: randomly taking the list of CA's without a thought to what the NSS trust on those CA are is NOT what is happening (Actually I think that was true before 2013 when ca-certificates were created -- by kai-- but most Linux OS's use ca-certificates, not their own curated list. We still can't control applications misusing the list (using tls CAs for code signing is just... wrong)).

One think your combined website is missing: version numbers.

Bob, we are in agreement that the email and server auth root cert lists at https://www.ccadb.org/resources are not intended for you / Linux OS root store, because you should continue to use the NSS root store as you have been.

For others, note that we added the distrust-after information to the CSV lists for email and server auth. And we will be adding similar (Microsoft-specific) columns to the CSV list for the Code Signing roots (Bug #1711597), and we are planning to add automation to keep the CCADB in sync with Microsoft's internal Root Cert Management Tool (RCMT) as per Bug #1711598. (currently Microsoft is manually maintaining the data in both places)

As Bob mentioned, the lists of root certs at https://www.ccadb.org/resources do not have version numbers, which is why folks who use those lists of root certs should check for changes in the lists whenever they do their application updates.

OK, ca-certificates will continue to use certdata.txt, which the addition of the microsoft object siging certs added (If the cert is already in certdata.txt, then the object signing trust is turned on, if it's not it's added with only the object signing trust flag on). It's a little annoying that it's not versioned, which means the version of the microsoft object signing list is whatever happens to be there when I make my build, which is rather non-ideal (it means the same version of ca-certificates build by redhat may be different than the same version number build by debian, depending on what day each decided to build their update).

BTW, this bug is about getting a single usable source for ca-certificates, which is why I'm kind of pushing back here.

bob

For anyone using the following two links in https://www.ccadb.org/resources, we are making a one-time change to the URLs to change the parameter from TrustBitsInclude to MicrosoftEKUs.

Code Signing Root Certificates
PEM of Root Certificates in Microsoft’s Root Store with Code Signing EKU (TXT)
PEM of Root Certificates in Microsoft’s Root Store with Code Signing EKU (CSV)

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: