1593776 - Sectigo: invalid subject:organizationalUnitName on DV certificates

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Matthias]

Attachment: cert_by_issuer_name_no_dcv.txt

Many certificates signed by Sectigo's (intermediate) roots append Sectigo-related brands in the OU fields, and with that adding subject information which is not validated to the certificates.

Example: https://crt.sh/?id=1609606775 - The company Cofano (who is the owner of cofanostack.com) does not have an organizational unit named "PositiveSSL" (PositiveSSL being a brand name of Sectigo, and by lack of other subject information implying Sectigo is the owner or controller of the domain, which is incorrect).

Also note that BR 7.1.4.2.2i disallows any use of "name, DBA, tradename, trademark, address, location, or other text that refers to a specific natural person or Legal Entity" in the OU field unless "subject:organizationName, , subject:givenName, subject:surname, subject:localityName, and subject:countryName" are also set and validated by the CA (this is partly an oversight in the BR according to the writer, but currently this is still an issue)

see also https://groups.google.com/forum/#!topic/mozilla.dev.security.policy/haYidcAYq8I

The attached file contains a query on the crt.sh certificate database with which I researched this issue, with the resulting table of OU values used. Note that the results are not only for Sectigo, and contain somewhat historical data.

Comment 1

[Wayne Thayer]

Robin: please acknowledge this bug and provide an incident report.

Comment 2

[Robin Alden]

I acknowledge the report.
We are aware that we have issued many end entity DV certificates with OU fields containing values that we regard as static text that does not identify the subject of the certificate.

I've got three different aspects of 7.1.4.2.2.i to offer.

1. Our prior understanding of 7.1.4.2.2.i

This is as we implemented it 7+ years ago.
We originally read it as saying that where information pertaining to the subject organization was included in an OU field:
a) That information must be validated per 3.2.2.1, and
b) It must only be included in an OV or EV certificate, i.e. it could not be included in DV.

I think that interpretation is reasonable if (and only if) the phrase "refers to a specific natural person or Legal Entity" means the same as "refers to the subject". That is how we read it, but I can now see it may not have been the intended reading.

2. The intent of 7.1.4.2.2.i

There are a couple of things going on here.
a) It probably intends to bolster the CA Warranty (9.6.1 4) that the CA (i) implemented a procedure for reducing the likelihood that the information contained in the Certificate's subject:organizationalUnitName attribute would be misleading.
We see that the avoidance of misleading information in the OU attribute is important and we believe we have honoured that intent.
b) It intends to prevent the inclusion of subject-related information in the OU attribute of a DV certificate, i.e. it is not OK to have a DV certificate with {subject:OU=Example Corp. Shipping Department, subjectAltName:DNSNAME=example.com}. We do not include subject-related information in the OU attribute of a DV certificate.

3. What 7.1.4.2.2.i really says

Looking at it now, it seems to say that you can only have meaningful, validated, OU information in a certificate which is either OV or EV, and which also includes subject:givenName and subject:surname attributes. There seems to be a very limited constituency of certificates to which this could apply. Although I haven't checked at the time of writing, I suspect that we have issued no certificates at all that meet those criteria.

It implies that it is OK to include in any certificate (DV, OV, EV) an OU attribute that DOES NOT include a name, DBA, tradename, trademark, address, location, or other text that refers to a specific natural person or Legal Entity.
"OU=Domain Control Validated" is therefore compliant to appear in any subject (DV, OV, EV) , unless some wit or miscreant were to register a trade mark or a corporation called "Domain Control Validated".

To be compliant with the broken language that is there now, I think we may need to change our policy to suppress all OU attributes, even those supplied by the applicant.
We will continue to plan and evaluate our response, but my current thinking is that we will not be undertaking a mass revocation in response to this issue as to do so would be a wholly disproportionate response.

I would like us to also propose a ballot to restate 7.1.4.2.2.i in a form that is clear, allows some sensible use of the OU attribute, and meets the consensus objectives of the forum and, of course, of Mozilla.

I will follow up with an incident report in the standard format.

Comment 3

[Ryan Sleevi]

We will continue to plan and evaluate our response, but my current thinking is that we will not be undertaking a mass revocation in response to this issue as to do so would be a wholly disproportionate response.

To remind Sectigo, "a wholly disproportionate response" is not a sufficient reason for violating the Baseline Requirements, per https://wiki.mozilla.org/CA/Responding_To_An_Incident#Revocation

Other CAs have worked to ensure timely revocation of certificates, even those that list improper CPS versions, in the spirit of keeping a healthy and compliant ecosystem, and ensuring that regardless of the reasons for revocation, the CA is wholly capable of abiding by the Baseline Requirements.

It implies that it is OK to include in any certificate (DV, OV, EV) an OU attribute that DOES NOT include a name, DBA, tradename, trademark, address, location, or other text that refers to a specific natural person or Legal Entity.

How is this consistent with the definition of "Subject Identity Information" in BRs 1.6.1, and the intersection of requirements from 3.2.5, 7.1.6.1, and 9.6.1 (5)?

Comment 4

[Matthias]

First, my opinion is that some information about the certificate can be inserted into the OU field, but that including a name, trademark, etc. in it's own OU field is not transparent for end users unless this is accompanied by the organization of that name, trademark etc. in the rest of the subject fields. If the end user with no knowledge of Sectigo's issued certificate structure looks for information in the certificate for whether they can trust the website they are on and see a probable third party trademark in the subject information, then that will confuse the user.

Second, I would like to notice that it has been 22 days since the last update; which makes me think timely incident responses (bug 1563579) are still a very current issue.

Comment 5

[Robin Alden]

Problem statement:

Sectigo have been inserting our brand or product names as well as those of certain of our sales partners and/or other descriptive text into the subject:OU fields of end entity certificates.
We regarded these OU values as static, i.e. not relating to the subject of the certificate.
These are Product Names, “Hosted by” statements, “Issued through” statements, “Provided by” statements, etc.
E.g. :
OU=<Product name>
OU=Hosted by <WebHost name>
OU=Provided by <Reseller name>
OU=Issued through <Customer name> E-PKI Manager
OU=Domain Control Validated

Section 7.1.4.2.2.i of the BRs attempts to describe what is permissible to be included in OU field values.

It has been brought to our attention in this bug and the referenced mdsp thread that the wider interpretation of the Baseline Requirements for the Issuance and Management of Publicly Trusted Certificates does not support this practice and therefore we have agreed that we will cease putting any text in an OU field which is not supplied by the certificate applicant.

1. How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date.

2019-11-01 We read a post on the mozilla.dev.security.policy mailing list that pointed out "that a lot of leaf certificates have organizationalUnitName specified without other organizational information such as organizationName. Many times this field is used for branding purposes, e.g. "issued through <someone's kpi manager>" or "SomeBrand SSL"."
https://groups.google.com/d/msg/mozilla.dev.security.policy/haYidcAYq8I/ezl4dUxqDAAJ

2. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.

(Times/dates in UTC)
2012 The BR language now in 7.1.4.2.2.i around OUs (apart from the givenName and surname) was proposed.
At an intervening date the requirement for givenName and surname were added to the OU language.
2019-11-01 We read the above-referenced post on m.d.s.p. and responded on that thread.
2019-11-04 Matthias opened this bug.
2019-11-13 We made our initial response on this bug.
2019-11-18 We created a development ticket for our CA to discontinue the use of OU fields other than for OU values provided by the applicant and pertaining to the subject.
2019-12-02 We prepared customer messaging around the change of our certificate profile.
2019-12-05 We sent email notifications to customers messaging around the change of our certificate profile.
2019-12-15 00:00 (Scheduled) We will release our code modification to discontinue the use of OU fields other than for OU values provided by the applicant and pertaining to the subject.

3. Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation.

We have not yet stopped adding these additional OU fields. We will stop doing so on 2019-12-15 00:00 (UTC).

4. A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued.

There are many.
The first such certificates would have been issued in 2002.
The last such certificates will be issued on 2019-12-14.
I will provide a count of the affected unexpired certificates in a followup.

5. The complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem.

Many may be found
E.g.
https://crt.sh/?OU=Hosted+by%25
https://crt.sh/?OU=PositiveSSL+Multi-Domain
https://crt.sh/?OU=COMODO+SSL+Unified+Communications

I will provide a list of the affected unexpired certificates in a followup.

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

We originally read the BR language as saying that where information pertaining to the subject organization was included in an OU field:
a) That information must be validated per 3.2.2.1, and
b) It must only be included in an OV or EV certificate, i.e. it could not be included in DV.
That interpretation was reasonable if (and only if) the phrase "refers to a specific natural person or Legal Entity" means the same as "refers to the subject". That is how we read it, but I can now see it was not how others read it and can accept that our reading was not the intended reading.

7. List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.

We have planned and coded for a change to our end entity certificate profiles to remove the extra OU fields.
We have communicated that change to our affected customer base.
The profile change will take effect 2019-12-15 00:00.

We do not propose to revoke the affected end entity certificates at this time.
These certificates will be left to expire.
The latest expiry of any affected certificate will be 2022-02-16.

In this case we believe that the harm caused by following the requirements of BR section 4.9.1 (Circumstances for Revocation) outweighs the risks that are passed on to individuals who rely on the web PKI.
While we appreciate Mozilla's guidance in https://wiki.mozilla.org/CA/Responding_To_An_Incident that <<Responses similar to “we deem this misissuance not to be a security risk” are not acceptable.>>, we do not believe that any security risk is presented by this non-compliance and we consider that there will be a large amount of effort required if we were to try and have the subscribers replace all of the affected certificates and that thousands of websites would likely be taken down because those maintaining them will not respond to the customer communication we send out concerning the affected certificates.

We think that the language of 7.1.4.2.2.i of the BRs is regrettably unclear in its wording and as written arguably forbids the presence of any OU field value at all in any issued certificate, contrary to what we now believe to be the intent of that section.
We will propose a ballot in the CA/B forum to restate 7.1.4.2.2.i in a form of words whose meaning is clear and which allows some sensible use of the OU attribute.

This incident and other high visibility incidents in 2019 have illustrated how a single, codified level of response for all certificates containing BR violations can lead to outcomes that are suboptimal for both relying parties and subscribers. Sectigo plans to drive dialog around establishing categories of BR violation, with appropriate codified responses for each. Our ultimate goal will be to advance a ballot to put these categories into effect.

Comment 6

[Robin Alden]

We implemented the planned certificate profile changes to remove the additional OU fields as planned and the profile change took effect
2019-12-15 00:00.

5. A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued.

The number of Sectigo-issued unexpired DV SSL certificates that included OU fields as at 2019-12-15 00:00 was 11,170,043.

4. The complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem.

A list of 11,170,043 crt.sh IDs is in a zipped CSV available here.

Comment 7

[Ryan Sleevi]

Robin: Per the policy on Revocation, please open a new incident regarding Sectigo's decision not to revoke these certificates, and please make sure to address the information required in such reports.

Comment 8

[Ben Wilson]

Setting June 15, 2020 for another review of this bug.

Comment 9

[Ben Wilson]

Bug 1620561 was opened for revocation delay related to this bug. I'm going to set this Next Update to July 15, and I'm considering closing this as covered by Bug #1620561.

Comment 10

[Ryan Sleevi]

We will propose a ballot in the CA/B forum to restate 7.1.4.2.2.i in a form of words whose meaning is clear and which allows some sensible use of the OU attribute.

There has been zero progress on this in 6 months. I appreciate some discussion happened in Bratislava, but I'm concerned that nothing concrete or tangible has turned up. One of the discussions was to, on a matter of first principles, forbid the OU entirely.

Rob, can you clarify whether anyone at Sectigo is still working on this, and if so, what a concrete timeline to proposing draft text here is? If the answer is that Bratislava showed it's "more complex", then perhaps a position paper from Sectigo discussing the trade-offs to show tangible discussion going forward?

Comment 11

[Nick France]

Ryan: Yes, it is still being worked on. Rich will post an update shortly on the discussions internally and with other CAs regarding the OU attribute, with a view to a future ballot proposal.

Comment 12

[Rich Smith]

We've been working behind the scenes with some of the other CAs to develop a CA/B Forum ballot to amend the BR, both to fix issues that already exist, as well as to clarify and define what is actually expected in terms of verification of the OU field. To directly address this bug, we stopped the practice of including any OU fields in DV certificates and thereafter for a number of reasons, the process of addressing the BR issues stalled both w/in Sectigo and in the other CAs with whom we've been discussing the issue. I've drafted a skeleton ballot proposal and sent it out to a couple of other CAs for review and feedback. I'll be out of the office next week, but plan to get something posted to the CA/B Forum validation working group by the end of the following week. At minimum, if we do nothing else, the BR needs clarifying because as Robin indicated in his initial response to this, the BR arguably currently does not allow the OU field to be used in ANY certificate, or at least no certificate that any CA is, to the best of my knowledge, actually issuing.

Comment 13

[Rob Stradling]

We have no further update.

Comment 14

[Rich Smith]

I've posted an initial draft to the CA/B Forum Server Certificate Validation sub-list which you can see here.

As this draft does not propose to allow OU field in DV certificates, we don't think the ultimate outcome of the CA/B Forum ballot has any further bearing on this bug, so I don't plan to post any additional updates regarding that subject here. Those interested can follow the commentary on the CA/B Forum listserv.

Comment 15

[Rich Smith]

As direct remediation of this bug Sectigo has ceased all issuance of DV certificates with OU fields populated. Separately, we have now proposed a ballot to fix the issues in the BR which partially led to our actions causing this bug. We will continue to engage in the process in the CA/B Forum, but feel that there are no further actions to be taken which directly relate to this bug.

Comment 16

[Rich Smith]

No further updates

Comment 17

[mpalmer]

I'd like to note that this section of the incident report doesn't appear to have been completely answered:

3. Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation.

We have not yet stopped adding these additional OU fields. We will stop doing so on 2019-12-15 00:00 (UTC).

Specifically, "a statement that you have not requires an explanation". I can't see any explanation of why it took over a month for issuance of these problematic certificates to cease in this bug report.

Comment 18

[Rich Smith]

In response to mpalmer comment 17:

Stopping including these OU fields required a code change to our CA system. It took time to implement, to test, to get approval for deployment, and to deploy this code change. The timeline described in the ticket contains a full explanation of actions and considerations taken by Sectigo to discontinue populating the OU field in a responsible manner for all stakeholders on 2019-12-15 00:00 (UTC).

Comment 19

[Rich Smith]

We have no further updates. We ceased issuing DV certificates with OU fields on 2019-12-15 as previously mentioned. We have submitted a proposal in the CA/B Forum to address the problems which exist with the OU field and we will continue to engage in that process. Ben, we feel that remediation of this bug is complete and would ask, if there are no further questions or comments, that this bug be closed.

Comment 20

[Ben Wilson]

I believe this bug can be closed, but I will schedule that to occur on or about 17-Sept-2020 in case there are further comments or concerns.

Comment 21

[Rob Stradling]

We have no further update.