Closed Bug 1429639 Opened 6 years ago Closed 6 years ago

DigiCert: BR 3.2.5 Validation of Authority Failure for OV Certs

Categories

(CA Program :: CA Certificate Compliance, task)

Product:
CA Program
Component:
CA Certificate Compliance
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: wthayer, Assigned: jeremy.rowley)

Details

(Whiteboard: [ca-compliance] [ov-misissuance])

Attachments

(1 file)

List of affected certificates
168.25 KB, application/vnd.ms-excel
Details 
Tim Hollebeek from DigiCert posted the following incident report to the mozilla.dev.security.policy forum:

There was a bug in our OEM integration that led to a lapse in the
verification of authenticity of some OV certificate requests coming in
through the reseller/partner system.

As you know, BR 3.2.5 requires CAs to verify the authenticity of a request
for an OV certificate through a Reliable Method of Communication (RMOC).
Email can be a RMOC, but in these cases, the email address was a constructed
email address as in BR 3.2.2.4.4.  Despite the fact that these addresses are
standardized in RFC 2142 or elsewhere, we do not believe this meets the
standard of "verified using a source other than the Applicant
Representative."

The issue was discovered by TBS Internet on Dec 30, 2018. Apologies for the
delay in reporting this. Because of the holidays, it took longer than we
wanted to collect the data we needed.  We patched the system to prevent
continued use of constructed emails for authenticity verification early, but
getting the number of impacted orgs took a bit more time. We are using the
lessons learned to implement changes that will benefit overall user security
as we migrate the legacy Symantec practices and systems to DigiCert.

Here's the incident report:

1.    How your CA first became aware of the problem (e.g. via a problem
report submitted to your Problem Reporting Mechanism, via a discussion in
mozilla.dev.security.policy, or via a Bugzilla bug), and the date.

Email from JP at TBS about the issue on Dec 30, 2017.

2.    A timeline of the actions your CA took in response.

A. Dec 30, 2017 - Received report that indirect accounts did not require a
third-party source for authenticity checks. Constructed emails bled from the
domain verification approval list to the authenticity approval list.
B. Dec 30, 2017 - Investigation began. Shut off email verification of
authenticity.
C. Jan 3, 2017 - Call with JP to investigate what he was seeing and
confirmed that all indirect accounts were potentially impacted.
D. Jan 3, 2017 - Fixed issue where constructed emails were showing as a
permitted address for authenticity verification.
E. Jan 5, 2017 - Invalidated all indirect order's authenticity checks.
Started calling on verified numbers to confirm authenticity for impacted
accounts.
F. Jan 6, 2017 - Narrowed scope to only identify customers impacted (where
the validation staff used a constructed email rather than a verified
number).
G. Jan 10, 2017 - This disclosure.

Ongoing:
H. Reverification of all impacted accounts
I. Training of verification staff on permitted authenticity verification

3.    Confirmation that your CA has stopped issuing TLS/SSL certificates
with the problem.

Confirmed. Email verification of authenticity remains disabled until we can
ensure additional safeguards.

4.    A summary of the problematic certificates. For each problem: number of
certs, and the date the first and last certs with that problem were issued.

There are 3,437 orgs impacted, with a total of 5,067 certificates.  The
certificates were issued between December 1st and December 30th.

5.    The complete certificate data for the problematic certificates. The
recommended way to provide this is to ensure each certificate is logged to
CT and then list the fingerprints or crt.sh IDs, either in the report or as
an attached spreadsheet, with one list per distinct problem.

Will add to CT once we grab it all.  I will provide a list of affected
certificates in a separate email (it's big, so it was getting this post
moderated).

6.    Explanation about how and why the mistakes were made or bugs
introduced, and how they avoided detection until now.

In truth, it comes down to a short timeframe to implement the
Symantec-DigiCert system integration and properly train everyone we hired.
We are implementing lessons learned to correct this and improve security
overall as we migrate legacy Symantec practices and systems to DigiCert. In
this case, there are mitigating controls.  For example, these are mostly
existing Symantec certs that are migrating to the DigiCert backend. The
verification by Symantec previously means that the number of potentially
problematic certs is pretty low. There's also a mitigating factor that we
did not use method 1 to confirm domain control. In each case, someone from
the approved constructed emails had to sign off on the certificate before
issuance.  This is limited to OV certificates, meaning EV certificates were
not impacted. Despite the mitigating factors, we believe this is a
compliance issue, even though we believe the security risk is minimal.

7.    List of steps your CA is taking to resolve the situation and ensure
such issuance will not be repeated in the future, accompanied with a
timeline of when your CA expects to accomplish these things.

A. We clarified in the system what is required for an authenticity check.
B. We removed email verification for authenticity checks until appropriate
new safeguards can be added.
C. We are re-validating authenticity for all potentially problematic
certificates and will revoke any that fail to validate (none have failed so
far)
D. We improved training for all validation specialists on the rules for
authenticity checking.
E. We are undergoing quarterly audits on the OEM system to ensure all checks
are in place (per the root transfer requirements from Google).
Assigning to Jeremy (couldn't find a Bugzilla account for Tim). Action items are:
- publish list of affected certificates
- complete re-validation and report results
Assignee: kwilson → jeremy.rowley
Flags: needinfo?(jeremy.rowley)
Whiteboard: [ca-compliance]
I was the reporter of this issue. I obtained a mis-validated certificate to demonstrate the vulnerability.

The weakness permitted to issue OV certificates with a domain I controlled associated with an organisation I was not affiliated with.

The POC certificate is:

subject= /C=FR/L=BAYEUX/O=MA SOLUTION/CN=www.tbsx509.org
issuer= /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=Thawte RSA CA 2018
-----BEGIN CERTIFICATE-----
MIIE2DCCA8CgAwIBAgIQCUXWg4dxIqaS4ioKdnW6ETANBgkqhkiG9w0BAQsFADBc
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3
d3cuZGlnaWNlcnQuY29tMRswGQYDVQQDExJUaGF3dGUgUlNBIENBIDIwMTgwHhcN
MTcxMjI5MDAwMDAwWhcNMTgxMjI5MTIwMDAwWjBOMQswCQYDVQQGEwJGUjEPMA0G
A1UEBxMGQkFZRVVYMRQwEgYDVQQKEwtNQSBTT0xVVElPTjEYMBYGA1UEAxMPd3d3
LnRic3g1MDkub3JnMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAuwXW
xEbS7Ss3phc8RWTswWtfA1aq3fMBATUbMGF8J2zSGPK3CNdgeWP+G/se4KHkCkSb
jzGZtRKM/eDWEV2y3zaZ1k6Ko77HTq6i8ZZnlNPSmBOISDrm396VOf7GuNC4XnkZ
btRq2vw7X9h4vJzIhO+5BQZxtXF7mwYVze+X8v+58rRpKTp9jVUuv+K/LphUNwqv
a7PQsEo906Mozf3zBSLCcex30MND8vARLLTJItpeii33qGEn1hB5X5wpz2Agp++X
I87llA3Gy+7zYbLLq/UsAu7aCXJ3xK8x3zZzWExkn2jwGy/yJbvx+D8jOA3BpA1L
02ntf9J6TJQYmdbjawIDAQABo4IBojCCAZ4wHwYDVR0jBBgwFoAUo8heZVTlMHjB
BeoHCmpZzLn+3lowHQYDVR0OBBYEFNflTHut0ljgd58IvaHuMuDRfzSFMCcGA1Ud
EQQgMB6CD3d3dy50YnN4NTA5Lm9yZ4ILdGJzeDUwOS5vcmcwDgYDVR0PAQH/BAQD
AgWgMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjA6BgNVHR8EMzAxMC+g
LaArhilodHRwOi8vY2RwLnRoYXd0ZS5jb20vVGhhd3RlUlNBQ0EyMDE4LmNybDBM
BgNVHSAERTBDMDcGCWCGSAGG/WwBATAqMCgGCCsGAQUFBwIBFhxodHRwczovL3d3
dy5kaWdpY2VydC5jb20vQ1BTMAgGBmeBDAECAjBvBggrBgEFBQcBAQRjMGEwJAYI
KwYBBQUHMAGGGGh0dHA6Ly9zdGF0dXMudGhhd3RlLmNvbTA5BggrBgEFBQcwAoYt
aHR0cDovL2NhY2VydHMudGhhd3RlLmNvbS9UaGF3dGVSU0FDQTIwMTguY3J0MAkG
A1UdEwQCMAAwDQYJKoZIhvcNAQELBQADggEBABWzTFc2+Eg4Pe5WjExbmMXSm5Dv
u8M6X2AVgLkBl0A5lI6M3RSGALrsWuYwXqAcsHJ2ExgFVI9kZwUKA/rwDp8AubK2
lIHH9Jk8obeX2i9cUkNZsLPi1du4Dn1JR+ecrMcaIDMSpVcZe60WkCIhhUa+/FQT
+pDgojmXIUoQIU6LrNcELGSSpScMBimKPPLsrcmb2ij5CC8IXiZkarhOXf2+SY0m
xmgzaCaV+TLTYUq5GVPrVa95tS1lUcLNEDOn5kXbSqv1SA2Lwgj236GzXNGSe7R1
8JhoV/FFnnLTGiRBgDCoGXis23F2fMX7Mgo57XtCtBvnbdHvPjX04y0wY1g=
-----END CERTIFICATE-----
Revalidation is ongoing. I'm working with validation to get a timeline. 

On JP's comment, note that in each case the domain controller did approve the certificate, albeit indirectly (through one of the give constructed email addresses). For the order in question, JP controlled one of the five constructed email addresses. The domain verification process notified and received approval from the constructed email contact to issue the requested cert for the requested org. Unfortunately, there wasn't the required tie between the account and org which meant any compromised email address could receive the same OV certificate.
Flags: needinfo?(jeremy.rowley)

(In reply to Jeremy Rowley from comment #4)
> Revalidation is ongoing. I'm working with validation to get a timeline. 

Jeremy: It's been over a month since this was discovered. Please provide an update on the revalidation of these certificates, along with a timeline for completion and revocation of any that can't be validated.

> On JP's comment, note that in each case the domain controller did approve
> the certificate, albeit indirectly (through one of the give constructed
> email addresses). For the order in question, JP controlled one of the five
> constructed email addresses. The domain verification process notified and
> received approval from the constructed email contact to issue the requested
> cert for the requested org. Unfortunately, there wasn't the required tie
> between the account and org which meant any compromised email address could
> receive the same OV certificate.

My understanding of this incident is that it doesn't require a 'compromised email address'. Please explain?
Flags: needinfo?(jeremy.rowley)
Whiteboard: [ca-compliance] → [ca-incident] - Need remediation status
We're currently at about 50% complete on re-validation. So far, the only certificate revoked is the one noted by JP. We're planning to revoke anyone not verified by Mid-March. 

It doesn't require a compromised email because the authenticity check wasn't working. The email holder still had to approve the request though. Summary:

1) Company A requests cert for Company B that includes a domain name controlled by Company A
2) Company B is verified by DigiCert in accordance with BRs
3) Company A accepts domain verification through a constructed email address (which is easy since Company A controls and operates the domain) e
4) Certificate issues with Company B listed as the subject with Company A's domain name.
Flags: needinfo?(jeremy.rowley)
Changing QA contact per https://bugzilla.mozilla.org/show_bug.cgi?id=1438254
QA Contact: gerv → wthayer
Jeremy: will you please also post updates to the m.d.s.p. thread as I requested?

https://groups.google.com/forum/#!topic/mozilla.dev.security.policy/hAjDAxLqrF0
Please provide an update on remediation here and on the m.d.s.p. thread.
Flags: needinfo?(jeremy.rowley)
okay - can I have until June 2? At that point, I'll have a list of revoked certs and have them all logged.
Flags: needinfo?(jeremy.rowley)
Whiteboard: [ca-incident] - Need remediation status → [ca-incident] - Next update 2-June 2018
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Product: NSS → CA Program
Whiteboard: [ca-incident] - Next update 2-June 2018 → [ca-compliance] [ov-misissuance]
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: