1574594 - Amazon Trust Services: Revoked Sample Certs - No SANs

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Trevoli (Amazon Trust Services)]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0

Steps to reproduce:

1. How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date.

July 30, 2019 ~4:45-5:00pm PST - Certificates are issued for the purpose of allowing Application Software Suppliers to test revoked certs (“Sample Certificates”). After each certificate was issued it was immediately revoked.

August 06, 2019 11:03 AM PST - During our regular certificate review we discovered that 3 of the Sample Certificates were missing their SANs.

August 06, 2019 11:03 AM PST - The root cause is determined as a transcription error when creating the first subject info containing template.

2. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.

July 30, 2019 ~4:45-5:00pm PST - Sample Certificates were issued then immediately revoked as part of the standard process
August 12, 2019 3:46 PM PST - Transcription issue is fixed, peer reviewed, and committed to the template
August 16, 2019 11:56 AM PST - Certificate issuance checklist is moved into a tool that tracks and enforces each step

3. Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation.

We have stopped issuing certs missing SANs.

4. A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued.

All issued and revoked July 30, 2019.
sca0a - X509v3 Subject Alternative Name is absent
sca2a - X509v3 Subject Alternative Name is absent
sca4a - X509v3 Subject Alternative Name is absent

5. The complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem.

sca0a - https://crt.sh/?id=1754186728&opt=cablint,x509lint
sca2a - https://crt.sh/?id=1754186776&opt=cablint,x509lint
sca4a - https://crt.sh/?id=1754186755&opt=cablint,x509lint

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

As mentioned in bug https://bugzilla.mozilla.org/show_bug.cgi?id=1569266 we previously had no source controlled templating of subject information for our Application Software Supplier sample certificates. For our January 2019 certificate issuance we created a draft of a template for the “revoked” certificates and used to it to issue these certificates. After we reviewed the certificates later https://bugzilla.mozilla.org/show_bug.cgi?id=1525710 and discovered an error with them we corrected the templated subject information. However, we unfortunately also incorrectly transcribed 3 of the 5 and as result we did not include the SANs in the template before committing the template to source control.

During our certificate template review on July 24, 2019 we reviewed the template for these certificates, saw the SANs were correctly included for the first cert, spot checked another (third cert) and passed this template. This would have been caught during testing however, due to resourcing we switched engineers midway through the pre-ceremony prep and there was not a clean handoff which resulted in a missed testing step.

7. List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.

In addition to correcting the template we’ve now moved our certificate issuance checklist into a tool that enforces peer review. As well as tracking and enforcement of steps that need to be performed. This would have mitigated the missed step that occurred during the handoff of work. It would have been clear which testing steps had been performed and which had not.

Further, all test output will be verified by two people not directly involved with the ceremony to validate that the proper results were achieved by the ceremony. This two person control will be applied to all future ceremonies.

Comment 1

[Trevoli (Amazon Trust Services)]

We haven't been updating every 7 days because we have no pending actions, but we wanted to check in and see if there are any questions or feedback. Thanks.

Comment 2

[Ryan Sleevi]

Thanks Trev. I'm not sure if Wayne meant to clear the NI, but resetting it :)

Comment 3

[Wayne Thayer]

It appears that all questions have been answered and remediation is complete.