Closed Bug 1649962 Opened 5 years ago Closed 5 years ago

SECOM: Incorrect OCSP Delegated Responder Certificate

Categories

(CA Program :: CA Certificate Compliance, task)

Product:
CA Program
Component:
CA Certificate Compliance
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: ryan.sleevi, Assigned: h-kamo)

Details

(Whiteboard: [ca-compliance] [ca-misissuance])

The following was originally reported to m.d.s.p. at https://www.mail-archive.com/dev-security-policy@lists.mozilla.org/msg13493.html

SECOM has issued one or more OCSP Delegated Responders, as defined within RFC 6960, Section 2.6 and Section 4.2.2.2, without including the id-pkix-ocsp-nocheck response, as required by the Baseline Requirements, Version 1, Section 13.2.5 through Version 1.7.0, Section 4.9.9

Example certificate: https://crt.sh/?id=1671982906

Please provide an incident report, including the timeline for revocation.

Ryan-san,

Thank you for your notice.
We are now investigating this issue.

Best regards,
Hisashi Kamo

Ryan-san,

Let us provide you our incident report.

  1. How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date.

We first became aware of the problem by the email "[Bug 1649962] New: SECOM: Incorrect OCSP Delegated Responder Certificate" received at 10:47 (Japan time) on July 2, 2020.

  1. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.

We confirmed that OCSP Signing was included in Extended Key Usage in the 3 intermediate CA certificates on July 2.
Then our plan is to reissue the intermediate CA certificates that do not include OCSP Signing in Extended Key Usage, starting from July 8 and completing by July 10. We are not changing the key pair for reissuing the certificates because changing the key pair in a short period of time will cause greater problems than the security benefits it may obtained.

  1. Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation.

Our CA has not stopped the issuance of the certificates with the problem.
All CAs are under the control of SECOM Trust Systems and it is not possible for any third party other than SECOM Trust Systems to create fraudulent OCSP responses, so that we consider that there is not any additional security risk related with this issued.
Since Key Usage of the intermediate CA certificate does not include Digital Signature and Basic Constraints is CA: TRUE, so we consider that OCSP response is impossible.
Deciding from the situation mentioned above, we would like to solve the problem by reissuing the intermediate CA certificates and distributing them after issuing, without revoking the intermediate CA certificates in question.

  1. A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued.

Number of the intermediate CA certificates in total: 3

  1. The complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem.

Please refer the contents of 4 above.

  1. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

Starting from January 1, 2019, it has been required to describe Extended Key Usage that includes the issued certificate in intermediate CA certificate, in accordance with Mozilla Root Store Policy. That’s why our CA described the all Extended Key Usage included in the certificate issued by the intermediate CA.
We once had the incident when only TLS Web Server Authentication was described in Extended Key Usage of the intermediate CA certificate and OCSP Signing was not included, the path validation of the OCSP server certificate issued by the intermediate CA failed. In order to deal with that incident, OCSP Signing was added to the Extended Key Usage of the intermediate CA certificate.

  1. List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.

We’ll make sure issuing intermediate CA certificates without including OCSP Signing in Extended Key Usage. We’ll also revise manuals and check sheets for preventing any recurrence of the problem.

Thank you for your consideration.

Best regards,
Hisashi Kamo

so we consider that OCSP response is impossible.

As discussed and demonstrated, this is not the case. It is very much possible and a very real risk. I don’t see a plan to address that risk; if anything, this plan amplifies the risk, as others have pointed out. Such a plan would fundamentally undermine trust in the issuer.

Are you sure you’ve properly evaluated the risk?

Flags: needinfo?(h-kamo)

Ryan-san,

Thank you for your comment.

It was certainly not “Impossible”. Sorry about that.
In cablint, if Digital Signature is not included in Extended Key Usage of the intermediate CA certificate, the following message is displayed, so we thought it could be impossible:
“NOTICE CA certificates without Digital Signature do not allow direct signing of OCSP responses.”
However, it did not have a clear basis regarding with RFC.
On the other hand, we believe that the additional security risk related with this issue is very limited by the following reasons:

  1. All CAs that include OCSP Signing in the Extended Key Usage of the intermediate CA certificate are operated by SECOM Trust Systems. SECOM Trust Systems solely manages the equipment and the private key of the intermediate CA certificate, so it is difficult to use the intermediate CA key pair inappropriately.
    These intermediate CAs only sign Certificates which much configured profile that conforms to the predetermined BR. Bypassing all of these configurations and signing the OCSP response data is as (or more) difficult as issuing a forged arbitrary End-Entity certificate, and our company controls it to prevent this.
  2. In addition, suppose even if it would be possible to create an inappropriate OCSP response, in order to use the OCSP response, it is necessary to make a fraud in the network between the signature verifier and the CA. Our company controls the network of CA side, and in order for any third party to make an OCSP response of a fraudulent root CA, it is necessary to control the network on the side that verifies the signature.

If there were any adversary which have ability to perform above1 and 2, that adversary should able to perform much more efficient attack toward Verifier, so we consider that there is no additional and realistic risk.

Thank you for your consideration.

Best regards,
Hisashi Kamo

Flags: needinfo?(h-kamo)

Ryan-san,

Please let us update our status.

Then our plan is to reissue the intermediate CA certificates that do not include OCSP Signing in Extended Key Usage, starting from July 8 and completing by July 10.

We reissued the intermediate CA Certificates by July10.
The CCADB update for those certificates have been done.

Best regards,
Hisashi Kamo

Are you saying you did not delay revocation for these?

Bypassing all of these configurations and signing the OCSP response data is as (or more) difficult as issuing a forged arbitrary End-Entity certificate, and our company controls it to prevent this.

Unfortunately, I don’t think it’s reasonable to take this statement at face value, and I remain deeply concerned about the remediation. To date, this bug has provided zero detailed evidence about those controls and design, how nothing could have gone wrong in the past, can go wrong now, or may go wrong in the future.

I understand you to believe it’s the same risk. I don’t, because you have not demonstrated anything to support or establish this. A proper incident response, as seen from other CAs, minimally includes all of the information necessary for a general member of the public to understand how this is not possible, and how it can be independently evaluated as not possible.

You are approaching/defining risk as you see it, from your perspective. This is the wrong approach. You should work to define risk as relying parties see if. Imagine if you had a third-party sub-CA in possession of such a certificate, and who you were already concerned about whether they were operating correctly. Further, imagine if they misissued a single OCSP response, your CA would be immediately distrusted, and you would be prevented from ever reapplying to a root store. If those were the “stakes”, what would you expect from this hypothetical third-party CA? What would you demand, if the very viability of your business was at stake? That’s the risk that Relying Parties and Browsers see, and that’s why “oh, we don’t think it’s an issue” is acceptable, for the same reason you wouldn’t accept such an answer from any third-party CA you had delegated to.

This is why it’s necessary to rotate keys and ensure secure key destruction. Revocation and rotating EKUs, as discussed in the m.d.s.p. thread, is unacceptable as a “complete” mitigation.

Flags: needinfo?(h-kamo)

Ryan-san,

Thank you for the comments.

We have completed the reissuance of the certificates in question as mentioned in Bugzilla #5

Now we are working on the following plan as the next response.
We will inform you of the detailed schedule once it is finalized.

  • Renewal of the CA keys (rotate keys)
  • Reissuance of the certificates of the existing users
  • Revocation of the old CAs (Ca cert revocation)
  • Destruction of the old CA keys (key destruction)

Even though we think our safety management is adequate as mentioned in the previous comment, considering the required information based on the discussion contents of you and m.d.s.p thoroughly, we believe that the key destruction is the most effective way, which make the relying parties understand clearly that we solve the problem.
Therefore, we will proceed the above plan.

The progress report will be updated in this Bugzilla accordingly.
We will continue our effort to make this mission done smoothly, so we highly appreciate your understanding.

Best regards,
Hisashi Kamo

Flags: needinfo?(h-kamo)

Ryan-san,

We are still continuing our effort to make this mission done early as possible.
Please let us provide the timeline schedule on Monday July 27th.

Thank you for your consideration.

Best regards,
Hisashi Kamo

Thanks for the update. I think that, as we look to push timelines back for an incident report again, it's important to think about how to provide details to help us understand the system. Decisions are ultimately data driven: we have the policy, and if there are violations to the policy, we need to understand the data driving that. Please consider bugs like Bug 1649964 in thinking about this.

Note: Explanations about the revocation delay largely belong on Bug 1652610, rather than this bug. For this bug, the relevant incident report is in understanding how things were missed and what's being done in the future (e.g. contributing engineering or financial resources to improved linting and/or other forms of interoperability testing).

Flags: needinfo?(h-kamo)
Flags: needinfo?(h-kamo)

Ryan-san,

Thank you for the comments.

Based on your comments, we will continue to work on additional reports and the schedule, but the current plans are as follows:

  • The key pair destruction plans for the two CAs are described below:

  1. JPRS Domain Validation Authority - G3
    https://crt.sh/?id=1671982906

  2. JPRS Organization Validation Authority - G3
    https://crt.sh/?id=1671982905

  • Create a new key pair and reissue the certificate from the new compliant CA.

  • With regard to the 45,000 End Entity certificates issued by two CAs, we take user’s availability into account, and also plan to do key destruction as soon as possible.

  • Among our customers, we recognize that there is a hosting company who do not have enough automation to set up a certificate and will set up certificates of 10,000 or more, or a customer who manually setting the 5,000 certificates on a server. Our plan also includes prospects for the migration of these customers.

  • We will consider these customers, provide sufficient support, and continue to talk with them for the early destruction of keys.
  • The detailed plans are as follows:

2020-07/29：will stop issuing from non-compliant CAs and get ready to issue from new compliant CAs.
2020-08-31：5% of the certificates will be renewed or reissued.
2020-09-30：25％ of the certificates will be renewed or reissued.
2020-10-31：30％ of the certificates will be renewed or reissued.
2020-11-30：35％ of the certificates will be renewed or reissued.
2020-12-31：45％ of the certificates will be renewed or reissued.
2021-01-31：70％ of the certificates will be renewed or reissued.
2021-02-28：85％ of the certificates will be renewed or reissued.
2021-03-31：100％ of the certificates will be renewed or reissued.
2021-04-30：Key destruction will be completed.

  • In addition, we do not plan mass revocation of the leaf certificates (to avoid expansion of CRL). Instead, we will establish a system to track these reissuing activities, grasp the progress, and manage and update the plan accordingly.

  • The destruction of the key pair at one of the following CAs is planned to be done on 2020-10-31.

SECOM Passport for Member PUB CodeSigning CA G1
https://crt.sh/?id=2517735005

Thank you for your consideration.

Best regards,
Hisashi Kamo

Regarding the revocation plans and timing, I responded on Bug 1652610, where such information is more appropriate, as per https://wiki.mozilla.org/CA/Responding_To_An_Incident#Revocation

What I think is critical for this incident is better understanding this particular statement:

We once had the incident when only TLS Web Server Authentication was described in Extended Key Usage of the intermediate CA certificate and OCSP Signing was not included, the path validation of the OCSP server certificate issued by the intermediate CA failed

Of the software examined, no such client was detected that behaved like this. Understanding more about this client seems useful, for the community, in preventing future incidents like this.

Once details are provided there, I think we'll be able to close this issue out and track progress on revocation in Bug 1652610

Flags: needinfo?(h-kamo)

Dear Kamo-san,
Please respond to Ryan's question here and then we'll close this bug and continue the discussion in Bug 1652610. In that bug, I'll ask as well about the extended timeframe. I'd like work on this to happen/close before March 2021.
Thanks,
Ben

Also, I am willing to close this bug and consolidate further discussion under SECOM's bug for delayed revocation, Bug #1652610. The comments in this bug contain many valuable disclosures and observations which are preserved for cross-reference in that bug. However, before I close this bug, I would like to understand what steps SECOM takes to ensure it is following and participating in discussions that highlight these types of issues. For instance, I assume that SECOM follows and occasionally participates in discussions about the CA incidents of other CAs in Bugzilla; changes and discussions of CA/Browser Forum Guidelines, and in particular, certificate profiles; and m.d.s.p posts, which discuss developments and interpretations of the applicable standards for CAs. (Section 2.1 of the Mozilla Root Store Policy states, "CAs MUST follow and be aware of discussions in the mozilla.dev.security.policy forum, where Mozilla's root program is coordinated. They are encouraged, but not required, to contribute to those discussions.") I think that understanding what additional steps you take to ensure this expectation is met, in order to detect and prevent future issues, is useful to ensure that no future interpretation issues arise.

Dear Ryan-san and Ben-san,

Thank you for your comments.
Please let us provide our answers.

We once had the incident when only TLS Web Server Authentication was described in Extended Key Usage of the intermediate CA certificate and OCSP Signing was not included, the path validation of the OCSP server certificate issued by the intermediate CA failed

Regarding with the contents when we confirmed at that time as above, we’d like to explain in details as follows:
Since the setting names and messages used in the following explanations are translated from the contents displayed in the Japanese Windows environment, so please note that the description contents may be different in the English Windows environment.

When we double-click the OCSP responder certificate issued from the intermediate CA that does not include EKU:OCSPSign on Windows and display it in the certificate viewer, the following sentence is displayed in “Certificate> General> Certificate information”: "None of the purposes of this certificate have been verified".

<Displayed contents of OCSP responder certificate when OCSPSign is not in EKU of intermediate CA certificate>

On the other hand, when the OCSP responder certificate issued from the intermediate CA including EKU:OCSPSign is displayed by the certificate viewer of Windows by the same procedure, the phrase, "Purpose of this certificate: OCSP signature", is displayed.

<Displayed contents of OCSP responder certificate when OCSPSign is in EKU of intermediate CA certificate>

From the above explained, we considered at that time that the presence or absence of OCSPSign in the intermediate CA certificate affects the warning display of the OCSP responder certificate.

Also, I am willing to close this bug and consolidate further discussion under SECOM's bug for delayed revocation, Bug #1652610. The comments in this bug contain many valuable disclosures and observations which are preserved for cross-reference in that bug. However, before I close this bug, I would like to understand what steps SECOM takes to ensure it is following and participating in discussions that highlight these types of issues.

We understand and accept that our active participation for discussions on m.d.p.s and Bugzilla of other companies is expected.
However, one concern is that when posting to m.d.p.s or Bugzilla of other companies, there is the check/approval process as an open document within our company, which may affect the posting frequency and speed.
In recent years, on the other hand, we have strengthened CA-related systems continuously. Every day, we check the contents of Bugzilla, or m.d.p.s, etc., and hold regular meetings in our company, to grasp the contents and extract the improvement points on the system side and the operation side, so that we carry out the process of updating systematically. In addition to that, we hold a regular report meeting including upper positions to share current CABF trends, medium- and long-term issues, and discussion contents. We have getting a certain effectiveness on the speed and quality of collecting information. From now on, we’ll make effort to participate more frequently in m.d.p.s and Bugzilla of other companies, so that we can discuss new ideas, findings and questions as much as we can.

Thank you for your consideration.

Best regards,
Hisashi Kamo

Flags: needinfo?(h-kamo)

Closing this bug. Please refer to Bug #1652610 for further proceedings.

Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Product: NSS → CA Program
Whiteboard: [ca-compliance] → [ca-compliance] [ca-misissuance]
You need to log in before you can comment on or make changes to this bug.