1705480 - SECOM: CP/CPS does not clearly specify domain validation methods

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Andrew Ayer]

SECOM has disclosed the following CP/CPS:

https://repo1.secomtrust.net/spcpp/pfw/pfwev2ca/Contents/PfWEVCA-CP-EN.pdf
https://repo1.secomtrust.net/spcpp/cps/SECOM-CPS-EN.pdf

for the CA https://crt.sh/?sha256=E1F2E95000F815E11C81490430B5D02C8D81D0D256C85DF68B516D6C27761926

Neither document specifies the subsections of BR 3.2.2.4 which SECOM uses to validate domain control, as required by MRSP 2.2 (3).

Section 3.2.7 of the CP contains a catch-all "Use other reasonable methods conforming to the Baseline Requirements" but it also describes two email-based methods which refer to "randomized values" (notably not the BR-defined "Random Value") without further elaboration. It is therefore unclear whether these two methods comply with the BRs.

Comment 1

[Hisashi Kamo]

Andrew-san,

Thank you for your comment.

In terms of domain validation, CP 3.2.7 1 and 2 are validated properly conforming to BR.
The CP of SECOM Passport for Web EV2.0 had some ambiguities as you pointed out.
Therefore, we will specify the subsections of BR 3.2.2.4 in the CP.
We will also correct the unclear contents of CP/ CPS after reconfirming the contents specified in each subsection.

In accordance with MRSP 2.2 (3), in addition to CP 3.2.7 1 and 2, the validation content of the BR 3.2.2.4 subsection adopted by our company is not as 3 "Use other reasonable methods conforming to Baseline Requirements", but specifically describes the BR subsection, and then let us further improve the accuracy of CP."

Let us supplement about "Random Value".
Description of "randomized values" instead of "Random Value" was due to the mistranslation from Japanese to English, thus we are going to correct it.
We utilize the "Random Value" appropriately as below.
Random Value: A value specified by a CA to the Applicant that exhibits at least 112 bits of entropy.

Thank you for your consideration.

Best regards,
Hisashi Kamo

Comment 2

[Andrew Ayer]

To be clear, this is an incident because your CP/CPS did not specify the subsections of BR 3.2.2.4 as required by Mozilla Root Store Policy. The expectations for responding to an incident, which Comment 1 fails to meet, are described here: https://wiki.mozilla.org/CA/Responding_To_An_Incident

Comment 3

[Hisashi Kamo]

Andrew-san,

Please let us provide our incident report as below.

1. How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date.

Our CA became aware of this incident via a notification email regarding this Bug 1705480, which was received at 2:52 (JPN time) on April 16, 2021.

2. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.

April 16～21, 2021
Regarding the operation of domain validation method, we checked BR and confirmed that it was validated without any problem conforming to BR 3.2.2.4.
We noticed there are some ambiguous descriptions in the CP of SECOM Passport for Web EV2.0, as you pointed out.

3. Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation.

Our CA has not stopped issuing the certificates.

4. A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued.

There is no certificates with problem, which have been issued.

5. The complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem.

There is no certificates with problem, which have been issued.

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

The reason why this incident happened was due to lack of accuracy in the description of CP/CPS.
Regarding to the CP/CPS confirmation, we review them at least once a year and the version is updated.
In the review, we carefully confirmed that they were compliant with BR and that the revised contents were reflected when the BR was revised.

7. List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.

The CP of SECOM Passport for Web EV2.0 had some ambiguities as you pointed out.
Therefore, we will specify the subsections 3.2.2.4 of BR in the CP.
We will also correct the unclear contents of CP/ CPS after reconfirming the contents specified in each subsection.
In accordance with MRSP 2.2 (3), in addition to CP 3.2.7 1 and 2, the validation content of the BR 3.2.2.4 subsection adopted by our company is not as 3 "Use other reasonable methods conforming to Baseline Requirements", but specifically describes the BR subsection, and then let us further improve the accuracy of CP.
After reviewing the entire CP, we are planning to update it at the end of May.
The update including supplement about "Random Value" that you commented.
Description of "randomized values" instead of "Random Value" was due to the mistranslation from Japanese to English, thus we are going to correct it.
We utilize the "Random Value" appropriately as below.
Random Value: A value specified by a CA to the Applicant that exhibits at least 112 bits of entropy.

For future CP/CPS reviews, we will add MRSP to the compliance checklist, in addition to BR. We will also raise the level of awareness for the person in charge, so that any similar incident should be prevented from now on.

Thank you for your consideration.

Best regards,
Hisashi Kamo

Comment 4

[Ryan Sleevi]

(In reply to Hisashi Kamo from comment #3)

2. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.

April 16～21, 2021
Regarding the operation of domain validation method, we checked BR and confirmed that it was validated without any problem conforming to BR 3.2.2.4.
We noticed there are some ambiguous descriptions in the CP of SECOM Passport for Web EV2.0, as you pointed out.

I've highlighted where this response fails to meet what is expected.

6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

The reason why this incident happened was due to lack of accuracy in the description of CP/CPS.

This is an explanation of what happened, but it does not explain how or why the mistakes were made, nor does it explain how they avoided detection until now.

Regarding to the CP/CPS confirmation, we review them at least once a year and the version is updated.
In the review, we carefully confirmed that they were compliant with BR and that the revised contents were reflected when the BR was revised.

Clearly, this did not happen. The purpose of the incident report is to better understand why it did not happen, because the current process is deficient.

7. List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.

For future CP/CPS reviews, we will add MRSP to the compliance checklist, in addition to BR. We will also raise the level of awareness for the person in charge, so that any similar incident should be prevented from now on.

From https://wiki.mozilla.org/CA/Responding_To_An_Incident#Incident_Report

For example, it’s not sufficient to say that “human error” of “lack of training” was a root cause for the incident, nor that “training has been improved” as a solution. While a lack of training may have contributed to the issue, it’s also possible that error-prone tools or practices were required, and making those tools less reliant on training is the correct solution. When training or a process is improved, the CA is expected to provide specific details about the original and corrected material, and specifically detail the changes that were made, and how they tie to the issue. Training alone should not be seen as a sufficient mitigation, and focus should be made on removing error-prone manual steps from the system entirely.

Raising the "level of awareness for the person in charge" is, effectively, stating human error and that training has been improved. This is not an acceptable level of introspection and response.

For example, there's a clear process failure, but SECOM has both failed to document its existing process for such reviews, which are already required both by the BRs and Mozilla, nor examined why those processes failed. That "the person in charge" may have made a mistake is not a root cause, nor the goal of these incident reports. A "meets expectations" incident report would have examined what factors made it possible for the person in charge to make mistakes, and would have looked at how other CAs h ave dealt with similar incidents. We've seen a number of suggestions from CAs facing incidents, and we've also see a majority of CAs avoid this incident entirely, so it's clear that SECOM is not meeting the bare minimum of industry expectations here.

Possible solutions involve multi-party review, more regular review (e.g. monthly or quarterly), and ensuring that there is a collective team ownership, reporting to SECOM leadership, that does regular review of CA incidents, to determine if SECOM is affected by any incidents another CA may encounter, as well as to look for possible improvements to their system. This is a clear systemic failure to stay aware of industry expectations (e.g. by subscribing to the Bugzilla component), and reports like Comment #1 / Comment #3 show a clear failure to take this issue seriously and make meaningful changes and improvements.

Comment 5

[Hisashi Kamo]

Ryan-san,

Thank you for your comment.

A timeline is a date-and-time-stamped sequence of all relevant events.

Our company states the procedure for implementing domain authorization in the CP.
Since the first version was published, the updates directly related to domain authorization have been done as follows:

• 2007/06/29
  SECOM Passport for Web EV CA Certificate Policy (PfW EV CP), first version was published.
• 2015/12/25
  SECOM Passport for Web EV CA Certificate Policy (PfW EV CP), modified for domain authorization description.
• 2018/08/01
  SECOM Passport for Web EV CA Certificate Policy (PfW EV CP), modified for domain authorization description.
• 2021/04/16～ 2021/04/21
  Regarding the operation of domain validation method for domain authorization, we checked BR and confirmed that it was validated without any problem in accordance with BR 3.2.2.4.
  Regarding SECOM Passport for Web EV CA Certificate Policy (PfW EV CP), as you pointed out, we confirmed that the description for subsection was missing and the described content was unclear.

This is an explanation of what happened, but it does not explain how or why the mistakes were made, nor does it explain how they avoided detection until now.

Compared to the compliance with BR and MRSP technical requirements, the operational requirements such as CP/CPS tended to be depend on the judgment of the person in charge. In many cases, the service manager received a report from the person in charge of operation and made a proper decision for each case. The operation status of CA and the contents to be described in the regulations (CP/CPS) should have required a mechanism to comprehensively manage all CAs, which was insufficient.

Possible solutions involve multi-party review, more regular review (e.g. monthly or quarterly), and ensuring that there is a collective team ownership, reporting to SECOM leadership, that does regular review of CA incidents, to determine if SECOM is affected by any incidents another CA may encounter, as well as to look for possible improvements to their system. This is a clear systemic failure to stay aware of industry expectations (e.g. by subscribing to the Bugzilla component), and reports like Comment #1 / Comment #3 show a clear failure to take this issue seriously and make meaningful changes and improvements.

To establish a new dedicated team for conducting compliance checks, we are discussing the details with our management. The dedicated team reviews changes in our regulations and operations per the revision of BR and MRSP, and makes recommendations to the service manager.
The team will also participate in quarterly internal audits and strive to detect issues at an early stage.
We will strengthen monitoring by the internal audit team to monitor and ensure whether the dedicated team conducts regular reviews and proper operation. The dedicated team will look into a checklist for each clause of BR and MRSP, and double-check the conformity every quarter to see if each of our own CA complies with the regulations (CP/CPS, etc.). In addition, the dedicated team will review the past incidents of other company’s CAs thoroughly, and provide effective feedback on what needs to be improved.

Thank you for your consideration.

Best regards,
Hisashi Kamo

Comment 6

[Ryan Sleevi]

Thanks. I think this latest response gets closer to looking for systemic issues and fixes, and is more robust against single points of failure.

It's unclear if this team is merely proposed, or is actually being created, and what the timeline is on that. Can you clarify?

Comment 7

[Hisashi Kamo]

Ryan-san,

Thank you for your comment.

Please let us comment as below.

We are planning the following as the role of the dedicated team. We can clarify the team and get started by doing the following. We plan to let it start working as a new team in May.

・This new team will be a “dedicated team” under the direct control of the “The Certification Services Improvement Committee” by SECOM listed in the current CP, and its role will be described in the internal regulations.
・Checking regularly the policies such as BR, Ballot, MRSP, etc.to clarify CA's response.
・Conducting internal audits quarterly (checking the issuance status of CAs and considering of the necessity for revoking intermediate CA certificates and closing CAs).
・Dealing with new construction and closure of root CAs and intermediate CAs.
・Dealing with posting for Bugzilla.
・Checking the Bugzilla (incident) posts of other companies' CAs to see if there are any similar problems as ours.
・Management of issues associated with CA system update.
・Management of CP/CPS revision.
・Regular training for the operational members.
・Before conducting the key ceremony or CA certificate issuance/revocation, the dedicated team checks the procedures of these works, and the confirmed result by the team is to be reported to the service manager, and the manager approves it, then the work shall be done.

Thank you for your consideration.

Best regards,
Hisashi Kamo

Comment 8

[Ryan Sleevi]

Can you provide an update on where things stand?

Note that, as per https://wiki.mozilla.org/CA/Responding_To_An_Incident#Keeping_Us_Informed

You should also provide updates at least every week giving your progress, and confirm when the remediation steps have been completed - unless Mozilla representatives agree to a different schedule by setting a “Next Update” date in the “Whiteboard” field of the bug. Such updates should be posted to the MDSP mailing list, if there is one, and the Bugzilla bug. The bug will be closed when remediation is completed.

It seems like this is something that the now-constructed dedicated team should have been on top of, as part of the compliance requirements, but it's unclear why this was not yet done. That's why it's good to get a status update here.

Comment 9

[Hisashi Kamo]

Ryan-san,

Please let us comment as below.

Can you provide an update on where things stand?

We’d like to update the progress for the dedicated team.

We plan to let it start working as a new team in May.

We officially launched the dedicated team by the in-house notification dated May 20, and the team’s mission has started.
The mission details shall be as we described in the comment 7.
As the first mission, we’ll work on the management of CP revision, which was pointed out in this Bugzilla and CP/CPS annual updates.
Then the CP, which reflects the following points, is scheduled to be updated on May 31st.

In accordance with MRSP 2.2 (3), in addition to CP 3.2.7 1 and 2, the validation content of the BR 3.2.2.4 subsection adopted by our company is not as 3 "Use other reasonable methods conforming to Baseline Requirements", but specifically describes the BR subsection, and then let us further improve the accuracy of CP.
After reviewing the entire CP, we are planning to update it at the end of May.
The update including supplement about "Random Value" that you commented.
Description of "randomized values" instead of "Random Value" was due to the mistranslation from Japanese to English, thus we are going to correct it.

From now on, our company will strive to update Bugzilla in a timely manner in accordance with https://wiki.mozilla.org/CA/Responding_To_An_Incident#Keeping_Us_Informed, and let the dedicated team function to fully meet the compliance requirements.

Thank you for your consideration.

Best regards,
Hisashi Kamo

Comment 10

[Hisashi Kamo]

Ryan-san, Andrew-san,

Please let us tell you that we updated the CP on May 31st as scheduled.
https://repo1.secomtrust.net/spcpp/pfw/pfwev2ca/Contents/PfWEVCA-CP-EN.pdf

Then the CP, which reflects the following points, is scheduled to be updated on May 31st.

    In accordance with MRSP 2.2 (3), in addition to CP 3.2.7 1 and 2, the validation content of the BR 3.2.2.4 subsection adopted by our company is not as 3 "Use other reasonable methods conforming to Baseline Requirements", but specifically describes the BR subsection, and then let us further improve the accuracy of CP.
    After reviewing the entire CP, we are planning to update it at the end of May.
    The update including supplement about "Random Value" that you commented.
    Description of "randomized values" instead of "Random Value" was due to the mistranslation from Japanese to English, thus we are going to correct it.

Thank you for your consideration.

Best regards,
Hisashi Kamo

Comment 11

[Ryan Sleevi]

I just want to note: It's been 17 days since Comment #10, and I'm not sure how Comment #8 was ambiguous, especially in light of other CA's incidents and need for regular communication, and Comment #9's acknowledgement of that.

For example, Comment #5 stated

The dedicated team will look into a checklist for each clause of BR and MRSP, and double-check the conformity every quarter to see if each of our own CA complies with the regulations (CP/CPS, etc.).

And it's unclear whether that's something that is presently ongoing, or whether SECOM views Comment #10 as the full completion/resolution of this event. In any event, the lack of of updates since Comment #10 is concerning, because the goal of such a team, mentioned in Comment #4, is to make sure that SECOM is aware of expectations.

What I think would be useful is for SECOM to clearly state, using what they've learned from this incident report, what they see the answers to Question 7 are, including both completed-and-to-be-completed actions, so that we can accurately determine whether or not those mitigations address both the symptoms of this incident, as well as the identified root cause(s).

Comment 12

[Hisashi Kamo]

Ryan-san,

Please let us comment as below.

We’d like to update the activity status of the dedicated team.
Our dedicated team is now mainly preparing for the annual WebTrust audit evaluated by the external audit firm, KPMG from July to September. Once again, the dedicated team confirms the policy requirements thoroughly for BR and MRSP and the contents of CP and CPS.
In addition, the CCADB ALV error can be resolved by the end of September 2021, so we believe that audit omissions will not occur in the future.
Regarding with CCADB ALV errors, since 2020/06/07 - 2021/06/06 is our audit evaluation period, we are proceeding with actions including the selection of targets for Webtrust audit with KPMG.

And also, the annual training for the operational members is scheduled to be conducted in early July.

Thank you for your consideration.

Best regards,
Hisashi Kamo

Comment 13

[Hisashi Kamo]

Ryan-san, Ben-san,

Please let us update as below.
The dedicated team conducted training on July1 and July 8, today.

And also, the annual training for the operational members is scheduled to be conducted in early July.

Thank you for your consideration.

Best regards,
Hisashi Kamo

Comment 14

[Ryan Sleevi]

I just want to note that, despite Comment #13, the delta between Comment #12 and Comment #13 has again shown that Comment #9 is not reliable (as previously highlighted in Comment #11)

In addition to not meeting https://wiki.mozilla.org/CA/Responding_To_An_Incident#Keeping_Us_Informed , it does not seem SECOM is meeting https://wiki.mozilla.org/CA/Responding_To_An_Incident#Incident_Report , which does suggest that SECOM is not currently organizationally capable of adhering to root program expectations.

The issues with Comment #12 regarding audits are separately being tracked in Bug 1717044, so I'll save my concerns with Comment #12 for that bug.

I don't know what more to say productively; the set of SECOM issues have significantly undermined confidence here, and I'm not sure that there is much SECOM could do at this point to meaningfully restore it. I'm not sure whether it's appropriate to start a wiki and have a conversation about next steps for SECOM, and simply close this issue, given that Comment #10 has updated the CP/CPS, and the remaining systemic issues seem to be beyond SECOM's ability to address.

Ben: Sending this to you. The immediate remediation (updating the CP/CPS) is complete, and the systemic understanding about what went wrong, or how it's being fixed, do not seem like something we'll be able to productively make progress on in this bug.

Comment 15

[Ben Wilson]

I intend to close this bug on or about 1-August-2021, but note the systemic issues illustrated by the other SECOM bugs.