Closed Bug 1716902 Opened 4 years ago Closed 3 years ago

E-Tugra: Forbidden Domain Validation Method 3.2.2.4.6

Categories

(CA Program :: CA Certificate Compliance, task)

Product:
CA Program
Component:
CA Certificate Compliance
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: fozzie, Assigned: dtokgoz)

Details

(Whiteboard: [ca-compliance] [policy-failure])

E-Tugra outlines in their CPS version 4.7 (dated 12/03/2021) that they utilise DCV 3.2.2.4.6.

The wording and placement of this within the section makes it seem like E-Tugra still uses this validation for new certificates, and does not just reuse data validated using this method prior to June 3, 2020.

Can E-Tugra confirm whether or not they are using this validation method for new certificates?

Assignee: bwilson → dtokgoz
Status: NEW → ASSIGNED
Whiteboard: [ca-compliance]

This validation method does not used for any domain validation for new certificates and did not used before. We will prepare Indecent report in days.

Davut,
If e-Tugra will be updating its CPS, it should also refer to the recent set of comments I've been making to various CAs' CPs and CPSes. See
https://wiki.mozilla.org/CA/Required_or_Recommended_Practices#CP.2FCPS_Documents_will_be_Reviewed.21 If that is done, then that will speed up my CPS review related to e-Tugra's inclusion request - https://bugzilla.mozilla.org/show_bug.cgi?id=1628720.
Thanks,
Ben

  1. How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in the MDSP mailing list, a Bugzilla bug, or internal self-audit), and the time and date.
    [Etugra]: We have received email from Ben Wilson of Mozilla and Bugzilla regarding this problem On June 17th 2021.

  2. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.
    [Etugra]:
    On March 03,2020 the validation method 3.2.2.4.6. was restricted on BR 1.6.8
    On June, 2020 E-Tugra stop to use validation method 3.2.2.4.6 and began to use validation method 3.2.2.4.18
    On June 17th, 2021 E-Tugra became aware of the problem as this was not included in CPS
    On June 20th, 2021 E-Tugra started to review its CPS that covers validation methods and other Certificate policy of root stores.

  3. Whether your CA has stopped, or has not yet stopped, certificate issuance or the process giving rise to the problem or incident. A statement that you have stopped will be considered a pledge to the community; a statement that you have not stopped requires an explanation.
    [Etugra]: CA operations are not stopped

  4. In a case involving certificates, a summary of the problematic certificates. For each problem: the number of certificates, and the date the first and last certificates with that problem were issued. In other incidents that do not involve enumerating the affected certificates (e.g. OCSP failures, audit findings, delayed responses, etc.), please provide other similar statistics, aggregates, and a summary for each type of problem identified. This will help us measure the severity of each problem.
    [Etugra]: There is no certificate that is not affected from this problem. This validation method is not used After June, 2020

  5. In a case involving certificates, the complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem. In other cases not involving a review of affected certificates, please provide other similar, relevant specifics, if any.
    [Etugra]: There is no certificate no affected from this problem. This validation method is not used After June, 2020

  6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.
    [Etugra]: We noticed that the expected discretion was not shown in updating and revising the CP CPS. We recognize that we should address policy requirements as soon as they become final and not wait for scheduled review times.
    It seems that controls are not sufficient within our CP/CPS review and update procedures.

  7. List of steps your CA is taking to resolve the situation and ensure that such situation or incident will not be repeated in the future, accompanied with a binding timeline of when your CA expects to accomplish each of these remediation steps.
    [Etugra]: We started to review and update CP/CPS. It will be completed and published at the end of June.
    We will enhance our procedures to check CP/CPS reviews with root store policies, BR and other guideless.
    We will update bug after CP/CPS updated and other action.

(In reply to Davut Tokgöz from comment #3)

  1. A timeline of the actions your CA took in response. A timeline is a date-and-time-stamped sequence of all relevant events. This may include events before the incident was reported, such as when a particular requirement became applicable, or a document changed, or a bug was introduced, or an audit was done.
    [Etugra]:
    On March 03,2020 the validation method 3.2.2.4.6. was restricted on BR 1.6.8
    On June, 2020 E-Tugra stop to use validation method 3.2.2.4.6 and began to use validation method 3.2.2.4.18
    On June 17th, 2021 E-Tugra became aware of the problem as this was not included in CPS
    On June 20th, 2021 E-Tugra started to review its CPS that covers validation methods and other Certificate policy of root stores.

Have you issued certificates between the 3rd of March, 2020, and June 2020 using validation method 3.2.2.4.6? And if so, could you include their certificate data as impacted certificates?

[Etugra]: There is no certificate that is not affected from this problem. This validation method is not used After June, 2020

This double negative implies that all certificates are affected, could you clarify?

[Etugra]: We started to review and update CP/CPS. It will be completed and published at the end of June.
We will enhance our procedures to check CP/CPS reviews with root store policies, BR and other guideless.
We will update bug after CP/CPS updated and other action.

Have you searched your certificate base for certificates that were misissued, and are you revoking any certificates that have the problem?

Hi Matthias
Due your comment, we reviewed the BR revisions many times. We understand that the domain validation method on 3.2.2.4.6 is not available after June 3rd in all version after 1.6.8. Please see pages 16 and 36 on latest version; June 3rd is mentioned as last usage date for this validation method.

Then I misunderstood your previous Comment 3.

It made me believe that method 3.2.2.4.6 was forbidden starting 2020-March-03 (as first mentioned in BR v1.6.8), instead of what the BR actually states and you just commented : BR v1.6.8 came into effect on 2020-March-03, which forbids use of 3.2.2.4.6 starting 2020‐June‐03.

Apart from that, your date notation for June 2020 wasn't clear on when you stopped using the validation method described in 3.2.2.4.6: On June, 2020 E-Tugra stop to use validation method 3.2.2.4.6 and began to use validation method 3.2.2.4.18. Could you therefore clarify which specific date in June this was and whether you have (searched for) certificates that have been issued on-or-after 2020-June-03 using the validation method described in section 3.2.2.4.6?

Hi Matthias
We controlled all our internal data and check the system updates there is no SSL certificate that has Domain Name Validation with method 3.2.2.4.6 on June 2020 and after.

Hi Ben
The New CPS is published with fixing Domain name validations, and some minor correction on some sections. https://e-tugra.com.tr/wp-content/uploads/2021/07/E-Tugra_SI_v4.8_EN.pdf .
We are reviewing our CP/CPS with https://wiki.mozilla.org/CA/Required_or_Recommended_Practices#CP.2FCPS_Documents_will_be_Reviewed.21 . Until now, we did not find a major missing.

Flags: needinfo?(bwilson)

(In reply to Davut Tokgöz from comment #3)

  1. List of steps your CA is taking to resolve the situation and ensure that such situation or incident will not be repeated in the future, accompanied with a binding timeline of when your CA expects to accomplish each of these remediation steps.
    [Etugra]: We started to review and update CP/CPS. It will be completed and published at the end of June.
    We will enhance our procedures to check CP/CPS reviews with root store policies, BR and other guideless.
    We will update bug after CP/CPS updated and other action.

This doesn't provide enough detail. What were your procedures before? What specifically will you do to enhance them? What is the timeline for those steps?

The CP/CPS updates and enhancement on internal procedures were completed. In these enhancement

  • We updated our Meeting Procedures that covers reviewing instructions of BR and other related policies. The frequency and the methodology of reviews was enhanced.
  • In addition to reviewing of the CP/CPS periodically with planned timeline, in every ballot of BR, EV Guideless and others, a review procedure was added.
  • Internal audit procedures was updated that covers updating of BR compliance Audit documents will be done after each Ballots.
  • Internal BR compliance Audit documents were enhanced.

Davut: Could you re-review Comment #8?

I don't believe you Comment #9 meets the goals, as covered in Responding to an Incident and previously discussed on list

Matthew had some specific questions, and your Comment #9 does not seem to address them. Specifically, what were they before? What was done to enhance?

Flags: needinfo?(dtokgoz)

Hi, this is update for Item 7
Before the incident, Internal BR and EVG compliance documents were being reviewed at scheduled periods i.e. annually. The reviewing dates were being planned and reviewed compliance documents were being approved by the management reviews in scheduled meetings. If an unscheduled review was required due to a change on BR or EVG, a review action is triggered by the information security management team by taking the initiative.
We were already using the correct validation methods but CP/CPS was not updated so based on this incident we updated CP/CPS on July 5th, 2021 with version 4.8
After current incident and to ensure similar incidents will not be repeated in the future, we revised our procedures and included a system as follows:
Against each ballot on CA/B forum on BR or EVG, or an update on root store policies, the changes will be reviewed in a defined period and all actions needed to stay in compliance with BR and EVG will be planned.
The results of the reviewing will have to be approved by the management in this period.
Following actions to be performed:
· If required, Change management on CP/CPS and internal management documents accordingly
· If required, Change management on BR and EVG compliance documents
· If required, Development and put in production plan of operational systems (CA, RA or etc).
· If required, all other actions like training, publishing or announcement etc.
For every change on BR or EVG with a ballot, or a change on a root store policy, a report (named compliance change report) will be created that covers all actions above and will be archived.
In addition to above, we have executed the following items:
· changed the agenda of management meetings by adding item "reviewing compliance change reports"
· enhanced our internal audits by adding questions covering whether all ballots and changes on root store policies were reviewed on time and will be audited

Flags: needinfo?(dtokgoz)

Against each ballot on CA/B forum on BR or EVG, or an update on root store policies, the changes will be reviewed in a defined period and all actions needed to stay in compliance with BR and EVG will be planned.

  1. What's the defined period?
  2. Can you share more insight into why this wasn't already part of your processes?
    That is, this seems like a bare minimum to meet the long-standing requirements, and we've seen CAs share such changes for years on other CA incidents. This suggests that E-Tugra may not be following existing CA incident bugs to learn good practices, and may also have other elements of missing controls for core expectations. I realize that asking a question like "Why weren't you already doing this?" may be greeted with a response of "Well, we didn't know we had to, and now we do", but hopefully you can understand this is precisely why it raises concerns about compliance overall. In Bug 1439128, Comment #4, E-Tugra mentioned multiple people are monitoring, so it seems like there may be a gap here worth improving.

For example, Bug 1693932 was filed 2021-02-19, Bug 1705904 filed 2021-04-17, Bug 1713978 filed 2021-06-01, while this bug was reported 2021-06-16. This was just a more recent incident of the same pattern, and so it seems reasonable to ask: Why did E-Tugra fail to learn from these other incidents, and why did E-Tugra fail to recognize the issue in its own systems?

Flags: needinfo?(dtokgoz)

Separately from Comment #12, I'm also noting that Comment #11 (to now) fails to meet the expectations in https://wiki.mozilla.org/CA/Responding_To_An_Incident#Keeping_Us_Informed . Presumably, the three people monitoring m.d.s.p (Bug 1439128, Comment #4) should have seen this discussion about the expectations.

E-Tugra develops itself to meet all requirements and we make improvements that we implement regularly within ourselves. In particular, we are constantly developing internal audits that monitor compliance with BR, ETSI, and all other required policies.
In our recent internal evaluations, when we question ourselves and improvements in ourselves, we do not see the possibility of any incompatibility other than the incompatibilities arising from misinterpreting the contents of the policies.
As stated in this bug, we are developing and implementing ways of keeping ourselves up-to-date against nonconformities as a result of disruptions and/or individual oversights that may occur in regular or irregular review timings.
Tracking all bugs in Bugzilla or discussions on other related groups is done always but not done in a plan. We are sure that these are good practices that we need to know and improve ourselves. We are reviewing all these in a way by searching when needed and to improve ourselves. We are sure that the priority of tracking these must be higher as much as others.
To close gaps that we can have and to completely destroy the ideas of that community that can cause concerns about compliance overall, we are working continuously on getting bigger our team and improving our process for compliance.
As indicated in Bug 1439128, still three people are monitoring. We are in the process of getting a bigger team that are experts in compliance areas.
Before requesting closing this bug from Ben, we wanted to wait for the response of Matias and Ryan for a while. With Comment11, we are sure that the remediation is completed.

Flags: needinfo?(dtokgoz)

I think this bug has probably run its course. Comment #14 fails to answer the direct and specific questions of Comment #12 and continues to provide broad statements without any meaningful, actionable details that can be used to help reassure the community.

I can understand that E-Tugra may feel Comment #14 is appropriate, given the similarity with Comment #11, but it's clear that there's a lack of meaningful incident management or self-examination here. The controls E-Tugra describes in Comment #14 are clearly and demonstrably deficient, as captured in Comment #12, but it does not seem E-Tugra recognizes this, judging by the latest reply.

Ben: I have no further questions, if only because I think it's unlikely to get a satisfactory answer to the questions posed in Comment #12. The questions are merely a reflection of trying to understand how a CA views incidents and looks to learn, and I think we've got enough signal here to know E-Tugra is not yet at the place we want/expect of CAs.

Let us give feedback which seems not answered fully in previous comments so that it will be a more understandable and satisfactory desired feedback on this bug discussion.

We would like to answer Ryan's questions in comment 12:

(Here, the first question in Ryan's comment 12 has not been answered completely due to a copy-paste error in my text editor.
What's the defined period? -> The period is at most in a week)

The feedback we've received from Ryan, it's important to explore how other CAs respond in similar situations and find a best solution based on others feedback. When we focused on this bug, we tried to find similar situations from different CAs.

With the occurrence of the event, there is a lack of BR and similar compliance monitoring and controls. As seen in this bug, software and application changes made as a result of our controls and their CPS reflections are not reflected timely. Here, although we have deprecated the domain name verification method (3.2.2.4.6) in question, its reflection in the CPS has been delayed mistakenly. This process needs to be completed together with all its reflections in a process that needs to happen.

Before the bug appeared, internal audits for compliance within our own system, such as BR and EV, were reviewed at scheduled periods i.e. annually. Review dates were planned and reviewed compliance documents were approved by management reviews at scheduled meetings. If an update was required due to a change in BR other compliance documents, an action was triggered by taking the initiative by the information security management team. And these actions include technical changes, software updates and documentation updates (for example, updating the CP/CPS).

As Ryan stated, we had a lack of regular follow-up of existing bugs or events in other lists before the bug appears. As we stated in Comment 14, although our procedures and guidelines seem sufficient. In order to improve further and overcome the deficiencies encountered in practice and to ensure cross-checks; we are increasing the number in our group responsible for compliance and audit operations and increasing the level of skills and experience and we are very successful so far.

Specific to this bug, in addition to the answer to the 7th item in the Incident Report in Comment 11:
In case improvement and change needs that will arise from other CA cases or discussions, we will ensure that the improvement processes trigger the new process specified in Comment 11. In this context, this created process will be a part of both compliance monitoring and self-improvement processes. We have updated the relevant improvement and compliance follow-up procedures and instructions for these described changes.

Ben: Can we close this bug. We believe that we completed this bug.

I will close this bug on Wed. 3-Nov-2021 unless anyone has any additional comments, questions, or suggestions.

Ben: Can we close the bug?

Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Flags: needinfo?(bwilson)
Resolution: --- → FIXED
Product: NSS → CA Program
Whiteboard: [ca-compliance] → [ca-compliance] [policy-failure]
You need to log in before you can comment on or make changes to this bug.