Closed Bug 1771482 Opened 3 years ago Closed 3 years ago

CFCA: Precertificate with postalCode and streetAddress swapped

Categories

(CA Program :: CA Certificate Compliance, task)

Product:
CA Program
Component:
CA Certificate Compliance
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: michel, Assigned: bixinlong)

Details

(Whiteboard: [ca-compliance] [ev-misissuance])

Attachments

(1 file)

6811189993_ocsp.txt
6.58 KB, text/plain
Details

Hello,
I found the precertificate https://crt.sh/?id=6811189993&opt=zlint issued by CFCA that seems to have the postalCode and streetAddress fields swapped.

Assignee: bwilson → bixinlong
Status: NEW → ASSIGNED
Whiteboard: [ca-compliance]

We need to have CFCA's incident report.

Flags: needinfo?(bixinlong)

(In reply to Ben Wilson from comment #1)

We need to have CFCA's incident report.

  1. Problem Report:
    CFCA recognized this problematic certificate via a email from Ben Wilson on June 1, 2022, which report by Michel Le Bihan on May 27, 2022.

  2. Timeline:
    June 1, 2022: After we received this, we immediately communicated with our certificate issuing department and confirmed the certificate was wrongly issued. Subsequently, we planned to revoke the certificate as soon as possible and re-issue a certificate. We contacted the customer and explain the situation, the customer resubmitted a application and we issued a new certificate on the same day, they hoped to revoke the certificate after it was replaced.
    June 2, 2022: The certificate was replaced on June 2, we revoked the wrong certificate on the same day.

  3. Statement
    CFCA has revoked the certificate.

  4. Summary
    CFCA only issued one certificate of the PostalCode and streetAddress swapped, no impact on other customers.

  5. Certificate Data:
    Please visit https://crt.sh/?id=6844251263&opt=zlint to check the new certificate data.

  6. Explanation:
    This problem is due to some staff error, the certificate-issuing officer mistakenly switched PostalCode and Street Addresses, and the certificate-reviewing officer didn’t notice.

  7. Steps:
    (1). We have communicated and trained with the issuing and auditing staff, they will more careful in issuing and reviewing certificates.
    (2). We're looking for possible Bug in the system, if they exist we will fixing them as soon as possible, to maintain the reliability of the system, reduce staff error.

(In reply to Ben Wilson from comment #1)

We need to have CFCA's incident report.

Hi Ben,

I just replied with a old account, could you help me delete comment #2 which report by Oliver Bi, thanks.

The following is CFCA's incident report.

  1. Problem Report:
    CFCA recognized this problematic certificate via a email from Ben Wilson on June 1, 2022, which report by Michel Le Bihan on May 27, 2022.

  2. Timeline:
    June 1, 2022: After we received this, we immediately communicated with our certificate issuing department and confirmed the certificate was wrongly issued. Subsequently, we planned to revoke the certificate as soon as possible and re-issue a certificate. We contacted the customer to explain the situation, the customer resubmitted a application and we issued a new certificate on the same day, they hoped to revoke the certificate after it was replaced.
    June 2, 2022: The certificate was replaced on June 2, we revoked the wrong certificate on the same day.

  3. Statement
    CFCA has revoked the certificate.

  4. Summary
    CFCA only issued one certificate of the PostalCode and streetAddress swapped, no impact on other customers.

  5. Certificate Data:
    Please visit https://crt.sh/?id=6844251263&opt=zlint to check the new certificate data.

  6. Explanation:
    This problem is due to some staff error, the certificate-issuing officer mistakenly switched PostalCode and Street Addresses, and the certificate-reviewing officer didn’t notice.

  7. Steps:
    (1). We have communicated and trained with the issuing and auditing staff, they will more careful in issuing and reviewing certificates.
    (2). We're looking for possible Bug in the system, if they exist we will fixing them as soon as possible, to maintain the reliability of the system, reduce staff error.

Flags: needinfo?(bixinlong)

(In reply to bixinlong from comment #3)

(In reply to Ben Wilson from comment #1)

We need to have CFCA's incident report.

Hi Ben,

I just replied with an old account, could you help me delete comment #2 which report by Oliver Bi, thanks.

The following is CFCA's incident report.

  1. Problem Report:
    CFCA recognized this problematic certificate via a email from Ben Wilson on June 1, 2022, which report by Michel Le Bihan on May 27, 2022.

  2. Timeline:
    June 1, 2022: After we received this, we immediately communicated with our certificate issuing department and confirmed the certificate was wrongly issued. Subsequently, we planned to revoke the certificate as soon as possible and re-issue a certificate. We contacted the customer to explain the situation, the customer resubmitted a application and we issued a new certificate on the same day, they hoped to revoke the certificate after it was replaced.
    June 2, 2022: The certificate was replaced on June 2, we revoked the wrong certificate on the same day.

  3. Statement
    CFCA has revoked the certificate.

  4. Summary
    CFCA only issued one certificate of the PostalCode and streetAddress swapped, no impact on other customers.

  5. Certificate Data:
    Please visit https://crt.sh/?id=6844251263&opt=zlint to check the new certificate data.

  6. Explanation:
    This problem is due to some staff error, the certificate-issuing officer mistakenly switched PostalCode and Street Addresses, and the certificate-reviewing officer didn’t notice.

  7. Steps:
    (1). We have communicated and trained with the issuing and auditing staff, they will more careful in issuing and reviewing certificates.
    (2). We're looking for possible Bug in the system, if they exist we will fixing them as soon as possible, to maintain the reliability of the system, reduce staff error.

Hello,
Thank you for your incident report. Does CFCA have preissuance or postissuance lints? If yes what tools are you using?

Flags: needinfo?(bixinlong)

Could you please also explain why this certificate is Revoked in CRL, but Good in OCSP?

Attached file 6811189993_ocsp.txt —

(In reply to Michel Le Bihan from comment #6)

Could you please also explain why this certificate is Revoked in CRL, but Good in OCSP?

We extract the log and analyze it. We find some cache data is not synchronized after the OCSP system update in May, we will update the test system today and update the production system in the next day. I think it will be solved after the update.

Flags: needinfo?(bixinlong)

Thank you for the info. Could you please also reply to my question in https://bugzilla.mozilla.org/show_bug.cgi?id=1771482#c5 ?

Please file a separate incident report for the OCSP issue.

Flags: needinfo?(bixinlong)

(In reply to Michel Le Bihan from comment #9)

Thank you for the info. Could you please also reply to my question in https://bugzilla.mozilla.org/show_bug.cgi?id=1771482#c5 ?

Please file a separate incident report for the OCSP issue.

Our certificate issuer checks via zlint or https://crt.sh, which are currently checked manually. We are already planning to add this functionality to our certificate software.

I will file a separate incident report for the OCSP issue today.

Flags: needinfo?(bixinlong)

(In reply to Michel Le Bihan from comment #9)

Thank you for the info. Could you please also reply to my question in https://bugzilla.mozilla.org/show_bug.cgi?id=1771482#c5 ?

Please file a separate incident report for the OCSP issue.

  1. Problem Report:
    CFCA found this while dealing with another incident, which also report by Michel Le Bihan on June 11, 2022.

  2. Timeline:
    June 1, 2022: We received an report about a certificate was wrongly issued , we immediately communicated with our certificate issuing department and confirmed the certificate was wrongly issued. Subsequently, we planned to revoke the certificate as soon as possible and re-issue a certificate. We contacted the customer to explain the situation, the customer resubmitted a application and we issued a new certificate on the same day, they hoped to revoke the certificate after it was replaced.
    June 2, 2022: The certificate was replaced on June 2, we revoked the wrong certificate on the same day.
    June 5, 2022: When I prepared the incident report, I checked the certificate status, I found an abnormality in the OCSP status of this certificate. Over the next few days, we extract the log and analyze it, we find some cache data is not synchronized after the OCSP system update in May.
    June 15, 2022: We updated the test system on June 15 and update the production system in June 16, the problem has been fixed when the system was updated.

  3. Statement
    CFCA have fixed this.

  4. Summary
    The incident caused the OCSP service to be abnormal from May 5th to June 15th. According to statistics, about 14 certificates were temporarily affected during this period, but these were resolved as the system was updated.

  5. Explanation:
    This problem is due some cache data is not synchronized after the OCSP system update in May.

  6. Steps:
    (1). We will pay more attention to the verification after system update.
    (2). We plan to add preissuance and postissuance lints on the certificate system, to ensure that errors can be discovered and resolved timely.

(In reply to bixinlong from comment #11)

(In reply to Michel Le Bihan from comment #9)

Thank you for the info. Could you please also reply to my question in https://bugzilla.mozilla.org/show_bug.cgi?id=1771482#c5 ?

Please file a separate incident report for the OCSP issue.

  1. Problem Report:
    CFCA found this while dealing with another incident, which also report by Michel Le Bihan on June 11, 2022.

Could you create a separate bug for this issue using the incident reporting guidelines of [0]? The OCSP incident seems unrelated to the swapped postalCode and streetAddress, and thus would require separate tracking.

[0] https://wiki.mozilla.org/CA/Responding_To_An_Incident#Incident_Report

(In reply to bixinlong from comment #10)

(In reply to Michel Le Bihan from comment #9)

Thank you for the info. Could you please also reply to my question in https://bugzilla.mozilla.org/show_bug.cgi?id=1771482#c5 ?

Please file a separate incident report for the OCSP issue.

Our certificate issuer checks via zlint or https://crt.sh, which are currently checked manually. We are already planning to add this functionality to our certificate software.

I will file a separate incident report for the OCSP issue today.

When do you plan to implement automated checks?

Flags: needinfo?(bixinlong)

(In reply to Michel Le Bihan from comment #13)

(In reply to bixinlong from comment #10)

(In reply to Michel Le Bihan from comment #9)

Thank you for the info. Could you please also reply to my question in https://bugzilla.mozilla.org/show_bug.cgi?id=1771482#c5 ?

Please file a separate incident report for the OCSP issue.

Our certificate issuer checks via zlint or https://crt.sh, which are currently checked manually. We are already planning to add this functionality to our certificate software.

I will file a separate incident report for the OCSP issue today.

When do you plan to implement automated checks?

We are currently working on it, as it involves requirements research, software development, system testing and bring the system online, I think it will take several months.

Flags: needinfo?(bixinlong)

(In reply to bixinlong from comment #10)

I will file a separate incident report for the OCSP issue today.

Did you create a new Bugzilla bug for the OCSP issue? Can you link to it?

It needs to be in a separate bug so it can be tracked separately. Your analysis and steps to resolve that problem should then be posted in that new bug.

I have created a new Bugzilla Bug for the OCSP issue, please see https://bugzilla.mozilla.org/show_bug.cgi?id=1778035

As a follow-up to Comment #3, were any other bugs in the system found? If so, how and when were they remediated?
Thanks,
Ben

Flags: needinfo?(bixinlong)

(In reply to Ben Wilson from comment #17)

As a follow-up to Comment #3, were any other bugs in the system found? If so, how and when were they remediated?
Thanks,
Ben

We have not found other new bugs in the system which will affecting certificate issue yet.

We'll continue to pay more attention to this. We've expanded the scope of the test, it may help us find some unexpected errors in time.

Thanks

Flags: needinfo?(bixinlong)

I will close this on or about Friday 19-Aug-2022 unless there are any additional or open issues where a response is needed.

Flags: needinfo?(bwilson)
Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Flags: needinfo?(bwilson)
Resolution: --- → FIXED
Product: NSS → CA Program
Whiteboard: [ca-compliance] → [ca-compliance] [ev-misissuance]
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: