Telia: Misissued certificate - FQDN without domain part (e_dnsname_not_valid_tld)

RESOLVED FIXED

Status

task
RESOLVED FIXED
6 months ago
8 days ago

People

(Reporter: pekka.lahtiharju, Assigned: pekka.lahtiharju)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [ca-compliance])

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36 Edge/16.16299

Steps to reproduce:

Telia did mass lint scan to find all invalid Telia certificates

Actual results:

Telia found 9 invalid certificates

Telia did mass lint scan to all its SSL certificates on Friday, February 8, 2019 to find all old certificates that are invalid. This mass scan revealed 9 previously unknown invalid certificates in 6 different error categories:

a) 2 CN IP value incorrectly in SAN DNS field (e_dnsname_not_valid_tld)(created in 2016)
b) 1 FQDN value incorrectly in SAN rfc822 field (e_ext_san_rfc822_name_present,e_subject_common_name_not_from_san)(created in 2016)
c) 1 CN SAN had leading space (e_dnsname_bad_character_in_label,e_subject_common_name_not_from_sa)(created in 2016)
d) 1 CN FQDN without domain part (e_dnsname_not_valid_tld)(created in 1Q2017)
e) 1 invalid wildcard format (e_dnsname_left_label_wildcard_correct)(created in 1Q2018)
f) 3 invalid OU value "-" (e_subject_contains_noninformational_value)(created in 2017-2Q2018)

Category d) incident report is in this bz. Others are in other incident reports created by Telia.

  1. How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, a discussion in mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the time and date.

Telia did mass lint scan to all its SSL certificates on Friday, February 8, 2019.

  1. A timeline of the actions your CA took in response.

Friday, February 8, 2019: Discovery by Telia itself
Friday, February 8, 2019: Preliminary analysis of the issue by Telia Security Board (e_dnsname_not_valid_tld). Not critical but invalid according to BR 4.9.1.1, rule7 thus decision was: "Should be revoked within 24 hours and MUST revoke a Certificate within 5 days"
Monday, February 11, 2019: Quick analysis that confirmed that similar error can't happen in current Telia systems.
February 11-15: Root cause analysis: Telia has added FQDN verification code that prevents values without TLD when such requirement came from BR several years ago. Normally prevention has worked but not in this particular case. We can see from the logs from March 30,2017 that code had detected this invalidity: "Wrong domains found; info=ev001eventprod". But nevertheless code then incorrectly accepted the CSR anyway. Next similar trial without TLD on the logs was correctly rejected few months later. Our hypothesis is that there was a bug related to error handling that was fixed soon after the accident. Because at the time there wasn't any reasonable way to run mass scan to our database we didn't scan older errors after the fix. Now Telia has a new CA software and mass database scans and also mass lint scans are possible and done after each major fix.
Friday, February 15, 2019: Revoked by Telia CA. Renewal of the certificate was started right after we found this problem. Renewal wasn't possible immediately so we let the invalid certificate live to maximum 5 days and then Telia CA operators revoked it and reported to me it was revoked. However, the original revocation failed and we only thought it was revoked in time. When I double-checked the status of all nine lint invalid certificates I found that the original revocation of this one was failed and operators revoked this one again successfully. Then I was able to verify it. To avoid this kind of delay in the future we decided to modify our process so that we verify all lint revocations within 5 days.

  1. Whether your CA has stopped, or has not yet stopped, issuing certificates with the problem. A statement that you have will be considered a pledge to the community; a statement that you have not requires an explanation.

Telia hasn't reproduced this error after this one. Can't happen again.

  1. A summary of the problematic certificates.
  2. The complete certificate data for the problematic certificates.

Only one of this kind exists. Affected certificate has CN+SAN value "ev001eventprod" and has serial 26649485c0947f748284e9d4c6bcfe8f and was created March 30, 2017 for three years. It can't be found from CT logs but Telia can give details of it if asked.

  1. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now.

Check root cause analysis above why this happened. This wasn't detected before by Telia, browsers, auditors or community because a) it caused no problems, b) it wasn't logged to CT, c) Telia didn't do lint check for old certificates before now. If similar problem would happen now the issue would be detected because of regular lint checks by Telia.

  1. List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things.

Telia has already resolved this like described above.

Assignee: wthayer → pekka.lahtiharju
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Whiteboard: [ca-compliance]

PEM of this problem is this:

-----BEGIN CERTIFICATE-----
MIIF+TCCA+GgAwIBAgIQJmSUhcCUf3SChOnUxrz+jzANBgkqhkiG9w0BAQsFADBG
MQswCQYDVQQGEwJGSTEUMBIGA1UECgwLVGVsaWFTb25lcmExITAfBgNVBAMMGFRl
bGlhU29uZXJhIFNlcnZlciBDQSB2MjAeFw0xNzAzMzAwNzQwMDNaFw0yMDAzMjkw
NzQwMDNaMGoxCzAJBgNVBAYTAlNFMQ8wDQYDVQQHDAZLYWxtYXIxGTAXBgNVBAoM
EFRlbGlhIENvbXBhbnkgQUIxFjAUBgNVBAsMDUlULU9wZXJhdGlvbnMxFzAVBgNV
BAMMDmV2MDAxZXZlbnRwcm9kMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKC
AQEAr7VlnpxI/ZwVfJvhGWtl8aV/YAySvVNp1/MkO7vXBa0greeGw17iiP5cR7tf
MEc9Gk2Fe9pvqREvRrQqzE4NCyYpT0XczaYJ1Lz0193+dHA1ARhZ3xcrAn5aNwNW
WjTq0GnfPvB+cfZNTAqJtvwbKn/3E0/2rIMybqNgdl9SKEzhL42fG++u2nxxN7FW
Q98v1tISBCdyIqmBUoa6dA9ZQnSIt6pzcQPwIqcS3hswGFelLVa3ptPKiLsN8x73
7f+Yc5GbzMs0Q9CINAladEj+9Fuy8Q9rWSn4MtAf1+gGhPgofbxRvkE5T7hqiyzb
H3EJnS1OmSI1LripTOEbDjRBPwIDAQABo4IBvTCCAbkwgY0GCCsGAQUFBwEBBIGA
MH4wLQYIKwYBBQUHMAGGIWh0dHA6Ly9vY3NwLnRydXN0LnRlbGlhc29uZXJhLmNv
bTBNBggrBgEFBQcwAoZBaHR0cDovL3JlcG9zaXRvcnkudHJ1c3QudGVsaWFzb25l
cmEuY29tL3RlbGlhc29uZXJhc2VydmVyY2F2Mi5jZXIwHwYDVR0jBBgwFoAUL0k8
KU/XByX5xozVZPVmPRKDIpUwTgYDVR0gBEcwRTBDBgZngQwBAgIwOTA3BggrBgEF
BQcCARYraHR0cDovL3JlcG9zaXRvcnkudHJ1c3QudGVsaWFzb25lcmEuY29tL0NQ
UzBNBgNVHR8ERjBEMEKgQKA+hjxodHRwOi8vY3JsLTMudHJ1c3QudGVsaWFzb25l
cmEuY29tL3RlbGlhc29uZXJhc2VydmVyY2F2Mi5jcmwwHQYDVR0lBBYwFAYIKwYB
BQUHAwEGCCsGAQUFBwMCMA4GA1UdDwEB/wQEAwIEsDAZBgNVHREEEjAQgg5ldjAw
MWV2ZW50cHJvZDAdBgNVHQ4EFgQUb4i27MYTiiXY6MirLqCMrOyE3xowDQYJKoZI
hvcNAQELBQADggIBAKjSWuU9E/UCXXtw2wIpTizv+ukEJ/dU3oMJNYNsdQO1860X
RCUjC9d4MBW7HW5GmD0OMRj2vQDwW3T/gGRV0t2pFI6H4IFDxTHkIDdIxpKGkLB6
GD/IE7IPcllPBtEBt2VmHu9EEJRS6csSn9dzyHU6givz0BbY6Do4fN2OEw6orYQn
6DX1NlCFdL8BrZOjGHZhiwtBHnc5soNlzzIyx7mzPFijXRB+Sc/AaTsV0nfWWj82
nUDjj6SbXLmwwQ7RvlFCRitBNZvMAXCo8spz+OtGEYuOmrD81gFmHIoOTWiKlwuv
LONW9q5zms8NKoEdHR9BkNiiyQaA3/+yyWyY8qy66JZoj5ZQlrIpl2NjLkIoIKyh
rNRyfT2MN9mi9PkZzGI9qaSmGHQIgS1/Cmf/3C+xwvLxzyDuhjrxjC7xarz+3nDJ
eBdTVMaomjUMiGWRHOuofhuXdacelX8lRMP8I6YVfvzglWQhkLouSa+sOiJeEslW
MDdrS+v9oo/nZ2ggwJYc6ZX5/sP+Yu9yhpO+HPRhMVC4endvmYFuwYDS/gLKoGvk
Dke3LkjPEJd66DicnNrbcQgHUHDyIFWYuF63vnbxFpHyNVp7b+FpdBJrt2cKgkMw
uhpqyBAbhAiIDMfcYSo0cZJRuulBIhJsoAxcsQRcrtC+5l6euTAiNNi/8RyR
-----END CERTIFICATE-----

QA Contact: kwilson → wthayer

Correcting bug type to task.

Type: defect → task
Flags: needinfo?(wthayer)

It appears that all questions have been answered and remediation is complete.

Status: ASSIGNED → RESOLVED
Closed: 8 days ago
Flags: needinfo?(wthayer)
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.