1426247 - Telia: Non-BR-Compliant OCSP Responder

Status: RESOLVED FIXED

(CA Program :: CA Certificate Compliance, task)

Description

[Wayne Thayer]

The OCSP responder for the TeliaSonera Root CA v1 is returning a good response for an invalid serial number as reported here: https://crt.sh/ocsp-responders?randomserial=Good&trustedBy=Mozilla&trustedFor=Server%20Authentication&trustedExclude=constrained,expired,onecrl&randomserial=Good&sort=2&dir=v

As per section 4.9.10 of the BRs, OCSP responders MUST NOT respond with a “good” status for unissued certificates. The effective date for this requirement was 2013-08-01.

Please provide an incident report in this bug, as described here:
https://wiki.mozilla.org/CA/Responding_To_A_Misissuance#Incident_Report

Comment 1

[pekka.lahtiharju]

We have investigated this and admit that there is an issue regarding Telia Root CA OCSP. We didn't realize this BR requirement was valid also for CA certificates before now incident report presented by you. Our SSL certificates are compliant but sub-cas aren't.

Our plan to fix the issue is this: We move to a completely new OCSP system on 27th Jan 2018. This issue is fixed then. Our OCSP vendor needs this extra time to fix their OCSP implementation because the current version doesn't properly support ONLINE OCSP when CA is OFFLINE like in this case.

Comment 2

[pekka.lahtiharju]

There is one week delay in our upgrade project so this issue will be solved on 3rd Feb 2018.

Comment 3

[Wayne Thayer]

The crt.sh report still shows a number of Telia OCSP responders that do not comply with the BRs. Please provide a status update.

Comment 4

[pekka.lahtiharju]

Telia changed OCSP system on Monday and since then we have had technical problems with the new OCSP. Some of the six responders have an incomplete database and we are in the process to update those. Because the data is partly incomplete we decided to configure the OCSP system to give temporarily good instead of unknown. Without that lot of our customers couldn´t use their client certificates which would be disasterous. We believe that the OCSP system is fixed during this week and we can convert back to normal BR specified OCSP answering model. Please, test again next week.

Comment 5

[Firefox Bug Husbandry Bot]

Changing QA contact per https://bugzilla.mozilla.org/show_bug.cgi?id=1438254

Comment 6

[Wayne Thayer]

I have confirmed that this problem has been fixed. Please provide the complete incident report as requested in the description.

Comment 7

[pekka.lahtiharju]

FULL INCIDENT REPORT
This bug (https://bugzilla.mozilla.org/show_bug.cgi?id=1426247) includes two different issues:

1. The first issue was that our OCSP of Offline Root CA gave "good" response for unissued certificates for a long time
This was reported by Mozilla bug at 19 Dec 2017 and confirmed by us 21 Dec 2017. At the time we were using OCSP software version that wasn't able to accept data from offline CA system to Online OCSP but we had a schedule that OCSP system will be updated few weeks later so we decided to wait that. Our OCSP vendor needed these weeks to fix this issue. This problem didn't affect any existing certificates but only when unissued serial number was used in OCSP query in context of our Root CA. Problem had probably started since BR started to require Root CA to be offline. This problem was fixed as expected when we updated to new OCSP system at 3 Feb 2018 but unfortunately the CA/OCSP update caused another problem reported below.

2. The second issue was that for six days about 0-6 percent of all our OCSP responses were incorrect.
This problem started 3 Feb 2018 18:00 EEST and was fully fixed 9 Feb 2018. We found this by ourselves in testing. It covered thousands of randomly selected certificates including SSL, client and mostly private certificates. Reason to the issue was that OCSP responders in our system had an incomplete database after the CA/OCSP migration. A small percentage of all serial numbers were missing from the OCSP database causing wrong “unknown” responses. Note! CRLs were correct all the time.  Because the OCSP data was partly incomplete on Monday 5 Feb 2018 we decided to configure the OCSP system to give temporarily "good" instead of "unknown" until a full recovery of OCSP database was finished. Because of the size of the database recovery took several days. We were forced to give "good" instead of "unknown" during these days because otherwise a lot of end users (thousands) couldn't use their legitimate client certificates to be able to login to critical IT systems used by e.g. national health care. The OCSP database was recovered 9 Feb 2018 and then we immediately restored the normal OCSP response configuration. The root cause to incomplete OCSP database was later related to CA data migration that was done previously when we changed our CA vendor. Thus this can't happen again because migration was one-time effort. A great majority of affected certificates were private certificates but some publicly trusted certificates also were affected because random 0-6 % were affected. Responses were more accurate at the end of the six day period. We can't any more tell which certificates gave incorrect "good" or “unknown” during the problem period.