Closed Bug 1994721 Opened 1 year ago Closed 1 year ago

CRLite false positive for GTS issued certificate

Categories

(Core :: Security: PSM, defect, P1)

Firefox 144
defect

Tracking

()

RESOLVED FIXED

People

(Reporter: felix.peters, Assigned: jschanck)

References

(Blocks 1 open bug)

Details

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:144.0) Gecko/20100101 Firefox/144.0

Steps to reproduce:

WebSSO login process for one of our VPN gateways opens a page in the domain openid-provider.gateway-public.supportcenter-mgmt-sit.ppifs-sandbox.net

Actual results:

Page load is rejected with SEC_ERROR_REVOKED_CERTIFICATE.
Caused by a false positive "Revoked" state of the certificate used, setting security.pki.crlite_mode to 1 or 3 in about:config allows the TLS connection / page load.

This is somewhat similar to Bug 1955023.

crlite-query result:

$ ./release/rust-query-crlite --update prod https openid-provider.gateway-public.supportcenter-mgmt-sit.ppifs-sandbox.net
INFO - Loaded 47 CRLite filter(s), most recent was downloaded: 0 hours ago
INFO - openid-provider.gateway-public.supportcenter-mgmt-sit.ppifs-sandbox.net Revoked

OCSP check by openssl:

$ openssl ocsp -noverify -no_nonce -issuer gateway-public-supportcenter-mgmt-sit-ppifs-sandbox-net-chain.pem -cert gateway-public-supportcenter-mgmt-sit-ppifs-sandbox-net.pem -text -url http://o.pki.goog/s/wr3/Zss
OCSP Request Data:
    Version: 1 (0x0)
    Requestor List:
        Certificate ID:
          Hash Algorithm: sha1
          Issuer Name Hash: 12AB48B9B763DA7E2D86C3DBB3C2773B29372D5A
          Issuer Key Hash: C781F5FD8E88D9003C4D63A2503124A0CE23FE23
          Serial Number: 66CBE67F37080DE61284B2FB0D6A43B4
OCSP Response Data:
    OCSP Response Status: successful (0x0)
    Response Type: Basic OCSP Response
    Version: 1 (0x0)
    Responder Id: C781F5FD8E88D9003C4D63A2503124A0CE23FE23
    Produced At: Oct 14 13:33:27 2025 GMT
    Responses:
    Certificate ID:
      Hash Algorithm: sha1
      Issuer Name Hash: 12AB48B9B763DA7E2D86C3DBB3C2773B29372D5A
      Issuer Key Hash: C781F5FD8E88D9003C4D63A2503124A0CE23FE23
      Serial Number: 66CBE67F37080DE61284B2FB0D6A43B4
    Cert Status: good
    This Update: Oct 14 13:33:27 2025 GMT
    Next Update: Oct 21 12:33:26 2025 GMT

    Signature Algorithm: sha256WithRSAEncryption
    Signature Value:
        26:72:ef:14:e2:36:37:dc:91:cd:b8:b3:ba:db:ba:e6:7f:55:
        ed:ac:50:2e:c2:27:c3:0b:c8:cf:63:8f:9c:8c:2b:ec:8a:42:
        c0:5a:ee:99:2c:8d:13:92:ec:95:8d:af:c7:59:2d:ad:f5:c6:
        d5:20:d9:4f:47:5a:c7:9c:70:e1:dd:42:3c:3c:29:8a:52:eb:
        a9:6d:8c:c4:ec:4d:9e:51:ad:8f:f5:ff:e5:27:9a:99:05:f1:
        d0:65:8d:d7:a2:49:6a:47:0a:a8:4d:8f:01:f7:54:18:74:f8:
        59:1a:02:b9:2c:0d:b3:bd:74:ca:4f:84:4e:e5:b2:01:e4:54:
        b0:e1:8e:98:34:04:fe:05:6f:6d:d0:8e:84:12:db:87:c6:a6:
        66:61:b1:08:ba:2f:08:56:66:0d:ea:d3:4a:ec:8b:a4:f3:9f:
        0d:b5:62:d6:c5:21:89:74:57:15:b5:88:81:ff:c4:f1:1b:18:
        29:a6:5c:a7:b1:2d:4b:f2:08:a7:ce:dc:7c:0c:76:18:71:4b:
        3b:36:96:82:a1:6a:b0:1b:8b:3b:84:78:5f:d4:4f:c5:86:f1:
        98:95:5c:00:30:38:64:a8:d7:b9:b1:4b:48:ce:6d:18:ac:95:
        50:7e:c5:c3:3c:43:49:4c:21:e9:a0:5c:52:da:d4:4a:4b:3a:
        87:e1:47:02
gateway-public-supportcenter-mgmt-sit-ppifs-sandbox-net.pem: good
  This Update: Oct 14 13:33:27 2025 GMT
  Next Update: Oct 21 12:33:26 2025 GMT

Certificate listed as good on crt.sh:
https://crt.sh/?id=21496330349

Expected results:

Certificate is considererd good during page load.

This, at least so far, only affects one of the many GTS-issued certificates we have in use.

The Bugbug bot thinks this bug should belong to the 'Core::Security: PSM' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Component: Untriaged → Security: PSM
Product: Firefox → Core

Thank you very much for this report!

CRLite incorrectly shows the certificate as revoked in the 20251005-1-default.filter.delta filter. The filter claims coverage of all Oak2026h1 timestamps earlier than "Sun Oct 5 01:18:39 PM UTC 2025" and the certificate has an SCT timestamp from that log at "Sun, 05 Oct 2025 12:45:10 PM UTC". However, the certificate was not present in the backend database when the 20251005-1 filters were generated.

The backend's CT log metadata for the 20251005-1 filters claims a maximum observed Oak2026h1 timestamp of "Sun Oct 5 01:20:05 PM UTC 2025". The latest covered timestamp should lag the maximum observed timestamp by the maximum merge delay of the log, which for Oak2026h1 is 86400 seconds. Here the latest covered timestamp lags the maximum observed timestamp by 86.4 seconds.

The root cause is unit confusion: in the calculation of the latest covered timestamp we incorrectly assumed that the MMD was given in milliseconds rather than seconds.

Assignee: nobody → jschanck
Severity: -- → S2
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Priority: -- → P1

I'll also add that Oct 5 coincides with when we rolled out the fix for Bug 1988663. That change made it so that we wait to add a certificate to the backend database until it is covered by the CT log metadata. The coverage calculation for holding back certificates did not have the same unit confusion bug, which is why there is a discrepancy between what the filter claimed was in the database and what was actually there.

We have deployed the fix and published new CRLite artifacts.

 $ ./target/release/rust-query-crlite -v --update prod https openid-provider.gateway-public.supportcenter-mgmt-sit.ppifs-sandbox.net
DEBUG - Fetching cert-revocations records from remote settings https://firefox.settings.services.mozilla.com/v1/buckets/security-state-staging/collections/
DEBUG - Found existing copy of 20251016-1-default.filter
INFO - Loaded 1 CRLite filter(s), most recent was downloaded: 0 hours ago
DEBUG - Loaded certificate from openid-provider.gateway-public.supportcenter-mgmt-sit.ppifs-sandbox.net
DEBUG - Issuer DN: C=US, O=Google Trust Services, CN=WR3
DEBUG - Subject DN: CN=*.gateway-public.supportcenter-mgmt-sit.ppifs-sandbox.net
DEBUG - Serial number: 66cbe67f37080de61284b2fb0d6a43b4
DEBUG - Issuer SPKI hash: OdSlmQD9NWJh4EbcOHBxkhygPwNSwA9Q91eounfbcoE=
INFO - openid-provider.gateway-public.supportcenter-mgmt-sit.ppifs-sandbox.net Good
Status: ASSIGNED → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
See Also: → 1995140
Blocks: crlite
You need to log in before you can comment on or make changes to this bug.