887321 - OCSP Stapling Telemetry

Status: RESOLVED FIXED

(Core :: Security: PSM, enhancement)

Description

[Patrick McManus [:mcmanus]]

Primarily, what I want to figure out is:
a] how often does stapling prevent a OCSP lookup to the CA
and b] how often are we carrying around stapled information when it doesn't add any new state for us (because the bloat has the CWND problem).

I think A is intellectually interesting in terms of evangleization, though other changes in our revocation strategy might make it less relevant. As a matter of web metrics I think its still an interesting data point in how much of the web's data is interested in using this.

B is a medium-sized deal and something we should definitely be looking at.

To do that, I'm interested in breaking down non-resumed handshakes into the following groups (though I'm happy with other arrangements that give similar information)

1] no stapled response but did not require CA Direct OCSP (CADOCSP?)
2] no stapled response but CADOCSP was needed
3] stapled response, no CADOCSP was triggered but that would be true even without the staple
 (i.e. the staple was already cached)
4] stapled response where some CADOCSP is triggered anyhow. (i.e. root or intermediate)
5] stapled response where

And I guess also adding a SSL_BYTES_BEFORE_CERT_CALLBACK_WITH[OUT]_STAPLING or something to help differentiate handshake sizes with and without stapled responses.

Comment 1

[_ck_]

This seemed like an appropriate ticket to ask this, apologies if not:  is there *any* way to tell in Firefox if OCSP stapling was used for the current connection? Wouldn't it be nice if that information was shown under "technical details" for security?

Because Firefox right now is certainly capable of refusing connection if an OCSP staple is invalid - when I test with expired stapling files, Firefox 25 will alert me to this and refuse to connect to the site (an override would be nice too as it is impossible to connect to a site otherwise if it has an ocsp staple but it is invalid).

Comment 2

[Dana Keeler (she/her) (use needinfo) [:keeler]]

(In reply to _ck_ from comment #1)
> This seemed like an appropriate ticket to ask this, apologies if not:  is
> there *any* way to tell in Firefox if OCSP stapling was used for the current
> connection? Wouldn't it be nice if that information was shown under
> "technical details" for security?

Currently that information isn't exposed. We're working on improving and expanding the security-relevant connection information exposed to the rest of the browser/addons, but that's still in development.

Comment 3

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

David, can you do this soon so we can uplift to Firefox 27. This will be important info to share when we share the revocation checking change proposal.

Comment 4

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Attachment: patch

Here's a simple initial implementation that will get us some of the data we want quickly.

Brian - I'm not sure about the concurrency requirements of this code - does access to Telemetry::Accumulate need to be locked? Or is it best-effort? Or not a concern?

Comment 5

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

Comment on attachment 8347364 [details] [diff] [review]
patch

Review of attachment 8347364 [details] [diff] [review]:
-----------------------------------------------------------------

::: security/manager/ssl/src/SSLServerCertVerification.cpp
@@ +922,4 @@
>      }
> +  } else {
> +    // no stapled OCSP response
> +    Telemetry::Accumulate(Telemetry::OCSP_STAPLING, 1);

I would switch the values for "1" and "2". That way, a lower value is always better.

::: toolkit/components/telemetry/Histograms.json
@@ +4675,5 @@
>      "kind": "enumerated",
>      "n_values": 8,
>      "description": "Type of handshake (1=resumption, 2=false started, 3=chose not to false start, 4=not allowed to false start)"
> +  },
> +  "OCSP_STAPLING": {

Please name the probe SSL_OCSP_STAPLING, so that it sorts with the rest of the SSL_* telemetry.

@@ +4678,5 @@
> +  },
> +  "OCSP_STAPLING": {
> +    "kind": "enumerated",
> +    "n_values": 8,
> +    "description": "Status of OCSP stapling on this handshake (1=none; 2=present, good; 3=present, expired; 4=present, other error)"

Please switch the meaning of 1 and 2 and update this description.

Comment 6

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Attachment: patch v1.1

Thanks, Brian. Here's the updated patch - carrying over r+.

https://hg.mozilla.org/integration/mozilla-inbound/rev/b2151f4603dc

Comment 7

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/b2151f4603dc

Comment 8

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Comment on attachment 8348148 [details] [diff] [review]
patch v1.1

[Approval Request Comment]
Bug caused by (feature/regressing bug #): OCSP stapling
User impact if declined: we won't have data on real-world OCSP stapling implementations
Testing completed (on m-c, etc.): on m-c for a few days, initial telemetry from Nightly is as expected: http://telemetry.mozilla.org/#nightly/29/SSL_OCSP_STAPLING
Risk to taking this patch (and alternatives if risky): low
String or IDL/UUID changes made by this patch: none

Comment 9

[bhavana bajaj [:bajaj]]

Comment on attachment 8348148 [details] [diff] [review]
patch v1.1

Approving the low risk patch. :dkeeler can you please ensure the telemetry looks right once this lands, thanks !

Comment 10

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Here's aurora: https://hg.mozilla.org/releases/mozilla-aurora/rev/90581f8d3cec
Beta will be more complicated as it looks like bug 929617 hasn't landed on 27 (and this depends on those changes).

Comment 11

[Dana Keeler (she/her) (use needinfo) [:keeler]]

https://hg.mozilla.org/releases/mozilla-beta/rev/741a8c5658a9
(landed with bug 929617, bug 943115, bug 938805, bug 932519, bug 934327, and bug 930209)

Comment 12

[u279076]

Given in-testsuite coverage this fix will not be manually verified by QA. If you believe this warrants extra QA attention please nominate for testing by removing this whiteboard tag and adding the verifyme keyword. Please also provide any details you have that may inform our testing.

Comment 13

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Sounds good. FWIW, from the telemetry dashboard, everything looks fine (about 6-7% successful stapled responses, 93% nothing stapled, and very few errors (relatively speaking)).