Make sure the LetsEncrypt data use-case is covered

RESOLVED FIXED

Status

Data Platform and Tools
Datasets: Telemetry Aggregates
P1
normal
RESOLVED FIXED
19 days ago
13 days ago

People

(Reporter: gfritzsche, Assigned: chutten)

Tracking

(Blocks: 3 bugs)

Details

MozReview Requests

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(1 attachment)

(Reporter)

Description

19 days ago
Due to bug 1406390 et al release data will no longer be aggregated from Firefox 58+ and will no longer be available for view on TMO.
Lets Encrypt seems to be using some of the release data published through aggregates.tmo, so we need to figure out a transition plan.
(Reporter)

Comment 1

19 days ago
JC, can you summarize what data LetsEncrypt needs?
Flags: needinfo?(jjones)

Comment 2

17 days ago
Georg,

Right now Let's Encrypt's https-over-time graph [1] is a Golang-clone [2][3] of rbarnes' time-series aggregator [4]. The intent is just to track adoption of HTTPS over time in some fashion that indicates how often users are negotiating TLS-protected sessions, as driving HTTPS-everywhere is the goal of Let's Encrypt.

Currently it's just pulling release data [3]. 

If I'm understanding correctly, the telemetry is not going away, it's just not going to be accessible on TMO? Perhaps this would be an opportunity to do the number-crunching on Mozilla's side and publish the summarized data directly?

[1] https://letsencrypt.org/stats/#percent-pageloads
[2] https://github.com/jcjones/ct-sql/tree/master/firefox-telemetry
[3] https://github.com/jcjones/ct-sql/blob/master/cmd/telemetry-update/main.go#L95
[4] https://ipv.sx/telemetry/general-v2.html?channels=release&measure=HTTP_PAGELOAD_IS_SSL&target=1&absolute=0&relative=1
Flags: needinfo?(jjones)
(Assignee)

Comment 3

16 days ago
Uh oh.

Network telemetry is predicated on the value of toolkit.telemetry.enabled which limits its collection to pre-release, even though it has releaseChannelCollection: opt-out. This means that we have no representative SSL data for Firefox users beyond Beta.

This is due to nsHttpChannel checking if the nsHttpHandler has telemetry enabled[1] which checks toolkit.telemetry.enabled[2].

This needs to be changed if we want SSL data from the release population. Simply popping the HTTP_{PAGELOAD|TRANSACTION}_IS_SSL accumulations out of the if(gHttpHandler->IsTelemetryEnabled()) block ought to do.

Better yet, we could remove the whole IsTelemetryEnabled part from nsHttpHandler and have the net telemetry be unconditionally accumulated.(presupposing that it isn't expensive to generate the data for accumulation.) Telemetry takes care of deciding what to store and send so that callers shouldn't have to.

(At the very least we ought to convert it to use Telemetry::CanRecordPrerelease())

[1]: http://searchfox.org/mozilla-central/rev/af86a58b157fbed26b0e86fcd81f1b421e80e60a/netwerk/protocol/http/nsHttpChannel.cpp#2268
[2]: http://searchfox.org/mozilla-central/rev/af86a58b157fbed26b0e86fcd81f1b421e80e60a/netwerk/protocol/http/nsHttpHandler.cpp#1753
Comment hidden (mozreview-request)
(Assignee)

Updated

16 days ago
See Also: → bug 1414388
thanks. I'll look at some of the other network data too. Some of it might be fairly expensive to create - but this is not.

Comment 6

16 days ago
mozreview-review
Comment on attachment 8925126 [details]
bug 1413258 - Collect SSL Telemetry on all channels

https://reviewboard.mozilla.org/r/196374/#review201634
Attachment #8925126 - Flags: review?(mcmanus) → review+
(Reporter)

Comment 7

13 days ago
(In reply to Chris H-C :chutten from comment #3)
> (At the very least we ought to convert it to use
> Telemetry::CanRecordPrerelease())

Should we have a follow-up bug on that?
That should avoid some misunderstandings.
(Reporter)

Updated

13 days ago
Blocks: 1414839
(Reporter)

Comment 8

13 days ago
(In reply to J.C. Jones [:jcj] from comment #2)
> If I'm understanding correctly, the telemetry is not going away, it's just
> not going to be accessible on TMO? Perhaps this would be an opportunity to
> do the number-crunching on Mozilla's side and publish the summarized data
> directly?

Yes, with the fix in comment 3 the data will be available, just not publicly right now.
Lets use bug 1414839 to scope out what we need to do here.
(Reporter)

Updated

13 days ago
Assignee: nobody → chutten

Comment 9

13 days ago
Pushed by chutten@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/0d15cd7ec41f
Collect SSL Telemetry on all channels r=mcmanus

Comment 10

13 days ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/0d15cd7ec41f
Status: NEW → RESOLVED
Last Resolved: 13 days ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.