1320051 - Stop sending saved-session pings from Firefox Desktop

Status: RESOLVED DUPLICATE of bug 1443603

(Toolkit :: Telemetry, defect, P4)

Description

[Georg Fritzsche [:gfritzsche]]

With bug 1286868 fixed we should soon stop sending "saved-session" pings.

Comment 1

[Georg Fritzsche [:gfritzsche]]

This is still blocked on the e10s analysis moving over to main pings.
We need to re-check with dzeber at some point on when we can move forward.

Comment 2

[Georg Fritzsche [:gfritzsche]]

See bug 1287481, comment 7 for the last status on this.

Comment 3

[Dave Zeber [:dzeber]]

I'm discussing this offline with :ben_ as to what the impact will be to the e10s analysis, and will update soon. (Sorry, got a bit behind because of workweek travel).

Comment 4

[Dave Zeber [:dzeber]]

Ben looked into this and concluded that the e10s analysis depends on having a full session available for each client. Limiting to a main ping per client introduces bias because of the fact that main pings come from different points into the session. Although the e10s work is winding down, we are planning to carry the methodology over to other projects (eg. transition webextensions).

If we stop sending the saved-session pings, we should put in place a way to reconstruct them if necessary for analysis. For example, the job that generates the E10s dataset could replicate the saved-session pings by selecting a sessionId for each client and aggregating the subsessions for that session.

Comment 5

[Benjamin Smedberg]

Dave/Ben, can you link me to the e10s analysis in question? In particular if it's one of the analyses that was using one ping per client, that's a pretty bogus thing we were doing and should have stopped doing that a while ago.

IMO that is not something that should block this, given the ongoing confusion about this ping and the duplicate data.

Comment 6

[Ben Miroglio [:bmiroglio]]

(In reply to Benjamin Smedberg [:bsmedberg] from comment #5)
> Dave/Ben, can you link me to the e10s analysis in question? In particular if
> it's one of the analyses that was using one ping per client, that's a pretty
> bogus thing we were doing and should have stopped doing that a while ago.

Here is the script that generates the data, which *does* take one ping per client:
https://github.com/mozilla/telemetry-batch-view/blob/master/src/main/scala/com/mozilla/telemetry/views/E10sExperiment.scala#L110

This is the analysis from a week in March:
https://github.com/benmiroglio/e10s_analyses/blob/master/beta/52/week7/e10s_experiment.ipynb

Comment 7

[Ben Miroglio [:bmiroglio]]

> IMO that is not something that should block this, given the ongoing
> confusion about this ping and the duplicate data.

I agree with Dave's assertion in comment #4 that we need a way to construct a full session view should we stop sending saved-session pings. This is important for continued analysis planned for WebExtensions--which will look similar to the E10s script/notebook linked in comment #6. Like the E10s analysis, we'll need a full session so the analysis isn't biased by the point in a session we observe from a main ping.

Comment 8

[Georg Fritzsche [:gfritzsche]]

From talking through this directly with Roberto, we ended up with the following:
- The current e10s / e10s multi analysis will continue to use saved-session and the existing notebooks.
- For future WebExtension analysis, SHIELD pref flip experiments [1] look like the right path & includes automatic viewers [2].
  Ben & Dave will get in touch with Gregg Lind and look at using this.
- Medium-term, as continued e10s analysis is needed, this moves over to pref flip experiments as well.
  This should mostly require landing & uplifting some prefs for SHIELD branching support.
- By end of the year, we will turn off saved-session pings.

Ben, does that sound right?
Can you file any breakout bugs that are needed?

1: https://docs.google.com/document/d/12aGlD0dFaTG_DMLxm6ANA-SPH_ClU7atbAgpPoSLJyM/edit#
2: https://moz-experiments-viewer.herokuapp.com/?ds=7&metrics=ALL&next=%2F&pop=ALL&scale=linear&showOutliers=false

Comment 9

[Ben Miroglio [:bmiroglio]]

Yes--sounds good :)

Comment 10

[Georg Fritzsche [:gfritzsche]]

We announced that saved-session will go away by end-of-year here:
https://mail.mozilla.org/pipermail/fhr-dev/2017-May/001316.html