1209535 - 5% sample for unified telemetry (saved-session pings) is missing on release 41.

Status: RESOLVED WONTFIX

(Toolkit :: Telemetry, defect)

Description

[Roberto Agostino Vitillo (:rvitillo)]

Only a handful of pings with build id 20150917150946 have environment/settings/telemetryEnabled set to false [1]. I would have expected to see far more pings with that flag disabled by this time.

[1] https://gist.github.com/vitillo/eb121b903cfc901a910b

Comment 1

[Georg Fritzsche [:gfritzsche]]

I've did quick checks on the latest 41 build, and i can send pings fine:
* on a new profile with a client id thats in the sample
* on my main profile when switching to a client id in that sample

I can see the pings submitted fine in Kibana using clientid 19cdea6c-ebde-cb49-8789-faacca03ed55 with:
https://kibana.shared.us-west-2.prod.mozaws.net/#/dashboard/elasticsearch/Telemetry%20client%20history%20&%20sizes 

document ids:
d1a8e73d-0a79-3740-ab7d-4ea7fea32b4d
cf863911-2963-c54b-a4ba-d169ee0c8a1d
c3cda861-a922-6f4b-baf3-00b2cfea3638
1ae4ac44-5771-f64d-a439-c2183e426b58

Comment 2

[Vladan Djeric (:vladan)]

This is actually a problem with the environment.settings.telemetryEnabled field. On Release 41, it reports environment.settings.telemetryEnabled even when the user did NOT opt into extended Telemetry.

FYI, there will be a 41.0.1 going out soon, so you could still fix this

Comment 3

[Vladan Djeric (:vladan)]

Hmm, actually, when i don't have extended Telemetry enabled, Release 41 doesn't even create a shutdown ping in my saved-telemetry-pings/ directory. So the issue could be more fundamental

Comment 4

[Georg Fritzsche [:gfritzsche]]

The telemetryEnabled measurement should be fine on 41:
https://hg.mozilla.org/releases/mozilla-release/file/523e01f0162c/toolkit/components/telemetry/TelemetryEnvironment.jsm#l987

We only save/send pings in 41 if your client id is in the 5% sample - if you are not in it, you don't get any Telemetry enabled without Telemetry opt-in ("toolkit.telemetry.enabled").

Comment 5

[Vladan Djeric (:vladan)]

After some further investigation and discussions on #datapipeline, we came to the conclusion that the real "issue" is that base-set Telemetry doesn't generate a saved-session ping on shutdown.

saved-session pings are used in Spark analyses and in the Telemetry dashboard.

This explains why Roberto didn't see telemetryEnabled=false pings in Spark, why i didn't see the saved-session pings on shutdown, why Telemetry dashes don't show additional Release channel submissions, etc

Additionally, I learned it wasn't sufficient to change the clientID in prefs (toolkit.telemetry.cachedClientID), but also had to change it in datareporting\state.json

Comment 6

[Katie Parlante]

Here's a kibana dash: https://kibana.shared.us-west-2.prod.mozaws.net/#/dashboard/elasticsearch/Fx41%20Release%20Telemetry

You can toggle the filters to show the saved-session pings.