Last Comment Bug 756152 - disable persistent telemetry to prevent telemetry data loss
: disable persistent telemetry to prevent telemetry data loss
Product: Toolkit
Classification: Components
Component: Telemetry (show other bugs)
: unspecified
: All All
-- normal (vote)
: mozilla15
Assigned To: Nathan Froyd [:froydnj]
: Georg Fritzsche [:gfritzsche]
Depends on:
  Show dependency treegraph
Reported: 2012-05-17 10:28 PDT by Nathan Froyd [:froydnj]
Modified: 2012-05-30 16:19 PDT (History)
8 users (show)
See Also:
Crash Signature:
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---

patch (3.82 KB, patch)
2012-05-17 12:01 PDT, Nathan Froyd [:froydnj]
taras.mozilla: review-
Details | Diff | Splinter Review
patch (3.39 KB, patch)
2012-05-17 17:25 PDT, Nathan Froyd [:froydnj]
taras.mozilla: review+
akeybl: approval‑mozilla‑aurora+
akeybl: approval‑mozilla‑beta+
Details | Diff | Splinter Review

Description User image Nathan Froyd [:froydnj] 2012-05-17 10:28:08 PDT
For persistent telemetry, we only send histogram data; we don't send other information that we collect during normal telemetry operation (amount of RAM, various startup times, etc.).  Due to my poor assumptions and the way telemetry data is aggregated on the server side, this approach discards (on the server side) that "extra" non-histogram information.

Loss of telemetry data is bad.  Given that restructuring the server side would be rather involved, switching persistent telemetry off for the short term is a better option.  We have open bugs (bug 748517 and related) for restructuring persistent telemetry to make it easier to save this extra information.  While those bugs can be fixed relatively quickly, shoving through the train process on such short notice doesn't seem advisable.
Comment 1 User image Nathan Froyd [:froydnj] 2012-05-17 10:29:30 PDT
Setting tracking flags.  I'm new to tracking flags, so hopefully I'm not screwing it up!
Comment 2 User image Nathan Froyd [:froydnj] 2012-05-17 12:01:46 PDT
Created attachment 624821 [details] [diff] [review]

Here's a patch that just disables the sending, but leaves everything else intact.  Tests still pass, thanks to the observer hook.
Comment 3 User image (dormant account) 2012-05-17 14:38:33 PDT
Comment on attachment 624821 [details] [diff] [review]

r-, please just change the check for persistence data existence
Comment 4 User image Nathan Froyd [:froydnj] 2012-05-17 17:25:05 PDT
Created attachment 624972 [details] [diff] [review]

Disabling by simply not loading data.
Comment 6 User image Alex Keybl [:akeybl] 2012-05-21 15:59:46 PDT
(In reply to Nathan Froyd (:froydnj) from comment #5)

Please nominate for aurora/beta approval tomorrow (5/22) and plan to land tomorrow as well so that this change makes it into FF13 beta 5.
Comment 7 User image Ed Morley [:emorley] 2012-05-22 06:47:04 PDT
Comment 8 User image Nathan Froyd [:froydnj] 2012-05-22 07:00:28 PDT
Comment on attachment 624972 [details] [diff] [review]

[Approval Request Comment]
Bug caused by (feature/regressing bug #): bug 707320 and followups.
User impact if declined: None, this feature has no user interaction.
Testing completed (on m-c, etc.): Tests in the test suite.
Risk to taking this patch (and alternatives if risky): Epsilon risk that something will go wrong and telemetry won't get sent in, period.  Unwilling to say zero risk. :)
String or UUID changes made by this patch: None.
Comment 9 User image Alex Keybl [:akeybl] 2012-05-22 11:29:35 PDT
Comment on attachment 624972 [details] [diff] [review]

[Triage Comment]
Understood on risk. Can you make sure to watch aurora/beta channel telemetry volume it's as expected?

Approved for Aurora 14 and Beta 13. Pleas land asap to make it into beta 5.
Comment 11 User image Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-05-29 10:38:55 PDT
Is there something QA can do to verify this fix?
Comment 12 User image Nathan Froyd [:froydnj] 2012-05-30 07:06:16 PDT
Nothing for QA here.
Comment 13 User image Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-05-30 10:42:49 PDT
Thanks Nathan.
Comment 14 User image Nathan Froyd [:froydnj] 2012-05-30 16:19:38 PDT
(In reply to Alex Keybl [:akeybl] from comment #9)
> Understood on risk. Can you make sure to watch aurora/beta channel telemetry
> volume it's as expected?

Aurora and Beta telemetry volume seem as expected so far, save for a brief dip (~30-40%) on the 24th and 25th, respectively.  I don't claim to understand this, seeing as how telemetry volume went up when we enabled persistent telemetry (since we were sending roughly 2x the number of pings).  One would therefore expect it to decrease by the same volume...I suppose it's possible people are not updating quickly?

In any event, Nightly telemetry volume has held more-or-less steady, so I think we're OK on the telemetry front.

Note You need to log in before you can comment on or make changes to this bug.