Open Bug 1303278 Opened 5 years ago Updated 4 years ago

Make the FX_SESSION_RESTORE_NUMBER_OF_TABS_RESTORED and FX_SESSION_RESTORE_NUMBER_OF_WINDOWS_RESTORED probes opt-out

Categories

(Firefox :: Session Restore, defect, P3)

defect
Points:
1

Tracking

()

Tracking Status
firefox51 --- affected

People

(Reporter: Dexter, Unassigned)

References

(Blocks 2 open bugs)

Details

(Whiteboard: [measurement:client])

As per bug 1276200 comment 26, we should discuss making FX_SESSION_RESTORE_NUMBER_OF_WINDOWS_RESTORED and FX_SESSION_RESTORE_NUMBER_OF_TABS_RESTORED opt-out.

(In reply to brendan c from bug 1276200 comment #26)
> Thanks Alessio -- yes exactly, I was thinking of the window(s) that open
> when FF is first launched. Is FX_SESSION_RESTORE_NUMBER_OF_WINDOWS_RESTORED
> an opt-out metric, or will it be promoted to opt-out? It seems reasonable to
> differentiate the session restoration window open events from the count of
> window open events initiated by user action within a session, but we may
> want to be able to look at both; and actually, window hoarding across
> sessions is a quite informative and interesting behavior.
> 
> I'm still unclear on the process for whitelisting probes for gen-pop--
> rweiss thoughts?
Blocks: 1252625
Points: --- → 1
Priority: -- → P3
Whiteboard: [measurement:client]
> I'm still unclear on the process for whitelisting probes for gen-pop--
> rweiss thoughts?
Flags: needinfo?(rweiss)
It's a "normal" data steward review, with enhanced scrutiny:

* what is the question that these metrics will be answering?
* who is responsible for analyzing or monitoring the results? (If necessary, need bugs filed to implement dashboarding and monitoring).
* do we already have automated tests for these measures?

rweiss can do that review if can provide that info. Or if rweiss is providing that info, I should probably do the review so that she's not reviewing her own proposal.
And please let's file bugs in the feature component, not in the telemetry component.
Component: Telemetry → Session Restore
Product: Toolkit → Firefox
:bsmedberg, you should probably provide the review.  I will answer as many questions as I can, and :bcolloran can follow up with additional details in case I am missing them.

1) We need to understand "window hoarding" as a user behavior.  Generally speaking, feature development such as Tab Center assert that management of tabs is more valuable to our users than management of windows.  This is an unsubstantiated claim.  We need to be able to compare the distribution of number of tabs compared to number of windows.

2) :bcolloran is the initial analyst who will be looking at these data. 

3) As for automated tests, I suspect :Dexter will have to answer those.
Flags: needinfo?(rweiss)
Flags: needinfo?(bcolloran)
Flags: needinfo?(alessio.placitelli)
(In reply to Rebecca Weiss from comment #4)
> 3) As for automated tests, I suspect :Dexter will have to answer those.

I'm afraid these probes don't have any test coverage. They were introduced by bug 1198898 [1]. If we promote these to opt-out, then we should be adding test coverage as part of this bug.

[1] - cset https://hg.mozilla.org/mozilla-central/rev/d6be25691040
Flags: needinfo?(alessio.placitelli)
(In reply to Alessio Placitelli [:Dexter] from comment #5)
> (In reply to Rebecca Weiss from comment #4)
> > 3) As for automated tests, I suspect :Dexter will have to answer those.
> 
> I'm afraid these probes don't have any test coverage. They were introduced
> by bug 1198898 [1]. If we promote these to opt-out, then we should be adding
> test coverage as part of this bug.
> 
> [1] - cset https://hg.mozilla.org/mozilla-central/rev/d6be25691040

I mean, if that's a requirement to promote the probe to opt-out.
> 2) :bcolloran is the initial analyst who will be looking at these data.

word
Flags: needinfo?(bcolloran)
Blocks: 1305041
No longer blocks: 1252625
(In reply to Benjamin Smedberg [:bsmedberg] from comment #2)
> It's a "normal" data steward review, with enhanced scrutiny:
> 
> * what is the question that these metrics will be answering?
> * who is responsible for analyzing or monitoring the results? (If necessary,
> need bugs filed to implement dashboarding and monitoring).
> * do we already have automated tests for these measures?

Benjamin, unfortunately these probes don't seem to have automated test coverage. Do we have to add tests before making the probes opt-out?
Flags: needinfo?(benjamin)
Are there reasons why we can't add the tests now? We could delay it if there's important blocking work (like Marionette) and it's an active/prioritized work item. But in general you need tests for data because we break them all the time by accident.
Flags: needinfo?(benjamin)
You need to log in before you can comment on or make changes to this bug.