1303278 - Make the FX_SESSION_RESTORE_NUMBER_OF_TABS_RESTORED and FX_SESSION_RESTORE_NUMBER_OF_WINDOWS_RESTORED probes opt-out

Status: NEW

(Firefox :: Session Restore, defect, P3)

Description

[Alessio Placitelli [:Dexter]]

As per bug 1276200 comment 26, we should discuss making FX_SESSION_RESTORE_NUMBER_OF_WINDOWS_RESTORED and FX_SESSION_RESTORE_NUMBER_OF_TABS_RESTORED opt-out.

(In reply to brendan c from bug 1276200 comment #26)
> Thanks Alessio -- yes exactly, I was thinking of the window(s) that open
> when FF is first launched. Is FX_SESSION_RESTORE_NUMBER_OF_WINDOWS_RESTORED
> an opt-out metric, or will it be promoted to opt-out? It seems reasonable to
> differentiate the session restoration window open events from the count of
> window open events initiated by user action within a session, but we may
> want to be able to look at both; and actually, window hoarding across
> sessions is a quite informative and interesting behavior.
> 
> I'm still unclear on the process for whitelisting probes for gen-pop--
> rweiss thoughts?

Comment 1

[Alessio Placitelli [:Dexter]]

> I'm still unclear on the process for whitelisting probes for gen-pop--
> rweiss thoughts?

Comment 2

[Benjamin Smedberg]

It's a "normal" data steward review, with enhanced scrutiny:

* what is the question that these metrics will be answering?
* who is responsible for analyzing or monitoring the results? (If necessary, need bugs filed to implement dashboarding and monitoring).
* do we already have automated tests for these measures?

rweiss can do that review if can provide that info. Or if rweiss is providing that info, I should probably do the review so that she's not reviewing her own proposal.

Comment 3

[Benjamin Smedberg]

And please let's file bugs in the feature component, not in the telemetry component.

Comment 4

[[e-mail]]

:bsmedberg, you should probably provide the review.  I will answer as many questions as I can, and :bcolloran can follow up with additional details in case I am missing them.

1) We need to understand "window hoarding" as a user behavior.  Generally speaking, feature development such as Tab Center assert that management of tabs is more valuable to our users than management of windows.  This is an unsubstantiated claim.  We need to be able to compare the distribution of number of tabs compared to number of windows.

2) :bcolloran is the initial analyst who will be looking at these data. 

3) As for automated tests, I suspect :Dexter will have to answer those.

Comment 5

[Alessio Placitelli [:Dexter]]

(In reply to Rebecca Weiss from comment #4)
> 3) As for automated tests, I suspect :Dexter will have to answer those.

I'm afraid these probes don't have any test coverage. They were introduced by bug 1198898 [1]. If we promote these to opt-out, then we should be adding test coverage as part of this bug.

[1] - cset https://hg.mozilla.org/mozilla-central/rev/d6be25691040

Comment 6

[Alessio Placitelli [:Dexter]]

(In reply to Alessio Placitelli [:Dexter] from comment #5)
> (In reply to Rebecca Weiss from comment #4)
> > 3) As for automated tests, I suspect :Dexter will have to answer those.
> 
> I'm afraid these probes don't have any test coverage. They were introduced
> by bug 1198898 [1]. If we promote these to opt-out, then we should be adding
> test coverage as part of this bug.
> 
> [1] - cset https://hg.mozilla.org/mozilla-central/rev/d6be25691040

I mean, if that's a requirement to promote the probe to opt-out.

Comment 7

[brendan c]

> 2) :bcolloran is the initial analyst who will be looking at these data.

word

Comment 8

[Alessio Placitelli [:Dexter]]

(In reply to Benjamin Smedberg [:bsmedberg] from comment #2)
> It's a "normal" data steward review, with enhanced scrutiny:
> 
> * what is the question that these metrics will be answering?
> * who is responsible for analyzing or monitoring the results? (If necessary,
> need bugs filed to implement dashboarding and monitoring).
> * do we already have automated tests for these measures?

Benjamin, unfortunately these probes don't seem to have automated test coverage. Do we have to add tests before making the probes opt-out?

Comment 9

[Benjamin Smedberg]

Are there reasons why we can't add the tests now? We could delay it if there's important blocking work (like Marionette) and it's an active/prioritized work item. But in general you need tests for data because we break them all the time by accident.

Comment 10

[BugBot [:suhaib / :marco/ :calixte]]

The severity field for this bug is relatively low, S3. However, the bug has 19 votes.
:dao, could you consider increasing the bug severity?

For more information, please visit auto_nag documentation.

Comment 11

[BugBot (nomail) [:suhaib / :marco/ :calixte]]

The last needinfo from me was triggered in error by recent activity on the bug. I'm clearing the needinfo since this is a very old bug and I don't know if it's still relevant.