Closed Bug 1464019 Opened 2 years ago Closed 2 years ago

Snippets can no longer be turned off when browser.newtabpage.activity-stream.asrouterExperimentEnabled is true

Categories

(Firefox :: Messaging System, defect, P2)

defect

Tracking

()

VERIFIED FIXED
Firefox 62
Iteration:
62.2 - Jun 4
Tracking Status
firefox60 --- unaffected
firefox61 --- verified
firefox62 --- verified

People

(Reporter: cmuresan, Assigned: k88hudson)

References

(Blocks 1 open bug)

Details

User Story

https://github.com/mozilla/activity-stream/compare/firefox-61b7...firefox-61b11

Attachments

(4 files)

[Affected versions]:
- Nightly 62.0a1 Build ID 20180522220730
- Beta 61.0b7    Build ID 20180521110957

[Affected Platforms]:
- All Windows
- All Mac
- All Linux

[Prerequisites]:
- Set "browser.newtabpage.activity-stream.feeds.snippets" to "true".
- Set "browser.onboarding.notification.finished" to "true".
- Set "browser.newtabpage.activity-stream.asrouterExperimentEnabled" to true.

[Steps to reproduce]:
1. Open a new tab and click the Gearwheel button.
2. Click the Snippets checkbox.
3. Open a New Tab and observe the bottom part of the page.

[Expected results]:
- Snippets are no longer displayed.

[Actual results]:
- Snippets are still displayed.

[Notes]:
- Attached a screen recording of the issue.
Assignee: nobody → khudson
Iteration: --- → 62.2 - Jun 4
Priority: -- → P1
Ok, looks like flipping the snippets pref in about:preferences will trigger the experiment to be turned off, so this will not need an uplift. However, we should still fix this for 62
Iteration: 62.2 - Jun 4 → 62.3 - Jun 18
Assignee: khudson → nobody
Priority: P1 → P2
@kate, are we sure this is correct? Right now if we turn off snippets from about:preferences#home, they will NOT be turned off and will still be displayed on subsequent new tabs. Is this the correct behavior?
Flags: needinfo?(khudson)
If our targeting parameters include that:
   browser.newtabpage.activity-stream.feeds.snippets must be true

and a user in the experiment who is seeing AS Router messages turns snippets off in about:preferences (i.e. sets browser.newtabpage.activity-stream.feeds.snippets to false), according to jkerim, they will get immediately dropped from the cohort. That means browser.newtabpage.activity-stream.asrouterExperimentEnabled would be flipped to false, and the snippets would disappear.

If this is true, the result is that:
- people in control for 61 are unaffected
- people in the experiment group are unaffected
- only people who manually flip their experiment pref would be effected 

That being said I appreciate this is kind of hard to verify/depends on shield targeting, so I'm also happy to just fix this and uplift it if we would be more comfortable with that
Flags: needinfo?(khudson)
What's your opinion Ciprian?
Flags: needinfo?(ciprian.georgiu)
I think I'm just going to fix this / uplift just in case.
Assignee: nobody → khudson
Iteration: 62.3 - Jun 18 → 62.2 - Jun 4
Commits pushed to master at https://github.com/mozilla/activity-stream

https://github.com/mozilla/activity-stream/commit/fef272f2b40295de03cf2d075394824ddc2b651d
Bug 1464019 - Turn off asrouter snippets when snippets.feed is is false

https://github.com/mozilla/activity-stream/commit/74225d53b3dfbd6239a408e6882afd1a4cfffe4b
Merge pull request #4172 from k88hudson/bug1464019

Bug 1464019 - Turn off asrouter snippets when snippets.feed is false
@Kate, I didn't know we can drop users out of an experiment that easily. Personally, I think we're better off being safe rather than being sorry.
Flags: needinfo?(ciprian.georgiu)
Attachment #8981937 - Flags: review?(edilee)
Comment on attachment 8981937 [details]
Bug 1464019 - Turn off asrouter snippets when snippets.feed is is false

https://reviewboard.mozilla.org/r/247946/#review254086
Attachment #8981937 - Flags: review?(edilee) → review+
User Story: (updated)
Depends on: 1462736
Comment on attachment 8981937 [details]
Bug 1464019 - Turn off asrouter snippets when snippets.feed is is false

Approval Request Comment
[Feature/Bug causing the regression]: AS Router snippets cannot be turned off by users if they are added to the experiment
[User impact if declined]: Experiment won't be able to be run
[Is this code covered by automated tests?]: Yes
[Has the fix been verified in Nightly?]: Not yet, but it has been in AS github code base / run locally for two days
[Needs manual test from QE? If yes, steps to reproduce]: try turning off snippets from the about:preferences#home page while experiment pref (browser.newtabpage.activity-stream.asrouterExperimentEnabled) is on. snippets should disappear.
[List of other uplifts needed for the feature/fix]: none
[Is the change risky?]: no
[Why is the change risky/not risky?]: feature is behind a pref
[String changes made/needed]: none
Attachment #8981937 - Flags: approval-mozilla-beta?
Flags: qe-verify+
I have verified that the issue is no longer reproducible on the latest Nightly 62.0a1 (Build ID 20180603221731) on Windows 10 x64, Mac 10.13.4 and Ubuntu 16.04 x64. Snippets are no longer displayed on subsequent tabs after they are disabled when browser.newtabpage.activity-stream.asrouterExperimentEnabled is true.
Status: RESOLVED → VERIFIED
Flags: qe-verify+
Flags: qe-verify+
Target Milestone: --- → Firefox 62
Comment on attachment 8981937 [details]
Bug 1464019 - Turn off asrouter snippets when snippets.feed is is false

Verified fixed on Nightly, let's take this in today's 61.0b11 build as well.
Attachment #8981937 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
I have verified that the issue is no longer reproducible on the latest Firefox Beta 61.0b11 (Build ID 20180604143143) on Windows 10 x64, Mac 10.13.5 and Arch Linux x64.
Flags: qe-verify+
User Story: (updated)
Blocks: 1471375
No longer blocks: 1471375
Component: Activity Streams: Newtab → Messaging System
You need to log in before you can comment on or make changes to this bug.