1573930 - Concerns around pref structure for 69 Discovery Stream launch

Status: VERIFIED FIXED

(Firefox :: New Tab Page, enhancement, P1)

Description

[Scott [:thecount] Downe]

Right now the pref used for Discovery Stream is not usable via rollouts.

If we made a secondary pref that's a pure FF pref we could then use that for rollouts, and the main Discovery Stream pref would be on top of that for a more generic control of the feature.

We can also extend this to two other similar prefs from our mitigation work.

So via a ff pref we can also turn on the basic layout, and change the spocs url.

To be clear, right now both of these work from AS prefs, we need a second controlling pref from FF so rollouts and holdback studies work better.

Comment 1

[Scott [:thecount] Downe]

[Tracking Requested - why for this release]: Risk mitigation for turning on a feature, this would allow us to turn it off via a FF pref vs a AS pref, which is much much easier.

Comment 2

[Scott [:thecount] Downe]

The three FF prefs I'm looking to add, which are by default empty unless we turn it on, are:

browser.newtabpage.activity-stream.discoverystream.enabled
This allows us to turn on and off the feature without dealing with an AS pref. It's worth noting that there is ALSO an AS pref that needs to be on too, which is on by default in 69. These together should turn on the feature.

browser.newtabpage.activity-stream.discoverystream.hardcoded-basic-layout
This allows us to revert to a version of newtab that looks like what we currently have on 68. This is a mitigation that can be used without an AS pref. The idea here is if we for whatever reason cannot ship because we're not comfortable with the layout, we can still ship the functionality without changing the user experiene.

browser.newtabpage.activity-stream.discoverystream.spocs-endpoint
This allows us to revert to the old spocs endpoint if something goes wrong on the server. By default it's empty and uses the AS controlled value.

Think of these as FF prefs that can override the AS prefs, but without either of these FF prefs we revert to the AS prefs.

Comment 3

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/activity-stream/pull/5254

Comment 4

[Scott [:thecount] Downe]

Attachment: Uplift 1573930 - FF prefs to control DS

Comment 5

[Scott [:thecount] Downe]

Attachment: Bug 1573930 - FF prefs to control DS and mitigation strat

Comment 6

[Scott [:thecount] Downe]

Comment on attachment 9085640 [details]
Bug 1573930 - FF prefs to control DS and mitigation strat

Beta/Release Uplift Approval Request

• User impact if declined: These changes allow a rollout to a larger feature to mitigate risk.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: To test:

First pref:

1. Set browser.newtabpage.activity-stream.discoverystream.enabled to true
2. Open a new tab, and ensure new tab shows 7 rows of story cards.
3. Set it to false, and ensure we now see the old AS with only 1 row of story cards.

Next pref:

1. Set browser.newtabpage.activity-stream.discoverystream.hardcoded-basic-layout to true.
2. Set browser.newtabpage.activity-stream.discoverystream.enabled to true
3. Open a new tab, ensure you see DS enabled with 3 cards.
4. Set it back to false, ensure you see the old AS with 3 cards.
5. Attached screen shots for each version of 3 cards to help verify. (DS 3 card experience and AS 3 card experience). Notice the slight difference in the popular topics section under the cards.

DS 3 card experience:
https://user-images.githubusercontent.com/197334/63046574-be675a80-bea0-11e9-97ae-108fd9aaa071.png

AS 3 card experience:
https://user-images.githubusercontent.com/197334/63046788-287fff80-bea1-11e9-8173-c9dcbe8a824b.png

Third pref:

1. Ensure you can see sponsored content.
2. Set browser.newtabpage.activity-stream.discoverystream.enabled to true.
3. Set browser.newtabpage.activity-stream.discoverystream.spocs-endpoint to "https://getpocket.cdn.mozilla.net/v3/firefox/unique-spocs"
4. Set browser.newtabpage.activity-stream.asrouter.devtoolsEnabled to true
5. Go to "about:home#devtools-ds" and scroll down until you see "spocs_endpoint https://getpocket.cdn.mozilla.net/v3/firefox/unique-spocs"
6. Set browser.newtabpage.activity-stream.discoverystream.spocs-endpoint to "spocs_endpoint https://spocs.getpocket.com/spocs".
7. Go to "about:home#devtools-ds" and scroll down until you see "spocs_endpoint https://spocs.getpocket.com/spocs" and ensure you see sponsored content.

• List of other uplifts needed: None
• Risk to taking this patch: Medium
• Why is the change risky/not risky? (and alternatives if risky): It's a bit larger than I would usually see for low risk.

Has tests though, and is on the low end of complicated changes.

• String changes made/needed: none

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c5f74c51b12f
FF prefs to control DS and mitigation strat r=Mardak

Comment 8

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/c5f74c51b12f

Comment 9

[Ciprian Muresan [:cmuresan], Ecosystem QA]

I have verified that first pref and next pref instructions work as expected on Windows 10, macOS 10.14, and Arch Linux 4.14.3. However I've ran into a problem when trying to verify the third pref instructions. Regardless of what I set browser.newtabpage.activity-stream.discoverystream.spocs-endpoint to I still see spocs, but I don't see anything in the Browser Toolbox Network tab related to either https://getpocket.cdn.mozilla.net/v3/firefox/unique-spocs or https://spocs.getpocket.com/spocs.

@scott, is seeing spocs enough to verify this issue? Or do we want to see the requests being made in the Networking tab? If it's the latter, do you think you could help me out a bit?

Comment 10

[Ed Lee :Mardak]

https://hg.mozilla.org/mozilla-central/rev/cf52300e04c9

Comment 11

[Ciprian Muresan [:cmuresan], Ecosystem QA]

After talking with Scott on slack I've managed to confirm the third pref instructions and I can see a unique-spocs POST request when the pref is changed to https://getpocket.cdn.mozilla.net/v3/firefox/unique-spocs and a spocs POST request when changing it to https://spocs.getpocket.com/spocs on Windows 10, macOS 10.14, and Arch Linux 4.14.3.
Based on this and comment 9, I'm marking the issue as Verified-Fixed.

Comment 12

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9085640 [details]
Bug 1573930 - FF prefs to control DS and mitigation strat

Makes it easier for us to control the rollout of the New Tab Page work in Fx69. Approved for 69.0b15.

Comment 13

[Alexandru Michis [:malexandru]]

https://hg.mozilla.org/releases/mozilla-beta/rev/432eb4511c02

Comment 14

[Scott [:thecount] Downe]

Noticed the steps to test in the bug were not as clear as the ones in github, so I updated them just now.

Comment 15

[Marius Coman [:mcoman], Ecosystem QA]

I have verified this issue with the latest Firefox Beta (69.0b15 Build ID - 20190819184224) installed, on Windows 10 x64, Arch Linux and Mac 10.14.5, using the steps provided in comment 6.