1458906 - A "Sponsored content" card is wrongly displayed on the first new tab opened, after the "Sponsored Stories" option was unchecked

Status: VERIFIED FIXED

(Firefox :: New Tab Page, defect, P1)

Description

[Marius Coman [:mcoman], Ecosystem QA]

Attachment: rec of the issue.gif

[Affected versions]:
- Firefox Beta 60.0b16
- Nightly 61.0a1

[Affected Platforms]:
- All Windows
- All Mac
- All Linux

[Prerequisites]:
- Have a clean new Firefox profile.
- Have the set-up for the "Title 1" variant configured (https://goo.gl/cL4Ue3).

[Steps to reproduce]:
1. Start the browser with the profile from prerequisites.
2. Open two new tabs and click the "Customize your New Tab page" button from the top right part of the page.
3. Uncheck the "Sponsored Stories" option.
4. Open a new tab and observe the "Recommended by Pocket" area.

[Expected result]:
- Only the "Pocket" recommendations cards are displayed.

[Actual results]:
- A "Sponsored content" card is also displayed.

[Additional Notes]:
- The issue is reproducible for all the variants except "Control".
- This issue is no longer reproducible for the next new tabs opened.
- Attached a screen recording of the issue.

Comment 1

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/activity-stream/pull/4139

Comment 2

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Hi Julien, Tim pinged me about this one. It is a potential RC3 build driver/ride-along.

Comment 3

[[github robot]]

Commits pushed to master at https://github.com/mozilla/activity-stream

https://github.com/mozilla/activity-stream/commit/8af9c738088ecffe0e5aadd93335faf79b6ec8bd
Fix Bug 1458906 - broadcast disabling of spocs

https://github.com/mozilla/activity-stream/commit/1073648d20ee7086fae29bcc6e7d893dcea79122
Merge pull request #4139 from rlr/rm-spocs

Fix Bug 1458906 - broadcast disabling of spocs

Comment 4

[Ricky Rosario [:rrosario, :r1cky]]

Attachment: Bug 1458906 - A "Sponsored content" card is wrongly displayed on the first new tab opened, after the "Sponsored Stories" option was unchecked

Review commit: https://reviewboard.mozilla.org/r/241576/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/241576/

Comment 5

[Ricky Rosario [:rrosario, :r1cky]]

Comment on attachment 8973041 [details]
Bug 1458906 - A "Sponsored content" card is wrongly displayed on the first new tab opened, after the "Sponsored Stories" option was unchecked

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/241576/diff/1-2/

Comment 6

[Kate Hudson :k88hudson]

Comment on attachment 8973041 [details]
Bug 1458906 - A "Sponsored content" card is wrongly displayed on the first new tab opened, after the "Sponsored Stories" option was unchecked

https://reviewboard.mozilla.org/r/241576/#review247392

Thanks, looks good

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4750a81367f3
A "Sponsored content" card is wrongly displayed on the first new tab opened, after the "Sponsored Stories" option was unchecked r=k88hudson

Comment 8

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/4750a81367f3

Comment 9

[Tim Spurway [:tspurway]]

:cmarius, the decision has been made that this bug isn't a driver for RC3, but should ride along if an RC3 becomes necessary.  Could you take another look at the fix that landed in Nightly and verify?

Comment 10

[Marius Coman [:mcoman], Ecosystem QA]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0
Firefox: 61.0a1, Build ID 20180504100129

I have verified the fix for this issue and I can confirm that is no longer reproducible on Windows 10 x64, Mac 10.13 and Ubuntu 14.04 x64 with the latest Nightly build (61.0a1) installed. 
Now, the "Sponsored content" is no longer displayed after it was disabled from the "about:preferences" page. 

However, keep in mind that if there won't be an RC3 build for Beta 60, this issue will soon affect the Firefox Release population that will be enrolled in the "Pocket Spocs Optimization Test" shield study, since the Fx60 is planned to be launched on May 9th.

Comment 11

[Ed Lee :Mardak]

Comment on attachment 8973041 [details]
Bug 1458906 - A "Sponsored content" card is wrongly displayed on the first new tab opened, after the "Sponsored Stories" option was unchecked

Approval Request Comment
[Feature/Bug causing the regression]: Pocket sponsored content
[User impact if declined]: Sponsored content stays visible on the first new tab after turning it off
[Is this code covered by automated tests?]: Yes
[Has the fix been verified in Nightly?]: Yes comment 10
[Needs manual test from QE? If yes, steps to reproduce]: For a rc? probably good to. Steps in comment 0
[List of other uplifts needed for the feature/fix]: None
[Is the change risky?]: A little
[Why is the change risky/not risky?]: Brand new code to fix the bug, but it's similar to the user just normally turning the section off and on again.
[String changes made/needed]: None

Comment 12

[Peter deHaan [:pdehaan]]

Any chance we want to reconsider an uplift, given recent articles such as https://www.theverge.com/2018/5/7/17326184/firefox-ads-sponsored-content-pocket-suggestions

I understand that this may not affect a huge number of users, but displaying sponsored content to people who _just_ disabled it may make a few people confused/angry.

Comment 13

[Oana Botisan, Desktop QA]

I tried to reproduce the bug using an older version of Nightly 61.0a1 (2018-05-03) on Windows 10 x64 by following the steps from comment 0 and from https://goo.gl/cL4Ue3, but I didn't get any Sponsored Stories in the Recommended by Pocket section. 

In the browser console there were thrown two errors that could explain why this happens:
 [Show/hide message details.] TelemetryStopwatch: key "TELEMETRY_PENDING_LOAD_MS" was already initialized

 [Show/hide message details.] TelemetryStopwatch: requesting elapsed time for nonexisting stopwatch. Histogram: "TELEMETRY_PENDING_LOAD_MS", key: "null"

Can you please give me some information about how can I verify this or at least to force the Sponsored Stories to appear?

Comment 14

[Ricky Rosario [:rrosario, :r1cky]]

(In reply to Oana Botisan from comment #13)
> Can you please give me some information about how can I verify this or at
> least to force the Sponsored Stories to appear?

For manual testing:
1- Save this file locally: https://www.dropbox.com/s/03u5m579hzsvznh/spocs.json
2- Go to about:config, and edit "browser.newtabpage.activity-stream.feeds.section.topstories.options" so that "stories_endpoint" is pointing to your local file ("file:///..../spocs.json")
3- Make sure "browser.newtabpage.activity-stream.showSponsored" is set to true as well.

That should do it.

Comment 15

[Oana Botisan, Desktop QA]

I managed to reproduce the issue on an older version of Nightly (2018-05-03) using Windows 10 x64 and the steps from comment 14.
I retested everything using the latest Nightly 62.0a1 and beta 61.0b4 on Windows 10 x64, Ubuntu 16.04 x64 and macOS 10.13. The bug is not reproducing anymore.

Comment 16

[Julien Cristau [:jcristau]]

Comment on attachment 8973041 [details]
Bug 1458906 - A "Sponsored content" card is wrongly displayed on the first new tab opened, after the "Sponsored Stories" option was unchecked

Fix an issue when disabling sponsored content, approved for 60.0.1

Comment 17

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-release/rev/540ffec9584e

Comment 18

[Julien Cristau [:jcristau]]

Comment on attachment 8973041 [details]
Bug 1458906 - A "Sponsored content" card is wrongly displayed on the first new tab opened, after the "Sponsored Stories" option was unchecked

actually let's get this one on esr60 too, in the interest of keeping release and esr60 in sync this cycle.

Comment 19

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-esr60/rev/423e30150b9a

Comment 20

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Added it to Firefox and ESR 60.0.1 release notes.

Comment 21

[Oana Botisan, Desktop QA]

Verified the fix on Firefox 60.0.1 and ESR 60.0.1 using Windows 10 x64, Ubuntu 16.04 x64 and macOS 10.13. The bug is not reproducing anymore.