1489619 - Loading spocs from cache doesn't properly reset campaign impression map

Status: VERIFIED FIXED

(Firefox :: New Tab Page, defect, P1)

Description

[Scott [:thecount] Downe]

To test:

1. Set browser.newtabpage.activity-stream.feeds.section.topstories.options to 
```
{"api_key_pref":"extensions.pocket.oAuthConsumerKey","hidden":false,"provider_icon":"pocket","provider_name":"Pocket","read_more_endpoint":"https://getpocket.com/explore/trending?src=fx_new_tab","stories_endpoint":"https://getpocket.com/v3/firefox/global-recs?version=3&consumer_key=$apiKey&locale_lang=en-US&feed_variant=default_test_20180731","stories_referrer":"https://getpocket.com/recommendations","topics_endpoint":"https://getpocket.cdn.mozilla.net/v3/firefox/trending-topics?version=2&consumer_key=$apiKey&locale_lang=en-US","show_spocs":true,"personalized":true}
```

2. Load the about:home page
3. restart the browser (this restart should have spocs loaded via cache)
4. load about:home page more than three times.
5. on the third time, the third card should not be a sponsored card.

Comment 1

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/activity-stream/pull/4408

Comment 2

[[github robot]]

Commit pushed to master at https://github.com/mozilla/activity-stream

https://github.com/mozilla/activity-stream/commit/ed31e9bd640c805214d12403fa1f1acbb01c7082
Fix bug 1489619 - Loading spocs from cache doesn't properly reset campaign impression map

Comment 3

[Ed Lee :Mardak]

https://hg.mozilla.org/mozilla-central/rev/95f8e6265f24

Comment 4

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/activity-stream/pull/4476

Comment 5

[Scott [:thecount] Downe]

Attachment: bug 1489619 - Loading spocs from cache doesn't properly reset campaign impression map

Comment 6

[Scott [:thecount] Downe]

Comment on attachment 9014993 [details]
bug 1489619 - Loading spocs from cache doesn't properly reset campaign impression map

[Beta/Release Uplift Approval Request]

Feature/Bug causing the regression: Bug 1485528

User impact if declined: Sponsored content might not be displayed when it could have been, which hurts revenue.

Is this code covered by automated tests?: Yes

Has the fix been verified in Nightly?: No

Needs manual test from QE?: Yes

If yes, steps to reproduce: 1. Set browser.newtabpage.activity-stream.feeds.section.topstories.options to 
```
{"api_key_pref":"extensions.pocket.oAuthConsumerKey","hidden":false,"provider_icon":"pocket","provider_name":"Pocket","read_more_endpoint":"https://getpocket.com/explore/trending?src=fx_new_tab","stories_endpoint":"https://getpocket.com/v3/firefox/global-recs?version=3&consumer_key=$apiKey&locale_lang=en-US&feed_variant=default_test_20180731","stories_referrer":"https://getpocket.com/recommendations","topics_endpoint":"https://getpocket.cdn.mozilla.net/v3/firefox/trending-topics?version=2&consumer_key=$apiKey&locale_lang=en-US","show_spocs":true,"personalized":true}
```

2. Load the about:home page
3. restart the browser (this restart should have spocs loaded via cache)
4. load about:home page more than three times.
5. on the third time, the third card should not be a sponsored card.

List of other uplifts needed: None

Risk to taking this patch: Low

Why is the change risky/not risky? (and alternatives if risky): Tests, small changes, already on nightly (not verified yet though), and the changes are limited to one file.

String changes made/needed: None

Comment 7

[Pascal Chevrel:pascalc (PTO until April 26)]

Comment on attachment 9014993 [details]
bug 1489619 - Loading spocs from cache doesn't properly reset campaign impression map

Bug potentially affecting revenue, fix landed on nightly 2 weeks ago without reported regressions, uplift approved for 63 beta 13, thanks.

Comment 8

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/releases/mozilla-beta/rev/2d8c01b9554c

Comment 9

[Daniel Bodea [:danibodea]]

I have attempted reproducing this issue, but I was unable. 

Can you please give some more details on the reproduction process? These are my questions:

1. Considering step 3: "restart the browser (this restart should have spocs loaded via cache)"; 
How exactly should I perform the restart in order to comply with the condition in the brackets?

2. Considering step 4: "load about:home page more than three times."
This is a bit confusing for me because if I open a new tab, then the "about:home" page is already loaded. Do I simply reload it by inputting "about:home" in the address bar an tapping Enter three times in a row?

3. Considering step 5: "on the third time, the third card should not be a sponsored card."
3.1 What is a sponsored card? How do I recognize a sponsored card?
3.2 The expected result is that the 3rd card would not be a "sponsored card", right?
3.3 The actual result (and the issue) is that the 3rd card is a "sponsored card", right? 

4. Are there other details that I should know before I retry to reproduce/verify it?

I am sorry for the low technical knowledge in this area and I ask for tolerance. Thank you! :)

Comment 10

[Scott [:thecount] Downe]

OK, I fixed the steps to test, looks like they were a bit out of date and also not optimal.

1. Set browser.newtabpage.activity-stream.feeds.section.topstories.options to

{"api_key_pref":"extensions.pocket.oAuthConsumerKey","hidden":false,"provider_icon":"pocket","provider_name":"Pocket","read_more_endpoint":"https://getpocket.com/explore/trending?src=fx_new_tab","stories_endpoint":"https://gist.githubusercontent.com/ScottDowne/164995d9535b4203846048bdee29d169/raw/d3bc36c1e2088ac6401139c1f098002f271bfe91/spoc.json","stories_referrer":"https://getpocket.com/recommendations","topics_endpoint":"https://getpocket.cdn.mozilla.net/v3/firefox/trending-topics?version=2&consumer_key=$apiKey&locale_lang=en-US","show_spocs":true,"personalized":true}

2. Load the about:home page, third card should be a link to THERINGER.COM and have this image https://img-getpocket.cdn.mozilla.net/direct?url=https%3A%2F%2Fs3.amazonaws.com%2Fpocket-hits%2F2018_SPOCs%2Ftheringer.png&resize=w618-h310

3. That about:home tab open you just did should have filled cache so the next time you open the browser, it should load the data from cache. This is where the bug is.

4. restart the browser

5. You should see the same card on the second load.

6. Refresh roughly 3 times.

Expected: On nightly, or where this bug is fixed, you should stop seeing THERINGER.COM after three or so refreshes.
Bug: On beta (currently) or where the bug isn't fixed, you'll keep seeing THERINGER.COM if you refresh more than three times (it'll display at 100 refreshes).

Hope that's better, let me know if that doesn't work for you.

Comment 11

[Scott [:thecount] Downe]

You can also confirm this by looking at the pref "browser.newtabpage.activity-stream.feeds.section.topstories.spoc.impressions"

If you see that as something like this: {"439":[1539099902637,1539099903220,1539099903709]} it means that third spoc has been displayed 3 times, and should stop on next refresh. If you see this: {"439":[1539099902637]} it means that spoc has been displayed once, and you should see it two more times. If you see something like this: {"439":[1539099902637,1539099903220,1539099903709,1539099902637,1539099903220,1539099903709]} it means the spoc has been displayed 6 times and it's a bug, should stop at 3. The actual numbers after "439" can be anything, I believe. So you won't see the same numbers as me but the "439" should always be the same. :)

Comment 12

[Marius Coman [:mcoman], Ecosystem QA]

I have verified that this issue is no longer reproducible with the latest versions of Firefox Beta (63.0b13 Build ID - 20181008155858) and "Firefox Nightly" (64.0a1 - Build ID 20181010100123) installed, on Windows 10 x64, Arch Linux and Mac 10.13.3. Now, the sponsored content card is no longer displayed after 3 refreshes.