Each "CFR Recommendation" for the "Facebook" and "Reddit" website is wrongly counted twice in the AS Router page

VERIFIED FIXED in Firefox 63

Status

()

defect
P1
normal
VERIFIED FIXED
9 months ago
23 days ago

People

(Reporter: mcoman, Assigned: andreio)

Tracking

(Blocks 1 bug)

Trunk
Firefox 65
All
Windows
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox63+ verified, firefox64+ verified, firefox65 verified)

Details

Attachments

(3 attachments)

Posted image rec of the issue.gif
[Affected versions]:
- Firefox 63 Build ID - 20181018182531
- Firefox 64.0b3 Build ID - 20181022150107
- Firefox 65.0a1 Build ID - 20181023222913

[Affected Platforms]:
- All Windows

[Prerequisites]:
- Have a new Firefox profile.
- Have the following prefs in the "about:config" page;
	- "browser.newtabpage.activity-stream.asrouter.messageProviders" pref set to "[{"id":"cfr", "cohort": "three_per_day", "frequency": {"custom": [{"period": "daily", "cap": 1}]}, "type":"local","localProvider":"CFRMessageProvider","enabled":true}]";
	- "browser.newtabpage.activity-stream.asrouter.devtoolsEnabled" set to "true"

[Steps to reproduce]:
1. Open the browser with the profile from prerequisites.
2. Open a new tab and click the "Facebook" top site.
3. Repeat step two another four times and restart the browser.
4. Go to the "about:newtab#asrouter" page.
5. Scroll down to the "FACEBOOK_CONTAINER_3" section.
6. Observe the number of the displayed impressions.
 
[Expected result]:
- Only one impression was displayed.

[Actual result]:
- Two impressions were displayed.

[Notes]:
- This issue affects the "Experiment 2" branch, where the recommendation should be displayed three times per add-on.
- This issue is also reproducible with the "CFR Recommendation" for the "Reddit" website.
- This issue is not reproducible on Linux and Mac.
- Attached a screen recording of the issue.
Assignee: nobody → andrei.br92
Iteration: --- → 65.1 (Nov 2)
Priority: -- → P1
Commit pushed to master at https://github.com/mozilla/activity-stream

https://github.com/mozilla/activity-stream/commit/251958adac3380f792ab35adba2cdbddef4b2323
Fix Bug 1501656 - Prevent onLocationChange from being called twice because of redirect events (#4525)
Status: NEW → RESOLVED
Closed: 9 months ago
Resolution: --- → FIXED
Blocks: 1502461
I have verified that this issue is no longer reproducible with the latest Firefox Nightly (65.0a1 Build ID - 20181029100347) installed, on Windows 10 x64. Now, the "CFR Recommendation" for the "Facebook" and "Reddit" website is counted correctly in the AS Router page.
Status: RESOLVED → VERIFIED
Blocks: 1471328
Comment on attachment 9020903 [details]
Bug 1501656 - Prevent CFR impression pings being sent twice because of redirect events

[Beta/Release Uplift Approval Request]

Feature/Bug causing the regression: Bug 1501656

User impact if declined: Our data for understanding add-on recommendations to users will be inaccurate, so we might make recommendations that either negatively impacts or does not fully optimize engagement.

Is this code covered by automated tests?: Yes

Has the fix been verified in Nightly?: Yes

Needs manual test from QE?: Yes

If yes, steps to reproduce: See initial comment in this bug for STR

List of other uplifts needed: None

Risk to taking this patch: Low

Why is the change risky/not risky? (and alternatives if risky): Small change to impression counting code; this code is isolated to the CFR experiment (which is not turned on by default yet)

String changes made/needed: None
Attachment #9020903 - Attachment description: Bug 1501656 - Prevent CFR impression pings being sent twice because of redirect events → [Uplift patch] Bug 1501656 - Prevent CFR impression pings being sent twice because of redirect events
Attachment #9020903 - Flags: approval-mozilla-beta?
[Tracking Requested - why for this release]: Although this is not a driver for a point-release, we would like to request a ride-along if this uplift is approved for 64. We have a planned study for 63 that will be unable to run without these changes.
Tracking for 63 and 64, please also request an uplift to release if you want it evaluated for the next dot release, thanks.
Flags: needinfo?(khudson)
Comment on attachment 9020903 [details]
Bug 1501656 - Prevent CFR impression pings being sent twice because of redirect events

[Beta/Release Uplift Approval Request]

Feature/Bug causing the regression: Bug 1501656

User impact if declined: No reliable release data will be available for us to assess the impact of add-on recommendations before launch, we will not be able to make the most optimal decisions about which add-ons to recommend to users in order to increase retention.

Is this code covered by automated tests?: Yes

Has the fix been verified in Nightly?: Yes

Needs manual test from QE?: Yes

If yes, steps to reproduce: See STR in the bug description

List of other uplifts needed: None

Risk to taking this patch: Low

Why is the change risky/not risky? (and alternatives if risky): Small change to CFR-specific code

String changes made/needed: None
Flags: needinfo?(khudson)
Attachment #9020903 - Flags: approval-mozilla-release?
Comment on attachment 9020903 [details]
Bug 1501656 - Prevent CFR impression pings being sent twice because of redirect events

Uplift approved for 64 Beta and 63.0.1, thanks.
Attachment #9020903 - Flags: approval-mozilla-release?
Attachment #9020903 - Flags: approval-mozilla-release+
Attachment #9020903 - Flags: approval-mozilla-beta?
Attachment #9020903 - Flags: approval-mozilla-beta+
Flags: qe-verify+
I managed to reproduce the initial issue on 63.0 build2 (20181018182531) using Windows 10 x64. I also can confirm that the fix is properly applied on the 63.0.1 build4 (20181030165643) - the "CFR Recommendation" is counted correctly in the AS Router page for the affected websites.
Waiting for beta 6 in order to check the 64 build.
As a follow up for the previous comment, 64.0b6 build1 (20181101155334) is also verified fixed on Windows 10 x64. Setting the corresponding flags.
Flags: qe-verify+
Attachment #9020903 - Attachment description: [Uplift patch] Bug 1501656 - Prevent CFR impression pings being sent twice because of redirect events → Bug 1501656 - Prevent CFR impression pings being sent twice because of redirect events
Component: Activity Streams: Newtab → Messaging System
You need to log in before you can comment on or make changes to this bug.