1492431 - The "CFR Recommendation", for the "Amazon" website, is wrongly displayed on "Experiment 1" and "Experiment 2" branches

Status: VERIFIED FIXED

(Firefox :: Messaging System, defect, P2)

Description

[Marius Coman [:mcoman], Ecosystem QA]

Attachment: rec of the issue.gif

[Affected versions]:
- Firefox 63 and above

[Affected Platforms]:
- All Windows
- All Linux
- All Mac

[Prerequisites]:
- Have a Firefox profile with the following prefs in the "about:config" page:
	- "browser.newtabpage.activity-stream.asrouterExperimentEnabled" with "true" value;
	- "browser.newtabpage.activity-stream.asrouter.messageProviders" with "[{"id":"cfr", "cohort": "one_per_day", "frequency": {"custom": [{"period": "daily", "cap": 1}]}, "type":"local","localProvider":"CFRMessageProvider","enabled":true}]" value.
	- "browser.search.region" set to "US";
	- "devtools.chrome.enabled" set to true.
- Have the "Browser Console" opened.

[Steps to reproduce]:
1. Open the browser with the profile from prerequisites.
2. Open a new tab and navigate to "www.amazon.com".
3. Focus the "Browser Console" and enter the following string: "NewTabUtils.activityStreamProvider.getTopFrecentSites({numItems: 25}).then(r => console.log(r.map(s=>`${s.url} [${s.frecency}]`).join('\n')))".
4. Repeat step 2 until the Amazon "frecency" score, from the browser console, is at least 10000.
5. Restart the browser and navigate to "www.amazon.com" in a new tab again.
6. Observe the right part of the "Address Bar".

[Expected result]:
- Only the "Page actions", "Save to Pocket" and "Bookmark this page" buttons are displayed.

[Actual result]:
- The "CFR Recommendation" button is also displayed.

[Notes]:
- Attached a screen recording of the issue.

Comment 1

[Kate Hudson :k88hudson]

I do not see this as of September 22 nightly (this may have been fixed by https://bugzilla.mozilla.org/show_bug.cgi?id=1492174), can you verify if this is still an issue?

Comment 2

[Tim Spurway [:tspurway]]

Marius, could you take another crack at reproducing this?

Comment 3

[Marius Coman [:mcoman], Ecosystem QA]

I have verified that this issue is no longer reproducible on Windows 10 x64, Arch Linux and Mac 10.13.3 with the latest "Firefox Nightly" (64.0a1 - Build ID 20180924100354) installed.

Also, using the "Mozregression" tool I have managed to find the regression window:

First good revision: fba337077a687723353e3b5bdae58a5f1996d516
Last bad revision: e7665879a630451639a42a2d75383b24928b90a2
Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=e7665879a630451639a42a2d75383b24928b90a2&tochange=fba337077a687723353e3b5bdae58a5f1996d516+

From the pushlog it seems that this issue was fixed in Bug 1492174.

Considering the above I am closing this issue as Resolved Fixed.

Comment 4

[Marius Coman [:mcoman], Ecosystem QA]

I have verified that this issue is no longer reproducible on Windows 10 x64, Arch Linux and Mac 10.13.3 with the latest "Firefox Beta" (63.0b9 - Build ID 20180924202103) installed. Now the "CFR Recommendation", for the "Amazon" website is no longer displayed on "Experiment 1" and "Experiment 2" branches.

Comment 5

[Ed Lee :Mardak]

https://hg.mozilla.org/mozilla-central/rev/95f8e6265f24