1878656 - Callout 3 v1.1 is not displayed if the Amazon link is opened from the Recent History

Status: RESOLVED WORKSFORME

(Firefox :: Messaging System, defect, P1)

Description

[Alexandru Trif, Desktop Test Engineering [:atrif]]

Found in

• 122.0

Affected versions

• 124.0a1 (2024-01-26)
• 123.0b6
• 122.0

Tested platforms

• Affected platforms: Windows 11x64, macOS 14
• Unaffected platforms: none

Preconditions

• browser.shopping.experience2023.enabled, set to true (doubleclick)
• browser.shopping.experience2023.optedIn, set to 0 (doubleclick, input ‘0’, hit enter)
• nimbus.debug to true
• messaging-system.rsexperimentloader.collection_id to nimbus-preview
• browser.newtabpage.activity-stream.asrouter.devtoolsEnabled to true
• security.ssl.enable_ocsp_stapling - False

Reach the following link: about:studies?optin_slug=review-checker-callouts-v11-experiment&optin_branch=treatment-a&optin_collection=nimbus-preview

Steps to reproduce

1. Open a random Amazon product and close the review checker.
2. Close the callout and advance system time with 24 hours.
3. Open Firefox and an Amazon product from App Menu > History > Recent History.
4. Close and reopen Firefox and an Amazon product from App Menu > History > Recent History.

Expected result

• Callout 3 v1.1 is displayed.

Actual result

• No callout is displayed.

Regression range

• N/A

Additional notes

• Attached a screen recording.
• Opening a new tab with a product will display the callout as expected.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

:atrif, if you think that's a regression, could you try to find a regression range using for example mozregression?

Comment 2

[Shane Hughes [:aminomancer]]

If you set the optedIn pref to 1, then it's callout 2 you should be seeing, not callout 3. Anyway, I followed the steps, but callout 2 is showing as expected.

The steps also don't seem to make sense:

1. Open a random Amazon product and close the review checker.
2. Close the callout and advance system time with 24 hours.
3. Open Firefox and an Amazon product from App Menu > History > Recent History.
4. Close and reopen Firefox and an Amazon product from App Menu > History > Recent History.

Step 4 is redundant. Callout 2 should show on step 3. And if it does show on step 3 (which it is for me), then it can't also show on step 4, since it can only be shown once per 5 days.

Comment 3

[Alexandru Trif, Desktop Test Engineering [:atrif]]

Attachment: 1878656.gif

(In reply to Shane Hughes [:aminomancer] from comment #2)

If you set the optedIn pref to 1, then it's callout 2 you should be seeing, not callout 3. Anyway, I followed the steps, but callout 2 is showing as expected.

I apologize for the confusion. I mistakenly left 1 there instead of 0. The optedin pref needs to be set to 0 for the issue to be reproduced. I will edit the description as well.

The steps also don't seem to make sense:

1. Open a random Amazon product and close the review checker.
2. Close the callout and advance system time with 24 hours.
3. Open Firefox and an Amazon product from App Menu > History > Recent History.
4. Close and reopen Firefox and an Amazon product from App Menu > History > Recent History.

Step 4 is redundant. Callout 2 should show on step 3. And if it does show on step 3 (which it is for me), then it can't also show on step 4, since it can only be shown once per 5 days.

I have made a screen recording for an easier understanding. In step 3 for me, the review checker is opened again instead of the callout. Please let me know if there is additional information needed. Thank you!

Comment 4

[Shane Hughes [:aminomancer]]

(In reply to Alexandru Trif, Desktop QA [:atrif] from comment #3)

I have made a screen recording for an easier understanding. In step 3 for me, the review checker is opened again instead of the callout. Please let me know if there is additional information needed, please let me now.

Ah, this is an issue that Carmen (:cfat) asked me about. The issue you are encountering is that the callout only triggers if you visit a PDP while the review checker is closed, but it's not closed in this case because it auto-activates up to 2 times, once per session and once per 24 hours. So if you started a new session and 24 hours passed (as it did in your steps), then the review checker can auto-activate again, before the callout can trigger.

The way to avoid this issue is to let the sidebar auto-activate twice before you start testing. After it's auto-activated twice, the callouts can start showing as normal. It stops after 2 times. So your steps would look like this

1. enroll, go to a PDP, the sidebar should auto-open.
2. close the sidebar. callout 1 should show.
3. restart firefox and advance system clock by +24h.
4. repeat steps 1-3 until the sidebar no longer auto-opens.
5. at this point, visiting a PDP should show callout 2/3 (depending on opted-in state)

Comment 5

[Alexandru Trif, Desktop Test Engineering [:atrif]]

Thank you for the detailed information:
If I use the suggested steps above the issue is not reproduced:

1. Enroll, go to a PDP, and close the sidebar
2. Click to close the callout 1 and close Firefox
3. Advance system clock by 24h and open Firefox
4. Visit a PDP page (sidebar opens) and close the sidebar and Firefox
5. Advance the system within 24 hours and visit a PDP.
   AR: Callout is displayed.

I think I found the problem. If follow the next steps without advancing time 24 hours a second time and without closing the sidebar at step 4 by only closing/ reopening Firefox callout 2 is not displayed even if the sidebar is not auto-opened:

1. Enroll, go to a PDP, and close the sidebar
2. Click to close the callout 1 and close Firefox
3. Advance system clock by 24h and open Firefox
4. Visit a PDP page (sidebar opens) and close Firefox (leave the sidebar open)
5. Reopen Firefox and visit a PDP page.
   AR: No callout is displayed on the first visit. Visiting a PDP in another tab will display the callout.

Feel free to close this as invalid if this is the intended behavior. The Callout is displayed only when opening a second tab with a PDP page while the review checker is not auto-opened. Thank you!

Comment 6

[Shane Hughes [:aminomancer]]

Thanks! Yes, this is the intended behavior. Sorry if that's not clear in the Figma spec. I think the designers were not aware that we were continuing the old "2 auto-opens" behavior from Review Checker 1.0, so they didn't factor that into the Figma spec.