1877160 - The Old version of the Review Checker Callout 2 is displayed after 5 days have passed and the Opted-in user reached a product details page

Status: VERIFIED FIXED

(Firefox :: Messaging System, defect)

Description

[Rares Doghi, Desktop QA]

Found in

• Nightly 124.0a1 (2024-01-29)

Affected versions

• Nightly 124.0a1 (2024-01-29)

Affected platforms

• All

Preconditions:
Set the browser.shopping.experience2023.enabled - TRUE
Set the browser.shopping.experience2023.optedIn - 1
Set nimbus.debug to true
Set messaging-system.rsexperimentloader.collection_id to nimbus-preview
Set browser.newtabpage.activity-stream.asrouter.devtoolsEnabled to true
security.ssl.enable_ocsp_stapling - False

Reach the following link in order to be enroled in the experiment:
about:studies?optin_slug=review-checker-callouts-v11-experiment&optin_branch=treatment-a&optin_collection=nimbus-preview

Steps to reproduce

1. Reach : https://www.amazon.com/XUEGW-Computer-Assembly-Required-Foldable/dp/B0CDS1238W/ref=sr_1_2_sspa?keywords=desk&qid=1706529514&sr=8-2-spons&sp_csd=d2lkZ2V0TmFtZT1zcF9hdGY&th=1
2. Close the Review Checker from the X Close button.
3. Click Got it at the 1st Callout panel.
4. Close Firefox and Advance system time 5 days.
5. Launch Firefox and Reach https://www.amazon.com/XUEGW-Computer-Assembly-Required-Foldable/dp/B0CDS1238W/ref=sr_1_2_sspa?keywords=desk&qid=1706529514&sr=8-2-spons&sp_csd=d2lkZ2V0TmFtZT1zcF9hdGY&th=1

Expected result

• The New Callout panel 2 should be displayed.

Actual result

• The Older version of the Callout 2 is displayed.
• Even after passing another 5 days the New Version of Callout 2 is not displayed at all.

Regression range
Not a Regression

Comment 1

[Jared Hirsch [:jhirsch] (he/him) (Needinfo please)]

Redirecting to Messaging System component, as this is a bug in an experiment recipe, not in the shopping code.

Comment 2

[Shane Hughes [:aminomancer]]

I'm not able to reproduce the first bullet:

The Older version of the Callout 2 is displayed.

That should be impossible, given that the experiment includes a dummy message that disables all the old messages.

Even after passing another 5 days the New Version of Callout 2 is not displayed at all.

I think this is because of a misconfiguration in the experiment, the use of screenImpressions which is not yet supported. I'll fix that. This is the same issue that caused bug 1877161, I think.

Comment 3

[Punam Dahiya [:pdahiya]]

Closing bug as per comment#2

Comment 5

[Rares Doghi, Desktop QA]

Verified as fixed.