1816905 - [Experiment] Branch B of the PDFjs Feature Callout experiment displays the wrong messages

Status: VERIFIED FIXED

(Firefox :: Messaging System, defect)

Description

[Maria Heres, inactive, contact Ecosystem QA]

Attachment: Screen Recording 2023-02-15 at 15.16.31.mov

[Affected Versions]:

• Firefox Release 110.0 (Build ID: 20230214051806)

[Affected Platforms]:

• Windows 10
• macOS 13.1
• Linux Mint 20.2

[Prerequisites]:

• You have set “nimbus.debug” to “true” from “about:config”.

[Steps to reproduce]:

1. Open Firefox.
2. Navigate to “about:studies?optin_slug=pdfjs-feature-callout&optin_branch=treatment-b&optin_collection=nimbus-preview” to enroll in the branch B of the “PDFjs Feature Callout” study.
3. Navigate to a page ending in “.pdf”.
4. Observe the PDF callout.

[Expected result]:

• The text of callout for branch B is:
  Slide 1: “Ditch the sketchy free online editor. Fill out forms, add comments, or take notes directly in Firefox.”
  Slide 2: “No more printing and scanning. Mark up documents, then save your changes.”

[Actual result]:

• The text of the callout in branch B is:
  Slide 1: “Fill out forms, add comments, or take notes directly in Firefox.”
  Slide 2: “Mark up PDFs, then save your changes.”

[Notes]:

• Attached is a recording of the issue.

Comment 1

[Meg Viar [:mviar]]

It looks like Treatment A is being show here instead of Treatment B.

Also note that based on the experiment specifications from product, the expected result differs from what's reported above. It should be:

Step 1

    Headline: Edit PDFs with our new text tool
    Body: Skip the search for free online editors. Fill out forms, add comments, or take notes directly in Firefox.
    CTA (Primary): Next

Step 2

    Headline: Sign documents with our new draw tool
    Body: No more printing and scanning. Mark up PDFs, then save your changes.

Comment 2

[Meg Viar [:mviar]]

The two branches in the experiment configuration are currently identical. Branch B needs to be updated to set the proper screen ID:
"pdfJsTourProgress": "{\"screen\":\"FEATURE_CALLOUT_1_B\",\"complete\":false}"

@jprickett can you update when that's ready?

Comment 3

[Jason Prickett]

The experiment has been updated, and branch B is now correctly using the proper screen ID of FEATURE_CALLOUT_1_B.

Comment 4

[Maria Heres, inactive, contact Ecosystem QA]

Based on the previous comment, I will mark this issue as fixed.

Comment 5

[Maria Heres, inactive, contact Ecosystem QA]

I have verified that the issue is no longer reproducible. The text displayed on Branch B now is the one in the first comment.
Tested using Firefox Release 110.0 (Build ID: 20230214051806) on Windows 10, macOS 13.1 and Linux Mint 20.2.