1550649 - Return-to-AMO about:welcome shows Trailhead cards and triggers experiments

Status: VERIFIED FIXED

(Firefox :: Messaging System, defect, P1)

Description

[Ed Lee :Mardak]

If trailhead state thinks it's something other than control, return-to-amo showing cards after the initial screen might pick up trailhead cards.

Comment 1

[Ed Lee :Mardak]

At latest, we'll need this for when we turn on experimentation, but better to get in to nightly 68 for earlier testing

Comment 2

[Ed Lee :Mardak]

Will track via uplift bug 1550861

Comment 3

[Ed Lee :Mardak]

STR:

1. Ensure trailhead.firstrun.branches is set to blank / no string value (to enable experimentation, and restart when changing)
2. Trigger usual return-to-amo (e.g., about:newtab#devtools -> Targeting -> Attribution Parameters -> Force Attribution)
3. Go to about:welcome (should see return to AMO)
4. Click "Get Started…" to see cards

Expected:

Cards show the usual RTAMO cards instead of Trailhead cards

Actual:

Trailhead cards shown (attachment 9065408 [details])

Comment 4

[Ed Lee :Mardak]

Attachment: trailhead cards shown incorrectly in RTAMO flow

Comment 5

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/activity-stream/pull/5036

Comment 6

[Ed Lee :Mardak]

https://hg.mozilla.org/mozilla-central/rev/c7a169c8670c

Comment 7

[Ed Lee :Mardak]

https://hg.mozilla.org/releases/mozilla-release/rev/df853c362664989e025d1206b59c37c435186a91

Comment 8

[Alexandra Moga [:LexaSV]]

I've verified this in Nightly 69.0a1 (latest) and in Beta 68.0b4 with Win10x64 and macOS High Sierra 10.13.6 by following the provided STRs.
While testing, I've run into the following issue: when I open about:welcome after having Forced attribution, the RTAMO overlay is not loaded and a blank page is displayed instead. I've attached a video that shows the issue.

Not sure if significant, but when I'm editing trailhead.firstrun.branches to "" I receive the following warning in the console:

LoginHelper: Couldn't parse origin for about:config 
Exception { name: "NS_ERROR_FAILURE", message: "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIURI.displayHostPort]", result: 2147500037, filename: "resource://gre/modules/LoginHelper.jsm", lineNumber: 219, columnNumber: 0, data: null

Another question: Comment 7 mentions that 67.0.5 should be fixed, however, If I follow the link to the corresponding build, I'm getting results for 67.0.1.

Comment 9

[Alexandra Moga [:LexaSV]]

Attachment: no-rtamo.gif

Comment 10

[Andrei Oprea [:andreio]]

I can confirm the issue. Loading about:welcome correctly triggers the RTMO message, on component mount it calls triggerOnboarding [0] which in turn causes a new type of overlay message to be set invalidating the RTMO overlay. Not sure yet what caused the regression but I'm looking into it for a fix.

The triggerOnboarding call is required to render the regular onboarding cards in the background (behind the RTAMO) message.

[0] https://github.com/mozilla/activity-stream/blob/3ef622799ccbb0e01c9cbbf3a0d05f1f4b37994b/content-src/asrouter/asrouter-content.jsx#L177

Comment 11

[Andrei Oprea [:andreio]]

Issue is caused by the updated targeting for the ONBOARDING_3 message (trailheadInterrupt targeting doesn't match). But if setupTrailhead is called you won't into the problem.

Conclusion: In order to test the different onboarding flows new profiles have to be created each time. Simply changing trailhead.firstrun.branches is not enough to reset the various configurations of the Trailhead experiment.

Comment 12

[Alexandra Moga [:LexaSV]]

I'm not sure I understand what should be the proper way to verify this fix. I have run two scenarios:

1. Conditions to trigger RTAMO flow

• create a new profile with the latest Nightly
• open about:config and set browser.newtabpage.activity-stream.asrouter.devtoolsEnabled to true
• go to about:newtab#devtools -> Targeting -> Attribution Parameters -> Force Attribution
• open about:welcome --> here I should see the RTAMO overlay, but I see the Trailhead card instead. The RTAMO overlay is displayed only after I'm restarting the profile
• click on Get started with Nightly in the RTAMO overlay--> the RTAMO onboarding card is displayed but it doesn't look like the original rtamo onboarding (see attachments rtamo onboarding cards.gif and original rtamo onboarding)

2. Same steps as above, but with trailhead.firstrun.branches set to ""

• triggeres the issue described in Comment 8

Please clarify which are the correct STRs for verifying this fix
Thanks

Comment 13

[Alexandra Moga [:LexaSV]]

Attachment: rtamo onboarding cards

Comment 14

[Alexandra Moga [:LexaSV]]

Attachment: original_rtamo_onboarding.png

Comment 15

[Ed Lee :Mardak]

Sorry, I meant to say set the pref trailhead.firstrun.branches value to be blank / empty string, so delete the whole entry or set the value to be nothing.

Comment 16

[Ed Lee :Mardak]

Attachment: empty pref value

Comment 17

[Alexandra Moga [:LexaSV]]

Verified fixed on latest Nightly and Beta 68 with Win10x64 and macOS High Sierra 10.13.6.
The issue I've mentioned in Comment 8 was happening because I was setting trailhead.firstrun.branches to "" , as mentioned in the STRs. In order for the fix to work the value for trailhead.firstrun.branches should be left empty.

I've attached a video showing that the correct rtamo cards are displayed.

Comment 18

[Alexandra Moga [:LexaSV]]

Attachment: correct rtamo cards.gif