Closed Bug 1550649 Opened 7 months ago Closed 6 months ago

Return-to-AMO about:welcome shows Trailhead cards and triggers experiments

Categories

(Firefox :: Messaging System, defect, P1)

defect

Tracking

()

VERIFIED FIXED
Firefox 68
Iteration:
69.1 - May 13 - 26
Tracking Status
firefox67 --- wontfix
firefox67.0.1 --- fixed
firefox68 --- fixed

People

(Reporter: Mardak, Assigned: k88hudson)

References

Details

(Keywords: github-merged)

Attachments

(5 files, 2 obsolete files)

If trailhead state thinks it's something other than control, return-to-amo showing cards after the initial screen might pick up trailhead cards.

At latest, we'll need this for when we turn on experimentation, but better to get in to nightly 68 for earlier testing

Assignee: nobody → khudson
Iteration: --- → 68.4 - Apr 29 - May 12
Blocks: 1550861
Priority: -- → P1

Will track via uplift bug 1550861

STR:

  1. Ensure trailhead.firstrun.branches is set to blank / no string value (to enable experimentation, and restart when changing)
  2. Trigger usual return-to-amo (e.g., about:newtab#devtools -> Targeting -> Attribution Parameters -> Force Attribution)
  3. Go to about:welcome (should see return to AMO)
  4. Click "Get Started…" to see cards

Expected:

Cards show the usual RTAMO cards instead of Trailhead cards

Actual:

Trailhead cards shown (attachment 9065408 [details])

Type: task → defect
Summary: Ensure return-to-AMO about:welcome doesn't trigger experimentation → Return-to-AMO about:welcome shows Trailhead cards and triggers experiments
Blocks: 1552280
Blocks: 1552366
Iteration: 68.4 - Apr 29 - May 12 → 69.1 - May 13 - 26
Keywords: github-merged
Status: NEW → RESOLVED
Closed: 6 months ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 68

I've verified this in Nightly 69.0a1 (latest) and in Beta 68.0b4 with Win10x64 and macOS High Sierra 10.13.6 by following the provided STRs.
While testing, I've run into the following issue: when I open about:welcome after having Forced attribution, the RTAMO overlay is not loaded and a blank page is displayed instead. I've attached a video that shows the issue.

Not sure if significant, but when I'm editing trailhead.firstrun.branches to "" I receive the following warning in the console:

LoginHelper: Couldn't parse origin for about:config 
Exception { name: "NS_ERROR_FAILURE", message: "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIURI.displayHostPort]", result: 2147500037, filename: "resource://gre/modules/LoginHelper.jsm", lineNumber: 219, columnNumber: 0, data: null

Another question: Comment 7 mentions that 67.0.5 should be fixed, however, If I follow the link to the corresponding build, I'm getting results for 67.0.1.

Flags: needinfo?(khudson)
Attached image no-rtamo.gif

I can confirm the issue. Loading about:welcome correctly triggers the RTMO message, on component mount it calls triggerOnboarding [0] which in turn causes a new type of overlay message to be set invalidating the RTMO overlay. Not sure yet what caused the regression but I'm looking into it for a fix.

The triggerOnboarding call is required to render the regular onboarding cards in the background (behind the RTAMO) message.

[0] https://github.com/mozilla/activity-stream/blob/3ef622799ccbb0e01c9cbbf3a0d05f1f4b37994b/content-src/asrouter/asrouter-content.jsx#L177

Issue is caused by the updated targeting for the ONBOARDING_3 message (trailheadInterrupt targeting doesn't match). But if setupTrailhead is called you won't into the problem.

Conclusion: In order to test the different onboarding flows new profiles have to be created each time. Simply changing trailhead.firstrun.branches is not enough to reset the various configurations of the Trailhead experiment.

I'm not sure I understand what should be the proper way to verify this fix. I have run two scenarios:

  1. Conditions to trigger RTAMO flow
  • create a new profile with the latest Nightly
  • open about:config and set browser.newtabpage.activity-stream.asrouter.devtoolsEnabled to true
  • go to about:newtab#devtools -> Targeting -> Attribution Parameters -> Force Attribution
  • open about:welcome --> here I should see the RTAMO overlay, but I see the Trailhead card instead. The RTAMO overlay is displayed only after I'm restarting the profile
  • click on Get started with Nightly in the RTAMO overlay--> the RTAMO onboarding card is displayed but it doesn't look like the original rtamo onboarding (see attachments rtamo onboarding cards.gif and original rtamo onboarding)
  1. Same steps as above, but with trailhead.firstrun.branches set to ""

Please clarify which are the correct STRs for verifying this fix
Thanks

Flags: needinfo?(andrei.br92)
Attached image rtamo onboarding cards (obsolete) —
Attached image original_rtamo_onboarding.png (obsolete) —

Sorry, I meant to say set the pref trailhead.firstrun.branches value to be blank / empty string, so delete the whole entry or set the value to be nothing.

Attached image empty pref value

Verified fixed on latest Nightly and Beta 68 with Win10x64 and macOS High Sierra 10.13.6.
The issue I've mentioned in Comment 8 was happening because I was setting trailhead.firstrun.branches to "" , as mentioned in the STRs. In order for the fix to work the value for trailhead.firstrun.branches should be left empty.

I've attached a video showing that the correct rtamo cards are displayed.

Status: RESOLVED → VERIFIED
Flags: needinfo?(khudson)
Flags: needinfo?(andrei.br92)
Attached image correct rtamo cards.gif
Attachment #9067961 - Attachment is obsolete: true
Attachment #9067966 - Attachment is obsolete: true
Component: Activity Streams: Newtab → Messaging System
You need to log in before you can comment on or make changes to this bug.