1595911 - Launch new Moments Page about Sync

Status: VERIFIED FIXED

(Firefox :: Messaging System, task, P1)

Description

[Jim T]

This is a placeholder bug for launching the sync moments page described here:
https://github.com/mozilla/bedrock/issues/8148

Unlike previous pages, this moment will be for existing firefox users, especially those who haven't connected their mobile device yet.

Targeting Criteria (Draft)

• URL: TBD
• Profile Age: older than 1 month
• Localess: Tier1
• Current Date: between X and Y (campaign window for restarts)
• Expiration: 7 days (time until restart)

Comment 1

[Liz Henry (:lizzard) (relman/hg->git project)]

Tania, looks like this is the bug for the new moments page we were discussing yesterday.
Jim, do these pages need any basic QA, or verification once they launch?

Comment 2

[Eric Renaud [:erenaud]]

Jim - noting the priority shift and that the page will instead be for Sync/FxA to drive that MAU metric.

This work is being managed in https://github.com/mozilla/bedrock/issues/8148

Hoosteeno - could you please complete the criteria, to match the 'big bang' desired here, should it differ from Jim's original set above?

URL: https://www.mozilla.org/firefox/welcome/3/
Profile Age: ?
Locales: Tier1 (de, en-CA, en-GB, en-US, fr)
Current Date: ?
Expiration: ?

Comment 3

[Tania Maity (:tmaity)]

QA verified the details given here and we don't find anything actionable. We require more information. What's the timeline to complete testing for this?

Comment 4

[Justin Crawford [:hoosteeno] [:jcrawford]]

Hoosteeno - could you please complete the criteria, to match the 'big bang' desired here, should it differ from Jim's original set above?

I think Jim is doing that in comment 0. Jim, let me know if I can help.

Comment 5

[Jim T]

That's still a placeholder example for whoever is owning this launch. Celeste is going to provide final targeting.

Comment 6

[Tim Spurway [:tspurway]]

Celeste, we are developing a process around streamlining Moments Page requests. Could you fill out this form as best you can?

Off-train Moments Page Request

• When (Firefox release or Date) do you expect this to show up to users?:
• What is the Business Goal or Context of the request (brief summary or link)?:
• How will success be measured?:
• Who is the directly accountable/responsible individual making this Messaging Request?
• Please describe the targeting (i.e. which users should see this message) in plain English (e.g. all users who are not signed-in to a Firefox Account in the en-US locale, US region, in Firefox 72 and higher) :
• Enterprise clients included?:
• Link to content or specification of the desired message:
• User impact if declined:
• User risk associated with releasing this Message (High/Med/Low/None):
• Why is the Message risky/not risky? (and alternatives if risky):

Comment 7

[Eric Renaud [:erenaud]]

Flagging the date range specification as 11/21/19 - 11/28/19 for showing the sync content/CTA page (/firefox/welcome/3)

Comment 8

[Celeste]

(In reply to Jim T from comment #5)

That's still a placeholder example for whoever is owning this launch. Celeste is going to provide final targeting.

Thanks Jim, let's discuss final targeting params today.

Comment 9

[Celeste]

(In reply to Tim Spurway [:tspurway] from comment #6)

Celeste, we are developing a process around streamlining Moments Page requests. Could you fill out this form as best you can?

Off-train Moments Page Request

• When (Firefox release or Date) do you expect this to show up to users?: 11/21/19-11/28/19
• What is the Business Goal or Context of the request (brief summary or link)?: To increase FxA MAU by helping users onboard into the Sync service.
• How will success be measured?: Button view, click through, conversion.
• Who is the directly accountable/responsible individual making this Messaging Request? Lindsey Shepard
• Please describe the targeting (i.e. which users should see this message) in plain English (e.g. all users who are not signed-in to a Firefox Account in the en-US locale, US region, in Firefox 72 and higher) : TBD
• Enterprise clients included?: No
• Link to content or specification of the desired message: TBD
• User impact if declined: Will not benefit from Sync.
• User risk associated with releasing this Message (High/Med/Low/None): Low
• Why is the Message risky/not risky? (and alternatives if risky): There is no real "trade off" for not completing the CTA. Users who choose not to use Sync can continue their current browsing habits.

Comment 10

[Tim Spurway [:tspurway]]

Thanks, Celeste!

Let's try to clear up the TBDs above.

Eric pointed to https://www.mozilla.org/firefox/welcome/3/ which looks like it has been localized into (de, en-CA, en-GB, en-US, fr) at least. So the content looks good.

For targeting, I suppose we wouldn't want new profiles to 'collide' with multiple Moments pages, so maybe we should show this to folks with profiles that are > 30 days old?

I also seems reasonable to show this to folks who don't have a Firefox Account, since that is the segment of the population that we are trying to reach with the FxA MAU KPI and that's the audience the /welcome/3/ page appears to be addressing.

Celeste, does that sound reasoable? Can I get your sign-off on the above Content and Targeting criteria?

This Moments message can be launched in Firefox 70 Release. I can't promise a 11/21/19 (that's tomorrow), as we have to encode the message, get QA to verify it in staging and have Release Management sign off before we can proceed. I can get that ball rolling as soon as we have sign-off on Content and Targeting.

Comment 11

[Celeste]

Yes, the content above is approved (the TBD was just a placeholder while I was looking for the link).

It seems like there is still some alignment needed on targeting at the Michele/Shep level. I would suggest the following, but would like to get final approval from Michele/Shep.

--Profile Age (>30days)
--Date Range (11/21 to 11/28)
--Does not have Sync enabled

Comment 12

[Celeste]

Michele, can you speak to the above?

Comment 13

[Tim Spurway [:tspurway]]

n/a

Comment 14

[Tim Spurway [:tspurway]]

[Tracking Requested - why for this release]:The request is to get this new Moments page into Firefox 70 Release ASAP. Nan will build the message and get it into the staging Remote Settings and will work with QA to get it verified.

Comment 15

[Nan Jiang [:nanj]]

This message has been deployed to stage, we can proceed with QA now.

Notes for QA:

Prerequisites

• Sync CFR messages from Remote Settings stage. You can reference this doc to see how to use the stage environment
• Turn on AS Router devtools by flipping the pref browser.newtabpage.activity-stream.asrouter.devtoolsEnabled
• This message is targeting all channels of Firefox 70, so the test should be done in 70 release
• The targeting locale is "en-*", "fr", and "de"
• Another targeting rule is around the browser profile age, it requires the profile age is older than 30 days. You can modify that by updating the created key in {profileDir}/times.json
• It also targets users who do not have Sync enabled

Steps to test

• Open Firefox 70
• Open the ASRouter devtools page by clicking the wrench icon on the top right corner of the newtab page
• On the devtools admin page, there is a "Messages" section, open the dropdown list of Show messages form, and select "cfr"
• Scroll down until you find the WNP_MOMENTS_SYNC message, click on the Show button, then reboot the browser
• Verify that the browser opens the page as https://www.mozilla.org/%LOCALE%/firefox/welcome/3/
• Reboot the browser again, then verify that it opens "about:home" other than the page above

Comment 16

[Ciprian Muresan [:cmuresan], Formerly of Ecosystem QA - please needinfo @mcoman for things regarding the team]

Hey Nan, we can start working on verifying the issue, but it would help us a lot if you would file a PI request for us to know what our deadline is and to track the work better.
As far as I can tell there won’t be a need to uplift anything to Beta, can you confirm?

Comment 17

[Nan Jiang [:nanj]]

(In reply to Ciprian Muresan [:cmuresan], Experiments QA from comment #16)

Hey Nan, we can start working on verifying the issue, but it would help us a lot if you would file a PI request for us to know what our deadline is and to track the work better.

tspurway / jim, do we have a PI for this?

As far as I can tell there won’t be a need to uplift anything to Beta, can you confirm?

Correct, this doesn't involve any code changes in Firefox, just adds a new remote CFR message that targets the current user on 70.

Comment 18

[Ron Manning]

PI request created: https://jira.mozilla.com/browse/PI-388

Comment 19

[Celeste]

Hey all, just wanted to come in an confirm that we have alignment and approval on targeting and are good to go. LGTM!

Comment 20

[Tim Spurway [:tspurway]]

I reset the qe-verify flag to indicate that this message is now signed off and ready to verify based on steps in comment #15. Ciprian, please let me (or Nan) know if you have any questions.

Comment 21

[Marius Coman [:mcoman], Ecosystem QA, PTO until 20th of October, for any testing requests ping :poiegas]

Launch new Moments Page about Sync

Firefox Release 70

We have tested Bug 1595911- Launch new Moments Page about Sync.

QA’s recommendation: YELLOW - SHIP IT CONDITIONALLY

Reasoning:

• We have found two new issues (Bug 1598658 and Bug 1598653) during testing that only affect the “Moments” page and not the trigger functionality.

• Unfortunately, we were unable to test the natural trigger of the "Moments" page in the first part of the day. However, we have worked together with Nan in the second part of the day and have found a way to do this. We will continue testing the trigger next week and we will send a follow up testing report.

Testing Summary:

• Verified that the page is successfully triggered from the AS Router page.

• Verified that all the elements from the “Moments” page are working correctly.

Tested Platforms:

• Windows 10 x64

• Mac OS 10.15

• Ubuntu 18.04 x64

Tested Firefox versions:

• Firefox Release 70.0.1

Comment 22

[Tim Spurway [:tspurway]]

Thanks, Marius.

Celeste, does your team consider the issues QA filed (Bug 1598658 and Bug 1598653) to be blocking the release of this feature?

Comment 23

[Eric Renaud [:erenaud]]

Hi Tim,

Please note 1598658 has been closed, resolved as invalid.

Please also note that 1598653 has been closed, resolved as won't fix - the back button behavior is due to the FxA redirect is not considered a blocker for launch.

We're good to go here. Thank you.

Comment 24

[Nan Jiang [:nanj]]

hey liz, can we get your signoff as :erenaud has cleaned up the QA concerns in comment 23?

Comment 25

[Liz Henry (:lizzard) (relman/hg->git project)]

Sure, looks good to me since QA has signed off and the team is still going to be around this afternoon to check on this and follow up if needed. Signing off for Relman!

Comment 26

[Nan Jiang [:nanj]]

This message has now been published to production. It's alive.

Comment 27

[Liz Henry (:lizzard) (relman/hg->git project)]

This only launched for 70 :)

Comment 28

[Marius Coman [:mcoman], Ecosystem QA, PTO until 20th of October, for any testing requests ping :poiegas]

I have verified that the "Moments" page is correctly displayed on the targeted Firefox locales for profiles older than 30 days that are not connected to Sync. Verified with the latest Firefox Release (70.0.1) installed on Windows 10 x64, Mac 10.15.1, and Arch Linux.

Comment 29

[Marius Coman [:mcoman], Ecosystem QA, PTO until 20th of October, for any testing requests ping :poiegas]

Marking this as Verified - Fixed according to comment 28.

Comment 30

[Nan Jiang [:nanj]]

This message has been retired from the CFR lineup according to the schedule.

Comment 31

[Celeste]

Just closing this out since we are live.