1464246 - Pref Flip startup.homepage_welcome_url to about:welcome

Status: RESOLVED FIXED

(Firefox :: General, enhancement)

Description

[Erica Wright [:ewright]]

Change the pref to redirect the firstrun page to the in-product experience.

Comment 1

[Erica Wright [:ewright]]

Attachment: Bug 1464246 - change the pref which will redirect the welcome page from a mozilla.org page to about:welcome.

Review commit: https://reviewboard.mozilla.org/r/246606/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/246606/

Comment 2

[Erica Wright [:ewright]]

note, to test on local nightly it is unbranded so you need to change the same pref in:
`browser/branding/unofficial/pref/firefox-branding.js` 
to `about:welcome` and run the browser with a new profile.

Comment 3

[Kate Hudson :k88hudson]

Looks good, I'm just wondering if we shouldn't block this ticket on localizing the onboarding experience?

Comment 4

[Erica Wright [:ewright]]

Good idea.

Comment 5

[Kate Hudson :k88hudson]

Comment on attachment 8980434 [details]
Bug 1464246 - change the pref which will redirect the welcome page from a mozilla.org page to about:welcome.

https://reviewboard.mozilla.org/r/246606/#review252932

::: browser/branding/aurora/pref/firefox-branding.js:7
(Diff revision 1)
>  
>  /* This Source Code Form is subject to the terms of the Mozilla Public
>   * License, v. 2.0. If a copy of the MPL was not distributed with this
>   * file, You can obtain one at http://mozilla.org/MPL/2.0/. */
>  
>  pref("startup.homepage_override_url", "https://www.mozilla.org/%LOCALE%/firefox/%VERSION%a2/whatsnew/");

Do you still want to keep the override_url if this is going away?

Comment 6

[Erica Wright [:ewright]]

Comment on attachment 8980434 [details]
Bug 1464246 - change the pref which will redirect the welcome page from a mozilla.org page to about:welcome.

https://reviewboard.mozilla.org/r/246606/#review252932

> Do you still want to keep the override_url if this is going away?

Yep, that's the url it links to on upgrade, ex: https://www.mozilla.org/en-US/firefox/61.0a1/whatsnew/. I haven't heard any discussion to remove that page.

Comment 7

[Pascal Chevrel:pascalc (PTO until April 26)]

Hi Kate, which Firefox version and channels are you targeting? 
if I am reading your patch correctly, you are changing the firstrun page for Dev Edition only.

Comment 8

[Pascal Chevrel:pascalc (PTO until April 26)]

(In reply to Pascal Chevrel:pascalc from comment #7)
> if I am reading your patch correctly, you are changing the firstrun page for
> Dev Edition only.

Forget this part of my comment, I didn't click on the full patch which modifies all occurrences of firefox-branding.js
  
I am still interested in knowing if we should track this for 62 though :)

Comment 9

[Kate Hudson :k88hudson]

The target for this is 62, yes

Comment 10

[Kate Hudson :k88hudson]

Comment on attachment 8980434 [details]
Bug 1464246 - change the pref which will redirect the welcome page from a mozilla.org page to about:welcome.

https://reviewboard.mozilla.org/r/246606/#review253684

::: browser/branding/aurora/pref/firefox-branding.js:7
(Diff revision 1)
>  
>  /* This Source Code Form is subject to the terms of the Mozilla Public
>   * License, v. 2.0. If a copy of the MPL was not distributed with this
>   * file, You can obtain one at http://mozilla.org/MPL/2.0/. */
>  
>  pref("startup.homepage_override_url", "https://www.mozilla.org/%LOCALE%/firefox/%VERSION%a2/whatsnew/");

Ah, ok thanks!

Comment 11

[Kate Hudson :k88hudson]

Can you re-ping me when this is unblocked/you're ready to merge?

Comment 12

[Kate Hudson :k88hudson]

Comment on attachment 8980434 [details]
Bug 1464246 - change the pref which will redirect the welcome page from a mozilla.org page to about:welcome.

https://reviewboard.mozilla.org/r/246606/#review254042

Comment 13

[Chris More [:cmore]]

Just want to double check that this pref flip is going to be tested in Shield or a funnelcake before we release the change to the welcome pref. I want to make sure we can quantify this change even if it is neutral for now.

Comment 14

[:Harald Kirschner :digitarald]

DevEd has a special first-run page with a lot more material. We'd need extra scrutiny to change that. Was that considered?

Comment 15

[Erica Wright [:ewright]]

(In reply to :Harald Kirschner :digitarald from comment #14)
> DevEd has a special first-run page with a lot more material. We'd need extra
> scrutiny to change that. Was that considered?

Thanks for pointing that out, Nightly also has a different page. I'll consult some folks about that.

Comment 16

[Liz Henry (:lizzard) (relman/hg->git project)]

Just to add some examples Pascal provided,

https://www.mozilla.org/en-US/firefox/nightly/firstrun/ should show the current nightly first run example
https://www.mozilla.org/en-US/firefox/61.0a2/firstrun/ is the 61 dev edition first run page.

Both of them look good, are up to date, and have nice, relevant content for users that invite community participation, so I'd hate to lose them.

Comment 17

[Liz Henry (:lizzard) (relman/hg->git project)]

Jeff, what do you think about continuing to support different welcome pages per branch?

Comment 18

[Erica Wright [:ewright]]

(In reply to Liz Henry (:lizzard) (needinfo? me) from comment #16)
> Just to add some examples Pascal provided,
> 
> https://www.mozilla.org/en-US/firefox/nightly/firstrun/ should show the
> current nightly first run example
> https://www.mozilla.org/en-US/firefox/61.0a2/firstrun/ is the 61 dev edition
> first run page.
> 
> Both of them look good, are up to date, and have nice, relevant content for
> users that invite community participation, so I'd hate to lose them.

Thanks Liz, the patch hasn't been updated yet, but for this first iteration it will only be affecting beta and release. When it does affect the other pages the about:welcome will be updated to have that relevant info.

Comment 19

[Jeff Griffiths (:canuckistani) (:⚡︎)]

(In reply to Liz Henry (:lizzard) (needinfo? me) from comment #17)
> Jeff, what do you think about continuing to support different welcome pages
> per branch?

This might complicate things:

1. I'd generally support in-tgree maintenance of the welcome page
2. I'd really like for there to be a distinct experience for Dev Edition
3. I think the dev experience team should think a bit on what we think an in-product welcome experience might look like but I bet it could be really cool. In the meantime I don't want dev edition to change to the default about:welcome.

Comment 20

[Tim Spurway [:tspurway]]

(In reply to Chris More [:cmore] from comment #13)
> Just want to double check that this pref flip is going to be tested in
> Shield or a funnelcake before we release the change to the welcome pref. I
> want to make sure we can quantify this change even if it is neutral for now.

There is a significant difference in the UX for about:welcome and https://www.mozilla.org/en-US/firefox/61.0/firstrun/.  about:welcome doesn't use an IFrame and opens FxA sign-in in a new tab, whereas mozilla.org opens it within it's IFrame on the same tab.   This means that if we run a Shield study, we will be testing both the alternative design and the fact that it's running in-browser rather than remotely.  If there were differences in tracked metrics, would we be sure of which change was the cause?  Is it a valid experiment or do we have too many differences?  

The metrics for the two cohorts are also tracked in different systems (GA vs. Fx telemetry).  Will this difference cast doubt on the actual results of the experiment?

Comment 21

[Chris More [:cmore]]

(In reply to Tim Spurway [:tspurway] from comment #20)
> (In reply to Chris More [:cmore] from comment #13)
> > Just want to double check that this pref flip is going to be tested in
> > Shield or a funnelcake before we release the change to the welcome pref. I
> > want to make sure we can quantify this change even if it is neutral for now.
> 
> There is a significant difference in the UX for about:welcome and
> https://www.mozilla.org/en-US/firefox/61.0/firstrun/.  about:welcome doesn't
> use an IFrame and opens FxA sign-in in a new tab, whereas mozilla.org opens
> it within it's IFrame on the same tab.   This means that if we run a Shield
> study, we will be testing both the alternative design and the fact that it's
> running in-browser rather than remotely.  If there were differences in
> tracked metrics, would we be sure of which change was the cause?  Is it a
> valid experiment or do we have too many differences?  
> 
> The metrics for the two cohorts are also tracked in different systems (GA
> vs. Fx telemetry).  Will this difference cast doubt on the actual results of
> the experiment?

Hi Tim.

Bug 1446023 is to remove the iframe on /firstrun/, so they should be again very similar once that is rolled out. 

Though, from talking to others, I am fine with not a/b testing about:welcome vs /firstrun/ (even if they were identical) because we know that about:welcome is just going to be the starting point of in-product onboarding going forward and a new baseline. Even if the results come back neutral, we would still launch about:welcome because it will be the baseline for more experiments in the future.

So the tl;dr, I am fine now with launching about:welcome with Fx62 release pending any other improvements/QA/regressions that are found (or not) between now and the merge date.

Comment 22

[Tim Spurway [:tspurway]]

Thanks, Chris.  That should simplify things considerably, as I was a little worried about first run overlap with some of the planned Onboarding experiments.

So to recap, the plan is to pref about:welcome on in Beta and Release, but to leave Nightly and Dev Edition as-is (pointing to their respecive mozilla.org content).  I will open separate bugs to switch Nightly and Dev Edition welcome pages to in-product experiences.

Comment 23

[Kate Hudson :k88hudson]

Comment on attachment 8980434 [details]
Bug 1464246 - change the pref which will redirect the welcome page from a mozilla.org page to about:welcome.

https://reviewboard.mozilla.org/r/246606/#review255586

::: browser/branding/aurora/pref/firefox-branding.js:8
(Diff revision 1)
>  /* This Source Code Form is subject to the terms of the Mozilla Public
>   * License, v. 2.0. If a copy of the MPL was not distributed with this
>   * file, You can obtain one at http://mozilla.org/MPL/2.0/. */
>  
>  pref("startup.homepage_override_url", "https://www.mozilla.org/%LOCALE%/firefox/%VERSION%a2/whatsnew/");
> -pref("startup.homepage_welcome_url", "https://www.mozilla.org/%LOCALE%/firefox/%VERSION%a2/firstrun/");
> +pref("startup.homepage_welcome_url", "about:welcome");

This needs to be changed back to the original URL for dev edition

::: browser/branding/nightly/pref/firefox-branding.js:6
(Diff revision 1)
>  /* This Source Code Form is subject to the terms of the Mozilla Public
>   * License, v. 2.0. If a copy of the MPL was not distributed with this
>   * file, You can obtain one at http://mozilla.org/MPL/2.0/. */
>  
>  pref("startup.homepage_override_url", "https://www.mozilla.org/projects/firefox/%VERSION%/whatsnew/?oldversion=%OLD_VERSION%");
> -pref("startup.homepage_welcome_url", "https://www.mozilla.org/projects/firefox/%VERSION%/firstrun/");
> +pref("startup.homepage_welcome_url", "about:welcome");

This also needs to be changed back

Comment 24

[Erica Wright [:ewright]]

Comment on attachment 8980434 [details]
Bug 1464246 - change the pref which will redirect the welcome page from a mozilla.org page to about:welcome.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/246606/diff/1-2/

Comment 25

[Erica Wright [:ewright]]

We're just waiting on l10n, I've tried to find if there's a summary for activity stream strings, will get back to you if I find more information on it.

Also, there is a better way to test this.
To run locally with branding in your mozconfig
 ac_add_options --with-branding=browser/branding/aurora // dev edition
 ac_add_options --with-branding=browser/branding/nightly // nightly
 ac_add_options --with-branding=browser/branding/official // beta and release

Comment 26

[Kate Hudson :k88hudson]

Comment on attachment 8980434 [details]
Bug 1464246 - change the pref which will redirect the welcome page from a mozilla.org page to about:welcome.

https://reviewboard.mozilla.org/r/246606/#review255596

Comment 27

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d52bdd020ef9
change the pref which will redirect the welcome page from a mozilla.org page to about:welcome. r=k88hudson

Comment 28

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/d52bdd020ef9