1209140 - /firstrun 42 - Open a second tab which will talk about Private Browsing with Tracking Protection

Status: VERIFIED FIXED

(Firefox :: General, defect)

Description

[Cory Price [:ckprice] (bugmail disabled, NI me!)]

For /firstrun 42, we would like to introduce Private Browsing with Tracking Protection into a 2nd tab. The design/copy for this content is complete.

As a consideration: in bug1185012comment32, we added logic which will open the Windows 10 welcome page if the user is opening Firefox in Windows 10 for the first time.

We haven't hashed out exactly how these two elements will interact with each other, but I wanted to get this bug filed, and see if we can get it on the radar for 42 RC.

Comment 1

[Alex Davis [:adavis]]

Per this A/B test, opening 2 tabs at first run had no negative impact on account creation (first tab).
https://mana.mozilla.org/wiki/display/FIREFOX/Two+Tabs+at+FirstRun

We concluded it was an efficient communication channel since a significant amount of users engaged with the 2nd tab.

Comment 2

[Cory Price [:ckprice] (bugmail disabled, NI me!)]

Had a regroup today with cmore, verdi, garethc, adavis, habber and have the following specifications:

W10Folk = users opening Firefox for the first time on Windows 10.

/firstrun (all users)
TAB1 = /firstrun URL
TAB2 = /firstrun/learnmore (new URL)
* Suppress the Windows 10 Welcome Page in this experience

/whatsnew (non Win10Folk)
TAB1 = /whatsnew URL

/whatsnew (Win10Folk)
TAB1 = /whastnew URL
TAB2 = /windows-10/welcome

Other Win10Folk
TAB1 = /windows-10/welcome

:jaws, :MattN - will you please have a quick review of this, and let me know if we can uplift this to 42 release?

Thanks!

Comment 4

[Cory Price [:ckprice] (bugmail disabled, NI me!)]

Hold on this one - discussing more with PMM.

Comment 5

[Cory Price [:ckprice] (bugmail disabled, NI me!)]

Attachment: /firstun 42 - Tab Specifications

Sorry for the churn, but we've received PMM direction that it's more important (for the time being) for Windows 10 users to see the Welcome page instead of the PB/TP content.

I've updated the specification in comment 2 with the attached document.

Comment 6

[Matthew N. [:MattN]]

Attachment: MozReview Request: Bug 1209140 - Open a second firstrun tab for Tracking Protection promotion. r=jaws

Bug 1209140 - Open a second firstrun tab for Tracking Protection promotion. r=jaws

Comment 7

[Matthew N. [:MattN]]

[Tracking Requested - why for this release]: Wanted by PMM for marketing tracking protection to new users

Comment 8

[Jared Wein [:jaws] (please needinfo? me)]

Comment on attachment 8673849 [details]
MozReview Request: Bug 1209140 - Open a second firstrun tab for Tracking Protection promotion. r=jaws

https://reviewboard.mozilla.org/r/22141/#review19827

Comment 9

[Matthew N. [:MattN]]

Comment on attachment 8673849 [details]
MozReview Request: Bug 1209140 - Open a second firstrun tab for Tracking Protection promotion. r=jaws

Approval Request Comment
[Feature/regressing bug #]: TP in PB
[User impact if declined]: No promotion of TP in PB for 42+ in new profiles
[Describe test coverage new/current, TreeHerder]: Only manual testing since it's for firstrun.
[Risks and why]: Low risk change to firstrun logic. This code shouldn't run otherwise.
[String/UUID change made/needed]: None. Webpage will be localized itself.

Comment 10

[Pulsebot]

https://hg.mozilla.org/integration/fx-team/rev/0409f20e3ee2

Comment 11

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/0409f20e3ee2

Comment 12

[Sylvestre Ledru [:Sylvestre]]

Matt, could you explain how QE can test that? Thanks
(btw, you are still marked as in PTO).

Comment 13

[Sylvestre Ledru [:Sylvestre]]

Comment on attachment 8673849 [details]
MozReview Request: Bug 1209140 - Open a second firstrun tab for Tracking Protection promotion. r=jaws

Sure, taking it as you did a great experiment on this.
Should be in 42 beta 8.

Comment 14

[Matthew N. [:MattN]]

(In reply to Sylvestre Ledru [:sylvestre] from comment #12)
> Matt, could you explain how QE can test that?

I updated the spec with testing instructions.

Cory, can you please accept my changes and make the document public to QE can see it?

Comment 15

[Matthew N. [:MattN]]

Cory, also please make sure the page is live as users will start to see it Tuesday on Beta.

Comment 16

[Alex Gibson [:agibson]]

(In reply to Matthew N. [:MattN] (PTO Oct. 8-12) from comment #15)
> Cory, also please make sure the page is live as users will start to see it
> Tuesday on Beta.

The page will not be ready by Tuesday. I suggest we use a temporary redirect for now to some other page. Cory, can you please file a bedrock bug for this?

Comment 17

[Cory Price [:ckprice] (bugmail disabled, NI me!)]

Document in attachment 8667973 [details] updated.

Bug 1215633 opened to add a temp redirect for URL.

Comment 18

[Matthew N. [:MattN]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/26ef53809ca7

Comment 19

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/releases/mozilla-beta/rev/968735b8ea8d

Comment 20

[Paul Silaghi, QA [:pauly]]

I've tested this on FF 42b9, using the instructions from https://goo.gl/10q1Ff:
> To simulate an upgrade to test whatsnew, decrease the version number in the browser.startup.homepage_override.mstone preference
Whatsnew page doesn't show up after restart.

> Firstrun can be tested by simply using a new profile
On Win 10, tab1=first run, tab2=https://www.mozilla.org/en-US/firefox/windows-10/welcome/?utm_source=firefox-browser&utm_medium=firefox-browser&utm_content=firstrun

> To simulate a user running on Windows 10 for the first time in an existing profile, set browser.usedOnWindows10 to false on Windows 10
No first run page, tab1=https://www.mozilla.org/en-US/firefox/windows-10/welcome/?utm_source=firefox-browser&utm_medium=firefox-browser

Please say which behavior is expected or not.

Comment 21

[Cory Price [:ckprice] (bugmail disabled, NI me!)]

A couple notes:

1. ALL /whatsnew 42 users should be seeing the /whatsnew URL. Maybe it's because bug 1196530 isn't pushed yet?

2. One test I don't see in comment 20 is a non Windows 10 users for /firstrun

Comment 22

[Paul Silaghi, QA [:pauly]]

(In reply to Cory Price [:ckprice] from comment #21)
> 2. One test I don't see in comment 20 is a non Windows 10 users for /firstrun
Worked fine, I mentioned only the suspicious behaviors

Comment 23

[Cory Price [:ckprice] (bugmail disabled, NI me!)]

I was able to test 42 successfully on a different update channel noted in bug 1196530 comment 4.

:pauly - can you confirm this as well please?

/firstrun 42 looks good.

Comment 24

[Paul Silaghi, QA [:pauly]]

I still don't see the whatsnew page.
I've tried the following scenarios, on Win 7:
1. Start FF 41, go to Help/About and update to FF 42 using the release standard update channel.
2. Set the update channel to release-localtest, start FF 41, go to Help/About and update to FF 42
3. Set the update channel to release-localtest, start FF 42 release, decrease the version number in the browser.startup.homepage_override.mstone preference to 41.0, restart FF
Thoughts?

Comment 25

[Cory Price [:ckprice] (bugmail disabled, NI me!)]

I think we are good on /firstrun as :pauly has successfully verified that functionality.

(In reply to Paul Silaghi, QA [:pauly] from comment #24)
> I still don't see the whatsnew page.
> 2. Set the update channel to release-localtest, start FF 41, go to
> Help/About and update to FF 42

Myself, rail and rbillings were able to verify /whatsnew. Your step 2 should have worked, you have to be sure you're updating the channel not through the prefs, but through the text file.

- be on 41
- browse to the channel-prefs.js file (for OSX /Applications/Firefox.app/contents/resources/defaults/pref)
- change channel to `release-localtest`  [may not work if you change it through about:config]
- close Firefox
- open Firefox
- go to About
- see update download and apply
- click 'Restart'
- tabs should open according to https://bugzilla.mozilla.org/attachment.cgi?id=8667973

:pauly, let me know if this works for you

Comment 26

[Paul Silaghi, QA [:pauly]]

Thanks Cory.
It seems to work now, although I did the same steps compared to last time.
Results:
> /whatsnew (non Win10Folk)
> TAB1 = /whatsnew URL (PB/TP)
1. PASS

> /whatsnew (Win10Folk)
> TAB1 = /whastnew URL (PB/TP)
> TAB2 = /windows-10/welcome/?utm_source=firefox-browser&utm_medium=firefox-browser
2. FAIL - tab2 didn't show up

> Win10Folk who aren’t upgrading/installing
> TAB1 = /windows-10/welcome/?utm_source=firefox-browser&utm_medium=firefox-browser
3. FAIL - tab1=about:home

Comment 27

[Paul Silaghi, QA [:pauly]]

Actually 2) and 3) seem logical, since FF is not at the first run.

Comment 28

[Cory Price [:ckprice] (bugmail disabled, NI me!)]

Thanks :pauly - first let's confirm the definition of Win10Folk

> Win10Folk = users opening Firefox for the *first time* on Windows 10.

(In reply to Paul Silaghi, QA [:pauly] from comment #26)
> > /whatsnew (Win10Folk)
> > TAB1 = /whastnew URL (PB/TP)
> > TAB2 = /windows-10/welcome/?utm_source=firefox-browser&utm_medium=firefox-browser
> 2. FAIL - tab2 didn't show up

Please confirm that this was the first time this test case opened in Windows 10. Probably an edge case here (e.g. browser update staged, then update OS).

> 
> > Win10Folk who aren’t upgrading/installing
> > TAB1 = /windows-10/welcome/?utm_source=firefox-browser&utm_medium=firefox-browser
> 3. FAIL - tab1=about:home

Can you confirm here as well that this was the first time this test case opened Firefox in Windows 10 (e.g. right after an OS update)?

Comment 29

[Paul Silaghi, QA [:pauly]]

I couldn't perform an OS update right now, but I tweaked the browser.usedOnWindows10=FALSE according to
"To simulate a user running on Windows 10 for the first time in an existing profile, set browser.usedOnWindows10 to false on Windows 10", and the "/windows-10/welcome/?utm_source=firefox-browser&utm_medium=firefox-browser" pages showed up correctly on cases 2) and 3).
Marking the bug as verified fixed.

Comment 30

[:Gijs (he/him)]

Do we need to keep this for 43/44/45 ?

Comment 31

[Cory Price [:ckprice] (bugmail disabled, NI me!)]

(In reply to :Gijs Kruitbosch from comment #30)
> Do we need to keep this for 43/44/45 ?

When do we have to tell you to remove for 43? We are gathering metrics on the effectiveness of the 2nd tab for /firstrun.

Comment 32

[:Gijs (he/him)]

(In reply to Cory Price [:ckprice] from comment #31)
> (In reply to :Gijs Kruitbosch from comment #30)
> > Do we need to keep this for 43/44/45 ?
> 
> When do we have to tell you to remove for 43? We are gathering metrics on
> the effectiveness of the 2nd tab for /firstrun.

Err, beta 43 is already out... so "earlier than today" if you wanted nobody on 43 to see it...

Comment 33

[Matthew N. [:MattN]]

(In reply to :Gijs Kruitbosch from comment #30)
> Do we need to keep this for 43/44/45 ?

FTR this was asked in one of our meetings and the answer was yes (which is why I didn't just land directly on 42 despite me thinking it was a better idea).

Comment 34

[Cory Price [:ckprice] (bugmail disabled, NI me!)]

Confirming that we are keeping this on for at least Firefox 43.

Comment 35

[Paul Silaghi, QA [:pauly]]

Firstrun page displayed correctly on FF 43b8 Win 7 and Win 10.
Whatsnew page couldn't be tested in the Beta cycle, according to https://bugzilla.mozilla.org/show_bug.cgi?id=1196530#c26

Comment 36

[Paul Silaghi, QA [:pauly]]

(In reply to Paul Silaghi, QA [:pauly] from comment #35)
> Firstrun page displayed correctly on FF 43b8 Win 7 and Win 10.
> Whatsnew page couldn't be tested in the Beta cycle, according to
> https://bugzilla.mozilla.org/show_bug.cgi?id=1196530#c26
Verified fixed FF 44b9 Win7 and Win10.