Closed Bug 1209140 Opened 5 years ago Closed 5 years ago

/firstrun 42 - Open a second tab which will talk about Private Browsing with Tracking Protection

Categories

(Firefox :: General, defect)

defect
Not set

Tracking

()

VERIFIED FIXED
Firefox 44
Tracking Status
firefox41 --- unaffected
firefox42 + verified
firefox43 --- verified
firefox44 --- verified

People

(Reporter: ckprice, Assigned: MattN)

References

Details

(Whiteboard: [fall-2015-engagement])

Attachments

(2 files)

For /firstrun 42, we would like to introduce Private Browsing with Tracking Protection into a 2nd tab. The design/copy for this content is complete.

As a consideration: in bug1185012comment32, we added logic which will open the Windows 10 welcome page if the user is opening Firefox in Windows 10 for the first time.

We haven't hashed out exactly how these two elements will interact with each other, but I wanted to get this bug filed, and see if we can get it on the radar for 42 RC.
Per this A/B test, opening 2 tabs at first run had no negative impact on account creation (first tab).
https://mana.mozilla.org/wiki/display/FIREFOX/Two+Tabs+at+FirstRun

We concluded it was an efficient communication channel since a significant amount of users engaged with the 2nd tab.
Duplicate of this bug: 1196548
Blocks: 1196540
No longer blocks: 1196548
Hold on this one - discussing more with PMM.
Flags: needinfo?(jaws)
Flags: needinfo?(MattN+bmo)
Sorry for the churn, but we've received PMM direction that it's more important (for the time being) for Windows 10 users to see the Welcome page instead of the PB/TP content.

I've updated the specification in comment 2 with the attached document.
Assignee: nobody → MattN+bmo
Status: NEW → ASSIGNED
Bug 1209140 - Open a second firstrun tab for Tracking Protection promotion. r=jaws
Attachment #8673849 - Flags: review?(jaws)
[Tracking Requested - why for this release]: Wanted by PMM for marketing tracking protection to new users
Comment on attachment 8673849 [details]
MozReview Request: Bug 1209140 - Open a second firstrun tab for Tracking Protection promotion. r=jaws

https://reviewboard.mozilla.org/r/22141/#review19827
Attachment #8673849 - Flags: review?(jaws) → review+
Comment on attachment 8673849 [details]
MozReview Request: Bug 1209140 - Open a second firstrun tab for Tracking Protection promotion. r=jaws

Approval Request Comment
[Feature/regressing bug #]: TP in PB
[User impact if declined]: No promotion of TP in PB for 42+ in new profiles
[Describe test coverage new/current, TreeHerder]: Only manual testing since it's for firstrun.
[Risks and why]: Low risk change to firstrun logic. This code shouldn't run otherwise.
[String/UUID change made/needed]: None. Webpage will be localized itself.
Attachment #8673849 - Flags: approval-mozilla-beta?
Attachment #8673849 - Flags: approval-mozilla-aurora?
https://hg.mozilla.org/mozilla-central/rev/0409f20e3ee2
Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 44
Matt, could you explain how QE can test that? Thanks
(btw, you are still marked as in PTO).
Flags: qe-verify+
Flags: needinfo?(MattN+bmo)
Comment on attachment 8673849 [details]
MozReview Request: Bug 1209140 - Open a second firstrun tab for Tracking Protection promotion. r=jaws

Sure, taking it as you did a great experiment on this.
Should be in 42 beta 8.
Attachment #8673849 - Flags: approval-mozilla-beta?
Attachment #8673849 - Flags: approval-mozilla-beta+
Attachment #8673849 - Flags: approval-mozilla-aurora?
Attachment #8673849 - Flags: approval-mozilla-aurora+
(In reply to Sylvestre Ledru [:sylvestre] from comment #12)
> Matt, could you explain how QE can test that?

I updated the spec with testing instructions.

Cory, can you please accept my changes and make the document public to QE can see it?
Flags: needinfo?(MattN+bmo) → needinfo?(cprice)
Cory, also please make sure the page is live as users will start to see it Tuesday on Beta.
(In reply to Matthew N. [:MattN] (PTO Oct. 8-12) from comment #15)
> Cory, also please make sure the page is live as users will start to see it
> Tuesday on Beta.

The page will not be ready by Tuesday. I suggest we use a temporary redirect for now to some other page. Cory, can you please file a bedrock bug for this?
Document in attachment 8667973 [details] updated.

Bug 1215633 opened to add a temp redirect for URL.
No longer depends on: 1215633
Flags: needinfo?(cprice)
I've tested this on FF 42b9, using the instructions from https://goo.gl/10q1Ff:
> To simulate an upgrade to test whatsnew, decrease the version number in the browser.startup.homepage_override.mstone preference
Whatsnew page doesn't show up after restart.

> Firstrun can be tested by simply using a new profile
On Win 10, tab1=first run, tab2=https://www.mozilla.org/en-US/firefox/windows-10/welcome/?utm_source=firefox-browser&utm_medium=firefox-browser&utm_content=firstrun

> To simulate a user running on Windows 10 for the first time in an existing profile, set browser.usedOnWindows10 to false on Windows 10
No first run page, tab1=https://www.mozilla.org/en-US/firefox/windows-10/welcome/?utm_source=firefox-browser&utm_medium=firefox-browser

Please say which behavior is expected or not.
Flags: needinfo?(MattN+bmo)
QA Contact: paul.silaghi
A couple notes:

1. ALL /whatsnew 42 users should be seeing the /whatsnew URL. Maybe it's because bug 1196530 isn't pushed yet?

2. One test I don't see in comment 20 is a non Windows 10 users for /firstrun
(In reply to Cory Price [:ckprice] from comment #21)
> 2. One test I don't see in comment 20 is a non Windows 10 users for /firstrun
Worked fine, I mentioned only the suspicious behaviors
I was able to test 42 successfully on a different update channel noted in bug 1196530 comment 4.

:pauly - can you confirm this as well please?

/firstrun 42 looks good.
I still don't see the whatsnew page.
I've tried the following scenarios, on Win 7:
1. Start FF 41, go to Help/About and update to FF 42 using the release standard update channel.
2. Set the update channel to release-localtest, start FF 41, go to Help/About and update to FF 42
3. Set the update channel to release-localtest, start FF 42 release, decrease the version number in the browser.startup.homepage_override.mstone preference to 41.0, restart FF
Thoughts?
Flags: needinfo?(cprice)
I think we are good on /firstrun as :pauly has successfully verified that functionality.

(In reply to Paul Silaghi, QA [:pauly] from comment #24)
> I still don't see the whatsnew page.
> 2. Set the update channel to release-localtest, start FF 41, go to
> Help/About and update to FF 42

Myself, rail and rbillings were able to verify /whatsnew. Your step 2 should have worked, you have to be sure you're updating the channel not through the prefs, but through the text file.

- be on 41
- browse to the channel-prefs.js file (for OSX /Applications/Firefox.app/contents/resources/defaults/pref)
- change channel to `release-localtest`  [may not work if you change it through about:config]
- close Firefox
- open Firefox
- go to About
- see update download and apply
- click 'Restart'
- tabs should open according to https://bugzilla.mozilla.org/attachment.cgi?id=8667973

:pauly, let me know if this works for you
Flags: needinfo?(paul.silaghi)
Flags: needinfo?(cprice)
Flags: needinfo?(MattN+bmo)
Thanks Cory.
It seems to work now, although I did the same steps compared to last time.
Results:
> /whatsnew (non Win10Folk)
> TAB1 = /whatsnew URL (PB/TP)
1. PASS

> /whatsnew (Win10Folk)
> TAB1 = /whastnew URL (PB/TP)
> TAB2 = /windows-10/welcome/?utm_source=firefox-browser&utm_medium=firefox-browser
2. FAIL - tab2 didn't show up

> Win10Folk who aren’t upgrading/installing
> TAB1 = /windows-10/welcome/?utm_source=firefox-browser&utm_medium=firefox-browser
3. FAIL - tab1=about:home
Flags: needinfo?(paul.silaghi) → needinfo?(cprice)
Actually 2) and 3) seem logical, since FF is not at the first run.
Thanks :pauly - first let's confirm the definition of Win10Folk

> Win10Folk = users opening Firefox for the *first time* on Windows 10.

(In reply to Paul Silaghi, QA [:pauly] from comment #26)
> > /whatsnew (Win10Folk)
> > TAB1 = /whastnew URL (PB/TP)
> > TAB2 = /windows-10/welcome/?utm_source=firefox-browser&utm_medium=firefox-browser
> 2. FAIL - tab2 didn't show up

Please confirm that this was the first time this test case opened in Windows 10. Probably an edge case here (e.g. browser update staged, then update OS).

> 
> > Win10Folk who aren’t upgrading/installing
> > TAB1 = /windows-10/welcome/?utm_source=firefox-browser&utm_medium=firefox-browser
> 3. FAIL - tab1=about:home

Can you confirm here as well that this was the first time this test case opened Firefox in Windows 10 (e.g. right after an OS update)?
Flags: needinfo?(cprice)
I couldn't perform an OS update right now, but I tweaked the browser.usedOnWindows10=FALSE according to
"To simulate a user running on Windows 10 for the first time in an existing profile, set browser.usedOnWindows10 to false on Windows 10", and the "/windows-10/welcome/?utm_source=firefox-browser&utm_medium=firefox-browser" pages showed up correctly on cases 2) and 3).
Marking the bug as verified fixed.
Status: RESOLVED → VERIFIED
Do we need to keep this for 43/44/45 ?
Flags: needinfo?(cprice)
(In reply to :Gijs Kruitbosch from comment #30)
> Do we need to keep this for 43/44/45 ?

When do we have to tell you to remove for 43? We are gathering metrics on the effectiveness of the 2nd tab for /firstrun.
Flags: needinfo?(cprice)
(In reply to Cory Price [:ckprice] from comment #31)
> (In reply to :Gijs Kruitbosch from comment #30)
> > Do we need to keep this for 43/44/45 ?
> 
> When do we have to tell you to remove for 43? We are gathering metrics on
> the effectiveness of the 2nd tab for /firstrun.

Err, beta 43 is already out... so "earlier than today" if you wanted nobody on 43 to see it...
Flags: needinfo?(cprice)
(In reply to :Gijs Kruitbosch from comment #30)
> Do we need to keep this for 43/44/45 ?

FTR this was asked in one of our meetings and the answer was yes (which is why I didn't just land directly on 42 despite me thinking it was a better idea).
Confirming that we are keeping this on for at least Firefox 43.
Flags: needinfo?(cprice)
Firstrun page displayed correctly on FF 43b8 Win 7 and Win 10.
Whatsnew page couldn't be tested in the Beta cycle, according to https://bugzilla.mozilla.org/show_bug.cgi?id=1196530#c26
(In reply to Paul Silaghi, QA [:pauly] from comment #35)
> Firstrun page displayed correctly on FF 43b8 Win 7 and Win 10.
> Whatsnew page couldn't be tested in the Beta cycle, according to
> https://bugzilla.mozilla.org/show_bug.cgi?id=1196530#c26
Verified fixed FF 44b9 Win7 and Win10.
You need to log in before you can comment on or make changes to this bug.