Closed Bug 1077740 Opened 10 years ago Closed 10 years ago

Reset legacy homepages to about:home

Categories

(Firefox :: General, enhancement)

enhancement
Not set
normal
Points:
3

Tracking

()

VERIFIED FIXED
Firefox 36
Iteration:
36.1
Tracking Status
firefox33 --- wontfix
firefox34 --- verified
firefox35 --- verified
firefox36 --- verified

People

(Reporter: cmore, Assigned: dao)

References

(Depends on 1 open bug)

Details

(Whiteboard: [fxgrowth])

Attachments

(4 files, 2 obsolete files)

We have millions of Firefox users that are using latest versions of Firefox that have their homepage set to one of the two websites:

https://www.google.com/firefox

http://start.mozilla.org

Start.mozilla.org traffic is many magnitudes larger than www.google.com/firefox and start.mozilla.org is also a website that we control.

After Firefox 4, Mozilla created a static copy of www.google.com/firefox and hosted it internally at start.mozilla.org and it is why these two sites look very similar. We would prefer if Google were to redirect google.com/firefox to start.mozilla.org, but that is unlikely to happen.

A few years ago to help address this problem, Mozilla changed a cname to point users to start.mozilla.org instead of google.com/firefox. While that is great that they aren't using googe.com/firefox as much, when those users upgrade, they have their homepage still set as start.mozilla.org.

I recommend that we add some code to the latest Firefox such that when Firefox finds values matching regex .*(www\.google\.com/firefox|start\.mozilla\.org).* that we override the value in browser.startup.homepage to about:home. We are actively trying to get users to upgrade their browser and it appears that even after upgrading from an old version of Firefox to a recent version, their homepages are still being set of these values. 

The current experience is not good for users or Mozilla. Why? Google.com/firefox has broken UX/content and start.mozilla.org isn't much better. If we combine about:home and about:newtab in the future, these users will also be left behind because of their hardcoded values.

We are currently running A/B tests on start.mozilla.org to get old users (Firefox 3.6 and below) to update, but for some reason we have *lots* of Firefox 32 users that have their browsers set to that homepage.

In the spirit of unifying our message and offerings to our Firefox users, we should get this changed asap. We have successful about:home snippet campaigns running and the users who have their homepages set as these values are not seeing those messages. Also, the design/branding on those legacy homepages are outdated.
Update: the page on Google is http (no SSL): http://www.google.com/firefox

Another reason to make this change is that users are searching from non-SSL pages.
Severity: normal → enhancement
OS: Mac OS X → All
Hardware: x86 → All
Flags: firefox-backlog+
Summary: Override browser.startup.homepage value when legacy homepages are present → reset legacy browser.startup.homepage values to "about:home"
Flags: qe-verify+
Chris, can you instrument the page ASAP to track the number (not content) of searches?
(In reply to John Jensen from comment #2)
> Chris, can you instrument the page ASAP to track the number (not content) of
> searches?

Just filed bug 1077856 to have event tracking added to the page.
Points: --- → 3
Assignee: nobody → dao
Status: NEW → ASSIGNED
Iteration: --- → 36.1
Depends on: 762966
Attached patch patch (obsolete) — Splinter Review
Not including https since those sites don't exist anyway.
Attachment #8506254 - Flags: review?(gavin.sharp)
Comment on attachment 8506254 [details] [diff] [review]
patch

http://www.google.de/firefox however does exist, for instance, so I suppose we should cover arbitrary tlds even though I'm not sure that we had actually set browser.startup.homepage to such values.
Attachment #8506254 - Flags: review?(gavin.sharp)
Attached patch patch v2Splinter Review
Attachment #8506254 - Attachment is obsolete: true
Attachment #8506258 - Flags: review?(gavin.sharp)
Comment on attachment 8506258 [details] [diff] [review]
patch v2

>diff --git a/browser/components/nsBrowserGlue.js b/browser/components/nsBrowserGlue.js

>+    if (currentUIVersion < 24) {
>+      // Reset homepage pref for users who have it set to start.mozilla.org
>+      // or google.com/firefox.
>+      const HOMEPAGE_PREF = "browser.startup.homepage";
>+      let uri = Services.prefs.getComplexValue(HOMEPAGE_PREF,
>+                                               Ci.nsIPrefLocalizedString).data;
>+      if (uri && (uri.startsWith("http://start.mozilla.org") ||
>+                  /^http:\/\/(www\.)?google\.[\w.]+\/firefox/.test(uri))) {

Might as well make the google.* regexp match the https version (unlikely but maybe some users manually tweaked them).

\w matches "_", so perhaps /^https?:\/\/(www\.)?google\.[a-z.]+\/firefox/i is a better regexp to use.

r=me with those changes.
Attachment #8506258 - Flags: review?(gavin.sharp) → review+
+1 to the regex adjustments matching other country domains. I also found this domain recently and not sure if it is in our out of scope: http://china-start.mozilla.net/
(In reply to Chris More [:cmore] from comment #8)
> +1 to the regex adjustments matching other country domains. I also found
> this domain recently and not sure if it is in our out of scope:
> http://china-start.mozilla.net/

I think that's out of scope here.

https://hg.mozilla.org/integration/fx-team/rev/e80f4dff89fb
https://hg.mozilla.org/mozilla-central/rev/e80f4dff89fb
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 36
Given that Firefox 36 is 2/24/2015 and the relative complexity of this patch, do you feel this is a candidate for an uplift before release in Fx 36?
Flags: needinfo?(gavin.sharp)
Yes, we can uplift this to 34.

Dao, can you request approvals?
Flags: needinfo?(gavin.sharp)
Comment on attachment 8506258 [details] [diff] [review]
patch v2

Approval Request Comment
[Feature/regressing bug #]: n/a
[User impact if declined]: some users see a legacy homepage with limited utility
[Describe test coverage new/current, TBPL]: no test coverage
[Risks and why]: low
[String/UUID change made/needed]: none
Attachment #8506258 - Flags: approval-mozilla-beta?
Attachment #8506258 - Flags: approval-mozilla-aurora?
It looks like this fix will remove a user's start pages if multiple pages are set and the first is start.mozilla.org. For ex,

http://start.mozilla.org|http://mozilla.org

Is this intentional and acceptable?
Flags: needinfo?(dao)
Not really intentional, but it sounds like a very unlikely edge case that I don't think is worth worrying about.
Flags: needinfo?(dao)
Perhaps trivial but as a user I would probably consider that data loss. (Although somewhat trivial data loss perhaps enough to make me upset by this homepage change.) Do you have any data to confirm that this is an edge case? If we don't know, is it really that hard to switch to about:home with a substitution regex instead of resetting the homepage?
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Attachment #8506258 - Flags: approval-mozilla-beta?
Attachment #8506258 - Flags: approval-mozilla-aurora?
Attachment #8506258 - Flags: checkin+
Attachment #8510115 - Flags: review?(gavin.sharp)
Summary: reset legacy browser.startup.homepage values to "about:home" → Reset legacy homepages to about:home
Attached patch combined patch for uplift (obsolete) — Splinter Review
Comment on attachment 8510115 [details] [diff] [review]
handle multiple home pages

>diff --git a/browser/components/nsBrowserGlue.js b/browser/components/nsBrowserGlue.js

>+        let value =
>+          Services.prefs.getComplexValue(HOMEPAGE_PREF, Ci.nsIPrefLocalizedString);

>+            Services.prefs.setComplexValue(HOMEPAGE_PREF,
>+                                           Ci.nsISupportsString, value);

Here you're passing what is theoretically a nsIPrefLocalizedString (value) but saying it's a nsISupportsString. This looks like it works anyways because of some automatic xpconnect string conversion stuff, but it's a bit confusing. It looks like setting a complex value of type nsIPrefLocalizedString works fine, so you could just do that. But actually, it might be clearer to just change the getComplexValue call to use nsISupportsString, since it's guaranteed to not be a localized pref if it has a user value.
Attachment #8510115 - Flags: review?(gavin.sharp) → review+
Approval Request Comment
[Feature/regressing bug #]: n/a
[User impact if declined]: some users see a legacy homepage with limited utility
[Describe test coverage new/current, TBPL]: no test coverage
[Risks and why]: low
[String/UUID change made/needed]: none
Attachment #8510121 - Attachment is obsolete: true
Attachment #8510989 - Flags: approval-mozilla-beta?
Attachment #8510989 - Flags: approval-mozilla-aurora?
juanb - Can you please test this feature on Nightly before uplift? 

The desired outcome is that if a user has one of their startpages set to https://www.google.com/firefox or http://start.mozilla.org that that page will be replaced with about:home.
Flags: needinfo?(jbecerra)
(In reply to Lawrence Mandel [:lmandel] (use needinfo) from comment #22)
> juanb - Can you please test this feature on Nightly before uplift? 
> 
> The desired outcome is that if a user has one of their startpages set to
> https://www.google.com/firefox or http://start.mozilla.org that that page
> will be replaced with about:home.

Here's a complete set of use-cases to test:

http://start.mozilla.org
http://start.mozilla.org/ 
http://start.mozilla.org/en-US/
http://start.mozilla.org/en-US/?foo=bar
http://start.mozilla.org/fr/
http://start.mozilla.org/?foo=bar

http://www.google.com/firefox
http://www.google.com/firefox?foo=bar
http://www.google.fr/firefox
http://www.google.de/firefox

Note: Also note that both domains are HTTP and not HTTPS. Neither of the sites load via HTTPS.
https://hg.mozilla.org/mozilla-central/rev/77acdd825695
Status: REOPENED → RESOLVED
Closed: 10 years ago10 years ago
Resolution: --- → FIXED
If I test this in Firefox nightly 36.0a1, should this work patch now?
(In reply to Chris More [:cmore] from comment #25)
> If I test this in Firefox nightly 36.0a1, should this work patch now?

Yes, but it's a one-time update. So to test, you should create a profile on an older version of Firefox/Nightly, set the homepage pref, and then use the more recent 36.0a1 with the same profile.
I've verified this issue with the latest Firefox 36 Nightly build (BuildID=20141029030207) on Windows 7 x64, Ubuntu 12.04 x86, and Mac OS X 10.9.5.

Scenario used to test:
1. Install and open Firefox 32.0.3 with a new profile.
2. Go to about:config and set browser.startup.homepage to a value from comment 23.
3. Restart 32.0.3 to confirm that the value is properly applied.
4. Install and open latest Nightly with same profile as at step #1.

The latest Nightly version overwrites the homepage value and opens with about:home as expected. Verified all values from comment 23 on Windows, and one of each of the two URLs on Ubuntu and Mac.

Also verified scenario with an older Nightly (Oct 09) which does not overwrite the homepage value. However after updating to the latest Nightly about:home is the homepage as expected.
Status: RESOLVED → VERIFIED
Flags: needinfo?(jbecerra)
QA Contact: florin.mezei
Thank you for verifying Florin. Did you also test with multiple home pages?
Flags: needinfo?(florin.mezei)
(In reply to Lawrence Mandel [:lmandel] (use needinfo) from comment #28)
> Thank you for verifying Florin. Did you also test with multiple home pages?

I've missed this case, sorry about that! I tested it today using the same Firefox 36 Nightly build as yesterday (BuildID=20141029030207) on Windows 7 x64, Ubuntu 12.04 x86, and Mac OS X 10.9.5.

I used the same scenario as in comment 27, covering the following values for the pref:
http://start.mozilla.org|http://openstandard.mozilla.org 
http://gmail.com|http://start.mozilla.org/en-US/?foo=bar|http://eclipse.org
http://openstandard.mozilla.org|http://start.mozilla.org/fr/ 
http://www.google.com/firefox|http://openstandard.mozilla.org 
http://gmail.com|http://www.google.com/firefox?foo=bar|http://eclipse.org
http://openstandard.mozilla.org|http://www.google.fr/firefox
Flags: needinfo?(florin.mezei)
Comment on attachment 8510989 [details] [diff] [review]
combined patch for uplift

Thanks Florin! Beta+ and Aurora+
Attachment #8510989 - Flags: approval-mozilla-beta?
Attachment #8510989 - Flags: approval-mozilla-beta+
Attachment #8510989 - Flags: approval-mozilla-aurora?
Attachment #8510989 - Flags: approval-mozilla-aurora+
Verified scenarios with single homepage (comment 27) and multiple homepages (comment 30) on Windows 7 x64, Mac OS X 10.9.5 and Ubuntu 14.04 x64, using:
- Firefox 34 beta 5 - BuildID=20141030172027
- Latest Firefox 35 Aurora - BuildID=20141031004003

Everything works as intended as the start/google homepage is correctly replaced by about:home.
Can I verify that this was in Firefox 34.0 and 34.0.5? I am seeing users who upgraded from 33.0 to 34.0 continue to use start.mozilla.org as their homepage.
Flags: needinfo?(gavin.sharp)
(In reply to Chris More [:cmore] from comment #35)
> Can I verify that this was in Firefox 34.0 and 34.0.5? I am seeing users who
> upgraded from 33.0 to 34.0 continue to use start.mozilla.org as their
> homepage.

Yes, this fix was included in 34.0 and later.

Are you seeing the same volume as in previous versions?

It's impossible to kill uses of this page 100% (the user can for example always override our reset by re-entering this URL, or might have a bookmark to those pages, etc.), but I would expect this to have made some difference.
Flags: needinfo?(gavin.sharp)
Well, I typically see 1,300,000 sessions from 33.0 per week on the start.mozilla.org. This past week, 33.0 dropped to 1,200,000 sessions and now we have 100,000 34.0 sessions on the site. It may not be a correlation, but it is interesting that the magnitude loss of 33.0 to the site is almost equal to the magnitude gain of 34.0 user. We'll have to give it a few weeks to see if the trend continues.

Hmmm, bookmark is a good point and possible. Though, not sure why someone would bookmark the start and also have a different start page. Though, users can do weird things at times.
It's also possible that users put this pref in user.js, that we can't write to prefs.js (or generally the profile directory) anymore or that some weird add-on is involved...
I really think there is a bug with this patch. Look at this trend. As they get upgraded, they continue to use start.mozilla.org. As 33 drops off, 34 picks up. I doubt the majority of that population is making a purposeful change back to start.mozilla.org. Yes, it is possible that some people change it back to start.mozilla.org, but we know that most users don't look at URL so they would have to remember it was called start.mozilla.org.
(In reply to Chris More [:cmore] from comment #39)
> Created attachment 8535206 [details]
> Firefox 33.x vs 34.x traffic to start.mozilla.org
> 
> I really think there is a bug with this patch.

I don't see where that might be, I think we've covered the edge cases and it was tested manually.

> Look at this trend. As they
> get upgraded, they continue to use start.mozilla.org. As 33 drops off, 34
> picks up. I doubt the majority of that population is making a purposeful
> change back to start.mozilla.org.

I wouldn't expect that either. My theory is that there's something preventing us from resetting the pref. Maybe this would also explain why start.mozilla.org stuck around in the first place for these users when we changed the home page.
 > I wouldn't expect that either. My theory is that there's something
> preventing us from resetting the pref. Maybe this would also explain why
> start.mozilla.org stuck around in the first place for these users when we
> changed the home page.

I would agree with your theory that it could be some kind of permissions issue in the pref. That seems logical and how they got stuck in the first place.

Gavin: For these up-to-date users who are stuck on start.mozilla.org and could have permissions issues, would the UITour API also have the same permissions issue? For example, if we had some code on the /whatsnew page that checked their start page settings through the API (only if that could be added to the API) and then replaced the substring with about:home just like this bug does, could the API tour do the substring replacement just like this bug?
Flags: needinfo?(gavin.sharp)
(In reply to Chris More [:cmore] from comment #41)
> Gavin: For these up-to-date users who are stuck on start.mozilla.org and
> could have permissions issues, would the UITour API also have the same
> permissions issue? For example, if we had some code on the /whatsnew page
> that checked their start page settings through the API (only if that could
> be added to the API) and then replaced the substring with about:home just
> like this bug does, could the API tour do the substring replacement just
> like this bug?

It wouldn't make a difference.
I think we only have a few options here:

a) think of a way to validate these "permissions issues" hypotheses via telemetry. We probably want to be able to detect:
 - is the clearUserPref effective immediately after it occurs?
 - for a given user, after we've reset the pref, does the pref return to its old value after a restart of Firefox?

b) try finding a way to get in touch with affected users, e.g. by putting a special message on the page prompting them to contact us, and then hopefully debugging the issue directly with at least one affected user
Flags: needinfo?(gavin.sharp)
I filed bug 1077740 for the telemetry.
Quick update that we have seen a 30% drop to traffic to start.mozilla.org since this bug went live into Fx 34 and it continues to drop rapidly as users upgrade to Fx 34. So, overall, this bug appears to be working! There is still a decent amount of Fx 34 users on the website, but it is 30% lower than previous fx 33 users.
Blocks: 1491332
You need to log in before you can comment on or make changes to this bug.