623110 - new default search engine overrides user-installed search engine with same name (Bing search in Firefox 4)

Status: VERIFIED FIXED

(Firefox :: Search, defect)

Description

[Mike Kaply [:mkaply]]

If a user on 3.6 has Bing installed and set as the default engine, when they upgrade to Firefox 4, that Bing is overwritten with the Firefox 4 Bing.

This breaks an existing revenue connection.

Considering Mozilla doesn't allow this, Firefox shouldn't be doing this.

If Bing is already the default engine, The Firefox Bing should be hidden upon install.

To reproduce:

1.Create a new profile with 3.6
2. Install Bing and make it the default - https://addons.mozilla.org/en-US/firefox/addon/10434/
3. Search result: http://www.bing.com/search?q=test&form=OSDSRC
4. Open same profile with Firefox 4
5. Search result: http://www.bing.com/search?q=test&form=MOZSBR&pc=MOZI

Expected result:

http://www.bing.com/search?q=test&form=OSDSRC

Comment 1

[Mike Kaply [:mkaply]]

Actually, if Bing is installed at all (even not the default, it should not be overwritten without the user opting in. This is the same requirement Mozilla has of add-ons.

Comment 2

[Kev Needham [:kev]]

Gavin, could you have a look at this?

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Asking for blocking, so we have it on the list of bugs which needs at least clarification.

Comment 4

[Mike Kaply [:mkaply]]

Also, although my example is the Bing search on AMO, our use case is with Personas Interactive, a Brand Thunder addon.

Firefox 4 breaks a revenue connection that the user has either opted into (via the AMO install) or agreed to with our non AMI install.

This is a huge deal.

Comment 5

[Mike Beltzner [:beltzner, not reading bugmail]]

Hey Kaply, no need to oversell. The actions on this bug to date speak to the fact that others recognize the severity of the issue at hand. Let's wait to hear from Gavin about what's causing this; I'll try not to speculate, though I have some suspicions!

Comment 6

[:Gavin Sharp [email: [e-mail]]]

We load engines in this priority order:
- addon-shipped
- app-shipped
- user-installed

and the first engine loaded with a given name wins. So yeah, our shipped Bing will overwrite the web-installed Bing (but not any addon-installed Bings).

I don't think there is much we can really do about that, short of changing the load prioritization, and that seems risky.

The case you're worried about seems to be an addon-install, which wouldn't be affected by this, so I'm leaning towards WONTFIX...

Comment 7

[Mike Kaply [:mkaply]]

It does overwrite addon-installed Bings.

We don't ship in searchplugins, we install Bing with addEngineWithDetails.

After going from Firefox 3.6 to Firefox 4, our Bing is overwritten with the Firefox Bing.

I'm sorry, but that's just not acceptable.

A user has chosen to use our Bing, connected revenue to us and Firefox is overwriting it.

Comment 8

[:Gavin Sharp [email: [e-mail]]]

(In reply to comment #7)
> It does overwrite addon-installed Bings.
> 
> We don't ship in searchplugins, we install Bing with addEngineWithDetails.

That's not "addon-installed". That's "hacky method used by addon to user-install a plugin". You could easily release an update to your addon that uses the supported search plugin installation method, and we can work together to find solutions that made it infeasible in the past - I'd be glad to help.

You're asking us to make a risky change to global engine load ordering to support your hacky addon, and that isn't really something I want to do.

Comment 9

[Mike Kaply [:mkaply]]

In order to comply with AMO policies, I can't ship my search engine like that.

Otherwise my search engine overwrites the default Bing without the user opting in.

Comment 10

[:Gavin Sharp [email: [e-mail]]]

You can give your bing search engine a different name.

Comment 11

[Mike Kaply [:mkaply]]

Then the user has two Bing engines. And I'm pretty sure that even if hide mine, if they click "restore defaults" in the search dialog, mine will magically reappear.

Even if we take my situation out of the equation, what Firefox is doing is wrong.

If a user installs Bing, and then upgrades to Firefox 4, the Firefox 4 Bing should not take precedent over the user installed Bing.

I don't understand how you can think that is OK.

You are stealing a revenue connection. There's really no other way to put it.

Comment 12

[:Gavin Sharp [email: [e-mail]]]

"stealing" is not a useful characterization, you don't need to resort to making those kinds of statements to get your point across. There is no malicious intent - we've always behaved this way, but this specific scenario (adding a new default engine) just rarely occurs, so it hasn't come up as a problem before.

Comment 13

[Mike Kaply [:mkaply]]

My problem is that you're trying to characterize the problem as how we've chosen to install our search engines.

The bug has nothing to do with how we install our search engines.

The bug is that on a Firefox upgrade, newly installed default Firefox search engines take precedent over user chosen engines (including the default.) That's a bug, plain and simple.

If a user installed Bing from AMO on Firefox 3.6 and then upgraded to Firefox 4, their Bing should not be replaced with the Firefox default Bing. That's simply wrong.

I don't understand how you can argue any other way.

If you can't fix it, remove Bing from Firefox 4.

Comment 14

[:Gavin Sharp [email: [e-mail]]]

(In reply to comment #13)
> The bug is that on a Firefox upgrade, newly installed default Firefox search
> engines take precedent over user chosen engines (including the default.) That's
> a bug, plain and simple.

I agree - there was never any debate about that.

The debate is about how important that bug is to fix in Firefox 4. There are several factors there:
a) It affects your addon specifically because of the implementation strategy you've chosen. This problem can be avoided by changing your addon.
b) It affects users who've manually installed the Bing plugin (from AMO or elsewhere), but likely not in a way that matters to them (they're unlikely to care which parameters are sent) and perhaps not in a way that matters to the authors of those search plugins (they may or may not care about the parameters being sent).
c) Changing the load ordering behaviour carries some risk.

It's clear that you think the negative effects of a) and b) outweigh the risk of c), but I'm not yet sure I agree.

Comment 15

[Chris B.]

I don't know if this is related, but I cannot get rid of Ask.com search engine (upper right corner) or set it to another default.  Every time I try to remove it or set it to another, say Google, the next time I browse Ask.com is set as the default again.  I have tried the under-the-hood settings to no avail.

Comment 16

[Mike Kaply [:mkaply]]

I've done some investigation into using the addon method of installing our search engine.

It brings up a couple challenges:

1. If the user has the addon installed search engine as the default and they uninstall the addon, their default search engine disappears. And you can't programmatically put it back when you uninstall, because it would be a duplicate of an existing engine.
2. For AMO installs, we would name the engine differently and if the user did not optin, they would have two engines in their list, Bing and Bing (NEW)

Comment 17

[Chris B.]

Never mind about comment 15.  I removed Ask.com search yesterday, set the default to Google, and Ask.com is gone from the list.  I remember having the problem in earlier builds (around Beta 6 and 7), but it is no longer an issue with Minefield 1/11/11.

Comment 18

[:Gavin Sharp [email: [e-mail]]]

(In reply to comment #16)
> 1. If the user has the addon installed search engine as the default and they
> uninstall the addon, their default search engine disappears. And you can't
> programmatically put it back when you uninstall, because it would be a
> duplicate of an existing engine.

Why does the "Default search engine disappear"? If you remove an addon with a search engine that overrides an app-shipped one, we should just revert to loading the app-shipped one.

> 2. For AMO installs, we would name the engine differently and if the user did
> not optin, they would have two engines in their list, Bing and Bing (NEW)

I don't understand this - you want to offer an addon that offers an additional search engine (Bing NEW), but make the search engine part optional? I don't think that's required by AMO policies. You can just unconditionally add Bing (NEW).

Comment 19

[:Gavin Sharp [email: [e-mail]]]

Attachment: patch

This changes the load ordering such that we prefer profile-installed engines over app-shipped engines. The concern with this is that people upgrading to Firefox 4 will suddenly start using old engines that were lying around in their profiles that they didn't even know existed, for whatever reason. I'm still very reluctant to do this.

Comment 20

[Mike Kaply [:mkaply]]

(In reply to comment #18)
> (In reply to comment #16)
> > 1. If the user has the addon installed search engine as the default and they
> > uninstall the addon, their default search engine disappears. And you can't
> > programmatically put it back when you uninstall, because it would be a
> > duplicate of an existing engine.
> 
> Why does the "Default search engine disappear"? If you remove an addon with a
> search engine that overrides an app-shipped one, we should just revert to
> loading the app-shipped one.

Not if they don't have exactly the same name.

I've found a workaround for this. On uninstall, if my search engine is the default, I add a new search engine and leave it behind. Unfortunately I have to name it differently.

> 
> > 2. For AMO installs, we would name the engine differently and if the user did
> > not optin, they would have two engines in their list, Bing and Bing (NEW)
> 
> I don't understand this - you want to offer an addon that offers an additional
> search engine (Bing NEW), but make the search engine part optional? I don't
> think that's required by AMO policies. You can just unconditionally add Bing
> (NEW).

No, I want to offer a search engine called "Bing" - If the user opts in, my Bing replaces the default Bing in every way.

I'm not going call my Engine Bing (NEW), I'm going to call it Bing.

Comment 21

[Mike Kaply [:mkaply]]

(In reply to comment #19)
> This changes the load ordering such that we prefer profile-installed engines
> over app-shipped engines. The concern with this is that people upgrading to
> Firefox 4 will suddenly start using old engines that were lying around in their
> profiles that they didn't even know existed, for whatever reason. I'm still
> very reluctant to do this.

The only issue here is Bing.

If a user installed a custom Yahoo that overrides the default on 3.6, when they go to 4, they'll still use Yahoo, won't they? If moving to 4 overrides every user installed search engine that happens to have a name that match an installed engine, we have a bigger problem.

The only issue here is what happens when an app engine comes it to being that never existed before...

Comment 22

[Andy Neal]

(In reply to comment #20)
> I'm not going call my Engine Bing (NEW), I'm going to call it Bing.

As a basic user of Firefox, if I see a search engine called the app-shipped name 'Bing', I expect it to be the TRUSTED app-shipped engine, not some sneaky one that has overwritten my install. Why are you trying to pull the wool over the eyes of users and trying to hide the origin of the engine? If you want a custom version, users should know and be well aware of its origin through its name.

The real bug here is that Firefox should not allow addon-installed and user-installed engines with the same name as app-shipped engines.

Comment 23

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Did you read mkaply's comments?  His users are specifically opting in to supporting a modified version of the search engine which supports his addon financially.

Comment 24

[Mike Kaply [:mkaply]]

> The real bug here is that Firefox should not allow addon-installed and
user-installed engines with the same name as app-shipped engines.

Actually a better situation is if the description field in the search engine was somehow made available to the user via tooltip or something else so they could see more information about the engine. It's quite difficult to convey "This is the Bing that you opted in to and was provided by Brand Thunder" in one search line. Separate bug.

Comment 25

[:Gavin Sharp [email: [e-mail]]]

The patch changes the load ordering to prefer user-installed (profile) search plugins over app-shipped search plugins. That means that if for whatever reason users have e.g. Google or Yahoo plugins in their profiles, then we'll start using them when they start using a build with that patch, rather than the shipped engines. That could be bad.

I'm much more comfortable just saying that app-installed engines can only be overridden by addon/"distro" shipped plugins (WONTFIXing this bug). That makes things harder for you to achieve your desired UX, but I'm not sure that should be considered a blocker for Firefox 4, since you have workarounds at your disposal.

Comment 26

[Mike Kaply [:mkaply]]

I do have workarounds, but they are even worse than what I originally did. The inability to access overridden search engines makes dealing with search engines in general a huge pain (for instance, I want to ship an engine called "Bing" in my addon but somehow be able to hide it and let the app "Bing" come through).

I'm having to do stupid things like naming engines "Bing " and "Bing  ". It's getting ridiculous.

I'll settle for a WONTFIX, but I still think what Firefox is doing is completely wrong.

Again, disregarding our situation completely, the fact that you install Bing from AMO and when you upgrade to Firefox 4 you're switched to Mozilla's Bing is a very bad bug. Whether it was intentional or not, it is called search hijacking, regardless of your change to the subject of this bug.

Comment 27

[Mike Beltzner [:beltzner, not reading bugmail]]

I think my recommendation here will be to prioritize user-installed search engines above all else, but have to think through the risk a bit more. Hope to have a call by tomorrow.

Comment 28

[Mike Beltzner [:beltzner, not reading bugmail]]

That indeed turns out to be my recommendation. Even at this late point, I think that we need to do what's right for users.

Adding QA and SUMO so they can keep an eye out: the deal here is that when Gavin makes this fix, any user-installed search engine that carries the same shortname (visible name of the search engine in the UI) will override the default search engines. Should this happen in ways that upset users, they can always reset the set of search engines to the default, thus removing their user-installed ones.

Comment 29

[Mike Beltzner [:beltzner, not reading bugmail]]

Juan/Cheng: please see comment 28

Comment 30

[:Gavin Sharp [email: [e-mail]]]

(In reply to comment #28)
> Should this happen in ways that upset users, they can
> always reset the set of search engines to the default, thus removing their
> user-installed ones.

No, that's not true. Resetting search engines doesn't remove any user-installed engines, it only restores the default ones. They'd need to manually remove the user-installed version, in which case we'll load the default again after a restart.

Comment 31

[:Gavin Sharp [email: [e-mail]]]

And actually, we also don't invalidate the cache when removing engines, so even if you do remove the user-installed one, we won't load the default one again until the cache is invalidated for some other reason (add another engine and restart, update and restart, etc.).

Comment 32

[:Gavin Sharp [email: [e-mail]]]

Attachment: patch

Comment 33

[:Gavin Sharp [email: [e-mail]]]

Hmm, I just found bug 344159, which explicitly changed this ordering to what it is now. I'll need to investigate that further.

Comment 34

[Mike Kaply [:mkaply]]

Gavin:

I own the CCK, so I can make whatever code changes are necessary to make this work.

Part of the CCK is that you can bundle searchengines. They are addon installed search engines (searchplugins) not app installed.

With these changes, the order is going to be user->addon->app, right?

Comment 35

[:Gavin Sharp [email: [e-mail]]]

With that patch, it will be addon/"distro"/user/app.

Do you recall why bug 344159 changed the relative order of app vs. user? There doesn't seem to be any rationale in the bug.

Comment 36

[Mike Kaply [:mkaply]]

I have no idea why bug 344159 did that. My only thought was that if a user had installed a search engine the CCK would win, but that doesn't make sense.

The CCK doesn't install engines, it uses a searchplugins directory, so those would win anyway.

On other note,  I've been thinking about this bug more, and I really don't get the problem here.

So as it stands today, a user would never a user installed Yahoo, Google, etc. because it's impossible to install a user search engine for an engine that exists in Firefox. And these engines have been there since day one.

The unique situation here is that we have a case where user could have installed an engine in Firefox 3.6 (Bing) that all of a sudden exists in 4.0.

So I don't understand your statement that users will all of a sudden get engines that they didn't have before. It's physically impossible for a user to have any engine installed that is named the same as a default engine (except Bing)

The only case where a  user could have a user installed engine that is the same as a system engine is Bing.

Am I missing something?

am

Comment 37

[:Gavin Sharp [email: [e-mail]]]

(In reply to comment #36)
> So I don't understand your statement that users will all of a sudden get
> engines that they didn't have before. It's physically impossible for a user to
> have any engine installed that is named the same as a default engine (except
> Bing)

Yes, it's impossible to get into that state via Firefox's add-engine mechanism, in builds that have the new search service (Firefox 2 and later). But it's possible for users to get into that state by manually copying plugins to their profile, or by third-party apps doing that, or by having installed those engines in pre-Firefox 2 builds, or in some other situations I may not have thought of.

Comment 38

[Mike Beltzner [:beltzner, not reading bugmail]]

I'd like to get this into b11 if possible.

Comment 39

[Justin Dolske [:Dolske]]

Pushed http://hg.mozilla.org/mozilla-central/rev/a55b45fb3d7c

Comment 40

[Steve Scott (pxbugz)]

Verified fixed in Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b12pre) Gecko/20110204 Firefox/4.0b12pre

Comment 41

[George Carstoiu]

Build identifier: Mozilla/5.0 (Windows NT 6.1; rv:2.0b12pre) Gecko/20110206 Firefox/4.0b12pre

Confirming issue is fixed.

Comment 42

[Sam Goldstein]

I have the same issue with my old browser's url search bar defaulted to Google, and not has been switched to Yahoo.  I've tried everything to get this back through the about:config portion of keyword:url, but no luck.  How can I get a solution for this?

Comment 43

[Vlad [QA]]

Verified fixed on Mozilla/5.0 (Windows NT 6.1; rv:2.2a1pre) Gecko/20110404 Firefox/4.2a1pre

Comment 44

[rh.breuer]

Windows 10 offers 2 lines with a firefox logo: One gives a window containing firefox with the default search engine bing, which I never chose, and apparantly more changes. If I click the other logo, a second window starts with the search engine Google I chose . Are there 2 versions of firefox on my desktop? Is there no risk to spoil the good one when I try to uninstall the other logo?

Comment 45

[Mark Banner (:standard8)]

(In reply to rh.breuer from comment #44)

Windows 10 offers 2 lines with a firefox logo: One gives a window containing firefox with the default search engine bing, which I never chose, and apparantly more changes. If I click the other logo, a second window starts with the search engine Google I chose . Are there 2 versions of firefox on my desktop? Is there no risk to spoil the good one when I try to uninstall the other logo?

Hi, sorry, but that's nothing to do with this issue that is long closed. It sounds like you really want our support site at https://support.mozilla.org/ where you can ask for help and they'll work with you to solve your issue.

Comment 46

[rh.breuer]

The spoilt thing might have come here as firefox 76 and then update to 79. Firefox 80 64bit german seems to be near.
If I got 2 versions by updating firefox, then both are firefox 79 64bit now. Or is it one mixture?

Comment 47

[Mark Banner (:standard8)]

(In reply to rh.breuer from comment #46)

The spoilt thing might have come here as firefox 76 and then update to 79. Firefox 80 64bit german seems to be near.
If I got 2 versions by updating firefox, then both are firefox 79 64bit now. Or is it one mixture?

Please stop commenting here, I have already said this is not an appropriate place. Please use the support site: https://support.mozilla.org/questions/new