337719 - For the Google search plugin, change country-specific TLD URLs to www.google.com

Status: RESOLVED FIXED

(Firefox :: Search, defect)

Description

[Basil Hashem [:baz]]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3

Across Firefox builds for different locales, the Google search plugin currently uses a variety of URLs for search. Some of the search URLs are www.google.com while others are country-specific top-level domains such as www.google.bg.

During a recent Fx 1.5.0.3 build exercise for Google CJKT distributiosn (where Google is the default search instead of Yahoo), it was discovered that Google wants to have all searches going to www.google.com instead of the country-specific TLD.

Concerns were raised about whether they really wanted traffic from say Taiwan to be going to www.google.com and potentially crossing the Pacific on every search but apparently they have implmented some sort of global DNS load balancing solution as well as IP detection for routing to the nearest server.

In particular, all locale builds should be checked but according to a cursory glance, the following is a preliminary list from Axel Hecht of country-specific TLD routing locales that need changing:

bg/browser/searchplugins/google-bg.src:   action="http://www.google.bg/search"
da/browser/searchplugins/google-dk.src:   action="http://www.google.dk/search"
de/browser/searchplugins/google-de.src:   action="http://www.google.de/search"
en-GB/browser/searchplugins/google-en-GB.src:   action="http://www.google.co.uk/search"
es-AR/browser/searchplugins/google-ar.src:   action="http://www.google.com.ar/search"
es-ES/browser/searchplugins/google-es-ES.src:   action="http://www.google.es/search"
fi/browser/searchplugins/google-fi.src:   action="http://www.google.fi/search"
fy-NL/browser/searchplugins/google-nl.src:   action="http://www.google.nl/search"
ga-IE/browser/searchplugins/google-ga-IE.src:   action="http://www.google.ie/search"
gu-IN/browser/searchplugins/google-gu.src:   action="http://www.google.co.in/search"
it/browser/searchplugins/google-it.src:   action="http://www.google.it/search"
ja-JP-mac/browser/searchplugins/google-jp.src:   action="http://www.google.co.jp/search"
ja/browser/searchplugins/google-jp.src:   action="http://www.google.co.jp/search"
ko/browser/searchplugins/google-ko.src:   action="http://www.google.co.kr/search"
lt/browser/searchplugins/google-lt.src:   action="http://www.google.lt/search"
mn/browser/searchplugins/google-mn.src:   action="http://www.google.mn/search"
nb-NO/browser/searchplugins/google-NO.src:   action="http://www.google.no/search"
nl/browser/searchplugins/google-nl.src:   action="http://www.google.nl/search"
nn-NO/browser/searchplugins/google.src:   action="http://www.google.no/search"
pl/browser/searchplugins/google-pl.src:   action="http://www.google.pl/search"
pt-BR/browser/searchplugins/google-pt-BR.src:    action="http://www.google.com.br/search"
ro/browser/searchplugins/google-ro.src:   action="http://www.google.ro/search"
ru/browser/searchplugins/google-ru.src:   action="http://www.google.ru/search"
zh-TW/browser/searchplugins/google-zh-TW.src:   action="http://www.google.com.tw/search"

Here is the official Google request for making this change.

From:  	"Patrick Brady" <[e-mail]>
To:  	"Basil Hashem" <[e-mail]>
Cc:  	[e-mail], [e-mail], [e-mail]
	
Subject: Re: Changing the default Google plugin search URL
Date:    Thu 05/11/06 07:11 PM
	
Basil,

Our feeling is that the best implementation is to send the traffic to www.google.com across the board.  I wanted to check with others on our team before changing the original bits, and we have confirmed that this is the best practice.  If you are so able, please update the existing locales to send traffic to google.com.

Send any questions or concerns my way.

Thanks,

Patrick

Reproducible: Always

Comment 1

[Axel Hecht [:Pike]]

If google has an idea on what the search query should look like, here is a 
stab at uncommon params given by some of our existing plugins:

ca/browser/searchplugins/google-ca.src:  <input name="hl" value="ca">
de/browser/searchplugins/google-de.src:<input name="meta" value="lr%3Dlang_de">
eu/browser/searchplugins/google-eu.src:<input name="hl" value="eu">
it/browser/searchplugins/google-it.src:<input name="hl" value="it">
ja-JP-mac/browser/searchplugins/google-jp.src:<input name="hl" value="ja">
ja-JP-mac/browser/searchplugins/google-jp.src:<input name="lr" value="lang_ja">
ja/browser/searchplugins/google-jp.src:<input name="hl" value="ja">
ja/browser/searchplugins/google-jp.src:<input name="lr" value="lang_ja">
ko/browser/searchplugins/google-ko.src:<input name="lr" value="lang_ko">
sl/browser/searchplugins/google-sl.src:<input name="hl" value="sl">
zh-TW/browser/searchplugins/google-zh-TW.src:<input name="hl" value="zh-TW">


The names differ, too.


bug 335460 looks like it needs input from here, too.

To clarify the request, is this a "nice to have" or something we want, or 
something we need, and which branches are affected?

I'm wondering if we need or should have distinct search plugins for google at 
all.
Gavin, do we use the Description field of the search plugin?

Comment 2

[:Gavin Sharp [email: [e-mail]]]

(In reply to comment #1)
> Gavin, do we use the Description field of the search plugin?

No, the description field isn't currently exposed in the UI (or on nsISearchEngine, I just realized, filed bug 337751 for that).

Comment 3

[Basil Hashem [:baz]]

I've asked Google for a list of definitive URL parameters that we should be using and if some of these values are dependent upon locale, etc...I will add these to the bug as soon as I have them.

I would rate this feature request as something we want (higher rating that nice to have) because it simplifies and unifies the way the Google search plugin works.

Comment 4

[Phil Ringnalda (:philor)]

(In reply to comment #1)
> de/browser/searchplugins/google-de.src:<input name="meta" value="lr%3Dlang_de">

Actually, that one's gone from 1.8, though not 1.8.0, because it causes the bug 333444 problem of ignoring the user's cookie-set language preference.

Comment 5

[Basil Hashem [:baz]]

Here are the URL parameters that the Google search plugin should generate:

q=MY+IMPORTANT+SEARCH - Search Term
start=0               - This is optional.  Mozilla should omit this parameter.
ie=utf-8              - Input encoding.  Mozilla should include this value.
oe=utf-8              - Output encoding.  Mozilla should include this value.
client=firefox-a      - Core Fx distribution with revenue sharing Mozilla should include this value.
rls=org.mozilla:en-US:official - Will vary based on locale. Mozilla should include this value.

In some selected locales, there also appear:
lang=lang_ja          - Mozilla can omit this parameter.
hl=ca                 - Interface language.  Mozilla should include this parameter. Value is language portion of locale (ja for Japanese).  For those languages that differ by country, should include both (pt-BR, zh-TW).
meta="lr=lang_de"     - Mozilla can omit this parameter.
lr=lang_ja            - Language restrict.  Mozilla can omit this parameter.

Comment 6

[Axel Hecht [:Pike]]

Actually, as has been mentioned in bug 338256, google pays attention to 
intl.accept_languages, I verified this here in amsterdam, going to google.com,
getting google nederlands to serve the content in either german or french,
depending on my pref. YAY! I guess it's time to rip out all our plugins in
favour of one.

Comment 7

[Axel Hecht [:Pike]]

For reference, I filed bug 340693 on intl search suggest results.

Comment 8

[Axel Hecht [:Pike]]

In bug 343849, we removed the explicit argument hl from the google search plugin.

We still need to resolve bug 340693 in order to find out how many variants of the google plugin we actually need.

Comment 9

[Basil Hashem [:baz]]

Can we decide if this bug needs action before Firefox 3 closes?

Comment 10

[Axel Hecht [:Pike]]

Right now, Japan is the only locale which has a separate google plugin, for the Japanese suggest corpus.

Kev is tracking a bunch of changes in behaviour on the google side that happened lately, I'm not sure if there's any action needed to make up for that. In particular, accept_lang seems to have stopped working.

This particular bug can be closed, though, I think.

Kev?

Comment 11

[Axel Hecht [:Pike]]

Kev, ping?

Comment 12

[Basil Hashem [:baz]]

I'm going to close this bug - any additional search suggest requests of Google is being tracked in bug 340693.