Closed Bug 903082 Opened 7 years ago Closed 7 years ago

Add Yahoo as a general search provider for specified locales for Fennec

Categories

(Firefox for Android :: General, defect)

26 Branch
ARM
Android
defect
Not set
normal

Tracking

()

VERIFIED FIXED
Firefox 27
Tracking Status
firefox26 + fixed
firefox27 --- verified
b2g-v1.2 --- fixed

People

(Reporter: krudnitski, Assigned: blassey)

References

(Blocks 1 open bug)

Details

Attachments

(1 file)

Can we look at adding Yahoo as a general search provider for the following locales: en-US, en-GB, de, fr, es-ES.

(If we get en-CA miraculously in time, we would also want Yahoo included for that locale as well.)

Yahoo has provided us with 3 different tags in case we wanted to test or had multiple search access points. 

*Please let me know which are actually used, and whether this is enough information to go upon or if more info from Yahoo is required.*

tags:
mozilla_mobile_search
mozilla_mobile2_search
mozilla_mobile3_search

I would like to see Yahoo simply added to our list so that it falls below 'Google'.
  ie for en-US:
     Google
     <Bing> when we also get Bing added <- bug to be filed right after this
     Yahoo
     Amazon
     Twitter
     Wikipedia

Please test for both phones and tablets.

The release target is Fx 26, although if this has any possibility of being pulled into Fx 25, do let me know as I want to keep Yahoo informed.

Please ensure the communities for the affected locales are informed and if there are any serious concerns raised as a result of this change.

Thanks, Karen
Blocks: 863469
Duplicate of this bug: 857707
Are the codes here really random? Or should the be tied to position in the search list or something?

I'd personally go for "as trivial as we can get away with".
Assignee: nobody → francesco.lodolo
tracking-fennec: ? → 25+
Assignee: francesco.lodolo → nobody
While we reconfirm the string, I wanted to log one change to the order of providers. Please have them configured the following way:

     Google
     Yahoo
     <Bing> when we also get Bing added <- bug to be filed right after this
     Amazon (or locale equivalent)
     Twitter (or locale equivalent)
     Wikipedia (or locale equivalent)

Yahoo should be in the 2nd position.

Thanks!
The Yahoo! search plugin is actually already in the mobile tree:

http://mxr.mozilla.org/mozilla-central/source/mobile/locales/en-US/searchplugins/yahoo.xml

its just not turned on in the list to ship:

http://mxr.mozilla.org/mozilla-central/source/mobile/locales/en-US/searchplugins/list.txt

We want to add one more param to the list and add it?
We definitely need to update the icon (see bug 913926), and also decide if we need to differentiate phones from tablets (now we can do it).
No, there is no differentiation needed between tablet and phone (at least not currently). Do I need to go to Yahoo to get a new icon?
damn - never mind my last question. I obviously just followed that bug AFTER responding to this one :/
Blocks: 916835
Summary: Add yahoo as a general search provider for specified locales for Fennec → Add Yahoo as a general search provider for specified locales for Fennec
wrt Bug 916835: here are the Yahoo engine identifiers that we currently track in FHR.

        "yahoo",                      -- US
        "yahoo-NO",
        "yahoo-answer-zh-TW",
        "yahoo-ar",
        "yahoo-bid-zh-TW",
        "yahoo-br",
        "yahoo-ch",
        "yahoo-cl",
        "yahoo-de",                   -- DE
        "yahoo-en-GB",                -- GB
        "yahoo-es",                   -- ES
        "yahoo-fi",
        "yahoo-france",               -- FR
        "yahoo-fy-NL",
        "yahoo-id",
        "yahoo-in",
        "yahoo-it",
        "yahoo-jp",
        "yahoo-jp-auctions",
        "yahoo-mx",
        "yahoo-sv-SE",
        "yahoo-zh-TW",

Looks like only en-CA would be needed for the Yahoo part of Bug 916835.

In implementing this bug, please make sure that the search engine IDs match this list. If they don't, or if en-CA makes it, let me know.
yahoo-it is definitely not part of Fennec (it's used only for Desktop), so not sure about this list.
(In reply to Francesco Lodolo [:flod] from comment #9)
> yahoo-it is definitely not part of Fennec (it's used only for Desktop), so
> not sure about this list.

That list is "provider identifiers that FHR will record, should the browser report that a search was performed using that provider". It's static, based on our partnerships, not on which search engines we currently ship.
This is tracking Fennec 25 and week 2 just wrapped up. The window for this landing in 25 beta is rapidly closing especially with the Summit taking up part of a week. If this should not be tracking 25 please renominate so we can retriage.
Akin to bug 903084, is this now 26+?
yes, happy to have this push to 26
tracking-fennec: 25+ → ---
Assignee: nobody → blassey.bugs
Attachment #813856 - Flags: review?(mark.finkle)
Attachment #813856 - Flags: review?(mark.finkle) → review+
As implemented, both this and the Bing patch in Bug 903084 will result in searches being recorded in FHR.

If the localized search engine list matches the pattern in Comment 8 (which is how desktop works), then Bug 916835 is done and dusted, with only QA left to do.
https://hg.mozilla.org/mozilla-central/rev/1ab6b9a0939b
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 27
Let's get an uplift nom then for FF26.
No.

This doesn't work properly yet.

When conducting a search through this newly added engine, my query ends up being: '{searchterms}' as the literal that gets fired off to Yahoo search. Each and every query ends up being '{searchterms}'.

Searching for 'toronto' yields the URL.

http://search.yahoo.com/search?p={searchterms}&ei=UTF-8&fr=mozilla_mobile_search
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
The substitution is case-sensitive:
{searchterms}  -->  {searchTerms}
Status: REOPENED → ASSIGNED
Depends on: 903084
fixed by bug 903084
Status: ASSIGNED → RESOLVED
Closed: 7 years ago7 years ago
Resolution: --- → FIXED
Status: RESOLVED → VERIFIED
FYI: we may be bring in this into Fx26...
Comment on attachment 813856 [details] [diff] [review]
yahoo_search_plugin.patch

Adding symbolic uplift nom -- this is intertwined with work in Bug 903084 (also requested for uplift), so uplifter: be aware of what actually landed!
Attachment #813856 - Flags: approval-mozilla-aurora?
I guess same question from Flod in the Bing bug (bug 903084), this only added en-US but comment #0 mentions more than that locale.
(In reply to Aaron Train [:aaronmt] from comment #24)
> I guess same question from Flod in the Bing bug (bug 903084), this only
> added en-US but comment #0 mentions more than that locale.

The other locales are fr, de, en-GB, and es. All have been consulted. I need a little clarification on what is now expected from these locales. Do they simply add Bing and Yahoo! to their region.properties in the agreed upon order in comment #0 ? Do they need to add anything to list.txt?
Jeff - I expect we need patches for each of the locales to ensure bing and yahoo are added to the specified locale's search list. They should be able to use the en-US as the guide and ensure google, yahoo and bing order is reflected appropriate with the rest of the locale's providers following.
(In reply to Karen Rudnitski [:kar] from comment #26)
> Jeff - I expect we need patches for each of the locales to ensure bing and
> yahoo are added to the specified locale's search list. They should be able
> to use the en-US as the guide and ensure google, yahoo and bing order is
> reflected appropriate with the rest of the locale's providers following.

I suppose my main question is are we asking these locales to create localized xml search files, as well as changing the order in each of their region.properties files, or are they only making changes to region.properties and using what is in en-US as the default fallback? I need to know this before I begin filing bugs for each of these locales.

Also, has any documentation been written on how users can change their own default search preferences in Fennec?
No changes to default is being asked for this timeframe (Fx 26). Therefore users will only see two new providers show up in their list after Google.

Changing search preferences is HUGELY improved come Fx 26, moreso than what was there previously (when it was managed through add-ons). We can ensure Roland has an article handy though in the meantime. 

On how this is implemented. My understanding is that each locale has its own list (which has in the past been customized if they so desired). We need each of the targeted (listed in comment 0) locale's list modified to include bing and yahoo with the parameters provided in en-US, and the search order to reflect 1. Google, 2. Yahoo and 3. Bing (and the rest of the list as per the locale's preference). 

NOTE: in comment 0, I have Bing in 2. and Yahoo in position 3, but I would like these reversed please to best maximize our partnerships in this release.

Does that clarify?
(In reply to Karen Rudnitski [:kar] from comment #28)
> No changes to default is being asked for this timeframe (Fx 26). Therefore
> users will only see two new providers show up in their list after Google.
> 

So does that mean we don't need an uplift here?
Flags: needinfo?(blassey.bugs)
Attachment #813856 - Flags: approval-mozilla-aurora? → approval-mozilla-aurora+
(In reply to lsblakk@mozilla.com [:lsblakk] from comment #29)
> (In reply to Karen Rudnitski [:kar] from comment #28)
> > No changes to default is being asked for this timeframe (Fx 26). Therefore
> > users will only see two new providers show up in their list after Google.
> > 
> 
> So does that mean we don't need an uplift here?

Ignore that, I've approved this and bug 903082 - looks like the comment I was reading was just about defaults.
Flags: needinfo?(blassey.bugs)
I think we're waiting for bug 923795 before uplifting this.
This is also waiting on a verification that changing our current defaults doesn't actually change the defaults of existing users. I think the discussion on Tuesday hinted that we'd change the settings of existing users.

I don't recall if anyone actually took an action item on this.
Once this is uplifted, and verified as Axel mentions, could I get a list of the specific files the localized patches should contain? I assume this involves changes to region.properties and list.txt, but should the localized patches also contain localized search XML files?
To be clear - this change will *not* change any defaults. Google is still the default and that has not changed in this release. Timing for when we want to introduce a change in defaults still needs to be confirmed, but this bug doesn't touch the default. Just adds two more engines.
(In reply to jbeatty from comment #33)
> Once this is uplifted, and verified as Axel mentions, could I get a list of
> the specific files the localized patches should contain? I assume this
> involves changes to region.properties and list.txt, but should the localized
> patches also contain localized search XML files?

I basically need to reword this to ensure I am understanding the question.

Are you asking whether the search string provided will route users to the localized page? i.e. I am a DE user and I want DE-localized results? (and therefore if the search string used in en-US will route DE users to the appropriate results?)

If that's not the question, I'm out of my depth and also adding Kev to see if he can add some of his search expertise to help answer the exact query (so I know what needs to be resolved).
(In reply to Karen Rudnitski [:kar] from comment #35)
> (In reply to jbeatty from comment #33)
> > Once this is uplifted, and verified as Axel mentions, could I get a list of
> > the specific files the localized patches should contain? I assume this
> > involves changes to region.properties and list.txt, but should the localized
> > patches also contain localized search XML files?
> 
> I basically need to reword this to ensure I am understanding the question.
> 
> Are you asking whether the search string provided will route users to the
> localized page? i.e. I am a DE user and I want DE-localized results? (and
> therefore if the search string used in en-US will route DE users to the
> appropriate results?)
> 
> If that's not the question, I'm out of my depth and also adding Kev to see
> if he can add some of his search expertise to help answer the exact query
> (so I know what needs to be resolved).

Yes, that's the question, but minus your parenthetical phrase. Essentially, are the localizers to create new search files containing an already localized search string (e.g., google.mx for Mexican l10n)? The alternative is what you describe in your parenthetical phrase, where the user is searhes through the en-US search string, and the search provider redirects according to the locale.
So, to address Google directly, similar to desktop we shouldn't have any locale specific files in place. For other search engines, we may need to provide localized versions of plugin xml files if the service provider doesn't redirect automagically based on geo/accept-lang headers/prefs in a cookie, etc. Traditionally Yahoo has used country specific hostnames for search, but I believe that's changed. Similarly Bing does their own redirection, but we should confirm with the team there that that is true.

Ideally there are no changes to XML files, but I'll talk to Joanne and we'll verify.
(In reply to Kev Needham [:kev] from comment #37)
> So, to address Google directly, similar to desktop we shouldn't have any
> locale specific files in place. For other search engines, we may need to
> provide localized versions of plugin xml files if the service provider
> doesn't redirect automagically based on geo/accept-lang headers/prefs in a
> cookie, etc. Traditionally Yahoo has used country specific hostnames for
> search, but I believe that's changed. Similarly Bing does their own
> redirection, but we should confirm with the team there that that is true.
> 
> Ideally there are no changes to XML files, but I'll talk to Joanne and we'll
> verify.

Thanks for the clarification, Kev, and for verifying with Joanne. I'll wait for confirmation, then file bugs asking the localizers to add Bing and Yahoo! to list.txt (inclusion) for FF26 and to region.properties (change defaults) for 27.
(In reply to Kev Needham [:kev] from comment #37)
> So, to address Google directly, similar to desktop we shouldn't have any
> locale specific files in place. For other search engines, we may need to
> provide localized versions of plugin xml files if the service provider
> doesn't redirect automagically based on geo/accept-lang headers/prefs in a
> cookie, etc. Traditionally Yahoo has used country specific hostnames for
> search, but I believe that's changed. Similarly Bing does their own
> redirection, but we should confirm with the team there that that is true.
> 
> Ideally there are no changes to XML files, but I'll talk to Joanne and we'll
> verify.

Any word from Joanne here? Would like to file bugs tonight if possible to ensure this is added to 26.
We're still waiting on official confirmation, but we're going to take an educated guess here given our timing. Where we feel Bing redirects, we don't believe Yahoo will do so.

Therefore for the Yahoo changes, can we make the change to the XML files as well for the 4 remaining locales (en-GB, de, fr, es-ES)? We believe the info already exists since the desktop should have already done the work in the past.
I have confirmed that Yahoo does not redirect.  Thanks, Joanne
Depends on: 931109
Depends on: 931110
Depends on: 931111
Depends on: 931112
Quick note: IIUC, it is desired that this change makes it for Firefox 26, currently in Aurora, right? However, currently is only applied into central, unless I'm missing something. At least, the yahoo.xml last date in aurora is from June, while in central is yesterday.
We're waiting on Bug 923795 before uplifting this, per Comment 31.
Are locales expected to take bug 923795 as well or just use "mozilla_mobile_search"?
https://hg.mozilla.org/mozilla-central/file/default/mobile/locales/en-US/searchplugins/yahoo.xml#l14
Flags: needinfo?(krudnitski)
And another question about Yahoo: what about suggestions? We tried with the French searchplugin (bug 931109 comment 8) but it suggests only results meaningful for English.
Comment on attachment 813856 [details] [diff] [review]
yahoo_search_plugin.patch

now that uplift has happened, this needs to land on beta
Attachment #813856 - Flags: approval-mozilla-beta?
Blocks: 923795
Attachment #813856 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
I'm confused. This bugs is marked as VERIFIED FIXED, but it is yet to land in mozilla-beta and mozilla-aurora, so I guess localized versions shouldn't land in advance to en-US in those repositories (who's waiting for whom?). Also, flod's concerns seem valid to me. Are we going to provide useless search suggestions in localized searches?
(In reply to Ricardo Palomares from comment #47)
> I'm confused. This bugs is marked as VERIFIED FIXED, but it is yet to land
> in mozilla-beta and mozilla-aurora, 

the status (in this case VERIFIED FIXED) refers to the release that a patch landed in, as designated by Target Milestone (in this case 27). For the status of branches, see status-firefox-<version>, which will flip to fixed/verified once the patches are uplifted.

> so I guess localized versions shouldn't
> land in advance to en-US in those repositories (who's waiting for whom?).
> Also, flod's concerns seem valid to me. Are we going to provide useless
> search suggestions in localized searches?

We'll only get search suggestions for the default (i.e. position #1) search engine. Having useful search suggestions is a requirement for being default. Right now only Google satisfies that requirement, which is why it is the default. Yahoo should be in position #2.
(In reply to Francesco Lodolo [:flod] from comment #44)
> Are locales expected to take bug 923795 as well or just use
> "mozilla_mobile_search"?
> https://hg.mozilla.org/mozilla-central/file/default/mobile/locales/en-US/
> searchplugins/yahoo.xml#l14

Yes, we should be using the top2 MozParam from bug 923795
(In reply to Ricardo Palomares from comment #47)
> I'm confused. This bugs is marked as VERIFIED FIXED, but it is yet to land
> in mozilla-beta and mozilla-aurora,

Note that this is already in mozilla-aurora, since it landed originally in mozilla-central and moved on to aurora yesterday. 
http://hg.mozilla.org/releases/mozilla-aurora/file/default/mobile/locales/en-US/searchplugins

What we're missing right now is the landing on beta, which I think is not far away given the approval-mozilla-beta+

(In reply to Brad Lassey [:blassey] (use needinfo?) from comment #49)
> Yes, we should be using the top2 MozParam from bug 923795
Thanks, I'll clear Karen's needinfo then since you answered both questions.
Flags: needinfo?(krudnitski)
Whiteboard: [checkin-needed-beta]
I stumbled upon bug 264822 ("Yahoo" should be "Yahoo!" in included search plugin) and bug 335102.

Can you confirm if "Yahoo" is the right choice to go for new searchplugins? I see "Yahoo" used a lot currently, even in the title of this bug or yahoo.com, but I'd like to be sure.
needinfo to Karen for Flod's question
Flags: needinfo?(krudnitski)
A good question - and am awaiting to hear back from their team.
Flags: needinfo?(krudnitski)
flod- thanks for asking, although seems we're ok to just keep it to 'yahoo' and not add any exclamation mark.
Thanks Karen, then we probably need to close bug 264822 as INVALID (it just got a patch).
You need to log in before you can comment on or make changes to this bug.