776781 - Add locale support to search suggestions

Status: RESOLVED INCOMPLETE

(Firefox for Android Graveyard :: General, defect)

Description

[Stefan Arentz | :st3fan | ⏰ EST | he/him]

The search suggestions currently default to english. Google does however support suggestions in other locales through the "hl" parameter.

% curl "https://www.google.com/complete/search?client=safari&q=bo&hl=en" 
["bo",[["boost mobile","","0"],["body central","","1"],["bodybuilding","","2"],["boat trader","","3"],["bofa","","4"],["bolt bus","","5"],["bonnaroo","","6"],["bones","","7"],["boston globe","","8"],["body by vi","","9"]],{"k":1,"q":"1EQTWwrigKmL3EMoyQtTay5hTu4"}]%                                                 

% curl "https://www.google.com/complete/search?client=safari&q=bo&hl=fr"
["bo",[["bon coin","","0"],["boulanger","","1"],["bouygues","","2"],["boursorama","","3"],["booking","","4"],["bon prix","","5"],["bonjour madame","","6"],["booska p","","7"],["botanic","","8"],["bob marley","","9"]],{"k":1,"q":"1EQTWwrigKmL3EMoyQtTay5hTu4"}]

Comment 1

[Stefan Arentz | :st3fan | ⏰ EST | he/him]

Attachment: Patch to add locale support to the search suggestions

Here is a patch to add basic locale support to the search suggestions. I have tested this by switching my phone's language to french. French suggestions are presented then.

The patch simply adds the "hl" parameter to the query sent to Google. I understand that this is likely not the correct way to go. I did try to modify the suggestTemplate in mobile/android/chrome/content/browser.js but it seems all the code around that is not able to deal with extra template parameters.

Maybe it is a good idea to simplify that code because there is a huge amount of code and machinery going there to just grab a simple URL template?

Comment 2

[Stefan Arentz | :st3fan | ⏰ EST | he/him]

Also, I would not mind working with someone to turn that patch into something that can be approved and included in Fennec.

Comment 3

[Brian Nicholson (:bnicholson)]

One option might be to change the OpenSearch XML (mxr.mozilla.org/mozilla-central/source/mobile/locales/en-US/searchplugins/google.xml).

Specifically, you could try changing:
  template="https://www.google.com/complete/search?client=firefox&q={searchTerms}"
to:
  template="https://www.google.com/complete/search?client=firefox&q={searchTerms}&hl={moz:locale}"

Comment 4

[Stefan Arentz | :st3fan | ⏰ EST | he/him]

Attachment: Patch to add locale support to the search suggestions v2

As per Brian's suggestion this patch now add the hl={moz:language} to the search description file. I'm not totally convinced that the sugestion url template should be part of the opensearch descriptor though.

Comment 5

[Brian Nicholson (:bnicholson)]

(In reply to Stefan Arentz [:st3fan] from comment #4)
> Created attachment 645265 [details] [diff] [review]
> Patch to add locale support to the search suggestions v2
> 
> As per Brian's suggestion this patch now add the hl={moz:language} to the
> search description file. I'm not totally convinced that the sugestion url
> template should be part of the opensearch descriptor though.

You shouldn't need to make any changes in Java. When the template is retrieved [1], it calls engine.getSubmission(), which gets the search URL for that term (in this case, the term is "__searchTerms__"). All placeholders, including {moz:locale}, will automatically be replaced with getSubmission(). We then do another substitution in Java, which replaces "__searchTerms__" with the actual query.

[1] http://mxr.mozilla.org/mozilla-central/source/mobile/android/chrome/content/browser.js#5713

Comment 6

[Stefan Arentz | :st3fan | ⏰ EST | he/him]

Oh but is moz:locale the right thing to use? I think google expects a two letter language code, not a locale specifier like en_CA. I will investigate :-)

Comment 7

[Stefan Arentz | :st3fan | ⏰ EST | he/him]

I tried adding {moz:locale} to the template URI but that does not follow the system locale. I switched my phone to French but Fennec runs in English and thus passes en-US to moz:locale. This is getting complicated. Not sure what the right thing to do is now.

Comment 8

[:Gavin Sharp [email: [e-mail]]]

See bug 343849. We explicitly removed this parameter in Desktop Firefox because Google's locale detection did the right thing in more cases. I'm sure the situation may be different on Mobile, but choosing which locale to use and finding a mapping between the various locale codes isn't a simple problem.

Comment 9

[Stefan Arentz | :st3fan | ⏰ EST | he/him]

(In reply to :Gavin Sharp (use [e-mail] for email) from comment #8)
> See bug 343849. We explicitly removed this parameter in Desktop Firefox
> because Google's locale detection did the right thing in more cases. I'm
> sure the situation may be different on Mobile, but choosing which locale to
> use and finding a mapping between the various locale codes isn't a simple
> problem.

Ok that makes sense. How do they do the locale detection though? Do we need to send an Accept-Locale header with the request?

Comment 10

[:Gavin Sharp [email: [e-mail]]]

I don't know how their locale detection works, it seems to be mostly IP geolocation (but they may also use Accept-Lang, which we already send this based on the build's locale).

Comment 11

[BMO Automation]

We have completed our launch of our new Firefox on Android. The development of the new versions use GitHub for issue tracking. If the bug report still reproduces in a current version of [Firefox on Android nightly](https://play.google.com/store/apps/details?id=org.mozilla.fenix) an issue can be reported at the [Fenix GitHub project](https://github.com/mozilla-mobile/fenix/). If you want to discuss your report please use [Mozilla's chat](https://wiki.mozilla.org/Matrix#Connect_to_Matrix) server https://chat.mozilla.org and join the [#fenix](https://chat.mozilla.org/#/room/#fenix:mozilla.org) channel.