670450 - Google search from about:home should not reveal anything about user's UI locale in URL

Status: RESOLVED DUPLICATE of bug 670451

(Firefox :: Search, defect)

Description

[Reşat SABIQ (Reshat)]

User Agent: Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0
Build ID: 20110615151330

Steps to reproduce:

Searched from about:home


Actual results:

Locale representing the locale of the Firefox installer used was indicated in the URL.


Expected results:

User's most preferred locale should be used instead (based on Accept-Language (and navigator.language)).

P.S. I'm reporting against FX 5, but suspect the same is true for Trunk.

Comment 1

[Reşat SABIQ (Reshat)]

Completely dropping rls HTTP param is an equally good, and perhaps even a better solution.

Comment 2

[Boris Zbarsky [:bzbarsky]]

I have no idea what this has to do with HTTP networking....

Comment 4

[Kev Needham [:kev]]

The "rls" attribute is used by Google affiliates, of which Mozilla is one. The locale included in this attribute has no bearing on the content returned by queries to Google, and is included for reporting only. The rls and client codes are required, and include information that is sent in the accept-language http headers, which Google uses in conjunction with GeoIP lookups or a user pref set in a cookie to set the language results are returned in.

The attributes as they appear in Firefox are by design.

Comment 5

[Boris Zbarsky [:bzbarsky]]

> and include information that is sent in the accept-language http headers

This bug report is about the fact that the rls value being sent does NOT match the Accept-Language HTTP header, and in particular that it's leaking personal information that in other situations we purposefully do not give out to web pages.

Comment 6

[Boris Zbarsky [:bzbarsky]]

In particular, the string seems to be hardcoded to the build-time locale in brand.properties, whereas it should probably be depending on the intl.accept_languages pref.

Comment 7

[Kev Needham [:kev]]

(In reply to comment #5)
> This bug report is about the fact that the rls value being sent does NOT
> match the Accept-Language HTTP header, and in particular that it's leaking
> personal information that in other situations we purposefully do not give
> out to web pages.

It's not supposed to match, although in a number of cases it does. What it's supposed to be is the locale code we use for the build. Apologies if this wasn't clear. Per comment #4, the parameter has nothing to do with the results generated, it's to facilitate reporting on queries generated by a particular Mozilla locale.

If this is a privacy concern, then it should probably be reported as such, but based on the initial report that wasn't my interpretation. It's not information that's sent anywhere other than Google, but if we want a privacy review, we can do that, it just wasn't clear from comment #0, which seemed to be focused on results being generated. Adding Alex and Sid for comment.

Comment 8

[Kev Needham [:kev]]

Ack. Want to flag this as a privacy concern for legal, but the bug gets marked confidential, which it shouldn't. Leaving in search and assigning to Alex for comment.

Comment 9

[Sid Stamm [:geekboy or :sstamm]]

kev, what do we gain from sending the rls parameter?  Is there any reason we shouldn't drop it or synchronize it with the Accept-Language header?

Comment 10

[:Gavin Sharp [email: [e-mail]]]

There are three components to the "rls" parameter's value: {moz:distributionID}, {moz:locale}, and {moz:official}.

- {moz:distributionID} is an arbitrary string ("org.mozilla") that is technically customizable at build-time when someone builds Firefox, but in practice is always going to be the same because as far as I know, no one really uses the build time parameter that controls its value (and so everyone just ends up getting the default string).
- {moz:locale} is the build-time locale, as mentioned here.
- {moz:official} indicates whether or not the build is using official branding

Comment 11

[Reşat SABIQ (Reshat)]

(In reply to comment #7)
> It's not
> information that's sent anywhere other than Google, but if we want a privacy
> review, we can do that, it just wasn't clear from comment #0, which seemed
> to be focused on results being generated.
In a rush, I forgot to add [privacy] keyword originally, but added it now.

(In reply to comment #6)
> In particular, the string seems to be hardcoded to the build-time locale in
> brand.properties, whereas it should probably be depending on the
> intl.accept_languages pref.
Please note that there are related but 2 distinct bugs, which is why i started with 2 bugs (pardon my tautology): 1 bug per issue. The issue in this bug (search from about:home) is that the build-time value is used. It may be from brand.properties, or it may even be from update.locale file: might be worth checking.
The issue in bug 670451 is that the locale from general.useragent.locale pref ("dynamic" value, and not the build-time value) is used in search from search bar. When this bug is resolved, i'll try verifying that the other bug is resolved as well: it's OK from my viewpoint to fix both in 1 bug, as long as everybody is aware that there are actually 2 issues, and apparently 2 places to fix.

Comment 12

[Reşat SABIQ (Reshat)]

(In reply to comment #11)
> it may even be from update.locale file
Actually, I have verified that the value does NOT come from update.locale file (by changing it manually and then searching again).

Comment 13

[Alex Fowler]

Asking Tom to review this and figure out an action plan...

Comment 14

[Tom Lowenthal]

I think Sid's questions are key to decide how to proceed. Waiting for Kev.

Comment 15

[Kev Needham [:kev]]

Sorry, I hadn't realized this was still waiting on an answer. The RLS parameter as it is constructed is what provide with search queries to Google as part of the agreement we have with them. The rls query identifies a search query as coming from an Mozilla-branded build of Firefox (the org.mozilla and official identify it as a Mozilla-distributed build that uses official branding), along with the locale of the build the query was generated from. The locale information is not used for any language detection or page rendering, and should not vary from the value generated at build time. It is included to give us some flexibility around locale defaults (e.g. excluding a given locale without the need to manage multiple RLS codes), and to provide aggregate reporting of query volumes.

Comment 16

[Reşat SABIQ (Reshat)]

(In reply to Kev Needham [:kev] from comment #15)
> The locale information is not used for any language
> detection or page rendering
> It is included to give us some flexibility around locale
> defaults (e.g. excluding a given locale without the need to manage multiple
> RLS codes), and to provide aggregate reporting of query volumes.
It appears to me there's no pressing reason to keep exposing the build-time locale. Especially, because for locales with a small number of users, this leads to making those users easily fingerprintable.
There'd be no harm done to Mozilla or Google if the reports were based on Accept-Language instead of the UA's build-time locale, but it would allow providing better privacy (especially to users of locales with a relatively small number of users). If it's a matter of a change in an agreement, then this could be a part of the next renewal of the agreement: commercial agreements usually do get renewed or tweaked once in a while. Besides, at a minimum, stuff like that could be handled over https, if it's really crucial to have (and https could go straight to Mozilla, if need be).

I would like to kindly request a decision on this issue by an official Mozilla Foundation body, rather than just somebody's personal judgement. I'm not sure, but perhaps the L10N team/org (as opposed to just a person from that team/org) could make a formal decision.

P.S. I realize that if your build-time locale is en-US, and your Accept-Locale is the same, there's no difference to you. But to illustrate to you the importance of this (particularly for some users), let's consider the other extreme. Imagine you use a locale used by n users in total. If n is a relatively low number, like 10, or 100, or 1000, and your build-time locale is travelling all over the Internet wherever you are, IMHO you are left with barely any privacy at all: every query you make (and most users make at least a few every day) discloses that you are 1 of only n users around the world even without any cookies being involved. Not showing understanding at least in this scenario, is equivalent to (Mozilla's (and Google's)) discouraging usage of localizations with a relatively small number of users. Is this really what the world has come to: we want to be able to generate our reports more easily, and if some users privacy goes down the drain because of this "more easily" consideration so be it? I hope not.

Comment 17

[Reşat SABIQ (Reshat)]

There has been a change in behavior relevant to searching from about:home (at least as of Firefox 21).
Before: locale revealed wasn't based on (couldn't be controlled by) general.useragent.locale pref value.
Currently: locale value revealed by about:home searches is controlled by general.useragent.locale pref value, but (unlike the issue in bug 670451) the locale revealed by about:home searches is unchangeable for the duration of the process.

P.S.
a. I.e., if the user changes the value of general.useragent.locale (e.g., from l1 to l2) after starting the app process, the value revealed by about:home searches after the change until the app is restarted is still l1, whereas the value revealed by search bar searches for the same timeframe is l2. 
b. After the app is restarted, both kinds of searches reveal the new locale value (l2).
P.P.S. So, the discrepancy between the searches from about:home vs. searches from the search bar has become less likely, but it still is very much possible. More importantly, a locale value is still revealed in the URL in these 2 workflows (as opposed to having been dropped from the URL) and, assuming a value has to be a part of the HTTP URL, the values revealed (both before and after re-starting the app) still aren't based on Accept-Language (and navigator.language).
P.P.P.S. IMHO, the following TODOs are worth considering (IMHO, 3. would be the best to go for, but 1. would still be a useful intermediate step if it's easier to implement (for the short-term), and 2. is as good as 3. except that it involves duplication (of an HTTP header by a URL param)):
1. Ensuring that the locale values revealed in URLs by various search workflows are always in synch between each other.
2. Ensuring that the locale values revealed in URLs by various search workflows are always in synch with Accept-Language (and navigator.language).
3. Dropping the locale values revealed in URLs by various search workflows once Accept-Language is used instead.

Comment 18

[:Gavin Sharp [email: [e-mail]]]

Bug 900865 and other related about:home changes have effectively removed the distinction between this bug and bug 670451.