300905 - Send "en-US" if "en" is first in language list and no other English dialects present

Status: RESOLVED FIXED

(Camino Graveyard :: Preferences, defect)

Description

[izzy mcmullin]

User-Agent:       Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8b3) Gecko/20050712 Camino/0.9a1+
Build Identifier: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8b3) Gecko/20050712 Camino/0.9a1+

only loads the gmail page in an asian language and i am unable to reload it in
english.

Reproducible: Always

Steps to Reproduce:
1.go to gmail login page
2.wha-la!
3.repeat un til you go insane

Actual Results:  
gmail is in an asian lang.

Expected Results:  
opened the english version of the page

Comment 1

[Samuel Sidler (old account; do not CC)]

What happened was Gmail changed what they were looking for in their prefs.
They're now looking for "en-US" instead of just "en". You can fix this now by
setting your dialect in prefs.js.

Some people will see this in Japanese, French, Chinese, whatever your second
language is in the intl.accept_languages preference of your prefs.js file.

Comment 2

[Samuel Sidler (old account; do not CC)]

Back -> pinkerton

More on this: See bug 161337, bug 281107, and bug 280814.

bug 161337 shows the initial checkin to "auto discover" this from system prefs.
But bug 280814 shows the reason we no longer discover the locale/dialect.

I think backing out part of bug 280814 (the part that removes the auto-discover
locale) would fix this. No?

Comment 3

[Samuel Sidler (old account; do not CC)]

CCing Bruce.

Bruce, is there another way we can do this? Maybe pull the locales and only use
them if it has not been specified. Sites that are looking for it, bypass the
first specified language and use the second one instead.

Comment 4

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Seems to me if the user just sets en-US, en, etc., in the System Prefs that
there wouldn't be a problem.  My accept string does include the dialect info,
and I didn't add it manually.

Comment 5

[Samuel Sidler (old account; do not CC)]

(In reply to comment #4)
> Seems to me if the user just sets en-US, en, etc., in the System Prefs that
> there wouldn't be a problem.  My accept string does include the dialect info,
> and I didn't add it manually.

Hmm... I'll check this, but if so, I suppose we should make a special case:

If a user specifies English but does not specify a locale, we specify the locale
for them.

Comment 6

[Samuel Sidler (old account; do not CC)]

Yep that's exactly the case. So, we need to say "set your locale if you want,
otherwise (if it's en) it's en-US."

Would that work?

Comment 7

[Bruce Davidson]

I don't understand exactly what it is that google is doing. Reporter, please can
you paste the accept-language header that Camino is sending to google that is
causing the problem. (Pages such as http://gemal.dk/browserspy/accept.php) will
tell you this information.)

Auto-detecting the locale was definitely the wrong idea, and I think we were
right to remove it. MacOS X's language selection includes dialect for some
language (e.g. mine has "British English" before "English", with "Australian
English" and "US English" both turned off). I can see other languages (e.g.
Portugese) with dialects present too.

I very much doubt that google are only looking for "en-us"; was the problem that
you didn't have "en" as a fall back? Samuel, can you post more details about any
testing you have done.

Comment 8

[Mark Mentovai]

The problem is that gmail is apparently understanding only en-US and not en
alone.  The localization defaults in Mac OS X include only en.  Unless the user
has modified the list to add a dialect (and this is a big "unless" for English
speakers), it won't include en-us.

It's Google's problem, but I've seen it on other sites in the past as well.  I
don't think it's unreasonable to transform en to en-US,en if English is present
in the list but no en dialects are.

Comment 9

[Bruce Davidson]

I'm not sure you're right that Google only accepts "en-us"...

If I remove en-gb from my language list, clear all my google cookies and log
into GMail with an accept-languages starting "en, fr, de, ..." I get a perfectly
sensible English interface and it sends me a www.google.com/Accounts cookie for
the language indicating "en".

It will be easier to debug if people could:
1. clear all mail.google.com, www.google.com (and possibly .google.com) cookies
before visiting GMail
2. Post their accept-language headers with the report
3. State what language they expected GMail to appear in (presumably the first in
the list above!) and what language they actually got it in.

Comment 10

[Mike Pinkerton (not reading bugmail)]

if i clear my google cookies (i had a "ja" cookie in there), then it all works
fine. I wonder if something else on their site mis-set the cookie and then gmail
just picked up on it.

Comment 11

[Samuel Sidler (old account; do not CC)]

Clearing cookies for me did not work last night. What seems to have happened is
Google looked for en-US (or any other dialect) and if it didn't see it, moved to
the second language in the list (which happened to be japanese for me).

I'm unable to test this now (I'm at work), but I'll try when I get home. Gmail
may have fixed the problem. If that's the case, users just need to clear their
cookies. I *hope* this is the case.

If not, I really don't see a problem with defining the dialect if it's not
already chosen. Gmail still works with en-GB/en-AU and I'm sure other sites
would act the same.

Comment 12

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

OK, I just created a fresh profile and tested.  First I set my languages to
English, French, Arabic (en fr;q=0.7 ar;q=0.3 from gemal.dk), then opened
Camino, went to Gmail, and logged in.  The whole process was in English, no
French involved.  Camino 2005071408 (v0.9a2), 10.3.9.

Comment 13

[Wevah (Nate Weaver)]

I was gonna say, I think that *Gmail* should really be checking for general 'en'
also, and not just 'en_US'. I'm 99.99% sure that this is/was a mistake on their end.

Comment 14

[Wevah (Nate Weaver)]

(In reply to comment #13)
> I was gonna say, I think that *Gmail* should really be checking for general 'en'
> also, and not just 'en_US'. I'm 99.99% sure that this is/was a mistake on
their end.

Oh, that's already been said. Remind me to not respond to bugmail when I'm not
fully awake.

Comment 15

[izzy mcmullin]

i emptied my cookies and now the site loads in english.

Comment 16

[Samuel Sidler (old account; do not CC)]

Gmail has fixed the problem. Unless there are other sites presenting this
problem, there's no reason for us to change our current approach.

Resolving... WORKSFORME.

Comment 17

[Simon Montagu :smontagu]

(In reply to comment #16)
> Gmail has fixed the problem. Unless there are other sites presenting this
> problem, there's no reason for us to change our current approach.
> 
> Resolving... WORKSFORME.

Reopening. See dependencies for some other sites presenting the problem, and see especially bug 327517 comment 8: "This behavior is a part of how ASP and IIS will handle locales, i.e., it's a Windows Server behavior..."

Comment 18

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Per smontagu's last comment, I think we're probably going to have to change to sending en-US when "English" is the top language and no other English dialects are listed, or something like that (i.e., comment 5/6/8).

I'm hoping than British, Australian, etc., English speakers actually change their language list appropriately and that we are really only special-casing lazy Americans anyway ;)

Comment 19

[Chris Lawson (gone)]

Tweaking summary to reflect what's actually going to happen, but let it be known that I believe this bug is entirely Tech Evangelism, and not something that Camino should be trying to work around. Note that all the dependencies are TE bugs.

Comment 20

[Stuart Morgan]

Attachment: fix

Converts any lonely 'en' to an 'en-US, en' pair.

Comment 21

[Maximilian Ruderburg [:max] (Arctic Revolution)]

Comment on attachment 246245 [details] [diff] [review]
fix

(Since I haven't given review before, treating my review as a second-r.)

r=me on the code, but I haven't run it through a functional test.

Comment 22

[Nick Kreeger]

Comment on attachment 246245 [details] [diff] [review]
fix

Looks good to me. Are we going to have to worry about other language packs other than en-US in the future?

Comment 23

[Stuart Morgan]

Checked in on trunk and MOZILLA_1_8_BRANCH.