529353 - Google uses the ambiguous/language family language code for Norwegian, so safebrowsing pages are in English

Status: RESOLVED FIXED

(Camino Graveyard :: Security, defect)

Description

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

The language code for Norwegian (Bokmål) is "nb-NO" and for Norwegian (Nynorsk) is "nn-NO".  There is no language code "no" (or "no-NO"), except for errors on Mac OS X < 10.4 and on google.com.

As a consequence, when our Norwegian users try to use any of our safebrowsing UI that goes to Google web pages, they get English pages instead of Norwegian:

Report unblocked phish:
  http://www.google.com/safebrowsing/report_phish/?tpl=generic&continue=http://caminobrowser.org/security/report-submitted.php&url=http://caminobrowser.org/documentation/security/safebrowsing/&hl=nb-no

Report incorrectly blocked non-phish:
  http://www.google.com/safebrowsing/report_error/?tpl=generic&continue=http://caminobrowser.org/security/report-submitted.php&url=http://caminobrowser.org/documentation/security/test-phishing/&hl=nb-no

Diagnostic for blocked malware site:
  http://google.com/safebrowsing/diagnostic?tpl=Camino&site=http://caminobrowser.org/documentation/security/test-malware/&hl=nb-no

In all cases, Google wants "&hl=no" instead. Safari has the same problem as we do. Firefox has the same problem with the incorrectly-blocked non-phish, but somehow manages to make the unblocked phish work:

Incorrectly blocked non-phish: 

http://nb-NO.phish-error.mozilla.com/?hl=nb-NO&url=http%3A%2F%2Fwww.mozilla.com%2Ffirefox%2Fits-a-trap.html 

which redirects to 

http://www.google.com/safebrowsing/report_error/?tpl=mozilla&continue=http%3A%2F%2Fwww.google.com%2Ftools%2Ffirefox%2Ftoolbar%2FFT2%2Fintl%2Fen%2Fsubmit_success.html&hl=nb-NO&url=http%3A%2F%2Fwww.mozilla.com%2Ffirefox%2Fits-a-trap.html

and shows up in English (note the "&hl=nb-NO").

Unblocked phish:

http://nb-no.phish-report.mozilla.com/?hl=nb-NO&url=http%3A%2F%2Fbrowserspy.dk%2Fbrowser.php

which redirects to

http://www.google.com/safebrowsing/report_phish/?tpl=mozilla&continue=http%3A%2F%2Fwww.google.com%2Ftools%2Ffirefox%2Ftoolbar%2FFT2%2Fintl%2Fno%2Fsubmit_success.html&hl=no&url=http%3A%2F%2Fbrowserspy.dk%2Fbrowser.php

and shows up in Norwegian (note the "&hl=no").

It would be useful if someone could file a bug against Google's language detection code (note the problem shows up all over google.com, for instance see the language select at the bottom of http://www.google.com/support/websearch/bin/answer.py?answer=106318&hl=nb-no).

However, I wondered if it was possible for us to add some code, say in http://mxr.mozilla.org/seamonkey/source/camino/src/safebrowsing/SafeBrowsingListManager.mm#493, to transform "nb-no" into "no" to make the pages we link to directly work correctly while we wait for Google to learn that "no" is not a proper language code?

Comment 1

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Stuart noted the RFCs do still contain "no" as the ambiguous or "language family" code for Norwegian, so Google's not completely in the wrong here.

Comment 2

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Attachment: Manipulate the language code for Norwegian

To answer my own question, yes, we can.  Now, what did I do wrong in doing so?

Comment 3

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Attachment: v2, accounts for the way the lang code changes depending on the lproj name

I re-read comment 0 and noted that all my urls there had "nb-no" in them, but testing today they only had "nb" in them.  Apparently the language code the OS reports depends on the name of the .lproj, and much to Marcello's dismay, we've been switching back and forth between old-style and new-style (ISO) lproj names.  This new patch accounts for either "nb" and "nb-no" being returned from the OS.

Comment 4

[Chris Lawson (gone)]

Comment on attachment 416624 [details] [diff] [review]
v2, accounts for the way the lang code changes depending on the lproj name

Looks good and results in Norwegian (or something that looks like Norwegian to someone whose familiarity with Norwegian consists pretty much entirely of "A møøse once bit my sister") with all three lproj variations (nb_NO, Norwegian, and "no.lproj"). I'd say that's good enough for an r+.

Comment 5

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Comment on attachment 416624 [details] [diff] [review]
v2, accounts for the way the lang code changes depending on the lproj name

Pink, this should be fairly self-explanatory.  I managed to code it, after all ;)

Comment 6

[Mike Pinkerton (not reading bugmail)]

Comment on attachment 416624 [details] [diff] [review]
v2, accounts for the way the lang code changes depending on the lproj name

sr=pink

Comment 7

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Landed on cvs trunk and CAMINO_2_0_BRANCH.