269469 - google.com - doesn't recognize firefox as supporting UTF-8

Status: RESOLVED WORKSFORME

(Tech Evangelism Graveyard :: English US, defect)

Description

[Jungshik Shin]

1. Type '가각' (this is UTF-8 : U+0AC00 U+AC01. In UTF-8, they're 0xEA 0xB0 0x80
0xEA 0xB0 0x81) in google search box (on the upper right side)
2. You'll be moved to 

http://www.google.co.kr/search?q=%EA%B0%80%EA%B0%81&start=0&start=0&ie=utf-8&oe=utf-8&client=firefox-a&rls=org.mozilla:ko-KR:official

3. Press 'news' button in the search result 

4. The url of the page  popping up is 

http://news.google.co.kr/news?q=%EA%B0%80%EA%B0%81&hl=en&lr=&newwindow=1&client=firefox-a&rls=org.mozilla:ko-KR:official&sa=N&tab=wn

Note that '%EA%B0%80%EA%B0%81' is interpreted as ISO-8859-1 and the search box
in the search result has 6 characters 'ê°€ê°' (instead of 2 Korean syllables)
that are represented as 0xEA 0xB0 0x80 0xEA 0xB0 0xB1 in ISo-8859-1. This
happens because news.google.co.kr (or news.google.com) doesn't recognize firefox
as supporting UTF-8. (the main google web site, www.google.com does this to
w3m-m17n and konqureror). 

If you chop off the URL to get rid of '&client=firefox.....' (i.e. just use
http://www.google.co.kr/search?q=%EA%B0%80%EA%B0%81&start=0&start=0&ie=utf-8&oe=utf-8),
pressing 'news' button in the search result page leads you to 

http://news.google.co.kr/news?q=%EA%B0%80%EA%B0%81&hl=en&lr=&newwindow=1&sa=N&tab=wn

in which '%EA ... %81' is correctly interpreted as UTF-8. 

I thought this was entirely google's fault and we don't have anything to do on
our side. Although it's still google's fault, I realize that we can avoid the
problem by dropping 'client=firefox....'. Without it, google does the right
thing recognizing gecko-based browsers as supporting UTF-8.

Comment 1

[Jungshik Shin]

s/ISO-8859-1/Windows-1252/

Comment 2

[Felix Miata]

Conforming summary to TFM item 10 at 
http://www.mozilla.org/projects/tech-evangelism/site/procedures.html#file-new

Reporter, when you file a TE bug, please follow the reporting guidelines.

Comment 3

[Jungshik Shin]

It doesn't matter whether it's co.kr or com, but to get a better attention, I'd
rather make it google.com.

Comment 4

[Jungshik Shin]

Related bug is bug 269600. 

Comment 5

[Jungshik Shin]

Just to make clear that it doesn't matter whether it's ko-KR or en-US and
whether it's google.com or google.co.kr, I'm listing two URLs one with
'client=...' and without. The former interprets '%EA%B0%80%EA%B0%81'
*incorrectl* as Windows-1252 while the latter interpret it *correctly* as UTF-8. 

http://news.google.com/news?q=%EA%B0%80%EA%B0%81&hl=en&lr=&newwindow=1&client=firefox-a&rls=org.mozilla:en-US:official&sa=N&tab=wn

http://news.google.com/news?q=%EA%B0%80%EA%B0%81&hl=en&lr=&newwindow=1&sa=N&tab=wn

I wonder why google relies on 'client=.....' string instead of just using
'User-Agent' string that includes 'Gecko'

I wrote to [e-mail] about the issue.

Comment 6

[Jungshik Shin]

darin and kat,
can you take care of this (ask someone in charge to fix) on the google side? I
wrote more than once to support@google, but nothing has happenend so far.

Comment 7

[Jungshik Shin]

Btw, we can work around this problem (comment #0) by dropping
'client=firefox...' here (or in a couple of other places)
http://lxr.mozilla.org/seamonkey/source/other-licenses/branding/firefox/locales/browserconfig.properties#2
however, I guess we don't wanna do that.

Comment 8

[Darin Fisher]

> can you take care of this (ask someone in charge to fix) on the google side?

done.  no eta for fix.

Comment 9

[Katsuhiko Momoi]

I'm curious. It is true that if you get into the URL mentioned by Jungshik,
there is this problem at Google news. I used FF 1.03 and went to google.co.kr
and entered the key word cited above. I was then taken to this URL:

http://www.google.co.kr/search?hl=ko&q=%EA%B0%80%EA%B0%81&btnG=Google+%EA%B2%80%EC%83%89&lr=

and of course when I clicked on the news link, there was no display problem.

So acknowledging that Google code may have a problem, how likely is it today to
get into this particular problem?

Comment 10

[Jungshik Shin]

Kat, please read my comment #0. It's rather easy to get into the 'trap' because
that's the page you get when you type a non-ASCII keyword in the google search
box on the upperright hand corner of firefox. I got back to this bug today
asking for your help because another Korean user reported the problem at
mozilla.or.kr

Comment 11

[Katsuhiko Momoi]

Sorry about that. I am so used to using either the Google home page or a toolbar
extension on Firefox that I hardly notice the upper right box. Actually it seems
that the Firefox/Google start/home page has the same problem.

Comment 12

[:Gavin Sharp [email: [e-mail]]]

this seems to have been fixed