792690 - Keyword search uses inconsistent character sets

Status: RESOLVED WORKSFORME

(Firefox :: Bookmarks & History, defect)

Description

[Martin von Gagern]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20100101 Firefox/15.0.1
Build ID: 20120913125421

Steps to reproduce:

1. Added keyword search for http://www.amazon.de/ top search field
2. Used that keyword search with "Türhänger" as the search term


Actual results:

A search for "TÌrhÀnger" was executed.

Looking at the content of the page, Amazon submits a hidden form field __mk_de_DE=ÅMÅZÕÑ to detect the encoding used for the form submission. The query part of the keyword search result had __mk_de_DE=%C5M%C5Z%D5%D1 (ISO-8859-1) but field-keywords=T%C3%BCrh%C3%A4nger (UTF-8). So it is quite understandable that the Amazon server got confused by this inconsistently encoded query.


Expected results:

A search for "Türhänger" should be executed. Either because the entered search term is encoded using ISO-8859-1, or because the __mk_de_DE field is encoded in the URL using UTF-8 as well.

The latter opens a route for a workaround: once the keyword has been added, one can manually change one variable to __mk_de_DE=%C3%85M%C3%85Z%C3%95%C3%91 to make the URL all UTF-8. After this modification, the query will execute as intended.

Comment 1

[Alice0775 White]

See http://kb.mozillazine.org/Using_keyword_searches#Specifying_a_non-UTF-8_encoding_in_the_query
You should manually add &mozcharset=ISO-8859-1 to created bookmark(i.e keyword search)

Comment 2

[Martin von Gagern]

(In reply to Alice0775 White from comment #1)
> You should manually add &mozcharset=ISO-8859-1 to created bookmark

OK, manually adding mozcharset is another way to work around the issue at hand. My core concern, however, is that this should work automatically, without any need for a manual fixup using either alternative.

When I generate a keyword search from a form field, then some encoding is used to store hidden fields in the query part of the URL. Either that encoding should be UTF-8 or a mozcharset parameter should automatically be added to the query. Preferrably the charset chosen for both of this should be the one a form submission would use as well, but that's not as important to me as the consitency between hidden fields and an mozcharset (resp. its default of UTF-8).

Comment 3

[[:Aleksej]]

WFM with the site described in comment #0 and 2014-07-29-03-02-02-mozilla-central-firefox-34.0a1.en-US.linux-x86_64.

Comment 4

[Mark Banner (:standard8)]

Also WFM with that site. It is possible there could be other sites broken, but without further information I don't think there's much we can do here - generally in this case it sounds like it was more a site issue.