520406 - german umlauts are not decoded correctly

Status: RESOLVED FIXED

(Mozilla Labs :: Snowl, defect)

Description

[Heiko Adams]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.1.3) Gecko/20090909 Fedora/3.5.3-1.fc11 Firefox/3.5.3
Build Identifier: 

Snow doesn't decode the german umlauts in the feed provided by the attached url correctly. Other newsreader like brief decode them correctly.

Reproducible: Always

Steps to Reproduce:
1. Open http://handelsblatt6.blogg.de/rss.xml
2. Subscribe to the feed using snowl
3. Open snowl news list and select the feed
4. Browse through the articles
Actual Results:  
German umlauts and some other special characters are not decoded correctly

Expected Results:  
German umlauts and some other special characters should be displayed correctly

Comment 1

[alta88]

this particular feed is encoded in ISO-8859-1 but the server sends as US-ASCII.  feeds with UTF8 encoding and <description> tags with CDATA show correctly, as does spiegel.de for example, which uses <content:encoding> tags for its content summary.  do other german feeds also fail or just this one?

more info can be found here:
http://www.rssboard.org/rss-specification
http://validator.w3.org/feed/

Comment 2

[Heiko Adams]

This feed has the same problem: http://www.pro-linux.de/backend/pro-linux.rdf

Comment 3

[alta88]

fixed by http://hg.mozdev.org/snowl/rev/86a914981a5b.

but i'm not sure what was intended by overriding mimetype, myk is this a proper fix?

Comment 4

[Myk Melez [:myk] [@mykmelez]]

The purpose was to prevent XMLHttpRequest from trying to parse the response, since the feed processor parses it itself, which makes the XMLHttpRequest parsing redundant and unnecessary.  I'm ok with this change if it fixes the bug, although note that it might have a performance impact.

Comment 5

[alta88]

so it seems the request needs to be for non text/plain to return a non-encoded result, which triggers a XMLHttpRequest parse.  if the request were of some rss mimetype, would this bypass XMLHttpRequest?  wouldn't it be a core inefficiency, since Fx would be doing two parses as well, so perhaps there should be certain mimetypes exempted?

Comment 6

[Myk Melez [:myk] [@mykmelez]]

Hmm, I'm not sure what you're asking; can you explain further?

Comment 7

[alta88]

if a mimetype request of text/plain bypasses XMLHttpRequest parsing, then a feed specific or custom mimetype (since feedprocessor is going to parse anyway) should also bypass XMLHttpRequest.  this would allow the data to retain non ascii values rather than have them returned hex-encoded as with text/plain.

in other words, if asking for text/plain results in encoding, and asking for xhtml results in non encoding but parsing, then asking for rss/xhtml (eg) should result in a non encoded and non parsed result doc.

Comment 8

[Myk Melez [:myk] [@mykmelez]]

Ah, I see. Hmm, it's worth a shot, although perhaps the MIME type should be something like text/dont-encode-me.

Comment 9

[alta88]

sure, that would be fine.  seems like there should be such a mimetype already; i don't know fx mime handling/code that deeply.

anything for performance!