All users were logged out of Bugzilla on October 13th, 2018

non 7-bit ascii characters replaced by question marks

RESOLVED WORKSFORME

Status

()

--
major
RESOLVED WORKSFORME
14 years ago
13 years ago

People

(Reporter: jarkko, Assigned: bugs)

Tracking

({intl})

Firefox Tracking Flags

(Not tracked)

Details

(URL)

(Reporter)

Description

14 years ago
User-Agent:       Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7) Gecko/20040803 Firefox/0.9.3
Build Identifier: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7) Gecko/20040803 Firefox/0.9.3

In the page at http://www.suunnistus.net (or http://solo.merita.fi), all
non-7-bit ascii characters that are not given as entities, are replaced by
question marks (and in some places by empty space). This wasn't happening in 0.9.2.

Reproducible: Always
Steps to Reproduce:
1. Go to http://www.suunnistus.net


Actual Results:  
Non-7-bit ascvii characters that are not encoded as entities, are replace by
question marks or empty spaces.

Expected Results:  
Show the characters (like ä, ö).

Comment 1

14 years ago
are all letters with diacritics rendered as question marks? Or, are only those
that are represented 'literally' (instead of with entities) affected? 

What's the character encoding in View | Encoding when you have the problem? It
should be ISO-8859-1. 

Can you try Mozilla 1.7.2 as well? 
Keywords: intl
(Reporter)

Comment 2

14 years ago
(In reply to comment #1)
> are all letters with diacritics rendered as question marks? Or, are only those
> that are represented 'literally' (instead of with entities) affected? 

Only those that are not entities. The problem seems to be that those failing
pages don't seem to have their charset specified in the document head.

> 
> What's the character encoding in View | Encoding when you have the problem? It
> should be ISO-8859-1.

It was UTF-8, and that's the reason why it failed. However, I hadn't changed
anything so it seems that UTF-8 is the default when the document doesn't specify
its charset. Is this new in 0.9.3?

When I set the auto-detect to universal, it seemed to work with all sites I
tested. Is there some reason why it couldn't be the default? I'm quite sure I
haven't ever changed the character encoding in Firefox by hand and it has worked
flawlessly until 0.9.3.

> 
> Can you try Mozilla 1.7.2 as well? 

It too had the auto-detect off, but its default seemed to be ISO-8859-1 so the
test pages worked fine.

Comment 3

14 years ago
Actually, universal autodetector tends to misdetect ISO-8859-1 as either
Shift_JIS or GB*, but your experience seems to be different. 

> it seems that UTF-8 is the default when the document doesn't specify
> its charset. Is this new in 0.9.3?

  I don't recall making such a change. If that's the case, obviously it's to be
reverted to ISO-8859-1 for the default en-US package. 

  Another possibility is that you went to the page in question by following a
link in a page encoded in UTF-8. Is it the case? In that case, that's a known
problem/issue. Mozilla uses the encoding of the referring page if the encoding
of the referred page is not specified. 

Steffen, do you know anything about this? 

Anyway, with bug 181541 fixed, Jarkko, you have a nice UI to set the default
encoding with. 

Comment 4

14 years ago
In new profiles, the default character encoding is still set to ISO-8859-1.

I can reproduce this problem - on Windows, that is - if I either set the default
character encoding to UTF-8, and visit these sites, or set the character
encoding of the current page (like this one) to UTF-8 and then visit these sites.
You have to clear your cache each time you try a new setting.
(Reporter)

Comment 5

14 years ago
(In reply to comment #4)
> In new profiles, the default character encoding is still set to ISO-8859-1.

Steffen,

This wasn't in a new profile, it was after I upgraded from 0.9.2 to 0.9.3 by
copying the new firefox executable over the old one (that's how it works in
OSX). After that, the sites mentioned here stopped showing correctly. No changes
in settings, I just shut down 0.9.2, upgraded, and launched again.

> 
> I can reproduce this problem - on Windows, that is - if I either set the default
> character encoding to UTF-8, and visit these sites, or set the character
> encoding of the current page (like this one) to UTF-8 and then visit these sites.
> You have to clear your cache each time you try a new setting.

I'm not surprised or worried about this, I think it's understandable that
problems arise when using the wrong charset. I just wonder why upgrading changed
the default setting.

Comment 6

14 years ago
(In reply to comment #5)
> I just wonder why upgrading changed the default setting.

So, in preference | general | language, 'default character encoding' was set to
UTF-8 (instead of ISO-8859-1) after you upgraded. That's really strange. I've
just built a firefox trunk build and its default is ISO-8859-1. I don't think
there's anything different between 0.9.x and the trunk in that regard. Could it
be an installer? Hmm...


Comment 7

14 years ago
Occasionally I have the same problem with http://www.whoa.nu and some other
websites; and it doesn't happen always, only sometimes. Isn't that a bit weird?

Comment 8

14 years ago
And, ironically, I have the same problem with Bugzilla right now...the Å
(a-ring) in my last name has turned into a question mark :-/

Comment 9

14 years ago
I am having this issue as well, and it's the copyright symbol which is appearing
as a question mark and I'd like to know how this can be fixed...

Comment 10

14 years ago
wfm on 10.3.8
Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8b2) Gecko/20050405
Firefox/1.0+

Updated

14 years ago
Assignee: firefox → bugs
Component: General → File Handling
QA Contact: general → aebrahim-bmo
This is an automated message, with ID "auto-resolve01".

This bug has had no comments for a long time. Statistically, we have found that
bug reports that have not been confirmed by a second user after three months are
highly unlikely to be the source of a fix to the code.

While your input is very important to us, our resources are limited and so we
are asking for your help in focussing our efforts. If you can still reproduce
this problem in the latest version of the product (see below for how to obtain a
copy) or, for feature requests, if it's not present in the latest version and
you still believe we should implement it, please visit the URL of this bug
(given at the top of this mail) and add a comment to that effect, giving more
reproduction information if you have it.

If it is not a problem any longer, you need take no action. If this bug is not
changed in any way in the next two weeks, it will be automatically resolved.
Thank you for your help in this matter.

The latest beta releases can be obtained from:
Firefox:     http://www.mozilla.org/projects/firefox/
Thunderbird: http://www.mozilla.org/products/thunderbird/releases/1.5beta1.html
Seamonkey:   http://www.mozilla.org/projects/seamonkey/

Comment 12

13 years ago
it happened to me pretty recently if I remember correct, it has most def.
occured at least once the past 3 months at the site www.whoa.nu (but it doesn't
happen very often though, that's for sure)
(In reply to comment #12)
> it happened to me pretty recently if I remember correct, it has most def.
> occured at least once the past 3 months at the site www.whoa.nu (but it doesn't
> happen very often though, that's for sure)

This WFM on the site you listed using Fx 1.5. From what I can tell, it was an encoding issue in the past and has, more or less, been remedied.

Erik and Jarkko, I'm going to close this bug as WORKSFORME. If you see this issue again using Firefox 1.5, and can reproduce it, please reopen the bug or leave a comment. Also, be sure to list the website this happens on and what version of OS X you're using.
Status: UNCONFIRMED → RESOLVED
Last Resolved: 13 years ago
Resolution: --- → WORKSFORME
(Reporter)

Comment 14

13 years ago
WFM now, too on 1.5/OSX.

Comment 15

13 years ago
WFM 99.99% of the time as well...and, just for the record - I'm using FF 1.5 on WinXP Pro
You need to log in before you can comment on or make changes to this bug.