49039 - national characters in a page intermittantly breaks rendering

Status: VERIFIED WORKSFORME

(Core :: DOM: HTML Parser, defect, P3)

Description

[R.K.Aa.]

first seen on linux M18 build 2000-081309

Noticed a query loaded from local disk broke some of the summary of bug 31919
"URLencode fails on URL with national ( æ ) character".

As it turns out: ANY occurance of æ, ø or å in a html page will now break
rendering of it. Tested "ñ" and "ö"  - those too break pages.

This bug doesn't break *saving*: The files are only trunkated when rendered.
Tempted to set it as a blocker however: This bug effectively makes moz more or
less completely useless for me. (Searches in bugzilla break, can't browser
national wepages etc.)

This bug is pretty new - days at most. Not sure of component - parser or
localization or what.

Comment 1

[R.K.Aa.]

regression keyw.

Comment 2

[R.K.Aa.]

adding hint in sumary about not using moz to read this bug right now.

Comment 3

[R.K.Aa.]

ahh this was fun: the error is intermittant. This page just loaded OK.
Lowering severity but this should be looked in to.

Comment 4

[R.K.Aa.]

wonder if i found a way to reproduce 100%:

open an ftp site in one browser window.
open a new browser window.
Try to load this bug then.

here, the first comment then ends this line:
"URLencode fails on URL with national (

Comment 5

[R.K.Aa.]

this is a pretty bad bug, as it means that a browser window launched from a
window currently displaying ftp treeview of some site, isn't really a full
fledged navigator window: Either it inherits "bad stuff" from treeview, or - for
some reason - it isn't properly initiated. 

Comment 6

[Asa Dotzler [:asa]]

OK, I can reproduce this with today's win32 and Mac builds.
If I have a window open to an FTP site and from that window I launch a new
window and try to laod http://bugzilla.mozilla.org/show_bug.cgi?id=49039 in this
new window it is trunkated at the æ character.

Comment 7

[Asa Dotzler [:asa]]

note: the source from view source is also truncated at that point.  Setting to
Platform/OS ALL

Comment 8

[Asa Dotzler [:asa]]

over to parser.

Comment 9

[rickg]

No real hard data yet, but this bug does not appear in viewer. That means 1) 
it's not parser; or 2) viewer's code path avoids the issue. Let's try mozilla 
now...

Comment 10

[rickg]

Ok, it took a while to track it down, but here it is. The scanner gets the data 
correctly from netlib, and tries to append it (in the parser). The scanner calls 
the i18n library: 

res = mUnicodeDecoder->Convert(aBuffer, &srcLength, unichars, &unicharLength);

which truncates the buffer after the 0xE6. I don't know why. Off to frank tang 
for more work.

Comment 11

[Frank Tang]

Please include reproduce procedure and /or test pages. 
With the current description, we have no way to reproduce / verify the issue. 
Please make some testing page for us. Thanks.

Comment 12

[R.K.Aa.]

ahh it's changed. With m18/linux/2000-082513 a browser-instance launched from
another (while it displays an ftp site) does NOT break text in this bug anymore.
Which was more or less how i discovered this bug. It seems to have passed away
in silence.

Comment 13

[R.K.Aa.]

i still see this in 2000091212

To reproduce:

go to an ftp:// URL
from that window: Open a new browser window
in the new window: go to http://www.dagbladet.no

All national characters (ז,ר,ו) will now display as "?"


(There is a cache bug also, so quitting mozilla and then starting again and
going to dagbladet.no will now show "?" instead of זרו even if cache is set to
refresh once per session. To verify that the site actually IS coded right, one
has to go to another site, delete disk and memory cache from prefs, close
mozilla, and then restart and go there again. Characters should now display זרו)

I think bug 52255 is a dup of this one.

Comment 14

[R.K.Aa.]

Oops - forgot the vital piece of info here: The bug no longer breaks all the
rest of the page, but it's still data-loss that it looses the ability to
interpret character-sets. Any language with more national characters than
norwegian can get very unreadable very quick as long as this bug isn't fixed.

Comment 15

[Frank Tang]

I cannot reproduce this. Mark it as workforme.

Comment 16

[bsharma]

updated qa contact.

Comment 17

[bsharma]

Verified on
build: 2001-05-07-06-Trunk
platform: WinNT