86792 - Anchors external to the current document using 8-bit characters do not work

Status: VERIFIED FIXED

(Core :: Internationalization, defect, P2)

Description

[Katsuhiko Momoi]

** Observed with 6/15/2001 Win32 trunk build **

Some Japanese web pages have a link to an anchor
in another document. And that anchor could be using 
8-bit characters like Shift)JIS Japanese. 

For example,

http://foobar.foo.bar/#??

8-bit anchors do work within the same document.

You can test this either via 2 web pages with a
link referring to an achor on another page.
Or you can try the direct URL to the anchor,
and it will not work, either.

Comment 1

[Katsuhiko Momoi]

TM to 0.9.2 and priority to P2.

Comment 2

[nhottanscp]

This was once fixed by bug 58819.
It used to be the anchor was not URL escaped in nsHTMLContentSink.cpp.
But now it's escaped, so I think it would work if it's unescaped.
This could be generic to even ASCII anchors if the anchor contains characters
which need URL escape.

Comment 3

[Frank Tang]

according to momoi, this is a very very important bug. 
Without this bug fixed, Netscape will lost chance to ship 
X,XXX,XXX copy of OEM bundle deal for YYYY company for product ZZZZ.
PDT: I cannot put important marketing information into public bug system.
Ask me and I can tell you what is X,XXXX,XXX , YYYY and ZZZZ.

Change the bug to ALL for platform and ALL for OS

Comment 4

[Frank Tang]

>8-bit anchors do work within the same document.
Is this true? I thought that we look at the problem together and conclude it 
does work in the same document but does not work if the anchor is refer to a 
different .html document. 

for inside netscape. I put some test cases under
http://warp/u/ftang/tmp/10_inf/
We cannot attach those test cases into bugzllia since they are 3rd party 
copyright html.

Comment 5

[Katsuhiko Momoi]

>> 8-bit anchors do work within the same document.

> Is this true? I thought that we look at the problem together and conclude it 
> does work in the same document but does not work if the anchor is refer to a 
> different .html document.

I think you are saying the same thing as I did. Within the same document,
8-bit characters as anchors seem to be working.

Comment 6

[Frank Tang]

Attachment: simple test case part 1

Comment 7

[Frank Tang]

Attachment: simple test case part 2- somehow this WORK, which cannot show the problem

Comment 8

[Frank Tang]

ok, I attach one test case here in the attachement. 
how to reproduce
1. open part 2 - 06/20/01 00:53
2. click on the 3rd link
expect result-
 open one new window
 and the page scroll to "A 3"
actaul result-
 open one new window and the page stay in the beginning
however, if now you
3. click on the part 2, and click the 4th link
it WILL scrool the the new window to the 4th link
It looks it does NOT work with the first link which open the window.


It seems the non ASCII anchor is working but not working while we mixed with 
target set or open new window....

Comment 9

[Frank Tang]

from my simplified test cases it seems the problem only happen while in the 
first click to bring up the document in a new named window. 
I suspect there are some JavaScript code which open the new window scrow up the 
URL 

Comment 10

[nhottanscp]

I tested ASCII anchor name which contains a space (URL escape is also applied to
space).
So HREF to an anchor <a NAME="ccc ccc">, like this <a href="anchor.html#ccc
ccc"> does not work (works with 4.x).

Comment 11

[Frank Tang]

Attachment: fix

Comment 12

[Frank Tang]

nhotta- did you produce the same fix? :)
internal test case at http://warp/u/ftang/tmp/10_inf

Comment 13

[Yuying Long]

Frank and Naoki:
I don't know if your fix covered some problem here, just in case:
I created a testcase(see attached file later) which contain Ascii and Japanese
(kanji, hirakana and half-width katakana) anchor name in different html file.  
The links in anchor2.html will call the anchors in another file anchor.html:
1. Save the document as shift-jis, inside the same document, it will work fine 
no matter ascii or Japanese anchor name.
2. If the files save as iso-8859-1, inside same document, only works when 
anchor name is ascii, the non-ascii anchor won't do anything.
3. When links in anchor2.html call anchors in anchor.html, only ascii anchor 
works fine, all other Japanese anchor will go to the beginning of the 
anchor.html page.

Comment 14

[Yuying Long]

Attachment: anchor2.html file. The links here will call anchors in anchor.html file

Comment 15

[Yuying Long]

Attachment: anchor.html file

Comment 16

[Frank Tang]

My local build which have the patch fix the case 3 - anchor2.html link to 
anchor.html
Not quite sure what do you mean in case 2, save as ISO-8859-1

Comment 17

[nhottanscp]

r=nhotta

About the allocation of ToNewCString, I see other places in mozilla doing it
differently.
e.g.
nsUnescape(NS_CONST_CAST(char*, filename.get()));
http://lxr.mozilla.org/mozilla/source/netwerk/streamconv/converters/nsIndexedToHTML.cpp#164

Then you could do it something like this.
    nsCAutoString unescapedRef(mRef);
    nsUnescape(NS_CONST_CAST(char*, unescapedRef.get()));

But that modifies the buffer without updating the length field of the class.
Unescape does not require to increase the buffer size but it causes
inconsistency inside the class.

Reassign to ftang.

Comment 18

[nhottanscp]

BTW, the anchor in nsHTMLContentSink.cpp used to be not escaped as I mentioned
before. You might want to check why it was changed and if the other part of
nsHTMLContentSink.cpp can handle unescaped URI then the unescape could be
applied at the assignment to 'mRef'. Or if you want to minimize the impact by
the change then the current way of copy before unescape would be fine.

Comment 19

[Frank Tang]

thanks for the review:
>But that modifies the buffer without updating the length field of the class.
Then I will keep using the current approach.

Comment 20

[vidur (gone)]

Hmm...there's also a nsXMLContentSink::ScrollToRef. There was some hope that
we'd be able to consolidate the redundant code in the various content sinks, but
that won't happen till sometime in the future. The HTML and XML versions look
considerably different, but probably should be virtually identical.

Comment 21

[Johnny Stenback (:jst)]

The XML content sink actually does the right thing now, as long as the URI we're
loading actually is a URL (i.e. nsIURL) since it calls GetRef() on the URL which
unescapes the ref.

sr=jst for the attached patch.

Comment 22

[chris hofmann]

a=chofmann

Comment 23

[Frank Tang]

fix and check in 

Comment 24

[Andreas Becker]

Switching QA contact to ylong for verification.

Comment 25

[Katsuhiko Momoi]

Thank you. I looked at this with an afternoon Win32 build from
6/21/2001. The original web pages that had this problem
no longer shows this problem. 

Comment 26

[Yuying Long]

The Sony VAIO menual page works fine now, however anchor still doesn't work very 
well so that I filed another bug 87700 for that.  
Mark this one as verified.