103954 - JS gets error

Status: VERIFIED FIXED

(Core :: Internationalization, defect, P2)

Description

[bobj]

Comment 1

[bobj]

If you try this page, with the JS console open, you see this error:
  Error: test is not defined

This page works with IE5.5 and Netscape 4.78.

Ran with today's commericial branch build:
  Mozilla/5.0 (Windows; U; Win95; en-US; rv:0.9.4) Gecko/20011009 Netscape6/6.2 

Comment 2

[Phil Schwartau]

Reassigning to Internationalization. It looks like another duplicate
of bug 63626. Any JS file that contains non-ASCII characters is failing
to load. This will cause failures in the JS Console anytime such missing JS
is invoked, and that appears to be what is happening here.

Comment 3

[Phil Schwartau]

Sure enough, if I try to load the site in a debug Mozilla build,
I get several of these alertboxes coming up:

ASSERTION: Could not convert external JavaScript to Unicode!: 'NS_SUCCEEDED(rv)'
file: d:\mozilla\content\base\src\nsScriptLoader.cpp.
line: 726

Comment 4

[bobj]

cc'd momoi, intl evangelist

Comment 5

[Roy Yokoyama]

->ftang

Comment 6

[Frank Tang]

I got a different result:
Error: Inversion is not defined
http://members.home.net/markdavis34/unicode/properties.js Line: 1

Comment 7

[Frank Tang]

It seems the problem is most of the js file have a copyright sign encode in
ISO-8859-1, and we try to load the page as UTF-8 since the HTML page is UTF-8.


nhotta, we should change the following code to skip the problem bytes and resume
the conversion again like what we did for parser. 

or we should try "ISO-8859-1" as the charset if we failed the first try. 

Can you help to fix this ?


content/base/src/nsScriptLoader.cpp
692 if (NS_SUCCEEDED(rv)) {
693 PRInt32 unicodeLength = 0;
694 695 rv = unicodeDecoder->GetMaxLength(string, stringLen, &unicodeLength);
696 if (NS_SUCCEEDED(rv)) {
697 typedef nsSharedBufferHandle<PRUnichar>* HandlePtr;
698 typedef nsAString* StrPtr;
699 HandlePtr handle = NS_AllocateContiguousHandleWithData(HandlePtr(0),
NS_STATIC_CAST(PRUint32, unicodeLength+1), StrPtr(0));
700 PRUnichar *ustr = (PRUnichar *)handle->DataStart();
701 702 rv = unicodeDecoder->Convert(string, (PRInt32 *) &stringLen, ustr,
703 &unicodeLength);
704 705 if (NS_SUCCEEDED(rv)) {
706 handle->DataEnd(handle->DataStart() + unicodeLength);
707 }
708 nsSharableString tempStr(handle);
709 request->mScriptText = tempStr;
710 }
711 }

Comment 8

[nhottanscp]

Can this be a generic option for unicode decoders, proceed with skipping 
unconverted characters instead of stop the process?

Comment 9

[bobj]

I don't thing this is a dup of of bug 63626.   Clearing status.

In bug 63626, the external .js file was in a different encoding of the HTML
and I assume it was not being served with the appropriate charset parameter
in the HTTP content-type header, so by default it is assume the encoding of
the external .js is the same as the HTML document.  Bug 63626 seems to be a
request for the browser/JS engine to notify that a conversion error was 
detected and warn the user.

I just filed a new bug that the charset parameter to the <script> appears
to be ignored: bug 106360

Comment 10

[bobj]

> It seems the problem is most of the js file have a copyright sign encode in
> ISO-8859-1, and we try to load the page as UTF-8 since the HTML page is UTF-8.

I don't think this is the case for this bug.  I looked at inversions.js:

    view-source:http://members.home.net/markdavis34/unicode/inversions.js

and I don't see any copyright signs, but I do see a lot of UTF-8 data.

view-source:http://members.home.net/markdavis34/unicode/inversions.js

I also looked at the HTTP headers, and there is no charset parameter being used
on the content-type:

  HTTP/1.1 200 OKCRLF
  Date: Tue, 23 Oct 2001 23:31:29 GMTCRLF
  Server: Apache/1.3.11 (Unix)CRLF
  Last-Modified: Fri, 02 Feb 2001 19:49:11 GMTCRLF
  ETag: "f9c7cd-11a1-3a7b0f37"CRLF
  Accept-Ranges: bytesCRLF
  Content-Length: 4513CRLF
  Connection: closeCRLF
  Content-Type: application/x-javascriptCRLF

So both the .html and the .js files seem to contain only UTF-8.

Comment 11

[bobj]

> Can this be a generic option for unicode decoders, proceed with skipping 
> unconverted characters instead of stop the process?

I think an option would be good, but we need to be careful when using the option.

Possibly for display, we could be lenient and skip over conversion errors, resync
and disply more data.  But for scripts, this might be dangerous.  In the script
case, it might be better to provide a warning or alert that a conversion error
occurred rather than silently failing.

Comment 12

[nhottanscp]

inversions.js has a copyright symbol at the top and it is encoded as "A9" in 
ISO-8859-1.
"var copyrightInversions = "Copyright © 2000 Mark Davis. All Righ"

In the code, UTF-8 decoder stops the conversion at "A9".

Comment 13

[nhottanscp]

Attachment: Changed to proceed for decoder errors instead of stop the process.

Comment 14

[Teruko Kobayashi]

Nominated this bug to nsbeta1.

Comment 15

[Frank Tang]

change line of {
+          if (NS_FAILED(rv)) {
to
+          if (NS_FAILED(rv)) 
+          {
in your new code. r=ftang

Comment 16

[Johnny Stenback (:jst)]

Ahem, the rest of the code in this file uses the style:

  if (...) {

and *not*:

  if (...)
  {

so leave the patch as is, consistency is more important than sticking to any
particular coding style.

sr=jst

Comment 17

[nhottanscp]

Checked in.

I think the alert issue can be a separate bug. 4.x does not alert.

Comment 18

[nhottanscp]

*** Bug 63626 has been marked as a duplicate of this bug. ***

Comment 19

[Teruko Kobayashi]

http://members.home.net/markdavis34/unicode/convert.html
is no longer in service.  Are there any test case for verifying this bug?

Comment 20

[Teruko Kobayashi]

This does not happen anymore in 20020102 build.

Comment 21

[bobj]

How do we know if the site is no longer active?
If necessary, we could contact Mark Davis to find out to where this has moved.