Closed Bug 88327 Opened 25 years ago Closed 25 years ago

XML Extras doesn't support international charsets

Categories

(Core :: XML, defect, P1)

defect

Tracking

()

VERIFIED FIXED
mozilla0.9.3

People

(Reporter: sandipc, Assigned: hjtoi-bugzilla)

References

()

Details

(Whiteboard: [pdt+][DIGBug][fixinhand])

Attachments

(1 file)

From Bugzilla Helper: User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0) BuildID: 2001060704 It looks like the XMLExtras objects assumes the response data is in the ISO8859- 1 charset. It ignores the charset part of the content type header that gets sent with the response. - Sandip Reproducible: Always Steps to Reproduce: 1) Submit a query using the XMLExtras send API 2) Have the server respond with an HTML page that contains non-ascii characters. a) I tested I18N support using "gb2312" simplified-chinese b) Set the content type to: text/html; charset-gb2312 c) Send the html containing the chinese chars d) Using the req.responseText API the characters are not converted correctly byt he XMLExtra object. It appears the object *ignores* the charset specified in the content-type. Actual Results: The resulting string doesn't have the correct set of characters. It looks like the XML extras object just takes each byte and stores them into the string without *any* character conversion. Expected Results: It should have converted the byte stream based on the character set specified in the content type header. Once this happens, the responseText attribute will have the correct representation of the string.
Whiteboard: DIGBug
Ok, I have this somewhat dissected. Thanks for the great testcase! In .send() we start a document load, and there we first try to set the document's charset. At that point the netlib did not have the data available, so we are left with default [Not sure why necko was not ready, sometimes it seems to have this info]. Parser tries to parse the data, but since it was not XML it did not set a correct charset on the document. While getting responseText we want to convert the internal buffer to the charset. If we have a document available we'll get the document's charset. We have the document and the rest is history. Now the fact that we have that document even though we did not receive XML data, or even anythinng remotely XML seems to indicate that we should just null the document. A somewhat reasonable check might be to see if we have any elements in the document. To make matters worse, we also have a typo in the charset detection code, which makes the detection always fail. I have a fix in my tree but I will clean it up a little. This should definitely go in the trunk and NS 0.9.2 aka RTM branch.
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Priority: -- → P1
Target Milestone: --- → mozilla0.9.3
Whiteboard: DIGBug → [DIGBug][fixinhand]
The changes look good, but there's something weird with the patch itself, I saved the file and here's what I see in emacs, ^M:s on some lines but not on others, what's up with that? :-) - mStatus = XML_HTTP_REQUEST_COMPLETED; + mStatus = XML_HTTP_REQUEST_COMPLETED;^M +^M Don't check those ^M:s in, other than that, sr=jst
AAARRRGHHHH! I noticed the funny lines in diff yet I thought I must have added whitespace or something to the end. All this mess happened because I used Cygwin cvs, which retains Unix linefeeds, and when I edited the files with Visual C++ it put Windows linefeeds on the new lines it creates. Bastards!
nsBranch ok, digbug, simple fix.
Keywords: nsBranch
Fix checked in. Please verify this by Thursday afternoon because I'd like to get this for RTM.
really fixed
Status: ASSIGNED → RESOLVED
Closed: 25 years ago
Resolution: --- → FIXED
For regression testing we have: http://green/heikki/post.html http://green/heikki/syncpost.html The craterlake server still seems to have some problems. Yesterday and this morning I was unable to access the testcase. Now I can access the test, but when I click the button I get server error.
OK, I got the server restored. Sorry it took a while...we are trying to cut a beta candidate of the proteus code this evening.
It doesn't work for the test case I originally submitted: http://craterlake.mcom.com/NSPipeTest.htm I tried it with this version of the 6.1 client: ftp://sweetlou/products/client/6.x/windows/32bit/x86/2001-07-06-13- 0.9.2/N6Setup.exe and it still fails.
It is not on the 0.9.2 branch yet. To get there, it needs to be verified on the trunk first. You need to test with a Mozilla/Netscape trunk build later than 7/4. I will be on vacation all next week (leaving tomorrow morning), so please be in contact with Nisheeth to make sure this will end up on the branch.
OS: Windows 2000 → All
Hardware: PC → All
Re-opening bug until the fix goes into the branch.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Whiteboard: [DIGBug][fixinhand] → [pdt+][DIGBug][fixinhand]
I tried testing this and I still get the same behavior that Heikki reported earlier. I can access the test on the server, but after I click the button the server returns an internal server error message. Just left voice mail for Greg White to fix the server end of things. Once we can get this tested with the branch build, we'll check the fix into the branch. Greg, please treat this as high priority because we are very close to the end for 6.1. The sooner we get this fix onto the 6.1 branch the better. Thanks a lot!
Status: REOPENED → ASSIGNED
craterlake is back again. Sorry for the confusion.
Also see http://bugzilla.mozilla.org/show_bug.cgi?id=90322, which will be much less important to DIG if we can use xmlextras for calendar.
I actually tested the fix Heikki put into the trunk on Monday against the isolated test case I put together on craterlake (Greg White's machine). I also ran it thru some other i18n pipe test suites that I have. The fix works for all of the above test cases! Thanks for the quick fix. - Sandip (sandipc@netscape.com)
Just checked in the fix to the branch.
Really marking fixed.
Status: ASSIGNED → RESOLVED
Closed: 25 years ago → 25 years ago
Resolution: --- → FIXED
This bug has been verified on the trunk and the branch.
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: