Closed Bug 782721 Opened 7 years ago Closed 7 years ago

Exising nsIUnicodeDecoder, nsBasicDecoder do not have a replacement character option when an unknown character is found.

Categories

(Core :: Internationalization, defect)

x86_64
Linux
defect
Not set

Tracking

()

RESOLVED DUPLICATE of bug 638379

People

(Reporter: bsurender, Unassigned)

References

(Blocks 1 open bug)

Details

The existing nsIUnicodeDecoder does not support a replacement character option if an unknown character is detected in the incoming charset to be decoded to UTF-16 code units.

http://mxr.mozilla.org/mozilla-central/source/intl/uconv/public/nsIUnicodeDecoder.h#61

Instead, the decoder stops and signals, or just recovers and continues.

The encoder however supports the above two options as well as replacement characters.
Component: DOM → Internationalization
Are kOnError_Signal and kOnError_Recover insufficient? Although those options may have some subtle differences from what the Encoding Standard requires, I don't think it's enough critical to block implementing TextDecoder.
(In reply to bsurender from comment #0)
> Instead, the decoder stops and signals, or just recovers and continues.

This is by design (though the behaviour could be changed to some extent if necessary): see bug 231267
Our decoder will replace unmapped character to what GetCharacterForUnMapped(). It is U+31FC for Shift_JIS, U+FFFD for other encodings.
Until bug 747762 has been fixed (but honestly I do not want break the compatibility with the Windows decoder deliberately), You can workaround the quirks as follows:
1. If the "fatal" flag is true, use kOnError_Signal and throw "EncodingError" on failure.
2. Otherwise, if GetCharacterForUnMapped() returns U+FFFD, use kOnError_Recover.
3. Otherwise, use kOnError_Signal and replaces the offending character to U+FFFD yourself.
Step 2 is optional. You can always handle the error yourself instead of inspecting the return value of GetCharacterForUnMapped().
Blocks: encoding
Assignee: bsurender → nobody
Do we still need this? Or should we close it as WONTFIX?
I found this was already covered by bug 638379.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 638379
You need to log in before you can comment on or make changes to this bug.