231267 - Charset converters should always notify callers of encoding errors

Status: RESOLVED FIXED

(Core :: Internationalization, defect)

Description

[Boris Zbarsky [:bzbarsky]]

According to smontagu (bug 174351 comment 7) there are converters that will
silently replace bogus bytes with U+FFFD.  Since there are cases when such bogus
bytes should be fatal errors, including cases when they could lead to security
holes, the converters should just do what the UTF-8 converter does -- return
NS_ERROR_ILLEGAL_INPUT and let the caller deal.

Note that nsIUnicodeDecoder.h has this 

 96   enum {
 97     kOnError_Recover,       // on an error, recover and continue
 98     kOnError_Signal         // on an error, stop and signal
 99   };

stuff and some noise about setting behavior to "signal", but all of that is unused.

Comment 1

[Boris Zbarsky [:bzbarsky]]

Note that I'm ok with leaving the error-recovery behavior if the caller _does_
get to affect it via the "signal" thing.  In fact, that's what would be most
preferable.

One possible impl would be to have the GetUnicodeDecoder call take a param for
whether errors should be automatically recovered from.  If that param is set,
instantiate a wrapper that does the error recovery and implements
nsIUnicodeDecoder, otherwise just use the raw decoder (and have that never do
error recovery).  That would allow us not to have this error-recovery code in
all the places we do now...

Comment 2

[Makoto Kato [:m_kato]]

This was already checked in a part of bug 174351.

Comment 3

[Boris Zbarsky [:bzbarsky]]

Does this need a unit test?

Comment 4

[Makoto Kato [:m_kato]]

Mochitest of bug 174351 contains this test.  Also, bug 289947 will contain unit test as script.

If we need unit test, I can add it.

Comment 5

[Boris Zbarsky [:bzbarsky]]

Having a unit test would be a good idea, yeah.