819868 - Consolidate code for converting from various encodings into a UTF-16 string

Status: RESOLVED WONTFIX

(Core :: Internationalization, defect)

Description

[Henri Sivonen (:hsivonen)]

There seems to be some code duplication among
nsContentUtils::ConvertStringFromCharset 
nsScriptLoader::ConvertToUTF16
nsXMLHttpRequest::AppendToResponseText
TextDecoder::Decode

These should (potentially with an exception for nsXMLHttpRequest::AppendToResponseText) use the same code. Furthermore, the code should probably be more careful about making the right tradeoff between wasting memory and avoiding copying. I suggest allocating the target string’s buffer for the worst case, decoding directly into the target string’s buffer and then having the string itself downsize its buffer if the string actually fits in a smaller jemalloc bucket than what it has.

See also bug 497204.

Comment 1

[Masatoshi Kimura [:emk]]

I'm considering making TextDecoder::Decode callable from C++ and using it wherever possible.

Comment 2

[Henri Sivonen (:hsivonen)]

The remaining cases are now sufficiently specialized that it doesn't make sense to unify them.