1488344 - Clean up UTF-8 conversion code in nsMIMEHeaderParamImpl.cpp

Status: NEW

(Core :: Networking, enhancement, P3)

Description

[Henri Sivonen (:hsivonen) (on vacation)]

In bug 1488115, some code from nsIUTF8ConverterService moved to nsMIMEHeaderParamImpl.cpp, which was the last caller.

It would make sense to take a closer look at this code and clean it up if possible.

 1) The code checks if the input conforms to the UTF-8 byte pattern and treats the input as UTF-8 if it matches the pattern even if a different encoding label is provided. Is this actually desirable or is this an artifact of trying to optimize around the uconv code being slow? Nowaways Encoding::DecodeWithoutBOMHandling(const nsACString&, nsACString&) is fast and even avoids a copy if the input is  a refcounted string and the input is valid UTF-8 and the encoding is UTF-8, if the input is ASCII and the encoding is an ASCII-compatible encoding or if the input stays in the ASCII state and the encoding is ISO-2022-JP.

 2) If it really is desirable to treat labeled-as-not-UTF-8 as UTF-8 when it matches the UTF-8 byte pattern, the check should only use IsUTF8() and not IsASCII() || IsUTF8. IsUTF8 is SIMD-accelerated anyway in the ASCII case.

 3) IS_7BIT_NON_ASCII_CHARSET is out of date. We support 3 encodings that are not ASCII-compatible and that can look like valid UTF-8 without being UTF-8. These are: UTF-16LE, UTF-16BE and ISO-2022-JP. It's better to first resolve the label and then pointer-compare the result against UTF_16BE_ENCODING, UTF_16LE_ENCODING and ISO_2022_JP_ENCODING than to try to do the check by looking at the label.

Comment 1

[Kershaw Chang [:kershaw]]

ni Jason to find an owner and set the priority.

Comment 2

[Jason Duell]

Really not sure of priority here.

Comment 3

[Selena Deckelmann :selenamarie :selena]

In support of our XPCOM cleanups, I'm prioritizing this. 

Kershaw - could you please have a look here at what's needed to be done?

Comment 4

[Kershaw Chang [:kershaw]]

Not work on this actively.