477577 - Encoding errors compiling TestAUSReadStrings.cpp

Status: RESOLVED FIXED

(Toolkit :: Application Update, defect)

Description

[Robert Strong (they/them - no direct email)]

From bug 473417 comment #89

Robert, this patch doesn't compile in all situations, your code is assuming
users are building on Latin-based versions of Windows.  Under Japanese Windows,
the compiler spits out a bunch of bad encoding errors and the compile fails.

Using UTF-8 strings directly in code like this trips up builds under Japanese
Windows, since the compiler is assuming (argh!) that source files are in the
default codepage of the system (i.e. codepage 932 (Shift-JIS) vs. 1252 (Windows
Latin-1)).

There might be a way of forcing the compiler to interpret the file as a UTF-8
file by adding a BOM but that will probably break builds on other platforms
with compilers that don't understand BOM's.  The safest way would be to simply
escape the UTF-8 strings you're using:

"Title Test - 测试 測試 Δοκιμή テスト Испытание"

would become

"Title Test - \xE6\xB5\x8B\xE8\xAF\x95 \xE6\xB8\xAC\xE8\xA9\xA6
\xCE\x94\xCE\xBF\xCE\xBA\xCE\xB9\xCE\xBC\xCE\xAE
\xE3\x83\x86\xE3\x82\xB9\xE3\x83\x88
\xD0\x98\xD1\x81\xD0\xBF\xD1\x8B\xD1\x82\xD0\xB0\xD0\xBD\xD0\xB8\xD0\xB5"

You could also read these strings in from a file, that would give you full
control over the assumed encoding.

Comment 1

[Robert Strong (they/them - no direct email)]

Attachment: patch

Pushed to mozilla-central
http://hg.mozilla.org/mozilla-central/rev/a419ff2cf0c5

Comment 2

[Robert Strong (they/them - no direct email)]

resolving -> fixed on trunk... will wait a cycle before pushing to mozilla-1.9.1

Comment 3

[Robert Strong (they/them - no direct email)]

Pushed to mozilla-1.9.1
http://hg.mozilla.org/releases/mozilla-1.9.1/rev/58ceb7cad61f

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1] (away from Feb 12th - Feb 22nd)]

John, could you please verify that this works on trunk and 1.9.1 branch?

Comment 5

[Robert Strong (they/them - no direct email)]

btw: Henrik, I believe the reason for setting the target milestone (from reading parts of the thread) to when it is fixed on the trunk was due to TM. As I understand it TM is no longer marking bugs landed in the TM repo as fixed and only mark the bugs as fixed after they have landed on trunk. If that is the case it seems counterintuitive to mark a bugs target milestone to a later milestone than the first milestone it actually landed for.

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+1] (away from Feb 12th - Feb 22nd)]

This is mostly off-topic but lets give a short answer... Setting the target milestone to 1.9.2a1 also applies to TM bugs. It doesn't matter in which product/component bugs resist. As what was said in the thread, the target milestone always represent the most recent milestone on trunk. And after branching for 1.9.1 this is 1.9.2a1 right now. It's really helpful for QA to have a straight handling, so sorry for all the changes Rob.