390127 - Non-ASCII characters in profile names are saved incorrectly; also in file:// listings; in builds since 2007062904, 0724, and 0725 (UTF-8 and UTF-16 problems). {Chinese, Cyrillics, Greek, Russian, German, umlauts, letters}

Status: VERIFIED FIXED

(Toolkit :: Crash Reporting, defect)

Description

[[:Aleksej]]

Debian GNU/Linux Lenny, LANG=en_GB.UTF-8.
Happens in Firefox, Thunderbird, Seamonkey trunk nightly builds from 20070729.

StR:
1. Create a profile with a non-ASCII character like “α”, “ü”, or “а” in its name (in this case, single characters were used).
...if you start Firefox with that profile now, the interface would look incorrectly.
2. Look at the profile's directory on disk and into profiles.ini


Expected results:

The character in UTF-8 in the directory name, Name, and Path variables in profiles.ini.


Actual Results (Minefield 2007072904)

When creating the profile from GUI:

PM:       α  (03b1) |   ü  (00fc) |   а  (0430)
IniName ceb1 (03b1) | c3bc (00fc) | b0d0 (0430) | CORRECT
Disk:           b1  |         fc  |         30
IniPath c2b1 (00b1) | c3bc (00fc) |   30 (0030)

The most significant byte from the UTF-16 representation is lost, the rest is used in the directory name on disk, and 00+rest is converted from UTF-16 into UTF-8 and used in the Path variable in profiles.ini.


CLI
CL:       α  (03b1)
IniName ceb1 (03b1) | CORRECT
Disk:   ceb1 (03b1) | CORRECT
IniPath c38ec2b1 (ce00b100)

The UTF-8 representation of the character gets 00s added to its bytes, and that is used in the Path variable in profiles.ini.


Regression ranges:

http://tinyurl.com/32osb2
Minefield:
2007062804(?) - WFM
2007062904 - happens

http://tinyurl.com/26qvjz
Thunderbird
2007072303 - WFM
2007072403 - happens

http://tinyurl.com/3cffoy
Seamonkey:
2007072402 - WFM
2007072501 - happens

Comment 1

[Boris Zbarsky [:bzbarsky]]

Hmm.... nothing in the ranges jumps out at me, and they're all different.  Could this have been some sort of build config change?

Comment 2

[Adam Guthrie]

Yep, it was due to enabling Breakpad on each platform (via adding MOZILLA_OFFICIAL to various Makefiles). (We figured this out in #breakpad on IRC, but I guess it never made it back to this bug.)

Comment 3

[[:Aleksej]]

Minefield file:// listings also don't support UTF-8 since that build.

Comment 4

[(not currently active) Ted Mielczarek]

This is definitely caused by Breakpad somehow.  Setting "Enabled=0" in application.ini causes this bug to disappear.

Comment 5

[(not currently active) Ted Mielczarek]

So uh, it breaks on this line:
http://bonsai.mozilla.org/cvsblame.cgi?file=/mozilla/toolkit/crashreporter/nsExceptionHandler.cpp&rev=1.26#351

If I stick a return NS_OK; right before that line, everything works.  If I stick one right after that line, I see the bug.  So confused.

Comment 6

[(not currently active) Ted Mielczarek]

It turns out that calling nsIFile::Append pre-XPCOM startup gets us into this situation.  Using AppendNative works.  Patch coming up.

Comment 7

[(not currently active) Ted Mielczarek]

Attachment: fix use of Append and GetNativePath

I haven't tested this out on Mac/Windows yet, I have a build going on the TryServer currently.  This fixes the issue on Linux, and also fixes some other areas that weren't unicode-safe on Windows.

Comment 8

[(not currently active) Ted Mielczarek]

You can test with this build:
https://build.mozilla.org/tryserver-builds/[e-mail]2

Comment 9

[(not currently active) Ted Mielczarek]

Comment on attachment 280778 [details] [diff] [review]
fix use of Append and GetNativePath

Doesn't break compiling on other platforms, and fixes the bug here.

Comment 10

[(not currently active) Ted Mielczarek]

Checked in.

Comment 11

[[:Aleksej]]

Seems to WFM with Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a8pre) Gecko/2007091704 Minefield/3.0a8pre and "æабцaαü".

Comment 12

[(not currently active) Ted Mielczarek]

Feel free to mark verified...

Comment 13

[[:Aleksej]]

Verified also with
Thunderbird version 3.0a1pre (2007091703)
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a8pre) Gecko/2007091701 SeaMonkey/2.0a1pre