1640131 - OSError.prototype.toString can fail with various language locales

Status: RESOLVED FIXED

(Toolkit Graveyard :: OS.File, defect)

Description

[Mark Banner (:standard8)]

STR:

• On a Ubuntu VM, I installed the Chinese character packs.
• Then I started Firefox with LANG=zh_CN.gb18030 firefox
• Hooked up the browser toolbox.
• In the console I enter OS.File.read("/home/testshouldntexist").catch(ex => ex.toString())

Expected Results

An error message that states the 'file was not found' in an appropriate language.

Actual Results

An error gets thrown:

TypeError: malformed UTF-8 character sequence at offset 2 osfile_unix_allthreads.jsm:113:43

This is currently breaking the search service on startup, as we're trying to log the error with the toString() conversion and that itself is throwing.

The error seems to come from the readString() line:

https://searchfox.org/mozilla-release/rev/6e64ba3b035d5c1829970c0e8ca9fab962d831ed/toolkit/components/osfile/modules/osfile_unix_allthreads.jsm#113

The definition for the ctypes function there, is to read char* pointers:

https://searchfox.org/mozilla-release/rev/6e64ba3b035d5c1829970c0e8ca9fab962d831ed/toolkit/components/osfile/modules/osfile_unix_allthreads.jsm#61-69

This should probably be changed to support native encoding. A workaround could be to use ReadStringReplaceMalformed but that wouldn't give a good error message in this case.

Comment 1

[Mark Banner (:standard8)]

Henri, do you have an idea of how many people use LANG=zh_CN.gb18030 or how an idea of how many people this bug would likely to affect? Its affect on the search service is mainly for new users, but it is bad once you hit it.

Also, do you have an idea for a solution here? We can do the workaround I suggested above, or worst case in the search service. Though I wonder if this could affect other parts of Firefox as well.

Comment 2

[Henri Sivonen (:hsivonen)]

(In reply to Mark Banner (:standard8) from comment #1)

Henri, do you have an idea of how many people use LANG=zh_CN.gb18030 or how an idea of how many people this bug would likely to affect?

Very few people. Ubuntu has never shipped with non-UTF-8 locale by default. Debian has shipped with UTF-8 locale by default since 2007. Red Hat has shipped with UTF-8 locale by default since 2002. GB18030 was published in 2005.

For all practical purposes, setting the glibc locale to a non-UTF-8 locale is asking for the system to break, but the attractive nuisance of non-UTF-8 locales stays available due to glibc backward compatibility principles.

Also, do you have an idea for a solution here?

I think the solution worth implementing is to switch to ReadStringReplaceMalformed to prevent Firefox functionality from breaking and to allow the error message to be unreadable. Ideally, we wouldn't expose any glibc-generated natural-language messages but would generate our own messages based on error numbers.

We already document that we don't support running with the Firefox profile placed at a non-UTF-8 file system path. However, as long as the profile is at an ASCII path, we've tried not to break of the glibc locale is set to a non-UTF-8 locale.

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:Yoric, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 4

[Mark Banner (:standard8)]

Thanks for the information Henri. I'm going to do the quick fix here, it sounds like users won't generally be hitting it, but given this stops the search service working if it does get hit, then I'd rather just have it fixed.

Comment 5

[Mark Banner (:standard8)]

Attachment: Bug 1640131 - When getting the error string for OSError, handle non-utf-8 character sets. r?Yoric!

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ce6192a7bdb5
When getting the error string for OSError, handle non-utf-8 character sets. r=Yoric

Comment 7

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/ce6192a7bdb5