1162084 - Arabic problems in convertUTF8toACE()

Status: RESOLVED FIXED

(Core :: Internationalization, defect)

Description

[Gervase Markham [:gerv]]

I recently added the following entries to the Public Suffix List (over in bug 1161823):
 
// xn--mgbai9azgqp6j ("Pakistan", Urdu/Arabic) : PK
پاکستان

// xn--mgbai9a5eva00b ("Pakistan", Urdu/Arabic, variant) : PK
پاكستان

However, they caused netwerk/test/unit/test_bug414122.js to fail, as follows:

0:03.02 LOG: Thread-1 ERROR NS_ERROR_FAILURE: Component returned failure code: 
0x80004005 (NS_ERROR_FAILURE) [nsIIDNService.convertUTF8toACE]

Therefore, we had to back the changes out. Both of these entries seem to convert correctly in an online converter:
https://manage.centralnic.com/support/idn_converter

Is this a bug in our converter, or a problem with the entries?

Gerv

Comment 1

[Jothan Frakes]

Is is possible to comment the second, variant string, and re-test?

Comment 2

[Gervase Markham [:gerv]]

Both strings cause a problem; removing only one didn't fix it. I have a reduced test case - I'll upload it.

Gerv

Comment 3

[Gervase Markham [:gerv]]

Attachment: effective_tld_names.dat

Make this your effective_tld_names.dat and then run the test.

Something odd is happening, though - when I copied and pasted back in from a patch, it worked. Is there some zero-width character in here or something which is causing an issue? More investigation tomorrow.

Gerv

Comment 4

[Simon Montagu :smontagu]

The entries contain a couple of Bidi control characters which fail the "prohibited character" test

Comment 5

[Simon Montagu :smontagu]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=09cd08e7bd1e

Comment 6

[Simon Montagu :smontagu]

Attachment: Add back the entries for Pakistan without the control characters

Comment 7

[Gervase Markham [:gerv]]

Comment on attachment 8602291 [details] [diff] [review]
Add back the entries for Pakistan without the control characters

r=gerv.

Jothan: I don't know where you got these strings from, but you may want to let your source know about this potential problem. Their copies of the strings embed invisible characters which are not permitted.

Gerv

Comment 8

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/45547d19c355

Comment 9

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/45547d19c355

Comment 10

[Jothan Frakes]

Was cut-n-paste from the ICANN page - I wonder if the paste buffer into notepad++ did that?

Glad it is resolved