792156 - Firefox can't open International URLs from Mac LaunchURL

Status: RESOLVED DUPLICATE of bug 309671

(Core :: Widget: Cocoa, defect)

Description

[Marc Kaufman]

User Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; MDDR; .NET4.0C; .NET4.0E; BRI/2)

Steps to reproduce:

URL: "http://www.vihtilä.fi" (the 'ä' is U+00E4) encoded as UTF8
Mac code:
CFURLRef destURL = CFURLCreateWithBytes(kCFAllocatorDefault, url, strlen(url), kCFStringEncodingUTF8, NULL);
Make a LSLaunchURLSpec with this destURL, null App specification (Firefox is default browser)
LSOpenFromURLSpec (...);


Actual results:

Firefox 12.0 opens, address bar is correct("www.vihtilä.fi"), but screen message is "Firefox can't find the server at "http://www.vihtil%c3%a4.fi"
Copying the address bar and pasting it back causes the correct site to open.


Expected results:

Firefox should open the address correctly. Problem does not happen with Safari 6.0 or Chrome 21.0.1180.89.
If I give the URL as percent encoded, it still doesn't work. If I give the URL from a UniChar (UTF16LE) string, the CFURLCreate returns NULL.
also see: https://support.mozilla.org/en-US/questions/936988

Comment 1

[Matthias Versen [:Matti]]

It should work if you use the Punycode

Comment 2

[Marc Kaufman]

It is not up to the caller to convert to Punycode, because that involves a number of steps including normalization of the Unicode form. Conversion is part of Firefox (as evidenced by the fact that I can paste the UTF8 string into the address bar and have the site open properly). The Firefox side of LSOpenFromURLSpec should process the CFURLRef the same way as pasting or open from a command line.

Comment 3

[Boris Zbarsky [:bzbarsky]]

Reopening.  Steven, do you know where this code lives on our side?  That is, what does LSOpenFromURLSpec end up calling into?

Comment 4

[Steven Michaud [:smichaud] (Retired)]

The code from comment #0 should result in the OS sending Firefox an "Apple event".  That gets handled here:

http://hg.mozilla.org/mozilla-central/annotate/3c68fdd4f77a/toolkit/xre/MacApplicationDelegate.mm#l331

Comment 5

[Steven Michaud [:smichaud] (Retired)]

(In reply comment #0)

> Firefox 12.0

Were you really testing with Firefox 12.0?  The current version is 15.0.1.

Please test with FF 15.0.1, and also with a current trunk (aka mozilla-central) nightly (http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/).

Comment 6

[Steven Michaud [:smichaud] (Retired)]

And what version of OS X were you testing on?

That may make a difference.

Comment 7

[Boris Zbarsky [:bzbarsky]]

So presumably the relevant line is:

   369    const char *argv[3] = {nullptr, "-url", [urlString UTF8String]};

and then we go ahead and run the commandline?  This should work, in general...

Marc, how do I actually go about reproducing this so I can debug what happens from there?

Comment 8

[Steven Michaud [:smichaud] (Retired)]

Testing with FF 15.0.1 and today's mozilla-central nightly on OS X 10.7.4, "open http://www.vihtilä.fi/" (from a Terminal prompt) fails exactly as described in comment #0.

In order to control which FF distro gets the Apple event, I tested with that FF distro already running.

Later I'll test on other versions of OS X.

(Boris, this should do as your testcase.)

Comment 9

[Marc Kaufman]

I originally tested witn 12.0, but have upgraded to 15.0 with the same result. I am using OSX 10.7. Apple has some problems with CFString (and CFURLRef), and can't make a CFURLRef from a UniChar string, for instance. I've reported those issues to Apple.

Comment 10

[Boris Zbarsky [:bzbarsky]]

Simon, thanks.  That's perfect.

When I run the command from comment 8, what I see is that [urlString UTF8String] in the code you linked to is returning "http://www.vihtil%C3%A4.fi/".  So it looks like something has munged the string to do %-escaping of non-ASCII characters before we ever get to this point.

And in fact, if I examine the AppleEvent passed in:

(gdb) po event  
<NSAppleEventDescriptor: 'GURL'\'GURL'{ '----':"http://www.vihtil%C3%A4.fi/" }>

So it looks like this is in fact a duplicate of bug 309671: something in the Apple code is converting from Unicode to %-escaped form...

Comment 11

[Marc Kaufman]

Here is a snippet of the code I am running:
	char* cURL = "http://www.vihtilä.fi/";

	CFURLRef destURLRef = CFURLCreateWithBytes(kCFAllocatorDefault, (UInt8*)cURL, strlen(cURL), kCFStringEncodingUTF8, NULL);
	
	LSLaunchURLSpec launchURLSpec;
	memset(&launchURLSpec, 0,  sizeof(launchURLSpec));

	launchURLSpec.itemURLs = CFArrayCreate(kCFAllocatorDefault, (const void **) &destURLRef, 1, &kCFTypeArrayCallBacks);
	
	if (noErr == LSOpenFromURLSpec(&launchURLSpec, NULL))
		success = true;

Comment 12

[Boris Zbarsky [:bzbarsky]]

Marc, can you please post a full C file I can compile and run to test this?

Comment 13

[Marc Kaufman]

Attachment: Command line project to demonstrate bug

Comment 14

[Boris Zbarsky [:bzbarsky]]

Thanks, looking into this.

Comment 15

[Boris Zbarsky [:bzbarsky]]

OK, so when I run that testcase, on OS 10.8, after the call to CFURLCreateWithBytes, if I examine destURLRef, I get:

(gdb) po destURLRef
http://www.vihtil%C3%A4.fi/

If I then look at the AppleEvent we get on our side, that also has the %-escaped hostname, and the error page I get in Firefox says:

  Firefox can't find the server at www.vihtil%c3%a4.fi.

So the escaping is happening in the CFURLCreateWithBytes call, as far as I can tell.  Again, on 10.8.

In bug 309671 comment 111 you said that there is some situation where you get a \xc4\xa4 somewhere?  Where is that, exactly?  Which OS X version are you on?

Comment 16

[Marc Kaufman]

I was getting \xC3\xA4 with FF 12.0. I'm running OSX 10.7.4. I'll retest on Monday with the actual Acrobat code and FF 15.0

Comment 17

[Marc Kaufman]

After the latest OS updates and running FF 15.0.1 I get: "Firefox can't find the server at www.vihtil%c3%a4.fi." Software  Mac OS X Lion 10.7.5 (11G56). I did have an OS update and FF update since the previous tests. So maybe we go back to the other bug and continue the fix there. -Marc

Comment 18

[Boris Zbarsky [:bzbarsky]]

Yeah, I think that's the best way to get somewhere here, since it looks like Apple is helpfully %-escaping our UTF-8.  :(