578751 - Use @loader_path instead of @executable_path in internal name of dylibs

Status: NEW

(Firefox Build System :: General, defect)

Description

[(not currently active) Ted Mielczarek]

Attachment: Use @loader_path instead of @executable_path

OS X 10.4's dyld added support for @loader_path which is like @executable_path, but looks in the directory containing whatever is linked to this dylib instead of the executable path. This seems like a more useful thing to do.

The only thing I'm not 100% sure on is how this affects things like binary components, where you really do want to load from @executable_path. If the modules being linked to are already in memory, presumably it doesn't matter, since they don't have to be found?

Comment 1

[Josh Aas]

Committed as a bustage fix, would be great to get review ASAP.

http://hg.mozilla.org/mozilla-central/rev/0c116ba35956

Comment 2

[Mark Banner (:standard8)]

So far, this has totally broken Thunderbird builds with errors similar to:

nsNativeModuleLoader::LoadModule("/Users/moztest/comm/main/tb/mozilla/dist/ShredderDebug.app/Contents/MacOS/components/libpref.dylib") - load FAILED, rv: 80004005, error:
	Unknown error: -2804

I'm hoping I just need to port the patch and rebuild. I've already tried clobbering specific areas, but that didn't work so I'm now doing a full clobber.

Comment 3

[Mark Banner (:standard8)]

Having done a clobber, and tried a fresh build from tinderbox, my current theory is that this patch breaks developers who want to run objdir builds.

Comment 4

[Mark Banner (:standard8)]

After having done some further testing, it seems like only Thunderbird shared builds are broken. 

Thunderbird static builds are fine, as are Firefox objdir runs. From tinderbox, SeaMonkey shared builds seem to be running ok as well.

Therefore I'm a bit confused as to what the issue is.

Comment 5

[Markus Stange [:mstange]]

My disable-libxul Firefox debug build with objdir is also broken.

Comment 6

[Peter Van der Beken [:peterv]]

(In reply to comment #5)
> My disable-libxul Firefox debug build with objdir is also broken.

Same here.

Comment 7

[Justin Wood (:Callek)]

(In reply to comment #4)
> After having done some further testing, it seems like only Thunderbird shared
> builds are broken. 

> SeaMonkey shared builds seem to be running ok as well.

Well I don't think the SM builds had been clobbered.

> Therefore I'm a bit confused as to what the issue is.

Given c#5 and c#6 I'm inclined to think we might need to re-evaluate/backout this patch from m-c

Comment 8

[Josh Aas]

The 737a08309d97 mozilla-central changeset contains the last of a series of bustage fixes. Using that changeset or higher with a clobbered build should work fine, is that not the case for anyone?

Comment 9

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Bug 580360 is an excellent example of why landing patches hours before code freeze without review is not a good idea.

Comment 10

[Benjamin Smedberg]

This has been backed out.

Comment 11

[Benjamin Smedberg]

Comment on attachment 457353 [details] [diff] [review]
Use @loader_path instead of @executable_path

The specific thing that this patch got wrong was that @loader_path/... is only correct if the component is in the same directory as xpcom. If it is in the components/ subdirectory, it has to be @loader_path/../libname.

And I'm not exactly sure what's supposed to happen for extensions, since it seems clear that @loader_path does *not* work correctly in that case.

Comment 12

[(not currently active) Ted Mielczarek]

Hence my caveats in comment 0. I'm not sure if this patch can work right. There's also an @rpath directive, it's possible we could use that and tweak it to work correctly.

Comment 13

[sann9270]

For information, because of the use of @executable_path in internal dylib, some third party products like Java applets that use the NSS dylib provided with Firefox to access Firefox keystore are broken now.

This occurs with the official release of Firefox 4.0 on MacOX Snow Leopard 10.6.x.

With previous versions of firefox (3.5/3.6) which use java embedding plug-in, @executable_path seemed to be resolved relative to firefox executables directory.

In Firefox 4.0 with Java for NAPI plugin, @executable_path is resolved relative to /Library/Java/Home/bin/java. The consequences of this new behaviour are:

- For third party dylib loaded by an applet and dynamically linked with Firefox libnss3.dylib using @executable_path/libnss3.dylib path, the third party library loading will fail as the system will search nss3 in java directory instead of firefox directory.

- Even if third party product directly preloads the libnss3.dylib by specifying an absolute path (/Applications/Firefox.app/Contents/MacOS/libnss3.dylib), the nss3 library loading will fail nonetheless because libnspr4.dylib contains sub-dependencies using @executable_path (e.g. @executable_path/libnssutil3.dylib).

- It is not possible to manually preload all dependent dylibs provided with Firefox from leaves to root, (nspr4 first, then plc4, ..., finally nss3). The loading of libnss3.dylib will fail because it seems that the system dyld linker still wants to load nss3 dependencies as @executable_path/libnspr4.dylib, even if /Applications/Firefox.app/Contents/MacOS/libnspr4.dylib has been successfully preloaded by the applet.

The conclusion is that it is not possible anymore for an applet to load the NSS dylib provided with firefox 4 to access the firefox keystore. One possible fix to this problem is to release new official versions of firefox 4 on MacOS with NSS related dylibs using @loader_path instead of @executable_path.

Remarks: there are 2 other possible workarounds, but, as it is the case with the first one, they all need to be implemented in the official Mozilla Firefox releases: First one is to re-integrate the Firefox 3.6 applet launcher mechanism (not reasonable). Second is to correctly set DYLB_LIBRARY_PATH when Firefox is launched from the Dock as it is done when Firefox is launched from terminal using the shell script firefox and run-mozilla.sh.

Comment 14

[pavrw]

Related to: https://bugzilla.mozilla.org/show_bug.cgi?id=654939

Comment 15

[(not currently active) Ted Mielczarek]

(In reply to comment #13)
> For information, because of the use of @executable_path in internal dylib,
> some third party products like Java applets that use the NSS dylib provided
> with Firefox to access Firefox keystore are broken now.

Note that this behavior has not changed in Firefox. We have always shipped our OS X binaries like this. This bug is still open...

> With previous versions of firefox (3.5/3.6) which use java embedding
> plug-in, @executable_path seemed to be resolved relative to firefox
> executables directory.
> 
> In Firefox 4.0 with Java for NAPI plugin, @executable_path is resolved
> relative to /Library/Java/Home/bin/java. The consequences of this new
> behaviour are:

This sounds like a change due to the Java plugin, exposed because we prefer the new version of Java over the old. Regardless, let's take this discussion to bug 654939.

Comment 16

[pavrw]

IMHO, This still needs to be discussed, even with bug 654939 marked as wontfix.

This issue is causing problems with Secmod and SunPKCS#11 Java objects also.

Any related progress? Can we consider this bug confirmed/ready to assign?

Comment 17

[pavrw]

A simple example of what's goin on:

java.io.IOException: dlopen(/Applications/Firefox.app/Contents/MacOS/libnss3.jnilib, 1): Library not loaded: @executable_path/libnssutil3.dylib
  Referenced from: /Applications/Firefox.app/Contents/MacOS/libnss3.jnilib
  Reason: image not found

A library that cannot reference itself???

This works properly for this and other libraries, when built with @loader_path

Comment 18

[(not currently active) Ted Mielczarek]

We do not have any plans to fix this any time soon. Dynamically loading the NSS libraries from the Firefox application directory is not something we intend to support.

Comment 19

[pavrw]

Going to discuss this on list. I hope to see you there ;). Thanks.