636845 - linux 64 bit builds do not properly load system plugins or extensions

Status: RESOLVED INCOMPLETE

(Firefox Build System :: General, defect)

Description

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: this patch sets the location of the system plugins and extensions folder based on being a 64bit os instead of the libdir setting

I have noticed that while running nighlties that system installed plugins (i.e. plugins installed through apt/yum) aren't detected by minefield.

Using strace I found that these builds are actually scanning /usr/lib/mozilla/plugins instead of /usr/lib64/mozilla/plugins on linux64.  The way that the code currently figures out whether to use /usr/lib64 or /usr/lib is to search for 'lib64' in the autoconf libdir [1].  Instead, I think it should be set to a commonly used value that works on the majority of distributions.

The patch I have attached changes the system plugin path from being based on the libdir autoconf.mk value to choosing the path based on whether the system is a 64bit Linux OS.  I have verified that the /usr/lib64/mozilla/plugins is the default for CentOS [2] and Fedora [3] and /usr/lib64 is a symlink for /usr/lib on Debian[4] and Ubuntu [5] which makes this change invisible.

I have also verified that this patch fixes the behaviour on my system.  I have made the same change for the system extensions directory which used the same method of figuring out whether or not to use /usr/lib64.

With my patch applied I have verified that my build is finding my system plugins and extensions.

Plugins:

access("/usr/lib64/mozilla/plugins", F_OK) = 0
open("/usr/lib64/mozilla/plugins", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 67
getdents(67, /* 8 entries */, 32768)    = 352
lstat("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/usr/lib64", {st_mode=S_IFDIR|0555, st_size=73728, ...}) = 0
lstat("/usr/lib64/mozilla", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/usr/lib64/mozilla/plugins", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/usr/lib64/mozilla/plugins/libtotem-narrowspace-plugin.so", {st_mode=S_IFREG|0755, st_size=84488, ...}) = 0
stat("/usr/lib64/mozilla/plugins/libtotem-narrowspace-plugin.so", {st_mode=S_IFREG|0755, st_size=84488, ...}) = 0
<snip>

Extensions:

access("/usr/lib64/mozilla/plugins", F_OK) = 0
open("/usr/lib64/mozilla/plugins", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 67
getdents(67, /* 8 entries */, 32768)    = 352
lstat("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/usr/lib64", {st_mode=S_IFDIR|0555, st_size=73728, ...}) = 0
lstat("/usr/lib64/mozilla", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/usr/lib64/mozilla/plugins", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/usr/lib64/mozilla/plugins/libtotem-narrowspace-plugin.so", {st_mode=S_IFREG|0755, st_size=84488, ...}) = 0
stat("/usr/lib64/mozilla/plugins/libtotem-narrowspace-plugin.so", {st_mode=S_IFREG|0755, st_size=84488, ...}) = 0

(this is extension is from the beagle-firefox-0.3.9-19.fc14.x86_64 package)

The same end can be achieved without modifying the build system by setting

PREFIX=/usr/local
ac_add_options --prefix=${PREFIX}
ac_add_options --libdir=${PREFIX}/lib64

but does not address the root problem.

[1] http://mxr.mozilla.org/mozilla-central/source/toolkit/xre/Makefile.in#236
[2]
[jhford@localhost ~]$ cat /etc/issue.net 
CentOS release 5.5 (Final)
Kernel \r on an \m
[jhford@localhost ~]$ file /usr/lib64
/usr/lib64: directory
[jhford@localhost ~]$ ls /usr/lib64/mozilla/plugins
[jhford@localhost ~]$ yum provides /usr/lib64/mozilla/plugins/*.so
<snip>
totem-mozplugin-2.16.7-7.el5.x86_64 : Mozilla plugin for Totem
Repo        : updates
Matched from:
Filename    : /usr/lib64/mozilla/plugins/libtotem-basic-plugin.so
Filename    : /usr/lib64/mozilla/plugins/libtotem-narrowspace-plugin.so
Filename    : /usr/lib64/mozilla/plugins/libtotem-complex-plugin.so
Filename    : /usr/lib64/mozilla/plugins/libtotem-gmp-plugin.so
Filename    : /usr/lib64/mozilla/plugins/libtotem-mully-plugin.so

[3]
[jhford@singe ~]$ cat /etc/issue.net
Fedora release 14 (Laughlin)
Kernel \r on an \m (\l)
[jhford@singe ~]$ file /usr/lib64
/usr/lib64: directory
[jhford@singe ~]$ ls /usr/lib64/mozilla/plugins
libjavaplugin.so                       libtotem-gmp-plugin.so
librhythmbox-itms-detection-plugin.so  libtotem-mully-plugin.so
libtotem-cone-plugin.so                libtotem-narrowspace-plugin.so
[jhford@singe ~]$ yum provides /usr/lib64/mozilla/plugins/librhythmbox-itms-detection-plugin.so
<snip>
rhythmbox-0.13.3-1.fc14.x86_64 : Music Management Application
Repo        : installed
Matched from:
Other       : Provides-match:
            : /usr/lib64/mozilla/plugins/librhythmbox-itms-detection-plugin.so
[4]
jhford@debian6-64bit:~$ cat /etc/issue.net 
Debian GNU/Linux 6.0
jhford@debian6-64bit:~$ file /usr/lib64
/usr/lib64: symbolic link to `lib'
jhford@debian6-64bit:~$ ls /usr/lib64/mozilla/plugins
flash-mozilla.so                       libtotem-gmp-plugin.so
librhythmbox-itms-detection-plugin.so  libtotem-mully-plugin.so
libtotem-cone-plugin.so                libtotem-narrowspace-plugin.so
jhford@debian6-64bit:~$ 

[5]
jhford@ubuntu1010-vm:~$ cat /etc/issue.net
Ubuntu 10.10
jhford@ubuntu1010-vm:~$ file /usr/lib64
/usr/lib64: symbolic link to `lib'
jhford@ubuntu1010-vm:~$ ls /usr/lib64/mozilla/plugins
flashplugin-alternative.so             libtotem-gmp-plugin.so
libjavaplugin.so                       libtotem-mully-plugin.so
librhythmbox-itms-detection-plugin.so  libtotem-narrowspace-plugin.so
libtotem-cone-plugin.so
jhford@ubuntu1010-vm:~$

Comment 1

[John Ford [:jhford] CET/CEST Berlin Time]

(In reply to comment #0)
> Extensions:
> 
> access("/usr/lib64/mozilla/plugins", F_OK) = 0
> open("/usr/lib64/mozilla/plugins", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) =
> 67
> getdents(67, /* 8 entries */, 32768)    = 352
> lstat("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
> lstat("/usr/lib64", {st_mode=S_IFDIR|0555, st_size=73728, ...}) = 0
> lstat("/usr/lib64/mozilla", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
> lstat("/usr/lib64/mozilla/plugins", {st_mode=S_IFDIR|0755, st_size=4096, ...})
> = 0
> lstat("/usr/lib64/mozilla/plugins/libtotem-narrowspace-plugin.so",
> {st_mode=S_IFREG|0755, st_size=84488, ...}) = 0
> stat("/usr/lib64/mozilla/plugins/libtotem-narrowspace-plugin.so",
> {st_mode=S_IFREG|0755, st_size=84488, ...}) = 0

Copy and paste don't seem to work all that well on linux

access("/usr/lib64/mozilla/extensions/{ec8030f7-c20a-464f-9b0e-13a3a9e97384}", F_OK) = 0
stat("/usr/lib64/mozilla/extensions/{ec8030f7-c20a-464f-9b0e-13a3a9e97384}", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/usr/lib64/mozilla/extensions/{ec8030f7-c20a-464f-9b0e-13a3a9e97384}", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 32
getdents(32, /* 2 entries */, 32768)    = 48
getdents(32, /* 0 entries */, 32768)    = 0
close(32)

Comment 2

[Benjamin Smedberg]

I don't think there is a good default: fedora-style distros use /usr/lib64, but debian (including ubuntu) use /usr/lib for 64 and /usr/lib32 for 32-bit, IIRC.

Comment 3

[(not currently active) Ted Mielczarek]

CCing some distro folks.

Comment 4

[Chris Coulson]

On Ubuntu, we do have a /usr/lib64, but it's just a symlink to /usr/lib. This will probably change in the future though (see https://wiki.ubuntu.com/MultiarchSpec#Filesystem%20layout)

Comment 5

[Wolfgang Rosenauer [:wolfiR]]

Most 64bit Linux archs are using lib64 (non-Debian based ones) for their 64bit variant but not all.
IPF aka IA64 (in case of SUSE Linux Enterprise) is using lib and has a very special library location as it uses an emulator to execute 32bit code.
So I agree with Benjamin that there is no good default but apparently the current behaviour worked for everyone somehow? I don't get why it isn't working for you? This is also a reason why one build which works perfectly on every Linux is not really possible.

Comment 6

[Christopher Aillon (sabbatical, not receiving bugmail)]

(Bug 311008 where this was originally implemented has some discussion related to this, too.)

The patch as written will force 32 bit builds created on a 64 bit build host to seek in /usr/lib64 which is definitely wrong behavior.

It looks like you're building with --libdir=/usr/local/lib which I think is wrong for a 64 bit build.  It is quite reasonable to expect that you build 64bit with --prefix=foo --libdir=$(prefix)/lib64.  This is what we do for every package in Fedora/RHEL, and I think that other distros do similar.

Comment 7

[John Ford [:jhford] CET/CEST Berlin Time]

(In reply to comment #6)
> (Bug 311008 where this was originally implemented has some discussion related
> to this, too.)
> 
> The patch as written will force 32 bit builds created on a 64 bit build host to
> seek in /usr/lib64 which is definitely wrong behavior.

I am not an autoconf expert, but it looks like HAVE_64BIT_OS is defined based on target pointer size [1] and explicitly checked to not be defined on builds targeting x86 [2].

[1]http://mxr.mozilla.org/mozilla-central/source/configure.in#1817
[2]http://mxr.mozilla.org/mozilla-central/source/configure.in#2589

Comment 8

[(not currently active) Ted Mielczarek]

Comment on attachment 515193 [details] [diff] [review]
this patch sets the location of the system plugins and extensions folder based on being a 64bit os instead of the libdir setting

I'm deferring review here to an actual Linux expert (there are quite a few CCed now).

Comment 9

[Christopher Aillon (sabbatical, not receiving bugmail)]

(In reply to comment #7)
> I am not an autoconf expert, but it looks like HAVE_64BIT_OS is defined based
> on target pointer size [1] and explicitly checked to not be defined on builds
> targeting x86 [2].

Sorry, you're indeed right, but that wasn't the relevant part of my post.

I don't feel this fix is correct since many 64bit unix builds simply don't have /usr/lib64.  I think the correct thing to do is one of:

a) update the m.o x86-64 build configs to set libdir=$(prefix)/lib64

This is the easiest and least regression prone thing to do, and is what I had in mind for 64 bit builds to do when I wrote the original patch to support this.

b) update configure.in to default to libdir=$(prefix)/lib64 if libdir is not set on 64bit platforms, and then unilaterally use $LIBDIR/mozilla and $DATADIR/mozilla

This would be most portable, however would mean that private builds with prefix=/usr/local would not load system plugins/extensions installed to prefix=/usr (which is the most likely place they will be).  There's also a chance of regressing other platforms.

Comment 10

[Yannick Patois]

Hi,

I also hit this problem with latest 5.0 x86_64 build.

Suddenly, when moving to 5.0 from 4.0.1 all my plugins disappeared (4.0.1 was also 64 bits but that was working then).

As I'm only using 64 bits mozilla on that machine, I just symlinked the plugin and extension directories from /usr/lib64/mozilla to /usr/lib/mozilla, which would likely not work if I still wanted to simultaneously run any 32 bits firefox.

To me, a 64 bits firefox should scan both lib and lib64, starting by lib64, and use what it found i them.


Thanks for your attention.