Last Comment Bug 762621 - Thunderbird 13 can't find its dynamic libs on Linux
: Thunderbird 13 can't find its dynamic libs on Linux
Status: RESOLVED FIXED
:
Product: MailNews Core
Classification: Components
Component: Build Config (show other bugs)
: 13
: x86 Linux
: -- normal (vote)
: Thunderbird 16.0
Assigned To: Mike Hommey [:glandium]
:
:
Mentors:
: 745518 764461 (view as bug list)
Depends on:
Blocks: 668869
  Show dependency treegraph
 
Reported: 2012-06-07 12:10 PDT by Gordon Lack
Modified: 2012-08-31 11:08 PDT (History)
8 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
+
fixed
+
fixed
+
fixed
unaffected


Attachments
Force loading NSPR before prldap60 (1.29 KB, patch)
2012-06-12 04:54 PDT, Mike Hommey [:glandium]
standard8: review+
standard8: approval‑comm‑aurora+
standard8: approval‑comm‑beta+
standard8: approval‑comm‑release+
Details | Diff | Splinter Review

Description Gordon Lack 2012-06-07 12:10:12 PDT
User Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:13.0) Gecko/20100101 Firefox/13.0
Build ID: 20120601045813

Steps to reproduce:

Upgrade from 12 (OK) to 13.
Tired starting 13


Actual results:

[mysys]: /local/i586/thunderbird/thunderbird
XPCOMGlueLoad error for file /local/i586/thunderbird/libxpcom.so:
libxul.so: cannot open shared object file: No such file or directory
Couldn't load XPCOM.



Expected results:

It should have started thunderbird.

I had to set LD_LIBRARY_PATH to /local/i586/thunderbird to get that to work.  Previous versions have not required this (and neither does Firefox v13).
Comment 1 Mark Banner (:standard8) 2012-06-11 05:46:41 PDT
Presumably you are using the builds from http://www.mozilla.org/thunderbird ? Which Linux OS are you using?

What does the output of:

# ldd /local/i586/thunderbird/thunderbird 

say?
Comment 2 Mike Hommey [:glandium] 2012-06-11 06:26:02 PDT
# ldd /local/i586/thunderbird/libxul.so

would be more useful.
Comment 3 Gordon Lack 2012-06-11 12:44:02 PDT
This is running on a Mandriva 2011 system.
It is the standard mozilla.org builds (indeed - it came along as a standard update, so v12 did *not* suffer this problem).

ldd outputs requested:

[mysys]: ldd /local/i586/thunderbird/thunderbird
        linux-gate.so.1 =>  (0xf7709000)
        libpthread.so.0 => /lib/libpthread.so.0 (0xf76cd000)
        libdl.so.2 => /lib/libdl.so.2 (0xf76c8000)
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xf75df000)
        libm.so.6 => /lib/libm.so.6 (0xf75b5000)
        libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0xf7597000)
        libc.so.6 => /lib/libc.so.6 (0xf741a000)
        /lib/ld-linux.so.2 (0xf770a000)

[mysys]: ldd /local/i586/thunderbird/libxul.so 
        linux-gate.so.1 =>  (0xf7796000)
        libpthread.so.0 => /lib/libpthread.so.0 (0xf5f2d000)
        librt.so.1 => /lib/librt.so.1 (0xf5f23000)
        libsmime3.so => not found
        libssl3.so => not found
        libnss3.so => not found
        libnssutil3.so => not found
        libXrender.so.1 => /usr/lib/libXrender.so.1 (0xf5f17000)
        libfreetype.so.6 => /usr/lib/libfreetype.so.6 (0xf5e8f000)
        libfontconfig.so.1 => /usr/lib/libfontconfig.so.1 (0xf5e5a000)
        libldap60.so => not found
        libprldap60.so => not found
        libldif60.so => not found
        libmozsqlite3.so => not found
        libasound.so.2 => /usr/lib/libasound.so.2 (0xf5d65000)
        libplds4.so => not found
        libplc4.so => not found
        libnspr4.so => not found
        libdl.so.2 => /lib/libdl.so.2 (0xf5d5f000)
        libmozalloc.so => not found
        libdbus-glib-1.so.2 => /usr/lib/libdbus-glib-1.so.2 (0xf5d37000)
        libdbus-1.so.3 => /lib/libdbus-1.so.3 (0xf5ced000)
        libglib-2.0.so.0 => /lib/libglib-2.0.so.0 (0xf5bfe000)
        libX11.so.6 => /usr/lib/libX11.so.6 (0xf5ac5000)
        libXext.so.6 => /usr/lib/libXext.so.6 (0xf5ab2000)
        libpangoft2-1.0.so.0 => /usr/lib/libpangoft2-1.0.so.0 (0xf5a83000)
        libpangocairo-1.0.so.0 => /usr/lib/libpangocairo-1.0.so.0 (0xf5a76000)
        libpango-1.0.so.0 => /usr/lib/libpango-1.0.so.0 (0xf5a2d000)
        libcairo.so.2 => /usr/lib/libcairo.so.2 (0xf5973000)
        libgobject-2.0.so.0 => /lib/libgobject-2.0.so.0 (0xf5924000)
        libgmodule-2.0.so.0 => /lib/libgmodule-2.0.so.0 (0xf591f000)
        libgtk-x11-2.0.so.0 => /usr/lib/libgtk-x11-2.0.so.0 (0xf54bd000)
        libatk-1.0.so.0 => /usr/lib/libatk-1.0.so.0 (0xf549c000)
        libgdk-x11-2.0.so.0 => /usr/lib/libgdk-x11-2.0.so.0 (0xf53ed000)
        libgdk_pixbuf-2.0.so.0 => /usr/lib/libgdk_pixbuf-2.0.so.0 (0xf53ca000)
        libXt.so.6 => /usr/lib/libXt.so.6 (0xf536d000)
        libgthread-2.0.so.0 => /lib/libgthread-2.0.so.0 (0xf5366000)
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xf527d000)
        libm.so.6 => /lib/libm.so.6 (0xf5253000)
        libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0xf5236000)
        libc.so.6 => /lib/libc.so.6 (0xf50b9000)
        /lib/ld-linux.so.2 (0xf7797000)
        libz.so.1 => /lib/libz.so.1 (0xf50a1000)
        libxml2.so.2 => /usr/lib/libxml2.so.2 (0xf4f54000)
        libgio-2.0.so.0 => /lib/libgio-2.0.so.0 (0xf4e27000)
        libpcre.so.0 => /lib/libpcre.so.0 (0xf4dea000)
        libxcb.so.1 => /usr/lib/libxcb.so.1 (0xf4dca000)
        libpixman-1.so.0 => /usr/lib/libpixman-1.so.0 (0xf4d46000)
        libpng12.so.0 => /usr/lib/libpng12.so.0 (0xf4d00000)
        libXfixes.so.3 => /usr/lib/libXfixes.so.3 (0xf4cfa000)
        libXinerama.so.1 => /usr/lib/libXinerama.so.1 (0xf4cf6000)
        libXi.so.6 => /usr/lib/libXi.so.6 (0xf4ce6000)
        libXrandr.so.2 => /usr/lib/libXrandr.so.2 (0xf4cdc000)
        libXcursor.so.1 => /usr/lib/libXcursor.so.1 (0xf4cd1000)
        libXcomposite.so.1 => /usr/lib/libXcomposite.so.1 (0xf4ccd000)
        libXdamage.so.1 => /usr/lib/libXdamage.so.1 (0xf4cc9000)
        libSM.so.6 => /usr/lib/libSM.so.6 (0xf4cc0000)
        libICE.so.6 => /usr/lib/libICE.so.6 (0xf4ca4000)
        libresolv.so.2 => /lib/libresolv.so.2 (0xf4c8c000)
        libXau.so.6 => /usr/lib/libXau.so.6 (0xf4c88000)
        libXdmcp.so.6 => /usr/lib/libXdmcp.so.6 (0xf4c81000)
Comment 4 Gordon Lack 2012-06-11 12:47:54 PDT
Here's an extract of an strace of the failed start-up (grep'ed for /lib).

It first finds libxul.so, then later goes looking for it in "standard" locations only and fails to find it.

4242  open("/lib/libpthread.so.0", O_RDONLY) = 3                                
4242  open("/lib/libdl.so.2", O_RDONLY) = 3                                     
4242  open("/usr/lib/libstdc++.so.6", O_RDONLY) = 3                             
4242  open("/lib/libm.so.6", O_RDONLY)  = 3                                     
4242  open("/usr/lib/libgcc_s.so.1", O_RDONLY) = 3                              
4242  open("/lib/libc.so.6", O_RDONLY)  = 3                                     
4242  lstat64("/local/i586/thunderbird/libxpcom.so", {st_mode=S_IFREG|0755, st_size=11824, ...}) = 0                                                            
4242  open("/local/i586/thunderbird/libldap60.so", O_RDONLY) = 4                
4242  open("/local/i586/thunderbird/libldap60.so", O_RDONLY) = 4                
4242  open("/local/i586/thunderbird/libprldap60.so", O_RDONLY) = 4              
4242  open("/local/i586/thunderbird/libprldap60.so", O_RDONLY) = 4              
4242  open("/lib/tls/i686/sse2/libplds4.so", O_RDONLY) = -1 ENOENT (No such file or directory)                                                                  
4242  stat64("/lib/tls/i686/sse2", 0xffeae58c) = -1 ENOENT (No such file or directory)                                                                          
4242  open("/lib/tls/i686/libplds4.so", O_RDONLY) = -1 ENOENT (No such file or directory)                                                                       
4242  stat64("/lib/tls/i686", 0xffeae58c) = -1 ENOENT (No such file or directory)                                                                               
4242  open("/lib/tls/sse2/libplds4.so", O_RDONLY) = -1 ENOENT (No such file or directory)                                                                       
4242  stat64("/lib/tls/sse2", 0xffeae58c) = -1 ENOENT (No such file or directory)                                                                               
4242  open("/lib/tls/libplds4.so", O_RDONLY) = -1 ENOENT (No such file or directory)
4242  stat64("/lib/tls", 0xffeae58c)    = -1 ENOENT (No such file or directory)
4242  open("/lib/i686/sse2/libplds4.so", O_RDONLY) = -1 ENOENT (No such file or directory)
4242  stat64("/lib/i686/sse2", 0xffeae58c) = -1 ENOENT (No such file or directory)
4242  open("/lib/i686/libplds4.so", O_RDONLY) = -1 ENOENT (No such file or directory)
4242  stat64("/lib/i686", 0xffeae58c)   = -1 ENOENT (No such file or directory)
4242  open("/lib/sse2/libplds4.so", O_RDONLY) = -1 ENOENT (No such file or directory)
4242  stat64("/lib/sse2", 0xffeae58c)   = -1 ENOENT (No such file or directory)
4242  open("/lib/libplds4.so", O_RDONLY) = -1 ENOENT (No such file or directory)
4242  stat64("/lib", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
4242  open("/usr/lib/tls/i686/sse2/libplds4.so", O_RDONLY) = -1 ENOENT (No such file or directory)
4242  stat64("/usr/lib/tls/i686/sse2", 0xffeae58c) = -1 ENOENT (No such file or directory)
4242  open("/usr/lib/tls/i686/libplds4.so", O_RDONLY) = -1 ENOENT (No such file or directory)
4242  stat64("/usr/lib/tls/i686", 0xffeae58c) = -1 ENOENT (No such file or directory)
4242  open("/usr/lib/tls/sse2/libplds4.so", O_RDONLY) = -1 ENOENT (No such file or directory)
4242  stat64("/usr/lib/tls/sse2", 0xffeae58c) = -1 ENOENT (No such file or directory)
4242  open("/usr/lib/tls/libplds4.so", O_RDONLY) = -1 ENOENT (No such file or directory)
4242  stat64("/usr/lib/tls", 0xffeae58c) = -1 ENOENT (No such file or directory)
4242  open("/usr/lib/i686/sse2/libplds4.so", O_RDONLY) = -1 ENOENT (No such file or directory)
4242  stat64("/usr/lib/i686/sse2", 0xffeae58c) = -1 ENOENT (No such file or directory)
4242  open("/usr/lib/i686/libplds4.so", O_RDONLY) = -1 ENOENT (No such file or directory)
4242  stat64("/usr/lib/i686", 0xffeae58c) = -1 ENOENT (No such file or directory)
4242  open("/usr/lib/sse2/libplds4.so", O_RDONLY) = -1 ENOENT (No such file or directory)
4242  stat64("/usr/lib/sse2", 0xffeae58c) = -1 ENOENT (No such file or directory)
4242  open("/usr/lib/libplds4.so", O_RDONLY) = -1 ENOENT (No such file or directory)
4242  stat64("/usr/lib", {st_mode=S_IFDIR|0755, st_size=20480, ...}) = 0
4242  open("/local/i586/thunderbird/libldif60.so", O_RDONLY) = 4
4242  open("/local/i586/thunderbird/libldif60.so", O_RDONLY) = 4
4242  open("/local/i586/thunderbird/libnspr4.so", O_RDONLY) = 4
4242  open("/local/i586/thunderbird/libnspr4.so", O_RDONLY) = 4
4242  open("/local/i586/thunderbird/libplc4.so", O_RDONLY) = 4
4242  open("/local/i586/thunderbird/libplc4.so", O_RDONLY) = 4
4242  open("/local/i586/thunderbird/libplds4.so", O_RDONLY) = 4
4242  open("/local/i586/thunderbird/libplds4.so", O_RDONLY) = 4
4242  open("/local/i586/thunderbird/libmozalloc.so", O_RDONLY) = 4
4242  open("/local/i586/thunderbird/libmozalloc.so", O_RDONLY) = 4
4242  open("/lib/librt.so.1", O_RDONLY) = 4
4242  open("/local/i586/thunderbird/libmozsqlite3.so", O_RDONLY) = 4
4242  open("/local/i586/thunderbird/libmozsqlite3.so", O_RDONLY) = 4
4242  open("/local/i586/thunderbird/libnssutil3.so", O_RDONLY) = 4
4242  open("/local/i586/thunderbird/libnssutil3.so", O_RDONLY) = 4
4242  open("/local/i586/thunderbird/libsoftokn3.so", O_RDONLY) = 4
4242  open("/local/i586/thunderbird/libsoftokn3.so", O_RDONLY) = 4
4242  open("/local/i586/thunderbird/libnss3.so", O_RDONLY) = 4
4242  open("/local/i586/thunderbird/libnss3.so", O_RDONLY) = 4
4242  open("/local/i586/thunderbird/libssl3.so", O_RDONLY) = 4
4242  open("/local/i586/thunderbird/libssl3.so", O_RDONLY) = 4
4242  open("/local/i586/thunderbird/libsmime3.so", O_RDONLY) = 4
4242  open("/local/i586/thunderbird/libsmime3.so", O_RDONLY) = 4
4242  open("/local/i586/thunderbird/libxul.so", O_RDONLY) = 4
4242  open("/local/i586/thunderbird/libxul.so", O_RDONLY) = 4
4242  open("/usr/lib/libXrender.so.1", O_RDONLY) = 4
4242  open("/usr/lib/libfreetype.so.6", O_RDONLY) = 4
4242  open("/usr/lib/libfontconfig.so.1", O_RDONLY) = 4
4242  open("/lib/libprldap60.so", O_RDONLY) = -1 ENOENT (No such file or directory)
4242  open("/usr/lib/libprldap60.so", O_RDONLY) = -1 ENOENT (No such file or directory)
4242  open("/lib/libxul.so", O_RDONLY)  = -1 ENOENT (No such file or directory)
4242  open("/usr/lib/libxul.so", O_RDONLY) = -1 ENOENT (No such file or directory)
4242  open("/local/i586/thunderbird/libxpcom.so", O_RDONLY) = 3
4242  open("/lib/libxul.so", O_RDONLY)  = -1 ENOENT (No such file or directory)
4242  open("/usr/lib/libxul.so", O_RDONLY) = -1 ENOENT (No such file or directory)
Comment 5 Gordon Lack 2012-06-11 12:51:20 PDT
In v121, thunderbird was a shell script (which set LD_LIBRARY_PATH).  v13 seems to have changed this.

Firefox underwent the same change earlier (v12 firefox was an executable too) but it still works.

I recall a thread about this for firefox earlier (about a year go?) but can't find the bug number (and it's not any of those mentioned in this ticket).
Comment 6 Mark Banner (:standard8) 2012-06-11 13:52:52 PDT
Can you check you've got a dependentlibs.text in your application directory and see what that contains?

From the strace:

4242  open("/local/i586/thunderbird/libprldap60.so", O_RDONLY) = 4
...
4242  open("/lib/libprldap60.so", O_RDONLY) = -1 ENOENT (No such file or directory)
4242  open("/usr/lib/libprldap60.so", O_RDONLY) = -1 ENOENT (No such file or directory)

That's interesting. Do you have a system auth based on LDAP installed?
Comment 7 Gordon Lack 2012-06-11 14:06:17 PDT
Yes - a dependentlib.list file is there (and is read as thunderbird starts).

[mysys]: cat dependentlibs.list 
libldap60.so
libprldap60.so
libldif60.so
libnspr4.so
libplc4.so
libplds4.so
libmozalloc.so
libmozsqlite3.so
libnssutil3.so
libsoftokn3.so
libnss3.so
libssl3.so
libsmime3.so
libxul.so


As for LDAP, I don't use LDAP, and nsswitch.conf has:

passwd:         files
shadow:         files
group:          files

(in fact everything is "files" except hosts, which is "files dns").

This is a home Linux workstation - no network-based auth at all.

But libprdldap60 is mentioned in the dependentlibs.list.
Comment 8 Mike Hommey [:glandium] 2012-06-11 14:26:43 PDT
(In reply to Gordon Lack from comment #7)
> But libprdldap60 is mentioned in the dependentlibs.list.

Except, looking at the strace, it depends on plds, which isn't loaded yet. If you move libprldap60.so after libplds4.so in dependentlibs.list, I bet this works.
Comment 9 Gordon Lack 2012-06-11 17:24:14 PDT
You are correct.

Editing dependentlibs.list to put libprldap60.so immediately after libplds4.so in the list enables thunderbird to start without setting LD_LIBRARY_PATH.
Comment 10 Mike Hommey [:glandium] 2012-06-12 04:54:25 PDT
Created attachment 632208 [details] [diff] [review]
Force loading NSPR before prldap60

This is a simple hackish fix that doesn't involve anything other than comm code. The idea is that we add nspr before prldap60, which depends on it. The result is that nspr is going to appear twice in dependentlibs.list, but it's not a problem at all because the second time it will see it, it will only reuse the same library it already loaded.

I'm curious to know why this bug wasn't caught before.
Comment 11 Mark Banner (:standard8) 2012-06-12 05:46:00 PDT
(In reply to Mike Hommey [:glandium] from comment #10)
> This is a simple hackish fix that doesn't involve anything other than comm
> code. The idea is that we add nspr before prldap60, which depends on it. The
> result is that nspr is going to appear twice in dependentlibs.list, but it's
> not a problem at all because the second time it will see it, it will only
> reuse the same library it already loaded.
> 
> I'm curious to know why this bug wasn't caught before.

I'm not sure why we didn't catch it, but we have discovered that for systems which have nspr installed for the system, we're loading the system version of libplds4.so, and then the Thunderbird version. Which obviously isn't good. Its possible that our tester base all have system nspr installed, or it could be that it was just missed somewhere.
Comment 12 Mark Banner (:standard8) 2012-06-12 05:47:56 PDT
Comment on attachment 632208 [details] [diff] [review]
Force loading NSPR before prldap60

I've checked this locally hacking the dependentlibs.list and using ldd and it works fine.

Thanks for the patch. I'll get it landed in a bit.
Comment 13 Mark Banner (:standard8) 2012-06-12 06:00:21 PDT
Moving to mailnews core as this affects SeaMonkey as well.
Comment 14 Mark Banner (:standard8) 2012-06-12 06:10:56 PDT
Checked in:

https://hg.mozilla.org/comm-central/rev/e0ce067587b7
Comment 15 Mark Banner (:standard8) 2012-06-12 06:12:31 PDT
Comment on attachment 632208 [details] [diff] [review]
Force loading NSPR before prldap60

[Triage Comment]
We need to take this onto branches for 13.0.1, so a=me. Ratty has also said that SeaMonkey want this fix.
Comment 17 Mark Banner (:standard8) 2012-06-12 07:27:37 PDT
Fixed for 13.0.1:

https://hg.mozilla.org/releases/comm-release/rev/fd59f0076d0c
Comment 18 Gordon Lack 2012-06-12 16:39:34 PDT
>> I'm curious to know why this bug wasn't caught before.

Some suggestions:

a) I'm running a 32-bit thunderbird on a 64-bit Linux.
b) I don't install to a system location.
c) The install location does not feature in the /etc/ld.so.conf locations, and neither in any LD_LIBRARY_PATH (as I don't ever set the other in any login environment I have, as it should be a per-application setting at best).
Comment 19 Philip Chee 2012-06-13 11:29:14 PDT
*** Bug 745518 has been marked as a duplicate of this bug. ***
Comment 20 Philip Chee 2012-06-13 11:29:59 PDT
*** Bug 764461 has been marked as a duplicate of this bug. ***
Comment 21 WD 2012-08-31 11:08:39 PDT
Just ran into this exact issue on a very stripped Linux VM.  (CERT's DebianFuzz).   I had to do two things to get Thunderbird 15 to run:
1) export LD_LIBRARY_PATH <path_to_thunderbird>
2) sudo apt-get install libdbus-glib-1-2

I would have liked Thunderbird to be able to run without needing to do either of the above.   Or is it that that's just the way packaging software for Linux is, and you have to assume that it has some minimum set of required libraries?

Note You need to log in before you can comment on or make changes to this bug.