223744 - [gtk2] Flash plugin crashes mozilla following bug 211587

Status: RESOLVED FIXED

(Core Graveyard :: Plug-ins, defect)

Description

[Kenneth Herron]

The flash plugin for linux (version 6.0r69 or 6.0r79) contains a dependency on a
gtk1-only symbol named "gtk_type_check_object_cast". The plugin works properly
in a gtk2 build of mozilla despite this symbol being missing.

However, the patch for bug 211587 changed plugins to be loaded with the dlopen()
flag RTLD_NOW, which forces dlopen() to resolve all symbols. In the process it
discovers that this symbol is missing, and returns an error. Mozilla handles
this by opening a couple additional libraries, then dlopen()ing the plugin again
using RTLD_LAZY. This might normally be expected to work.

However, my system's dl support (glibc 2.2.5 on Mandrake linux 9.0) seems to be
buggy here. When the initial call to dlopen() the plugin fails, the dl subsystem
apparently doesn't unmap the plugin or do the internal bookkeeping to record
that it wasn't opened successfully. A second call to dlopen() will succeed even
with RTLD_NOW set, but the plugin's symbols aren't fully resolved, and
attempting to access the plugin leads to a crash.

How to reproduce:

1) Compile mozilla with gtk2 support.
2) Install the flash plugin.
3) Attempt to run mozilla.

Launching "mozilla -g" with flash version 6.0r79 (the current version) yields
the following:

> dist/bin/mozilla -g
dist/bin/run-mozilla.sh -g dist/bin/mozilla-bin
MOZILLA_FIVE_HOME=/home/kherron/cvs/mozilla-gtk2/dist/bin
 
LD_LIBRARY_PATH=/home/kherron/cvs/mozilla-gtk2/dist/bin:/home/kherron/cvs/mozilla-gtk2/dist/bin/plugins:/usr/local/lib/mre/mre-1.6a
DISPLAY=:0.0
FONTCONFIG_PATH=/etc/fonts:/home/kherron/cvs/mozilla-gtk2/dist/bin/res/Xft
DYLD_LIBRARY_PATH=/home/kherron/cvs/mozilla-gtk2/dist/bin:/usr/local/lib/mre/mre-1.6a
    
LIBRARY_PATH=/home/kherron/cvs/mozilla-gtk2/dist/bin:/home/kherron/cvs/mozilla-gtk2/dist/bin/components:/usr/local/lib/mre/mre-1.6a
      
SHLIB_PATH=/home/kherron/cvs/mozilla-gtk2/dist/bin:/usr/local/lib/mre/mre-1.6a
         
LIBPATH=/home/kherron/cvs/mozilla-gtk2/dist/bin:/usr/local/lib/mre/mre-1.6a
       ADDON_PATH=/home/kherron/cvs/mozilla-gtk2/dist/bin
      MOZ_PROGRAM=dist/bin/mozilla-bin
      MOZ_TOOLKIT=
        moz_debug=1
     moz_debugger=
which: no ddd in
(/home/kherron/bin:/usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin:/usr/sbin:/sbin)
/usr/bin/gdb dist/bin/mozilla-bin -x /tmp/mozargs7034
GNU gdb 5.2.1-2mdk (Mandrake Linux)
Copyright 2002 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i586-mandrake-linux-gnu"...
(gdb) handle SIG32 nostop noprint pass
Signal        Stop      Print   Pass to program Description
SIG32         No        No      Yes             Real-time event 32
(gdb) r
Starting program: /home/kherron/cvs/mozilla-gtk2/xpfe/bootstrap/mozilla-bin
Type Manifest File: /home/kherron/cvs/mozilla-gtk2/dist/bin/components/xpti.dat
+++ JavaScript debugging hooks installed.
nsNativeComponentLoader: autoregistering begins.
nsNativeComponentLoader: autoregistering succeeded
nNCL: registering deferred (0)
WARNING: dependent window created without a parent, file
/home/kherron/cvs/mozilla/xpfe/bootstrap/nsWindowCreator.cpp, line 128
GFX: dpi=101 t2p=0.0714286 p2t=14 depth=16
WEBSHELL+ = 1
LoadPlugin: failed to initialize shared library libXt.so
[/home/kherron/.mozilla/plugins/libflashplayer.so: undefined symbol:
gtk_type_check_object_cast]
LoadPlugin: failed to initialize shared library libXext.so
[/home/kherron/.mozilla/plugins/libflashplayer.so: undefined symbol:
gtk_type_check_object_cast]
LoadPlugin() /home/kherron/.mozilla/plugins/libflashplayer.so returned 81b7060
GetMIMEDescription() returned "application/x-shockwave-flash:swf:Shockwave
Flash;application/futuresplash:spl:FutureSplash Player"

Program received signal SIGSEGV, Segmentation fault.
0x00048d3e in ?? ()
Current language:  auto; currently c
(gdb) bt 12
#0  0x00048d3e in ?? ()
#1  0x4255df22 in ns4xPlugin::GetValue(nsPluginVariable, void*) (this=0x81b7c08,
    variable=nsPluginVariable_NameString, value=0xbfffdce4)
    at /home/kherron/cvs/mozilla/modules/plugin/base/src/ns4xPlugin.cpp:828
#2  0x425917f8 in nsPluginFile::GetPluginInfo(nsPluginInfo&) (this=0xbfffde90,
    info=@0xbfffdea0)
    at /home/kherron/cvs/mozilla/modules/plugin/base/src/nsPluginsDirUnix.cpp:392
#3  0x42573135 in nsPluginHostImpl::ScanPluginsDirectory(nsIFile*,
nsIComponentManager*, int, int*, int) (this=0x81b52b8, pluginsDir=0x0,
compManager=0x809aa10, aCreatePluginList=1,
    aPluginsChanged=0xbfffe008, checkForUnwantedPlugins=0)
    at /home/kherron/cvs/mozilla/modules/plugin/base/src/nsPluginHostImpl.cpp:4924
#4  0x42573419 in
nsPluginHostImpl::ScanPluginsDirectoryList(nsISimpleEnumerator*,
nsIComponentManager*, int, int*, int) (this=0x81b52b8, dirEnum=0x81b5ab0,
compManager=0x809aa10,
    aCreatePluginList=1, aPluginsChanged=0xbfffe0a8, checkForUnwantedPlugins=0)
    at /home/kherron/cvs/mozilla/modules/plugin/base/src/nsPluginHostImpl.cpp:5016
#5  0x42573801 in nsPluginHostImpl::FindPlugins(int, int*) (this=0x81b52b8,
    aCreatePluginList=1, aPluginsChanged=0xbfffe148)
    at /home/kherron/cvs/mozilla/modules/plugin/base/src/nsPluginHostImpl.cpp:5098
#6  0x425734b2 in nsPluginHostImpl::LoadPlugins() (this=0xbfffdce4)
    at /home/kherron/cvs/mozilla/modules/plugin/base/src/nsPluginHostImpl.cpp:5036
#7  0x425722ab in nsPluginHostImpl::GetPluginFactory(char const*, nsIPlugin**) (
    this=0x81b52b8, aMimeType=0x4252b5a9 "application/x-java-vm",
aPlugin=0xbfffe298)
    at /home/kherron/cvs/mozilla/modules/plugin/base/src/nsPluginHostImpl.cpp:4486
#8  0x42511174 in nsJVMManager::StartupJVM() (this=0x81b4b58)
    at /home/kherron/cvs/mozilla/modules/oji/src/nsJVMManager.cpp:655
#9  0x42511b51 in nsJVMManager::MaybeStartupLiveConnect() (this=0x81b4b58)
    at /home/kherron/cvs/mozilla/modules/oji/src/nsJVMManager.cpp:910
#10 0x42516aef in nsJVMManager::StartupLiveConnect(JSRuntime*, int&)
(this=0xbfffdce4,
    runtime=0x80e9468, outStarted=@0x426b5c60)
    at /home/kherron/cvs/mozilla/modules/oji/src/nsJVMManager.h:143
#11 0x4242741d in nsJSEnvironment::Init() ()
    at /home/kherron/cvs/mozilla/dom/src/base/nsJSEnvironment.cpp:1958
(More stack frames follow...)

Comment 1

[Kenneth Herron]

Attachment: Sample program to illustrate the dlopen problem

This simple program illustrates the basic problem underlying problem with the
dl library. Run it as follows:

$ gcc -o dltest dltest.c -ldl
$ ./dltest .mozilla/plugins/libflashplayer.so
.mozilla/plugins/libflashplayer.so: undefined symbol: gtk_window_get_type
returned 0x0x8049690

If you run the program through strace, you'll see the dl library mmaps
libflashplayer.so on the first call to dlopen() and doesn't munmap it. The
second dlopen() call doesn't reopen or remap the file; its apparently returning
a handle that was set up during the first call despite the first call failing
to resolve all symbols.

Clearly, this is a bug in the dl library rather than mozilla, but it may be
widespread enough to warrant a workaround.

Comment 2

[Andreas Lange]

Always something is buggy in some combination... :-(

I suppose the best solution is to go back to have RTLD_LAZY all over and ifdef
RTLD_NOW to the only confirmed case that needs it, Solaris.

Comment 3

[Boris Zbarsky [:bzbarsky]]

Well, hp-ux needs it too, no?

Comment 4

[Andreas Lange]

Both Solaris and HP-UX needs libXm, Solaris definately needs RTLD_NOW and it's
likely that HP-UX does as well (This concerns the Acrobat plug in).

If we changed back to RTLD_LAZY, would the reporter's plugin work again, or
would it cause intermittent crashes when it in some code path actually calls the
missing symbol? Perhaps you could test that, Kenneth?

I suppose the best thing to avoid any plug in related crash would be to only
have RTLD_NOW in all places. That would make us not load a few plug ins, otoh,
we would never have a Mozilla crash caused by a badly linked plug in.

Acrobat 4 is one example. That one references a CreateQueue() that isn't defined
anywhere, and this causes _some_ pdf's to make mozilla crash and burn. Otoh,
it's quite simple to work around (a "void CreateQueue() {}" compiled shared
seems to suffice, or upgrade to Acro5), and I don't know if there is any valid
case where missing symbols are ok?

Comment 5

[Kenneth Herron]

I've been building mozilla on linux with gtk2 and using flash revision 6.0r69
for quite a while with no particular problems. The sample URL has a flash object
on the home page, and I've been visiting the site almost daily for months with
no trouble.

I first encounted the problem reported here on oct. 24th after updating my
source tree from cvs and recompiling. The previous source update had been 10/13.
After encountering the crash, I downloaded a fresh copy of the flash plugin,
which upgraded me from 6.0r69 to 6.0r79. The only effect of this upgrade was to
change when the crash occurred. With 6.0r69, the crash was occurring when
viewing a page containing a flash object. With 6.0r79, the crash occurs during
mozilla startup.

Further, if I change the first PR_LoadLibraryWithFlags() call
(http://lxr.mozilla.org/seamonkey/source/modules/plugin/base/src/nsPluginsDirUnix.cpp#322)
to use a flag value of 0, implying that it should load the library lazy, then
mozilla starts and the plugin functions normally.

Comment 6

[Andreas Lange]

Attachment: Some ifdefs to resolve this

Here is a patch that should resolve this while still preserving PR_LD_NOW in
some cases. The code gets a bit cluttered with all the ifdefs and comments, but
I think the comments are necessary for someone to understand all of the
conditions just by looking at the code.

Comments?

Comment 7

[Christopher Blizzard (:blizzard)]

This is the sound of my head exploding.

Comment 8

[Andreas Lange]

Comment on attachment 134500 [details] [diff] [review]
Some ifdefs to resolve this

Oh. Your queue grew that much? Sorry.

Comment 9

[Christopher Blizzard (:blizzard)]

No, the gtk-mixed symbol problem is a dumb one to have.  I guess someone started
including gtk 1.2 symbols in their Xt plugin and we're shipping gtk2.

God, I love unix so much.

Comment 10

[Boris Zbarsky [:bzbarsky]]

Comment on attachment 134500 [details] [diff] [review]
Some ifdefs to resolve this

sr=bzbarsky; I see no better solutions for now...

blizzard, any other way we could possibly fix this?

Comment 11

[Christopher Blizzard (:blizzard)]

Doesn't flash have a setting for disabling the gtk code?  I seem to remember
that they had something like this.

Comment 12

[Andreas Lange]

Should this get in before 1.6b?

No response to Blizzard's question from anyone at Macromedia yet...

Comment 13

[Wily Yuen]

I also encountered a page with the crash problem, when Flash 6.0 r79 plugin was
installed on a GTK2 build of Mozilla Firebird 0.7 on Linux.

http://www.hkepc.com/phpBB-HK/

(Note: It is a Chinese page so you will need Chinese fonts to handle it properly)

When I browsed the sub-forum and tried to go back to the main forum page,
Firebird crashed. This problem does not exist on GTK1.2 build.

When I removed the Flash plugin, I can go back and forth from sub-forum page to
main forum page on that site with no problem.

Comment 14

[Andreas Lange]

Comment on attachment 134500 [details] [diff] [review]
Some ifdefs to resolve this

Summary: Bug 211587 changed how ldopen() loaded plug-ins for Solaris. Bug
164021 (regarding HP-UX) then extended this change to all other platforms,
since the semantics seemed sane. 

It turned out that at least dlopen() on Linux can't handle this as it should -
the latest flash plugin crashes GTK2 builds (Mozilla/Firebird).

This patch ifdefs the described changes to the platforms that have a confirmed
need, and brings back the previous lax behaviour for all other platforms.
(sr=bz)

Comment 15

[Asa Dotzler [:asa]]

Comment on attachment 134500 [details] [diff] [review]
Some ifdefs to resolve this

a=asa (on behalf of drivers) for checkin to Mozilla 1.6 Beta.

Comment 16

[Andreas Lange]

Anyone with cvs access, please feel free to check in.

Comment 17

[Boris Zbarsky [:bzbarsky]]

Doesn't the patch still need r= from blizzard?

Comment 18

[Boris Zbarsky [:bzbarsky]]

Checking in nsPluginsDirUnix.cpp;
/cvsroot/mozilla/modules/plugin/base/src/nsPluginsDirUnix.cpp,v  <-- 
nsPluginsDirUnix.cpp
new revision: 1.34; previous revision: 1.33
done

Fixed for 1.6b.