12.27 KB, application/octet-stream
3.00 KB, text/plain
7.84 KB, text/plain
3.77 KB, patch
Robert Relyea: review+
|Details | Diff | Splinter Review|
3.40 KB, patch
Robert Relyea: review+
|Details | Diff | Splinter Review|
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1b99) Gecko/20090605 Firefox/3.5b99 Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1b99) Gecko/20090605 Firefox/3.5b99 Firefox always crashes under Fedora 11 using 10.0.22.87 flash player. Fedora 11 and flash player are necessary to cause the crash. Reproducible: Always Steps to Reproduce: 1. Under Fedora 11 2. flash player under ~/.mozilla/plugins 3. Browser webpage a few seconds Actual Results: Crash Expected Results: No Crash
Can you give a stacktrace of the crash? https://developer.mozilla.org/En/How_to_get_a_stacktrace_for_a_bug_report
(In reply to comment #2) > Can you please create a new profile, and try to reproduce there. Are you using > an official build of firefox, or a linux distro release? As I've mentioned this happens with a new profile. I'm using Firefox trunk version. I'll try to post stacktrace later.
I mean,, are you using a version from the official mozilla ftp site, or a something through the package manager? Can you give the full version info, something like Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2a1pre) Gecko/20090610 Minefield/3.6a1pre (.NET CLR 3.5.30729) ID:20090610042525.
Also, what version of the Flash plugin are you using? Does Flash work in other browsers?
Reproduced a crash on youtube with the latest flash plugin from adobe.com. Waiting for the stack trace now.
Note: this was with version 10, r22.
Sorry, that's 10.0 r22 in about:plugins. To be more specific.
Crashes Firefox 3.0.10 as well.
I'm also using Mozilla's builds here. Not Fedora's.
Michelle, could the Adobe team have a look at this bug too?
I cannot repro on Ubuntu 8.10 with Flash 10.0.22.87-1 installed. Means I cannot run a debug session on it to get more information.
caillon, you might want to have a look at this - seems F11-specific and affects both Fx 3 and Fx 3.5.
So, based on my understanding: * Fedora's firefox binaries running on F11 works * Mozilla's firefox binaries running on F11 crashes Offhand, I have no idea what's up :( Martin, when you get a chance, can you help pinpoint what the issue is? Possibly a glibc mismatch?
Created attachment 382659 [details] Crash Log Since Crash Reporter doesn't work for (a clear regression - earlier bug reported did NOT required GConf2) me, I'm posting crash log/dump here. Again: clean profile, flash-plugin-10.0.22.87, Firefox nightly.
Reproduced. Sure enough, Firefox 3.5b4 distributed with Fedora works with 10.0.22.87 while Firefox 3.0.10 downloaded from Mozilla crashes. This is easily reproducible by just booting the Fedora i686 live CD. Here's the reason: libflashplayer.so wants to load libcurl.so during initialization. For some reason, the dlopen() call fails when Firefox 3.0.10 loads libflashplayer.so, but succeeds when Firefox 3.5b4 does the same thing.
I can confirm and emphasize further: With F11 the flash-plugin 10.0 r22 crashes all my mozillas: Firefox 3.5b4 (official Fedora distribution), my own builds in F11 -gcc 4.4.0, Minefield 3.6a1, and Seamonkey 2.0b1pre. Unfortunately I can't provide crash reports as the crashreporter won't build with gcc 4.4.0.
(In reply to comment #19) > Unfortunately I can't provide crash reports as the crashreporter won't build > with gcc 4.4.0. That's true but you should be able to catch the stack by running your builds inside gdb. Do you have debug builds?
(In reply to comment #18) > Reproduced. Sure enough, Firefox 3.5b4 distributed with Fedora works with > 10.0.22.87 while Firefox 3.0.10 downloaded from Mozilla crashes. This is easily > reproducible by just booting the Fedora i686 live CD. > > Here's the reason: libflashplayer.so wants to load libcurl.so during > initialization. For some reason, the dlopen() call fails when Firefox 3.0.10 > loads libflashplayer.so, but succeeds when Firefox 3.5b4 does the same thing. Strange, I've manually removed libcurl RPM from my Fedora 11 installation and Firefox still crashes. Probably the problem lies somewhere else.
Please also see http://kb2.adobe.com/cps/142/tn_14266.html. From this location you can also download a debug version of Flash. That could help to track down this issue if one of you is familiar with gdb.
(In reply to comment #20) > (In reply to comment #19) > > Unfortunately I can't provide crash reports as the crashreporter won't build > > with gcc 4.4.0. > > That's true but you should be able to catch the stack by running your builds > inside gdb. Do you have debug builds? I'm afraid I did not make debug builds. Unfortunately I am not too familiar with gdb. But give me some time, perhaps I will be successful... BTW Flashplayer ver 9 works fine with these F11 builds. Also from the same source, but built in Windows XP with VS 2005 works fine, also with Flash 10.
Creating a debug build is simple and only needs some small modifications to your .mozconfig. Everything is documented here: https://developer.mozilla.org/En/Simple_Firefox_build Debugging information can be found here: https://developer.mozilla.org/En/Debugging_Mozilla_on_Linux_FAQ
According to http://forums.fedoraforum.org/showthread.php?p=1217086, the problem has to do with a conflict between the /lib/libfreebl3.so that Fedora 11 provides as part of its nss packages and the libfreebl3.so that Firefox uses internally as part of its own nss. Flash 10 uses libcurl which in turn uses libfreebl3. One of the suggested workarounds is to remove Firefox's libfreebl3.so. If you are building your own Firefox, using the --with-system-nss option avoids the conflict as well. I am guessing that the Firefox that comes with Fedora was built with --with-system-nss. There is another crash that happens, at least for me, when Flash actually tries to play a sound, and it attempts to load libasound_module_pcm_pulse.so. That may be the crash people are seeing with the Fedora provided Firefox.
(In reply to comment #25) > There is another crash that happens, at least for me, when Flash actually tries > to play a sound, and it attempts to load libasound_module_pcm_pulse.so. That > may be the crash people are seeing with the Fedora provided Firefox. That one looks like to be the same as bug 496034 which also happens on other distributions and for video too.
Sadly there is no useful information in it. :/
(In reply to comment #28) > Sadly there is no useful information in it. :/ Yes, I noticed that. The good thing, however, is that I now got some debugging experience..:)- It is true that the Fedora distributed Firefox is built with --with-system-nss I will now try that, and see if I also get the sound bug..
This bug is also present in the Linux version of Firefox 3.5 rc2.
This same issue is happening on OpenSuse 11 and OpenSuse 11.1 for us with slightly different results. It's very easy for me to replicate. If I download and use version 3.0.X from Mozilla, Flash 10 is very stable and everything works fine. When I use 3.5 it always crashes at shutdown after playing flash content. This is happening even though both versions reside on the same server and in theory have access to identical libraries. 3.5 also seems to crash when you play flash content in a second window and then close it and return back to the original window instance. If you feel that installing the debug Flash version + running against gdb will produce output....I'm very willing to help. It's easily replicated here.
Created attachment 386423 [details] gdb backtrace of 3.5 under Fedora 11 gdb backtrace with most symbols available
Craig thanks but this backtrace looks like another bug. There is no Flash stack frame listed. You should better file a new Audio/Video bug.
Could there be some differences in the ABI of Fedora's libfreebl3.so? Looks like trunk mozilla-central is using NSS 3.12.4 RC0, and mozilla-1.9.1 is using NSS 3.12.3, while Fedora has NSS_3_12_4_FIPS1_WITH_CKBI_1_75 plus a few changes: http://cvs.fedoraproject.org/viewvc//rpms/nss/F-11/nss.spec?view=markup
Redhat just pushed out Firefox 3.5 final to the yum mirrors. It no longer crashes on Flash or video for me. I imagine this bug will rear its head again when Mozilla does a Firefox update, and Fedora (and SuSE?) users upgrade outside of the package system. It's fairly obvious that there is some library issue. I was wondering if someone would like me to go back to Mozilla's Firefox 3.5 and to try some different things (such as setting LD_LIBRARY_PATH to my local Firefox install)? This has been very frustrating; although it appears that RedHat is at fault, perhaps the Firefox startup script needs to accommodate this situation.
(In reply to comment #18) Mike Melanson <email@example.com> wrote: > Here's the reason: libflashplayer.so wants to load libcurl.so during > initialization. For some reason, the dlopen() call fails when Firefox 3.0.10 > loads libflashplayer.so, but succeeds when Firefox 3.5b4 does the same thing. So, flash player crashes when dlopen fails? That's clearly a flashplayer bug. dlopen can fail at various times for various reasons. Crashing seems like the wrong response. As for dlopen failures with NSS, that's a known bug, already fixed on trunk.
Actually, this could also be if some distro distributes an incomplete set of NSS .so files. NSS adds new .so files from time to time, and every time we do, there's always some distro that ships an incomplete set, causing problems until they ship the complete set.
> > So, flash player crashes when dlopen fails? That's clearly a flashplayer bug. > dlopen can fail at various times for various reasons. Crashing seems like the > wrong response. > > As for dlopen failures with NSS, that's a known bug, already fixed on trunk. > > *** This bug has been marked as a duplicate of bug 494107 *** Fixed in trunk? I'm not noticing that - today's trunk still crashes when visiting a web page which contains flash. > To anyone who still wants to use Mozilla's Firefox in Fedora: If you have installed Mozilla's Firefox under root user then just delete all libns*.so files in the root Firefox directory. If you have installed it under your user account (and you can update it), then use this script to avoid crashes (call it, e.g. fx and put it in your $PATH): ---------------------------------- #! /bin/bash cd "/firefox/installation/dir" || exit 1 mkdir -p save mv libns* save &> /dev/null exec ./firefox & sleep 60 mv save/libns* . ----------------------------------
Fedora Project has apparently solved the problem for Fedora-fc11. The problem is native on the DVD install set. I removed /opt/Adobe/Reader9/Reader/intellinux/lib/libflashsupport.so, /usr/lib/mozilla, /usr/lib/flash-plugin and all firefox directories under /usr as a start to set the stage. I used yum to erase 'flash-plugin' and 'firefox' and root to remove the rest. I then used yum to install firefox, which updated three other packages, and used yum to install 'flash-plugin'. Fedora-fc11 yum sites installed firefox-3.5.1 and Adobeflash-10.0-r22. At the end of this process every site which crashed my browser operated correctly and everything I had removed had been replaced, except for the usual Java, etc., plugins.
(In reply to comment #40) >> As for dlopen failures with NSS, that's a known bug, already fixed on trunk. > > Fixed in trunk? I'm not noticing that - today's trunk still crashes when > visiting a web page which contains flash. Fixed on the NSS trunk. NSS is its own separate project with its own repository (which is Mozilla's CVS repository). Any other repositories that contain copies of NSS sources, such as any of Mozilla's Hg repositories, or any of Red Hat's repositories, are downstream repositories, and they contain snapshots of NSS (matching certain CVS tags in the NSS repository). The NSS team does not control when the maintainers of these downstream repositories update their copies. However, we do request that they only take snapshots of "official" NSS release tags, unless they're prepared to fully support NSS, because the NSS team cannot support versions other than its own releases. The NSS team produces new release tags rather frequently, generally when any downstream project needs an update. > To anyone who still wants to use Mozilla's Firefox in Fedora: > > If you have installed Mozilla's Firefox under root user then just delete all > libns*.so files in the root Firefox directory. Um, no. It's true that you can just remove the NSS shared libraries from Mozilla's directory to use the OS's "native" copy, but to do that, you really need to delete ALL the NSS files from the Firefox directory, and removing libns*.so doesn't do that. I suggest the following variant of your shell script: cd "/firefox/installation/dir" || exit 1 mkdir -p save touch libfoopy3.so mv lib*3.so save &> /dev/null || true exec ./firefox & This will move all of NSS's files and also libsqlite3.so, which is OK if your system has a copy in /usr/lib, but not otherwise. If your firefox doesn't start, then move libsqlite3.so back from save to .
https://bugzilla.redhat.com/show_bug.cgi?id=505365#c21 "there was a change between Fedora 10 and Fedora 11 that might be related. In F11, glibc has started to depend on libfreebl3.so, in order to centralize all operating system use of cryptography. But at the same time, there was a desire to keep the list of dependencies as small as possible. Usually, libfreebl3.so depends on libraries from nspr.rpm Bob Relyea had invented mechanisms to use a dynamic runtime decision, whether libfreebl3.so should use some internal minimal nspr replacements, or whether it should link against the real nspr. The idea was, if libfreebl3.so gets referenced by glibc, it shall use its internal nspr replacements (stubs), and in other scenarios it should load the real nspr libs."
It looks like symbols in nsslowhash.c are only defined with FREEBL_NO_DEPEND, which Fedora 11 defines but Mozilla does not. http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/security/nss/lib/freebl/manifest.mn&rev=1.57&mark=73-75#73 Anyone seeing "version `NSSRAWHASH_3.12.3' not found" messages? When are the symbols in nsslowhash used? Should they be defined even without FREEBL_NO_DEPEND? What is the equivalent API without FREEBL_NO_DEPEND?
They are private to softoken.
(In reply to comment #45) > They are private to softoken. I don't think I understand. They are exported with FREEBL_NO_DEPEND: http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/security/nss/lib/freebl/freebl_hash.def&rev=1.2&mark=61#61 Are you saying that something similar exists in softoken but is private without FREEBL_NO_DEPEND? Or are you saying that these symbols should only be used by softoken?
Sorry, my comment 45 was mistaken. I believe these symbols define a private interface between glibc and freebl on Red Hat Linux. I believe it is not intended to be a public interface to be called by just any application software that wishes to do so. I believe Mozilla itself should be making no direct calls to those functions, and hence there should be no unresolved externals naming those functions when linking any Mozilla software. I may be mistaken about that interface being private. My words below presume the interface is indeed private. Since it is intended to be a (Red Hat) Linux system private interface, it makes sense (to me) that it would be provided in Red Hat Linux's system builds, and not in Mozilla's builds which are intended to run on all Linux distros. So, I think it is likely appropriate that those symbols should only be defined in Fedora builds and not in Mozilla builds. I see that the problem you are dealing with is that an attempt to link freebl into a .so is complaining about the missing symbols. I think this is because the mapfile is the wrong mapfile. There are two .def files in lib/freebl, which are freebl.def and freebl_hash.def http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/security/nss/lib/freebl/manifest.mn&rev=1.57&mark=78,80#73 During the make, one of them gets copied into OBJDIR. If you have the wrong one in OBJDIR, and you try to build, you will get this error. Perhaps you changed the setting of FREEBL_NO_DEPEND but then did not do a clean build?
I can reproduce the crash. Fedora 11 system, using flash-plugin installed from Adobe yum repository. I crash using a downloaded nightly firefox 3.5 build from ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-1.9.1/ I also crash using a local debug build of the Firefox 3.5 branch (using internal nss, not system nss). In order to trigger the crash, I simply navigated to www.youtube.com and clicked on the first video.
I guess the problem is caused by having two different freebl libraries used in the process. I suspect the globally installed flash-plugin may use the system-wide installed libfreebl (a): # ldd /usr/lib/flash-plugin/libflashplayer.so |grep -i nss libnss3.so => /lib/libnss3.so (0x00624000) libnssutil3.so => /lib/libnssutil3.so (0x00800000) while the Firefox application uses the libfreebl (b) that was shipped/built together with the application. (a) was built using FREEBL_NO_DEPEND (b) was built without I don't understand what conflict this can produce, but I assume that Bob Relyea can tell us, and maybe he has an idea for a fix, he invented the FREEBL_NO_DEPEND solution. Maybe one library attempts to free memory that the other library created?
My installed flash-plugin.rpm is version 10.0.32.18 I downloaded http://fpdownload.macromedia.com/get/flashplayer/installers/archive/fp10_debug_archive.zip I extracted fp10_debug_archive/10r32_18/flashplayer_10r32_18_linux_debug.tar.gz I moved the installed /usr/lib/flash-plugin/libflashplayer.so away I copied file libflashplayer.so extracted from flashplayer_10r32_18_linux_debug.tar.gz to /usr/lib/flash-plugin/libflashplayer.so I started my debug version of Firefox in gdb. Unfortunately this didn't give me a better stack trace. I still see: #0 0x00000000 in ?? () #1 0xaa0af7d0 in ?? () from /usr/lib/flash-plugin/libflashplayer.so #2 0xaa23c285 in ?? () from /usr/lib/flash-plugin/libflashplayer.so #3 0xaa436d00 in ?? () from /usr/lib/flash-plugin/libflashplayer.so #4 0xaa0b55d6 in ?? () from /usr/lib/flash-plugin/libflashplayer.so #5 0xaa0a7fac in ?? () from /usr/lib/flash-plugin/libflashplayer.so #6 0xaa09a319 in ?? () from /usr/lib/flash-plugin/libflashplayer.so #7 0xaa09d599 in ?? () from /usr/lib/flash-plugin/libflashplayer.so #8 0xb7d42180 in ?? () #9 0xb7d4091c in ?? () #10 0x00000001 in ?? () #11 0x00000009 in ?? () #12 0xb7d84b20 in ?? () #13 0xb7d84bb0 in ?? () #14 0x00000000 in ?? ()
(In reply to comment #44) > > Anyone seeing "version `NSSRAWHASH_3.12.3' not found" messages? Yes. When I start my local debug build (internal nss) in gdb, then I get: $ ./firefox -no-remote -P tryfips -g -d gdb ./run-mozilla.sh -g -d gdb ./firefox-bin -no-remote -P tryfips MOZILLA_FIVE_HOME=. LD_LIBRARY_PATH=.:./plugins:. DISPLAY=:0.0 DYLD_LIBRARY_PATH=.:. LIBRARY_PATH=.:./components:. SHLIB_PATH=.:. LIBPATH=.:. ADDON_PATH=. MOZ_PROGRAM=./firefox-bin MOZ_TOOLKIT= moz_debug=1 moz_debugger=gdb perl: ./libfreebl3.so: version `NSSRAWHASH_3.12.3' not found (required by /lib/libcrypt.so.1) perl: ./libfreebl3.so: version `NSSRAWHASH_3.12.3' not found (required by /lib/libcrypt.so.1) perl: ./libfreebl3.so: version `NSSRAWHASH_3.12.3' not found (required by /lib/libcrypt.so.1) perl: ./libfreebl3.so: version `NSSRAWHASH_3.12.3' not found (required by /lib/libcrypt.so.1) /usr/bin/gdb ./firefox-bin -x /tmp/mozargs.iAvn5G GNU gdb (GDB) Fedora (220.127.116.1190302-37.fc11) ...
inside the directory where I run ./firefox : $ strings ./libfreebl3.so |grep -i rawrash => nothing $ strings /lib/libfreebl3.so |grep -i rawhash NSSRAWHASH_3.12.3 I used "thread apply all backtrace" to inspect the other threads at the same we have crasahed. Threads number 1, 2, 3 and 4 have damaged threads. The other threads have usual correct looking threads. Looks like memory corruption to me.
I tried to step into plugin init code. nsNPAPIPluginInstance::InitializePlugin 1030 NS_TRY_SAFE_CALL_RETURN(error, (*fCallbacks->newp)((char*)mimetype, &fNPP, (PRUint16)mode, count, (char**)names, (char**)values, NULL), fLibrary,this); But I can't. When I use "s" command, the next thing that happens is fork and the crash. I re-attempted to debug, and just before this call I used set follow-fork-mode child Didn't work, gdb didn't give me any chance to try further, but printed it' starting bash, then starting ps, then firefox crashed and the stack was shown.
Even if both freebl libraries where built the same way, you will run into problems if you have both installed. bob
(In reply to comment #54) > Even if both freebl libraries where built the same way, you will run into > problems if you have both installed. But people never saw this bug prior to Fedora 11. My primary development system has been the latest Fedora version since 2006, I did always have system nss installed globally, and I had always developed Mozilla using my local builds with internal nss, and I never saw such a problem. I believe this is a new problem.
Prior to Fedora 11, we didn't have libgcrypt using freebl. Now that libgcrypt used freebl, users which continue to try to use their own copy of freebl will have problems removing the private freebl copy should solve the problem. bob
I confirm that I don't crash when having moved libfreebl3.so away, but what should our message be? People don't use "their own copy" of libfreebl, they have downloaded a software package and attempt to run it. Should Mozilla stop shipping libfreel in their Linux binaries? Should Mozilla begin to produce distribution+release specific Linux downloads? Should Mozilla provide a document "how to tweak your downloaded binary so it won't crash on your system" and force all Linux users to read it?
The takeaway message for most users will be "use Ubuntu, because it works". Firefox is a critical application for Fedora, and Flash is a critical plugin.
I suggest that we add ifeq ($(OS_ARCH),Linux) DEFAULT_GMAKE_FLAGS += FREEBL_NO_DEPEND=1 endif to mozilla/security/manager/Makefile.in.
Which Fedora are you running? Mine is Fedora-11. Downloads for auto update left me with Firefox-3.5.2 and that works perfectly. Every image gets shown, all videos get shown. Exactly WHAT are you talking about?! I don't have to go to Adobe to get flash; it gets downloaded automatically from Fedora. DUH! Use yum.
(In reply to comment #51) > $ ./firefox -no-remote -P tryfips -g -d gdb > ./run-mozilla.sh -g -d gdb ./firefox-bin -no-remote -P tryfips > MOZILLA_FIVE_HOME=. > LD_LIBRARY_PATH=.:./plugins:. > DISPLAY=:0.0 > DYLD_LIBRARY_PATH=.:. > LIBRARY_PATH=.:./components:. > SHLIB_PATH=.:. > LIBPATH=.:. > ADDON_PATH=. > MOZ_PROGRAM=./firefox-bin > MOZ_TOOLKIT= > moz_debug=1 > moz_debugger=gdb > perl: ./libfreebl3.so: version `NSSRAWHASH_3.12.3' not found (required by > /lib/libcrypt.so.1) > perl: ./libfreebl3.so: version `NSSRAWHASH_3.12.3' not found (required by > /lib/libcrypt.so.1) > perl: ./libfreebl3.so: version `NSSRAWHASH_3.12.3' not found (required by > /lib/libcrypt.so.1) > perl: ./libfreebl3.so: version `NSSRAWHASH_3.12.3' not found (required by > /lib/libcrypt.so.1) Attachment 394923 [details] [diff] will help here (allowing arguments to be passed to firefox-bin).
(In reply to comment #56) > removing the private freebl copy should solve the problem. This will only work until a new symbol is added to libfreebl.so and newer applications start to depend on this symbol. (In reply to comment #54) > Even if both freebl libraries where built the same way, you will run into > problems if you have both installed. I don't see why this needs to be the case. Provided newer versions of libraries maintain backward compatibility, it should be sufficient to merely ensure that the library with the newest ABI is loaded first. (If newer versions do not maintain backward compatibility then they'll need to use different soname / versioned symbols / etc.) The problem here is that the ABI depends on how the library was configured (but the soname does not). (In reply to comment #59) > I suggest that we add > > ifeq ($(OS_ARCH),Linux) > DEFAULT_GMAKE_FLAGS += FREEBL_NO_DEPEND=1 > endif > > to mozilla/security/manager/Makefile.in. Always building with FREEBL_NO_DEPEND on Linux looks to me like it would work around the problem here (because the FREEBL_NO_DEPEND=1 version of the ABI is a superset of the FREEBL_NO_DEPEND= ABI.) I can't comment on whether this would impact performance at all (given that some apps will know that they'll need nspr anyway). But why not make the ABI independent of the FREEBL_NO_DEPEND=1 environment variable? If Fedora's libcrypt wants the functionality provided through NSSRAWHASH_3.12.3, then why assume that no other clients want that functionality?
Also crashed "Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:18.104.22.168) Gecko/20090824 Firefox/3.5.3".
As of today, Fedora 11 doesn't use Firefox/3.5b99 and, so far, doesn't issue Firefox/3.5.3 Use yum and get the correct updates for Fedora 11 from the 'update' and 'fedora' sites. As usual, all beta testers can ignore that advice.
Firefox 3.5.2 as provided by Fedora 11 ("Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:22.214.171.124) Gecko/20090803 Fedora/3.5.2-2.fc11 Firefox/3.5.2") doesn't crash when I access pages using Flash (for this one I use the 64bit Flash Plugin as downloaded at http://labs.adobe.com/downloads/flashplayer10.html). However, if I install a separate copy of Firefox, like the "Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:126.96.36.199) Gecko/20090824 Firefox/3.5.3" using the 32bit Flash Plugin from the adobe-linux-i386 repository, it crashes.
Removing libfreebl3.so from the Firefox program directory (of my individual installation, not the one provided by Fedora) seems to solve the problem.
Even trying to remove libfreebl3.so from the Firefox program directory, /usr/lib/firefox-3.5.2/, on Fedora 11 wont work; there is none there. Maybe that's why the Mozilla.com version doesn't work on Fedora 11; Mozilla uses it's private version of libfreebl3.so?! Or is it the other way around? Does Fedora compile a static version into their Firefox? My Fedora 11 installation has /usr/lib/libfreebl3.chk, /usr/lib/libfreebl3.so as part of the regular installation. Why provide a private version? Obviously, if flash works with Fedora 11 firefox-3.5.2 on an independent download from Adobe there is probably some peculiarity of the Mozilla.com version of firefox that is starting to drift from Posix standards. In other words, it ain't compliant any more.
I have a libfreebl3.so in the /lib directory which I guess is the one that Firefox (3.5.2) as provided by Fedora uses. I additionally have Firefox 3.5.3 (the candidate 1 build) installed in /opt/firefox, there was a libfreebl3.so in exactly this directory. After I deleted the libfreebl3.so in /opt/firefox, this instance (3.5.3) could handle Flash well (before it crashed). The Firefox 3.5.2 installation provided by Fedora 11 worked well in regard to Flash all the way. One another thing I haven't mentioned yet: I updated from Fedora 10 to 11 this week. With Fedora 10 there was no problem with Flash at all (also not with my own installation in /opt/firefox), it only started after the update to Fedora 11.
I filed bug 513024 for the workaround suggested in comment 59. I'd appreciate it if someone experiencing this crash (with Fedora 11) could test the build linked there, please. Note that the build is based on trunk, so I suggest either back up your profile or "mkdir ~/tmp/new-profile" and run with "-profile ~/tmp/new-profile -no-remote".
Created attachment 398822 [details] [diff] [review] Proposed patch Always build with FREEBL_NO_DEPEND=1 on current versions of Linux. When I build NSS on Fedora 11 and set LD_LIBRARY_PATH to point to the NSS libraries in my build tree, I can't even use vi to edit a file because of this bug (NSSRAWHASH_3.12.3 not found, required by /lib/libcrypt.so.1). Any NSS-based product that uses its own NSS libraries rather than the system NSS libraries could be affected by this bug when it runs on a Fedora/RHEL system >= Fedora 11.
Wan-Teh, I sympathize with the problem you're having, but Sun's NSS team (or at least I personally) need to understand the full ramifications of this proposed change better before approving this. The number 1 top priority of Sun's NSS team for 3.12.5 is to eliminate, on all platforms, the present perception that 3.12.5 is a regression, as compared to 3.11.x, with respect to ability to load and initialize the necessary NSS shared libraries. I don't want to have to release 3.12.6 and/or 3.12.7 before we eliminate that perception. I fear that unintended consequences may cause the change proposed in this patch to turn into a case of "one step forward, and one step back". Let's discuss this on Thursday morning.
Perhaps ifdef Mozillla && Linux would be the right compromise? bob
(In reply to comment #72) > Perhaps ifdef Mozillla && Linux would be the right compromise? It is not only Mozillla but any app that wants to ship a libfreebl.so and run on Fedora 11, so it would be ifdef WANTS_TO_BE_COMPATIBLE_WITH_FEDORA_11. I'm curious as to what Fedora's libcrypt needs in NSSRAWHASH_3.12.3 that isn't available through FREEBLVectorStr::p_HASH_GetRawHashObject? Or is NSSRAWHASH_3.12.3 merely making something public, that should be public in all builds?
1. In general, you should probably be using system NSS for your application. If you are using mozilla provided by the OS provider, it probably already does, it's just the mozilla builds that are a problem. If you're app believes it really needs to include it's own version of NSS on linux, it could set FREEBL_NO_DEPEND itself. 2. NSSRAWHASH is pretty much strictly for gcrypt. The FREEBLVector interface is private (and a good way to get your app to break on various NSS releases as the NSS team may feel free to change that API if necessary).
(In reply to comment #74) > 1. In general, you should probably be using system NSS for your application. That prevents the app from using new features in NSS. > 2. NSSRAWHASH is pretty much strictly for gcrypt. The FREEBLVector interface > is private (and a good way to get your app to break on various NSS releases as > the NSS team may feel free to change that API if necessary). Thanks. FREEBLVector was the only interface to libfreebl3.so, so I infer from that that libfreebl3.so was only ever intended to be used within NSS. Fedora's libcrypt.so.1 apparently needed a feature that NSS didn't provide and so this feature was provided through new NSSRAWHASH ABI on libfreebl3.so. Why would other clients not want this feature?
Nelson: You can feel free to mark my patch review-. My intention is to call your attention to this potential issue if any of Sun's NSS-based applications bundle NSS libraries (rather than using the system NSS libraries) on Linux and need to support a Fedora or RHEL release >= Fedora 11. Bob: It is very hard to ship a product that uses system NSS and some bleeding edge NSS feature, because if that feature has a bug without a workaround, you are hosed. For a recent example, see bug 515279. (Chromium uses system NSS and the new NSS function CERT_PKIXVerifyCert.)
I'm afraid that, at the moment, no one working on NSS at Sun knows (or remembers, if he ever did know) all the implications of FREEBL_NO_DEPEND. If it's absolutely not a problem for us, then I absolutely do not oppose it. At the moment, I just don't know. We need to bring more of Sun's remaining NSS staff up to speed on this. Maybe we can have a conference call to discuss this on Wednesday or Thursday?
Longer term, we should build libfreebl3.so with FREEBL_NO_DEPEND=1 on Linux. The way we build libfreebl3.so on Linux should not differ between Red Hat and Sun or between Linux distributions. Perhaps I will adapt my patch For now, you can wait until your applications actually encounter this problem to make a decision. The symptom of this bug is a "libfreebl3.so: version `NSSRAWHASH_3.12.3' not found (required by /lib/libcrypt.so.1)" error message. You can't miss it. I will create an alternative patch for the SOFTOKEN_3_13_BRANCH.
> That prevents the app from using new features in NSS. Most Linux distributions stay pretty up-to-date on their version of NSS. It's better to just require the appropriate NSS version. If you use rpm or apt, those requirements are usually picked up automatically by your app because NSS has versioned it's API (the binary has the right symbols to determine dependencies). > I infer from that that libfreebl3.so was only ever intended to be used within NSS. yes. > Fedora's libcrypt.so.1 apparently needed a feature that NSS didn't provide and > so this feature was provided through new NSSRAWHASH ABI on libfreebl3.so. > Why would other clients not want this feature? The feature is available to NSS in general. libcrypt is part of the base glib of the OS. They needed a restrictive (non-crypto)service provided by libfreebl in a FIPS certified way. It's a very specific need, and the bar is pretty high before I would suggest an app use it. The feature is only available in Linux (will not even work on non-linux).
> I will create an alternative patch for the SOFTOKEN_3_13_BRANCH. I would be OK with that. > Longer term, we should build libfreebl3.so with FREEBL_NO_DEPEND=1 > on Linux. The way we build libfreebl3.so on Linux should not differ > between Red Hat and Sun or between Linux distributions. I'm not against that, but remember Sun builds on very old versions of Linux. There may be some issues in the NO_DEPEND mode. I believe that it's all clear, but it really hasn't been tested on something like RHEL 2.1 or earlier. (Actually it's not even turned on on any version of RHEL from Red Hat, and won't be until the FIPS evaluation has been completed. -- on the other hand libgcrypt doesn't try to use it on those platforms either). bob
(In reply to comment #79) > The feature is available to NSS in general. libcrypt is part of the base glib > of the OS. They needed a restrictive (non-crypto)service provided by libfreebl > in a FIPS certified way. It's a very specific need, and the bar is pretty high > before I would suggest an app use it. The feature is only available in Linux > (will not even work on non-linux). Thanks for the explanation. My understanding of the issues here is not sufficient to grasp why the bar is high or why other apps wouldn't need it in a FIPS certified way. However, may I make a suggestion for consideration (by people who understand the issues better than I): Currently FREEBL_NO_DEPEND looks like it controls two different features. 1) Enables the new NSSLOWHASH functionality. 2) Enables on-demand loading of nspr4 and nssutil3. Although these features are currently coupled, it doesn't look like they need to be. IIUC only the second feature needs to have the (possibly-recent-)Linux-dependency. If the FREEBL_NO_DEPEND environment variable controlled only the second feature then the ABI could be consistent wrt configuration.
Created attachment 399664 [details] [diff] [review] Proposed patch (for SOFTOKEN_3_13_BRANCH) v2 Set FREEBL_NO_DEPEND = 1 for Linux 2.6 or later in mozilla/security/nss/lib/freebl/config.mk on the SOFTOKEN_3_13_BRANCH. This patch should be less controversial.
Comment on attachment 399664 [details] [diff] [review] Proposed patch (for SOFTOKEN_3_13_BRANCH) v2 Neither the original patch nor this patch works, because mozilla/security/nss/lib/freebl/manifest.mn tests FREEBL_NO_DEPEND. With the current makefiles, FREEBL_NO_DEPEND must be set in the environment or on the gmake command line.
A patch in coreconf/Linux2.6.mk should work, though..... bob
No. manifest.mn is the very first makefile included by mozilla/security/nss/lib/freebl/Makefile: http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/security/nss/lib/freebl/Makefile&rev=1.108&mark=45,51#41 So any variable tested by manifest.mn must come from the environment or the gmake command line.
That's why there are not supposed to ever be any ifdefs or other conditionals in manifest.mn
FWIW, I have worked around this bug by using nspluginwrapper. My .mozilla/plugins/ directory contains the following (lines folded here): lrwxrwxrwx 1 root root 45 2009-10-02 05:39 npwrapper.so -> /usr/lib/mozilla/plugins- wrapped/npwrapper.so* lrwxrwxrwx 1 root root 66 2009-10-02 05:38 nswrapper_32_32.libflashplayer.so -> /usr /lib/mozilla/plugins-wrapped/nswrapper_32_32.libflashplayer.so On an x86_64 computer, I do not have the problem with flash, and this was because I've been using nspluginwrapper all along.
I've provided some updates in the Fedora bug at https://bugzilla.redhat.com/show_bug.cgi?id=505365 I've created a wiki page describing the understood issue around the Mozilla and NSS library conflict https://fedoraproject.org/wiki/Mozilla_NSS_Conflict and linked it from the "common bugs" pages of fedora 11/12. It contains a workaround for end users (delete libraries from downloaded Linux builds.) The bugfix (as proposed in comment 69) has been applied to Firefox trunk and Firefox 3.6.pre already. We'll propose in bug 513024 that the fix gets applied to Firefox 3.5.x stable branch, too. According to my testing it works. There remain crashes using 64 bit versions of the flash plugin with 64 bit Mozilla. Some flash pages (e.g. http://www.youtube.com/falltvpreview ) always crash on 64 bit Firefox, whether the Firefox binary was produced by Fedora or by Mozilla. And the page works fine using 32 bit flash. Therefore this is suspected to be a separate bug. Some people have reported they crash using 32 bit fedora firefox and 32 bit flash, but I've never been able to reproduce this. Let's consider this a separate unconfirmed bug. We'll collect information about this in Fedora bug https://bugzilla.redhat.com/show_bug.cgi?id=533959 This bug 497251 says "Firefox always crashes using flash. I can confirm "always crashes" when having the mentioned library conflict. I therefore propose to close this bug as "fixed" and "fixed1.9.2", and once we commit bug 513024 to 1.9.1, declare it as fixed1.9.1, too. I propose to open a separate bug report for people crashing on "some" flash content using "64 bit flash", after they have ensured they don't suffer from the library conflict issue described at https://fedoraproject.org/wiki/Mozilla_NSS_Conflict
IMO bug 513024 is only a workaround. The bug is that the ABI of libfreebl3.so depends on the environment variable FREEBL_NO_DEPEND at build time, and this bug should be kept open to track that until it is fixed.
Karl, I'd agree with you, the better plan is to change NSS, so it will always build with FREEBL_NO_DEPEND on Linux. However, it's a common practice in NSS to drive the build using environment variables. If you look at the makefile you have modified, you'll find several other variables that are set to build NSS in the desired way, for example: DEFAULT_GMAKE_FLAGS += MOZILLA_CLIENT=1 DEFAULT_GMAKE_FLAGS += NO_MDUPDATE=1 ... etc.
I don't mind keeping this bug, but I believe this is fixed. I've filed bug 527557 to suggest changing NSS default Linux FREEBL_NO_DEPEND compilation mode. If you think the makefile change is a undesirable workaround, we could: - close this bug - make bug 513024 depend on bug 527557 - undo bug 513024 (on trunk) once Firefox picks up a fixed NSS (at some later time)
(In reply to comment #90) > If you look at the makefile you have modified, you'll find several other > variables that are set to build NSS in the desired way, for example: I haven't checked all the variables there but > DEFAULT_GMAKE_FLAGS += MOZILLA_CLIENT=1 AFAICS this doesn't change the ABI of any library. > DEFAULT_GMAKE_FLAGS += NO_MDUPDATE=1 This doesn't seem to be used in Mozilla's NSS version: http://mxr.mozilla.org/mozilla-central/search?string=NO_MDUPDATE&filter=[Nn]O_MDUPDATE
(In reply to comment #91) > I've filed bug 527557 to suggest changing NSS default Linux FREEBL_NO_DEPEND > compilation mode. I've commented there. I'm OK with closing this, if people think that appropriate, provided there is an open bug report that tracks the problem.
Created attachment 411403 [details] [diff] [review] Patch v3 Proposed patch v3. Ideally the patch shouldn't be limited to the future softoken branch, but applied to the current stable version of NSS. I understand that directory freebl must not be modified, therefore I propose to add this fix to the directory one level above. This patch assumes that export/unexport commands are available in all "make" software we're using. The patch will define FREEBL_NO_DEPEND on Linux by default. (Maybe it's unnecessary, but I added a mechanism to request the current default mode using FREEBL_NO_DEPEND=0 ) FWIW, reporting another scenario where this problem bites people: On Fedora 11, when testing a local build (without freebl_no_depend), it's even impossible to run perl or curl.)
Created attachment 411406 [details] [diff] [review] Patch v4 removed invalid "then" from makefile patch
Comment on attachment 411406 [details] [diff] [review] Patch v4 Kai, thank you for the patch. Your patch helps when we do "make" in the mozilla/security/nss or mozilla/security/nss/lib directory. But it's possible to do "make" in mozilla/security/nss/lib/freebl. In fact, I do that often when I work on freebl. It's fine to fix this bug only on the SOFTOKEN_3_13_BRANCH.
The ideal place to make this change is in coreconf/Linux.mk. It's out of the FIPS boundary. It doesn't need an ifdef LINUX. I would be tempted to make it conditional on ifndef FREEBL_DEPEND Rather than interpreting FREEBL_NO_DEPEND value, but I'm ok with kai's formulation.
Flash Player 10.1 beta is available for Linux (x86_32) now: http://labs.adobe.com/downloads/flashplayer10.html This Player should no longer crash when encountering the conflict described in this bug. Note that it won't completely load or show any content, but that's a lesser evil than crashing outright.
I had almost the exact same symptoms suddenly appear a few days ago. turns out I had conflicting flash support installed. Problem appears after yum updating: mplayer-1.0-0.111.20090923svn.fc11.i586 mencoder-1.0-0.111.20090923svn.fc11.i586 I guess firefox was handling the conflict until the update fix is to remove libflashsupport pkg [sonik@alienproject ~]$ rpm -qa | grep -i flash libflashsupport-000-0.5.svn20070904.i386 flash-plugin-10.0.32.18-release.i386 [sonik@alienproject ~]$ rpm -ql libflashsupport-000-0.5.svn20070904.i386 /usr/lib/libflashsupport.so [sonik@alienproject ~]$ rpm -ql flash-plugin-10.0.32.18-release.i386 /usr/lib/flash-plugin /usr/lib/flash-plugin/LICENSE /usr/lib/flash-plugin/README /usr/lib/flash-plugin/homecleanup /usr/lib/flash-plugin/libflashplayer.so /usr/lib/flash-plugin/setup /usr/share/doc/flash-plugin-10.0.32.18 /usr/share/doc/flash-plugin-10.0.32.18/readme.txt [sonik@alienproject ~]$ sudo yum erase libflashsupport-000-0.5.svn20070904.i386 [sudo] password for sonik: Loaded plugins: dellsysidplugin2, refresh-packagekit Setting up Remove Process Resolving Dependencies --> Running transaction check ---> Package libflashsupport.i386 0:000-0.5.svn20070904 set to be erased --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Removing: libflashsupport i386 000-0.5.svn20070904 installed 11 k Transaction Summary ================================================================================ Remove 1 Package(s) Reinstall 0 Package(s) Downgrade 0 Package(s) Is this ok [y/N]: y Downloading Packages: Running rpm_check_debug Running Transaction Test Finished Transaction Test Transaction Test Succeeded Running Transaction Erasing : libflashsupport-000-0.5.svn20070904.i386 1/1 Removed: libflashsupport.i386 0:000-0.5.svn20070904 Complete!
Bob, we should target this bug for the FIPS revalidation (always build freebl with FREEBL_NO_DEPEND=1 on Linux).
If you can supply a patch, I think it's appropriate for inclusion. (It will probably involve moving some changes to manifest.mn into Makefile for config.mk bob
Oh, and any patch that you add should have an additional environment variable to optionally turn off FREEBL_NO_DEPEND if it's the default. bob
Created attachment 442573 [details] [diff] [review] Patch v5 (checked in) Bob, please review this patch. Re: comment 104 You can turn off FREEBL_NO_DEPEND on the make command line: make FREEBL_NO_DEPEND= I'd like to avoid an additional environment variable to turn off FREEBL_NO_DEPEND. Is this acceptable?
I'd prefer the environment variable.
Comment on attachment 442573 [details] [diff] [review] Patch v5 (checked in) r+ if the ifeq line also includes && FREEBL_FORCE_DEPEND bob
(In reply to comment #107) > (From update of attachment 442573 [details] [diff] [review]) > r+ if the > > ifeq line also includes > && FREEBL_FORCE_DEPEND Bob, I understand we want the default to be "no depend". I guess you intended to request something like && (FREEBL_FORCE_DEPEND != 0) That is, if nobody asked to force, it will be "no depend". Bob, do you agree? Is it already too late to include this patch for the revalidation?
Quite right. It should default to NO_DEPEND on Lunix unless someone purposefully requests depend. It's not yet too late to get in. bob
Comment on attachment 442573 [details] [diff] [review] Patch v5 (checked in) I checked in the patch on the NSS trunk (NSS 3.12.7). Checking in coreconf/Linux.mk; /cvsroot/mozilla/security/coreconf/Linux.mk,v <-- Linux.mk new revision: 1.44; previous revision: 1.43 done Checking in nss/lib/freebl/Makefile; /cvsroot/mozilla/security/nss/lib/freebl/Makefile,v <-- Makefile new revision: 1.112; previous revision: 1.111 done Checking in nss/lib/freebl/config.mk; /cvsroot/mozilla/security/nss/lib/freebl/config.mk,v <-- config.mk new revision: 1.25; previous revision: 1.24 done Checking in nss/lib/freebl/manifest.mn; /cvsroot/mozilla/security/nss/lib/freebl/manifest.mn,v <-- manifest.mn new revision: 1.58; previous revision: 1.57 done And on the SOFTOKEN_3_13_BRANCH (Softoken 3.13). Checking in nss/lib/freebl/Makefile; /cvsroot/mozilla/security/nss/lib/freebl/Makefile,v <-- Makefile new revision: 188.8.131.52; previous revision: 184.108.40.206 done Checking in nss/lib/freebl/config.mk; /cvsroot/mozilla/security/nss/lib/freebl/config.mk,v <-- config.mk new revision: 220.127.116.11; previous revision: 18.104.22.168 done Checking in nss/lib/freebl/manifest.mn; /cvsroot/mozilla/security/nss/lib/freebl/manifest.mn,v <-- manifest.mn new revision: 22.214.171.124; previous revision: 1.57 done
Created attachment 450800 [details] [diff] [review] Set FREEBL_NO_DEPEND to 0 to force a "depend" build (checked in) Rather than adding FREEBL_FORCE_DEPEND (which would require documenting the precedence if both FREEBL_NO_DEPEND and FREEBL_FORCE_DEPEND were set), I suggest that we just use a FREEBL_NO_DEPEND of 0 to force a "depend" build. This requires changing ifdef FREEBL_NO_DEPEND to a comparison with 1 ifeq ($(FREEBL_NO_DEPEND),1) Bob, what do you think?
Slavo: The patch I checked in for this bug yesterday caused new memory leaks on the "memleak dopushups Linux" tinderbox. Here is a new stack: nss_Init/SECMOD_LoadModule/SECMOD_LoadModule/secmod_LoadPKCS11Module/secmod_ModuleInit/FC_Initialize/nsc_CommonInitialize/RNG_RNGInit/freebl_RunLoaderOnce/PR_CallOnce/freebl_LoadDSO/FREEBL_GetVector/FREEBL_InitStubs/dlopen@@GLIBC_2.1/_dlerror_run/_dl_catch_error/dlopen_doit/_dl_open/_dl_catch_error/dl_open_worker/_dl_map_object_deps/malloc Note the FREEBL_InitStubs call in FREEBL_GetVector, which is introduced by the patch. This leak in dlopen can be safely ignored. (We call dlclose here: http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/security/nss/lib/freebl/stubs.c&rev=1.8&mark=533,536,541,543,555#529 ) Could you please add a suppression for it? Perhaps the suppression should start from dlopen: dlopen@@GLIBC_2.1/_dlerror_run/_dl_catch_error/dlopen_doit/_dl_open/_dl_catch_error/dl_open_worker/_dl_map_object_deps/malloc This part of the stack is common to many of your existing suppressions, so this leak seems like a bug of dlopen. Thanks.
Added stack: **/FREEBL_InitStubs/dlopen@@GLIBC_2.1/** Checking in ignored; /cvsroot/mozilla/security/nss/tests/memleak/ignored,v <-- ignored new revision: 1.80; previous revision: 1.79 done
Comment on attachment 450800 [details] [diff] [review] Set FREEBL_NO_DEPEND to 0 to force a "depend" build (checked in) r+ rrelyea
Comment on attachment 450800 [details] [diff] [review] Set FREEBL_NO_DEPEND to 0 to force a "depend" build (checked in) I checked in this patch on the NSS trunk (NSS 3.12.7) and SOFTOKEN_3_13_BRANCH (Softoken 3.13). Checking in coreconf/Linux.mk; /cvsroot/mozilla/security/coreconf/Linux.mk,v <-- Linux.mk new revision: 1.45; previous revision: 1.44 done Checking in nss/lib/freebl/Makefile; /cvsroot/mozilla/security/nss/lib/freebl/Makefile,v <-- Makefile new revision: 1.113; previous revision: 1.112 done Checking in nss/lib/freebl/config.mk; /cvsroot/mozilla/security/nss/lib/freebl/config.mk,v <-- config.mk new revision: 1.26; previous revision: 1.25 done Checking in nss/lib/freebl/Makefile; /cvsroot/mozilla/security/nss/lib/freebl/Makefile,v <-- Makefile new revision: 126.96.36.199; previous revision: 188.8.131.52 done Checking in nss/lib/freebl/config.mk; /cvsroot/mozilla/security/nss/lib/freebl/config.mk,v <-- config.mk new revision: 184.108.40.206; previous revision: 220.127.116.11 done
Now nsslowhash.h doesn't get installed in dist/public/nss as we need downstream for building softoken in fedora and rhel. I will enter a separate bug and refer to this one.