Closed Bug 700835 Opened 13 years ago Closed 13 years ago

[Mac] Firefox 8 and up crash with Apple's latest Java updates for OS X 10.6 and 10.7 closing tab/window containing Java applet

Categories

(Core Graveyard :: Plug-ins, defect)

Product:
Core Graveyard
Component:
Plug-ins
Version:
8 Branch
Platform:
All
macOS
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

(firefox8 fixed, firefox9 fixed, firefox10 fixed, firefox11 fixed)

Status:
VERIFIED FIXED
Milestone:
mozilla11
Tracking Flags:
Tracking Status
firefox8 --- fixed
firefox9 --- fixed
firefox10 --- fixed
firefox11 --- fixed

People

(Reporter: mario_grgic, Assigned: smichaud)

References

(
URL
)

Details

(Keywords: verified-beta, Whiteboard: [qa!] STR in comment #67, best summary in comment #44, rdar://10444452)

Crash Data

Attachments

(4 files, 2 obsolete files)

Gdb trace of crash (C and Java stacks)
57.77 KB, text/plain
Details
Patch v3 (another suggestion from Josh)
12.66 KB, patch
smichaud
: review+
akeybl
: approval-mozilla-aurora+
akeybl
: approval-mozilla-beta+
Details | Diff | Splinter Review
Patch v3 (bool -> PRBool)
11.85 KB, patch
smichaud
: review+
Details | Diff | Splinter Review
Plugin crash
38.05 KB, text/plain
Details
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:8.0) Gecko/20100101 Firefox/8.0 Build ID: 20111104165243 Steps to reproduce: This happens with Firefox 8.0 and Mac OS X 10.6.8 and java version "1.6.0_29" Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-10M3527). Visit http://java.com/en/download/testjava.jsp wait for the java applet to load and display the JRE info. Close the tab and firefox crashes. This is reproducible every time.
I was able to reproduce this using Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:8.0) Gecko/20100101 Firefox/8.0. Here is my crash report: https://crash-stats.mozilla.com/report/index/bp-556d9081-4ec7-4e97-8c9b-f141e2111108
Status: UNCONFIRMED → NEW
Crash Signature: [@ mozilla::plugins::parent::_forceredraw ]
Ever confirmed: true
Summary: Firefox 8 on Mac OS X 10.6.8 with Java 6 build 1.6.0_29-b11-402 crashes → [Mac] Firefox 8 with Java 6 build 1.6.0_29-b11-402 crashes after tab close
This is explosive and very recent -- all the crash reports I saw with this signature at https://crash-stats.mozilla.com/ have today's date! It cooincides with new Apple updates for Java (for OS X 10.6 and 10.7): http://support.apple.com/kb/DL1360 http://support.apple.com/kb/DL1421 So I'll bet this is an Apple bug. Though (since the crashes happen in our code) it's also possible that Apple's done something to trigger a Mozilla bug. I'll work on getting gdb crash stacks, which are often more accurate than Breakpad crash stacks.
Component: General → Plug-ins
Product: Firefox → Core
QA Contact: general → plugins
Severity: normal → critical
It currently ranks as the #1 top crasher in early Firefox 8 data.
Mac crasher, not the top overall crash. (In reply to Marcia Knous [:marcia] from comment #4) > It currently ranks as the #1 top crasher in early Firefox 8 data.
As best I can tell, these crashes no longer happen in current trunk code. The last mozilla-central nightly with which I can reproduce them (on OS X 10.7.2 with Apple's Java for OS X Lion Update 1 installed) is: firefox-2011-11-04-03-11-47-mozilla-central The first nightly with which I can no longer reproduce them is: firefox-2011-11-05-03-11-11-mozilla-central The STR from comment #0 is *not* 100% effective. In builds that do crash, I crash perhaps 25%-50% of the time. So Marcia and Mario, please double-check my "degression-range". I should shortly have a proper regression range.
My experience on my 10.7.2 machine is that I crash consistently with the steps in Comment 0 when I load that URL. Using Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:10.0a1) Gecko/20111104 Firefox/10.0a1, I can reproduce the crash loading the URL and then closing the tab with a clean profile. Using Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:10.0a1) Gecko/20111105 Firefox/10.0a1 I cannot reproduce the crash loading the URL and then closing the tab with a clean profile. Java configuration is: Java SE 6 Update 29.
I was able to reproduce the crash by closing the tab or hitting the back button. I also had a crash on using the latest UX nightly https://crash-stats.mozilla.com/report/index/bp-4d81069e-0f32-496c-84b3-2a6982111109
(In reply to comment #8) What's the relation between the latest UX nightly and current mozilla-central code? Are you also able to reproduce with the firefox-2011-11-04-03-11-47-mozilla-central nightly? Are you no longer able to reproduce with the firefox-2011-11-05-03-11-11-mozilla-central nightly? How about with today's mozilla-central nightly?
Not able to reproduce this on the same machine using Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:7.0.1) Gecko/20100101 Firefox/7.0.1.
I find the STR from comment #0 is much more likely to be effective (to lead to a crash) if I wait at least 10 seconds after http://java.com/en/download/testjava.jsp has finished loading before I close the tab it's in.
As best I can tell, here's the regression range for this bug: firefox-2011-07-10-03-07-56-mozilla-central firefox-2011-07-11-03-08-29-mozilla-central I don't crash with the first of these two nightlies. I crash easily with the second (if I wait at least 10 seconds after http://java.com/en/download/testjava.jsp has finished loading before closing the tab/window it's in). Marcia, could you double-check that?
URL: http://java.com/en/download/testjava.jsp
I can't tell (just by looking) which patch in either range triggered or untriggered this bug. I'll have to do an hg bisect on each end.
I crash using Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:8.0a1) Gecko/20110711 Firefox/8.0a1, which is the second build. But I was not able to crash using the first build. So I see the same thing you do on my machine. Should we file a Radar bug on this? (In reply to Steven Michaud from comment #12) > As best I can tell, here's the regression range for this bug: > > firefox-2011-07-10-03-07-56-mozilla-central > firefox-2011-07-11-03-08-29-mozilla-central > > I don't crash with the first of these two nightlies. I crash easily with > the second (if I wait at least 10 seconds after > http://java.com/en/download/testjava.jsp has finished loading before closing > the tab/window it's in). > > Marcia, could you double-check that?
The _forcedraw crash is up to #6 in the list of Mac topcrashers :-( > Should we file a Radar bug on this? Let's wait until we have more information. This may still turn out to be unambiguously our bug (and only very incidentally Apple's).
By the way, in earlier nightlies my bp crash signatures are at numerical addresses. For example: bp-8a6f00d4-cf33-4531-a302-3a7712111109 bp-db95ee70-86a1-45fa-9f28-4f4372111109
(In reply to Steven Michaud from comment #9) > (In reply to comment #8) > > What's the relation between the latest UX nightly and current > mozilla-central code? > > Are you also able to reproduce with the > firefox-2011-11-04-03-11-47-mozilla-central nightly? Are you no longer able > to reproduce with the firefox-2011-11-05-03-11-11-mozilla-central nightly? > > How about with today's mozilla-central nightly? I'm not sure I know the answer to any of those question. I just tried again with today's nightly (opened nightly, went to About Nightly, updated) and I haven't crashed but after going to the test page, then back to this bug without crashing I clicked the link to the test page and Firefox became completely unresponsive 3 out of 4 times for over a minute (then I force quit it).
Verdi, I'll try to deal with your questions later (once I have a better handle on this bug). For now I'm going to stick to the trunk/mozilla-central nightlies.
> For now I'm going to stick to the trunk/mozilla-central nightlies. Ones that I've downloaded separately, rather than getting them via updates.
Blocks: 701148
BTW, we turned updates off for Mac while we investigate this (bug 701148)
After much effort, I'm still unable to reproduce these crashes in custom builds (from code that should crash, according to the regression ranges). So I'm beginning to wonder if the crashes were triggered by something else than code changes. Ted, are you aware of any changes to the build system (for mozilla-central at least) that happened during the regression range from comment #12, or the "degression range" from comment #6? If not, is there some way we can reconstruct changes to the build system over time?
By the way, I'm aware of about:buildconfig, and am looking at its results to see if I can glean anything from them.
The above means it isn't likely a code issue, as 10.0a2 is what was on mozilla-central @ 9:00 am yesterday. So unless this fix came in today, it points to a build config / builder issue. Steven, can you use the 10.0a2 debug build to get the symbols and stack you need? It's at ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2011-11-09-mozilla-aurora-debug/
(Following up comment #21 and comment #22) Possible false alarm -- I got one of the regression ranges wrong (not the dates, but which patches were actually included). I *hope* things aren't yet quite that scary. I should know more in about 30 minutes.
(In reply to Steven Michaud from comment #16) > By the way, in earlier nightlies my bp crash signatures are at numerical > addresses. For example: > > bp-8a6f00d4-cf33-4531-a302-3a7712111109 > bp-db95ee70-86a1-45fa-9f28-4f4372111109 We don't keep nightly symbols in Soccoro for ever, so old nightlies won't have functions resolved from addresses. IIRC we keep the symbols for the last 30 nightlies.
I finally managed it! Fortunately what I implied above about this being a non-code issue is incorrect. I got this trace in a custom build made with the same patch at tip as the firefox-2011-11-04-03-11-47-mozilla-central nightly.
JNFRunLoop is a class in the JavaNativeFoundation subframework of the JavaVM framework -- so it's part of Apple's JVM. I've no idea why or how it's running Mozilla code. This behavior certainly looks suspicious. But I haven't yet finished my analysis, and it's too early to draw conclusions. I'm too tired, and will stop for the day. I pick this up again tomorrow morning.
One final teaser for the day: The jemalloc patch (for bug 694896) seems to be what "fixed" these crashes, at least on OS X 10.7.2! Now all we need to do is figure out why :-)
Wait, so does this only happen on 10.7? The jemalloc fix would seem to imply that
I checked crash stats for OS - Mac OS X 10.6.8, Mac OS X 10.7.0 and Mac OS X 10.7.2. It's now over 1000 crashes on 8.0 (throttled).
> Wait, so does this only happen on 10.7? The jemalloc fix would seem to imply that No. It also happens on 10.6, at least as often as on 10.7. But the jemalloc fix also changed behavior on 10.6. And I haven't yet tested on 10.6. In fact I probably have another solid day's work before I'll have gathered all the necessary information and can start to draw conclusions. And to suggest ways to fix or work around this bug.
Assignee: nobody → smichaud
Status: NEW → ASSIGNED
> But the jemalloc fix also changed behavior on 10.6. As I recall, we were doing something Very Wrong when registering jemalloc on mac. It's entirely possible it wasn't causing problems on 10.6 until this java update.
Do we see the crash on 10.6 in beta but not aurora? That would suggest that it may be fixed by the jemalloc change.
The crashes don't happen with the 7.0.1 release (on 10.6 or 10.7), which didn't have jemalloc on 10.7. So even on 10.7, the jemalloc patch only indirectly "fixes" these crashes. I hope to have more to say later today, when I've had a chance to do more tests.
Ah, comment 34 is wrong. FF10 turns on jemalloc on 10.5, 10.6, and 10.7, afaict. So if we're seeing crashes in FF8, then the problem there isn't our messed-up registration of jemalloc.
Adding another related signature.
Crash Signature: [@ mozilla::plugins::parent::_forceredraw ] → [@ mozilla::plugins::parent::_forceredraw ] [@ nsFrame::Init ]
I found one 6.0.2 crash on the original signature. Not sure it's related since there are no 7.x crashes and that's where the bulk of our user base was when this started. bp-1e10b3d1-acd8-4a3d-89ae-cbbfb2111108
I'm making progress -- I've now found the patch that actually triggered these crashes (at least on OS X 10.7): Bug 670079: Stop caching plugin instances. We don't ever want to restart instances. r=jst author Josh Aas <joshmoz@gmail.com> Fri Jul 08 12:39:22 2011 -0400 (at Fri Jul 08 12:39:22 2011 -0400) changeset 72551 b4725ef65601 The whole jemalloc business seems to have been a red herring. Yes, enabling jemalloc (landing the patch for bug 694896) in the 2011-11-05 mozilla-central nightly seems to "fix" the crashes (at least on OS X 10.7). And I also found that disabling jemalloc (backing out the patch for bug 414946) in the 2011-07-11 mozilla-central nightly seems to have triggered them (at least on OS X 10.7). But, like I also said, we don't see these crashes in FF 7.0.1 on OS X 10.7, and FF 7.0.1 doesn't have jemalloc. Testing the mozilla-central nightlies from 2011-07-08 (where the patch for bug 414946 was landed) to 2011-07-10 (just before the patch for bug 414946 was backed out) is difficult on OS X 10.7 -- thanks to problems with jemalloc, those builds are extremely crashy. But you can programmatically turn off jemalloc by setting the NO_MAC_JEMALLOC environment variable (I set it to "1"). You can do that at a Terminal prompt, then run FF from there. When you do that you find this bug's crashes start with the following mozilla-central nightly: firefox-2011-07-09-03-07-51-mozilla-central And I've already found (by trial and error) that the patch for bug 670069 is the one (in that regression range) that triggered them. I'll be playing with various changes to that patch, to find the smallest change that will "fix" this bug's crashes. Even then I might not have found the cure. There may yet be another layer of the onion that I need to peel. But I'm confident I'll find a reasonable fix (or workaround) today or tomorrow.
> And I've already found (by trial and error) that the patch for bug > 670069 is the one (in that regression range) that triggered them. That should be bug 670079.
STR to reproduce this bug's crashes with today's mozilla-central nightly: 1) Open a Terminal prompt and in it enter the following: export NO_MAC_JEMALLOC=1 2) Run today's mozilla-central from the same Terminal prompt. For example: ./[PathToFirefoxApp]/Contents/MacOS/firefox-bin 3) Visit this bug's URL (http://java.com/en/download/testjava.jsp). 4) Wait for a few seconds after the Java applet has finished loading, then close the tab/window it's in. About 50% of the time I crash, in mozilla::plugins::parent::_forceredraw. Please try this out, Marcia, and let us know if it works for you.
Yes, I can reproduce with your steps using Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:11.0a1) Gecko/20111110 Firefox/11.0a1. (In reply to Steven Michaud from comment #41) > STR to reproduce this bug's crashes with today's mozilla-central nightly: > > 1) Open a Terminal prompt and in it enter the following: > > export NO_MAC_JEMALLOC=1 > > 2) Run today's mozilla-central from the same Terminal prompt. For > example: > > ./[PathToFirefoxApp]/Contents/MacOS/firefox-bin > > 3) Visit this bug's URL (http://java.com/en/download/testjava.jsp). > > 4) Wait for a few seconds after the Java applet has finished loading, > then close the tab/window it's in. > > About 50% of the time I crash, in > mozilla::plugins::parent::_forceredraw. > > Please try this out, Marcia, and let us know if it works for you.
Attached patch Fix (obsolete) — — Splinter Review
Turns out this is Apple's bug: On both OS X 10.7 and 10.6, the Java plugin from Apple's latest Java updates can call NPN_ForceRedraw() on a plugin instance after the browser has called NPP_Destroy() on the plugin instance, and the browser has deleted the nsNPAPIPluginInstance object corresponding to the plugin instance. The crashes happen when NPN_ForceRedraw() (mozilla::plugins::parent::_forceredraw()) dereferences a pointer to the deleted nsNPAPIPluginInstance object. (As best I can tell, previous versions of Apple's Java plugin never call NPN_ForceRedraw() at any time.) But this is also partially our fault: The patch for bug 670079 dropped support for the NPPVpluginKeepLibraryInMemory variable (set by the plugin with a call to NPN_SetValue()). But we apparently didn't announce this anywhere (I can't find anything in the plugin-futures mailing list archives), and this variable remains documented (and undeprecated) on MDN (https://developer.mozilla.org/en/NPN_SetValue). In any case, Apple's current Java updates and the previous versions all call NPN_SetValue() to turn on NPPVpluginKeepLibraryInMemory, which (prior to bug 670079's removal of support for it) turns on plugin caching. When plugin caching is on, a plugin instance (its nsNPAPIPluginInstance object) doesn't get deleted when the plugin instance is destroyed. Instead it gets "cached", and potentially re-used to instantiate another plugin instance. This behavior masks Apple's bug -- when NPN_ForceRedraw() gets called on a destroyed plugin instance, its nsNPAPIPluginInstance object has generally not yet been deleted. Yes, there were problems with our implementation of support for NPPVpluginKeepLibraryInMemory (it's supposed to mean that the *plugin* stays in memory, not the *plugin instance*). And yes, the behavior of Apple's new Java plugins is clearly wrong. But I think our only reasonable course of action is to back out the patch for bug 670079, at least temporarily. That's (basically) what my patch does. (Though it preserves a few of the cosmetic changes from Josh's patch for bug 670079.) Before relanding that patch (if we choose to do so), we'll need to give plugin vendors plenty of warning, and time to change their plugins.
Attachment #573835 - Flags: review?(joshmoz)
Update: Josh doesn't like the idea of simply restoring the old plugin-caching code. His work since then assumes that old instance objects will never be re-used. So he wants the code changed to cache them but never re-use them. We're both currently working on that.
Maybe I'm confused but isn't caching but never re-using effectively going to leak nsNPAPIPluginInstance objects? When do they get removed from cache and released?
> When do they get removed from cache and released? They'll continue to get released as they always were. You'll see when we post the new patch.
Comment on attachment 573909 [details] [diff] [review] Patch v2 (Josh's suggested changes) These changes to my patch don't add any new code (they just remove code), and are otherwise just cosmetic (e.g. changes to method names). This should, I think, make it safe to use it in FF 8.0.1. The idea is that it's safer to use "existing" code (code that we know worked in the past). And no, this patch shouldn't cause any leaks.
Attachment #573909 - Attachment description: Patch v2 (Josh → Patch v2 (Josh's suggested changes)
Attachment #573909 - Flags: review?(joshmoz)
Attachment #573835 - Attachment is obsolete: true
Attachment #573835 - Flags: review?(joshmoz)
Steven: Can I get a tryserver build with that patch?
> Steven: Can I get a tryserver build with that patch? Not for the most recent patch. But here's one built from my original patch: http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/smichaud@pobox.com-1f88c57f0cfd/try-macosx64/firefox-11.0a1.en-US.mac.dmg
Remember that you need to set the NO_MAC_JEMALLOC environment variable when testing it. Otherwise jemalloc masks the bug (stops the crashes from happening).
Testing with the build in Comment 51, I do not crash when loading the site in the URL. I do have the NO_MAC_JEMALLOC environment variable set.
Testing with the build in Comment 51, I do not crash when loading the site in the URL and closing the tab. I do have the NO_MAC_JEMALLOC environment variable set.
Comment on attachment 573909 [details] [diff] [review] Patch v2 (Josh's suggested changes) Still haven't heard from Josh. Redirecting the review request to jst, who's agreed to take it on.
Attachment #573909 - Flags: review?(joshmoz) → review?(jst)
Attachment #573909 - Attachment is patch: true
Comment on attachment 573909 [details] [diff] [review] Patch v2 (Josh's suggested changes) Review of attachment 573909 [details] [diff] [review]: ----------------------------------------------------------------- ::: dom/plugins/base/nsNPAPIPluginInstance.cpp @@ +970,5 @@ > + return NS_OK; > +} > + > +nsresult > +nsNPAPIPluginInstance::ShouldCache(bool* shouldCache) We can probably just return the result here, no need for nsresult.
Attachment #573909 - Flags: review?(jst) → review+
Carry over Josh's r+.
Attachment #573909 - Attachment is obsolete: true
Attachment #573968 - Flags: review+
Comment on attachment 573968 [details] [diff] [review] Patch v3 (another suggestion from Josh) Landed on mozilla-inbound: http://hg.mozilla.org/integration/mozilla-inbound/rev/8ac0226cb220
Summary: [Mac] Firefox 8 with Java 6 build 1.6.0_29-b11-402 crashes after tab close → [Mac] Firefox 8 and up crash badly with Apple's latest Java updates for OS X 10.6 and 10.7
Summary: [Mac] Firefox 8 and up crash badly with Apple's latest Java updates for OS X 10.6 and 10.7 → [Mac] Firefox 8 and up crash with Apple's latest Java updates for OS X 10.6 and 10.7 closing tab/window containing Java applet
Attachment #573968 - Flags: approval-mozilla-aurora?
Comment on attachment 573968 [details] [diff] [review] Patch v3 (another suggestion from Josh) [Triage Comment] Approving for aurora and beta, since we're considering this for an 8.0.1 chemspill. Please also prepare for landing on release, and provide a try build (of the latest patch) that QA can verify the fix with.
Attachment #573968 - Flags: approval-mozilla-beta+
Attachment #573968 - Flags: approval-mozilla-aurora?
Attachment #573968 - Flags: approval-mozilla-aurora+
> provide a try build (of the latest patch) that QA can verify the fix > with I've started a build. It should be available by tomorrow morning. > Approving for aurora and beta > Please also prepare for landing on release I'll shortly post another version of my patch for beta and release, which will have "bool" changed to "PRBool" where appropriate. I'll land right away on aurora.
> I'll land right away on aurora. Actually "patch" doesn't work with aurora (because it's too different from trunk), so I'll also need to do translate my patch to aurora by hand.
Comment on attachment 573968 [details] [diff] [review] Patch v3 (another suggestion from Josh) Landed on mozilla-aurora: http://hg.mozilla.org/releases/mozilla-aurora/rev/dcfa8d61f3c5
https://hg.mozilla.org/mozilla-central/rev/8ac0226cb220
Status: ASSIGNED → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla11
My patch for this bug won't be in today's mozilla-central nightly. But here's a tryserver build to test with: http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/smichaud@pobox.com-f3b5cce53f90/try-macosx64/firefox-11.0a1.en-US.mac.dmg The patch should be in today's aurora nightly. But note that on the aurora branch (like on the trunk) you'll need to test with the NO_MAC_JEMALLOC environment variable set -- otherwise jemalloc will mask the bug.
STR for current trunk and aurora branch nightlies (for trunk nightlies dated 2011-11-05 and later, for aurora nightlies dated 2011-11-09 and later): 1) Open a Terminal prompt and in it enter the following: export NO_MAC_JEMALLOC=1 2) Run FF from the same Terminal prompt. For example: ./[PathToFirefoxApp]/Contents/MacOS/firefox-bin 3) Visit this bug's URL (http://java.com/en/download/testjava.jsp). 4) Wait for a few seconds after the Java applet has finished loading, then close the tab/window it's in. About 50% of the time I crash, in mozilla::plugins::parent::_forceredraw. STR for every other version of FF: 1) Visit this bug's URL (http://java.com/en/download/testjava.jsp). 2) Wait for a few seconds after the Java applet has finished loading, then close the tab/window it's in. About 50% of the time I crash, in mozilla::plugins::parent::_forceredraw.
Whiteboard: STR in comment #67
Whiteboard: STR in comment #67 → STR in comment #67, best summary in comment #44
Testing with Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:10.0a2) Gecko/20111112 Firefox/10.0a2 and following the STR in Comment 67 I have not crashed. Checking on 10.6 next. I have a Apple plugin container crash that is happening during my testing, but I believe it related to the Unity Player since it happening when a game that is loading needs that player.
> Apple plugin container crash What's this? Not a crash in plugin-container? (Note that Apple's Java plugin is 64-bit, and runs in process (in firefox-bin). Or more precisely the "server" process does. There's another "client" process (part of the Java plugin, not plugin-container) that also runs, and (for example) displays to the screen.)
(In reply to comment #70) Like you said, not related to this bug. But it *does* sound related to bug 699180.
Testing with Steven's Tryserver build from Comment 66, everything looks good so far on 10.6. I have also tested using the Aurora nightly build and have no experienced no crashes on 10.6.
per triage meeting: smichaud is going to also land this fix on relbranch GECKO80_2011110416_RELBRANCH in http://hg.mozilla.org/releases/mozilla-release.
If QA wants to do more testing on this, it was suggested in the meeting today that we load pages with lot of plugins. This should be more focused on the Java plugin but we can try some ad hoc testing with all plugins just to catch any potential edge cases.
Comment on attachment 573988 [details] [diff] [review] Patch v3 (bool -> PRBool) Landed on mozilla-release GECKO80_2011110416_RELBRANCH: http://hg.mozilla.org/releases/mozilla-release/rev/5925d01fa121
I have verified this using the steps from comment67 on: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:8.0) Gecko/20100101 Firefox/8.0 and on Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0) Gecko/20100101 Firefox/9.0 beta 1 Firefox crashes every time. I have tried to reproduce this issue on Aurora and Nightly using both steps to reproduce from comment67 and I got no crash. Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:10.0a2) Gecko/20111113 Firefox/10.0a2 Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:11.0a1) Gecko/20111113 Firefox/11.0a1 The same thing happens on MacOS X 10.7: Firefox crashes on these builds: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:8.0) Gecko/20100101 Firefox/8.0 Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:9.0) Gecko/20100101 Firefox/9.0 build 1 No crash on Aurora and Nightly: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:10.0a2) Gecko/20111113 Firefox/10.0a2 Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:11.0a1) Gecko/20111113 Firefox/11.0a1
Crash Signature: [@ mozilla::plugins::parent::_forceredraw ] [@ nsFrame::Init ] → [@ mozilla::plugins::parent::_forceredraw ] [@ @0x0 | mozilla::plugins::parent::_forceredraw ] [@ nsFrame::Init ]
Hardware: x86 → All
Crash Signature: [@ mozilla::plugins::parent::_forceredraw ] [@ @0x0 | mozilla::plugins::parent::_forceredraw ] [@ nsFrame::Init ] → [@ mozilla::plugins::parent::_forceredraw ] [@ @0x0 | mozilla::plugins::parent::_forceredraw ] [@ nsFrame::Init ] [@ nsBlockFrame::Init ]
Crash Signature: [@ mozilla::plugins::parent::_forceredraw ] [@ @0x0 | mozilla::plugins::parent::_forceredraw ] [@ nsFrame::Init ] [@ nsBlockFrame::Init ] → [@ mozilla::plugins::parent::_forceredraw ] [@ @0x0 | mozilla::plugins::parent::_forceredraw ] [@ nsFrame::Init ] [@ nsBlockFrame::Init ] [@ nsNPAPIPluginInstance::ForceRedraw ]
Bug report submitted to Apple (rdar://10444452) Latest Java updates access plugin instance after destruction, crash FF 8 As of Apple's latest Java updates (Java for Mac OS 10.6 Update 6 and Java for Mac OS X Lion Update 1), Apple's Java plugin can call an NPAPI function on a plugin instance after the plugin instance has been destroyed. In Firefox 8.0 and up, this causes crashes every time a tab or window is closed, or the Back button is pressed, if the tab or window contains a Java applet. This bug causes a very large number of crashes: In the few days after these Java updates were released, these crashes quickly became the #1 topcrasher on all versions of OS X, in all versions of Firefox. This issue was first reported at https://bugzilla.mozilla.org/show_bug.cgi?id=700835, and is being followed up there. The specific details are that Apple's latest Java updates can call NPN_ForceRedraw() on a plugin instance after the browser has called NPP_Destroy() on the plugin instance, and the browser has deleted the nsNPAPIPluginInstance object corresponding to the plugin instance. The crashes happen when NPN_ForceRedraw() (mozilla::plugins::parent::_forceredraw()) dereferences a pointer to the deleted nsNPAPIPluginInstance object. In older versions of Firefox this bug is masked (and these crashes prevented) by the fact they support plugin caching for plugins that (like Apple's Java plugin) call NPN_SetValue() to turn on NPPVpluginKeepLibraryInMemory. FF 8.0 no longer supports plugin caching. Though we'll be (partially) turning it back on to work around these crashes.
Whiteboard: STR in comment #67, best summary in comment #44 → STR in comment #67, best summary in comment #44, rdar://10444452
How soon can we expect these changes to come to Firefox 8?
Using the interwoven Teamsite CMS, FF8 will crash upon uploading or submitting a file. Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:8.0) Gecko/20100101 Firefox/8.0
(In reply to comment #80) Does the "interwoven Teamsite CMS" use Java? If not, then what you report sounds completely unrelated. In that case please open a new bug, and in it be sure to provide: 1) A publicly accessible URL or testcase with which to reproduce the problem. 2) What versions of Firefox the problem occurs on. 3) Whether or not the problem occurs with current mozilla-central nightlies.
(Following up comment #81) Also: 4) What version(s) of OS X you're using.
(In reply to comment #78) > How soon can we expect these changes to come to Firefox 8? Soon, but we don't yet know exactly when.
Verified on: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:8.0.1) Gecko/20100101 Firefox/8.0.1 - NO CRASH Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0) Gecko/20100101 Firefox/9.0 beta 1 - CRASH Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:10.0a2) Gecko/20111114 Firefox/10.0a2 - NO CRASH Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:11.0a1) Gecko/20111115 Firefox/11.0a1 - NO CRASH I've updated all my plugins to the latest version. Silverlight plugin installed is 4.0.60831.0. (JAVA SE 6 Update 29) On MacOS X 10.7: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:8.0.1) Gecko/20100101 Firefox/8.0.1 - NO CRASH Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:9.0) Gecko/20100101 Firefox/9.0 beta 1 - CRASH Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:10.0a2) Gecko/20111114 Firefox/10.0a2 - NO CRASH Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:11.0a1) Gecko/20111115 Firefox/11.0a1 - NO CRASH The issue doesn't reproduce anymore on Firefox 8.0.1
Can we please check this on 10.5 as well?
I tested Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:8.0.1) Gecko/20100101 Firefox/8.0.1. 10.5 had an updated version of Java which is different than 10.6 and 10.7 (Version 1.5.0_30). Testing with this version, I haven't been able to generate any crashes while open and closing Java version tabs. (In reply to Marcia Knous [:marcia] from comment #85) > Can we please check this on 10.5 as well?
Apple has a 10.7.3 seed out (Build 11D16) so I tested to see if anything was different - https://crash-stats.mozilla.com/report/index/bp-1b46ea1d-0399-442b-934e-7a1892111115 is my crash on Firefox 8.
If/when Apple fixes this bug, it'll be in their next Java updates for OS X 10.6 and 10.7.
And no, Apple fixing this bug won't mess up my patch :-)
Try run for d45763120aad is complete. Detailed breakdown of the results available here: https://tbpl.mozilla.org/?tree=Try&rev=d45763120aad Results (out of 275 total builds): success: 227 warnings: 19 failure: 29 Builds available at http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/Callek@gmail.com-d45763120aad
Can't reproduce with 10.6.8, Java 1.6.29 and FF8.0.1. Thanks.
This never landed on default on mozilla-release, so pushed it there with another mozilla-release push I was doing: http://hg.mozilla.org/releases/mozilla-release/pushloghtml?changeset=919ff0e4ee9a
I have tried to reproduce this issue on: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:8.0.1) Gecko/20100101 Firefox/8.0.1 build 2 and also on Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:8.0.1) Gecko/20100101 Firefox/8.0.1 build 2 Firefox didn't crash. Issue no longer reproducible on Firefox 8.0.1 build 2. See also comment76 and comment84.
http://crash-stats.mozilla.com/ still lists a few of these crashes for FF 8.0.1: https://crash-stats.mozilla.com/query/query?product=Firefox&version=Firefox%3A8.0.1%28beta%29&version=Firefox%3A8.0.1&platform=mac&range_value=1&range_unit=weeks&date=11%2F22%2F2011+12%3A36%3A42&query_search=signature&query_type=contains&query=forceredraw&reason=&build_id=&process_type=any&hang_type=any&do_query=1 And I've just found out it's still pretty easy to reproduce these crashes on pages with *lots* of Java applets -- for example http://www.grandfunkrailroad.com/station.htm (which I found in a comment on one of the crash reports). Apparently if there are enough Java applets on the page, there will be too many of them to fit in the plugin cache, and some of them will get deleted as soon as the tab/window closes, or you navigate away from the page. I'm not sure what we can do about this ... though of course we could increase the size of the cache. So Apple still needs to fix this bug!
As you'd expect, it's also quite easy to crash FF 7.0.1 playing around on http://www.grandfunkrailroad.com/station.htm (if you have one of Apple's latest Java updates installed): bp-38164bfa-123b-47ec-9f19-5b0812111122 So the 8.0.1 crashes have nothing to do with how we (partially) re-enabled plugin caching.
Whiteboard: STR in comment #67, best summary in comment #44, rdar://10444452 → [qa+] STR in comment #67, best summary in comment #44, rdar://10444452
I've verified this using the link from comment96, and Firefox crashes if I click one of the buttons. Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0) Gecko/20100101 Firefox/9.0 beta 3 Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:9.0) Gecko/20100101 Firefox/9.0 beta 3 The crash report is: bp-cd7a19d5-a675-4977-a464-3e33d2111125 Using the steps from comment67, Firefox is not crashing. I've tested this on: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0) Gecko/20100101 Firefox/9.0 beta 3 Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:9.0) Gecko/20100101 Firefox/9.0 beta 3 The Java version installed is: JAVA SE 6 Update 29 See also comment75, comment84 and also comment94 Considering this, it's still resolved fixed?
> Considering this, it's still resolved fixed? This bug now has 98 comments. Also, the number of crashes should be greatly reduced. Let's open a new bug on the remaining issue -- that we haven't yet worked around Apple's bug on pages that have lots of Java applets.
(In reply to Steven Michaud from comment #98) > > Considering this, it's still resolved fixed? > > This bug now has 98 comments. Also, the number of crashes should be greatly > reduced. > > Let's open a new bug on the remaining issue -- that we haven't yet worked > around Apple's bug on pages that have lots of Java applets. It sounds like we can mark this verified and open a cloned bug to deal with subsequent cases? Please confirm Steven.
> It sounds like we can mark this verified and open a cloned bug to > deal with subsequent cases? Please confirm Steven. Sounds good to me.
VERIFIED fixed -- Steven, can you please clone this bug for the subsequent cases? I'm not confident that I can accurately describe them, thanks.
Status: RESOLVED → VERIFIED
Keywords: verified-beta
Whiteboard: [qa+] STR in comment #67, best summary in comment #44, rdar://10444452 → [qa!] STR in comment #67, best summary in comment #44, rdar://10444452
> Steven, can you please clone this bug for the subsequent cases? Will do, when I get back from vacation (later this week).
Blocks: 705931
Filed Bug 705931 to track this issue. Steven - feel free to embellish it when any other information you wish. (In reply to Steven Michaud from comment #102) > > Steven, can you please clone this bug for the subsequent cases? > > Will do, when I get back from vacation (later this week).
No longer blocks: 705931
Blocks: 702238
It could seem that this fix broke stuff on Linux. My test-case is NemID, the national login system in Denmark (e.g. eboks.dk --> click "Log på privat"). It is a java applet. Cf. above the fix was pushed to Aurora on the 10th. I checked my build of Aurora from Nov 8. It works. The build from Nov. 11 crashed. More importantly, with the release of Fx10, this also crashes now. | 9.0a2-0.20111108 | + | |---------------------------------+---| | 10.0a2-0.20111111 | - | | Fri 11 Nov 2011 12:48:21 PM GMT | | |---------------------------------+---| +: works ; - crash I use Archlinux 64 bit with OpenJDK 6.b24_1.11 and icedtea 1.1.4. [Versions of JDK > 6 does not seem to work with NemID (in my case at least)]. The console output before crashing is: java version "1.6.0_24" OpenJDK Runtime Environment (IcedTea6 1.11) (ArchLinux-6.b24_1.11-1-x86_64) OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode) Feb 7, 2012 10:27:01 PM <WARN> DANID_DIGITAL_SIGNATUR appletdk.pbs.applet.bootstrap.new - Not using whitelisted class classloader: net.sourceforge.jnlp.runtime.JNLPClassLoader Feb 7, 2012 10:27:01 PM <INFO> Thread-4 - init Feb 7, 2012 10:27:01 PM <INFO> Thread-4 - selected loglevel: INFO Feb 7, 2012 10:27:01 PM <INFO> Thread-4 - This browser is a sun.applet.PluginAppletViewer[frame0,749,301,200x250,invalid,layout=java.awt.BorderLayout,title=,resizable,normal] Feb 7, 2012 10:27:02 PM <INFO> Thread-4 - start Assertion failure: rt->onOwnerThread(), at /build/src/mozilla-release/js/src/jsapi.cpp:6316 Aborted See also the Arch bug: https://bugs.archlinux.org/task/28235
Rasmus, please open a new bug and make it block this one.
Steve, I have opened bug 725117. Hopefully, I managed to `block [ 700835 ] out'. Thanks.
Depends on: 725117
No longer depends on: 725117
Product: Core → Core Graveyard
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: