Closed Bug 1093683 Opened 10 years ago Closed 8 years ago

[10.10] Firefox crashes opening or saving files (in the OS X native file picker) on Yosemite

Categories

(Core :: Widget: Cocoa, defect)

Product:
Core
Component:
Widget: Cocoa
Version:
33 Branch
Platform:
x86
macOS
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

()

Status:
RESOLVED WORKSFORME
Tracking Flags:
Tracking Status
firefox33 --- wontfix
firefox34 --- wontfix
firefox35 --- wontfix
firefox36 - affected
firefox37 --- affected

People

(Reporter: feranick, Unassigned)

Details

(Keywords: crash, regression, topcrash-mac)

Crash Data

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:33.0) Gecko/20100101 Firefox/33.0
Build ID: 20141027150301

Steps to reproduce:

The issue is random but fairly consistent. It happens when trying for example to save a succession of images (right lick on "Save Image As"). 


Actual results:

Firefox crashes


Expected results:

It should have not crashed
More details to reliably reproduce:

1. Search for an image in google image search. 
2. Select the image
3. Right click on it and select "save image as"
4. Keep repeating step 3.

After 2-3 times Firefox crashes.
Here it is:

AdapterDeviceID: 0x 116
AdapterDriverVersion: 
AdapterVendorID: 0x8086
Add-ons: cutyfox%40apps.metzweb.net:1.4.1,cryptocat%40crypto.cat:2.2.2,foxyproxy%40eric.h.jung:4.4.1,%7B972ce4c6-7e08-4474-a285-3208198ce6fd%7D:33.0.2,%7Bd10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d%7D:2.6.5
BuildID: 20141027150301
CrashTime: 1415133249
EMCheckCompatibility: true
EventLoopNestingLevel: 1
FramePoisonBase: 7ffffffff0dea000
FramePoisonSize: 4096
InstallTime: 1414727567
Notes: AdapterVendorID: 0x8086, AdapterDeviceID: 0x 116GL Layers? GL Context? GL Context+ GL Layers+ 
ProductID: {ec8030f7-c20a-464f-9b0e-13a3a9e97384}
ProductName: Firefox
ReleaseChannel: release
SecondsSinceLastCrash: 13822
StartupTime: 1415133020
Theme: classic/1.0
Throttleable: 1
URL: https://www.google.com/search?q=image&safe=off&source=lnms&tbm=isch&sa=X&ei=3TdZVOmqHJPesAT-voCQBQ&ved=0CAgQ_AUoAQ&biw=2520&bih=1475#safe=off&tbm=isch&q=firefox&facrc=_&imgdii=_&imgrc=11TW4NCefhmuMM%253A%3BjyJakPlKHxQy7M%3Bhttp%253A%252F%252Fwww.webvisionsevent.com%252Fuserfiles%252Ffirefox_logo-only_RGB.png%3Bhttp%253A%252F%252Fwww.webvisionsevent.com%252F2014%252F01%252Ffirefox-the-privacy-loving-browser%252F%3B2169%3B2081
Vendor: Mozilla
Version: 33.0.2
useragent_locale: en-US

This report also contains technical information about the state of the application when it crashed.
Out of curiosity, does this reproduce in safe mode? What about a new profile?

Steven, can you make anything out of these crashreports and/or can you reproduce? I'm wondering if this is a new 10.10 issue; all the crashes are in libdispatch (ie in OS X). Not sure what we're doing that's triggering that.
Component: Untriaged → Widget: Cocoa
Flags: needinfo?(smichaud)
Flags: needinfo?(feranick)
Product: Firefox → Core
It seems that the crashes are happening when the Adblock plus (2.6.5) extension is enabled...
Flags: needinfo?(feranick)
OS X has had file picker crashers for many years.  These are just a new variant of them, specific to Yosemite.  They're not a Firefox bug.

Though I've tried many times, I've never been able to reproduce any of them.  But sometime in the next few days I will try with Adblock Plus 2.6.5, to see what happens.
Flags: needinfo?(smichaud)
Nicola:  You *may* be able to make these crashes go away, at least temporarily, by restarting your computer.
Thanks, Steven. I tried that and everything works for a little while (maybe a few hours) but then it comes back. In any case, thanks very much for prompt action and response.
These crashes, or ones like them, are now the #1 Mac topcrasher on the 34 branch:

https://crash-stats.mozilla.com/report/list?signature=libdispatch.dylib%400x84bd&product=Firefox&query_type=contains&range_unit=weeks&process_type=any&platform=mac&version=Firefox%3A34.0b&version=Firefox%3A34.0b9&version=Firefox%3A34.0b8&version=Firefox%3A34.0b7&version=Firefox%3A34.0b6&version=Firefox%3A34.0b5&version=Firefox%3A34.0b4&version=Firefox%3A34.0b3&version=Firefox%3A34.0b2&version=Firefox%3A34.0b1&hang_type=any&date=2014-11-14+20%3A00%3A00&range_value=1#tab-comments

Most of the comments refer to uploading files.  But saving and uploading files is all done with the native OS X filepicker -- which is where these crashes happen.

I'll try to find a few hours next week to spend on this bug.  The first thing I'll do is try to repro with Adblock Plus.
Status: UNCONFIRMED → NEW
Crash Signature: [@ libdispatch.dylib@0x84bd ]
Ever confirmed: true
Keywords: crash, topcrash-mac
Nicola:

In preparation for my trying to repro these crashes, please tell me more about what you mean by the following:

> It happens when trying for example to save a succession of images

If possible, please give me particular, publicly accessible pages, and specific steps to follow.  I'd be happy if they worked as seldom as 50% of the time.
Flags: needinfo?(feranick)
https://crash-stats.mozilla.com/report/list?signature=libdispatch.dylib%400x84bd seems to be exclusive to Yosemite, but pretty high up in the charts for a Mac-only crash now.
Marcia, have you seen that in your Yosemite testing?
Summary: Firefox crashes when saving files → [10.10] Firefox crashes when saving files
Flags: needinfo?(mozillamarcia.knous)
Summary: [10.10] Firefox crashes when saving files → [10.10] Firefox crashes when opening or saving files (in the OS X native file picker)
Summary: [10.10] Firefox crashes when opening or saving files (in the OS X native file picker) → [10.10] Firefox crashes opening or saving files (in the OS X native file picker) on Yosemite
(In reply to Robert Kaiser (:kairo@mozilla.com) from comment #12)
> https://crash-stats.mozilla.com/report/list?signature=libdispatch.
> dylib%400x84bd seems to be exclusive to Yosemite, but pretty high up in the
> charts for a Mac-only crash now.
> Marcia, have you seen that in your Yosemite testing?

I haven't seen this, but it looks as if most of the URL were involved with uploading files of some kind. I can say in previous Apple OS upgrades we have had issues with the file picker as Steven notes in Comment 7.
Flags: needinfo?(mozillamarcia.knous)
OK, I can trivially reproduce this after upgrading to 10.10.1 (I tried before the update, still with 10.10.0, and everything worked fine) by setting prefs to let me chose where to download to and trying to download something (just now tried downloading Aurora for Android). See bp-50a85702-8fb9-4a9c-93a8-9f1262141119.
[Tracking Requested - why for this release]:
This is trivial to reproduce and makes it impossible to download any file when set to "Always ask me where to save files". It may be caused by something Apple did in 10.10.1 but it still is us crashing when the user wants to do something in Firefox so we should track.
Severity: normal → critical
status-firefox33: --- → affected
status-firefox34: --- → affected
status-firefox35: --- → affected
status-firefox36: --- → affected
tracking-firefox34: --- → ?
tracking-firefox35: --- → ?
tracking-firefox36: --- → ?
tracking-firefox37: --- → ?
Keywords: regression, reproducible
Sorry, what I can reproduce is a different signature and so far only on Nightly. Not sure if it's the same thing or not.
Keywords: reproducible
https://crash-stats.mozilla.com/report/list?product=Firefox&signature=libdispatch.dylib%400x84bd still says this happens on all channels, and it's pretty high in stats, so I'll leave it nominated still.
All those crashes seem to be on "Mac OS X 10.10.0 14A389" actually. And the comment on bp-902d6987-3016-44a1-9e22-a61dc2141119 nicely says "Welcome to the latest OS X build. Firefox isn't the only thing crashing."
Robert:

Is OS X 10.10.1 actually out now, or were you testing with a DP?

Note that Nicola reported he's only able to reproduce these crashes with AdBlock Plus installed and activated (version 2.6.5 in his case).  So whoever's trying to reproduce these crashes should do it with AdBlock Plus installed.

Socorro's correlations (for this crash) also mention the following:

29% (60/208	15% (136/894	{b9db16a4-6edc-47ec-a1f4-b86292ed211d} (Video DownloadHelper, https://addons.mozilla.org/addon/3006)
11% (22/208	5% (44/894	{b9bfaf1c-a63f-47cd-8b9a-29526ced9060} (Download YouTube As MP4, https://addons.mozilla.org/addon/11869)

Nicola:

Please, please give me the information I've requested in comment #11.
(In reply to Steven Michaud from comment #19)
> Is OS X 10.10.1 actually out now, or were you testing with a DP?

I did get it through the update function on the app store. Note that I said the crash happens with 10.10.0 though, apparently.
(In reply to Steven Michaud from comment #19)
> Socorro's correlations (for this crash) also mention the following:

Yes, no really significant correlations with anything, not ABP nor anything else. The download stuff probably only shows up because this has to do with file dialogs and those are used more by people who coincide with users doing those kinds of downloads.
Yes, I now also see OS X 10.10.1 in App Store Updates.  I'm currently downloading it.

I wonder if it will make any difference with these crashes :-)

Do note, though, that it may just change their signature.
Here's another signature of what also appear to be file picker crashes, also entirely on OS X 10.10.

https://crash-stats.mozilla.com/report/list?signature=AppKit%400x801d47&product=Firefox&query_type=contains&range_unit=weeks&process_type=any&platform=mac&hang_type=any&date=2014-11-19+17%3A00%3A00&range_value=1#tab-sigsummary
Crash Signature: [@ libdispatch.dylib@0x84bd ] → [@ libdispatch.dylib@0x84bd ] [@ AppKit@0x801d47 ]
Nicola:

In addition to what I've already requested of you in comment #11, please try upgrading to OS X 10.10.1, to see if it makes these crashes go away for you.
Apparently OS X 10.10.1 *doesn't* fix this bug:

https://crash-stats.mozilla.com/search/?signature=~libdispatch.dylib%400x84bd&platform_version=%3D10.10.1

Though it's still possible that it lessens its frequency.
From what I gather by reading the comments, it doesn't sound like we can ship with this bug. We're going to need a fix ASAP in order to ship 34 on time.

Steven - Can you take this bug? If you can find a fix by tomorrow, we can consider taking the fix in beta10.
status-firefox33: affected → wontfix
status-firefox37: --- → affected
tracking-firefox34: ? → +
tracking-firefox35: ? → +
tracking-firefox36: ? → +
tracking-firefox37: ? → +
Flags: needinfo?(smichaud)
> We're going to need a fix ASAP in order to ship 34 on time.

That's very unrealistic.

This is an Apple bug.  Similar bugs have existed for years.  They happen deep in system code.  We've never been able to reproduce any of them, never mind work around them.

It's true this is (now) a topcrasher.  But that's happened before.  All we were able to do then is wait helplessly until it stopped being a topcrasher.

If we have any useful contacts in Apple, we should probably reach out to them.  But I think it's extremely unlikely they'll be able/willing to help -- after all even we can't reproduce these crashes.
Flags: needinfo?(smichaud)
Thank you for the clarification Steven. I think you've set the expectations for this bug pretty well at this point. I'd like to correct my comment 26 to state that this does not block Firefox 34. (As such, I'm dropping tracking.) I will try to direct some Apple contacts at this bug but agree that we shouldn't count on this being fixed until we have STR.
tracking-firefox34: + → ---
tracking-firefox35: + → ---
tracking-firefox37: + → ---
I'm going to do a full-court press, over the next few days, to see if I can figure out how to reproduce these crashes.  I've always failed in the past, so I think it's unlikely I'll succeed.  But this new variant of Apple's old file picker crashes seems to happen more often than any of its predecessors have for a long time.  Hopefully that increases my chances of success.
All my time was eaten by another bug.  So I'm just starting my full court press now.

This bug's existing signatures aren't in the top 10 for OS X 10.10.1.  But another signature (which I'm now adding to this bug) is currently #6 among the 10.10.1 topcrashers.  It's hard to tell much (yet) from its crash stacks -- but the comments indicate that it's also a file picker problem.

https://crash-stats.mozilla.com/signature/?platform_version=~10.10.1&product=Firefox&signature=AppKit%400x801ca7&_columns=date&_columns=product&_columns=version&_columns=build_id&_columns=platform&_columns=reason&_columns=address&page=1
Crash Signature: [@ libdispatch.dylib@0x84bd ] [@ AppKit@0x801d47 ] → [@ libdispatch.dylib@0x84bd ] [@ AppKit@0x801d47 ] [@ AppKit@0x801ca7 ]
Crash Signature: [@ libdispatch.dylib@0x84bd ] [@ AppKit@0x801d47 ] [@ AppKit@0x801ca7 ] → [@ libdispatch.dylib@0x84bd ] [@ libdispatch.dylib@0x197e ] [@ AppKit@0x801d47 ] [@ AppKit@0x801ca7 ]
Atos translation of first of Nicola's crash stacks from comment #4:

_dispatch_async_f2 (in libdispatch.dylib) + 105
__NSXPCCONNECTION_IS_CALLING_OUT_TO_ERROR_BLOCK__ (in Foundation) + 6
__97-[NSXPCConnection
  _sendInvocation:proxyNumber:remoteInterface:withErrorHandler:timeout:userInfo:]_block_invoke311
  (in Foundation) + 694
_xpc_connection_reply_callout (in libxpc.dylib) + 46
_xpc_connection_call_reply (in libxpc.dylib) + 35
_dispatch_client_callout (in libdispatch.dylib) + 7
_dispatch_queue_drain (in libdispatch.dylib) + 1099
_dispatch_queue_invoke (in libdispatch.dylib) + 201
_dispatch_root_queue_drain (in libdispatch.dylib) + 462
_dispatch_worker_thread3 (in libdispatch.dylib) + 90
_pthread_wqthread (in libsystem_pthread.dylib) + 732
start_wqthread (in libsystem_pthread.dylib) + 12
_dispatch_barrier_sync_f (in libdispatch.dylib) + 59
It seems that all of the crashes at libdispatch.dylib@0x84bd take place at the following instruction, where rax == 0x5a5a5a5a5a5a5a5a:

mov    qword [ds:rax+0x10], r8

So it seems likely that these crashes are an Apple use-after-free, possibly made more likely to happen by jemalloc's memory poisoning.
Atos translation of a crash at AppKit@801d47 (bp-9a19b4bf-d19b-4503-8891-60bc22141109):

-[NSScrollView _setForcesContentInsetsLayout:] (in AppKit) + 15
-[NSTableView setLayer:] (in AppKit) + 332
-[NSView _dropLayerBackingWithoutFlushDisable] (in AppKit) + 223
-[NSView _childrenLostLayerTreeAncestor] (in AppKit) + 406
-[NSClipView _recursiveLostLayerTreeHostAncestor] (in AppKit) + 42
-[NSView _setSuperview:] (in AppKit) + 897
-[NSClipView _setSuperview:] (in AppKit) + 152
-[NSView removeFromSuperview] (in AppKit) + 434
-[NSView removeFromSuperviewWithoutNeedingDisplay] (in AppKit) + 37
-[NSView _finalizeWithReferenceCounting] (in AppKit) + 999
-[NSView dealloc] (in AppKit) + 150
-[NSScrollView dealloc] (in AppKit) + 651
objc_object::sidetable_release(bool) (in libobjc.dylib) + 235
CFRelease (in CoreFoundation) + 303
-[__NSArrayI dealloc] (in CoreFoundation) + 124
objc_object::sidetable_release(bool) (in libobjc.dylib) + 235
CFRelease (in CoreFoundation) + 303
-[__NSDictionaryM dealloc] (in CoreFoundation) + 268
objc_object::sidetable_release(bool) (in libobjc.dylib) + 235
_object_remove_assocations (in libobjc.dylib) + 409
objc_destructInstance (in libobjc.dylib) + 131
object_dispose (in libobjc.dylib) + 21
-[NSResponder dealloc] (in AppKit) + 138
-[NSView dealloc] (in AppKit) + 181
-[NSScrollView dealloc] (in AppKit) + 651
objc_object::sidetable_release(bool) (in libobjc.dylib) + 235
(anonymous namespace)::AutoreleasePoolPage::pop(void*) (in libobjc.dylib) + 574
_CFAutoreleasePoolPop (in CoreFoundation) + 49
CFRunLoopTimerInvalidate (in CoreFoundation) + 604
__CFRunLoopDoTimer (in CoreFoundation) + 1112
__CFRunLoopDoTimers (in CoreFoundation) + 300
__CFRunLoopRun (in CoreFoundation) + 2023
CFRunLoopRunSpecific (in CoreFoundation) + 295
RunCurrentEventLoopInMode (in HIToolbox) + 234
ReceiveNextEventCommon (in HIToolbox) + 178
_BlockUntilNextEventMatchingListInModeWithFilter (in HIToolbox) + 70
_DPSNextEvent (in AppKit) + 963
-[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] (in AppKit) + 193

Parts of this are pretty clearly corrupt -- for example the repeats of objc_object::sidetable_release(bool).  But it gives some indication that trouble is being caused when an autorelease pool is released.
By the way, I did try to reproduce these crashes, on OS X 10.10.0, using AdBlock Plus and downloading various kinds of things.  I didn't see any crashes, probably because I have too little information about precisely how these crashes are triggered.  I know from past experience that there's no point continuing to do this -- the chances of success are far too remote.

But I am going to keep digging and thinking, at least for another day.
Atos translation of a crash at libdispatch.dylib@0x197e (bp-2b8e4ec3-a566-4298-99be-049122141123):

_os_object_retain (in libdispatch.dylib) + 45
_xpc_connection_unpack_message (in libxpc.dylib) + 180
_xpc_connection_mach_event (in libxpc.dylib) + 1877
_dispatch_client_callout4 (in libdispatch.dylib) + 8
_dispatch_mach_msg_invoke (in libdispatch.dylib) + 444
_dispatch_queue_drain (in libdispatch.dylib) + 570
_dispatch_mach_invoke (in libdispatch.dylib) + 231
_dispatch_root_queue_drain (in libdispatch.dylib) + 462
_dispatch_worker_thread3 (in libdispatch.dylib) + 90
_pthread_wqthread (in libsystem_pthread.dylib) + 728
start_wqthread (in libsystem_pthread.dylib) + 12
_dispatch_barrier_sync_f (in libdispatch.dylib) + 59
Atos translation of a crash at AppKit@0x801ca7 (bp-77482d07-9610-403d-acc0-dac1b2141124):

-[NSScrollView _setForcesContentInsetsLayout:] (in AppKit) + 15
-[NSTableView setLayer:] (in AppKit) + 332
-[NSSegmentedCell _calculateSelectedSegmentForPoint:] (in AppKit) + 58
-[NSView _childrenLostLayerTreeAncestor] (in AppKit) + 406
-[NSClipView _recursiveLostLayerTreeHostAncestor] (in AppKit) + 42
-[NSView _setSuperview:] (in AppKit) + 897
-[NSClipView _setSuperview:] (in AppKit) + 152
-[NSView removeFromSuperview] (in AppKit) + 434
-[NSView removeFromSuperviewWithoutNeedingDisplay] (in AppKit) + 37
-[NSView _finalizeWithReferenceCounting] (in AppKit) + 999
-[NSView dealloc] (in AppKit) + 150
-[NSScrollView dealloc] (in AppKit) + 651
objc_object::sidetable_release(bool) (in libobjc.dylib) + 235
CFRelease (in CoreFoundation) + 303
-[__NSArrayI dealloc] (in CoreFoundation) + 124
objc_object::sidetable_release(bool) (in libobjc.dylib) + 235
CFRelease (in CoreFoundation) + 303
-[__NSDictionaryM dealloc] (in CoreFoundation) + 268
objc_object::sidetable_release(bool) (in libobjc.dylib) + 235
_object_remove_assocations (in libobjc.dylib) + 409
objc_destructInstance (in libobjc.dylib) + 131
object_dispose (in libobjc.dylib) + 21
-[NSResponder dealloc] (in AppKit) + 138
-[NSView dealloc] (in AppKit) + 181
-[NSScrollView dealloc] (in AppKit) + 651
objc_object::sidetable_release(bool) (in libobjc.dylib) + 235
(anonymous namespace)::AutoreleasePoolPage::pop(void*) (in libobjc.dylib) + 574
_CFAutoreleasePoolPop (in CoreFoundation) + 49
CFRunLoopTimerInvalidate (in CoreFoundation) + 604
__CFRunLoopDoTimer (in CoreFoundation) + 1112
__CFRunLoopDoTimers (in CoreFoundation) + 300
__CFRunLoopRun (in CoreFoundation) + 2023
CFRunLoopRunSpecific (in CoreFoundation) + 295
RunCurrentEventLoopInMode (in HIToolbox) + 234
ReceiveNextEventCommon (in HIToolbox) + 430
_BlockUntilNextEventMatchingListInModeWithFilter (in HIToolbox) + 70
_DPSNextEvent (in AppKit) + 963
-[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] (in AppKit) + 193
-[NSEvent window] (in AppKit) + 85
cache_getImp (in libobjc.dylib) + 191
-[NSApplication sendEvent:] (in AppKit) + 3710
Look like nothing is going to happen for this. Untracking it for 36...
tracking-firefox36: + → -
I have no idea if this is useful for anyone, but I thought I would throw it out.

We were encountering a file picker crash on Postbox when clicking on Photos in Yosemite. It was deep inside the Apple code. We discovered that it happened on Firefox but was mysteriously fixed in this regression range:

http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=00b264c7cced&tochange=97aa3da59001

(Note that was all before Yosemite even existed).

After a lot of building, we finally discovered that this NSPR change:

https://bugzilla.mozilla.org/show_bug.cgi?id=871064

fixed the crash.

So thread priority has some bearing on file picker crashes.

Just an FYI.
This works for me with latest Nightly, 49.0a1, 20160511030221, Mac 10.11.2.  Resolving as such, please reopen if you have reliable steps to reproduce.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → WORKSFORME
Flags: needinfo?(feranick)
You need to log in before you can comment on or make changes to this bug.