1200580 - Crash in mozilla::DisplayItemClip::IntersectWith

Status: VERIFIED FIXED

(Core :: Layout, defect)

Description

[Michael Henretty [:mikehenrty][:mhenretty]]

When running a certain Gaia Integration tests with Mulet on an optimized build (debug and prod), we are getting the crash from below. To run the test, download latest Mulet and run the following command:

RUNTIME=/PATH/TO/MULET/firefox TEST_FILES=/PATH/TO/gaia/apps/system/test/marionette/pinning_the_web_test.js make test-integration

This crashes about 50% of the time on my Mac Pro. It is also crashing on Linux in c-i: https://treeherder.mozilla.org/logviewer.html#?job_id=2634427&repo=b2g-inbound

* thread #1: tid = 0x3e4fa, 0x00000001043db9c9 XUL`mozilla::DisplayItemClip::IntersectWith(mozilla::DisplayItemClip const&) + 409, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
  * frame #0: 0x00000001043db9c9 XUL`mozilla::DisplayItemClip::IntersectWith(mozilla::DisplayItemClip const&) + 409
    frame #1: 0x00000001043e6e37 XUL`mozilla::ContainerState::ProcessDisplayItems(nsDisplayList*) + 903
    frame #2: 0x00000001043edd35 XUL`mozilla::FrameLayerBuilder::BuildContainerLayerFor(nsDisplayListBuilder*, mozilla::layers::LayerManager*, nsIFrame*, nsDisplayItem*, nsDisplayList*, mozilla::ContainerLayerParameters const&, mozilla::gfx::Matrix4x4 const*, unsigned int) + 8533
    frame #3: 0x000000010444b177 XUL`nsDisplayList::PaintRoot(nsDisplayListBuilder*, nsRenderingContext*, unsigned int) + 999
    frame #4: 0x000000010447ca2c XUL`nsLayoutUtils::PaintFrame(nsRenderingContext*, nsIFrame*, nsRegion const&, unsigned int, unsigned int) + 4780
    frame #5: 0x000000010449c346 XUL`PresShell::RenderDocument(nsRect const&, unsigned int, unsigned int, gfxContext*) + 1414
    frame #6: 0x0000000103a4393e XUL`mozilla::dom::CanvasRenderingContext2D::DrawWindow(nsGlobalWindow&, double, double, double, double, nsAString_internal const&, unsigned int, mozilla::ErrorResult&) + 1374
    frame #7: 0x0000000103644fad XUL`mozilla::dom::CanvasRenderingContext2DBinding::drawWindow(JSContext*, JS::Handle<JSObject*>, mozilla::dom::CanvasRenderingContext2D*, JSJitMethodCallArgs const&) + 1293
    frame #8: 0x0000000103a06eb1 XUL`mozilla::dom::GenericBindingMethod(JSContext*, unsigned int, JS::Value*) + 401
    frame #9: 0x000000010547ef9c XUL`js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) + 444
    frame #10: 0x000000010549082d XUL`Interpret(JSContext*, js::RunState&) + 35629
    frame #11: 0x0000000105487cbd XUL`js::RunScript(JSContext*, js::RunState&) + 381
    frame #12: 0x000000010547f3c5 XUL`js::Invoke(JSContext*, JS::CallArgs const&, js::MaybeConstruct) + 1509
    frame #13: 0x0000000105468042 XUL`js::Invoke(JSContext*, JS::Value const&, JS::Value const&, unsigned int, JS::Value const*, JS::MutableHandle<JS::Value>) + 610
    frame #14: 0x00000001057f1fef XUL`JS_CallFunctionValue(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>, JS::HandleValueArray const&, JS::MutableHandle<JS::Value>) + 111
    frame #15: 0x0000000102c365b2 XUL`nsXPCWrappedJSClass::CallMethod(nsXPCWrappedJS*, unsigned short, XPTMethodDescriptor const*, nsXPTCMiniVariant*) + 5250
    frame #16: 0x000000010246c230 XUL`PrepareAndDispatch + 576
    frame #17: 0x000000010246b07b XUL`SharedStub + 91
    frame #18: 0x000000010241340a XUL`(anonymous namespace)::MessageLoopIdleTask::Run() + 58
    frame #19: 0x00000001024134a7 XUL`(anonymous namespace)::MessageLoopTimerCallback::Notify(nsITimer*) + 23
    frame #20: 0x0000000102467322 XUL`nsTimerImpl::Fire() + 994
    frame #21: 0x000000010245af47 XUL`nsTimerEvent::Run() + 215
    frame #22: 0x000000010245ee11 XUL`nsThread::ProcessNextEvent(bool, bool*) + 1041
    frame #23: 0x0000000102488301 XUL`NS_ProcessPendingEvents(nsIThread*, unsigned int) + 81
    frame #24: 0x000000010419d874 XUL`nsBaseAppShell::NativeEventCallback() + 116
    frame #25: 0x00000001041f7509 XUL`nsAppShell::ProcessGeckoEvents(void*) + 297
    frame #26: 0x00007fff8cc6ca01 CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17
    frame #27: 0x00007fff8cc5eb8d CoreFoundation`__CFRunLoopDoSources0 + 269
    frame #28: 0x00007fff8cc5e1bf CoreFoundation`__CFRunLoopRun + 927
    frame #29: 0x00007fff8cc5dbd8 CoreFoundation`CFRunLoopRunSpecific + 296
    frame #30: 0x00007fff8f3a356f HIToolbox`RunCurrentEventLoopInMode + 235
    frame #31: 0x00007fff8f3a32ea HIToolbox`ReceiveNextEventCommon + 431
    frame #32: 0x00007fff8f3a312b HIToolbox`_BlockUntilNextEventMatchingListInModeWithFilter + 71
    frame #33: 0x00007fff947de8ab AppKit`_DPSNextEvent + 978
    frame #34: 0x00007fff947dde58 AppKit`-[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 346
    frame #35: 0x00000001041f6ae6 XUL`-[GeckoNSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 86
    frame #36: 0x00007fff947d3af3 AppKit`-[NSApplication run] + 594
    frame #37: 0x00000001041f7c8c XUL`nsAppShell::Run() + 236
    frame #38: 0x0000000104ade349 XUL`nsAppStartup::Run() + 41
    frame #39: 0x0000000104b3748d XUL`XREMain::XRE_mainRun() + 3325
    frame #40: 0x0000000104b37835 XUL`XREMain::XRE_main(int, char**, nsXREAppData const*) + 645
    frame #41: 0x0000000104b37c43 XUL`XRE_main + 227
    frame #42: 0x0000000100001bfd firefox`main + 1773
    frame #43: 0x00000001000011b4 firefox`start + 52

Comment 1

[Gregor Wagner [:gwagner]]

Attachment: lldb bt

sorry for my horrible lldb skills....
Roc, any idea whats going on here?

Comment 2

[Michael Henretty [:mikehenrty][:mhenretty]]

Note, this bug is easier to reproduce if you apply my patch from:
https://github.com/mozilla-b2g/gaia/pull/31584

That said, this bug also happens on master.

Comment 3

[Michael Henretty [:mikehenrty][:mhenretty]]

Attachment: [Marionette JS Test Case] Make Mulet Crash

Here is a reduced test case that happens on the latest Master of mulet. You can also reproduce the crash manually by running Mulet then clicking on the Rocketbar, then typing in `about:blank` and clicking return.

I would say this crash happens about 50% of the time.

Comment 4

[Michael Henretty [:mikehenrty][:mhenretty]]

Crash report from reproducing manually:
https://crash-stats.mozilla.com/report/index/94137db4-e759-47f4-8bd3-d48972150901

Comment 5

[Michael Henretty [:mikehenrty][:mhenretty]]

Ah crap, I guess there are no symbols there, ignore comment 4.

Comment 6

[Gregor Wagner [:gwagner]]

Markus seems to know this code as well.

Comment 7

[Markus Stange [:mstange]]

I'm currently looking into a very similar crash in bug 1198492, bug 1198798 and bug 1181135. I should know more by the end of the day.

Comment 8

[Daniel Veditz [:dveditz]]

calling this a use-after-free based on the lldb bt showing mHdr is 0x5a5a5a5a....

Comment 9

[Markus Stange [:mstange]]

The patches in bug 1156238 and bug 1181135 should fix this. This bug only affects Firefox 42 and 43.

Comment 10

[Mats Palmgren (inactive)]

Markus, do you think bug 1181135 alone fixed this, or is bug also 1156238 needed?
If needed, could you prioritize the reviews in that bug?  Thanks.

Comment 11

[Markus Stange [:mstange]]

Bug 1156238 is not needed.

Comment 12

[Mats Palmgren (inactive)]

OK, great.  Let's dupe this then since that bug is already public.

Daniel, since my Core credentials was removed I can no longer mark
things private.  Could you hide the testcase attachment, comment 0
and comment 3, before making this bug public please?  Thanks.

Comment 13

[Daniel Veditz [:dveditz]]

It's usually better to have a security bug be "fixed by" its dupe than actually mark it as a dupe. otherwise we lose track of the fact that we fixed a security bug, which has implications for verification, advisories, and potential branch back-ports.

We don't need to hide those testcases, we just need to keep the bug hidden until the fix ships.

Comment 14

[Matt Wobensmith [:mwobensmith][:matt:]]

Hi Michael, does this fix work for you?

Comment 15

[Michael Henretty [:mikehenrty][:mhenretty]]

Hi Matt, thanks for following up. Indeed this seems to have at least fixed the bug that was causing our integration test to fail. As supporting evidence, we have since removed the workaround [1] that we initially put in place for this problem.

1.) https://github.com/mozilla-b2g/gaia/commit/5aa7b14d5d652f0821a7be9591caa50051cdd97b#diff-cd1a583e0231e554eae454cdda4df856R13

Comment 16

[Matt Wobensmith [:mwobensmith][:matt:]]

Super, thanks Michael. I'm marking verified based on this. Much appreciated.