1456534 - Crash in InvalidArrayIndex_CRASH | MergeState::ProcessItemFromNewList

Status: VERIFIED FIXED

(Core :: Web Painting, defect, P1)

Description

[Ryan VanderMeulen [:RyanVM]]

This bug was filed from the Socorro interface and is
report bp-f370098c-33c5-414f-ba43-e19ff0180424.
=============================================================

Noticed while verifying bug 1451384. This appears to have started back up again on build 20180424100107. Mac/Linux look like nullptr crashes, but there's a Windows one that looks maybe wildptr? Maybe a regression from bug 1439809 or bug 1452805?

Top 10 frames of crashing thread:

0 libmozglue.dylib MOZ_CrashPrintf mfbt/Assertions.cpp
1 XUL InvalidArrayIndex_CRASH xpcom/ds/nsTArray.cpp:26
2 XUL MergeState::ProcessItemFromNewList xpcom/ds/nsTArray.h:1029
3 XUL RetainedDisplayListBuilder::MergeDisplayLists layout/painting/RetainedDisplayListBuilder.cpp:487
4 XUL MergeState::ProcessItemFromNewList layout/painting/RetainedDisplayListBuilder.cpp:279
5 XUL RetainedDisplayListBuilder::MergeDisplayLists layout/painting/RetainedDisplayListBuilder.cpp:487
6 XUL MergeState::ProcessItemFromNewList layout/painting/RetainedDisplayListBuilder.cpp:279
7 XUL RetainedDisplayListBuilder::MergeDisplayLists layout/painting/RetainedDisplayListBuilder.cpp:487
8 XUL MergeState::ProcessItemFromNewList layout/painting/RetainedDisplayListBuilder.cpp:279
9 XUL RetainedDisplayListBuilder::MergeDisplayLists layout/painting/RetainedDisplayListBuilder.cpp:487

=============================================================

Comment 1

[Calixte Denizet (:calixte)]

There are 8 crashes (from 6 installations) in nightly 61 with buildid 20180424100107. In analyzing the backtrace, the regression may have been introduced by patch [1] to fix bug 1439809.

[1] https://hg.mozilla.org/mozilla-central/rev?node=02904a082859

Comment 2

[Andrew McCreight [:mccr8]]

#2 Windows top crash for the 4-25 Nightly.

Comment 3

[Matt Woodrow (:mattwoodrow)]

Looks like this reproduces on tweetdeck.twitter.com for some people, but I can't get it to crash on either of my machines.

Comment 4

[Paul Bone [:pbone]]

This crashes for me, it seems to be when I'm interacting with tweetdeck, just now https://crash-stats.mozilla.com/report/index/e6626d54-4da1-4278-8387-2f2f90180430#tab-bugzilla it crashed while I was scrolling my main list of tweets.  I don't know what exactly triggers it but I'll see if it keeps happening.

It's happened for me on both my desktop and laptop.  Both running Linux Mint with similar settings, although the desktop has NVidia graphics and the laptop has intel graphics.

Thanks.

Comment 5

[Paul Bone [:pbone]]

I know the following:

It crashes if tweetdeck is the active tab in one of the windows.  It doesn't matter whether that window is focused (or another application on a different workspace).  It doesn't matter whether i'm interacting with tweetdeck or not.  But it does have to be the selected tab.

Comment 6

[Fabien Cazenave [:kaze]]

This happens to me as well, mostly with tweetdeck (Ubuntu 16.04, Intel graphics, latest Nightly).

I’ve had this bug for 4 days now but still can’t find any reliable STR. I confirm that it crashes even without any user interaction — sometimes it takes a few seconds, sometimes it takes about an hour, which makes it difficult to get reliable STRs.

Comment 7

[Fabien Cazenave [:kaze]]

Warning: pure speculation. Steps That Might Reproduce:

 • open TweetDeck
 • scroll the main column until you get two tweets in a row that embed an image
 • on one of these two tweets, hover the “retweet” icon
 • wait for ~10 seconds
 • move the pointer up to hover the embedded image

→ crash. I’ve crashed my TweetDeck tab 10 times in a couple minutes with these steps, though not 100%.

Comment 8

[Fabien Cazenave [:kaze]]

Even simpler:

 • open TweetDeck
 • scroll the main column until you get two tweets in a row that embed an image (e.g. [1] and [2])
 • wait for ~10 seconds

→ crash.

The position of the pointer is not important — even with the pointer over the address bar, the tab will crash.
This doesn’t crash 100% of the time but when it does, it happens within 10 seconds.
If the tab doesn’t crash after 10 seconds, I hit F5, re-scroll to the same position and wait 10 seconds → crash.

[1] https://twitter.com/Barbara102006/status/991031039328858112
[2] https://twitter.com/AlsBoy/status/990961173238550528

Comment 9

[Fabien Cazenave [:kaze]]

Hmm, the “two tweets that embed an image” is not helping, I get crashes on any kind of tweet now.

 • open TweetDeck
 • scroll the main column a bit
 • wait for ~10 seconds

→ crash.

I hope this helps a bit more than the noise I’ve added. :-/

Comment 10

[Ryan VanderMeulen [:RyanVM]]

#2 overall content process crash -> blocking.

Comment 11

[Matt Woodrow (:mattwoodrow)]

I still can't reproduce this unfortunately :(

I did a try build with some more detailed assertion messages: https://treeherder.mozilla.org/#/jobs?repo=try&revision=df8c2dd44b3e189a22dabc6a73d50fc725742144

Can anyone reproduce it with those builds, and link me to the crash report. Thanks!

Comment 12

[Paul Bone [:pbone]]

The panel may not need to be scrolled.  I've had at least one crash after I scrolled for a bit, returned it to the top (home) position, and then it crashed.

I'll try to reproduce it with that build, or a debug build, maybe rr, later today.

Comment 13

[Paul Bone [:pbone]]

I've been procrastinating on twitter for about an hour now.  It hasn't crashed.  I wonder if twitter have changed something in tweetdeck that no-longer triggers this.

Comment 14

[Paul Bone [:pbone]]

I spoke to soon, it literally crashed just now.  I don't see any extra asserts,  Web console says:

nsLoginManager: searchLogins: `formSubmitURL` or `httpRealm` is recommended nsLoginManager.js:445
XML Parsing Error: no root element found
Location: https://tweetdeck.twitter.com/metrics
Line Number 1, Column 1:
metrics:1:1
Invalid URI. Load of media resource  failed.
tweetdeck.twitter.com
XML Parsing Error: no root element found
Location: https://tweetdeck.twitter.com/metrics
Line Number 1, Column 1: metrics:1:1
Invalid URI. Load of media resource  failed. tweetdeck.twitter.com
XML Parsing Error: no root element found
Location: https://tweetdeck.twitter.com/metrics
Line Number 1, Column 1:
metrics:1:1
Invalid URI. Load of media resource  failed. tweetdeck.twitter.com
XML Parsing Error: no root element found
Location: https://tweetdeck.twitter.com/metrics
Line Number 1, Column 1:
metrics:1:1
Invalid URI. Load of media resource  failed. tweetdeck.twitter.com
XML Parsing Error: no root element found
Location: https://tweetdeck.twitter.com/metrics
Line Number 1, Column 1:
metrics:1:1

stderr says:

(firefox:20026): GLib-GObject-WARNING **: /build/glib2.0-prJhLS/glib2.0-2.48.2/./gobject/gsignal.c:3486: signal name 'selection_changed' is invalid for instance '0x7f755dc6fe70' of type 'MaiAtkType13b'

(firefox:20026): GLib-GObject-WARNING **: /build/glib2.0-prJhLS/glib2.0-2.48.2/./gobject/gsignal.c:3486: signal name 'selection_changed' is invalid for instance '0x7f755dc6fe70' of type 'MaiAtkType13b'

(firefox:20026): GLib-GObject-WARNING **: /build/glib2.0-prJhLS/glib2.0-2.48.2/./gobject/gsignal.c:3486: signal name 'selection_changed' is invalid for instance '0x7f755dc6fe70' of type 'MaiAtkType13b'

(firefox:20026): GLib-GObject-WARNING **: /build/glib2.0-prJhLS/glib2.0-2.48.2/./gobject/gsignal.c:3486: signal name 'selection_changed' is invalid for instance '0x7f755dc6fe70' of type 'MaiAtkType13b'

(firefox:20026): GLib-GObject-WARNING **: /build/glib2.0-prJhLS/glib2.0-2.48.2/./gobject/gsignal.c:3486: signal name 'selection_changed' is invalid for instance '0x7f755dc6fe70' of type 'MaiAtkType13b'

(firefox:20026): GLib-GObject-WARNING **: /build/glib2.0-prJhLS/glib2.0-2.48.2/./gobject/gsignal.c:3486: signal name 'selection_changed' is invalid for instance '0x7f755dc6fe70' of type 'MaiAtkType13b'

(firefox:20026): GLib-GObject-WARNING **: /build/glib2.0-prJhLS/glib2.0-2.48.2/./gobject/gsignal.c:3486: signal name 'selection_changed' is invalid for instance '0x7f755dc6fe70' of type 'MaiAtkType13b'

(firefox:20026): GLib-GObject-WARNING **: /build/glib2.0-prJhLS/glib2.0-2.48.2/./gobject/gsignal.c:3486: signal name 'selection_changed' is invalid for instance '0x7f755dc6fe70' of type 'MaiAtkType13b'

(firefox:20026): GLib-GObject-WARNING **: /build/glib2.0-prJhLS/glib2.0-2.48.2/./gobject/gsignal.c:3486: signal name 'selection_changed' is invalid for instance '0x7f755dc6fe70' of type 'MaiAtkType13b'

(firefox:20026): GLib-GObject-WARNING **: /build/glib2.0-prJhLS/glib2.0-2.48.2/./gobject/gsignal.c:3486: signal name 'selection_changed' is invalid for instance '0x7f755dc6fe70' of type 'MaiAtkType13b'

(firefox:20026): GLib-GObject-WARNING **: /build/glib2.0-prJhLS/glib2.0-2.48.2/./gobject/gsignal.c:3486: signal name 'selection_changed' is invalid for instance '0x7f755dc6fe70' of type 'MaiAtkType13b'

(firefox:20026): GLib-GObject-WARNING **: /build/glib2.0-prJhLS/glib2.0-2.48.2/./gobject/gsignal.c:3486: signal name 'selection_changed' is invalid for instance '0x7f755dc6fe70' of type 'MaiAtkType13b'

(firefox:20026): GLib-GObject-WARNING **: /build/glib2.0-prJhLS/glib2.0-2.48.2/./gobject/gsignal.c:3486: signal name 'selection_changed' is invalid for instance '0x7f755dc6fe70' of type 'MaiAtkType13b'

(firefox:20026): GLib-GObject-WARNING **: /build/glib2.0-prJhLS/glib2.0-2.48.2/./gobject/gsignal.c:3486: signal name 'selection_changed' is invalid for instance '0x7f755dc6fe70' of type 'MaiAtkType13b'
[Parent 20026, Gecko_IOThread] WARNING: pipe error (58): Connection reset by peer: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 353

###!!! [Parent][MessageChannel] Error: (msgtype=0x16007F,name=PBrowser::Msg_Destroy) Channel error: cannot send/recv


###!!! [Parent][MessageChannel] Error: (msgtype=0x16007F,name=PBrowser::Msg_Destroy) Channel error: cannot send/recv


(firefox:20026): GLib-GObject-WARNING **: /build/glib2.0-prJhLS/glib2.0-2.48.2/./gobject/gsignal.c:3486: signal name 'load_complete' is invalid for instance '0x7f754e380bf0' of type 'MaiAtkType139'


Something I tried was turning on the gecko profiler.  It crashed pretty quickly after I did that.

Here's the crash: https://crash-stats.mozilla.com/report/index/fdbf7586-2c9f-45de-9da9-f786b0180502

Comment 15

[Paul Bone [:pbone]]

I now have non-profiler crashes.  I don't know...

Comment 16

[Matt Woodrow (:mattwoodrow)]

The crash report contains the crash reason I was hoping for, but still not enough to figure out what the problem is.

Trying even more assertions: https://treeherder.mozilla.org/#/jobs?repo=try&revision=aa1adf51a42ca069b742ee693e000a5e80c8ead2

Comment 17

[Paul Bone [:pbone]]

Here's a crash with that latest build: https://crash-stats.mozilla.com/report/index/035ad75a-5d09-4b07-8529-036350180503  I didn't get gdb attached in time for a stack trace, sorry.

Comment 18

[Paul Bone [:pbone]]

Seeing some of this from gdb, I've not seen this message before:

  warning: Corrupted shared library list: 0x7f6931017000 != 0x7f6955b96800

Anyway, looks like it crashed on one of your AssertUniqueItems assertions.

#0  0x00007f697249c4b9 in AssertUniqueItem(nsDisplayItem*) ()
   from /tmp/t/firefox/libxul.so
#1  0x00007f69724ebf7d in nsDisplayBackgroundImage::AppendBackgroundItemsToTop(nsDisplayListBuilder*, nsIFrame*, nsRect const&, nsDisplayList*, bool, mozilla::ComputedStyle*, nsRect const&, nsIFrame*) () from /tmp/t/firefox/libxul.so
#2  0x00007f69722c428d in nsFrame::DisplayBackgroundUnconditional(nsDisplayListBuilder*, nsDisplayListSet const&, bool) () from /tmp/t/firefox/libxul.so
#3  0x00007f69722d7733 in nsFrame::DisplayBorderBackgroundOutline(nsDisplayListBuilder*, nsDisplayListSet const&, bool) () from /tmp/t/firefox/libxul.so
#4  0x00007f697241b106 in nsBoxFrame::BuildDisplayList(nsDisplayListBuilder*, nsDisplayListSet const&) () from /tmp/t/firefox/libxul.so
#5  0x00007f69722fda13 in nsIFrame::BuildDisplayListForChild(nsDisplayListBuilder*, nsIFrame*, nsDisplayListSet const&, unsigned int) ()
   from /tmp/t/firefox/libxul.so
#6  0x00007f697240d894 in nsBoxFrame::BuildDisplayListForChildren(nsDisplayListBuilder*, nsDisplayListSet const&) () from /tmp/t/firefox/libxul.so
#7  0x00007f6972431d11 in nsSliderFrame::BuildDisplayListForChildren(nsDisplayListBuilder*, nsDisplayListSet const&) () from /tmp/t/firefox/libxul.so
#8  0x00007f697241b118 in nsBoxFrame::BuildDisplayList(nsDisplayListBuilder*, nsDisplayListSet const&) () from /tmp/t/firefox/libxul.so
#9  0x00007f69722fcbd0 in nsIFrame::BuildDisplayListForChild(nsDisplayListBuilder*, nsIFrame*, nsDisplayListSet const&, unsigned int) ()
   from /tmp/t/firefox/libxul.so
#10 0x00007f697240d894 in nsBoxFrame::BuildDisplayListForChildren(nsDisplayListBuilder*, nsDisplayListSet const&) () from /tmp/t/firefox/libxul.so
#11 0x00007f697241b118 in nsBoxFrame::BuildDisplayList(nsDisplayListBuilder*, nsDisplayListSet const&) () from /tmp/t/firefox/libxul.so
#12 0x00007f69722f7f2b in nsIFrame::BuildDisplayListForStackingContext(nsDisplayListBuilder*, nsDisplayList*, bool*) () from /tmp/t/firefox/libxul.so
#13 0x00007f69722fc955 in nsIFrame::BuildDisplayListForChild(nsDisplayListBuilder*, nsIFrame*, nsDisplayListSet const&, unsigned int) ()
   from /tmp/t/firefox/libxul.so
#14 0x00007f6972335e8b in mozilla::ScrollFrameHelper::AppendScrollPartsTo(nsDisplayListBuilder*, nsDisplayListSet const&, bool, bool) ()
   from /tmp/t/firefox/libxul.so
#15 0x00007f6972340d3c in mozilla::ScrollFrameHelper::BuildDisplayList(nsDisplayListBuilder*, nsDisplayListSet const&) () from /tmp/t/firefox/libxul.so
#16 0x00007f69722fd3ec in nsIFrame::BuildDisplayListForChild(nsDisplayListBuilder*, nsIFrame*, nsDisplayListSet const&, unsigned int) ()
   from /tmp/t/firefox/libxul.so
#17 0x00007f69722fe12c in DisplayLine(nsDisplayListBuilder*, nsRect const&, nsLineList_iterator&, int, int&, nsDisplayListSet const&, nsBlockFrame*, mozilla::css::TextOverflow*, unsigned int) [clone .isra.1012] ()
   from /tmp/t/firefox/libxul.so
#18 0x00007f6972305f61 in nsBlockFrame::BuildDisplayList(nsDisplayListBuilder*, nsDisplayListSet const&) () from /tmp/t/firefox/libxul.so

Comment 19

[Paul Bone [:pbone]]

In the debug build I get a different stack trace, because it hits an assertion that wasn't compiled-in before:

Here I think: https://searchfox.org/mozilla-central/source/layout/painting/nsDisplayList.cpp#3207

#0  0x00007f82a34842fb in nsDisplayItem::GetClipWithRespectToASR(nsDisplayListBuilder*, mozilla::ActiveScrolledRoot const*) const () from /tmp/t/firefox-dbg/libxul.so
#1  0x00007f82a34ad05a in nsDisplayList::GetClippedBoundsWithRespectToASR(nsDisplayListBuilder*, mozilla::ActiveScrolledRoot const*, nsRect*) const ()
   from /tmp/t/firefox-dbg/libxul.so
#2  0x00007f82a33a1b85 in nsDisplayWrapList::UpdateBounds(nsDisplayListBuilder*) () from /tmp/t/firefox-dbg/libxul.so
#3  0x00007f82a34ad480 in nsDisplayTransform::UpdateBounds(nsDisplayListBuilder*) () from /tmp/t/firefox-dbg/libxul.so
#4  0x00007f82a34a96a5 in MergeState::ProcessItemFromNewList(nsDisplayItem*, mozilla::Maybe<Index<MergedListUnits> > const&) () from /tmp/t/firefox-dbg/libxul.so
#5  0x00007f82a34a9319 in RetainedDisplayListBuilder::MergeDisplayLists(nsDisplayList*, RetainedDisplayList*, RetainedDisplayList*, mozilla::Maybe<mozilla::ActiveScrolledRoot const*>&) () from /tmp/t/firefox-dbg/libxul.so
#6  0x00007f82a34a9680 in MergeState::ProcessItemFromNewList(nsDisplayItem*, mozilla::Maybe<Index<MergedListUnits> > const&) () from /tmp/t/firefox-dbg/libxul.so
#7  0x00007f82a34a9319 in RetainedDisplayListBuilder::MergeDisplayLists(nsDisplayList*, RetainedDisplayList*, RetainedDisplayList*, mozilla::Maybe<mozilla::ActiveScrolledRoot const*>&) () from /tmp/t/firefox-dbg/libxul.so
#8  0x00007f82a34c422c in RetainedDisplayListBuilder::AttemptPartialUpdate(unsigned int, mozilla::DisplayListChecker*) () from /tmp/t/firefox-dbg/libxul.so
#9  0x00007f82a329789b in nsLayoutUtils::PaintFrame(gfxContext*, nsIFrame*, nsRegion const&, unsigned int, nsDisplayListBuilderMode, nsLayoutUtils::PaintFrameFlags) ()
   from /tmp/t/firefox-dbg/libxul.so
#10 0x00007f82a3252b04 in mozilla::PresShell::Paint(nsView*, nsRegion const&, unsigned int) () from /tmp/t/firefox-dbg/libxul.so
#11 0x00007f82a308d004 in nsViewManager::ProcessPendingUpdatesPaint(nsIWidget*) () from /tmp/t/firefox-dbg/libxul.so
#12 0x00007f82a308d23c in nsViewManager::ProcessPendingUpdatesForView(nsView*, bool) () from /tmp/t/firefox-dbg/libxul.so
#13 0x00007f82a308d2d1 in nsViewManager::ProcessPendingUpdates() () from /tmp/t/firefox-dbg/libxul.so
#14 0x00007f82a322db86 in nsRefreshDriver::Tick(long, mozilla::TimeStamp) () from /tmp/t/firefox-dbg/libxul.so

Comment 21

[Ryan VanderMeulen [:RyanVM]]

Assigning this to Matt since it's a P1, though I recognize that we're still trying to figure out exactly what's broken here :)

Comment 22

[Matt Woodrow (:mattwoodrow)]

Thanks Paul! I have a possible lead based on that information, this try build attempts to fix it:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=1383c43cdd67520bc094c7f37a48fd850eaedb79

Comment 23

[Paul Bone [:pbone]]

Based on what you've learnt, do you think there's anything I can do to try to provoke it into crashing?  Otherwise I plan to just leave it open all day, if it doesn't crash all day then I think we're good.  Cheers.

Comment 24

[Paul Bone [:pbone]]

That looks like its fixed it, no crashes all day long.  I'd like to confirm by seeing the rate in crash stats drop off once this is in nightly also.

Thanks.

Comment 25

[Matt Woodrow (:mattwoodrow)]

Attachment: Bug 1456534 - Make sure we do a full display list rebuild on the next frame after creating a displayport during painting.

Review commit: https://reviewboard.mozilla.org/r/241706/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/241706/

Comment 26

[Markus Stange [:mstange]]

Comment on attachment 8973218 [details]
Bug 1456534 - Make sure we do a full display list rebuild on the next frame after creating a displayport during painting.

https://reviewboard.mozilla.org/r/241706/#review247598

Comment 27

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2fe7f347fbbd
Make sure we do a full display list rebuild on the next frame after creating a displayport during painting. r=mstange

Comment 28

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/2fe7f347fbbd

Comment 29

[Paul Bone [:pbone]]

Thank you.

Comment 30

[Ryan VanderMeulen [:RyanVM]]

Looks like this patch didn't completely eliminate the crashes, but it certainly dropped the frequency pretty significantly. I've filed bug 1459997 to track the remaining crashes.