1430258 - Intermittent awsy\test_memory_usage.py TestMemoryUsage.test_open_tabs | application crashed [@ SubDocEnumCb]

Status: RESOLVED FIXED

(Core :: Web Painting, defect, P2)

Description

[Treeherder Bug Filer]

Filed by: ncsoregi [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=155995312&repo=mozilla-inbound

https://queue.taskcluster.net/v1/task/e-ckT1GtTeepm0Vo4I-eQA/runs/0/artifacts/public/logs/live_backing.log

19:20:21     INFO - mozcrash Saved app info as Z:\task_1515782112\build\blobber_upload_dir\010d2c71-f779-4747-aaf2-d6f13c562fe3.extra
2073
19:20:21     INFO - PROCESS-CRASH | awsy\test_memory_usage.py TestMemoryUsage.test_open_tabs | application crashed [@ SubDocEnumCb]
2074
19:20:21     INFO - Crash dump filename: c:\users\genericworker\appdata\local\temp\tmp5nmzie\minidumps\010d2c71-f779-4747-aaf2-d6f13c562fe3.dmp

Comment 1

[Intermittent Failures Robot]

14 failures in 788 pushes (0.018 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 6
* mozilla-central: 6
* autoland: 2

Platform breakdown:
* windows7-32: 11
* windows10-64-ccov: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1430258&startday=2018-01-08&endday=2018-01-14&tree=all

Comment 2

[Andrei Ciure[:aciure]]

There were 35 failures in the last 7 days related to this bug. This seems to affect Windows platform, mainly Windows 7 all build types.

:bc can you please take a look at this?

Comment 3

[Bob Clary [:bc] (inactive)]

A Browser crash is not an issue with the framework. Moving to Core: Layout: Web Painting. Dupe of bug 1429078 ?

Comment 4

[Intermittent Failures Robot]

39 failures in 657 pushes (0.059 failures/push) were associated with this bug in the last 7 days. 

This is the #18 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* mozilla-inbound: 15
* mozilla-central: 15
* autoland: 7
* try: 2

Platform breakdown:
* windows7-32: 34
* windows7-32-nightly: 3
* windows10-64: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1430258&startday=2018-01-15&endday=2018-01-21&tree=all

Comment 5

[Bob Clary [:bc] (inactive)]

This appears to have first appeared in <https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=6575fa09e55f5405a69b7c64c050174cd0536922&selectedJob=155006558> when the patch for Bug 1404392 landed. Bug 1429375 was filed but this error was missed.

See bug 1429078 comment 0 that bug 1418840 might be the cause. Matt: Is this something you could look into?

Comment 6

[Miko Mynttinen]

Most of the newer crashes seem to happen when accessing the anonymous view between the subdoc view and the root view [1].

[1]: https://searchfox.org/mozilla-central/rev/e9a067a401e41017766f4ab90bc3906603273d51/layout/painting/RetainedDisplayListBuilder.cpp#607

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Here the stack from one of the latest awsy test jobs. Most of the crashes are happening on Windows 7 32bit. The 64bit awsy jobs don't show this crash.

23:45:25     INFO - Crash reason:  EXCEPTION_ACCESS_VIOLATION_READ
23:45:25     INFO - Crash address: 0x8
23:45:25     INFO - Assertion: Unknown assertion type 0x00000000
23:45:25     INFO - Process uptime: 1066 seconds
23:45:25     INFO - 
23:45:25     INFO - Thread 0 (crashed)
23:45:25     INFO -  0  xul.dll!SubDocEnumCb [RetainedDisplayListBuilder.cpp:2d21df989ee6 : 607 + 0x9]
23:45:25     INFO -     eip = 0x5df5a4d1   esp = 0x002882a0   ebp = 0x002882ac   ebx = 0x0028832c
23:45:25     INFO -     esi = 0x00000000   edi = 0x1bdc3000   eax = 0x00000000   ecx = 0x1bdc3000
23:45:25     INFO -     edx = 0x0fb00a78   efl = 0x00010206
23:45:25     INFO -     Found by: given as instruction pointer in context
23:45:25     INFO -  1  xul.dll!nsDocument::EnumerateSubDocuments(bool (*)(nsIDocument *,void *),void *) [nsDocument.cpp:2d21df989ee6 : 8583 + 0x7]
23:45:25     INFO -     eip = 0x5d3d5004   esp = 0x002882b4   ebp = 0x00288314
23:45:25     INFO -     Found by: call frame info
23:45:25     INFO -  2  xul.dll!GetModifiedFrames [RetainedDisplayListBuilder.cpp:2d21df989ee6 : 638 + 0x11]
23:45:25     INFO -     eip = 0x5df47c34   esp = 0x0028831c   ebp = 0x00288334
23:45:25     INFO -     Found by: call frame info
23:45:25     INFO -  3  xul.dll!RetainedDisplayListBuilder::AttemptPartialUpdate(unsigned int) [RetainedDisplayListBuilder.cpp:2d21df989ee6 : 965 + 0xa]
23:45:25     INFO -     eip = 0x5df30181   esp = 0x0028833c   ebp = 0x002883b4
23:45:25     INFO -     Found by: call frame info
23:45:25     INFO -  4  xul.dll!nsLayoutUtils::PaintFrame(gfxContext *,nsIFrame *,nsRegion const &,unsigned int,nsDisplayListBuilderMode,nsLayoutUtils::PaintFrameFlags) [nsLayoutUtils.cpp:2d21df989ee6 : 3836 + 0xe]
23:45:25     INFO -     eip = 0x5ddfee6c   esp = 0x002883bc   ebp = 0x00289d00
23:45:25     INFO -     Found by: call frame info
23:45:25     INFO -  5  xul.dll!mozilla::PresShell::Paint(nsView *,nsRegion const &,unsigned int) [PresShell.cpp:2d21df989ee6 : 6481 + 0x1c]
23:45:25     INFO -     eip = 0x5ddd3ec0   esp = 0x00289d08   ebp = 0x00289e30
23:45:25     INFO -     Found by: previous frame's frame pointer
23:45:25     INFO -  6  xul.dll!mozilla::PresShell::Paint(nsView *,nsRegion const &,unsigned int) [PresShell.cpp:2d21df989ee6 : 6481 + 0x1c]
23:45:25     INFO -     eip = 0x5ddd3ec0   esp = 0x00289d18   ebp = 0x00289e30
23:45:25     INFO -     Found by: stack scanning
23:45:25     INFO -  7  mozglue.dll!BaseAllocator::free(void *) [mozjemalloc.cpp:2d21df989ee6 : 4233 + 0x9]
23:45:25     INFO -     eip = 0x7422798d   esp = 0x00289d3c   ebp = 0x00289e30
23:45:25     INFO -     Found by: stack scanning
23:45:25     INFO -  8  xul.dll!mozilla::dom::TabChild::RecvRenderLayers(bool const &,unsigned __int64 const &) [TabChild.cpp:2d21df989ee6 : 2715 + 0x34]
23:45:25     INFO -     eip = 0x5db4d1c7   esp = 0x00289e38   ebp = 0x00289eb4
23:45:25     INFO -     Found by: call frame info
23:45:25     INFO -  9  xul.dll!mozilla::dom::TabChild::ForcePaint(unsigned __int64) [TabChild.cpp:2d21df989ee6 : 3519 + 0x18]
23:45:25     INFO -     eip = 0x5db4189a   esp = 0x00289ebc   ebp = 0x00289ed0
23:45:25     INFO -     Found by: call frame info
23:45:25     INFO - 10  xul.dll!`anonymous namespace'::HangMonitorChild::InterruptCallback [ProcessHangMonitor.cpp:2d21df989ee6 : 346 + 0xd]
23:45:25     INFO -     eip = 0x5db376b0   esp = 0x00289ed8   ebp = 0x00289f08
23:45:25     INFO -     Found by: call frame info
23:45:25     INFO - 11  xul.dll!InterruptCallback [ProcessHangMonitor.cpp:2d21df989ee6 : 1143 + 0x5]
23:45:25     INFO -     eip = 0x5db3760f   esp = 0x00289f10   ebp = 0x0028a140
23:45:25     INFO -     Found by: call frame info
23:45:25     INFO - 12  xul.dll!InvokeInterruptCallback [Runtime.cpp:2d21df989ee6 : 534 + 0x3]
23:45:25     INFO -     eip = 0x5eaf61f5   esp = 0x00289f14   ebp = 0x0028a140
23:45:25     INFO -     Found by: call frame info
23:45:25     INFO - 13  xul.dll!js::GetProperty(JSContext *,JS::Handle<JSObject *>,JS::Handle<JS::Value>,JS::Handle<jsid>,JS::MutableHandle<JS::Value>) [NativeObject.h:2d21df989ee6 : 1619 + 0xf]
23:45:25     INFO -     eip = 0x5ea09799   esp = 0x00289f78   ebp = 0x00289fd4
23:45:25     INFO -     Found by: stack scanning
23:45:25     INFO - 14  xul.dll + 0x2ff99bc
23:45:25     INFO -     eip = 0x5f5299bc   esp = 0x00289fdc   ebp = 0x1b1f30d0
23:45:25     INFO -     Found by: call frame info
23:45:25     INFO - 15  xul.dll!js::GetProperty(JSContext *,JS::Handle<JSObject *>,JS::Handle<JS::Value>,JS::Handle<jsid>,JS::MutableHandle<JS::Value>) [NativeObject.h:2d21df989ee6 : 1619 + 0xf]
23:45:25     INFO -     eip = 0x5ea09799   esp = 0x00289ff8   ebp = 0x0028a014
23:45:25     INFO -     Found by: stack scanning
23:45:25     INFO - 16  xul.dll!js::Proxy::get(JSContext *,JS::Handle<JSObject *>,JS::Handle<JS::Value>,JS::Handle<jsid>,JS::MutableHandle<JS::Value>) [Proxy.cpp:2d21df989ee6 : 362 + 0x18]
23:45:25     INFO -     eip = 0x5eb05508   esp = 0x0028a01c   ebp = 0x0028a070
23:45:25     INFO -     Found by: call frame info
23:45:25     INFO - 17  xul.dll!js::GetProperty(JSContext *,JS::Handle<JSObject *>,JS::Handle<JS::Value>,js::PropertyName *,JS::MutableHandle<JS::Value>) [jsobj.h:2d21df989ee6 : 804 + 0x22]
23:45:25     INFO -     eip = 0x5ec277a5   esp = 0x0028a078   ebp = 0x0028a0a0
23:45:25     INFO -     Found by: stack scanning
23:45:25     INFO - 18  xul.dll!GetPropertyOperation [Interpreter.cpp:2d21df989ee6 : 219 + 0x19]
23:45:25     INFO -     eip = 0x5ec27b84   esp = 0x0028a0f8   ebp = 0x0028a118
23:45:25     INFO -     Found by: call frame info with scanning
23:45:25     INFO - 19  xul.dll!GetPropertyOperation [Interpreter.cpp:2d21df989ee6 : 219 + 0x19]
23:45:25     INFO -     eip = 0x5ec27b84   esp = 0x0028a100   ebp = 0x0028a118
23:45:25     INFO -     Found by: stack scanning
23:45:25     INFO - 20  xul.dll!js::LooselyEqual(JSContext *,JS::Handle<JS::Value>,JS::Handle<JS::Value>,bool *) [Interpreter.cpp:2d21df989ee6 : 837 + 0xd]
23:45:25     INFO -     eip = 0x5ec3103a   esp = 0x0028a110   ebp = 0x0028aaaa
23:45:25     INFO -     Found by: stack scanning
23:45:25     INFO - 21  xul.dll!JSContext::handleInterrupt() [Runtime.cpp:2d21df989ee6 : 613 + 0x6]
23:45:25     INFO -     eip = 0x5eafe82b   esp = 0x0028a148   ebp = 0x0028aaaa
23:45:25     INFO -     Found by: stack scanning
23:45:25     INFO - 22  xul.dll!Interpret [Interpreter.cpp:2d21df989ee6 : 2133 + 0x21]
23:45:25     INFO -     eip = 0x5ec302ec   esp = 0x0028a160   ebp = 0x0028aaaa
23:45:25     INFO -     Found by: call frame info
23:45:25     INFO - 23  xul.dll!CompareLayers [nsFrame.cpp:2d21df989ee6 : 888 + 0x8]
23:45:25     INFO -     eip = 0x5de29245   esp = 0x0028a174   ebp = 0x0028ad00
23:45:25     INFO -     Found by: stack scanning
23:45:25     INFO - 24  xul.dll!nsIFrame::MarkNeedsDisplayItemRebuild() [nsFrame.cpp:2d21df989ee6 : 1047 + 0x4]
23:45:25     INFO -     eip = 0x5de3e001   esp = 0x0028a1c0   ebp = 0x0028ad48
23:45:25     INFO -     Found by: call frame info with scanning
23:45:25     INFO - 25  xul.dll!nsIFrame::MarkNeedsDisplayItemRebuild() [nsFrame.cpp:2d21df989ee6 : 1047 + 0x4]
23:45:25     INFO -     eip = 0x5de3e001   esp = 0x0028a200   ebp = 0x0028ad7c
23:45:25     INFO -     Found by: call frame info with scanning
23:45:25     INFO - 26  xul.dll!nsBlockFrame::nsBlockFrame(nsStyleContext *,nsQueryFrame::ClassID) [nsBlockFrame.h:2d21df989ee6 : 423 + 0x14]
23:45:25     INFO -     eip = 0x5de15d00   esp = 0x0028a238   ebp = 0x0677287f
23:45:25     INFO -     Found by: call frame info with scanning
23:45:25     INFO - 27  mozglue.dll!arena_t::MallocSmall(unsigned int,bool) [mozjemalloc.cpp:2d21df989ee6 : 2952 + 0x6]
23:45:25     INFO -     eip = 0x74224c9b   esp = 0x0028a240   ebp = 0x0677287f
23:45:25     INFO -     Found by: stack scanning
23:45:25     INFO - 28  xul.dll!PLDHashTable::Add(void const *,std::nothrow_t const &) [PLDHashTable.cpp:2d21df989ee6 : 589 + 0x6]
23:45:25     INFO -     eip = 0x5cb89f07   esp = 0x0028a24c   ebp = 0x0677287f
23:45:25     INFO -     Found by: stack scanning
23:45:25     INFO - 29  mozglue.dll!RedBlackTree<arena_chunk_map_t,ArenaRunTreeTrait>::Insert(RedBlackTree<arena_chunk_map_t,ArenaRunTreeTrait>::TreeNode *) [rb.h:2d21df989ee6 : 365 + 0x5]
23:45:25     INFO -     eip = 0x7422442f   esp = 0x0028a278   ebp = 0x0028a294
23:45:25     INFO -     Found by: stack scanning

Comment 8

[Intermittent Failures Robot]

15 failures in 148 pushes (0.101 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* mozilla-inbound: 6
* autoland: 5
* mozilla-central: 4

Platform breakdown:
* windows7-32: 15

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1430258&startday=2018-01-23&endday=2018-01-23&tree=all

Comment 9

[Matt Woodrow (:mattwoodrow)]

Attachment: fix-subdoc

Made all the null checks fallible, since clearly we can't rely on things being sane.

Note that this also makes sure we recurse into the right subdoc.

Comment 10

[Miko Mynttinen]

Comment on attachment 8945105 [details] [diff] [review]
fix-subdoc

Review of attachment 8945105 [details] [diff] [review]:
-----------------------------------------------------------------

::: layout/painting/RetainedDisplayListBuilder.cpp
@@ +578,5 @@
>    nsDisplayListBuilder* builder;
>    nsTArray<nsIFrame*> modifiedFrames;
>  };
>  
> +nsIFrame* GetRootFrameForPainting(nsDisplayListBuilder* aBuilder, nsIDocument* aDocument)

For consistency, could you please use:
static nsIFrame*
GetRootFrameForPainting

Comment 11

[Liz Henry (:lizzard) (relman/hg->git project)]

Let me know if you want to uplift this to beta once it lands on m-c.

Comment 12

[Intermittent Failures Robot]

58 failures in 701 pushes (0.083 failures/push) were associated with this bug in the last 7 days. 

This is the #12 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* autoland: 22
* mozilla-inbound: 18
* mozilla-central: 12
* try: 6

Platform breakdown:
* windows7-32: 54
* windows10-64-ccov: 3
* windows7-32-nightly: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1430258&startday=2018-01-22&endday=2018-01-28&tree=all

Comment 15

[Intermittent Failures Robot]

6 failures in 735 pushes (0.008 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 4
* mozilla-inbound: 1
* mozilla-central: 1

Platform breakdown:
* windows7-32: 6

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1430258&startday=2018-01-29&endday=2018-02-04&tree=all

Comment 16

[Matt Woodrow (:mattwoodrow)]

Looks like I accidentally put the wrong bug number on the commit message.

This landed as https://hg.mozilla.org/integration/mozilla-inbound/rev/fa67d0e45eaa

Comment 17

[Ryan VanderMeulen [:RyanVM]]

Is this worth consideration for backporting to 59 as well? Note that it'll need a rebased patch if the answer is yes.

Comment 18

[Julien Cristau [:jcristau]]

This crash signature seems to be nightly only?