1191465 - Plugin Container crash Nightly: [@ SweepArenaList<T> ]

Status: VERIFIED FIXED

(Core :: JavaScript: GC, defect, critical)

Description

[Semtex]

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:42.0) Gecko/20100101 Firefox/42.0
Build ID: 20150805030208

Steps to reproduce:

There is no clear STR, simply since today Nightly crash with this sig:

https://crash-stats.mozilla.com/report/index/ffb4b3f7-b15a-424d-91e3-fda002150805

or this

https://crash-stats.mozilla.com/report/index/0855cc6d-5161-4674-bb3b-721642150805

Only if there is some Flash content, it is very random, Nightly works and suddenly crash, when crash report window is open then Windows inform about Plugin container crash. 

Problem occur with Nightly cset: https://hg.mozilla.org/mozilla-central/rev/f3b757156f69

Comment 1

[Jim Jeffery not reading bug-mail 1/2/11]

I just got same crash sig https://crash-stats.mozilla.com/report/index/bp-54e026e0-7c5f-4297-8323-965e22150805

Today's Nightly 
Flash 18.0.0.209 
Win7 x64 
m-c 32bit build
non-e10s enabled

Setting to NEW

Comment 2

[Emanuel Hoogeveen [:ehoogeveen]]

Given the signature, obviously GC-related.

Comment 3

[IU]

Attachment: WinDbg stack trace

Comment 4

[WildcatRay]

https://crash-stats.mozilla.com/report/index/bp-42e8645b-836f-4075-99ed-b699e2150805

Comment 6

[Andrew McCreight [:mccr8]]

FYI, we may get a huge spike in GC crashes on Nightly. I've seen a bunch of similar bugs filed, and people who get it are crashing a lot.

Comment 7

[Terrence Cole [:terrence]]

The only GC related patches that landed in the 8/5 nightly are: 
	5ba973a43c12	Terrence Cole — Bug 1189906 - Remove the unused UseSavedRoots enum; r=jonco
	027800a23ccb	Terrence Cole — Bug 1189112 - Part 2: simplify rooting of ScriptsAndCountsVector with PersistentRooted; r=nbp
	abc018892155	Terrence Cole — Bug 1189112 - Part 1: Use TraceableVector to simplify tracing of ScriptsAndCountsVector; r=nbp
	d1288e84b4a0	Terrence Cole — Bug 1189072 - Make DefaultTracer for struct types call T::trace; r=fitzgen
	d6dea3334b6c	Terrence Cole — Bug 1188620 - Use PersistentRooted for asyncActivation roots; r=fitzgen
	0aed5c00a735	Terrence Cole — Bug 1188445 - Allow PersistentRooted to store StaticTraceable; r=sfink
	b05f6533036e	Terrence Cole — Bug 1189809 - Remove the ill-fated DynamicTraceable; r=jonco

Comment 8

[:dmajor]

Crash-stats definitely points to the 8/5 nightly.

Comment 10

[Terrence Cole [:terrence]]

I've backed out:
	027800a23ccb	Terrence Cole — Bug 1189112 - Part 2: simplify rooting of ScriptsAndCountsVector with PersistentRooted; r=nbp
	abc018892155	Terrence Cole — Bug 1189112 - Part 1: Use TraceableVector to simplify tracing of ScriptsAndCountsVector; r=nbp
	d6dea3334b6c	Terrence Cole — Bug 1188620 - Use PersistentRooted for asyncActivation roots; r=fitzgen

These are as likely as anything in there to be at fault and I haven't landed anything on top of them yet. Hopefully tomorrows nightly will tell us more.

Comment 11

[Terrence Cole [:terrence]]

Thanks, bugzilla!

Comment 12

[IU]

Does the stack trace I attached help at all?

Comment 13

[IU]

I think this is the relevant excerpt:

STACK_TEXT:  
004beeb4 5d6b4444 004bef14 004bf078 09151200 xul!SweepArenaList<JSScript,js::AutoClearTypeInferenceStateOnOOM *>+0x33c
004bef5c 5d6b5d2b 004bf078 004bf078 09151200 xul!js::gc::GCRuntime::sweepPhase+0xa5
004bef80 5d6b577c 004bf078 00000003 0000002f xul!js::gc::GCRuntime::incrementalCollectSlice+0x12b
004befd4 5d6b58e2 00000001 004bf078 0000002f xul!js::gc::GCRuntime::gcCycle+0xf2
004bf068 5d6b5636 00000001 0000002f 00000028 xul!js::gc::GCRuntime::collect+0xe0
004bf0c8 5da7d41e 0000002f 00000028 00000000 xul!js::gc::GCRuntime::gcSlice+0x39
004bf0e8 5da7d3a4 00000001 00000028 00000000 xul!nsJSContext::GarbageCollectNow+0x76
004bf0f8 5d74c9ee 0c4fa320 00000000 096b4be4 xul!InterSliceGCTimerFired+0x15
004bf1c8 5db32e3f 14c8add0 00003c8c 00000000 xul!nsTimerImpl::Fire+0xf7
004bf1fc 5d752606 14c8add0 02346590 02346580 xul!nsTimerEvent::Run+0x37
004bf2f4 5d7514d1 02343160 00000000 004bf327 xul!nsThread::ProcessNextEvent+0x2da
004bf328 5d74fb63 023560c0 d673e9db 096b4be0 xul!mozilla::ipc::MessagePump::Run+0x57
004bf360 5d74f9a4 02343160 00000001 5da9ae00 xul!MessageLoop::RunHandler+0x20
004bf380 5d751b42 09699440 00000000 004bf3a0 xul!MessageLoop::Run+0x19
004bf390 5d751d49 096b4be0 09699440 004bf3b4 xul!nsBaseAppShell::Run+0x34
004bf3a0 5da7fb94 096b4be0 004bf6f1 0c704e80 xul!nsAppShell::Run+0x1d
004bf3b4 5db390db 09699440 004bf5f8 004bf614 xul!nsAppStartup::Run+0x22
004bf588 5db39300 00000003 004bf720 005e7268 xul!XREMain::XRE_mainRun+0x503
004bf5a4 5dbed944 00000000 0234d100 004bf720 xul!XREMain::XRE_main+0x1a7
004bf6f8 000f16d9 00000003 005e7268 004bf720 xul!XRE_main+0x35
004bf894 000f1322 0231c100 005e7370 005e7268 firefox!do_main+0x159
004bf93c 000f10de 000f257b 000f257b 00000000 firefox!NS_internal_main+0x122
004bf950 000f24fe 005e7268 ffffc100 005e7770 firefox!wmain+0xbe
004bf998 744b3744 ff2ea000 744b3720 5eb4415d firefox!__tmainCRTStartup+0xfe
004bf9ac 7724a064 ff2ea000 5dce6eba 00000000 KERNEL32!BaseThreadInitThunk+0x24
004bf9f4 7724a02f ffffffff 7726d7b8 00000000 ntdll!__RtlUserThreadStart+0x2f
004bfa04 00000000 000f257b ff2ea000 00000000 ntdll!_RtlUserThreadStart+0x1b


STACK_COMMAND:  .cxr 0x0 ; kb

FAULTING_SOURCE_LINE:  c:\builds\moz2_slave\m-cen-w32-ntly-000000000000000\build\src\js\src\jsgc.cpp

FAULTING_SOURCE_FILE:  c:\builds\moz2_slave\m-cen-w32-ntly-000000000000000\build\src\js\src\jsgc.cpp

FAULTING_SOURCE_LINE_NUMBER:  5193

SYMBOL_STACK_INDEX:  0

SYMBOL_NAME:  xul!SweepArenaList<JSScript,js::AutoClearTypeInferenceStateOnOOM *>+33c

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: xul

IMAGE_NAME:  xul.dll

DEBUG_FLR_IMAGE_TIMESTAMP:  55c212ee

FAILURE_BUCKET_ID:  NULL_CLASS_PTR_READ_CALL_c0000005_xul.dll!SweepArenaList_JSScript,js::AutoClearTypeInferenceStateOnOOM_*_

BUCKET_ID:  APPLICATION_FAULT_NULL_CLASS_PTR_READ_NULL_POINTER_READ_CALL_xul!SweepArenaList_JSScript,js::AutoClearTypeInferenceStateOnOOM_*_+33c

ANALYSIS_SOURCE:  UM

FAILURE_ID_HASH_STRING:  um:null_class_ptr_read_call_c0000005_xul.dll!sweeparenalist_jsscript,js::autocleartypeinferencestateonoom_*_

FAILURE_ID_HASH:  {e7cda7bb-f367-77b3-a938-0629ff8c301c}

Followup: MachineOwner
---------

Comment 14

[User Dderss]

I have a new crash report https://crash-stats.mozilla.com/report/index/1af2a990-e0ce-4a3c-9ba8-a670c2150806 with similar signature:

Frame 0: Module: xul.dll -- Signature: SweepArenaList<JSScript, js::AutoClearTypeInferenceStateOnOOM* __ptr64> -- source: js/src/jsgc.cpp

Frame 1: Module: xul.dll -- Signature: js::gc::GCRuntime::sweepPhase(js::SliceBudget&) -- source: js/src/jsgc.cpp

...

-- is it really this bug or it should be filed separately?

Comment 15

[Andrew McCreight [:mccr8]]

(In reply to User Dderss from comment #14)
> -- is it really this bug or it should be filed separately?

It is likely to be the same thing, given that you are on the 08-05 Nightly. This crash seems to be happening to a lot of people.

Comment 16

[User Dderss]

I have a new crash that is connected to #bug 1191409, which is said to be duplicate of this bug:

https://crash-stats.mozilla.com/report/index/7d6859aa-36f2-4bbf-9e7e-273f12150806

Crash signature in this case is not among mentioned above, though: 

Frame: 0 	nsCOMPtr<nsIContent>::operator=(nsIContent*) in xpcom/glue/nsCOMPtr.h

Frame: 1 	nsGlobalWindow::CleanUp() in dom/base/nsGlobalWindow.cpp

Comment 17

[Robert Kaiser]

I'm on vacation, dmajor is looking at the data this week and he commented here already anyhow.

Comment 18

[:dmajor]

Terrence: Bug 1191400 comment 0 is likely the same issue and has reliable STR.

Comment 20

[Andrew McCreight [:mccr8]]

Somewhere in the dupe chain somebody said that this URL reproduces the crash.

Comment 22

[IU]

This crash is still happening: bp-76eb8931-9aec-44d3-808f-a65c72150809

I attached a WinDbg log to this report and even posted comment 13, which is an excerpt suggesting where the crash is being triggered, and yet no one is looking into it.

Clearly the backing out bugs 1189112 and 1188620 served only to decrease the frequency of crashes and has resolved nothing.

Comment 23

[Virtual_ManPL [:Virtual] 🇵🇱 - (please needinfo? me - so I will see your comment/reply/question/etc.)]

[Tracking Requested - why for this release]: Regression and TOP1 crash for Firefox c42 now

Comment 24

[User Dderss]

I have a JS crash https://crash-stats.mozilla.com/report/index/b71c935a-aaff-4598-ad91-ccbc12150809 with different signature:

https://crash-stats.mozilla.com/report/index/b71c935a-aaff-4598-ad91-ccbc12150809

js::ObjectValueMap::findZoneEdges()

js::gc::GCRuntime::findZoneEdgesForWeakMaps()

Is it the same thing or should be filed as different bug?

By the way, the crash has happened without me doing anything with the browser. It has just died on its on during processing regular JS timer/interval functions.

Comment 25

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

It looks like this appeared in the 8/5 nightly, disappeared in 8/7, and then re-appeared in 8/8. Here's the data I'm looking at:

https://crash-stats.mozilla.com/search/?product=Firefox&signature=~SweepArenaList%3CT%3E&build_id=%3E%3D20150804030000&version=42.0a1&_facets=signature&_facets=build_id&_columns=date&_columns=signature&_columns=build_id&_columns=product&_columns=version&_columns=platform#facet-build_id

These are the ranges:

Original regression:
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=5b54831761b1&tochange=f3b757156f69

Fix:
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=07befc6f54e7&tochange=2fad09a0540d

Re-regression:
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=91de9c670800&tochange=943b79d9c65f

Given these ranges, it seems almost certain that bug 1181908 is the actual regressor. I'm going to back it out again.

Comment 26

[David Baron :dbaron: 🏴󠁵󠁳󠁣󠁡󠁿 ⌚UTC-7]

Pulling that backout (and nothing else) into my build fixed a reliable crash on google docs that I was seeing (matching the reports in bug 1191259 / bug 1191788).

Comment 31

[:dmajor]

At this point I'm convinced that we've identified the regressing change. Based on yesterday's backout this should be fixed starting in the August 10 nightly builds.

Comment 32

[Sylvestre Ledru [:sylvestre]]

Tracking in case we reopen it

Comment 34

[Tracy Walker [:tracy]]

Crash stats verifies this has been fixed. I looked through all of the signatures. There are no notable volume crashes since this was resolved. A dozen of the signatures associated with this had one or 2 crashes on a build from the 10th and/or 11th.

Comment 37

[Chris Peterson [:cpeterson]]

@ Terrence, I can still reproduce this crash with Nightly 43 build 2015-08-18.

STR:
1. Load http://www.dash-player.com/demo/manifest-test/
2. Paste the following manifest URL into the video player: https://mozillalive-a.akamaihd.net/Akamai_Public/ngrp:Nomad-1_all/manifest.mpd
3. Press the LOAD button and then the play button.

RESULT:
CRASH!

bp-25b34bfc-3769-4a2a-8200-638af2150810
bp-b111c394-343d-40b6-8e71-f9fd02150810

Comment 38

[Chris Peterson [:cpeterson]]

Never mind. My crash reports in comment 37 were old reports from before this bug was fixed.

Comment 40

[Common User Network Terminal]

(In reply to Chris Peterson [:cpeterson] from comment #38)
> Never mind. My crash reports in comment 37 were old reports from before this
> bug was fixed.

It isn't fixed. 

https://crash-stats.mozilla.com/report/list?product=Firefox&signature=SweepArenaList%3CT%3E#tab-reports

It's been a bug since v38.0.1 and it still isn't fixed.

Comment 41

[Milan Sreckovic [:milan] (needinfo for best results)]

There certainly are crashes with that signature - are these (or should they be) tracked in another bug?

Comment 43

[Andrew McCreight [:mccr8]]

[@ SweepArenaList<T> ] is a generic GC crash signature that can have a variety of causes. This particular bug is about a particular source of this, so it should remain closed. If somebody wants to have an open bug for the current crashes with this signature, please file a new bug. They are likely caused by a separate underlying issue.