1275172 - Crash in MustSkipMarking<T>

Status: RESOLVED WORKSFORME

(Core :: JavaScript: GC, defect, P5)

Description

[Ting-Yu Chou [:ting] (away)]

This bug was filed from the Socorro interface and is 
report bp-49fb5e98-1f2e-4a76-946a-de9942160523.
=============================================================

This is #12 of Nightly 20160522030240, there are 4 crashes from 2 installations.

https://crash-stats.mozilla.com/report/index/4233af46-56f9-462b-8ed2-8e1982160523
https://crash-stats.mozilla.com/report/index/1d4243af-41d0-4514-a3f1-81b242160523
https://crash-stats.mozilla.com/report/index/e6483136-2d86-4d28-bef8-2e4822160523

Comment 1

[Nicholas Nethercote [inactive]]

Terrence, any ideas?

Comment 2

[Terrence Cole [:terrence]]

There are actually 3 unrelated crashes here, but whatever.

The crash at https://crash-stats.mozilla.com/report/index/4233af46-56f9-462b-8ed2-8e1982160523 is literally impossible. It's crashing on a nullptr where the line above it is a check for nullptr. The crash at https://crash-stats.mozilla.com/report/index/1d4243af-41d0-4514-a3f1-81b242160523 is a crash at 0x0000000100000000, which dollars to cents is a single-bit-flip error on a nullptr. The crash at https://crash-stats.mozilla.com/report/index/49fb5e98-1f2e-4a76-946a-de9942160523 is marking an interned atom, which by definition is not dead. There appears to just be garbage where a pointer should be. Normally, I'd expect this to be heap corruption, but given the other crashes here, I'm more likely to blame hardware. And indeed, we can see that both of these reporters are using AMD64 family 6 model 60 stepping 3 | 4 CPUs.

So, I'm not sure what we can do here.

Comment 3

[Nicholas Nethercote [inactive]]

Ok, thanks for looking.

Comment 4

[BugBot [:suhaib / :marco/ :calixte]]

Crash volume for signature 'MustSkipMarking<T>':
 - nightly (version 50): 66 crashes from 2016-06-06.
 - aurora  (version 49): 134 crashes from 2016-06-07.
 - beta    (version 48): 170 crashes from 2016-06-06.
 - release (version 47): 492 crashes from 2016-05-31.
 - esr     (version 45): 44 crashes from 2016-04-07.

Crash volume on the last weeks:
             Week N-1   Week N-2   Week N-3   Week N-4   Week N-5   Week N-6   Week N-7
 - nightly          5         10         16          9          6          9          9
 - aurora          34         16         22         14         21         23          2
 - beta            48         16         13         40         22         16          4
 - release         49         64         62         54         99         97         41
 - esr              2          1          0          0          3          1          7

Affected platforms: Windows, Linux

Comment 5

[[:philipp]]

[Tracking Requested - why for this release]:
the volume of this crash is starkly rising in 49 beta builds and currently making up around 0.50% of browser crashes there. graph: http://bit.ly/2aDMFeU

Comment 6

[Liz Henry (:lizzard) (relman/hg->git project)]

Tracking since this looks like a problem in early beta 49. Naveed can you help find someone to investigate?

Comment 7

[BugBot [:suhaib / :marco/ :calixte]]

Crash volume for signature 'MustSkipMarking<T>':
 - nightly (version 51): 19 crashes from 2016-08-01.
 - aurora  (version 50): 80 crashes from 2016-08-01.
 - beta    (version 49): 2115 crashes from 2016-08-02.
 - release (version 48): 62 crashes from 2016-07-25.
 - esr     (version 45): 41 crashes from 2016-05-02.

Crash volume on the last weeks (Week N is from 08-22 to 08-28):
            W. N-1  W. N-2  W. N-3
 - nightly       3       6       8
 - aurora       24      40       6
 - beta        714     669     279
 - release      24      19       9
 - esr           1       0       1

Affected platforms: Windows, Linux

Crash rank on the last 7 days:
           Browser   Content     Plugin
 - nightly #516      #139
 - aurora  #147      #45
 - beta    #15       #14
 - release #1087     #157
 - esr     #3368

Comment 8

[Liz Henry (:lizzard) (relman/hg->git project)]

This is at least in the top 20 crashes on beta. It likely doesn't block the release, but I would like someone to look at it for future releases since it seems to consistently be a problem.

Comment 9

[David Bolter [:davidb] (NeedInfo me for attention)]

Terrence, could you check out some additional reports to see if this might be more than a hardware issue?
https://crash-stats.mozilla.com/signature/?signature=MustSkipMarking<T>#reports

Note CPU breakdown for this crash sig:
x86 	1081 	97.7%
amd64 	14 	1.3%
arm 	12 	1.1%

Comment 10

[Terrence Cole [:terrence]]

There are still a ton of different bugs landing this this pile. The individual crash reports are still totally unactionable, but the volume is such that there is likely a real bug somewhere. Lots of these stacks are under JSScript traversal, so there is probably a bug with someone failing to trace or sweep a script, somewhere.

Comment 11

[BugBot [:suhaib / :marco/ :calixte]]

Crash volume for signature 'MustSkipMarking<T>':
 - nightly (version 52): 8 crashes from 2016-09-19.
 - aurora  (version 51): 18 crashes from 2016-09-19.
 - beta    (version 50): 443 crashes from 2016-09-20.
 - release (version 49): 1381 crashes from 2016-09-05.
 - esr     (version 45): 7 crashes from 2016-06-01.

Crash volume on the last weeks (Week N is from 10-03 to 10-09):
            W. N-1  W. N-2
 - nightly       4       4
 - aurora       16       2
 - beta        354      89
 - release    1100     280
 - esr           0       2

Affected platforms: Windows, Linux

Crash rank on the last 7 days:
           Browser   Content     Plugin
 - nightly #364      #292
 - aurora  #206      #99
 - beta    #45       #33
 - release #56       #41
 - esr

Comment 12

[Wayne Mery (:wsmwk)]

I sampled the 3 of the 4 users who had the most crashes in the past month, as determined by their email address. Below are the 5-10 most recent crashes for each

bp-58717a2f-63c2-43da-a391-9b2a62161111	2016-11-11 10:19:12 	js::BaseShape::traceChildrenSkipShapeTable   
bp-a26127ea-88c6-4327-8c74-4ad302161111	2016-11-11 10:16:38 	nsCOMPtr_base::~nsCOMPtr_base | nsTimeout::~nsTimeout   
bp-f7636ce3-f1e4-42cf-93ac-2bff22161111	2016-11-11 10:14:48 	js::TenuringTracer::traverse<T>   
bp-cabfe057-840c-47dd-9bd5-0bd682161111	2016-11-11 05:01:09 	js::detail::HashTable<T>::lookup | js::detail::HashTable<T>::lookupForAdd | EvalScriptGuard::lookupInEvalCache   
bp-99339828-4f46-4c80-8768-a27b42161111	2016-11-11 05:00:59 	js::detail::HashTable<T>::lookup | bp-js::detail::HashTable<T>::lookupForAdd | EvalScriptGuard::lookupInEvalCache   
bp-a6d3b4eb-1e33-4e2f-a1ac-e65f92161102	2016-11-02 10:34:38 	JS::GCHashSet<T>::sweep   
bp-fc1b3673-4ab2-4288-840b-3a7912161102	2016-11-02 10:34:08 	js::gc::StoreBuffer::putValue    


bp-494f7c6a-7819-4cdf-8610-ef5522161111	2016-11-11 17:33:28 	js::jit::BacktrackingAllocator::pickStackSlots   
bp-2255aed0-c175-4aef-a260-29fd12161111	2016-11-11 16:31:21 	msvcr120.dll@0xf20c | huge_ralloc   
bp-b928fa98-cd32-486d-a148-f80a62161111	2016-11-11 16:06:24 	SnowWhiteKiller::Trace   
bp-7ff6734b-f651-4765-8326-13f5f2161110	2016-11-10 19:48:18 	nsPurpleBuffer::Block::VisitEntries<T>   
bp-d9c723b8-a739-4a3a-a31e-92db42161110	2016-11-10 19:39:05 	js::jit::BacktrackingAllocator::go   
bp-34e98889-f097-4d53-b756-8103a2161110	2016-11-10 18:33:54 	nsIFrame::GetOffsetToCrossDoc    
bug 1263916 has the highest crash rate for the signatures above


bp-b97122e4-91d9-4278-a685-121342161107	2016-11-07 15:47:56 	JS::GCHashSet<T>::sweep   
bp-a07e2c6e-01df-4917-917f-4014f2161107	2016-11-07 15:43:19 	mozilla::CSSStyleSheet::TraverseInner   
bp-c7e0ad46-dbee-42d5-9ae6-c56eb2161107	2016-11-07 15:43:09 	MustSkipMarking<T>   
bp-acaf91b5-444a-4015-9cb5-1ddbe2161107	2016-11-07 15:42:29 	js::ConcatStrings<T>   
bp-dbe073ca-d218-4f16-8df9-b13c22161107	2016-11-07 15:40:48 	UnmarkGrayTracer::onChild   
bp-56eb179b-d2b6-4720-ac00-18c082161107	2016-11-07 15:40:31 	nsDocShell::SetupNewViewer   
bp-176fd0b3-5fdc-433f-ab68-6de022161107	2016-11-07 12:44:05 	js::UnmarkScriptData   
bp-ea226f31-e999-4dba-9835-a02ff2161107	2016-11-07 12:42:46 	jit | UNKNOWN   
bp-794c4f58-502e-4f14-b6af-51b592161106	2016-11-06 10:15:31 	js::gc::StoreBuffer::putCell    
bug 1257309 has the highest crash rate for the signatures above

Comment 13

[Wayne Mery (:wsmwk)]

#74 crash for Thunderbird 45.4.0

Comment 14

[BugBot [:suhaib / :marco/ :calixte]]

Crash volume for signature 'MustSkipMarking<T>':
 - nightly (version 53): 35 crashes from 2016-11-14.
 - aurora  (version 52): 68 crashes from 2016-11-14.
 - beta    (version 51): 2194 crashes from 2016-11-14.
 - release (version 50): 11510 crashes from 2016-11-01.
 - esr     (version 45): 67 crashes from 2016-07-06.

Crash volume on the last weeks (Week N is from 01-02 to 01-08):
            W. N-1  W. N-2  W. N-3  W. N-4  W. N-5  W. N-6  W. N-7
 - nightly       5      11       7       9       1       0       2
 - aurora        9      13      14       8      11       8       0
 - beta        300     282     342     406     341     315     109
 - release    1275    1312    1627    1765    2163    2230     751
 - esr           5       4      10       7       6       4       8

Affected platforms: Windows, Linux

Crash rank on the last 7 days:
           Browser   Content   Plugin
 - nightly #488      #119
 - aurora  #239      #120
 - beta    #42       #33
 - release #38       #24
 - esr     #1888

Comment 15

[Julien Cristau [:jcristau] (back April 22)]

Too late for firefox 52, mass-wontfix.

Comment 16

[Wayne Mery (:wsmwk)]

#51 crash for Thunderbird 52.2.1. There were a couple reports in TB54.0b3 [1]. But no reports yet for TB55.0b2 (not surprising -
 Thunderbird beta has few users - 0.1% of release)

[1] bp-291ca412-ac8b-45b4-824d-ed4670170808 bp-929c29b6-1517-42cc-8246-4ca040170808

Comment 18

[Wayne Mery (:wsmwk)]

No version 60 crashes for Thunderbird

Comment 19

[Jon Coppeard (:jonco)]

Looks like memory corruption.

Comment 20

[BugBot [:suhaib / :marco/ :calixte]]

Closing because no crashes reported for 12 weeks.

Comment 21

[BugBot [:suhaib / :marco/ :calixte]]

Since the bug is closed, the stalled keyword is now meaningless.
For more information, please visit auto_nag documentation.