858032 - (SadJit) [meta] crashes in EnterBaseline / EnterJit

Status: NEW

(Core :: JavaScript Engine, defect, P5)

Description

[Robert Kaiser]

This bug was filed from the Socorro interface and is 
report bp-374cf0cd-fdfc-4118-bf30-9629d2130404 .
============================================================= 


This is probably the generic signature we're now seeing for crashes in JITed code now with the new Baseline Compiler where we have seen EnterMethodJIT before when we were still in JägerMonkey.

So this bug is a catch-all for those issues at this time, and not actionable per se (still better to start clean here than to add it on the old EnterMethodJIT one).


Do we have better mechanisms to debug those crashes with BC than we had with JM?

Comment 1

[Jan de Mooij [:jandem]]

All of these seem to be bug 858022 so far.

Comment 2

[Robert Kaiser]

FYI, bug 595351 was the meta on EnterMethodJIT.

Oh, and for specific reproducible crashes with EnterBaseline (or specific cases you could find with debugging on minidumps), please file separate bugs blocking this one.


(In reply to Jan de Mooij [:jandem] from comment #1)
> All of these seem to be bug 858022 so far.

Ah, good. I just felt we should have a generic one for those signatures in any case, just like we had for EnterMethodJIT. I'm happy about any specific cases we find and can eliminate, of course. :)

Comment 3

[Robert Kaiser]

Scoobidiver, this is a meta bug tracking the EnterBaseline issue, as I described in comment #0. Let's put flags on being startup, native, or whatever on the bugs filed on specific issues, e.g. bug 858083 which is the [native-crash] we mainly are seeing now or bug 858022 which is the x86 one making up most of the Firefox crashes there atm.

Also, this is not a regression, you can see it either as new feature crashing (in terms of Baseline being a new compiler we added) or as a shift of signatures (in terms of some crashes we've seen in JM's EnterMethodJIT now being shifted to EnterBaseline with the BC landing).

Comment 4

[Scoobidiver (away)]

It's #3 browser crasher in 23.0b3, #5 in 24.0a2, and #6 in 25.a1.

Comment 5

[Robert Kaiser]

Yes, as this is a signature that gobbles up all crashes in the Baseline JIT, this being a topcrash across channels is expected. I think it's a bit lower than EnterMethodJIT and JeagerShot signatures were in combined volume, though, so the Baseline JIT Compiler might have fixed some crashes that we had with the JägerMonkey JIT. But that's mostly speculation, given the complexity of that area (also in terms of related signatures).

Comment 6

[Robert Kaiser]

I'm adding unactionable here as this bug directly is not directly actionable, tracking a generic signature. There's things we can do to get more info about those crashes, and if we find reproducible steps for this signature, we should file separate bugs and go for fixes there.

Comment 7

[alex_mayorga]

Nightly crashes consistently when adding attachments in Yahoo! Mail, see bug 935491 please.

Comment 8

[me]

I just got this, https://crash-stats.mozilla.com/report/index/0c1ef456-3f11-436b-8d6c-6f3e82140612

Comment 10

[Liz Henry (:lizzard) (relman/hg->git project)]

Noting that this is still a topcrash in Firefox 34 and is the #2 topcrash for Firefox 34.0b4 and b5.

Comment 12

[alex_mayorga]

¡Hola Kairo!

This moved 9^ to become top crash #3 https://crash-stats.mozilla.com/topcrasher/products/Firefox/versions/42.0b in the past week

Is it worth updating the Version on this bug?

¡Gracias!
Alex

Comment 13

[Robert Kaiser]

As mentioned in the summary, this is a meta bug. All kinds of crashes in JITed code get registered with this signature, and there are quite a few so it gets into topcrash ranks. There is nothing we can do without concrete testcases though, and those should go into separate bugs. When looking at stats, just ignore this signature.

Comment 16

[Cosmin Muntean [:cmuntean], Ecosystem QA]

From the crash signature [@ EnterBaseline ] , the affected versions are:
- Nightly: 47.0a1, 46.0a1	
- Firefox: 44.0
- Aurora: 46.0a2, 45.0a2, 44.0a2
- Beta: 45.0b2, 45.0b1, 44.0b99, 44.0b9, 44.0b8, 44.0b7, 44.0b6, 44.0b4, 44.0b2, 44.0b11, 44.0b1

From the crash signature [@ @0x0 | EnterBaseline ] , the affected versions are:
- Nightly: 46.0a1	
- Firefox: 44.0
- Aurora: 46.0a2, 45.0a2, 44.0a2
- Beta: 45.0b1, 44.0b8, 44.0b4, 44.0b2, 44.0b11, 44.0b1,

Comment 17

[BugBot [:suhaib / :marco/ :calixte]]

Crash volume for signature 'EnterBaseline':
 - nightly (version 50): 64 crashes from 2016-06-06.
 - aurora  (version 49): 89 crashes from 2016-06-07.
 - beta    (version 48): 110 crashes from 2016-06-06.
 - release (version 47): 3032 crashes from 2016-05-31.
 - esr     (version 45): 531 crashes from 2016-04-07.

Crash volume on the last weeks:
             Week N-1   Week N-2   Week N-3   Week N-4   Week N-5   Week N-6   Week N-7
 - nightly         19         13          6          9          3          4          3
 - aurora          25         14         13         16          9          5          0
 - beta            23         14         18         12         21          9          8
 - release        469        500        469        427        465        393        122
 - esr             43         51         50         43         54         52         50

Affected platforms: Windows, Mac OS X, Linux

Comment 18

[BugBot [:suhaib / :marco/ :calixte]]

Crash volume for signature 'EnterBaseline':
 - nightly (version 51): 42 crashes from 2016-08-01.
 - aurora  (version 50): 58 crashes from 2016-08-01.
 - beta    (version 49): 113 crashes from 2016-08-02.
 - release (version 48): 331 crashes from 2016-07-25.
 - esr     (version 45): 670 crashes from 2016-05-02.

Crash volume on the last weeks (Week N is from 08-22 to 08-28):
            W. N-1  W. N-2  W. N-3
 - nightly      14      15       6
 - aurora       21      16       8
 - beta         37      35      20
 - release     106      80      39
 - esr          60      48      45

Affected platforms: Windows, Mac OS X, Linux

Crash rank on the last 7 days:
           Browser   Content     Plugin
 - nightly #104      #170
 - aurora  #54       #73
 - beta    #411      #1570
 - release #176      #107
 - esr     #155

Comment 20

[BugBot [:suhaib / :marco/ :calixte]]

Crash volume for signature 'EnterBaseline':
 - nightly (version 52): 18 crashes from 2016-09-19.
 - aurora  (version 51): 14 crashes from 2016-09-19.
 - beta    (version 50): 47 crashes from 2016-09-20.
 - release (version 49): 351 crashes from 2016-09-05.
 - esr     (version 45): 933 crashes from 2016-06-01.

Crash volume on the last weeks (Week N is from 10-03 to 10-09):
            W. N-1  W. N-2
 - nightly      11       7
 - aurora       14       0
 - beta         35      12
 - release     266      80
 - esr         101      96

Affected platforms: Windows, Mac OS X, Linux

Crash rank on the last 7 days:
           Browser   Content     Plugin
 - nightly #80
 - aurora  #138      #176
 - beta    #405      #1115
 - release #244      #219
 - esr     #117

Comment 21

[BugBot [:suhaib / :marco/ :calixte]]

Crash volume for signature 'EnterBaseline':
 - nightly (version 53): 76 crashes from 2016-11-14.
 - aurora  (version 52): 98 crashes from 2016-11-14.
 - beta    (version 51): 328 crashes from 2016-11-14.
 - release (version 50): 2177 crashes from 2016-11-01.
 - esr     (version 45): 2593 crashes from 2016-07-06.

Crash volume on the last weeks (Week N is from 01-02 to 01-08):
            W. N-1  W. N-2  W. N-3  W. N-4  W. N-5  W. N-6  W. N-7
 - nightly      23      10       6       6      13       9       4
 - aurora       23      18      18      13       6      12       0
 - beta         43      42      61      37      52      48      28
 - release     338     320     393     360     329     260      88
 - esr         132     162     186     197     137     134     150

Affected platforms: Windows, Mac OS X, Linux

Crash rank on the last 7 days:
           Browser   Content   Plugin
 - nightly #61       #254
 - aurora  #76       #164
 - beta    #262      #877
 - release #213      #227
 - esr     #119

Comment 22

[BugBot [:suhaib / :marco/ :calixte]]

Crash volume for signature 'EnterBaseline':
 - nightly (version 54): 14 crashes from 2017-01-23.
 - aurora  (version 53): 3 crashes from 2017-01-23.
 - beta    (version 52): 17 crashes from 2017-01-23.
 - release (version 51): 146 crashes from 2017-01-16.
 - esr     (version 45): 3114 crashes from 2016-08-03.

Crash volume on the last weeks (Week N is from 01-30 to 02-05):
            W. N-1  W. N-2  W. N-3  W. N-4  W. N-5  W. N-6  W. N-7
 - nightly      10
 - aurora        2
 - beta         13
 - release      73       0
 - esr         186     210     172     173     132     162     186

Affected platforms: Windows, Mac OS X, Linux

Crash rank on the last 7 days:
           Browser   Content   Plugin
 - nightly #43
 - aurora  #323      #93
 - beta    #561      #864
 - release #273      #178
 - esr     #107

Comment 24

[Andrew]

I replaced the RAM on my computer and the error is gone.

Comment 25

[Julien Cristau [:jcristau] (back April 22)]

Too late for firefox 52, mass-wontfix.

Comment 27

[Ted Campbell [:tcampbell]]

Adding EnterBaselineMethod back to signatures so that historical crash rate graph is less misleading.

Comment 28

[Ted Campbell [:tcampbell]]

Bug 1034706 has lots of detailed (but perhaps slightly outdated) analysis.

Comment 29

[Ted Campbell [:tcampbell]]

In general, the majority of crashes on these signatures are due to bad hardware. This is either due to known-bad CPUs (See Bug 1281759), or failing RAM.

Comment 33

[alex_mayorga]

¡Hola y'all!

Found this crash over at https://support.mozilla.org/es/questions/1360629#answer-1465751
Updated flags per https://crash-stats.mozilla.org/signature/?product=Firefox&signature=js%3A%3Ajit%3A%3AMaybeEnterJit
Hope this is helpful.

¡Gracias!

Comment 34

[Iain Ireland [:iain]]

Thanks for the report!

I took a quick look at the minidump. Based on the disassembly at the point of the crash, it looks like we tried loading the JSClass from a JS object to check a flag, and one of the pointers in the object->shape->class chain was bogus. (The flag in question is the NON_NATIVE flag, but I don't think that helps.) This shouldn't happen; the object's data must have been corrupted somehow before we hit this code. Unfortunately, there isn't enough data in the dump to look back in time and figure out how that happened. (The only thing I've managed to work out is that it looks like the object pointer was correctly tagged.)

If we see a number of similar crashes, that might help us narrow things down, but for now I don't think there's anything we can do here. This is a very high-level crash signature that will catch many crashes in jit-compiled code, even if those crashes aren't related to each other. (I just looked at a random sample of the half-dozen most recent dumps with this signature, and none of them matched this pattern, or each other.)

Comment 35

[BugBot [:suhaib / :marco/ :calixte]]

The severity field for this bug is set to S4. However, the bug has the topcrash keyword.
:sdetar, could you consider increasing the severity of this top-crash bug? If the crash isn't "top" anymore, could you drop the topcrash keyword?

For more information, please visit auto_nag documentation.

Comment 36

[BugBot [:suhaib / :marco/ :calixte]]

The bug is linked to a topcrash signature, which matches the following criteria:

• Top 20 desktop browser crashes on release (startup)
• Top 20 desktop browser crashes on beta (startup)
• Top 10 content process crashes on beta
• Top 10 content process crashes on release
• Top 5 desktop browser crashes on Linux on release (startup)
• Top 10 AArch64 and ARM crashes on release (startup)

For more information, please visit auto_nag documentation.

Comment 37

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Any idea why EnterJit is more frequent since Friday?

Comment 40

[Donal Meehan [:dmeehan]]

Any idea why EnterJit is more frequent since Friday?

Comment 41

[Jules A]

I get it instantly when trying to open a simple spreadsheet in Excel Online (it does have hyperlinks though). It does it even with Tampermonkey disabled so it's not 1848391.