635295 - Crash in GMail [@ js::mjit::JaegerShot ]

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect)

Description

[Ben Turner (not reading bugmail, use the needinfo flag!)]

Today's nightly on OS X 64bit is killing me. I crashed three times in GMail before I simply disabled JM. Each time I crashed I was clicking the 'Newer' link to move to the next message in my inbox.

bp-fb14a3aa-2664-4489-a899-9c8342110218
bp-c309e061-d8c9-4852-a05a-03c3a2110218
bp-78f24736-ac63-4bcf-8ab1-18c892110218

Comment 1

[David Anderson [:dvander] - inactive, e-mail if emergency]

Yuck. Would you be able to generate a core dump (set ulimit -c and launch from Terminal) or get it in gdb?

Comment 2

[David Anderson [:dvander] - inactive, e-mail if emergency]

Err "ulimit -c unlimited"

Comment 3

[Ben Turner (not reading bugmail, use the needinfo flag!)]

Ok, coredump sent to dvander.

Comment 4

[David Anderson [:dvander] - inactive, e-mail if emergency]

The active script is from gmail and we're faulting on an invalid EIP. We came from here:

0x0000000117f681c0:	mov    $0xfffb800000000000,%r11
0x0000000117f681ca:	cmp    %r11,%r12
0x0000000117f681cd:	jb     0x117f6881e
-----------------------------------^ Goes to stubs::UncachedCall (obj guard)

0x0000000117f681d3:	mov    $0x1019ccf20,%r11
0x0000000117f681dd:	cmp    %r11,0x8(%r15)
0x0000000117f681e1:	jne    0x117f6881e
-----------------------------------^ js_FunctionClass guard

0x0000000117f681e7:	mov    0x30(%r15),%r15
0x0000000117f681eb:	mov    $0x100faa9d0,%r11
0x0000000117f681f5:	cmp    %r11,0x60(%r15)
0x0000000117f681f9:	jne    0x117f6881e
-----------------------------------^ js_fun_call guard

0x0000000117f681ff:	mov    $0xfffb800000000000,%r11
0x0000000117f68209:	cmp    %r11,%r8
0x0000000117f6820c:	jb     0x117f68910
-----------------------------------^ Goes to ic::NativeCall for JSOP_FUNCALL

0x0000000117f68212:	mov    $0x1358bf948,%r11
-----------------------------------^ callee guard

0x0000000117f6821c:	cmp    %r11,%r9
0x0000000117f6821f:	jne    0x12252e598
-----------------------------------^ invalid, but we take this branch.

Comment 5

[David Anderson [:dvander] - inactive, e-mail if emergency]

ic.fastGuardedObject is non-NULL
ic.fastGuardedNative is NULL
ic.pools[0] is non-NULL
ic.pools[1], [2] are NULL

So, this means the IC thinks it is in the following state:
 (1) The fast-path is linked.
 (2) The only stub generated is a "full call stub", which is linked from the
     OOL path.

What seems to have happened is that fastGuardedNative got NULL'd, but it wasn't patched.

Comment 6

[(Out Sep 17-23) Ryan VanderMeulen]

Attachment: Windows 7 crash stack

I've been seeing a lot of crashes in GMail as well since updating my build last night. Windows 7 x64 SP1. Different stack, though. Maybe related?

Comment 7

[Mauricio Collares [:collares]]

I also saw a bunch of crash reports like this one:

https://crash-stats.mozilla.com/report/index/bp-be2ccd6b-29a7-444a-8c49-222f22110218
https://crash-stats.mozilla.com/report/index/bp-f34ef5f6-c10c-4f7e-946f-0b5152110218
https://crash-stats.mozilla.com/report/index/bp-68631e86-3117-4feb-9599-31e3e2110218
https://crash-stats.mozilla.com/report/index/bp-d415a890-3acb-467b-97e6-def6d2110218

Also, this might be unrelated, but I was getting crashes like [1] since a few days ago when trying to change the status message on Gmail Chat. The newer ones happen while doing other things as well and are much more frequent, though.

[1] https://crash-stats.mozilla.com/report/index/bp-dea892ea-491e-4f7b-a115-e08b22110216

Comment 8

[Mauricio Collares [:collares]]

Please ignore the [1] link above. I got a backtrace of it at http://pastebin.com/AwhRWUbv and it's probably not JS related.

Comment 9

[David Anderson [:dvander] - inactive, e-mail if emergency]

Phew. I managed to get a reproducible, shell test case that crashes. There is an extremely subtle bug in sweepCallICs.

The test case doesn't crash without some instrumentation: in my build, I made ExecutablePools allocation requests take more than the max "small" allocation size. This way, every single IC gets its own EP. This means code gets released instantly, exposing crashes and refcounting bugs easier.

Comment 10

[David Anderson [:dvander] - inactive, e-mail if emergency]

Attachment: test case

gmail is using Function.bind, which gives back a native that can be garbage collected much quicker than most natives. The bug is that the relinking condition in sweepCallICs does not cover the following case:

 (1) ic.fastGuardedObject is non-NULL - the inline path is linked.
 (2) ic.fastGuardedNative is non-NULL - there is a native IC as well.
 (3) The object in (1) is live.
 (4) The object in (2) is dead.

This will remove the native stub, but the condition at the bottom of sweepCallICs will see |ic.fastGuardedObject| as non-NULL, and won't relink the jump.

Comment 11

[David Anderson [:dvander] - inactive, e-mail if emergency]

Attachment: fix

Comment 12

[Luke Wagner [:luke]]

Comment on attachment 513655 [details] [diff] [review]
fix

Good job with the repro case.

From IRL: followup bug ideas: when fastFunDead and !ic.hasJsFunCheck, assert that ic.funJump is already pointing to ic.slowPathStart.  Also, when fastFunDead, patch ic.hotJumpOffset to jump to some js_SafeCrash.

Comment 13

[David Anderson [:dvander] - inactive, e-mail if emergency]

http://hg.mozilla.org/tracemonkey/rev/39e3a00ea6e8
http://hg.mozilla.org/mozilla-central/rev/49c49bcf67bd