968168 - Faulty: crash in ImageLayerComposite::Disconnect due to scribbled vptr of LayerComposite base, under ShadowLayerParent::ActorDestroy

Status: RESOLVED FIXED

(Core :: Graphics, defect)

Description

[Benoit Jacob [:bjacob] (mostly away)]

Found by Christoph Diehl's "Faulty" fuzzer, see bug 777067

Hit while running webm-video reftests.

Comment 1

[Benoit Jacob [:bjacob] (mostly away)]

Attachment: Faulty session

Comment 2

[Benoit Jacob [:bjacob] (mostly away)]

This Faulty/GDB session is very cluttered, as it took me time to understand what was going on. So let's paste the relevant bits and comment on them to show what is happening here.

Relevant part of the stack:

(gdb) bt
#0  0x00002aaaade5742b in mozilla::layers::ImageLayerComposite::Disconnect (this=0x2aaace03cc00)
    at /hack/mozilla-central/gfx/layers/composite/ImageLayerComposite.cpp:59
#1  0x00002aaaade96591 in mozilla::layers::ShadowLayerParent::ActorDestroy (this=0x2aaac3b22780, why=
    mozilla::ipc::IProtocolManager<mozilla::ipc::IProtocol>::Deletion) at /hack/mozilla-central/gfx/layers/ipc/ShadowLayerParent.cpp:59
#2  0x00002aaaad78048b in mozilla::layers::PLayerParent::DestroySubtree (this=0x2aaac3b22780, why=
    mozilla::ipc::IProtocolManager<mozilla::ipc::IProtocol>::Deletion) at ./PLayerParent.cpp:292

Source code at crash location:

(gdb) l
56      void
57      ImageLayerComposite::Disconnect()
58      {
59        Destroy();        /// <-- CRASH HERE
60      }

The 'this' object:

(gdb) p this
$1 = (mozilla::layers::ImageLayerComposite *) 0x2aaace03cc00
(gdb) set print pretty on
(gdb) p *this
$2 = (mozilla::layers::ImageLayerComposite) {
  <mozilla::layers::ImageLayer> = {
    <mozilla::layers::Layer> = {
      _vptr$Layer = 0x2aaab419fe20, 
 [... snip ...]
  },
  <mozilla::layers::LayerComposite> = {
    _vptr$LayerComposite = 0x3e800000320,
 [... snip ...]

Are these vptr's pointing to valid memory?

(gdb) p *((void**)0x2aaab419fe20)
$20 = (void *) 0x2aaaade57210
(gdb) p *((void**)0x3e800000320)
Cannot access memory at address 0x3e800000320

So the _vptr$LayerComposite is definitely invalid (and we don't quite know about the _vptr$Layer, but it seems probably valid, as in this DEBUG build free'd memory is filled with 0xa5 bytes)

Is this invalid vptr relevant to the crash at hand?

Disassembly around crash location:

(gdb) disassemble 'mozilla::layers::ImageLayerComposite::Disconnect'
Dump of assembler code for function mozilla::layers::ImageLayerComposite::Disconnect():
   0x00002aaaade57400 <+0>:     push   %rbp
   0x00002aaaade57401 <+1>:     mov    %rsp,%rbp
   0x00002aaaade57404 <+4>:     sub    $0x10,%rsp
   0x00002aaaade57408 <+8>:     mov    %rdi,-0x8(%rbp)
   0x00002aaaade5740c <+12>:    mov    -0x8(%rbp),%rdi
   0x00002aaaade57410 <+16>:    mov    %rdi,%rax
   0x00002aaaade57413 <+19>:    add    $0x1e0,%rax
   0x00002aaaade57419 <+25>:    mov    0x1e0(%rdi),%rdi
   0x00002aaaade57420 <+32>:    mov    %rdi,-0x10(%rbp)
   0x00002aaaade57424 <+36>:    mov    %rax,%rdi
   0x00002aaaade57427 <+39>:    mov    -0x10(%rbp),%rax
=> 0x00002aaaade5742b <+43>:    callq  *0x18(%rax)
   0x00002aaaade5742e <+46>:    add    $0x10,%rsp
   0x00002aaaade57432 <+50>:    pop    %rbp
   0x00002aaaade57433 <+51>:    retq   
End of assembler dump.

Register values:

(gdb) info registers 
rax            0x3e800000320    4294967296800
rbx            0x0      0
rcx            0x1      1
rdx            0x2aaaade96550   46912550561104
rsi            0x1      1
rdi            0x2aaace03cde0   46913089162720
rbp            0x2aaad87035b0   0x2aaad87035b0
rsp            0x2aaad87035a0   0x2aaad87035a0
r8             0x20     32
r9             0x101010101010101        72340172838076673
r10            0x2aaaabb974f0   46912513864944
r11            0x2aaaabb98d2e   46912513871150
r12            0x7fffffffa9c0   140737488333248
r13            0x2aaad87049c0   46913264044480
r14            0x0      0
r15            0x3      3
rip            0x2aaaade5742b   0x2aaaade5742b <mozilla::layers::ImageLayerComposite::Disconnect()+43>
eflags         0x10202  [ IF RF ]
cs             0x33     51
ss             0x2b     43
ds             0x0      0
es             0x0      0
fs             0x0      0
gs             0x0      0


So we are crashing as we are trying to jump to an address around the value of %rax, and that value is that of the invalid _vptr$LayerComposite, 0x3e800000320.

So this bad _vptr$LayerComposite is definitely the cause of the crash.

The question is how did _vptr$LayerComposite get that bad value?

0x3e800000320 looks like two small uint32 values next to each other.

0x3e8 == 1000
0x320 == 800

So this 0x3e800000320 64bit value reinterprets as the pair of uint32's with values (1000, 800).

That is exactly the standard reftest resolution.

In summary, the probably cause of the crash is that the _vptr$LayerComposite in a ImageLayerComposite was scribbled with the bits from a gfx IntSize.

It might still be true that this is a use-after-free, but at this point I don't know. It could also be brutal scribbling of a live object.

Renaming the bug accordingly.

Comment 3

[Benoit Jacob [:bjacob] (mostly away)]

After investigating bug 968204, I wonder if the same could be happening here: a layer gets reinterpreted as a layer of a different type after Faulty corrupts the 'type' enum value passed in a message. That could explain why we end up reading the contents of a gfx IntSize where we expect a base class vptr...

Comment 4

[Benoit Jacob [:bjacob] (mostly away)]

Classification: PLayer protocol, possible layer type mismatch, hard

Comment 5

[Andrew McCreight [:mccr8]]

This sounds at least sec-high.

Comment 6

[Benoit Jacob [:bjacob] (mostly away)]

Fixed by the landing of PLayerTransaction type checks before casting layers, bug 968833.

Comment 7

[Matt Wobensmith [:mwobensmith][:matt:]]

Marking [qa-] for verification, lack of test case.