1114993 - [openh264] ASAN heap-buffer-overflow in WelsDec::WelsInitRefList

Status: RESOLVED FIXED

(Core :: Audio/Video: GMP, defect)

Description

[Nils]

Attachment: repro.264

The attached testcase crashes the ASAN build of h264dec of the upcoming openh264 firefox branch (https://github.com/cisco/openh264/tree/v1.3-Firefox36).

This affects the ASM optimised 32-bit build. The non-asm 32 and 64bit builds crash on a read violation, probably because ASAN is bailing early on the read.

ASAN output:

=================================================================
==28876==ERROR: AddressSanitizer: heap-buffer-overflow on address 0xf485d817 at pc 0x080d7410 bp 0xffd4d5c8 sp 0xffd4d1ac
WRITE of size 81920 at 0xf485d817 thread T0
    #0 0x80d740f in __asan_memset /home/bobthebuilder/llvm/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cc:383:5
    #1 0x816b19e in WelsDec::WelsInitRefList(WelsDec::TagWelsDecoderContext*, int) (/home/nils/264/h264dec-ff-1.3+0x816b19e)
    #2 0x8150c8c in WelsDec::DecodeCurrentAccessUnit(WelsDec::TagWelsDecoderContext*, unsigned char**, TagBufferInfo*) (/home/nils/264/h264dec-ff-1.3+0x8150c8c)
    #3 0x814bd9c in WelsDec::ConstructAccessUnit(WelsDec::TagWelsDecoderContext*, unsigned char**, TagBufferInfo*) (/home/nils/264/h264dec-ff-1.3+0x814bd9c)
    #4 0x812b3cb in WelsDecodeBs (/home/nils/264/h264dec-ff-1.3+0x812b3cb)
    #5 0x811f593 in WelsDec::CWelsDecoder::DecodeFrame2(unsigned char const*, int, unsigned char**, TagBufferInfo*) (/home/nils/264/h264dec-ff-1.3+0x811f593)
    #6 0x81170a9 in H264DecodeInstance(ISVCDecoder*, char const*, char const*, int&, int&, char const*) (/home/nils/264/h264dec-ff-1.3+0x81170a9)
    #7 0x811a074 in main (/home/nils/264/h264dec-ff-1.3+0x811a074)
    #8 0xf7494a82  (/lib/i386-linux-gnu/libc.so.6+0x19a82)
    #9 0x807e69b in _start (/home/nils/264/h264dec-ff-1.3+0x807e69b)

0xf485d817 is located 0 bytes to the right of 86039-byte region [0xf4848800,0xf485d817)
allocated by thread T0 here:
    #0 0x80f25eb in malloc /home/bobthebuilder/llvm/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:40:3
    #1 0x819525f in WelsMalloc (/home/nils/264/h264dec-ff-1.3+0x819525f)

SUMMARY: AddressSanitizer: heap-buffer-overflow /home/bobthebuilder/llvm/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cc:383 __asan_memset
Shadow bytes around the buggy address:
  0x3e90bab0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x3e90bac0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x3e90bad0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x3e90bae0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x3e90baf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x3e90bb00: 00 00[07]fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x3e90bb10: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x3e90bb20: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x3e90bb30: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x3e90bb40: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x3e90bb50: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
==28876==ABORTING

Comment 1

[Daniel Veditz [:dveditz]]

Ethan: same questions as bug 1114996 comment 1

Comment 2

[Ethan Hugg [:ehugg]]

According to my testing, this one is fixed by this commit which has now been merged into the v1.3-Firefox36 branch:
https://github.com/cisco/openh264/commit/5c114c3ebb6d6477e5cbf1c52b9f08a7cb4d18c0

Comment 3

[sijchen]

same in https://bugzilla.mozilla.org/show_bug.cgi?id=1114990

Comment 4

[Ethan Hugg [:ehugg]]

OpenH264 1.3 has this fix and is now downloaded for Firefox 34+ so marking this as fixed.