1224369 - UBSan: index out of bounds [@vp8_loop_filter_row_normal]

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect, P1)

Description

[Tyson Smith [:tsmith]]

Attachment: test_case.vp8

This was found by fuzzing libvpx (commit c6641709a707ccb98cbdf785428659e44d4f2c8b) and it appears to be in our branch.

https://dxr.mozilla.org/mozilla-central/source/media/libvpx/vp8/common/loopfilter.c#222

vp8/common/vp8_loopfilter.c:222:35: runtime error: index 225 out of bounds for type 'unsigned char [64]'
    #0 0x77cca4 in vp8_loop_filter_row_normal (/home/user/Desktop/libvpx/simple_decoder_ub_asan+0x77cca4)
    #1 0x83f283 in decode_mb_rows (/home/user/Desktop/libvpx/simple_decoder_ub_asan+0x83f283)
    #2 0x837e2d in vp8_decode_frame (/home/user/Desktop/libvpx/simple_decoder_ub_asan+0x837e2d)
    #3 0x5d052d in vp8dx_receive_compressed_data (/home/user/Desktop/libvpx/simple_decoder_ub_asan+0x5d052d)
    #4 0x5ca52c in vp8_decode (/home/user/Desktop/libvpx/simple_decoder_ub_asan+0x5ca52c)
    #5 0x4ecdde in vpx_codec_decode (/home/user/Desktop/libvpx/simple_decoder_ub_asan+0x4ecdde)
    #6 0x4eb189 in main /home/user/code/libvpx/examples/simple_decoder.c:135:11
    #7 0x7f121a2a5ec4 in __libc_start_main /build/buildd/eglibc-2.19/csu/libc-start.c:287
    #8 0x41dad5 in _start (/home/user/Desktop/libvpx/simple_decoder_ub_asan+0x41dad5)

Comment 1

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: 1224369-p1-gtest-list-of-test-cases.patch

Part 1: Test cases given as list.

No actual test changes from before. This will help with this bug and future
ones, to easily add more test cases.

Comment 2

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: 1224369-p2-gtest.patch

Part 2: Added vp8/ivf test case.

Comment 3

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

This issue has the same cause as bug 1224363 (filter_level not clamped to 0-63), though the effect appears elsewhere, so there is no more need for a fix.

Comment 4

[Ralph Giles (:rillian)]

Comment on attachment 8690624 [details] [diff] [review]
1224369-p1-gtest-list-of-test-cases.patch

Review of attachment 8690624 [details] [diff] [review]:
-----------------------------------------------------------------

Nice.

Comment 5

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/06c9e401ee3c
https://hg.mozilla.org/integration/mozilla-inbound/rev/6b7901d50318

I might have been to eager to check-in, as I believe this bug should be sec-low or even no-sec. Also this bug is actually fixed by the patch in bug 1224363 (which is sec-low or could be no-sec, as confirmed by an libvpx expert in bug 1224363 comment 15).
The patches here are just a gtest test case.

But sorry if I should have asked for official approval first.
Could you please lower the sec rating?

Comment 6

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/06c9e401ee3c
https://hg.mozilla.org/mozilla-central/rev/6b7901d50318

Comment 7

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

sec-moderate was low enough for landing.