1479831 - OpenH264: shift exponent is negative in codec/decoder/core/src/cabac_decoder.cpp

Status: RESOLVED FIXED

(Core :: Audio/Video: GMP, defect)

Description

[Tyson Smith [:tsmith]]

Attachment: testcase.264

Found while fuzzing openh264 revision f92a006bb05dce89f312df8a641a65abf09076c8

Build with "-fsanitize=undefined"

To reproduce:
./h264dec testcase.264 /dev/null

codec/decoder/core/src/cabac_decoder.cpp:148:28: runtime error: shift exponent -1 is negative
    #0 0x775427 in WelsDec::DecodeBinCabac(WelsDec::SWelsCabacDecEngine*, WelsDec::SWels_Cabac_Element*, unsigned int&) codec/decoder/core/src/cabac_decoder.cpp:148:28
    #1 0x71856e in WelsDec::DecodeCabacIntraMbType(WelsDec::TagWelsDecoderContext*, WelsDec::TagNeighborAvail*, int) codec/decoder/core/src/parse_mb_syn_cabac.cpp:88:3
    #2 0x71856e in WelsDec::ParseMBTypeBSliceCabac(WelsDec::TagWelsDecoderContext*, WelsDec::TagNeighborAvail*, unsigned int&) codec/decoder/core/src/parse_mb_syn_cabac.cpp:370
    #3 0x6c5741 in WelsDec::WelsDecodeMbCabacBSliceBaseMode0(WelsDec::TagWelsDecoderContext*, WelsDec::TagNeighborAvail*, unsigned int&) codec/decoder/core/src/decode_slice.cpp:1091:3
    #4 0x6d341f in WelsDec::WelsDecodeMbCabacBSlice(WelsDec::TagWelsDecoderContext*, WelsDec::TagNalUnit*, unsigned int&) codec/decoder/core/src/decode_slice.cpp:1443:3
    #5 0x6d8425 in WelsDec::WelsDecodeSlice(WelsDec::TagWelsDecoderContext*, bool, WelsDec::TagNalUnit*) codec/decoder/core/src/decode_slice.cpp:1555:12
    #6 0x59adc4 in WelsDec::DecodeCurrentAccessUnit(WelsDec::TagWelsDecoderContext*, unsigned char**, TagBufferInfo*) codec/decoder/core/src/decoder_core.cpp:2549:16
    #7 0x595d93 in WelsDec::ConstructAccessUnit(WelsDec::TagWelsDecoderContext*, unsigned char**, TagBufferInfo*) codec/decoder/core/src/decoder_core.cpp:2252:10
    #8 0x55a69e in WelsDecodeBs codec/decoder/core/src/decoder.cpp:798:7
    #9 0x52e365 in WelsDec::CWelsDecoder::DecodeFrame2(unsigned char const*, int, unsigned char**, TagBufferInfo*) codec/decoder/plus/src/welsDecoderExt.cpp:570:3
    #10 0x52c4f4 in WelsDec::CWelsDecoder::DecodeFrameNoDelay(unsigned char const*, int, unsigned char**, TagBufferInfo*) codec/decoder/plus/src/welsDecoderExt.cpp:495:11
    #11 0x516b49 in H264DecodeInstance(ISVCDecoder*, char const*, char const*, int&, int&, char const*, char const*, int, bool) codec/console/dec/src/h264dec.cpp:226:17
    #12 0x51c34f in main codec/console/dec/src/h264dec.cpp:510:3
    #13 0x7f26f58ec82f in __libc_start_main /build/glibc-Cl5G7W/glibc-2.23/csu/../csu/libc-start.c:291
    #14 0x41d638 in _start (h264dec+0x41d638)

Comment 1

[xiaotian shi]

The issue has been addressed by openh264 #PR 3011

Comment 2

[Tyson Smith [:tsmith]]

Verified with commit 1b3980b3437e83f30001e9b7dfdf4a98e69b87bc