963483 - OpenH264: global-buffer-overflow crash [@WelsDec::DeblockingInterMb]

Status: RESOLVED FIXED

(Core :: WebRTC: Audio/Video, defect)

Description

[Christoph Diehl [:posidron]]

Attachment: testcase.264

codec/decoder/core/src/deblocking.cpp:279

  void_t FilteringEdgeLumaH (SDeblockingFilter* pFilter, uint8_t* pPix, int32_t iStride, uint8_t* pBS) {
    int32_t iIndexA;
    int32_t iAlpha;
    int32_t iBeta;
    FORCE_STACK_ALIGN_1D (int8_t, tc, 4, 16);

    GET_ALPHA_BETA_FROM_QP (pFilter->iLumaQP, pFilter->iSliceAlphaC0Offset, pFilter->iSliceBetaOffset, iIndexA, iAlpha,
                            iBeta);

    if (iAlpha | iBeta) {
*     TC0_TBL_LOOKUP (tc, iIndexA, pBS, 0);
      pFilter->pLoopf->pfLumaDeblockingLT4Ver (pPix, iStride, iAlpha, iBeta, tc);
    }
    return;
  }

Tested with https://github.com/cisco/openh264/commit/fcd7a13816

Comment 1

[Christoph Diehl [:posidron]]

Attachment: callstack.txt

Comment 2

[wayne]

Will look into this. But it may need some time.

Comment 3

[wayne]

Hi Christoph, the root cause is found the same as 963602, which is already fixed in our local test for syntax checking. It is due to the error iSliceAlphaC0Offset parse leadingto invalid array access. will be updated after checking code is reviewed.

Comment 4

[wayne]

Hi Christoph, the bug has been fixed in cisco master branch.
Could u please check it? Thanks.

Comment 5

[Christoph Diehl [:posidron]]

Testcase in ASan/UBSan build shows:

codec/decoder/core/inc/dec_golomb.h:176:5: runtime error: left shift of 15131 by 19 places cannot be represented in type 'int'

global-buffer-overflow crash is fixed.

Tested with https://github.com/cisco/openh264/commit/d468404822

Comment 6

[wayne]

Hi Christoph, the new issue has been fixed in latest cisco master branch.

Could u please have a check?

Thanks for your advice.

Comment 7

[Christoph Diehl [:posidron]]

Fixed - including the the UBSan runtime error.

Tested with https://github.com/cisco/openh264/commit/6854e06796