958958 - OpenH264: global-buffer-overflow crash [@WelsDec::IdctResAddPred_c]

Status: RESOLVED FIXED

(Core :: WebRTC: Audio/Video, defect)

Description

[Christoph Diehl [:posidron]]

Attachment: testcase.264

Similar to https://bugzilla.mozilla.org/show_bug.cgi?id=958935 but has a different stack and uses kiStride2.

codec/decoder/core/src/decode_mb_aux.cpp:102

void_t IdctResAddPred_c (uint8_t* pPred, const int32_t kiStride, int16_t* pRs) {
  [...]
  const int32_t kiStride2   = kiStride << 1;
  [...]
  pDst[i + kiStride2] = pClip[ ((32 + kT1 - kT2) >> 6) + pDst[i + kiStride2] ];
  [...]
}

Tested with https://github.com/cisco/openh264/commit/4a8a9aabc1

Comment 1

[Christoph Diehl [:posidron]]

Attachment: callstack

Comment 2

[wayne]

root cause found, reason same as 958948, pull request can be seen via https://github.com/cisco/openh264/pull/146

Comment 3

[wayne]

Hi Christoph:
   fix it. Could you verify it on cisco/master branch? thanks!

Comment 4

[Christoph Diehl [:posidron]]

Fixed.

Tested with https://github.com/cisco/openh264/commit/fcd7a13816

Comment 5

[Al Billings [:abillings - ex-MoCo]]

What versions of Firefox were affected by this? What version took the github fix into it (if any yet)?