958935 - OpenH264: global-buffer-overflow crash [@WelsDec::IdctResAddPred_c]

Status: RESOLVED FIXED

(Core :: WebRTC: Audio/Video, defect)

Description

[Christoph Diehl [:posidron]]

Attachment: testcase.264

codec/decoder/core/src/decode_mb_aux.cpp:101

void_t IdctResAddPred_c (uint8_t* pPred, const int32_t kiStride, int16_t* pRs) {
  [...]
  pDst[i + kiStride] = pClip[ ((32 + kT1 + kT2) >> 6) + pDst[i + kiStride] ];
  [...]
}

Tested with https://github.com/cisco/openh264/commit/4a8a9aabc1

Comment 1

[Christoph Diehl [:posidron]]

Attachment: callstack

Comment 2

[wayne]

we used the latest decoder to test the "testcase.264" from https://github.com/cisco/openh264/commit/14f6c4fa72159fa9353722ef029e68c9beabbe82 
and no crash occurs in Mac 64.

Comment 3

[Christoph Diehl [:posidron]]

Hi wayne, it still crashes for me. Note that you need to test it against an ASan enabled build.

Comment 4

[wayne]

we're looking into the bugs, and this bug seems to be caused by look-up table optimization. we will give each bug a deep solution. Please give us a little more time.

Comment 5

[wayne]

root cause found, i.e., the combination of 958904 & 958948. pull request can be found by https://github.com/cisco/openh264/pull/146

Comment 6

[wayne]

Hi Christoph:
   fix it. Could you verify it on cisco/master branch? thanks!

Comment 7

[Christoph Diehl [:posidron]]

Fixed.

Tested with https://github.com/cisco/openh264/commit/fcd7a13816

Comment 8

[Al Billings [:abillings - ex-MoCo]]

What was the first affected version of Firefox with this? Was the fix from comment 7 integrated into trunk?

Comment 9

[Christoph Diehl [:posidron]]

(In reply to Al Billings [:abillings] from comment #8)
> What was the first affected version of Firefox with this? 

OpenH264 is not in our tree yet.