567951 - Picture region checks in VideoData::Create are incorrect

Status: RESOLVED INACTIVE

(Core :: Audio/Video: Playback, defect)

Description

[Matthew Gregan [:kinetik]]

In nsVideoInfo, the rectangle specified by mPicture should be located completely within the rectangle specified by mFrame.  The stride of the frame may be (usually is) larger than the frame width.  The sanity checks in VideoData::Create are incorrect:

Say, for a frame which is 800 pixels wide:

  if (!AddOverflow32(aInfo.mPicture.x, aInfo.mPicture.width, picXLimit) ||
      picXLimit > PRUint32(aBuffer.mPlanes[0].mStride) ||

x = 60
width = 780
picXLimit = 840
mStride = 864

This check will succeed as picXLimit < mStride, but the picture region extends outside of the valid frame rectangle.

Comment 1

[Nils Ohlmeier [:drno]]

Mass closing do to inactivity.
Feel free to re-open if still needed.