701259 - (CVE-2011-3665) crashes on videos when scaling to large width/height

Status: VERIFIED FIXED

(Core :: Audio/Video, defect)

Description

[sczimmer]

Attachment: example website

when visiting the website:
<video width="999999999" height="999999999" controls autoplay>
<source src="avideo.ogg" type="video/ogg">
</video><video controls autoplay>

firefox crashes in LinearScaleYUVToRGB32Row, an inline assembly function defined in gfx/ycbcr/yuv_row_posix.cpp
it looks like firefox is writing xmm1 to rgb_buf width times but width is too big so it writes xmm1 until it reaches unmapped memory and crashes then

I tested in firefox 5 on 32-bit linux and firefox 11 on 64-bit linux and it crashes as described above. I also tested in firefox 3.6 and it doesn't crash

marking as security because possibly exploitable memory corruption

Comment 1

[sczimmer]

Attachment: video to go with website

Comment 2

[sczimmer]

Attachment: backtrace for firefox 11 on 64-bit linux

Comment 3

[Chris Pearce [:cpearce (Not reading bugmail)]]

Attachment: Patch v1

Reject attempts to scale a BasicPlanarYCbCrImage if width or height are larger than PlanarYCbCrImage::MAX_DIMENSION.

Comment 4

[Matthew Gregan [:kinetik]]

It seems like this should be rejected at a higher level.  mScaleHint is being set to a size computed from an area with width==height==nscoord_MAX, which seems bogus.

Comment 5

[Chris Pearce [:cpearce (Not reading bugmail)]]

What did you have in mind specifically, Matthew?

Comment 6

[Robert O'Callahan (:roc) (email my personal email if necessary)]

You're probably right, but defense in depth is fine here.

Comment 7

[Chris Pearce [:cpearce (Not reading bugmail)]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/9f27da086fe3

Comment 8

[Ian Melven :imelven]

from edmorley : https://hg.mozilla.org/mozilla-central/rev/09ad1943c19a

Comment 9

[Ian Melven :imelven]

(In reply to Ian Melven :imelven from comment #8)
> from edmorley : https://hg.mozilla.org/mozilla-central/rev/09ad1943c19a

scratch that, wrong bug - https://hg.mozilla.org/mozilla-central/rev/9f27da086fe3 fixes this

Comment 10

[Chris Pearce [:cpearce (Not reading bugmail)]]

Comment on attachment 574204 [details] [diff] [review]
Patch v1

Requesting approval on Aurora and Beta because this is a crasher and it is potentially exploitable. I think it would be quite hard to write an actual exploit though.

Comment 11

[LegNeato]

Comment on attachment 574204 [details] [diff] [review]
Patch v1

[triage comment]
Approved for beta and aurora. Please land as soon as possible.

Comment 12

[Chris Pearce [:cpearce (Not reading bugmail)]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/62676e1529c2

Comment 13

[Chris Pearce [:cpearce (Not reading bugmail)]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/62676e1529c2

Comment 14

[Chris Pearce [:cpearce (Not reading bugmail)]]

https://hg.mozilla.org/releases/mozilla-beta/rev/05255b6a0f42

Comment 15

[Daniel Veditz [:dveditz]]

The code being patched here doesn't exist on the 1.9.2 branch, I assume this bug is more recent then and doesn't affect older branches?

Comment 16

[u279076]

Testcase does not crash 2011-12-08 Nightly and Aurora, and Firefox 9.0b5

Comment 18

[Raymond Forbes[:rforbes]]

rforbes-bugspam-for-setting-that-bounty-flag-20130719