Last Comment Bug 701259 - (CVE-2011-3665) crashes on videos when scaling to large width/height
: crashes on videos when scaling to large width/height
: verified-aurora, verified-beta
Product: Core
Classification: Components
Component: Audio/Video (show other bugs)
: unspecified
: x86_64 Linux
: -- normal (vote)
: mozilla11
Assigned To: Chris Pearce (:cpearce)
: Maire Reavy [:mreavy]
Depends on:
  Show dependency treegraph
Reported: 2011-11-09 18:09 PST by sczimmer
Modified: 2014-06-26 13:47 PDT (History)
5 users (show)
rforbes: sec‑bounty+
See Also:
Crash Signature:
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---

example website (140 bytes, text/html)
2011-11-09 18:09 PST, sczimmer
no flags Details
video to go with website (278.62 KB, text/plain)
2011-11-09 18:10 PST, sczimmer
no flags Details
backtrace for firefox 11 on 64-bit linux (19.84 KB, text/plain)
2011-11-09 18:16 PST, sczimmer
no flags Details
Patch v1 (1.49 KB, patch)
2011-11-13 17:06 PST, Chris Pearce (:cpearce)
roc: review+
christian: approval‑mozilla‑aurora+
christian: approval‑mozilla‑beta+
Details | Diff | Splinter Review

Description sczimmer 2011-11-09 18:09:48 PST
Created attachment 573399 [details]
example website

when visiting the website:
<video width="999999999" height="999999999" controls autoplay>
<source src="avideo.ogg" type="video/ogg">
</video><video controls autoplay>

firefox crashes in LinearScaleYUVToRGB32Row, an inline assembly function defined in gfx/ycbcr/yuv_row_posix.cpp
it looks like firefox is writing xmm1 to rgb_buf width times but width is too big so it writes xmm1 until it reaches unmapped memory and crashes then

I tested in firefox 5 on 32-bit linux and firefox 11 on 64-bit linux and it crashes as described above. I also tested in firefox 3.6 and it doesn't crash

marking as security because possibly exploitable memory corruption
Comment 1 sczimmer 2011-11-09 18:10:30 PST
Created attachment 573400 [details]
video to go with website
Comment 2 sczimmer 2011-11-09 18:16:29 PST
Created attachment 573401 [details]
backtrace for firefox 11 on 64-bit linux
Comment 3 Chris Pearce (:cpearce) 2011-11-13 17:06:37 PST
Created attachment 574204 [details] [diff] [review]
Patch v1

Reject attempts to scale a BasicPlanarYCbCrImage if width or height are larger than PlanarYCbCrImage::MAX_DIMENSION.
Comment 4 Matthew Gregan [:kinetik] 2011-11-13 17:22:35 PST
It seems like this should be rejected at a higher level.  mScaleHint is being set to a size computed from an area with width==height==nscoord_MAX, which seems bogus.
Comment 5 Chris Pearce (:cpearce) 2011-11-13 18:14:51 PST
What did you have in mind specifically, Matthew?
Comment 6 Robert O'Callahan (:roc) (email my personal email if necessary) 2011-11-13 19:58:25 PST
You're probably right, but defense in depth is fine here.
Comment 8 Ian Melven :imelven 2011-11-14 19:23:25 PST
from edmorley :
Comment 9 Ian Melven :imelven 2011-11-14 19:29:05 PST
(In reply to Ian Melven :imelven from comment #8)
> from edmorley :

scratch that, wrong bug - fixes this
Comment 10 Chris Pearce (:cpearce) 2011-11-14 19:45:13 PST
Comment on attachment 574204 [details] [diff] [review]
Patch v1

Requesting approval on Aurora and Beta because this is a crasher and it is potentially exploitable. I think it would be quite hard to write an actual exploit though.
Comment 11 christian 2011-11-15 13:38:52 PST
Comment on attachment 574204 [details] [diff] [review]
Patch v1

[triage comment]
Approved for beta and aurora. Please land as soon as possible.
Comment 12 Chris Pearce (:cpearce) 2011-11-15 16:41:17 PST
Comment 13 Chris Pearce (:cpearce) 2011-11-15 16:41:55 PST
Comment 14 Chris Pearce (:cpearce) 2011-11-15 18:04:55 PST
Comment 15 Daniel Veditz [:dveditz] 2011-12-07 14:41:18 PST
The code being patched here doesn't exist on the 1.9.2 branch, I assume this bug is more recent then and doesn't affect older branches?
Comment 16 Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2011-12-08 15:24:38 PST
Testcase does not crash 2011-12-08 Nightly and Aurora, and Firefox 9.0b5
Comment 18 Raymond Forbes[:rforbes] 2013-07-19 18:28:16 PDT

Note You need to log in before you can comment on or make changes to this bug.