1304999 - Crash in mozilla::layers::AttemptVideoScale

Status: RESOLVED FIXED

(Core :: Graphics: Layers, defect, P3)

Description

[[:philipp]]

This bug was filed from the Socorro interface and is 
report bp-1d61ae89-21d0-4711-be1b-851742160923.
=============================================================
Crashing Thread (21)
Frame 	Module 	Signature 	Source
0 	xul.dll 	mozilla::layers::AttemptVideoScale 	gfx/layers/basic/BasicCompositor.cpp:445
1 	xul.dll 	mozilla::layers::BasicCompositor::DrawQuad(mozilla::gfx::RectTyped<mozilla::gfx::UnknownUnits, float> const&, mozilla::gfx::IntRectTyped<mozilla::gfx::UnknownUnits> const&, mozilla::layers::EffectChain const&, float, mozilla::gfx::Matrix4x4Typed<mozilla::gfx::UnknownUnits, mozilla::gfx::UnknownUnits> const&, mozilla::gfx::RectTyped<mozilla::gfx::UnknownUnits, float> const&) 	gfx/layers/basic/BasicCompositor.cpp:555
2 		@0x13 	
3 	xul.dll 	mozilla::layers::RenderLayers<mozilla::layers::ContainerLayerComposite>(mozilla::layers::ContainerLayerComposite*, mozilla::layers::LayerManagerComposite*, mozilla::gfx::IntRectTyped<mozilla::RenderTargetPixel> const&) 	gfx/layers/composite/ContainerLayerComposite.cpp:627
4 	xul.dll 	mozilla::layers::RenderIntermediate<mozilla::layers::ContainerLayerComposite>(mozilla::layers::ContainerLayerComposite*, mozilla::layers::LayerManagerComposite*, mozilla::gfx::IntRectTyped<mozilla::gfx::UnknownUnits> const&, RefPtr<mozilla::layers::CompositingRenderTarget>) 	gfx/layers/composite/ContainerLayerComposite.cpp:736
5 	xul.dll 	mozilla::layers::ContainerPrepare<mozilla::layers::ContainerLayerComposite>(mozilla::layers::ContainerLayerComposite*, mozilla::layers::LayerManagerComposite*, mozilla::gfx::IntRectTyped<mozilla::RenderTargetPixel> const&) 	gfx/layers/composite/ContainerLayerComposite.cpp:434
6 	xul.dll 	mozilla::layers::ContainerLayerComposite::Prepare(mozilla::gfx::IntRectTyped<mozilla::RenderTargetPixel> const&) 	gfx/layers/composite/ContainerLayerComposite.cpp:872
7 	xul.dll 	mozilla::layers::ContainerPrepare<mozilla::layers::ContainerLayerComposite>(mozilla::layers::ContainerLayerComposite*, mozilla::layers::LayerManagerComposite*, mozilla::gfx::IntRectTyped<mozilla::RenderTargetPixel> const&) 	gfx/layers/composite/ContainerLayerComposite.cpp:401
8 	xul.dll 	mozilla::layers::ContainerLayerComposite::Prepare(mozilla::gfx::IntRectTyped<mozilla::RenderTargetPixel> const&) 	gfx/layers/composite/ContainerLayerComposite.cpp:872
9 	xul.dll 	mozilla::layers::ContainerPrepare<mozilla::layers::ContainerLayerComposite>(mozilla::layers::ContainerLayerComposite*, mozilla::layers::LayerManagerComposite*, mozilla::gfx::IntRectTyped<mozilla::RenderTargetPixel> const&) 	gfx/layers/composite/ContainerLayerComposite.cpp:401
10 	xul.dll 	mozilla::layers::ContainerLayerComposite::Prepare(mozilla::gfx::IntRectTyped<mozilla::RenderTargetPixel> const&) 	gfx/layers/composite/ContainerLayerComposite.cpp:872
11 	xul.dll 	mozilla::layers::LayerManagerComposite::Render(mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&) 	gfx/layers/composite/LayerManagerComposite.cpp:934
12 	xul.dll 	mozilla::layers::LayerManagerComposite::UpdateAndRender() 	gfx/layers/composite/LayerManagerComposite.cpp:478
13 	xul.dll 	mozilla::layers::LayerManagerComposite::EndTransaction(mozilla::TimeStamp const&, mozilla::layers::LayerManager::EndTransactionFlags) 	gfx/layers/composite/LayerManagerComposite.cpp:399
14 	xul.dll 	mozilla::layers::CompositorBridgeParent::CompositeToTarget(mozilla::gfx::DrawTarget*, mozilla::gfx::IntRectTyped<mozilla::gfx::UnknownUnits> const*) 	gfx/layers/ipc/CompositorBridgeParent.cpp:1260
15 	xul.dll 	mozilla::layers::CompositorVsyncScheduler::ComposeToTarget(mozilla::gfx::DrawTarget*, mozilla::gfx::IntRectTyped<mozilla::gfx::UnknownUnits> const*) 	gfx/layers/ipc/CompositorBridgeParent.cpp:590
16 	xul.dll 	mozilla::layers::CompositorVsyncScheduler::Composite(mozilla::TimeStamp) 	gfx/layers/ipc/CompositorBridgeParent.cpp:491

this is a crashing signature on windows appearing to happen within new code-path from bug 1266491 that seems to affect pre-release builds starting beginning with firefox 50 so far.
although there has been one single crash with 49.0b1 as well: https://crash-stats.mozilla.com/report/index/e0a609e0-f3dc-4b6c-903d-ca9752160804

Comment 1

[Milan Sreckovic [:milan] (needinfo for best results)]

WrappingTextureSourceYCbCrBasic::GetSurface returns nullptr?

Comment 2

[Sotaro Ikeda [:sotaro]]

(In reply to Milan Sreckovic [:milan] from comment #1)
> WrappingTextureSourceYCbCrBasic::GetSurface returns nullptr?

Yea, it seems like nullptr access.

Comment 3

[Sotaro Ikeda [:sotaro]]

Bug 1254011 seems to affect to the crash. It defer DataSourceSurface allocation and YCbCr color conversion.

Comment 4

[Sotaro Ikeda [:sotaro]]

(In reply to Sotaro Ikeda [:sotaro] from comment #3)
> Bug 1254011 seems to affect to the crash. It defer DataSourceSurface
> allocation and YCbCr color conversion.

In oom situation, DataSourceSurface allocation could fail.

Comment 5

[Sotaro Ikeda [:sotaro]]

Attachment: patch - Add nullptr check of YCbCr to rgb surface

Comment 6

[Sotaro Ikeda [:sotaro]]

Attachment: patch - Add nullptr check of YCbCr to rgb surface

Comment 7

[Sotaro Ikeda [:sotaro]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=1aa20b6e2482

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/4980a2aea6fb
Add nullptr check of YCbCr to rgb surface r=jrmuizel

Comment 9

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/4980a2aea6fb

Comment 10

[Sylvestre Ledru [:Sylvestre]]

Sotaro, as this is impacting 50 & 51, do you want to uplift that? 
If yes, could you fill the uplift request? Thanks

Comment 11

[Sotaro Ikeda [:sotaro]]

Attachment: patch for gecko50 - Add nullptr check of YCbCr to rgb surface

Comment 12

[Sotaro Ikeda [:sotaro]]

Comment on attachment 8795982 [details] [diff] [review]
patch - Add nullptr check of YCbCr to rgb surface

Approval Request Comment
[Feature/regressing bug #]: bug 1254011
[User impact if declined]: Could cause a crash during out of memory and BasicCompositor(no gpu acceleration).
[Describe test coverage new/current, TreeHerder]:Tested locally.
[Risks and why]: Low. The patch just add checking of nullptr.
[String/UUID change made/needed]: none

Comment 13

[Sylvestre Ledru [:Sylvestre]]

Comment on attachment 8795982 [details] [diff] [review]
patch - Add nullptr check of YCbCr to rgb surface

Fix a crash, seems low risk, taking it.
Should be 50 beta 5.

Comment 14

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/57ad439e58a9

Comment 15

[Carsten Book [:Tomcat]]

 https://hg.mozilla.org/releases/mozilla-beta/rev/37417e763286faffc6d50d93fe0fee3da795dee7

Comment 16

[BugBot [:suhaib / :marco/ :calixte]]

Crash volume for signature 'mozilla::layers::AttemptVideoScale':
 - nightly (version 52): 0 crashes from 2016-09-19.
 - aurora  (version 51): 1 crash from 2016-09-19.
 - beta    (version 50): 935 crashes from 2016-09-20.
 - release (version 49): 1 crash from 2016-09-05.
 - esr     (version 45): 0 crashes from 2016-07-25.

Crash volume on the last weeks (Week N is from 10-17 to 10-23):
            W. N-1  W. N-2  W. N-3  W. N-4
 - nightly       0       0       0       0
 - aurora        0       0       1       0
 - beta         42     410     387      90
 - release       1       0       0       0
 - esr           0       0       0       0

Affected platform: Windows

Crash rank on the last 7 days:
           Browser     Content   Plugin
 - nightly
 - aurora
 - beta    #869
 - release
 - esr

Comment 17

[Sotaro Ikeda [:sotaro]]

crash of firefox49 seems to be caused by a different problem.
 > report 701fd78b-c004-44ef-9710-18bbd2161011 

place of crash is different and bug 1254011 fix does not exit in firefox49, only bug 1266491 exists.

Comment 18

[Sotaro Ikeda [:sotaro]]

(In reply to Sotaro Ikeda [:sotaro] from comment #17)
> crash of firefox49 seems to be caused by a different problem.
>  > report 701fd78b-c004-44ef-9710-18bbd2161011 

  https://crash-stats.mozilla.com/report/index/701fd78b-c004-44ef-9710-18bbd2161011