1495025 - VP9 Profile 2 10 bits fails to decode with hardware decoder

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect, P3)

Description

[Jean-Yves Avenard [:jya]]

Attachment: The World in HDR10.webm

Try to open a video with a machine that supports VP9 profile 2 10 bits (Intel 8th gen or nvidia 10xx and later)

You get a decoding error.

Reason for this is we always attempt to decode into a NV12 surface, and the decoder won't let you.

Comment 1

[Jean-Yves Avenard [:jya]]

Attachment: Bug 1495025 - P1. Search for alternative pixel format when stream change.

When decoding a vp9 profile 2 (10 bits), the MF_E_TRANSFORM_STREAM_CHANGE message is returned. We need to look for alternative format type other than NV12: 10/16 bits.

When using those formats, we can no longer assume that the D3D11ShareHandleImage can use NV12. So we will convert to RGBA32 on the fly via a MFT.

Comment 2

[Jean-Yves Avenard [:jya]]

Attachment: Bug 1495025 - P2. Use lambda for callback.

I find it easier to read than a function pointer making you search elsewhere to see what it's about

Depends on D7294

Comment 3

[Jean-Yves Avenard [:jya]]

Attachment: Bug 1495025 - P3. Store original IMFMediaType's subtype in D3D11SharedHandleImage.

This allows more easily the creation of the MFT required to convert to a RGBA32 image when doing a readback.

Depends on D7295

Comment 4

[Jean-Yves Avenard [:jya]]

Attachment: Bug 1495025 - P4. Add COLOR_16 type

Depends on D7296

Comment 5

[Jean-Yves Avenard [:jya]]

Attachment: Bug 1495025 - P5. Add Windows P010 and P016 support for software decoder

As we do not have an IMF nor D3D11 NV12 image, we always require a full copy of the data that will deinterleave the chroma channels.

Depends on D7316

Comment 6

[Chris Pearce [:cpearce (Not reading bugmail)]]

Comment on attachment 9013330 [details]
Bug 1495025 - P1. Search for alternative pixel format when stream change.

Chris Pearce (:cpearce) has approved the revision.

Comment 7

[Chris Pearce [:cpearce (Not reading bugmail)]]

Comment on attachment 9013331 [details]
Bug 1495025 - P2. Use lambda for callback.

Chris Pearce (:cpearce) has approved the revision.

Comment 8

[Chris Pearce [:cpearce (Not reading bugmail)]]

Comment on attachment 9013333 [details]
Bug 1495025 - P3. Store original IMFMediaType's subtype in D3D11SharedHandleImage.

Chris Pearce (:cpearce) has approved the revision.

Comment 9

[Chris Pearce [:cpearce (Not reading bugmail)]]

Comment on attachment 9013378 [details]
Bug 1495025 - P5. Add Windows P010 and P016 support for software decoder

Chris Pearce (:cpearce) has approved the revision.

Comment 10

[Nicolas Silva [:nical]]

Comment on attachment 9013371 [details]
Bug 1495025 - P4. Add COLOR_16 type

Nicolas Silva [:nical] has approved the revision.

Comment 11

[Jean-Yves Avenard [:jya]]

Attachment: Bug 1495025 - P6. Remove now unused paramater.

Depends on D7316

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/208624601a18
P1. Search for alternative pixel format when stream change. r=cpearce
https://hg.mozilla.org/integration/autoland/rev/c548d816019d
P2. Use lambda for callback. r=cpearce
https://hg.mozilla.org/integration/autoland/rev/c3b43ee1092e
P3. Store original IMFMediaType's subtype in D3D11SharedHandleImage. r=cpearce
https://hg.mozilla.org/integration/autoland/rev/25895d283d47
P4. Add COLOR_16 type r=nical
https://hg.mozilla.org/integration/autoland/rev/528dbc463c22
P5. Add Windows P010 and P016 support for software decoder r=cpearce
https://hg.mozilla.org/integration/autoland/rev/263d4f722174
P6. Remove now unused paramater. r=bryce

Comment 13

[Cristian Brindusan [:cbrindusan]]

Backed out 6 changesets (Bug 1495025) for mochitest-webgl2 failures in test_2_conformance2__textures__misc__npot-video-sizing.html.

Backout: https://hg.mozilla.org/integration/autoland/rev/a22ea1fdf4cb5eea00a73306d2b0d005e26d4a6c

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=pending,running,success,testfailed,busted,exception&revision=263d4f72217465543fc7e0ecda66b8db2cd5a1d9&selectedJob=203374400

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=203374400&repo=autoland&lineNumber=2373

Comment 14

[Jean-Yves Avenard [:jya]]

I can't reproduce that failure with identical patches.

I don't think it had any relations with that webgl failure.

https://treeherder.mozilla.org/#/jobs?repo=try&selectedJob=203847594&revision=c54ca4cb1c36244c4e1c36239c5dc5e23130f10f

I'll reland shortly

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/c62823871aca
P1. Search for alternative pixel format when stream change. r=cpearce
https://hg.mozilla.org/integration/mozilla-inbound/rev/f1afe7e2a9e3
P2. Use lambda for callback. r=cpearce
https://hg.mozilla.org/integration/mozilla-inbound/rev/7fd1f6103294
P3. Store original IMFMediaType's subtype in D3D11SharedHandleImage. r=cpearce
https://hg.mozilla.org/integration/mozilla-inbound/rev/9f59a50dcc6d
P4. Add COLOR_16 type r=nical
https://hg.mozilla.org/integration/mozilla-inbound/rev/68efa7588ba8
P5. Add Windows P010 and P016 support for software decoder r=cpearce
https://hg.mozilla.org/integration/mozilla-inbound/rev/24d67618f6b9
P6. Remove now unused paramater. r=bryce

Comment 16

[Narcis Beleuzu [:NarcisB]]

Backed out for webgl2 failures on test_2_conformance2__textures__misc__npot-video-sizing.html

Push link: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&resultStatus=testfailed,busted,exception,runnable&revision=24d67618f6b943e45d6d1131c1bc442274f3d330
Backout link: https://hg.mozilla.org/integration/mozilla-inbound/rev/9f2c6ae03c6d5cab74ec0965002edc5776911748
Log link: https://treeherder.mozilla.org/logviewer.html#?job_id=203868605&repo=mozilla-inbound&lineNumber=4806

Comment 17

[Jean-Yves Avenard [:jya]]

That test is invalid and based on wrong premises.

Yes, the encoded image is 1920x1080

however it has a display aspect ratio set at [SAR 171:176 DAR 19:11]

So it is to be displayed as 1920x1111.

In the test we read:
    // It was found that Firefox's videoHeight doesn't report the
    // correct answer for this video resource, though the uploaded
    // video texture is the correct size.
so videoHeight must be 1111, if it's not it's wrong.

Why that test ever passed is a mystery, it shouldn't

Where does that test come from?

Comment 18

[Jean-Yves Avenard [:jya]]

FWIW, Chrome displays it as 1920x1112
Edge displays it as 1865x1080

so all browsers have similar behaviour

Comment 19

[Jean-Yves Avenard [:jya]]

Attachment: Bug 1495025 - P7. Silence compilation warning

Comment 20

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/107add19c310
P1. Search for alternative pixel format when stream change. r=cpearce
https://hg.mozilla.org/integration/mozilla-inbound/rev/2851d7aead76
P2. Use lambda for callback. r=cpearce
https://hg.mozilla.org/integration/mozilla-inbound/rev/5e7ea76f73f3
P3. Store original IMFMediaType's subtype in D3D11SharedHandleImage. r=cpearce
https://hg.mozilla.org/integration/mozilla-inbound/rev/9818fa498ba5
P4. Add COLOR_16 type r=nical
https://hg.mozilla.org/integration/mozilla-inbound/rev/b88586cb4fdd
P5. Add Windows P010 and P016 support for software decoder r=cpearce
https://hg.mozilla.org/integration/mozilla-inbound/rev/982f91f69b39
P6. Remove now unused paramater. r=bryce

Comment 21

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/107add19c310
https://hg.mozilla.org/mozilla-central/rev/2851d7aead76
https://hg.mozilla.org/mozilla-central/rev/5e7ea76f73f3
https://hg.mozilla.org/mozilla-central/rev/9818fa498ba5
https://hg.mozilla.org/mozilla-central/rev/b88586cb4fdd
https://hg.mozilla.org/mozilla-central/rev/982f91f69b39

Comment 22

[Kelsey Gilbert [:jgilbert]]

(In reply to Jean-Yves Avenard [:jya] from comment #17)
> That test is invalid and based on wrong premises.
> 
> Yes, the encoded image is 1920x1080
> 
> however it has a display aspect ratio set at [SAR 171:176 DAR 19:11]
> 
> So it is to be displayed as 1920x1111.
> 
> In the test we read:
>     // It was found that Firefox's videoHeight doesn't report the
>     // correct answer for this video resource, though the uploaded
>     // video texture is the correct size.
> so videoHeight must be 1111, if it's not it's wrong.
> 
> Why that test ever passed is a mystery, it shouldn't
> 
> Where does that test come from?

As the path hints, it's from the webgl2 conformance tests.

Comment 23

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2a9f2418655a
P7. Silence compilation warning r=mattwoodrow

Comment 24

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/2a9f2418655a