1848193 - Make H264-avcc works on Windows

Status: RESOLVED FIXED

(Core :: Audio/Video: Web Codecs, task)

Description

[C.M.Chang[:chunmin]]

In WebCodecs, VideoDecoder is expected to return the error if decode fails, but decoding some corrupt H264 data is unexpectedly okay on Windows.

More specifically, this test creates a corrupt H264 data in AVCC format, and then try decoding it. The expected result is to get an error, but the expected error isn't returned from our decoder.

Input data must conform to Annex B of ISO/IEC 14496-10. The data must include the start codes. The decoder skips bytes until it finds a valid sequence parameter set (SPS) and picture parameter set (PPS) in the byte stream.

Since Windows's H264 decoder only accepts H264 data in Annex B format, H264ChangeMonitor will convert the the data into AnnexB format if the format is AVCC. What happens is that the conversion is unexpected okay, and surprisingly the converted data cause no issue to our decoder. The decoder will return MF_E_TRANSFORM_NEED_MORE_INPUT in the test mentioned above, but which is a normal signal asking for more data.

I think we should rework AnnexB::ConvertSampleToAnnexB so that we can catch the corrupt data during the conversion.

Comment 1

[C.M.Chang[:chunmin]]

Attachment: Bug 1848193 - Add debug logs for close

Comment 2

[C.M.Chang[:chunmin]]

Attachment: Bug 1848193 - Check H264 data when converting from AVCC to AnnexB

Depends on D185971

Comment 3

[C.M.Chang[:chunmin]]

Attachment: Bug 1848193 - Enable WebCodecs on Windows

Depends on D185972

Comment 4

[C.M.Chang[:chunmin]]

Attachment: Bug 1848193 - Change expectations in VideoDecoder's WPTs

Depends on D186040

Comment 5

[C.M.Chang[:chunmin]]

Attachment: Bug 1848193 - Skip conversion for 4-byte AVCC data

ConvertSampleTo4BytesAVCC converts all kinds of given AVCC data into
4-byte-header AVCC after varifying their data. However, if the header of
the given data is already 4-byte, there is no need to do the conversion.
The 4-byte-header AVCC data only need to check if the data is corrupt.

This patch addes a shortcut for 4-byte AVCC data to skip convesion
(replacing its internal data) and do the corrupt-check only.

Depends on D186041

Comment 6

[BugBot [:suhaib / :marco/ :calixte]]

There are some r+ patches which didn't land and no activity in this bug for 2 weeks.
:chunmin, could you have a look please?
If you still have some work to do, you can add an action "Plan Changes" in Phabricator.
For more information, please visit BugBot documentation.

Comment 7

[C.M.Chang[:chunmin]]

I just rebase the patches on top of the latest upstream and tested them on try: https://treeherder.mozilla.org/jobs?repo=try&revision=505c089a81cf7e439777d1f45cb85c314e74b97c&selectedTaskRun=Bq1pAIqcR5-H8ruqZKEfqw.0

The patches are going to be landed once the tests are green.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/bd64307bf3b5
Add debug logs for close r=padenot
https://hg.mozilla.org/integration/autoland/rev/65f68d603511
Check H264 data when converting from AVCC to AnnexB r=media-playback-reviewers,alwu
https://hg.mozilla.org/integration/autoland/rev/dbc5a6f310e5
Enable WebCodecs on Windows r=padenot
https://hg.mozilla.org/integration/autoland/rev/0a245641f43b
Change expectations in VideoDecoder's WPTs r=padenot
https://hg.mozilla.org/integration/autoland/rev/7cb9a33ef25b
Skip conversion for 4-byte AVCC data r=media-playback-reviewers,aosmond

Comment 9

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/bd64307bf3b5
https://hg.mozilla.org/mozilla-central/rev/65f68d603511
https://hg.mozilla.org/mozilla-central/rev/dbc5a6f310e5
https://hg.mozilla.org/mozilla-central/rev/0a245641f43b
https://hg.mozilla.org/mozilla-central/rev/7cb9a33ef25b