Closed Bug 881514 Opened 11 years ago Closed 9 years ago

Pass the YouTube MSE conformance testsuite

Categories

(Core :: Audio/Video: Playback, defect, P3)

defect

Tracking

()

RESOLVED FIXED

People

(Reporter: kinetik, Unassigned)

References

(Blocks 1 open bug, )

Details

Attachments

(1 file, 1 obsolete file)

This will end up being a tracking bug for various failing tests.  Right now with the patch in bug 855130 we're passing the first 21 or so tests and a handful of later ones, but not all of those are valid passes (e.g. CanPlayClearKey etc. pass despite there being no support for that functionality).
Attached patch hacks v0 (obsolete) — Splinter Review
Some required (hacky) changes for passing the first handful of tests:

* avc1.640008 seems bogus, 0x08 doesn't map to an official AVC level as far as I can tell.  Will follow up with test authors.
* mp4a.40.5 is for AAC-HC, which I'm not sure if we plan/intend to support.
* SourceBuffer#append is from an older version of the spec.  Will follow up with test authors.
Depends on: 896883
Depends on: 896884
No longer blocks: 1031965
Comment on attachment 779041 [details] [diff] [review]
hacks v0

This patch is no longer necessary (at least, if you test "tip of tree" and select webm/vp9
Attachment #779041 - Attachment is obsolete: true
Depends on: 1064128
Failing as of today, ignoring everything after test 58 as it's EME rather than MSE:

18 - Bug 1064128 - SourceBuffer::timestampOffset unimplemented
19 - Bug 1064128 - SourceBuffer::timestampOffset unimplemented
36 - Bug 1064128 - SourceBuffer::timestampOffset unimplemented

27 - Times out waiting for media element's ended event after append/abort/duration/EOS/play sequence

29 - Fails with "Estimated source buffer size is (2) which should be greater than or equal to (12)"

31 - Times out, playback doesn't start
32 - Times out, playback doesn't start

38 - Tries to addSourceBuffer(audio/mp4; codecs="mp4a.40.2") and fails
39 - Tries to addSourceBuffer(video/mp4; codecs="avc1.640028") and fails

42 - Bug 896884 - MediaSource::duration not fully implemented

43 - audio splicing/etc. unimplemented
44 - audio splicing/etc. unimplemented
45 - audio splicing/etc. unimplemented

48 - Times out while waiting for a seek to complete
49 - Times out while waiting for a seek to complete

55 - Bug 238041 - Event timestamp epoch (was bug 77992, but that mutated)

58 - InvalidStateError setting xhr.responseType to "arraybuffer" before calling Open()
Depends on: 1097381
Depends on: 1096089
Depends on: 1113282
Depends on: 1113370
Depends on: 1116643
Depends on: 1116644
Depends on: 1116645
Depends on: 1116647
Depends on: 1116648
Depends on: 1116649
Depends on: 1116650
Depends on: 1116651
Depends on: 1116653
Depends on: 1116655
Depends on: 1116657
Depends on: 1116658
Depends on: 1116659
Depends on: 1116661
Summary: Pass the MSE conformance testsuite → Pass the YouTube MSE conformance testsuite
No longer depends on: 1096089
Depends on: 1130263
Depends on: 1130265
As of Today (Friday, 6th 2015) with bug 1125776 (and upcoming bug 1129732) we pass all tests but
29. VideoBufferSize (bug 1130265)
42. MediaSourceDuration (bug 1116651)
47. Seek (intermittent bug 1130263)
Depends on: 1132832
All tests are now green.
Depends on: 1133396
Depends on: 1133397
Depends on: 1133408
FWIW I'm still seeing failures on the latest-tip MSE/2015 tests #48 and #54 with the latest nightly 2015-02-16 on Windows 8.1 x64.
Are those intermittent failures or they always occur?
If a test fails, all subsequent tests will show as "red", so to determine if a test did fail, you need to check the logs or reload the page.
Flags: needinfo?(bobsfree)
Win7.1 x64 and 2015-02-16 nightly.

In test #48: Always becomes TIMED OUT.

In test #54: Shows a failure like below.
EventTimestamp:    0: (0, 126.409433) 
EventTimestamp:  sb1.buffered.length 1 
EventTimestamp:    0: (0, 43.877422) 
EventTimestamp:  sb0.buffered.length 1 
EventTimestamp:  ms.sb count = 2 
EventTimestamp:  video.networkState = 2 
EventTimestamp:  video.readyState = 3 
EventTimestamp:  video.currentTime = 0.119954 
TestRunner:  Assert failed: event.timeStamp is (2253.7928867624596) which should be greater than or equal to (1424172586756) 
TestRunner:  checkEq passed: XHR length is (500000). 
TestRunner:  checkNE passed: request index is (0). 
TestRunner:  checkEq passed: XHR requestType is (GET). 
TestRunner:  checkEq passed: request index is (-1). 
TestRunner:  checkEq passed: XHR length is (1500000). 
TestRunner:  checkNE passed: request index is (0). 
TestRunner:  checkEq passed: XHR requestType is (GET). 
TestRunner:  checkEq passed: request index is (-1). 
TestRunner:  checkGr passed: Date.now() is (1424172586756). 
EventTimestamp:  MS test started 
TestRunner:  Starting test 54:EventTimestamp with timeout 30000

And #32 and #33 intermittently become TIMED OUT
Don't know if anybody has noticed it but the timer is wrong.
I noticed this issue on some YouTube videos but couldn't repeat it.
(In reply to Jean-Yves Avenard [:jya] from comment #7)
> Are those intermittent failures or they always occur?
> If a test fails, all subsequent tests will show as "red", so to determine if
> a test did fail, you need to check the logs or reload the page.

I can't seem to reproduce the failures on test #48 with Nightly 2015-02-18 in safe mode anymore, so it is possible these are intermittent, involved extensions or were resolved. However, test #54 consistently fails every time in safe mode.

EventTimestamp:    0: (0, 126.409433) 
EventTimestamp:  sb1.buffered.length 1 
EventTimestamp:    0: (0, 43.877422) 
EventTimestamp:  sb0.buffered.length 1 
EventTimestamp:  ms.sb count = 2 
EventTimestamp:  video.networkState = 2 
EventTimestamp:  video.readyState = 3 
EventTimestamp:  video.currentTime = 0.109931 
TestRunner:  Assert failed: event.timeStamp is (312850.103492221) which should be greater than or equal to (1424322375489) 
TestRunner:  checkEq passed: XHR length is (500000). 
TestRunner:  checkNE passed: request index is (0). 
TestRunner:  checkEq passed: XHR requestType is (GET). 
TestRunner:  checkEq passed: request index is (-1). 
TestRunner:  checkEq passed: XHR length is (1500000). 
TestRunner:  checkNE passed: request index is (0). 
TestRunner:  checkEq passed: XHR requestType is (GET). 
TestRunner:  checkEq passed: request index is (-1). 
TestRunner:  checkGr passed: Date.now() is (1424322375489). 
EventTimestamp:  MS test started 
TestRunner:  Starting test 54:EventTimestamp with timeout 30000
Flags: needinfo?(bobsfree)
Priority: P2 → P3
Depends on: 1163617
Depends on: 1163618
Depends on: 1163620
Depends on: 1163621
Depends on: 1163622
Depends on: 1163623
Depends on: 1163624
Depends on: 1163625
Depends on: 1163626
Depends on: 1163619
all those bugs are tracking bugs.. they aren't designed to be closed :(
Component: Audio/Video → Audio/Video: Playback
Closing this now as we pass all tests (the duration tests are invalid and what they are testing is invalid: the buffered range is only ever updated asynchronously, so testing the buffered range right after changing the duration is invalid)
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: