Pass the YouTube MSE conformance testsuite

RESOLVED FIXED

Status

()

defect
P3
normal
RESOLVED FIXED
6 years ago
4 years ago

People

(Reporter: kinetik, Unassigned)

Tracking

(Blocks 1 bug)

Trunk
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(URL)

Attachments

(1 attachment, 1 obsolete attachment)

(Reporter)

Description

6 years ago
This will end up being a tracking bug for various failing tests.  Right now with the patch in bug 855130 we're passing the first 21 or so tests and a handful of later ones, but not all of those are valid passes (e.g. CanPlayClearKey etc. pass despite there being no support for that functionality).
(Reporter)

Comment 1

6 years ago
Posted patch hacks v0 (obsolete) — Splinter Review
Some required (hacky) changes for passing the first handful of tests:

* avc1.640008 seems bogus, 0x08 doesn't map to an official AVC level as far as I can tell.  Will follow up with test authors.
* mp4a.40.5 is for AAC-HC, which I'm not sure if we plan/intend to support.
* SourceBuffer#append is from an older version of the spec.  Will follow up with test authors.
(Reporter)

Updated

6 years ago
Depends on: 896883
(Reporter)

Updated

6 years ago
Depends on: 896884
No longer blocks: 1031965
(Reporter)

Comment 2

5 years ago
Comment on attachment 779041 [details] [diff] [review]
hacks v0

This patch is no longer necessary (at least, if you test "tip of tree" and select webm/vp9
Attachment #779041 - Attachment is obsolete: true
(Reporter)

Updated

5 years ago
Depends on: 1064128
(Reporter)

Comment 3

5 years ago
Failing as of today, ignoring everything after test 58 as it's EME rather than MSE:

18 - Bug 1064128 - SourceBuffer::timestampOffset unimplemented
19 - Bug 1064128 - SourceBuffer::timestampOffset unimplemented
36 - Bug 1064128 - SourceBuffer::timestampOffset unimplemented

27 - Times out waiting for media element's ended event after append/abort/duration/EOS/play sequence

29 - Fails with "Estimated source buffer size is (2) which should be greater than or equal to (12)"

31 - Times out, playback doesn't start
32 - Times out, playback doesn't start

38 - Tries to addSourceBuffer(audio/mp4; codecs="mp4a.40.2") and fails
39 - Tries to addSourceBuffer(video/mp4; codecs="avc1.640028") and fails

42 - Bug 896884 - MediaSource::duration not fully implemented

43 - audio splicing/etc. unimplemented
44 - audio splicing/etc. unimplemented
45 - audio splicing/etc. unimplemented

48 - Times out while waiting for a seek to complete
49 - Times out while waiting for a seek to complete

55 - Bug 238041 - Event timestamp epoch (was bug 77992, but that mutated)

58 - InvalidStateError setting xhr.responseType to "arraybuffer" before calling Open()

Updated

5 years ago
Depends on: 1096089
Depends on: 1113282
Depends on: 1113370
Depends on: 1116643
Depends on: 1116644
Depends on: 1116645
Depends on: 1116647
Depends on: 1116648
Depends on: 1116649
Depends on: 1116650
Depends on: 1116651
Depends on: 1116653
Depends on: 1116655
Depends on: 1116657
Depends on: 1116658
Depends on: 1116659
Depends on: 1116661
Summary: Pass the MSE conformance testsuite → Pass the YouTube MSE conformance testsuite
No longer depends on: 1096089
Depends on: 1130263
Depends on: 1130265
As of Today (Friday, 6th 2015) with bug 1125776 (and upcoming bug 1129732) we pass all tests but
29. VideoBufferSize (bug 1130265)
42. MediaSourceDuration (bug 1116651)
47. Seek (intermittent bug 1130263)
Depends on: 1132832
All tests are now green.
Depends on: 1133396
Depends on: 1133397
Depends on: 1133408

Comment 6

4 years ago
FWIW I'm still seeing failures on the latest-tip MSE/2015 tests #48 and #54 with the latest nightly 2015-02-16 on Windows 8.1 x64.
Are those intermittent failures or they always occur?
If a test fails, all subsequent tests will show as "red", so to determine if a test did fail, you need to check the logs or reload the page.
Flags: needinfo?(bobsfree)

Comment 8

4 years ago
Win7.1 x64 and 2015-02-16 nightly.

In test #48: Always becomes TIMED OUT.

In test #54: Shows a failure like below.
EventTimestamp:    0: (0, 126.409433) 
EventTimestamp:  sb1.buffered.length 1 
EventTimestamp:    0: (0, 43.877422) 
EventTimestamp:  sb0.buffered.length 1 
EventTimestamp:  ms.sb count = 2 
EventTimestamp:  video.networkState = 2 
EventTimestamp:  video.readyState = 3 
EventTimestamp:  video.currentTime = 0.119954 
TestRunner:  Assert failed: event.timeStamp is (2253.7928867624596) which should be greater than or equal to (1424172586756) 
TestRunner:  checkEq passed: XHR length is (500000). 
TestRunner:  checkNE passed: request index is (0). 
TestRunner:  checkEq passed: XHR requestType is (GET). 
TestRunner:  checkEq passed: request index is (-1). 
TestRunner:  checkEq passed: XHR length is (1500000). 
TestRunner:  checkNE passed: request index is (0). 
TestRunner:  checkEq passed: XHR requestType is (GET). 
TestRunner:  checkEq passed: request index is (-1). 
TestRunner:  checkGr passed: Date.now() is (1424172586756). 
EventTimestamp:  MS test started 
TestRunner:  Starting test 54:EventTimestamp with timeout 30000

And #32 and #33 intermittently become TIMED OUT

Comment 9

4 years ago
Don't know if anybody has noticed it but the timer is wrong.
I noticed this issue on some YouTube videos but couldn't repeat it.

Comment 10

4 years ago
(In reply to Jean-Yves Avenard [:jya] from comment #7)
> Are those intermittent failures or they always occur?
> If a test fails, all subsequent tests will show as "red", so to determine if
> a test did fail, you need to check the logs or reload the page.

I can't seem to reproduce the failures on test #48 with Nightly 2015-02-18 in safe mode anymore, so it is possible these are intermittent, involved extensions or were resolved. However, test #54 consistently fails every time in safe mode.

EventTimestamp:    0: (0, 126.409433) 
EventTimestamp:  sb1.buffered.length 1 
EventTimestamp:    0: (0, 43.877422) 
EventTimestamp:  sb0.buffered.length 1 
EventTimestamp:  ms.sb count = 2 
EventTimestamp:  video.networkState = 2 
EventTimestamp:  video.readyState = 3 
EventTimestamp:  video.currentTime = 0.109931 
TestRunner:  Assert failed: event.timeStamp is (312850.103492221) which should be greater than or equal to (1424322375489) 
TestRunner:  checkEq passed: XHR length is (500000). 
TestRunner:  checkNE passed: request index is (0). 
TestRunner:  checkEq passed: XHR requestType is (GET). 
TestRunner:  checkEq passed: request index is (-1). 
TestRunner:  checkEq passed: XHR length is (1500000). 
TestRunner:  checkNE passed: request index is (0). 
TestRunner:  checkEq passed: XHR requestType is (GET). 
TestRunner:  checkEq passed: request index is (-1). 
TestRunner:  checkGr passed: Date.now() is (1424322375489). 
EventTimestamp:  MS test started 
TestRunner:  Starting test 54:EventTimestamp with timeout 30000
Flags: needinfo?(bobsfree)

Updated

4 years ago
Priority: P2 → P3

Updated

4 years ago
Depends on: 1163617

Updated

4 years ago
Depends on: 1163618

Updated

4 years ago
Depends on: 1163620

Updated

4 years ago
Depends on: 1163621

Updated

4 years ago
Depends on: 1163622

Updated

4 years ago
Depends on: 1163623

Updated

4 years ago
Depends on: 1163624

Updated

4 years ago
Depends on: 1163625

Updated

4 years ago
Depends on: 1163626

Updated

4 years ago
Depends on: 1163619
all those bugs are tracking bugs.. they aren't designed to be closed :(
Component: Audio/Video → Audio/Video: Playback
Closing this now as we pass all tests (the duration tests are invalid and what they are testing is invalid: the buffered range is only ever updated asynchronously, so testing the buffered range right after changing the duration is invalid)
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.