Bug 700208 (AreWePlayingYet)

AreWePlayingYet.org tracking bug

NEW
Unassigned

Status

()

defect
8 years ago
3 years ago

People

(Reporter: kinetik, Unassigned)

Tracking

(Blocks 1 bug)

Trunk
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [games], )

Reporter

Description

8 years ago
SoundCloud published an HTML 5 <audio> interoperability test.  We're failing 8 tests:

1. Seeking to unbuffered position with seamless playback
2. Supports MP3 format (bug 562730)
3. Consistent timeupdate interval (15ms - 250ms)
4. Property "played" (bug 462959)
5. Property "playbackRate" (bug 495040)
6. Property "defaultPlaybackRate" (bug 495040)
7. Property "currentTime"
8. Event "ratechange" (bug 495040)

Of the ones we don't already have bugs for:

1. Test is broken (serves unseekable Ogg and expects to seek, also could fail
   intermittently depending on media download speed).
3. Expects delta time between timeupdate events to be >= 15ms and <= 250ms, in Firefox
   the test sees a delta time of 260-285ms.  This is a real bug.
7. Test is broken (serves unseekable Ogg and expects to seek, can pass intermittently
   depending on media download speed).
Reporter

Updated

8 years ago
Depends on: 700210
Reporter

Comment 1

8 years ago
Filed bug 700210 for #3.

I've attempted to contact the owners about the broken tests.
Reporter

Comment 2

8 years ago
When I say "unseekable Ogg", what I mean is that the server does not support range requests, so Firefox is only able to seek in buffered parts of the file.

#1 is unreliable because it attempts to seek to audio.buffered.end(0) + 60, but depending on download speed it's possible for audio.buffered.end(0) to equal audio.duration, so the test not complete.

#7 is unreliable because it can intermittently pass depending on media download speed, as a seek into a buffered area will succeed but a seek into an unbuffered area will fail due to the first issue mention in this comment.
Reporter

Comment 3

8 years ago
Additionally, it's not clear to me what #1 intends to test, but seeking to an unbuffered area will not be seamless as there will be network and decoding delays involved in the seek.

If it's expected that the old playback location continues to play until the seek completes, this isn't the case in Firefox nor in any implementation I'm aware of.  The test doesn't (and can't) detect this behaviour, anyway.

What the test appears to be checking is whether playback continues automatically after seeking into an unbuffered area--if this is the intended action the test should probably be renamed to reflect what is being tested.

Comment 4

8 years ago
Hi Matthew!

I must say you caught us by surprise since we're today doing a final work marathon on this project to be released tomorrow.
Anyway your input is very valuable and the Github repo (http://github.com/soundcloud/areweplayingyet) is open now. If you have any suggestions to improve a test just create a Pull Request.

Going to fix some tests now! ;)
After running the full test on Firefox Beta 8 (beta channel), the count is still at 0 at the bottom of website.

Comment 6

8 years ago
Antoine as I wrote above, you shouldn't consider anything final until Wednesday. Thank you for your feedback.
Reporter

Comment 7

8 years ago
The seeking issue is resolved, so #7 now passes.  I think #1 should pass now, but I suspect the test is timing out before it has a chance to complete.  The tests have a default timeout of 10s, which may be too low when running some of them over a slowish international link.

Two new tests are failing: AAC support (we don't, no bug to reference, but the reasoning is roughly the same as for MP3), and 'stalled' (which should pass, not sure what's going on there).

Comment 8

8 years ago
Hi Matthew!

Let's talk about the #1. Do you understand the use case? If implemented right, it allows fast playback of _any_ part of an audio file (without the need to buffer everything until that point) and then if no more user interaction happens it should continue playback smoothly.

About AAC support... You should now more about that than me but didn't it go open source recently?

The stalled event is not working on Firefox... The loadstart event is triggered, then we halt the transfer of data for 3500ms (the spec requires 3000 but we gave some space) where stalled should be triggered.

Cheers,
Tomás
Apple release their ALAC (Apple Lossless Audio Codec, equivalent to FLAC) under an open source license, but AFAIK, AAC is still encumbered with patents.

Comment 10

8 years ago
@Paul Do you think ALAC could be a good way of erradicating the problem of having different audio formats for different browsers?
Tomas, ALAC is unfortunately a lossless audio codec, and is intended for very high quality/high bitrate streams, so it might not be very useful for our purpose, which is streaming media.

Comment 12

8 years ago
@Paul, acknowledged. So what are your ideas on how this could be solved?

Comment 13

8 years ago
@Matthew: The first test was indeed timing out. Have look at it now! http://areweplayingyet.org/support-seeking-unbuffered-position
Tomás, the codec issue is hard. Apple and MS could support Ogg like Google, Opera, and Mozilla. In some years the patents on mp3 will expire and we can add that to Firefox. A shim that does audio decompression in js might work in the meantime.

Fortunately, the <audio> element lets you specify multiple <source> elements!

Comment 15

8 years ago
Can anyone look at this test source and validate that is valid (passing or not)? Webkit people are saying that seeking should only happen after the canplay event was triggered but this makes Firefox fail.

Comment 17

8 years ago
It _made_ Firefox fail because canplay is triggered again after the seek so there would be an infinite loop. Passing now.
Reporter

Comment 18

8 years ago
Requiring the canplay event before seeking corresponds to HAVE_FUTURE_DATA.  It should be possible to seek as soon as you reach HAVE_METADATA (and receive the loadedmetadata event).

Comment 19

8 years ago
Matthew can you please link me to the spec part that defines that? Thanks
Reporter

Comment 20

8 years ago
Here: http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#seeking

Note that it's an error to seek while in readyState HAVE_NOTHING.  All other states are valid, including HAVE_METADATA.

Here: http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#the-ready-states

Under HAVE_METADATA: "The API will no longer raise an exception when seeking."

Also note in that section that canplay is fired when the readyState is HAVE_FUTURE_DATA.

So this change is wrong, and should be reverted: https://github.com/soundcloud/areweplayingyet/commit/37d10a57d420807cc6838f8e94d56f5198b01798
Reporter

Comment 21

8 years ago
It's probably better to have these discussions in the AreWePlayingYet issue tracker so they're more visible to other interested parties.  This bug is intended to track Firefox bugs, and also track notes on the tests before the AWPY issue tracker was opened up.  I'll file AWPY test bugs there from now on.
Reporter

Updated

8 years ago
Depends on: 702870
Reporter

Comment 22

8 years ago
Loop test fails on Linux (with PulseAudio) due to bug 582513.
Depends on: 582513
Alias: AreWePlayingYet
Reporter

Comment 24

7 years ago
The failing tests need to be reviewed to confirm they're valid tests, and if so, why they're failing.  That can be done separately for each test if someone would like to file individual bugs that block this one.
Whiteboard: [games]

Comment 25

7 years ago
Hey guys,
We made minors update to AWPY:
 - Introduced check for Opus support http://areweplayingyet.org/support-format-opus :)
 - Split the test 'support-seeking-unbuffered-position' into 3 tests to take into account different codecs and removed the former one. (you can read more about the motivations: https://github.com/soundcloud/areweplayingyet/pull/41)
  * http://areweplayingyet.org/support-seeking-unbuffered-position-ogg
  * http://areweplayingyet.org/support-seeking-unbuffered-position-mp3
  * http://areweplayingyet.org/support-seeking-unbuffered-position-aac

We will also introduce an equivalent test for Opus.
Or if you want to contribute you can propose a patch :)

Comment 26

7 years ago
In this day, 
Firefox nightly 20.0a1 (2012-11-26) have 42 test passed/47. Firefox fail for:
* Seeking to unbuffered position with continuous playback after seeking (MP3)
* Seeking to unbuffered position with continuous playback after seeking (AAC)
* Supports MP3 format
* Supports AAC format
=> Normal: problems with patent. It's work if you choose Ogg.
* Event "stalled"  I can't find a bug for it in bigzilla.


Firefox 17.0 have 39/47. The difference it's for:
* Property "playbackRate"
* Property "defaultPlaybackRate"
* Event "ratechange"

Comment 27

7 years ago
(In reply to vulcain from comment #26)
> * Event "stalled"  I can't find a bug for it in bigzilla.
I think it's bug 702870.

Comment 28

7 years ago
Firefox 17 on android 4.1 have 38/47:
* Seeking to unbuffered position with continuous playback after seeking (AAC)
* Seeking to unbuffered position with continuous playback after seeking (MP3)
* Seeking to unbuffered position with continuous playback after seeking (Ogg)
* Support MP3 format
* Consistent time between play() and actual playback (< 20ms)
* Property "playbackRate"
* Property "defaultPlaybackRate"
* Event "stalled"
* Event "ratechange"

with Firefox 20.0a  nightly on Android 4.1 :42/47
* Seeking to unbuffered position with continuous playback after seeking (AAC)
* Seeking to unbuffered position with continuous playback after seeking (Ogg)
* Support MP3 format
* Consistent time between play() and actual playback (< 20ms)
* Event "stalled"

Comment 29

7 years ago
For information on Linux 3.2.0-34-generic x86_64:
Opera 12.11 have 39/47 (fail for seek all format, don't support Opus, mp3, aac; Event "abort"; Event "stalled", Seeking to unbuffered position with continuous playback after seeking with ogg, mp3, aac). 
Chromium 20.0.1132.47 have 19/47
Google Chrome 23.0.1271.91 have 43/47 and fail for Event "stalled" and Seeking to unbuffered position with continuous playback after seeking for mp3, aac, Ogg. Amazing, i don't remember that Chrome could play opus codec.

On Android 4.1, Opera mobile 12.10 have 35/47 Chrome have 29/47, Dolphin  Browser 9.0.3 have 29/47, Android Browser 4.1.2 have 29/47.
Component: Audio/Video → Audio/Video: Playback
With Firefox 48 on Windows 10:
* Event "stalled"
* Event "abort"

None of the previous reports mention this "abort" failure, I am not sure if it is a new test or a regression.
Firefox 48 on android 4.4 have 42/47:
* Consistent time between play() and actual playback (< 20ms)
* Property "buffered"
* Event "stalled"
* Event "canplaythrough"
* Event "abort"

None of the previous Android reports mention these "buffered", "canplaythrough" or "abort" failures, I am not sure if they are new tests or regressions.
[update] With Firefox 48 to 51 on Windows 10 we have 44/47:
* Property "buffered" (I missed this failure on #c30 above)
* Event "stalled"
* Event "abort"

None of the previous Android reports mention these "buffered", or "abort" failures, I am not sure if they are new tests or regressions.
You need to log in before you can comment on or make changes to this bug.