1553215 - Throw when the MediaStream passed to MediaStreamAudioSourceNode does not have an audio track, and implement the correct behaviour

Status: RESOLVED FIXED

(Core :: Web Audio, defect, P3)

Description

[Paul Adenot (:padenot)]

We need to take the first track alphabetically, basically.

Comment 1

[Paul Adenot (:padenot)]

Attachment: Bug 1553215 - Start testing MediaStreamAudioSourceNode in WPT. r?karlt,pehrsons

It's not complete I think, but it covers the constructor and some bits of
behaviour.

Comment 2

[Paul Adenot (:padenot)]

Attachment: Bug 1553215 - Implement the correct behaviour for MediaStreamAudioSource. r?pehrsons,karlt

It should pick and lock to the right track, regardless of if it's still in the
MediaStream.

The test fix is because we don't expose tracks until the HTMLMediaElement has
loaded loading I think.

This alignes with a couple spec changes:

• https://github.com/WebAudio/web-audio-api/issues/264
• https://github.com/WebAudio/web-audio-api/pull/1829/files

and also throws when no valid track are found in the MediaStream, either because
it's all video tracks or because it has no tracks.

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug is a task, but please change it back in case of error.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/80181b642f81
Start testing MediaStreamAudioSourceNode in WPT. r=karlt,pehrsons
https://hg.mozilla.org/integration/autoland/rev/71154c8ef814
Implement the correct behaviour for MediaStreamAudioSource.  r=karlt,pehrsons

Comment 5

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/17255 for changes under testing/web-platform/tests

Comment 6

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Can't merge web-platform-tests PR due to failing upstream checks:
Github PR https://github.com/web-platform-tests/wpt/pull/17255
* Taskcluster (pull_request) (https://tools.taskcluster.net/task-group-inspector/#/NvEH3Cd7RCyeeAAoi3J4zA)

Comment 7

[Cosmin Sabou [:CosminS]]

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=pending%2Crunning%2Csuccess%2Ctestfailed%2Cbusted%2Cexception&classifiedState=unclassified&fromchange=71154c8ef814a3e6b10c9789325d4ed5529cf641&tochange=bb2560cbaecfd954eb0f2c14948c95a03ff12f09&searchStr=mda%2Cwindows&group_state=expanded&selectedJob=250959379

Failure log:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=250959379&repo=autoland&lineNumber=60076
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=250958537&repo=autoland&lineNumber=49874

Backout link: https://hg.mozilla.org/integration/autoland/rev/bb2560cbaecfd954eb0f2c14948c95a03ff12f09

17:38:42 INFO - TEST-PASS | dom/media/webaudio/test/test_audioContextSuspendResumeClose.html | An exception was not thrown
17:38:42 INFO - Buffered messages finished
17:38:42 INFO - TEST-UNEXPECTED-FAIL | dom/media/webaudio/test/test_audioContextSuspendResumeClose.html | An exception was not thrown
17:38:42 INFO - SimpleTest.ok@SimpleTest/SimpleTest.js:275:18
17:38:42 INFO - expectNoException@dom/media/webaudio/test/webaudio.js:22:3
17:38:42 INFO - tryToCreateNodeOnClosedContext/<@dom/media/webaudio/test/test_audioContextSuspendResumeClose.html:44:7
17:38:42 INFO - tryToCreateNodeOnClosedContext@dom/media/webaudio/test/test_audioContextSuspendResumeClose.html:37:40
17:38:42 INFO - afterResume/<@dom/media/webaudio/test/test_audioContextSuspendResumeClose.html:308:7
17:38:42 INFO - TEST-PASS | dom/media/webaudio/test/test_audioContextSuspendResumeClose.html | The context is in closed stat

[task 2019-06-10T17:44:36.782Z] 17:44:36 INFO - TEST-OK | /webaudio/the-audio-api/the-mediastreamaudiosourcenode-interface/mediastreamaudiosourcenode-ctor.html | took 1490ms
[task 2019-06-10T17:44:36.783Z] 17:44:36 INFO - TEST-START | /webaudio/the-audio-api/the-mediastreamaudiosourcenode-interface/mediastreamaudiosourcenode-routing.html
[task 2019-06-10T17:44:36.791Z] 17:44:36 INFO - Closing window 6442450945
[task 2019-06-10T17:44:36.835Z] 17:44:36 INFO - PID 3017 | [Child 3097, Main Thread] WARNING: No active window: file /builds/worker/workspace/build/src/js/xpconnect/src/XPCJSContext.cpp, line 662
[task 2019-06-10T17:44:36.851Z] 17:44:36 INFO - PID 3017 | [Parent 3017, Main Thread] WARNING: '!mSelection', file /builds/worker/workspace/build/src/editor/libeditor/EditorBase.cpp, line 4893
[task 2019-06-10T17:44:36.851Z] 17:44:36 INFO - PID 3017 | [Parent 3017, Main Thread] WARNING: '!editActionData.CanHandle()', file /builds/worker/workspace/build/src/editor/libeditor/EditorBase.cpp, line 1302
[task 2019-06-10T17:44:36.887Z] 17:44:36 INFO - PID 3017 | [Parent 3017, Main Thread] WARNING: nsIEditActionListener installed, this editor becomes slower: 'mActionListeners.Length() != 1', file /builds/worker/workspace/build/src/editor/libeditor/EditorBase.cpp, line 2136
[task 2019-06-10T17:44:36.964Z] 17:44:36 INFO - PID 3017 | ++DOCSHELL 0x7f19becca000 == 1 [pid = 3210] [id = {f5ac1c10-c6c1-48a9-aaa1-f7603f5aa95a}]
[task 2019-06-10T17:44:37.041Z] 17:44:37 INFO - PID 3017 | ++DOMWINDOW == 1 (0x7f19c01f4d40) [pid = 3210] [serial = 1] [outer = (nil)]
[task 2019-06-10T17:44:37.045Z] 17:44:37 INFO - PID 3017 | [Child 3210, Main Thread] WARNING: No active window: file /builds/worker/workspace/build/src/js/xpconnect/src/XPCJSContext.cpp, line 662
[task 2019-06-10T17:44:37.073Z] 17:44:37 INFO - PID 3017 | ++DOMWINDOW == 2 (0x7f19befab000) [pid = 3210] [serial = 2] [outer = 0x7f19c01f4d40]
[task 2019-06-10T17:44:37.190Z] 17:44:37 INFO - PID 3017 | ### XPCOM_MEM_BLOAT_LOG defined -- logging bloat/leaks to /tmp/tmpZIx365.mozrunner/runtests_leaks_829_tab_pid3247.log
[task 2019-06-10T17:44:37.340Z] 17:44:37 INFO - PID 3017 | Couldn't convert chrome URL: chrome://branding/locale/brand.properties
[task 2019-06-10T17:44:37.340Z] 17:44:37 INFO - PID 3017 | [Child 3247, Main Thread] WARNING: Could not get the program name for a cubeb stream.: 'NS_SUCCEEDED(rv)', file /builds/worker/workspace/build/src/dom/media/CubebUtils.cpp, line 388
[task 2019-06-10T17:44:37.381Z] 17:44:37 INFO - PID 3017 | ++DOMWINDOW == 3 (0x7f19c0168400) [pid = 3210] [serial = 3] [outer = 0x7f19c01f4d40]
[task 2019-06-10T17:44:37.590Z] 17:44:37 INFO - PID 3017 | ++DOMWINDOW == 4 (0x7f19be8e0c00) [pid = 3210] [serial = 4] [outer = 0x7f19c01f4d40]
[task 2019-06-10T17:44:38.458Z] 17:44:38 INFO -
[task 2019-06-10T17:44:38.458Z] 17:44:38 INFO - TEST-UNEXPECTED-FAIL | /webaudio/the-audio-api/the-mediastreamaudiosourcenode-interface/mediastreamaudiosourcenode-routing.html | MediaStreamAudioSourceNode captures the right track. - assert_true: Other track seem to be routed to the AudioContext? expected true got false
[task 2019-06-10T17:44:38.458Z] 17:44:38 INFO - analyse@http://web-platform.test:8000/webaudio/the-audio-api/the-mediastreamaudiosourcenode-interface/mediastreamaudiosourcenode-routing.html:97:11
[task 2019-06-10T17:44:38.458Z] 17:44:38 INFO - Test.prototype.step_timeout/<@http://web-platform.test:8000/resources/testharness.js:1652:22
[task 2019-06-10T17:44:38.458Z] 17:44:38 INFO - Test.prototype.step@http://web-platform.test:8000/resources/testharness.js:1594:25
[task 2019-06-10T17:44:38.459Z] 17:44:38 INFO - Test.prototype.step_func/<@http://web-platform.test:8000/resources/testharness.js:1618:35
[task 2019-06-10T17:44:38.459Z] 17:44:38 INFO - setTimeout handlerTest.prototype.step_timeout@http://web-platform.test:8000/resources/testharness.js:1651:16
[task 2019-06-10T17:44:38.459Z] 17:44:38 INFO - analyse@http://web-platform.test:8000/webaudio/the-audio-api/the-mediastreamaudiosourcenode-interface/mediastreamaudiosourcenode-routing.html:129:11
[task 2019-06-10T17:44:38.459Z] 17:44:38 INFO - Test.prototype.step_timeout/<@http://web-platform.test:8000/resources/testharness.js:1652:22
[task 2019-06-10T17:44:38.460Z] 17:44:38 INFO - Test.prototype.step@http://web-platform.test:8000/resources/testharness.js:1594:25
[task 2019-06-10T17:44:38.460Z] 17:44:38 INFO - Test.prototype.step_func/<@http://web-platform.test:8000/resources/testharness.js:1618:35
[task 2019-06-10T17:44:38.460Z] 17:44:38 INFO - setTimeout handlerTest.prototype.step_timeout@http://web-platform.test:8000/resources/testharness.js:1651:16
[task 2019-06-10T17:44:38.460Z] 17:44:38 INFO - @http://web-platform.test:8000/webaudio/the-audio-api/the-mediastreamaudiosourcenode-interface/mediastreamaudiosourcenode-routing.html:131:9
[task 2019-06-10T17:44:38.461Z] 17:44:38 INFO - TEST-OK | /webaudio/the-audio-api/the-mediastreamaudiosourcenode-interface/mediastreamaudiosourcenode-routing.html | took 1673ms

Comment 8

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR was closed without merging

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fad5e3bf0695
Start testing MediaStreamAudioSourceNode in WPT. r=karlt,pehrsons
https://hg.mozilla.org/integration/autoland/rev/992d3dd0cee0
Implement the correct behaviour for MediaStreamAudioSource.  r=karlt,pehrsons

Comment 10

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Can't merge web-platform-tests PR due to failing upstream checks:
Github PR https://github.com/web-platform-tests/wpt/pull/17255
* Taskcluster (pull_request) (https://tools.taskcluster.net/task-group-inspector/#/Cg6CKL36SkaCNAM-ZtNx2g)

Comment 11

[Oana Pop-Rus]

Backed out 2 changesets (bug 1553215) for Web platform tests failures in mediastreamaudiosourcenode-routing.html

Push that started the failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception%2Crunnable&revision=992d3dd0cee045520f093b21e8ed1746fcee112a&searchStr=debug%2Ctest%2Cwpt&selectedJob=251365828

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=251365828&repo=autoland&lineNumber=49712

Backout: https://hg.mozilla.org/integration/autoland/rev/4489b0e4df415ea32fa9ddc8aa5dfd77bf599cdd

Comment 12

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR was closed without merging

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d5a6a97abef4
Start testing MediaStreamAudioSourceNode in WPT. r=karlt,pehrsons
https://hg.mozilla.org/integration/autoland/rev/85228e5172e3
Implement the correct behaviour for MediaStreamAudioSource.  r=karlt,pehrsons

Comment 14

[Razvan Maries]

https://hg.mozilla.org/mozilla-central/rev/d5a6a97abef4
https://hg.mozilla.org/mozilla-central/rev/85228e5172e3

Comment 15

[Eric Shepherd [:sheppy]]

For documentation purposes, this is related to the addition of MediaStreamTrackAudioSourceNode -- its main purpose is to specify what the "first" audio track is. Among other things, see this blog post: https://blog.paul.cx/post/mediastreamtrackaudiosourcenode/

Comment 16

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Can't merge web-platform-tests PR due to failing upstream checks:
Github PR https://github.com/web-platform-tests/wpt/pull/17255
* Taskcluster (pull_request) (https://tools.taskcluster.net/task-group-inspector/#/Y4qc-scGR6WaTBFihiT9nQ)

Comment 17

[Andreea Pavel [:apavel]]

Please see above comment.

Comment 18

[Paul Adenot (:padenot)]

James, what is the process to sort out the problem in comment 16? I've done a fix to the test. I believe it should pass in Firefox now. Chrome does not have the feature yet, it's supposed to fail there.

Comment 19

[James Graham [:jgraham]]

Sorry, I was on PTO. Where's the test fix? You should be able to push it to the GitHub PR branch and we'll reland it in m-c once we downstream the PR. Or do you mean it was fixed in some other m-c push/bug?

Note that the Chrome problem is instability, not merely the test failing (which doesn't cause the CI to fail).

Comment 20

[Paul Adenot (:padenot)]

It was fixed in bug 1560251. The code is correct in Gecko, how should we proceed ?

Comment 21

[James Graham [:jgraham]]

Merge the upstream PR, which will allow that PR to apply and then end up with a fixed test upstream, I think. I'll do that.

Comment 22

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged

Comment 23

[Eric Shepherd [:sheppy]]

Documentation updates:

• Added Exceptions section to the reference page for MediaStreamAudioSourceNode; listed the InvalidStateError that occurs if the stream has no audio tracks.
• Added Usage notes section to the same page, explaining that the MediaStreamTrackAudioSourceNode node type is generally the better choice now, and why. Also includes a subsection explaining what the order of tracks is (sorted by ID using Unicode point ordering).
• Updated Firefox 69 for developers