1244035 - Volume changes can't take effect immediately in DecodedStream

Status: RESOLVED DUPLICATE of bug 1695265

(Core :: Audio/Video: Playback, defect, P3)

Description

[JW Wang [:jwwang] [:jw_wang]]

DecodedStream::SendAudio() sends all audio samples in the queue to the source media stream at a time and all audio samples are applied with the same volume. If the total duration of the audio queue is 10s, it means next volume change will not take effect until 10s later which is bad.

In order for volume changes to take effect as soon as possible, we should send audio data at a minimum amount without causing audio underrun. However, it is hard to determine the minimum value in the current push model of DecodedStream.

We should change to pull model as we did for AudioStream and DecodedAudioDataSink.

Comment 1

[JW Wang [:jwwang] [:jw_wang]]

With this bug, test_browserElement_inproc_AudioChannelMutedByDefault.html fails when changing AMPLE_AUDIO_USECS [1] from 1000000 to 2000000 in fixing bug 1242783.

When AMPLE_AUDIO_USECS is 2000000, setMuted(false) [2] won't take effect on the audio samples until 2 seconds are played. We won't have enough audible samples to pass the test [3] for audio.ogg is only 2.3 seconds long.

[1] https://hg.mozilla.org/mozilla-central/file/aa90f482e16db77cdb7dea84564ea1cbd8f7f6b3/dom/media/MediaDecoderStateMachine.cpp#l95
[2] https://hg.mozilla.org/mozilla-central/file/aa90f482e16db77cdb7dea84564ea1cbd8f7f6b3/dom/browser-element/mochitest/browserElement_AudioChannelMutedByDefault.js#l57
[3] https://hg.mozilla.org/mozilla-central/file/aa90f482e16db77cdb7dea84564ea1cbd8f7f6b3/dom/browser-element/mochitest/file_browserElement_AudioChannelMutedByDefault.html#l35

Comment 2

[Chris Peterson [:cpeterson]]

P2

Comment 3

[Robert O'Callahan (:roc) (email my personal email if necessary)]

If we played back all media element audio+video through the MediaStreamGraph, we wouldn't need to do this, we could use the MSG volume control.

Comment 4

[JW Wang [:jwwang] [:jw_wang]]

What if we send a media stream generated by capturing a media element over peer connection to the remote side? The remote side should be muted if we call |element.volume = 0| on the local side.

Comment 5

[Robert O'Callahan (:roc) (email my personal email if necessary)]

I'm not sure that's true. Better ask Pehrson or jesup.

Comment 6

[Karl Tomlinson (:karlt)]

For MediaElementAudioSourceNode, 

http://webaudio.github.io/web-audio-api/#idl-def-MediaElementAudioSourceNode

"The HTMLMediaElement must behave in an identical fashion after the
MediaElementAudioSourceNode has been created, except that the rendered audio
will no longer be heard directly, but instead will be heard as a consequence
of the MediaElementAudioSourceNode being connected through the routing
graph. Thus pausing, seeking, volume, src attribute changes, and other aspects
of the HTMLMediaElement must behave as they normally would if not used with a
MediaElementAudioSourceNode."

It may be that both media element sources and destinations should apply their
volumes.

Comment 7

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Mass change P2 -> P3

Comment 8

[Alastor Wu [:alwu]]

This has been done in bug1695265.