1216417 - Resuming a paused media element playing a stream does not play audio

Status: RESOLVED FIXED

(Core :: Audio/Video: MediaStreamGraph, defect, P1)

Description

[Shian-Yow Wu [:swu]]

When working on bug 1206621, we found an issue which is reproducible on TV simulator.

STR:
1. Launch TV simulator, go to TV deck and play the video.  Make sure audio is available.
2. Press 'Home' key and switch to Apps deck.  The TV app will be switched to background, and both vidio & audio are unavailable(expected).
3. Press 'Home' key and switch back to TV deck.  The TV app will be switched to foreground, and video keeps playing, but audio is NOT available(not expected).

When switching TV app back to foreground, the setMuted(false) will be called to unmute the video.  The setMuted(false) has been executed all the way down to HTMLMediaElement::SetVolumeInternal() with effectiveVolume=1, which looks fine.  So there might be other factors that caused such unmute failure.  One thing note, when the same video playback finished and replayed by TV app, the audio becomes available.

[Blocking Requested - why for this release]:

Audio policy management is changed after v2.1. We need the corresponding modification to be merged into TV System App.

Comment 1

[Shian-Yow Wu [:swu]]

[Blocking Requested - why for this release]:

Audio policy management is changed after v2.1. We need the corresponding modification to be merged into TV System App.

Comment 2

[Shian-Yow Wu [:swu]]

Alastor and Evan, do you have any comments?

Comment 3

[Evelyn Hung]

Evan just told me |setMuted()| is only API exposed to Gaia, so in theory we called |setMuted(false)| means System app allow this audio channel to play.

Comment 4

[Alastor Wu [:alwu]]

After discuss with Shian-Yow, the muted status of the MediaElement seems no problem.
We need to check the audio data flow, maybe it got stuck at someplace.

Comment 5

[Evan Tseng [:evanxd]]

Weird, no such problem in phone build. What's different between Phone and TV's gecko?

Comment 6

[Shian-Yow Wu [:swu]]

As suggested by Alastor, we enabled the log in MediaDecoderStateMachine::OnAudioDecoded() as below and make sure the audio sample keeps coming in after setMuted(false) to unmute.  However still no audio heard.

https://dxr.mozilla.org/mozilla-central/source/dom/media/MediaDecoderStateMachine.cpp#591

JW, do you have any comments?

Comment 7

[Shian-Yow Wu [:swu]]

(In reply to Evan Tseng [:evanxd][:愛聞插低] from comment #5)
> Weird, no such problem in phone build. What's different between Phone and
> TV's gecko?

TV simulator uses same gecko as phone.

Comment 8

[Mahendra Potharaju [:mahe]]

Blocks 2.5 for TV.

Comment 9

[Alastor Wu [:alwu]]

From the bug situation (comment4 & comment6), it's more like the audio playback issue.
Change the component.

Comment 10

[JW Wang [:jwwang] [:jw_wang]]

(In reply to Shian-Yow Wu [:swu] from comment #6)
> As suggested by Alastor, we enabled the log in
> MediaDecoderStateMachine::OnAudioDecoded() as below and make sure the audio
> sample keeps coming in after setMuted(false) to unmute.  However still no
> audio heard.
> 
> https://dxr.mozilla.org/mozilla-central/source/dom/media/
> MediaDecoderStateMachine.cpp#591
> 
> JW, do you have any comments?

Do you mean https://hg.mozilla.org/mozilla-central/file/f7b746b4e913/dom/media/MediaDecoderStateMachine.cpp#l591?

I can't see how it is relevant.

Comment 11

[Shian-Yow Wu [:swu]]

(In reply to JW Wang [:jwwang] from comment #10)
> (In reply to Shian-Yow Wu [:swu] from comment #6)
> > As suggested by Alastor, we enabled the log in
> > MediaDecoderStateMachine::OnAudioDecoded() as below and make sure the audio
> > sample keeps coming in after setMuted(false) to unmute.  However still no
> > audio heard.
> > 
> > https://dxr.mozilla.org/mozilla-central/source/dom/media/
> > MediaDecoderStateMachine.cpp#591
> > 
> > JW, do you have any comments?
> 
> Do you mean
> https://hg.mozilla.org/mozilla-central/file/f7b746b4e913/dom/media/
> MediaDecoderStateMachine.cpp#l591?
> 
> I can't see how it is relevant.

Sorry, should be this one:
https://hg.mozilla.org/mozilla-central/file/f7b746b4e913/dom/media/MediaDecoderStateMachine.cpp#l607

Comment 12

[JW Wang [:jwwang] [:jw_wang]]

Can you provide pr log with NSPR_LOG_MODULES=timestamp:1,MediaDecoder:4,MediaSample:4? I can't tell much why audio samples are not coming. It could be that playback is paused or reader is set idle...

Comment 13

[Shian-Yow Wu [:swu]]

Attachment: Log with NSPR_LOG_MODULES=timestamp:1,MediaDecoder:4

Here is the log.  Please find 'XXXXXX' for marked comments, thanks!

Comment 14

[JW Wang [:jwwang] [:jw_wang]]

====== WindowVolumeChanged 4807 aVolume=1.000000 aMuted=1
2015-10-21 04:37:43.932358 UTC - 829409024[7f684ce2d340]: Decoder=7f6834982a00 StopPlayback()
2015-10-21 04:37:43.932402 UTC - 829409024[7f684ce2d340]: Decoder=7f6834982a00 Dispatching SetIdle() audioQueue=1007369 videoQueue=133334
2015-10-21 04:37:43.932412 UTC - 829409024[7f684ce2d340]: Decoder=7f6834982a00 Not starting playback [playStatePermits: 0, mIsAudioPrerolling: 0, mIsVideoPrerolling: 0, mAudioOffloading: 0]

Seems nothing wrong with MDSM. The reader is set idle because we have enough audio and video in the queue.

Comment 15

[Shian-Yow Wu [:swu]]

Attachment: Log with NSPR_LOG_MODULES=timestamp:1,MediaDecoder:4,MediaSample:4

Just found that I missed to turn on MediaSample flag in the previous log.  This is the one with that enabled.

It seems to me no obvious difference between audio available/unavailable part from the log.  Which means the problem maybe not be related to MDSM, as stated in comment 14.

Comment 16

[James Cheng[:JamesCheng]]

Hi Alastor,

When I try to reproduce this symptom, I found that there is a warning message when pressing |Home Key| during video playback.

[Parent 21260] WARNING: Audio Buffer is not full by the end of the callback.: 'Available() == 0 || mSampleWriteOffset == 0'

https://dxr.mozilla.org/mozilla-central/rev/f7b746b4e91307448cb0746a41f677bfc23908b0/dom/media/AudioBufferUtils.h#87

Once this message printed out.

The audio stop outputting.

When go back to TV Deck, it may recover playing audio when video played to the end and start to play next round.

But I found that sometimes the video start to play, it has chance to print out the [Parent 21260] WARNING: Audio Buffer is not full by the end of the callback.: 'Available() == 0 || mSampleWriteOffset == 0' log again.

Once this log occurred, the audio cannot to playback.

Do you have any idea about this?


Thank you very much.

Comment 17

[Alastor Wu [:alwu]]

Because I'm not familiar with the AudioBuffer, change the NI to Paul.
Paul is working on web audio/audio streaming, he might know about that.

---

Hi, Paul,
Do you have any idea about comment16?
Thanks!

Comment 18

[Paul Adenot (:padenot)]

This means that the graph has stopped running of an audio callback, because it has found that there is no audio track in the graph.

Comment 19

[Blake Wu [:bwu][:blakewu]]

Per comment 18, change the component.

Comment 20

[Blake Wu [:bwu][:blakewu]]

Hi Maire, 
This bug is related to MSG. Can someone from your team help check this bug? 
It happens on TV simulator[1].  

[1]https://developer.mozilla.org/en-US/Firefox_OS/TVs_connected_devices

Comment 21

[Randell Jesup [:jesup] (needinfo me)]

[09:55]	padenot	if there is a MediaStream that has an audio track in the graph, we keep running off a AudioCallbackDriver
[09:55]	padenot	if that's is not true, then we run off a SystemClockDriver
[09:55]	padenot	at the transition between the two, you get, most of the time, not enough audio to fill the buffer
[09:55]	padenot	hence the message
[09:56]	padenot	also, the TV build uses special tracks iirc
[09:56]	padenot	tracks that contain video composited by their hardware compositor and the like
[09:56]	padenot	might be the same for audio

So I think looking at the difference between the tracks in the TV build and regular phone builds is the key.  Clearly regular phone builds restart the track when it's unmuted.  You could also turn on MSG logging (mediastreamgraph:4 or 5), which might help trackdown the details.  Warning: it's a lot of logging.

Comment 22

[Maire Reavy [:mreavy]]

Randell is going to take this one for now at least until we learn more. Paul is jammed with TPAC work, and I'm hoping this one is an easy fix once we get all the data.

Getting logs from a "working" phone build and a "non-working" TV build with MSG logging (mediastreamgraph:4 or 5) to compare would really help.  @swu and @jacheng -- can one of you get those for us?

Comment 23

[James Cheng[:JamesCheng]]

Hi Maire,

|Getting logs from a "working" phone build|

Do you know is there any existing sample or user scenario that uses MSG on phone build(b2g desktop)?

I want to find a quick way for logging the |working| case on phone build.

Thank you!

Comment 24

[Maire Reavy [:mreavy]]

Let's capture logs for the following scenario:
Start a video on the phone, switch to another app (which forces the video to the background), and then bring the video to the foreground.  Please be sure to confirm that audio resumes playing.

That's probably the closest scenario to what we're trying to do on the TV that isn't working.

Thanks!

Comment 25

[Shian-Yow Wu [:swu]]

James will provide the MSG log for comparison.

Comment 26

[James Cheng[:JamesCheng]]

Hi Maire,

Basically, we are not sure that "phone build" can work well.

Naively, I put TV Deck app into phone build and try to make all things the same as TV build.

My reproducing step is as below,

1. I run the TV Deck app, the video can be played with sound. 

2. I press |Home| to the app list, the sound is still playing(I'm not sure this is normal or abnormal).

3. I press another app trying to make the audio(from TV Deck) mute. The sound is still playing(Not the same  situation I met in TV build). I'm not sure if it is the UX design for phone but I cannot produce the logs with the same scenario as TV's.

In conclusion, I failed to verify the scenario on phone build.

Hi Evelyn,

Do you know how to verify this scenario in phone build or anything I ignored to build into GAIA?

Thank you.

Comment 27

[James Cheng[:JamesCheng]]

Attachment: NSPR_LOG_MODULES=MediaStreamGraph:5.log

Hi Maire,

After consulting from Evelyn and Luke,

By Bug 1190434,

We need to revert this commit to make the phone build relevant to TV build.

git revert -m1 ccbd181bc816ab3a6a6290bc3b80d77d4919d79d

Unfortunately, this issue can also be reproduced on phone build.

Attach the PRLOG gotten from phone build for your reference.

The reproduce step is

1. Enter TV Deck. (playback normally)
2. Leave TV Deck.(audio is mute) 
3. Enter again.(audio is still mute)

Thanks for your help.

Comment 28

[James Cheng[:JamesCheng]]

Attachment: dummy.tar.gz

upload the test asset.

Comment 29

[James Cheng[:JamesCheng]]

Attachment: 0001-add-TV-Deck-app-to-phone-build.patch

Once you want to reproduce the issue. 
Please follow below steps,

1. Follow MDN to setup the b2g desktop
https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/Building_the_Firefox_OS_simulator

2. Extract attachment 8677908 [details] and put it into /gaia/profile/ (the test asset is in  gaia/profile/dummy/...). This step is necessary since TV Deck will try to load data from this, but phone build will not generate this dummy folder.

3. Apply this patch to make TV Deck app build into phone.

4. make clean and make gaia again.

5. run b2g desktop.


4.

Comment 30

[Blake Wu [:bwu][:blakewu]]

(In reply to James Cheng[:JamesCheng] from comment #29)
> Created attachment 8677914 [details] [diff] [review]
> 0001-add-TV-Deck-app-to-phone-build.patch
> 
> Once you want to reproduce the issue. 
> Please follow below steps,
> 
> 1. Follow MDN to setup the b2g desktop
> https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/
> Building_the_Firefox_OS_simulator
> 
> 2. Extract attachment 8677908 [details] and put it into /gaia/profile/ (the
> test asset is in  gaia/profile/dummy/...). This step is necessary since TV
> Deck will try to load data from this, but phone build will not generate this
> dummy folder.
> 
> 3. Apply this patch to make TV Deck app build into phone.
> 
> 4. make clean and make gaia again.
> 
> 5. run b2g desktop.
> 
> 
> 4.
Have you finished your steps? :)

Comment 31

[James Cheng[:JamesCheng]]

Yes, Sorry for the typo...

Is there any way to modify the my own comment?

Thank you.

Comment 32

[Randell Jesup [:jesup] (needinfo me)]

Andreas - This seems to be related to AudioChannel issues; can you take a look?

Comment 33

[Josh Cheng [:josh]]

Hi James, I tagged "typo" to your comment 29. It will hide automatically.
You can retype your comment as new one. Thanks!

Comment 34

[James Cheng[:JamesCheng]]

Revise comment #29
==========================================
Once you want to reproduce the issue. 
Please follow below steps,

1. Follow MDN to setup the b2g desktop
https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/Building_the_Firefox_OS_simulator

2. Extract attachment 8677908 [details] and put it into /gaia/profile/ (the test asset is in  gaia/profile/dummy/...). This step is necessary since TV Deck will try to load data from this, but phone build will not generate this dummy folder.

3. Apply this patch to make TV Deck app build into phone.

4. make clean and make gaia again.

5. run b2g desktop.


Thank you Josh.

Comment 35

[Andrea Marchesini [:baku]]

This is what I see debugging this issue:

1. we have 2 registered AudioChannelAgents.
2. audiochannel-notification is correctly sent with 'active' state.
3. SetMuted(false) is correctly called.

I see the video and I hear the audio. Then I press 'home'. The app goes in background, but the audio continues.

1. no setMuted(true)
2. No unregistration occurred. No other notification.

then I open the tv app again, the video is shown, the audio is still there.

Comment 36

[Andrea Marchesini [:baku]]

Ok, what I wrote in the previous comment was already written in comment 26.
But also reverting the commit ccbd181b I cannot see this issue: the video is correctly muted when the app is in background, and it's unmuted when the app comes back to foreground.

Questions:
I'm using m-c. Do I maybe need a particular version of gecko?
I'm currently using a b2g desktop build. Do I need to run this code on a real device in order to reproduce this issue?

Comment 37

[Randell Jesup [:jesup] (needinfo me)]

NI James Cheng as well

Comment 38

[Shian-Yow Wu [:swu]]

(In reply to Andrea Marchesini (:baku) from comment #36)
> Ok, what I wrote in the previous comment was already written in comment 26.
> But also reverting the commit ccbd181b I cannot see this issue: the video is
> correctly muted when the app is in background, and it's unmuted when the app
> comes back to foreground.
> 
> Questions:
> I'm using m-c. Do I maybe need a particular version of gecko?
> I'm currently using a b2g desktop build. Do I need to run this code on a
> real device in order to reproduce this issue?

Have you tried reproducing this issue on B2G desktop with TV build[1]?
The TV build and phone build are based on same gecko(latest m-c), the only difference is gaia.

The video in channel 1 of TV deck is just 5 seconds long, and it repeats when finished.  The muted audio issue will be disappeared when video been replayed.  So it's easier to see the issue by switching to channel 2(use 'UP' key), which is another video with longer duration.

[1]https://developer.mozilla.org/en-US/Firefox_OS/TVs_connected_devices

Comment 39

[James Cheng[:JamesCheng]]

(In reply to Andrea Marchesini (:baku) from comment #36)
> Ok, what I wrote in the previous comment was already written in comment 26.
> But also reverting the commit ccbd181b I cannot see this issue: the video is
> correctly muted when the app is in background, and it's unmuted when the app
> comes back to foreground.
> 
> Questions:
> I'm using m-c. Do I maybe need a particular version of gecko?
> I'm currently using a b2g desktop build. Do I need to run this code on a
> real device in order to reproduce this issue?

Could you please make sure you used the longer video as comment#38 mentioned for verifying?

The audio can continue to playback once the video auto-replayed.

Thanks.

Comment 40

[Alastor Wu [:alwu]]

Hi, Baku,
This issue is only happen in TV build. See comment 38.

Do you think this issue related with AudioChannel?
IMMO, I think it's not related with AudioChannel.
Because the bug situation is only muting the audio, but the AudioChannel API would mute the whole video, not just audio.

Need your help here, thanks!

Comment 41

[Andrea Marchesini [:baku]]

Some more info about the debugging:

AudioChannelService is propagating the information correctly. mozilla::dom::HTMLMediaElement::SetVolumeInternal is called with muted true and then false when the app comes visible again. The difference is that: mSrcStreamIsPlaying is false, so we don't change the stream volume here: HTMLMediaElement::SetVolumeInternal().

But also changing the code so that we call |stream->SetAudioOutputVolume(this, effectiveVolume);| ignore mSrcStreamIsPlaying, the correct volume (effectiveVolume = 1) we don't fix the issue.

Comment 42

[Andreas Pehrson [:pehrsons]]

(In reply to Andrea Marchesini (:baku) from comment #41)
> Some more info about the debugging:
> 
> AudioChannelService is propagating the information correctly.
> mozilla::dom::HTMLMediaElement::SetVolumeInternal is called with muted true
> and then false when the app comes visible again. The difference is that:
> mSrcStreamIsPlaying is false, so we don't change the stream volume here:
> HTMLMediaElement::SetVolumeInternal().
> 
> But also changing the code so that we call
> |stream->SetAudioOutputVolume(this, effectiveVolume);| ignore
> mSrcStreamIsPlaying, the correct volume (effectiveVolume = 1) we don't fix
> the issue.

If the stream is paused, there's no audio output registered for the media element. When it gets added again on playing, the volume is set: https://dxr.mozilla.org/mozilla-central/rev/2b333a1d94e805a59c619ee41a6dec7fdcce505d/dom/html/HTMLMediaElement.cpp#3174

Comment 43

[Andreas Pehrson [:pehrsons]]

The AudioOutputs in MediaStreamGraph are not set up properly. Looking at James' MediaStreamGraph:5 log (attachment 8677907 [details]), and filtering on '7f41a3bc08c0' which seems to be the playback stream in question, I get the following:

> -1405274368[7f41a3bc2d00]: Adding media stream 7f41a3bc08c0 to the graph
> -1405274368[7f41a3bc2d00]: Adding MediaInputPort 7f4190e65640 (from 7f41a3bc0500 to 7f41a3bc08c0) to the graph
> <snip>
> -1405274368[7f41a3bc2d00]: MediaStream 7f41a3bc08c0 bufferStartTime=0.000000 blockedTime=0.000000
> -1405274368[7f41a3bc2d00]: MediaStream 7f41a3bc08c0 bufferStartTime=0.000000 blockedTime=0.000000
> -1405274368[7f41a3bc2d00]: MediaStream 7f41a3bc08c0 writing video frame 7f419144dc00 (540x360)
> -1405274368[7f41a3bc2d00]: MediaStream 7f41a3bc08c0 bufferStartTime=0.000000 blockedTime=0.000000
> -1606420736[7f4190f032c0]: MediaStream 7f41a3bc08c0 writing 1792 samples for 0.568889 to 0.609524 (samples 25088 to 26880)
The last line above is audio being written at [1].

> -1606420736[7f4190f032c0]: MediaStream 7f41a3bc08c0 writing video frame 7f4190e6ad00 (540x360)
> -1606420736[7f4190f032c0]: MediaStream 7f41a3bc08c0 bufferStartTime=0.000000 blockedTime=0.000000
> -1606420736[7f4190f032c0]: MediaStream 7f41a3bc08c0 writing 640 samples for 0.609524 to 0.624036 (samples 26880 to 27520)
> <snip>
> -1606420736[7f4190f032c0]: MediaStream 7f41a3bc08c0 writing video frame 7f4190fa8800 (540x360)
> -1606420736[7f4190f032c0]: MediaStream 7f41a3bc08c0 bufferStartTime=0.000000 blockedTime=0.000000
> -1606420736[7f4190f032c0]: MediaStream 7f41a3bc08c0 writing 640 samples for 3.535238 to 3.549751 (samples 155904 to 156544)
> -1606420736[7f4190f032c0]: MediaStream 7f41a3bc08c0 bufferStartTime=0.000000 blockedTime=0.000000
> -1606420736[7f4190f032c0]: MediaStream 7f41a3bc08c0 bufferStartTime=0.000000 blockedTime=0.000000
> -1606420736[7f4190f032c0]: MediaStream 7f41a3bc08c0 bufferStartTime=0.000000 blockedTime=0.000000
Here, audio and video is no longer played out. The outputs were removed by the media element when it got paused.

> <snip>
> -1606420736[7f4190f032c0]: MediaStream 7f41a3bc08c0 bufferStartTime=0.000000 blockedTime=0.000000
> -1606420736[7f4190f032c0]: MediaStream 7f41a3bc08c0 bufferStartTime=0.000000 blockedTime=0.000000
> -1606420736[7f4190f032c0]: MediaStream 7f41a3bc08c0 bufferStartTime=0.000000 blockedTime=0.000000
> -1606420736[7f4190f032c0]: MediaStream 7f41a3bc08c0 writing video frame 7f41915f6b00 (540x360)
> -1606420736[7f4190f032c0]: MediaStream 7f41a3bc08c0 writing video frame 7f4191b66300 (540x360)
> -1606420736[7f4190f032c0]: MediaStream 7f41a3bc08c0 bufferStartTime=0.000000 blockedTime=0.000000
> -1606420736[7f4190f032c0]: MediaStream 7f41a3bc08c0 bufferStartTime=0.000000 blockedTime=0.000000
> -1606420736[7f4190f032c0]: MediaStream 7f41a3bc08c0 bufferStartTime=0.000000 blockedTime=0.000000
> -1606420736[7f4190f032c0]: MediaStream 7f41a3bc08c0 writing video frame 7f4191b79600 (540x360)
> -1606420736[7f4190f032c0]: MediaStream 7f41a3bc08c0 bufferStartTime=0.000000 blockedTime=0.000000
> -1606420736[7f4190f032c0]: MediaStream 7f41a3bc08c0 bufferStartTime=0.000000 blockedTime=0.000000
> -1606420736[7f4190f032c0]: MediaStream 7f41a3bc08c0 writing video frame 7f41915cb800 (540x360)
Here, we have connected the outputs again, but only video is written. Either an audio output is not added (unlikely, pretty straight forward, see [2]) or it's not being picked up by the graph (definitely more complex, see [3]).

We should add some logs to MediaStreamGraph regarding audio outputs (and video probably) because there are none at all now.

My hunch is that this is a latent bug exposed by bug 1189506, since it swapped stream blocking for Add/RemoveAudio/VideoOutput() when pausing streams, so marking it as a dependency.


[1] https://dxr.mozilla.org/mozilla-central/rev/2b333a1d94e805a59c619ee41a6dec7fdcce505d/dom/media/MediaStreamGraph.cpp#677
[2] https://dxr.mozilla.org/mozilla-central/rev/2b333a1d94e805a59c619ee41a6dec7fdcce505d/dom/media/MediaStreamGraph.cpp#1764
[3] https://dxr.mozilla.org/mozilla-central/rev/2b333a1d94e805a59c619ee41a6dec7fdcce505d/dom/media/MediaStreamGraph.cpp#578

Comment 44

[Andreas Pehrson [:pehrsons]]

[23:04:03] <jesup> pehrsons: yeah, I was wondering about the AddAudioOutput()/RemoveAudioOutput() might be involved

[23:07:24] <jesup> pehrsons: is it possible we're not updating UpdateSrcMediaStreamPlaying()

[11:01:14] <pehrsons> jesup: baku|away: well since the video output was set up we know that HTMLMediaElement did all its stream updating, so I believe in an MSG issue. We should log changes to mAudioOutputs and how those are handled, i.e., the process of updating mAudioOutputStreams.

[11:03:18] <pehrsons> jesup: baku|away: well actually, we do set the volume from HTMLMediaElement but there's no "Audio output key not found" in the log. So the audio output is there but it wasn't picked up.

[11:06:13] <pehrsons> jesup: baku|away: This looks like a candidate: https://dxr.mozilla.org/mozilla-central/rev/2b333a1d94e805a59c619ee41a6dec7fdcce505d/dom/media/MediaStreamGraph.cpp#583

[11:09:27] <pehrsons> jesup: baku|away: We cleared mAudioOutputStreams when mAudioOutputs went empty, then we come back but the tracks haven't changed in the StreamBuffer (mTracksDirty is set to true only in AddTrack() and ForgetUpTo(), none of which are touched by Add/RemoveAudioOutput()), so we exit early.


PlayAudio() seems to be happy as long as mAudioOutputs and mAudioOutputStreams have the same length, but we should assert that. I can get a patch up soonish.

Comment 45

[Andreas Pehrson [:pehrsons]]

Seems like bug 1163958 caused this issue but bug 1189506 exposed it.

Comment 46

[Andreas Pehrson [:pehrsons]]

Attachment: MozReview Request: Bug 1216417 - Make sure audio output streams are created after adding an audio output. r?baku

Bug 1216417 - Make sure audio output streams are created after adding an audio output. r?baku

Comment 47

[Andreas Pehrson [:pehrsons]]

Attachment: MozReview Request: Bug 1216417 - Add logging for audio and video output changes in MSG. r?roc

Bug 1216417 - Add logging for audio and video output changes in MSG. r?roc

Comment 48

[Andreas Pehrson [:pehrsons]]

James, could you build with these two patches and try to reproduce again? If it is still failing another MediaStreamGraph:5 log would be good, thanks!

Comment 49

[James Cheng[:JamesCheng]]

Congratulations, I cannot reproduce this issue on my TV build.

Thank you Andreas.

I will try phone build later.

Comment 50

[James Cheng[:JamesCheng]]

Phone build also cannot reproduce this issue by Andreas' patch.

Comment 51

[Andrea Marchesini [:baku]]

Comment on attachment 8680471 [details]
MozReview Request: Bug 1216417 - Make sure audio output streams are created after adding an audio output. r?baku

https://reviewboard.mozilla.org/r/23627/#review21133

LGTM but ask a review to a media peer too!

Comment 52

[Andreas Pehrson [:pehrsons]]

Comment on attachment 8680471 [details]
MozReview Request: Bug 1216417 - Make sure audio output streams are created after adding an audio output. r?baku

Bug 1216417 - Make sure audio output streams are created after adding an audio output. r?baku

Comment 53

[Andrea Marchesini [:baku]]

Comment on attachment 8680471 [details]
MozReview Request: Bug 1216417 - Make sure audio output streams are created after adding an audio output. r?baku

https://reviewboard.mozilla.org/r/23627/#review21135

Comment 54

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Comment on attachment 8680471 [details]
MozReview Request: Bug 1216417 - Make sure audio output streams are created after adding an audio output. r?baku

https://reviewboard.mozilla.org/r/23627/#review21223

Comment 55

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Comment on attachment 8680472 [details]
MozReview Request: Bug 1216417 - Add logging for audio and video output changes in MSG. r?roc

https://reviewboard.mozilla.org/r/23629/#review21225

Comment 56

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/65e251b9db24
https://hg.mozilla.org/integration/mozilla-inbound/rev/dd5adab01830

Comment 57

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/65e251b9db24
https://hg.mozilla.org/mozilla-central/rev/dd5adab01830

Comment 58

[Andreas Pehrson [:pehrsons]]

Comment on attachment 8680471 [details]
MozReview Request: Bug 1216417 - Make sure audio output streams are created after adding an audio output. r?baku

Approval Request Comment
[Feature/regressing bug #]: Caused by Bug 1163958, exposed by Bug 1189506.
[User impact if declined]: Resuming a media element with a stream may not play audio, as observed in TV and phone builds.
[Describe test coverage new/current, TreeHerder]: Manually tested per comment 49 and comment 50.
[Risks and why]: Low. Very low complexity fix.
[String/UUID change made/needed]: None.

This approval request applies to both patches on this bug.

Comment 59

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Comment on attachment 8680471 [details]
MozReview Request: Bug 1216417 - Make sure audio output streams are created after adding an audio output. r?baku

I looked at the patches in MozReview and given that the change was tested + has been in Nightly for a few days, it seems safe to uplift to Aurora44.

Comment 60

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/releases/mozilla-b2g44_v2_5/rev/65e251b9db24
https://hg.mozilla.org/releases/mozilla-b2g44_v2_5/rev/dd5adab01830

Comment 61

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/7daa3893eab7
https://hg.mozilla.org/releases/mozilla-aurora/rev/ae4965be0983

Comment 62

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/releases/mozilla-b2g44_v2_5/rev/5e5c9c327bbc
https://hg.mozilla.org/releases/mozilla-b2g44_v2_5/rev/56103c13d304

Comment 63

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/releases/mozilla-b2g44_v2_5/rev/7daa3893eab7
https://hg.mozilla.org/releases/mozilla-b2g44_v2_5/rev/ae4965be0983

Comment 64

[Sylvestre Ledru [:Sylvestre]]

Comment on attachment 8680471 [details]
MozReview Request: Bug 1216417 - Make sure audio output streams are created after adding an audio output. r?baku

Just like Ritu: Taking it
Should be in 43 beta 2.

Comment 65

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/releases/mozilla-beta/rev/37d6cc360023
https://hg.mozilla.org/releases/mozilla-beta/rev/09bdeec1b65c