Closed Bug 1249098 Opened 4 years ago Closed 4 years ago

Firefox always sends full band Opus audio even though the negotiation specifies wideband audio

Categories

(Core :: WebRTC: Audio/Video, defect, P1)

44 Branch
defect

Tracking

()

RESOLVED FIXED
mozilla47
Tracking Status
firefox44 --- wontfix
firefox45 --- wontfix
firefox46 --- fixed
firefox47 --- fixed
Blocking Flags:

People

(Reporter: rdaware, Assigned: bwc)

References

Details

Attachments

(2 files)

When WebRTC negotiates the Opus audio with Firefox version 44, FF will always send FB (full band) stereo audio, even though the negotiation specifies WB (wide band) mono.
The first byte (TOC header) will have the value of 0x7c, instead of the 0x48 value that we expected.

Our endpoint sends the following SDP answer to FF. 
m=audio 9000 RTP/SAVPF 111\r\n
a=rtpmap:111 opus/48000/2\r\n
a=ptime:20\r\n
a=maxptime:20\r\n
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\n
a=setup:passive\r\n
a=rtcp-mux\r\n
a=sendonly\r\n
a=fmtp:111 minptime=20
a=fmtp:111 maxplaybackrate=16000; sprop-maxcapturerate=16000; maxaveragebitrate=40000; useinbandfec=1; usedtx=1

This problem doesn’t exist in FF version 43 or earlier versions.
Product: Firefox → Core
Component: Untriaged → WebRTC: Audio/Video
The change in 44 was due to changing the getUserMedia/MSG sample rate from 16Khz to 32Khz
See Also: → 953265
We were aware of this bug, but it has been low priority because it had no "real world" impacts.  A compliant Opus decoder will decode FB even if it expects WB.  IIUC the reason this bug has become an issue is because a  legacy device is using a very old version of Opus (a version prior to 1.0).  I need to know how common this device is in the field?  Is this a device that's just in a lab?  Or is it still popular in the field?  How likely is it that those devices will try to talk with Cisco Spark outside of a lab?
Severity: critical → normal
Flags: needinfo?(paulej)
The impact to our servers is big. For our audio servers which support 16 kHz sampling, we have to do Opus-decoding, down-sampling and Opus-encoding again. The quality could be an issue. The biggest impact is the CPU cost in our servers.
backlog: --- → webrtc/webaudio+
Rank: 10
Priority: -- → P1
Assignee: nobody → docfaraday
Flags: needinfo?(paulej)
Attachment #8720978 - Flags: review?(rjesup) → review+
Comment on attachment 8720978 [details]
MozReview Request: Bug 1249098: Support maxplaybackrate for opus. r=jesup

https://reviewboard.mozilla.org/r/35533/#review32201

::: media/webrtc/signaling/src/sdp/SdpAttribute.h:1244
(Diff revision 1)
> +      maxplaybackrate(48000)

Can we hoist this to a #define/etc somewhere?  We have that here and in CodecConfig.h and maybe elsewhere
Comment on attachment 8720978 [details]
MozReview Request: Bug 1249098: Support maxplaybackrate for opus. r=jesup

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/35533/diff/1-2/
Attachment #8720978 - Attachment description: MozReview Request: Bug 1249098: Support maxplaybackrate for opus. r?jesup → MozReview Request: Bug 1249098: Support maxplaybackrate for opus. r=jesup
Comment on attachment 8720978 [details]
MozReview Request: Bug 1249098: Support maxplaybackrate for opus. r=jesup

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/35533/diff/2-3/
Can you verify that one of the builds here does what you expect?

http://archive.mozilla.org/pub/firefox/try-builds/bcampen@mozilla.com-3f5f2ff13beff674e444f3a73663da296d2727eb/
Flags: needinfo?(rdaware)
Hi Paul -- Please see Comment 8.  We want to verify that this fix resolves the problem.  Any help you can give Rahul and his team to get this verified ASAP is much appreciated. Thanks.
Flags: needinfo?(paulej)
We are doing verification now. Will post the results shortly.
Sure, I'll request that this is checked.
Flags: needinfo?(paulej)
We tested on Firefox nightly 47.0a1 (2016-02-25) version.
Firefox still sends out full band opus audio when we negotiate for wide band audio.
Flags: needinfo?(rdaware)
(In reply to rdaware from comment #12)
> We tested on Firefox nightly 47.0a1 (2016-02-25) version.
> Firefox still sends out full band opus audio when we negotiate for wide band
> audio.

The candidate fix is not in Nightly yet.  You need to test with the build Byron referenced in Comment 8.
Flags: needinfo?(rdaware)
I tested with build given by Byron. We are able to receive wide band audio but we are receiving stereo even though we are negotiating for mono.
Flags: needinfo?(rdaware)
(In reply to rdaware from comment #14)
> I tested with build given by Byron. We are able to receive wide band audio
> but we are receiving stereo even though we are negotiating for mono.

Mono can be a separate bug.
I agree with Byron that mono can be a separate bug, but we (the collective "we") are running out of time in the Fx47 cycle for any additional work -- whether in this bug or a separate one.  

Rahul/Paul -- How important is it to support negotiating and sending mono?  Do you *need* that ASAP as well?  Is there anything else you need related to negotiating Opus?  I need firm answers to these questions immediately.  Thanks.
Flags: needinfo?(rdaware)
Flags: needinfo?(paulej)
What the team is telling me is that the existing clients cannot play stereo audio to the user.  Would it be possible to force mono and then resolve the stereo/mono issue as another bug?
Flags: needinfo?(paulej)
From our tests, in the previous version, FF43, the default channels is set to Mono but in FF 44, it is set to Stereo. This part of code must also have been changed from FF 43 to FF 44.
(In reply to duanpei from comment #19)
> From our tests, in the previous version, FF43, the default channels is set
> to Mono but in FF 44, it is set to Stereo. This part of code must also have
> been changed from FF 43 to FF 44.

We increased the bandwidth allocated to Opus around then; and the Opus encoder will cut features (like Stereo) if the bandwidth is too low.  I don't think we changed the codec channels setting.
https://hg.mozilla.org/mozilla-central/rev/6e6aaff5337b
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla47
Flags: needinfo?(rdaware)
Blocks: 818618
Comment on attachment 8720978 [details]
MozReview Request: Bug 1249098: Support maxplaybackrate for opus. r=jesup

Approval Request Comment
[Feature/regressing bug #]:

   Bug 953265 (sort of)

[User impact if declined]:

   Wasted bandwidth when interoping with endpoints that cannot make use of full-band opus, or even failure to interop with endpoints that cannot handle full-band opus.

[Describe test coverage new/current, TreeHerder]:

   There are some new unit-tests that cover the signaling aspects.

[Risks and why]: 

   Fairly low, mostly rote modifications, although a fair number.

[String/UUID change made/needed]:

   None.
Attachment #8720978 - Flags: approval-mozilla-aurora?
See Also: → 818618
Comment on attachment 8720978 [details]
MozReview Request: Bug 1249098: Support maxplaybackrate for opus. r=jesup

It would be good to fix this bandwidth issue, ok to uplift to aurora. 
Some extra test coverage added
Attachment #8720978 - Flags: approval-mozilla-aurora? → approval-mozilla-aurora+
I'm hitting conflicts trying to apply this to aurora. Can we get a rebased patch?
Flags: needinfo?(docfaraday)
Flags: needinfo?(docfaraday)
You need to log in before you can comment on or make changes to this bug.