Closed Bug 1643204 Opened 4 months ago Closed 17 days ago

hear no audio when joining calls on MIT (W3C) WebEx instance

Categories

(Core :: DOM: Navigation, defect, P2)

defect

Tracking

()

RESOLVED FIXED
82 Branch
Tracking Status
firefox-esr68 --- unaffected
firefox-esr78 --- unaffected
firefox76 --- unaffected
firefox77 --- unaffected
firefox78 --- unaffected
firefox79 + fixed
firefox80 + verified
firefox81 + verified
firefox82 --- verified

People

(Reporter: dbaron, Assigned: mattwoodrow)

References

(Blocks 1 open bug, Regression)

Details

(Keywords: regression, Whiteboard: [wfh], [wptsync upstream])

Attachments

(2 files)

When I join a W3C meeting held on WebEx, in particular, for the CSSWG's first Wednesday call, I hear no audio.

This is a recent regression. Bisecting on nightlies gave this range but then mozregression failed to switch to taskcluster.

I've observed the problem on both Windows and Linux.

After updating mozregression, it did work on taskcluster builds, and I got to this regression range, which is a little bit surprising, but perhaps not a ton.

Flags: needinfo?(matt.woodrow)
Regressed by: 1620679

dholbert said:

I initially had no audio, and then I opened YouTube in another tab and started playing a video to sanity-check my audio, and then when I closed that & switched back to WebEx, it started working

So the "broken audio" situation may not be permanent.

Component: WebRTC → DOM: Content Processes

Matt, should we back out your web progress listener changes from bug 1620679 if it's causing this WebEx regression and possibly tp6 regression bug 1642998?

needinfo' jimm to try reproducing this bug.

Severity: -- → S2
Component: DOM: Content Processes → DOM: Navigation
Flags: needinfo?(jmathies)
Priority: -- → P2

David, how were you testing this to bisect?

I found a test instance on webex.com, but I haven't figured out to know if I'm getting audio from it (since it's empty).

Flags: needinfo?(matt.woodrow) → needinfo?(dbaron)

I was testing during the 30 minutes after the 60 minute CSS Working Group call (linked from comment 0), which is set up for 90 minutes. I'm not sure if we have any WebEx test setup access or if we'd need to ask somebody at W3C to set up a WebEx call that we could test with...

Flags: needinfo?(dbaron)

Also, the way I could tell is that I expect to hear a beep sound when I join the call successfully.

Thanks, the beep helped!

It looks like the webpage is using document.open() while we're loading, and this previously fired a 'pageshow' event and now doesn't.

From what I can tell, Chrome doesn't fire one either though, and WebEx is working just fine there.

I think this might be a site issue with UA sniffing, and then expecting us to do different things.

If I override my UA to MacOS X/Safari 12, then I get the beep and it seems to work correctly.

I haven't found any other UA setting that works (Chrome mode can't detect my speakers).

The change to not fire pageshow when cancelling the old load seems to be correct according to the spec, so hopefully we can get WebEx to not rely on it.

Duplicate of this bug: 1638454

detection is done in https://join-test.webex.com/webappng/scripts/vendor.8f815900.js
and https://akamaicdn.webex.com/pb/web/40.4.3.173/thinClientSupportAPI.js?rnd=2020060506

    isUnSupported: function (i, e) {
      var n = this._getClientInfo(),
      t = i || n.os,
      o = e || n.browser;
      return this.isMobileDevice() ? !0 : t.mac && this.notSupportMac(t.mac) ? !0 : t.win && o.ie && this.notSupportWindows(t.win, o.ver) ? !0 : /maxthon|opr/.test(r) ? !0 : o.chrome && o.ver <= 32 ? !0 : o.firefox && o.ver <= 28 ? !0 : o.safari && o.ver < 7 ? !0 : !1
    },
    isUnSupportedForNewWeb: function (i, e) {
      var n = this._getClientInfo(),
      t = i || n.os,
      o = e || n.browser;
      return this.isMobileDevice() ? !0 : t.mac && this.notSupportMac(t.mac) ? !0 : t.win && o.ie && this.notSupportWindows(t.win, o.ver) ? !0 : /maxthon|opr/.test(r) ? !0 : o.chrome && o.ver < 52 ? !0 : o.firefox && o.ver < 47 ? !0 : o.safari && o.ver < 11 ? !0 : !1
    },

[Tracking Requested - why for this release]:

WebEx audio no longer works in 79 Nightly.

@ Karl, does the WebCompat team have a contact at WebEx? And what is the process for requesting a temporary UA override in the WebCompat extension?

We need to:

  1. Ask WebEx to fix their UA detection.
  2. Then, while waiting for WebEx to fix their side, either:
  • Back out or fix Matt's change from bug 1620679
  • Or add a temporary UA override for WebEx in the WebCompat extension

Since Matt's change makes Firefox pageshow event behavior more like Chrome (IIUC Matt's comment 9), it seems like we should add a UA override instead of backing out Matt's changes.

Flags: needinfo?(jmathies) → needinfo?(kdubost)

Adding [wfh] whiteboard tag because this is a video conferencing bug.

Root Cause: --- → External Software Affecting Firefox
Whiteboard: [wfh]

Mike, can you please help find an assignee for this bug?

Flags: needinfo?(miket)

I don't have any contact.
maybe mike or adam knows.

Flags: needinfo?(kdubost)

stpeter, I believe you may have some WebEx contacts, is that correct?

Flags: needinfo?(miket) → needinfo?(stpeter)

Chris, can you make a call whether we need to back this out on our side or will we go the outreach route?

Flags: needinfo?(cpeterson)

(In reply to Gian-Carlo Pascutto [:gcp] from comment #17)

Chris, can you make a call whether we need to back this out on our side or will we go the outreach route?

Matt, unfortunately, I think we need to back out your web progress listener changes from bug 1620679 until WebEx fixes their site.

dbaron points out that crafting a UA override for WebEx might not be straightforward because WebEx instances exist at many different URLs.

Flags: needinfo?(cpeterson) → needinfo?(matt.woodrow)

We'll need clear instructions telling WebEx how to fix their site so it continues to work in Firefox versions with and without the changes in bug 1620679.

Depends on: 1648476

Uploaded a patch to backout the subset of my change that is causing issues (skipping firing "pageshow" for aborted document loads).

The safari UA override continues to work correctly both before and after this patch.

We can land this anytime if we decide not to go ahead with the UA spoofing.

Flags: needinfo?(matt.woodrow)

I've reached out to the Webex team about this.

Flags: needinfo?(stpeter)

:cpeterson do you know if work is planned on this bug for 79?

Flags: needinfo?(cpeterson)

[Tracking Requested - why for this release]:

(In reply to Kim Moir [:kmoir] ET from comment #23)

:cpeterson do you know if work is planned on this bug for 79?

Yes, Matt Woodrow will land his backout patch in Fx79 Beta. We'll leave the code (and regression) in Fx80 Nightly.

We've contacted Webex developers and they're investigating. If Webex hasn't deployed a server fix when Fx80 rides from Nightly to Beta, we might just publish a SUMO article describing a Webex workaround (switch tabs away from Webex and then back to activate audio) instead of backing out the code from Beta again.

Flags: needinfo?(cpeterson) → needinfo?(matt.woodrow)
Keywords: leave-open

Would be great if we could get this reviewed and nominated for Beta uplift in time for the 79.0b3 build starting tomorrow evening PDT.

We verified with current Beta build 79.0b2(64-bit), this issue could not be reproduced.

Comment on attachment 9159556 [details]
Bug 1643204 - Back out changes to not fire pageshow event for aborted document loads

Beta/Release Uplift Approval Request

  • User impact if declined: Audio not working on Cisco WebEx calls, until tab is switched
  • Is this code covered by automated tests?: No
  • Has the fix been verified in Nightly?: Yes
  • Needs manual test from QE?: No
  • If yes, steps to reproduce:
  • List of other uplifts needed: None
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): Just backing out a subset of the regressing change.
  • String changes made/needed:
Flags: needinfo?(matt.woodrow)
Attachment #9159556 - Flags: approval-mozilla-beta?

(In reply to Benrui She from comment #26)

We verified with current Beta build 79.0b2(64-bit), this issue could not be reproduced.

To be clear, was there a change on Cisco's end to make this start working or are you saying that Beta79 isn't showing the problem even though nothing has changed on your end?

Flags: needinfo?(beshe)

(In reply to Ryan VanderMeulen [:RyanVM] from comment #28)

(In reply to Benrui She from comment #26)

We verified with current Beta build 79.0b2(64-bit), this issue could not be reproduced.

To be clear, was there a change on Cisco's end to make this start working or are you saying that Beta79 isn't showing the problem even though nothing has changed on your end?

We didn't have any change in Cisco Webex Meeting Web Client code related to this issue.

Flags: needinfo?(beshe)

(In reply to Benrui She from comment #29)

We didn't have any change in Cisco Webex Meeting Web Client code related to this issue.

Interesting! Does is still reproduce for you with a Nightly build?

Flags: needinfo?(beshe)

Comment on attachment 9159556 [details]
Bug 1643204 - Back out changes to not fire pageshow event for aborted document loads

Per discussion with Matt, we're going to take this on Beta just to play it safe. Approved for 79.0b3.

Attachment #9159556 - Flags: approval-mozilla-beta? → approval-mozilla-beta+

(In reply to Chris Peterson [:cpeterson] from comment #19)

We'll need clear instructions telling WebEx how to fix their site so it continues to work in Firefox versions with and without the changes in bug 1620679.

Is that work on someone's plate?

Flags: needinfo?(cpeterson)

(In reply to David Baron :dbaron: 🏴󠁵󠁳󠁣󠁡󠁿 ⌚UTC-8 from comment #0)

When I join a W3C meeting held on WebEx, in particular, for the CSSWG's first Wednesday call, I hear no audio.

This is a recent regression. Bisecting on nightlies gave this range but then mozregression failed to switch to taskcluster.

I've observed the problem on both Windows and Linux.

Hi David! Do you still remember on which Webex site did you have this problem?
We need to double check that site on Webex side.
Thank you!

Flags: needinfo?(beshe)

(In reply to Benrui She from comment #34)

(In reply to David Baron :dbaron: 🏴󠁵󠁳󠁣󠁡󠁿 ⌚UTC-8 from comment #0)
Hi David! Do you still remember on which Webex site did you have this problem?
We need to double check that site on Webex side.

Benrui, David told me the Webex site was mit.webex.com.

We also had reports (in bug 1648476) of this audio problem on meetingsamer12.webex.com.

Flags: needinfo?(cpeterson)

(In reply to Julien Cristau [:jcristau] from comment #33)

(In reply to Chris Peterson [:cpeterson] from comment #19)

We'll need clear instructions telling WebEx how to fix their site so it continues to work in Firefox versions with and without the changes in bug 1620679.

Is that work on someone's plate?

Yes. I will make sure that happens. stpeter is talking with Webex engineers at Cisco. (And Benrui commenting in this bug works at Cisco. :)

Hi Benrui,

(In reply to Benrui She from comment #34)

Hi David! Do you still remember on which Webex site did you have this problem?
We need to double check that site on Webex side.

from David's initial report my guess would be that he experienced the initial issue on mit.webex.com as that server is regularly used by the W3C. David can you confirm this?

Flags: needinfo?(dbaron)

(In reply to Matt Woodrow (:mattwoodrow) from comment #8)

Thanks, the beep helped!

It looks like the webpage is using document.open() while we're loading, and this previously fired a 'pageshow' event and now doesn't.

From what I can tell, Chrome doesn't fire one either though, and WebEx is working just fine there.

Matt, where you using the WebEx test meeting page here https://www.webex.com/test-meeting.html or what page were you using to determine the root cause?

Flags: needinfo?(matt.woodrow)

(In reply to Nils Ohlmeier [:drno] from comment #38)

Matt, where you using the WebEx test meeting page here https://www.webex.com/test-meeting.html or what page were you using to determine the root cause?

Yes, I was using the test-meeting instance. Load that, ignore attempts to install the desktop app and wait for it to offer the web client, and then connect and wait for a beep.

I can no longer get it to offer me the web client at all, so I can't reproduce any more.

Flags: needinfo?(matt.woodrow)

(In reply to Nils Ohlmeier [:drno] from comment #37)

from David's initial report my guess would be that he experienced the initial issue on mit.webex.com as that server is regularly used by the W3C. David can you confirm this?

yes, that's correct.

Flags: needinfo?(dbaron)

Hey Matt, sounds like this might be resolved WFM without the backout for 80?

Flags: needinfo?(matt.woodrow)

I just observed this on nightly 2020-07-08 on Windows 10 on mit.webex.com (which is running version 40.6.1.179 of the Cisco Webex Meetings Web App).

(As noted before, the workaround of switching tabs made the audio start.)

:dbaron In future, can you grab the server version so that we can communicate it to Webex? You can do this by clicking the hamburger at top left, then Webex Support, Support, About.

(In reply to Peter Saint-Andre [:stpeter] from comment #43)

:dbaron In future, can you grab the server version so that we can communicate it to Webex? You can do this by clicking the hamburger at top left, then Webex Support, Support, About.

Ah, I need to find the version starting from the page from before joining the meeting. Following the above instructions from it gives:

Page version: 40.6.6.8
Desktop app version: 39.7.7.27
API version: 40.6.4.178

Here are my steps to reproduce on the main Webex test site, without needing to actually create a Webex account or connect multiple computers to the same meeting room:

  1. In Firefox 80 Nightly, load https://www.webex.com/test-meeting.html
  2. Enter a test username and email address.
  3. Ignore offers to download the native Webex application.
  4. Click the "Join from your browser" click.
  5. The "Join Meeting Test" page's video preview shows a frozen video frame from my camera. This is the bug!
  6. If I switch to a new Firefox tab and then switch back to the Webex tab, the video preview is now live video, as expected.

I'm using Windows 10. The video preview works fine in Firefox 78 Release. (I also tried testing Firefox 79 Beta, but I'm running into different problems there. My Firefox 79 Beta is not able unable to connect on "Join Meeting Test" page.)

(In reply to Nils Ohlmeier [:drno] from comment #38)

(In reply to Matt Woodrow (:mattwoodrow) from comment #8)

Thanks, the beep helped!

It looks like the webpage is using document.open() while we're loading, and this previously fired a 'pageshow' event and now doesn't.

From what I can tell, Chrome doesn't fire one either though, and WebEx is working just fine there.

Matt, where you using the WebEx test meeting page here https://www.webex.com/test-meeting.html or what page were you using to determine the root cause?

and @Chris
I checked with our UI engineer and found that in the meeting preview page, we do call 'document.open' for some iframe. And the result is that the video tag won't play automatically. So will that 'pageshow' event impact video tag's property?
Thanks!

Benrui

(In reply to Benrui She from comment #46)

(In reply to Nils Ohlmeier [:drno] from comment #38)

(In reply to Matt Woodrow (:mattwoodrow) from comment #8)

Thanks, the beep helped!

It looks like the webpage is using document.open() while we're loading, and this previously fired a 'pageshow' event and now doesn't.

From what I can tell, Chrome doesn't fire one either though, and WebEx is working just fine there.

Matt, where you using the WebEx test meeting page here https://www.webex.com/test-meeting.html or what page were you using to determine the root cause?

and @Chris
I checked with our UI engineer and found that in the meeting preview page, we do call 'document.open' for some iframe. And the result is that the video tag won't play automatically. So will that 'pageshow' event impact video tag's property?
Thanks!

Benrui
I'd write a test.html for this issue. It's easy to reproduce it

<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8">
    <meta name="description" content="WebRTC code samples">
    <meta id="theme-color" name="theme-color" content="#ffffff">
    <base target="_blank">
    <title>Test video</title>
</head>
<body>
<div id="container">
    <h1>Test Firefox Nightly video element play not working issue</h1>
</div>
</body>
<script>
var ifr = document.createElement("IFRAME");
ifr.width = 1024;
ifr.height = 640;
document.body.appendChild(ifr);
// setTimeout(function() {  //if open this setTimeout, the video is OK
    var doc = ifr.contentDocument;
    doc.open();
    doc.write([
        '<!DOCTYPE html>',
        '<html>',
        '<head><meta charset="UTF-8"/>',
        '<base target="_blank">',
        '</head>',
        '<body>',
        'test Video:',
        '<video id="video" autoplay></video>',
        '<script src=\"./test.js\"><\/script>',
        '</body>',
        '</html>'
    ].join(''));
    doc.close();
// }, 0);
</script>
</html>

The test.js file content:

function testVideo() {
    var constraints = {
        video: {deviceId: undefined}
    };
    navigator.mediaDevices.getUserMedia(constraints).then(function(stream) {
        var videoElement = document.querySelector("video");
        videoElement.srcObject = stream;
        videoElement.onplaying = function() {console.error("!!!!in playing")};
        videoElement.play();
    });
}
testVideo();

Open the test.html in Firefox Nightly(80.0a1 - I tested), the video will freezed, and click play on right click menu of the video element, can't work too.

And if I open the setTimeout in test.html, it works.

<script>
var ifr = document.createElement("IFRAME");
ifr.width = 1024;
ifr.height = 640;
document.body.appendChild(ifr);
setTimeout(function() { //if open this setTimeout, the video is OK
    var doc = ifr.contentDocument;
    doc.open();
    doc.write([
        '<!DOCTYPE html>',
        '<html>',
        '<head><meta charset="UTF-8"/>',
        '<base target="_blank">',
        '</head>',
        '<body>',
        'test Video:',
        '<video id="video" autoplay></video>',
        '<script src=\"./test.js\"><\/script>',
        '</body>',
        '</html>'
    ].join(''));
    doc.close();
}, 0);
</script>

And if move the append iframe inside setTimeout as below, it not work

<script>
var ifr = document.createElement("IFRAME");
ifr.width = 1024;
ifr.height = 640;

setTimeout(function() { 
    document.body.appendChild(ifr); //move into setTimeout, not work
    var doc = ifr.contentDocument;
    doc.open();
    doc.write([
        '<!DOCTYPE html>',
        '<html>',
        '<head><meta charset="UTF-8"/>',
        '<base target="_blank">',
        '</head>',
        '<body>',
        'test Video:',
        '<video id="video" autoplay></video>',
        '<script src=\"./test.js\"><\/script>',
        '</body>',
        '</html>'
    ].join(''));
    doc.close();
}, 0);
</script>

As the test result, seems if document.open called while the html rendering (not only the page first loading, also the dynamic rendering) will trigger this issue. Please have a check.

Thanks for the testcase!

It looks like the broken case happens when we start a new Document.open while the current Document hasn't yet loaded.

Previously, this cancelled the existing load (of about:blank in this case), and fired the pageshow event, which also updates our internal 'visible' state, required before we'll allow video playback.

Now we skip firing the pageshow event (correctly, I believe), but we're also not firing it in when the Document.open load finishes. This means we never get the event, and we never mark our Document as being visible and ready for video playback.

Flags: needinfo?(matt.woodrow)
Assignee: nobody → matt.woodrow
Status: NEW → ASSIGNED

Attached file Bug 1643204 - Fire pageshow event when we finish loading a Document.open load, if we haven't already fired one for the Document. r?smaug — Details

Matt, with your proposed pageshow event change, will the Cisco engineers still need to make any Webex code changes to support Firefox?

Flags: needinfo?(matt.woodrow)

(In reply to Chris Peterson [:cpeterson] from comment #50)

Attached file Bug 1643204 - Fire pageshow event when we finish loading a Document.open load, if we haven't already fired one for the Document. r?smaug — Details

Matt, with your proposed pageshow event change, will the Cisco engineers still need to make any Webex code changes to support Firefox?

Yeah, that's correct.

We might want to back out the pageshow changes from beta again, rather than trying to uplift the new patch.

Flags: needinfo?(matt.woodrow)
Blocks: 1655677

FYI, the in progress patch here fixes bug 1655677.

Per comment 51 I pushed the backout patch (same as comment 32 for beta 79) to beta 80.

[Tracking Requested - why for this release]:

@ Relman: this Webex bug is riding from Fx81 Nightly to Fx81 Beta. The "fix" for Fx79 and Fx80 was a backout of the original regressing patch.

@ Matt: what is the next step for landing this fix? We'll need to uplift it to Fx81 Beta, too.

Flags: needinfo?(matt.woodrow)
Pushed by mwoodrow@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/da1424ee1d11
Fire pageshow event when we finish loading a Document.open load, if we haven't already fired one for the Document. r=smaug
Flags: needinfo?(matt.woodrow)
Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/25251 for changes under testing/web-platform/tests
Whiteboard: [wfh] → [wfh], [wptsync upstream]

Backed out for wpt failure on reload.window.html and throw-on-dynamic-markup-insertion-counter-construct.html

Backed out link:
Log link1: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=314131564&repo=autoland&lineNumber=3695
Log link2: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=314139505&repo=autoland&lineNumber=2429

Flags: needinfo?(matt.woodrow)
Backout by nbeleuzu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/7373e9dde6cf
Backed out changeset da1424ee1d11 for wpt failure on reload.window.html CLOSED TREE
Flags: needinfo?(matt.woodrow)
Pushed by mwoodrow@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/35784e7b70b0
Fire pageshow event when we finish loading a Document.open load, if we haven't already fired one for the Document. r=smaug
Upstream PR merged by moz-wptsync-bot

I verified that 82 Nightly is now fixed and 81 Beta is still broken. We need to uplift this fix to 81 Beta.

Matt Woodrow is out on PTO until September 9, so someone else will need to uplift.

Ryan, Matt Woodrow is out on PTO until September 9, so someone else will need to uplift his fix to 81 Beta.

Flags: needinfo?(ryanvm)

Olli, can you please request Beta approval on this?

Flags: needinfo?(ryanvm) → needinfo?(bugs)

Comment on attachment 9166173 [details]
Bug 1643204 - Fire pageshow event when we finish loading a Document.open load, if we haven't already fired one for the Document. r?smaug

Beta/Release Uplift Approval Request

  • User impact if declined: MIT (W3C) WebEx and possibly other websites broken
  • Is this code covered by automated tests?: Yes
  • Has the fix been verified in Nightly?: Yes
  • Needs manual test from QE?: Yes
  • If yes, steps to reproduce: See https://bugzilla.mozilla.org/show_bug.cgi?id=1643204#c45
  • List of other uplifts needed: None
  • Risk to taking this patch: Medium
  • Why is the change risky/not risky? (and alternatives if risky): This changes page load code and changing that is always a bit risky.
  • String changes made/needed: NA
Flags: needinfo?(bugs)
Attachment #9166173 - Flags: approval-mozilla-beta?
Flags: qe-verify+
Status: ASSIGNED → RESOLVED
Closed: 17 days ago
Resolution: --- → FIXED
Target Milestone: --- → 82 Branch

Comment on attachment 9166173 [details]
Bug 1643204 - Fire pageshow event when we finish loading a Document.open load, if we haven't already fired one for the Document. r?smaug

Approved for 81.0b6.

Attachment #9166173 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
QA Whiteboard: [qa-triaged]

Hi all,

I was able to reproduce the issue on 81.0b1 (64-bit) and 81.0b5 (64-bit) using windows 10.
Verified fixed on beta 81.0b6 (64-bit) and Nightly 82.0a1 (2020-09-03) (64-bit)

Regards,
Vir

(In reply to Virginia Balducci from comment #69)

Hi all,

I was able to reproduce the issue on 81.0b1 (64-bit) and 81.0b5 (64-bit) using windows 10.
Verified fixed on beta 81.0b6 (64-bit) and Nightly 82.0a1 (2020-09-03) (64-bit)

Regards,
Vir

Thanks you all!
We will verify on WebEx side and update here.

Hi all,

This is verified fixed on Firefox Release 80.0.1 (64-bit) .

Regards,
Vir

QA Whiteboard: [qa-triaged]
Flags: qe-verify+
You need to log in before you can comment on or make changes to this bug.