1091537 - Link clicker UI IE support

Status: RESOLVED FIXED

(Hello (Loop) :: Client, defect)

Description

[Romain Testard [:RT]]

The TokBox IE plug-in helps provide support for IE8 through to IE11.
This is enabled on the TokBox SDK from release 2.2.9.1
This bug tracks the process to support the IE plug-in from TokBox on Firefox Hello link clicker UI.

Comment 1

[:shell escalante]

let's investigate what it takes to fix this.  let's see how big the changes are to get working.

Comment 2

[Jared Wein [:jaws] (please needinfo? me)]

Attachment: Patch

As discussed over Vidyo. This, along with bug 1073415 should get calls working. I'd like to test it with you when you are reviewing this.

Comment 3

[Dan Mosedale (:dmosedale, :dmose)]

Comment on attachment 8517679 [details] [diff] [review]
Patch

Review of attachment 8517679 [details] [diff] [review]:
-----------------------------------------------------------------

r=dmose for the code changes themselves.  I'm happy to test on a call with you in the morning.

Comment 4

[Jared Wein [:jaws] (please needinfo? me)]

Attachment: Patch w/ r+

Thanks!

Comment 5

[Jared Wein [:jaws] (please needinfo? me)]

https://hg.mozilla.org/integration/fx-team/rev/af91cf5d1e66

Comment 6

[Nils Ohlmeier [:drno]]

Sorry, but I don't even know what qe-verify means for this bug:

A simple manual verification with any version of IE on any Windows which shows that the link clicker page loads and shows the start is probably doable.
Doing the same thing from IE 8 all the way to IE 11 and across all different version of Windows would be substantial effort.

Actually verifying that calls work with all version of IE on all Windows flavors is even more effort. Verifying that all features in an established call works as expected is even more effort.

Doing all this in an automated way is even more unrealistic, as we at Mozilla don't have the tools, experience or resources to test features like this on IE.

There is a long list of open question from my perspective, which should have been answered before this landed:
- How are users going to report bugs for the IE plugin? Bugzilla doesn't even allow you to specify IE as your browser, or?
- Who is going to take care of the bug reports?
- How do we debug and solve problems on code, which we don't own?

Comment 7

[Mark Banner (:standard8)]

(In reply to Nils Ohlmeier [:drno] from comment #6)
> There is a long list of open question from my perspective, which should have
> been answered before this landed:
> - How are users going to report bugs for the IE plugin? Bugzilla doesn't
> even allow you to specify IE as your browser, or?

I think we need to remember that the standalone client is basically a web page - just like our other websites - and we accept bugs on those which are IE (or another browser) specific.

There's a lot of infrastructure around the IE plugin (and the SDK) to set up and make the calls actually work. That's what we need to be testing and ensuring works. In theory, we have this issue for all reasonably current browsers (i.e. the at least the "browser unsupported" view should work in most cases - see bug 1092239 for example).

> - Who is going to take care of the bug reports?

There's at least two levels of bugs here: 1) things that the standalone client tries to do which IE doesn't support. Just like this bug. 2) issues in the IE plugin and SDK supplied by TokBox (or even their servers).

> - How do we debug and solve problems on code, which we don't own?

I think we already have this with the SDK itself - if we have an issue, then we analyse to see if its in our code or theirs. If we think it is in the sdk then we have a discussion with support folks and it gets fixed (or they explain what we're doing wrong).

Given the plugin is something extra to the sdk, and not cross-browser, it might be worth setting up an additional component for bugs that we think are the plugin-only, but it may be too early for that just yet.


In case there's something I've missed with respect to how we intend on handle these types of bugs, needinfoing RT and Adam as they will know more.

Comment 8

[Romain Testard [:RT]]

Mark I agree with what you say.
Safari support will also come at some point through the SDK (coming with March 2015 SDK release) and there are considerations for iOS and Android support through apps leveraging again the SDK for iOS and Android.

Comment 9

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/af91cf5d1e66

Comment 10

[u279076]

Before I can mark this qe-verify+ I think we need to have a meeting about what QE can support. The last time we discussed this (a couple weeks ago) it was agreed to that QE would not need to support this. There are significant resourcing concerns and we are just now beginning to test Firefox 35 features. I am not willing to put people on supporting this until we've at least verified our own builds work.

In other words, we need to support Firefox before IE and we've not yet reached a point where I can confidently say Firefox is well enough supported.

Comment 11

[Randell Jesup [:jesup] (needinfo me)]

https://hg.mozilla.org/releases/mozilla-aurora/rev/9f8521d8d21f

Comment 12

[u279076]

(In reply to Anthony Hughes, QA Mentor (:ashughes) from comment #10)
> In other words, we need to support Firefox before IE and we've not yet
> reached a point where I can confidently say Firefox is well enough supported.

Flagging myself so I know to come back to this asap.

Comment 13

[Nils Ohlmeier [:drno]]

(In reply to Mark Banner (:standard8) from comment #7)
> (In reply to Nils Ohlmeier [:drno] from comment #6)
> > There is a long list of open question from my perspective, which should have
> > been answered before this landed:
> > - How are users going to report bugs for the IE plugin? Bugzilla doesn't
> > even allow you to specify IE as your browser, or?
> 
> I think we need to remember that the standalone client is basically a web
> page - just like our other websites - and we accept bugs on those which are
> IE (or another browser) specific.

Sure if the expectation here is that QE only has to verify that the page loads and shows the "Start Call" button that is fine and doable.

> There's a lot of infrastructure around the IE plugin (and the SDK) to set up
> and make the calls actually work. That's what we need to be testing and
> ensuring works.

And that is where the problem starts for me. Obviously we could once manually verify that a call from IE works.
But what about code changes on our side, code changes in IE, code changes in the TokBox plugin? How do we learn about these, who tests them?

> In theory, we have this issue for all reasonably current
> browsers (i.e. the at least the "browser unsupported" view should work in
> most cases - see bug 1092239 for example).

I think this boils down to the question if Mozilla should become a calling service provider which worries about getting the maximum network reach of its service (like TokBox does it), or if Mozilla with Firefox Hello does something primarily for its Firefox users and besides Firefox covers only what is reasonably easy to cover with the given resources (my assumption here is that Firefox <-> Chrome interop is reasonably tested - but Firefox <-> IE interop has never been tested, at least not by us).
 
> > - How do we debug and solve problems on code, which we don't own?
> 
> I think we already have this with the SDK itself - if we have an issue, then
> we analyse to see if its in our code or theirs. If we think it is in the sdk
> then we have a discussion with support folks and it gets fixed (or they
> explain what we're doing wrong).

I think this comparing apples with oranges.
Scenario one is a JavaScript SDK which we have in source code and which gets executed in Firefox. That is reasonably easy to debug for us. And besides the copyright problem we could potentially fix issues in the JS SDK our self. Using this JS SDK allows a certain percentage of Firefox users to successfully place calls between each other which would not be possible without TokBox's help.
Scenario two is a binary plugin, for which we don't have the source code, nor do we know what that plugin is doing on the users machine. It runs in the browser of a competitor. So we have no easy way to debug any issues with it. Still we give our brand name for the combination of these binaries. And users will have an expectation that it will work as good as with Firefox.
 
Even if Mozilla QA would test the hell out of the IE + TokBox combination what would we do with any problems found during that testing that are not related just to our code of the standalone page?
For example problems with USB headsets, video quality issues,...

Comment 14

[Nils Ohlmeier [:drno]]

And sorry for spamming this bug with these high level question and concerns, but I'm not aware of any meta bug or other high level IE feature integration documentation where the expectations and responsibilities  are properly documented.

Comment 15

[Mark Banner (:standard8)]

I think as Anthony said in comment 10 we need to have a meeting on what we can do, the expectations etc. As Anthony's already flagged it to himself, I'll let him take this further.

Comment 16

[Randell Jesup [:jesup] (needinfo me)]

Comment on attachment 8518495 [details] [diff] [review]
Patch w/ r+

Approval Request Comment
Landed on aurora per IRC with lsblakk with a=loop-only

Comment 17

[u279076]

(In reply to Mark Banner (:standard8) from comment #15)
> I think as Anthony said in comment 10 we need to have a meeting on what we
> can do, the expectations etc. As Anthony's already flagged it to himself,
> I'll let him take this further.

Indeed I did - I plan to suggest a support proposal in tomorrow's 10:30am PT meeting. Assuming we agree to the proposal or some modification of it, I will follow up here in this bug.

Comment 18

[Adam Roach [:abr]]

I'm sympathetic to the problems that Nils raises, but these are fundamentally the same issues as we have with Chrome interop at the moment. If we run across a problem, we isolate it to "our code" or "not our code," and if it's not our code, we hand it off to TokBox.

The value proposition of the toolkit includes isolating us from the vagaries of different browser implementations (including the IE plugin), and we should plan our testing accordingly.

That said, we do need some short list of browsers we're going to test the standalone client in. The list of candidates gets big quickly:

* Firefox Linux
* Firefox Mac
* Firefox Windows
* Firefox Android
* Chrome/Chromium Linux
* Chrome/Chromium Mac
* Chrome/Chromium Windows
* Chrome/Chromium Android
* sBrowser Android
* Bowser iOS
* Internet Explorer Windows
* Opera Linux
* Opera Mac
* Opera Windows
* Opera Android

...and that's ignoring the multiple versions of the browsers. The problem gets even bigger when native WebRTC support comes to IE and Safari (assuming this happens at some point).

I'm tagging Romain here so that he can make a priority call about which platforms we need to actively test against. If we need a larger testing team to support what we need to support, then we should take steps to figure out how we're resourcing that.

Comment 19

[Romain Testard [:RT]]

I think our principles should be:
1 Prioritize testing on Firefox always
2 Testing in third party browsers should be done in order of browser usage share on the link clicker page (the scope of the testing being obviously limited to what features we support on specific browsers).
3 Browsers with less than 1% hits on the clicker page should be considered non supported

We don't have reports regarding browsers most used on the link clicker page although I'll see what could be done on Kibana with Katie.

The list will probably look like what follows although I'll confirm once I get clearer stats from the link clicker page:
1 Firefox Windows
2 Firefox Mac
3 Firefox Android
4 Firefox Linux
5 Chrome/Chromium for Windows
6 Chrome/Chromium for Mac
7 Chrome/Chromium for Android
8 Safari for iOS 
9 IE11 (Windows)
10 Android sBrowser 
11 Safari for Mac
12 IE8 (Windows)
13 Opera for Windows
14 IE9 (Windows)
15 IE10 (Windows)

Comment 20

[Romain Testard [:RT]]

Clearing NI as we decided not to support IE plug-ins.

Comment 21

[u279076]

(In reply to Romain Testard [:RT] from comment #20)
> Clearing NI as we decided not to support IE plug-ins.

Just closing the loop here on what QA could theoretically support if we ever decide to change that decision (this applies to *any* browser we'd want to support with plugins).

Regarding test coverage, we have web automation in place for loading the standalone page in WebRTC-supported browsers. In addition, Nils is working on an end-to-end test that should help for Firefox but this will not benefit the plugin. We will depend on Tokbox's test coverage for their plugin, whether it be automated or manual.

Regarding bug investigations, we can provide some support for initially investigating bugs that involve users outside Firefox. However, we will rely on Tokbox to investigate issues involving their plugin.

The priorities in comment 19 seem sane to me. QA will always put Mozilla's products first, third-parties who support WebRTC second, and all others last. That said our team has limited resources and any future test coverage will need to be prioritized against our new 2015 priorities. For any further questions on this I recommend contacting Marc Schifer and/or Clint Talbert.