Closed Bug 862883 Opened 6 years ago Closed 6 years ago

Enable webrtc mochitest automation for FxAndroid

Categories

(Core :: WebRTC, defect, P2)

All
Android
defect

Tracking

()

RESOLVED FIXED
mozilla23

People

(Reporter: jsmith, Assigned: jsmith)

References

(Blocks 1 open bug)

Details

(Whiteboard: [WebRTC][blocking-webrtc-][android-gum+][qa-])

Attachments

(1 file, 3 obsolete files)

Turn on support for mochitests in webrtc on FxAndroid.
Blocks: webrtc-tests
I've added a patch here so that we can grab this and run a try build run to find out what failures we get when we run the mochitests on Android.
Whiteboard: [WebRTC][android-gum+]
Assignee: nobody → jsmith
Looking at the try run, looks like we passed on all gUM tests, so we can land a patch to turn those on right now.

However, every single peer connection test failed here with a reference error of "mozRTCPeerConnection is not defined." I thought the build patch that landed for WebRTC enabled PC support, but the tests are failing here on reference errors.

gcp - Any ideas why?
Flags: needinfo?(gpascutto)
Does the test put the media.peerconnection.enabled flag on Android?
Flags: needinfo?(gpascutto)
(In reply to Gian-Carlo Pascutto (:gcp) from comment #5)
> Does the test put the media.peerconnection.enabled flag on Android?

Yes. See the attached patch - it runs all tests with media.peerconnection.enabled set to true.
OS: All → Android
Whiteboard: [WebRTC][android-gum+] → [WebRTC][blocking-webrtc-][android-gum+]
Priority: -- → P2
Talked this over with Randell in the weekly meeting - he thinks the problem is that the build be generated on try with the mochitests isn't being compiled with the MOZ_WEBRTC flag, so the Peer Connection bits aren't being picked up.

I'll follow up with gcp in next week's meeting to find out why.
Putting needinfo on gcp for comment 7 in case he has ideas why the builds being ran in try are not building the WebRTC code on FxAndroid correctly.
Flags: needinfo?(gpascutto)
>Yes. See the attached patch - it runs all tests with media.peerconnection.enabled 
>set to true.

But it doesn't set media.navigator.enabled to true.
Flags: needinfo?(gpascutto)
(In reply to Gian-Carlo Pascutto (:gcp) from comment #9)
> >Yes. See the attached patch - it runs all tests with media.peerconnection.enabled 
> >set to true.
> 
> But it doesn't set media.navigator.enabled to true.

That shouldn't matter. We landed a patch originally that removed the requirement that media.navigator.enabled had to be set.

If that was true, then I'd expect the gUM tests to be failing right now. But that's not happening here.
Thing is I'm chasing mochitest crashes, and I don't have this problem at all:
https://tbpl.mozilla.org/?tree=Try&rev=43d69806d706
Hmm...okay. Maybe we changed something then after we preffed gUM or PC on in desktop. Okay, I'll modify the patch to enable both then.
So I had another try push today that didn't include the prefs flip:
https://tbpl.mozilla.org/?tree=Try&rev=286bad8d4740

Don't see the problem there at all.
(In reply to Gian-Carlo Pascutto (:gcp) from comment #13)
> So I had another try push today that didn't include the prefs flip:
> https://tbpl.mozilla.org/?tree=Try&rev=286bad8d4740
> 
> Don't see the problem there at all.

Okay. I'll update the above patch for something that can be landed then.
I just landed the code to make things pass on inbound, so a patch to enable them would be beat.
..neat.
Attachment #738559 - Attachment is obsolete: true
Attachment #744697 - Attachment is obsolete: true
Attachment #744701 - Flags: review?(gpascutto)
Attachment #744701 - Flags: feedback?(martijn.martijn)
Attachment #744701 - Flags: feedback?(hskupin)
For people flagged down for review/feedback:

- review on gcp to ensure these tests then effectively will run on FxAndroid
- feedback on whimboo to see if he's okay with the framework changes to move towards the exclude list model, rather than the weird UA stuff we were doing originally
- feedback on mw22 to ensure I did the b2g.json setup correct
Blocks: 868035
Comment on attachment 744701 [details] [diff] [review]
Enable WebRTC mochitests on FxAndroid v1

Apparently the perm pref is defined incorrectly.
Attachment #744701 - Flags: review?(gpascutto)
Attachment #744701 - Flags: review-
Attachment #744701 - Flags: feedback?(martijn.martijn)
Attachment #744701 - Flags: feedback?(hskupin)
Attachment #744701 - Attachment is obsolete: true
Comment on attachment 744706 [details] [diff] [review]
Enable WebRTC mochitests on FxAndroid v2

This time, with the right config pref.
Attachment #744706 - Flags: review?(gpascutto)
Attachment #744706 - Flags: feedback?(martijn.martijn)
Attachment #744706 - Flags: feedback?(hskupin)
Comment on attachment 744706 [details] [diff] [review]
Enable WebRTC mochitests on FxAndroid v2

Looks good to me, but you prolly want an r+ from an actual peer (I know next to nothing about the rests).
Attachment #744706 - Flags: review?(gpascutto) → review?(rjesup)
Comment on attachment 744706 [details] [diff] [review]
Enable WebRTC mochitests on FxAndroid v2

Review of attachment 744706 [details] [diff] [review]:
-----------------------------------------------------------------

Looks fine to me except for the question you can find below.

::: dom/media/tests/mochitest/head.js
@@ +116,3 @@
>    SimpleTest.waitForExplicitFinish();
> +  SpecialPowers.pushPrefEnv({'set': [
> +    ['media.peerconnection.enabled', true],

We should also remove this preference given that peer connection is enabled by default.

::: testing/mochitest/b2g.json
@@ +203,5 @@
>      "dom/file/test/test_workers.html":"",
>      "dom/file/test/test_write_read_data.html":"",
>      "dom/imptests/editing/conformancetest/test_event.html":"",
>      "dom/imptests/editing/conformancetest/test_runtest.html":"",
> +    "dom/media/tests/mochitest":"",

Can we already run all our mochitests on B2G?
Attachment #744706 - Flags: feedback?(hskupin) → feedback+
(In reply to Henrik Skupin (:whimboo) from comment #25)
> Comment on attachment 744706 [details] [diff] [review]
> Enable WebRTC mochitests on FxAndroid v2
> 
> Review of attachment 744706 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> Looks fine to me except for the question you can find below.
> 
> ::: dom/media/tests/mochitest/head.js
> @@ +116,3 @@
> >    SimpleTest.waitForExplicitFinish();
> > +  SpecialPowers.pushPrefEnv({'set': [
> > +    ['media.peerconnection.enabled', true],
> 
> We should also remove this preference given that peer connection is enabled
> by default.

Actually, we can't right now. FxAndroid requires the preference to be set in order to use the mozRTCPeerConnection API right now (i.e. it's preffed off by default).

> 
> ::: testing/mochitest/b2g.json
> @@ +203,5 @@
> >      "dom/file/test/test_workers.html":"",
> >      "dom/file/test/test_write_read_data.html":"",
> >      "dom/imptests/editing/conformancetest/test_event.html":"",
> >      "dom/imptests/editing/conformancetest/test_runtest.html":"",
> > +    "dom/media/tests/mochitest":"",
> 
> Can we already run all our mochitests on B2G?

Don't think so. The WebRTC APIs are not implemented on B2G yet, so the tests will fail if we run them on B2G right now. This is being worked on though.
Try run looks green outside of the one failure we already know about in bug 866093. That's been fixed, so we should be good to land so long as I get the r+ from jesup.
(In reply to Jason Smith [:jsmith] from comment #26)
> > ::: testing/mochitest/b2g.json
> > @@ +203,5 @@
> > >      "dom/file/test/test_workers.html":"",
> > >      "dom/file/test/test_write_read_data.html":"",
> > >      "dom/imptests/editing/conformancetest/test_event.html":"",
> > >      "dom/imptests/editing/conformancetest/test_runtest.html":"",
> > > +    "dom/media/tests/mochitest":"",
> > 
> > Can we already run all our mochitests on B2G?
> 
> Don't think so. The WebRTC APIs are not implemented on B2G yet, so the tests
> will fail if we run them on B2G right now. This is being worked on though.

So what does it then mean when we add this line? Doesn't it include all of the mochitests in the current set of executable tests?
(In reply to Henrik Skupin (:whimboo) from comment #28)
> (In reply to Jason Smith [:jsmith] from comment #26)
> > > ::: testing/mochitest/b2g.json
> > > @@ +203,5 @@
> > > >      "dom/file/test/test_workers.html":"",
> > > >      "dom/file/test/test_write_read_data.html":"",
> > > >      "dom/imptests/editing/conformancetest/test_event.html":"",
> > > >      "dom/imptests/editing/conformancetest/test_runtest.html":"",
> > > > +    "dom/media/tests/mochitest":"",
> > > 
> > > Can we already run all our mochitests on B2G?
> > 
> > Don't think so. The WebRTC APIs are not implemented on B2G yet, so the tests
> > will fail if we run them on B2G right now. This is being worked on though.
> 
> So what does it then mean when we add this line? Doesn't it include all of
> the mochitests in the current set of executable tests?

It's the opposite. The line I'm adding here adds the directory of tests to the exclude list, not the include list.

See http://hg.mozilla.org/mozilla-central/file/da429c311864/testing/mochitest/b2g.json.
Comment on attachment 744706 [details] [diff] [review]
Enable WebRTC mochitests on FxAndroid v2

Thanks!
Attachment #744706 - Flags: feedback?(martijn.martijn) → feedback+
Comment on attachment 744706 [details] [diff] [review]
Enable WebRTC mochitests on FxAndroid v2

Review of attachment 744706 [details] [diff] [review]:
-----------------------------------------------------------------

::: dom/media/tests/mochitest/head.js
@@ +116,3 @@
>    SimpleTest.waitForExplicitFinish();
> +  SpecialPowers.pushPrefEnv({'set': [
> +    ['media.peerconnection.enabled', true],

We need to keep the ability for people to disable it, at least to provide a firestop for security issues
Attachment #744706 - Flags: review?(rjesup) → review+
Keywords: checkin-needed
https://hg.mozilla.org/mozilla-central/rev/987686854a21
Status: NEW → RESOLVED
Closed: 6 years ago
Flags: in-testsuite+
Resolution: --- → FIXED
Target Milestone: --- → mozilla23
Whiteboard: [WebRTC][blocking-webrtc-][android-gum+] → [WebRTC][blocking-webrtc-][android-gum+][qa-]
You need to log in before you can comment on or make changes to this bug.