1188120 - Can we remove test_zmedia_cleanup.html?

Status: RESOLVED FIXED

(Core :: WebRTC, defect, P2)

Description

[Nils Ohlmeier [:drno]]

My understanding of the test_zmedia_cleanup.html file is, that it is suppose to tear down the WiFi interface on the B2G emulator after executing all of the WebRTC mochitests. It's hack around the fact that mochitest does not have setup and teardown functions for test groups.

But apparently our tests start and tear down the WiFi network for every single test execution any way.

So we have two options here:
1) Remove the test_zmedia_cleanup.html test case as it appears to be doing nothing.
2) Remove the tearDownNetwork() call from networkTestFinished() to speed up the test execution.

Comment 1

[Nils Ohlmeier [:drno]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=45aaa311a756

Comment 2

[Randell Jesup [:jesup] (needinfo me)]

Martin - did you do anything with zmedia_cleanup?  I seem to remember some bugs going by involving it and identity.

Comment 3

[Nils Ohlmeier [:drno]]

(In reply to Randell Jesup [:jesup] from comment #2)
> Martin - did you do anything with zmedia_cleanup?  I seem to remember some
> bugs going by involving it and identity.

I saw it in Martin's identity test mochitest.ini file and removed it from there as well.

If we can trust the result of this try run: https://treeherder.mozilla.org/#/jobs?repo=try&revision=45aaa311a756 it seem to be safe to remove the zmedia_cleanup test.

Comment 4

[Martin Thomson [:mt:]]

Yeah, Nils has a good handle on this.  The test exists to deal with some infidelities in the b2g testing, but I believe that we have a better option for handling this.

Comment 5

[Nils Ohlmeier [:drno]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=e182465e2ac6

Comment 6

[Nils Ohlmeier [:drno]]

(In reply to Nils Ohlmeier [:drno] from comment #5)
> https://treeherder.mozilla.org/#/jobs?repo=try&revision=e182465e2ac6

This is testing the second alternative of keeping zmedia_cleanup in place, but instead do not tear down the network interface after each test. The advantages might be a) faster test execution (as the network interfaces are only brought up and teared down once each) b) this might fix the strange intermittent problem in bug 1184319.

Comment 7

[Dan Minor [:dminor]]

From what I can tell, this test was added to help cleanup network state for B2G and doesn't actually test anything WebRTC related. It is now preventing Bug 1204281 from being resolved and I'm sure I've seen it fail intermittently fairly often elsewhere (e.g. Bug 1251132).

Is there anything preventing us from removing this test now that B2G is tier-3?

Comment 8

[Dan Minor [:dminor]]

It occurred to me that since this is the last test to run, we might just move shutdown crashes / timeouts to the test before it in the list. That said, if the test isn't really testing anything, we should still get rid of it.

Comment 9

[Nils Ohlmeier [:drno]]

Exactly. I fear that if we remove the test intermittents will only move to second from last. Although that might no happen, as the timing of test_zmedia_cleanup is obviously super fast compared to any of the real tests.
But as pointed out correctly with the new status of B2G I think it is safe to remove this test.

Which BTW also brings up the question if we can remove the code which ensures that network is up on B2G?

Comment 10

[Nils Ohlmeier [:drno]]

Attachment: Bug 1188120: remove B2G networking test code

Review commit: https://reviewboard.mozilla.org/r/62954/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/62954/

Comment 11

[Dan Minor [:dminor]]

Comment on attachment 8768972 [details]
Bug 1188120: remove B2G networking test code

https://reviewboard.mozilla.org/r/62954/#review59990

LGTM

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b54fe0985395
remove B2G networking test code r=dminor

Comment 13

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/b54fe0985395