1045398 - Consider fully disabling mochitest-bc and mochitest-dt on the B2G release branches

Status: RESOLVED FIXED

(Release Engineering :: Release Automation: Other, defect)

Description

[Ryan VanderMeulen [:RyanVM]]

Do we get any useful test coverage by running the browser-chrome and devtools mochitests on the B2G release branches(b2g28, b2g30, b2g32)? We don't ship any browser builds from those branches and IIUC, the tests really don't cover much in the way of B2G functionality.

To cut down on wasted load and intermittent oranges that aren't likely to be fixed, I'd like to shut them off if they don't serve any useful purpose.

Comment 1

[Ryan VanderMeulen [:RyanVM]]

Rob/Gavin, I guess I'm really looking for your feedback on this. Feel free to loop in any others you want to weigh in.

Comment 2

[Rob Campbell [:rc] (:robcee)]

I'll defer to Paul. He's more looped-in to the B2G testing universe.

Comment 3

[Paul Rouget [:paul]]

Alex knows these things (he's on holiday, coming back next week).

Comment 4

[:Gavin Sharp [email: [e-mail]]]

As far as I know there is no b2g coverage in any browser-chrome tests, though I suppose in theory there could be some coverage of Gecko features. Probably not any that b2g depends on that it doesn't have its own coverage for, though, and if we're not shipping browser builds from those branches I see little point in running browser-chrome tests.

Comment 5

[Alexandre Poirot [:ochameau]]

There isn't any b2g specific tests in browser-* suites.
But as gavin said, there can still be some useful platform tests, that aren't testing anything related to browser chrome. Instead they focus on some lower level platform code.
There shouldn't be many, but I know at least one or two related to webapps API.
  http://mxr.mozilla.org/mozilla-central/source/dom/tests/browser/browser.ini#33
But it looks like they are disabled.

At the end I would say that there is a quite limited value in running them on b2g branches.
Regarding browser-devtools, I think there is absolutely no value in running them.

We may introduce some new kind of browser-* suite once Mulet is ready where we would test some b2g specifics.
It will allow us to test in the same suite browser chrome client code against b2g remote debugging server.

Comment 6

[Ryan VanderMeulen [:RyanVM]]

This is probably past my ability to drive forward at this point. James, might you be able to help here?

Comment 7

[James Lal [:lightsofapollo]]

Heh "absolutely no value" to me means lets turn them off :)

Comment 8

[Ryan VanderMeulen [:RyanVM]]

I was asking for someone to write the patch ;)

Comment 9

[Massimo Gervasini [:massimo]]

Attachment: [buildbot-configs] Bug 1045398 - disabling mochitest-bc and mochitest-dt on the B2G release branches.patch

Removes mochitest-devtools-chrome and mochitest-browser-chrome from mozilla-b2g* branches

Comment 10

[Massimo Gervasini [:massimo]]

Attachment: differences

and this is the list of removed builders

Comment 11

[Jordan Lund (:jlund)]

bbot-cfg patch is in production

Comment 12

[Ryan VanderMeulen [:RyanVM]]

And working perfectly, thanks!