771756 - Stop running talos on Android XUL

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Phil Ringnalda (:philor)]

The same story as bug 771421 - the only people who have any interest in Android XUL are b2g, and they are only interested in keeping OOP alive until they get their own tests running, so mochitests and maybe C/J/R. Nobody wants to see XUL talos anymore, especially not the tegra pool that has to run it.

"<mfinkle>: worth filing a bug to shut XUL talos down"

Comment 1

[Justin Wood (:Callek)]

From our last Mobile:Testing meeting (June 27) this was discussed and determined that we could not shut down testing for Android XUL until we get win8 testing up and running properly.

I'll make sure we talk about this again this coming week.

Comment 2

[Aki Sasaki (not active)]

Currently seeing a lot of backlog on android tests, a la bug 772458.

Comment 3

[Ed Morley [:emorley]]

Callek, re comment 1: not even just talos?

Comment 4

[Armen [:armenzg]]

I believe disabling talos for XUL should be the right thing to do.

Blassey ^? consent?

Comment 5

[Brad Lassey [:blassey] (use needinfo?)]

reopening because we eventually want to stop running tests on XUL fennec. As I understand it these tests are useful to B2G because they test e10s and the Metro work because they test similar XUL code. Once those teams/projects have their testing up to snuff we should turn these tests off.

So, to summarize, from the mobile team's perspective go ahead and turn them off. We should get sign off from desktop (for the Metro testing, Asa is already aware of this) and B2G that they no longer need XUL fennec as a proxy for their testing. 

Adding Asa and cjones to the CC so they can comment.

Comment 6

[Armen [:armenzg]]

In this bug, we're specifically asking about *talos/performace* rather than unit testing.

I don't there should be any inconvenience on stopping to measure the performance of a product that we don't ship anymore.

I assume we want to keep talos performance for mozilla-beta/mozilla-release since we're still shipping the XUL version for tablets; correct?

Comment 7

[Jim Mathies [:jimm]]

(In reply to Brad Lassey [:blassey] from comment #5)
> reopening because we eventually want to stop running tests on XUL fennec. As
> I understand it these tests are useful to B2G because they test e10s and the
> Metro work because they test similar XUL code. Once those teams/projects
> have their testing up to snuff we should turn these tests off.

FWIW, Metro front end code (which is a fork of XUL Fennec front end) moved to browser/metro and is already undergoing major changes. So the Fennec XUL tests in /mobile no longer matter to us.

Comment 8

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

xul-fennec talos tests are not useful for b2g.

Comment 9

[Aki Sasaki (not active)]

(In reply to Chris Jones [:cjones] [:warhammer] from comment #8)
> xul-fennec talos tests are not useful for b2g.

Are xul fennec unit tests useful for b2g?

Comment 10

[Aki Sasaki (not active)]

Attachment: remove android-xul from talos

Ran test-masters and reconfiged my test staging master with it.
Android XUL unit tests stayed; Android XUL talos disappeared.

Comment 11

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

(In reply to Aki Sasaki [:aki] from comment #9)
> (In reply to Chris Jones [:cjones] [:warhammer] from comment #8)
> > xul-fennec talos tests are not useful for b2g.
> 
> Are xul fennec unit tests useful for b2g?

reftests are useful because we don't have those running on b2g yet.  Hopefully soon.  We had mochitests running but now hit a problem so the xul-fennec ones are temporarily useful again.

Comment 12

[Aki Sasaki (not active)]

Does that include crashtest and jsreftest?
Sounds like we can reopen bug 771421 at least and turn off android-xul browser-chrome.

Comment 13

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

crashtest is useful while reftest harness not spun up for b2g.  jsreftest useful but less so.

Comment 14

[Armen [:armenzg]]

Comment on attachment 640808 [details] [diff] [review]
remove android-xul from talos

Review of attachment 640808 [details] [diff] [review]:
-----------------------------------------------------------------

I will attach the diff of builder_list.py ftr.

Comment 15

[Armen [:armenzg]]

Attachment: we're only removing talos for XUL

Comment 16

[Aki Sasaki (not active)]

Comment on attachment 640808 [details] [diff] [review]
remove android-xul from talos

http://hg.mozilla.org/build/buildbot-configs/rev/627672f99d93

Comment 17

[Aki Sasaki (not active)]

This made it to production today.