677332 - turn off broken or duplicate n900 tests

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Aki Sasaki (not active)]

The mobile team wants to keep the n900s running for extra coverage that the tegras don't provide.

In the meantime, let's turn off anything the tegras *do* provide, or that are obviously broken on n900.

Comment 1

[Aki Sasaki (not active)]

* Ts is broken (perma-red).
* browser-chrome is perma-orange.
* crashtest is perma-orange.
* xpcshell is perma-orange.

We should definitely turn off the above on n900.

* We're running Tdhtml on tegra; dup.
* We're running Tp4m and Tp4m_nochrome on tegra; Tp4 and Tp4_nochrome are a dup.
* We're running Tsspider, Tsvg, and Twinopen on tegra; dup.

* We're running Tpan and Tzoom on both, but Tpan and Tzoom are more stable on n900.
* We're running mochitests on both tegra and n900, but:

539$ diff n900.txt tegra.txt | more
0a1,2
> Harness_sanity
> MochiKit-1.4.2/tests
4,5d5
< content/xul/document/test
< content/xul/templates/tests
7a8,10
> dom/tests/mochitest/ajax/jquery
> dom/tests/mochitest/ajax/mochikit
> dom/tests/mochitest/ajax/scriptaculous
10a14,18
> dom/tests/mochitest/dom-level2-html
> dom/tests/mochitest/orientation
> dom/tests/mochitest/storageevent
> editor/composer/test
> intl/uconv/tests
12a21,26
> layout/inspector/tests
> layout/xul/test
> modules/libjar/test/mochitest
> parser/htmlparser/tests/mochitest
> toolkit/components/microformats/tests
> toolkit/xre/test

We're missing content/xul/document/test and content/xul/templates/tests from tegra mochitests.

We should add those two directories to tegra mochitests, and stabilize Tpan/Tzoom on tegra.  Other than that our test coverage on tegra is the same as, or greater than, n900 test coverage.

Comment 2

[Aki Sasaki (not active)]

Any objections to turning off:

* Ts
* browser-chrome
* crashtest
* xpcshell
* Tdhtml
* Tp4
* Tp4_nochrome
* Tsspider
* Tsvg
* Twinopen

on n900 ?

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

I agree to turning off the perma-orange and crashes, but why are we turning off tests that are running on android?  AFAIC everything that runs on n900 runs on android, is there something I am missing?

Comment 4

[Aki Sasaki (not active)]

(In reply to Joel Maher (:jmaher) from comment #3)
> I agree to turning off the perma-orange and crashes, but why are we turning
> off tests that are running on android?

We're turning off tests on n900 that are also running on Android, to

1) slowly turn Maemo into a tier 3 platform (unsupported by MoCo)
2) help lower tinderbox mail queue load
3) help get rid of production-mobile-master and staging-mobile-master, which IT doesn't want to have to move, but other portions of IT want the ESX server space back
4) no one looks at the n900 results; running these and maintaining them is wasting time+resources

> AFAIC everything that runs on n900
> runs on android, is there something I am missing?

Comment 1?

(In reply to Aki Sasaki [:aki] from comment #1)
> We're missing content/xul/document/test and content/xul/templates/tests from
> tegra mochitests.
> 
> We should add those two directories to tegra mochitests, and stabilize
> Tpan/Tzoom on tegra.  Other than that our test coverage on tegra is the same
> as, or greater than, n900 test coverage.

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

< content/xul/document/test
< content/xul/templates/tests

hmm, I don't see those on the spreadsheet when I ran the tests.  We have moved those tests from mochitest-plain to mochitest-chrome.  So mochitests are a non-issue.

the tpan/tzoom problems are related to the bug 662936 which just recently is starting to make some headway on getting resolved.

Comment 6

[Aki Sasaki (not active)]

To clarify:

[12:34]	<aki>	jmaher: are you basically saying you don't object to n900s being turned off?
[12:36]	<jmaher>	aki: I don't object at all

Comment 7

[Mark Finkle (:mfinkle) (use needinfo?)]

(In reply to Aki Sasaki [:aki] from comment #2)
> Any objections to turning off:
> 
> * Ts

This is perma-red, but is actually giving us rwal, valid data. In fact it just caught a Ts N900-only regression. The perma-red is a lie.

> * browser-chrome

I think we have some tests run here that are not currently enabled for Android

> * crashtest
> * xpcshell

OK with me unless someone else has issues

> * Tdhtml
> * Tp4
> * Tp4_nochrome
> * Tsspider
> * Tsvg
> * Twinopen

These are currently more stable on N900

Comment 8

[Aki Sasaki (not active)]

What does "stable" mean in this case?

Comment 9

[Aki Sasaki (not active)]

Attachment: turn off n900 xpcshell, crashtest

Also, kill the emails we get from pmm.

Comment 10

[Aki Sasaki (not active)]

Comment on attachment 551572 [details] [diff] [review]
turn off n900 xpcshell, crashtest

http://hg.mozilla.org/build/buildbot-configs/rev/079e6310a921

Comment 11

[Aki Sasaki (not active)]

Going to mark this as fixed.
6.0 shipped, so we're < 6 weeks away to Maemo tier 3.