630940 - stop running unittests on build machines for releases

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P4)

Description

[[e-mail] (:bhearsum)]

For 1.9.2 and mozilla-central we already run release tests on test machines, we should just disable the build machine runs.

For 1.9.1, we don't do any testing on release builds on test machines, but I don't see why we shouldn't be able to.

Comment 1

[Armen [:armenzg]]

The reason we didn't is because the perma-oranges that got fixed for enabling unit tests on the minis would require backporting which is not free.

That is why we decided not to enable unit tests on the minis for 1.9.1 and 1.9.2.

Comment 2

[Armen [:armenzg]]

(In reply to comment #0)
> For 1.9.2 and mozilla-central we already run release tests on test machines, we
> should just disable the build machine runs.
>
I am sorry I did not catch this at first.
We do *not* run unit tests on the test machines for 1.9.2 (nor for normal builds neither for releases).
http://hg.mozilla.org/build/buildbot-configs/file/69e900bf01ea/mozilla-tests/config.py#l499

Unfortunately, we have to keep on running unit tests on the builders unless you think you can convince developers to back port the test fixes they had to land for clearing perma-oranges.

I vote for WONTFIX.

Comment 3

[John Ford [:jhford] CET/CEST Berlin Time]

for mozilla-central builds, we shouldn't be running tests on builders though

Comment 4

[Armen [:armenzg]]

(In reply to comment #3)
> for mozilla-central builds, we shouldn't be running tests on builders though

It is disabled on the builders (but not for releases which has to be fixed - I have it assigned).

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

Armen is referring to bug 631546. I think we should fix that there. If we're running tests on minis for 1.9.2 then we should find out why, otherwise I think there's nothing else to do here.

Comment 6

[[e-mail] (:bhearsum)]

(In reply to comment #1)
> The reason we didn't is because the perma-oranges that got fixed for enabling
> unit tests on the minis would require backporting which is not free.
> 
> That is why we decided not to enable unit tests on the minis for 1.9.1 and
> 1.9.2.

Are you sure this is accurate for 1.9.2? I see tons of 1.9.2 tests running on minis...

Comment 7

[Armen [:armenzg]]

bhearsum are you sure you are not looking at the talos jobs?

This is how it should be:
* 1.9.1/1.9.2 talos on minis
* 1.9.1/1.9.2 unit tests on builders

Can we please close this bug? or am I missing something?

Comment 8

[[e-mail] (:bhearsum)]

Ah, you're right. Sorry for the confusion!