507540 - Stop running build-and-unittest jobs

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Description

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

There are dep. bugs tracking getting unittests running properly on debug builds, separate from the builds. Once that is running with similar results, we can stop running our current all-in-one build-and-unittest jobs.

This is important to reduce per-push load on our infrastructure.

Comment 1

[Chris AtLee [:catlee]]

Something we could do in the short term is disable the tests portion of the current unittest build.  So we'd do the unittest build, and then hand off the build to the 'mochitests' and 'everythingelse' builders.

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #1)
> Something we could do in the short term is disable the tests portion of the
> current unittest build.  So we'd do the unittest build, and then hand off the
> build to the 'mochitests' and 'everythingelse' builders.

IIRC, ted said that 'make check' isn't packaged currently and would have to stay with the builder. Only takes a couple of minutes to run though.

Comment 3

[alice nodelman [:alice] [:anode]]

Futuring till blocking bug is fixed.

Comment 4

[Chris AtLee [:catlee]]

Attachment: Stop running tests in unittest builder

This is phase 1 of this bug - turning off most of the tests that are run as part of the current unittest build+tests builder.  Once we have unittests on debug and optimized builds running, we can stop running this builder entirely.

Comment 5

[[e-mail] (:bhearsum)]

Comment on attachment 401025 [details] [diff] [review]
Stop running tests in unittest builder

yay!

Comment 6

[Chris AtLee [:catlee]]

Comment on attachment 401025 [details] [diff] [review]
Stop running tests in unittest builder

changeset:   416:3968af736cff

Comment 7

[Chris AtLee [:catlee]]

Looks good

Comment 8

[Chris AtLee [:catlee]]

Actually, reopening this.  Phase 2 is to shut of the unittest builds altogether, but we need to wait until problems with optimized tests and debug tests are ironed out.

Comment 9

[Chris AtLee [:catlee]]

Attachment: Allow unittests to be toggled on per-platform basis

Comment 10

[Chris AtLee [:catlee]]

Attachment: buildbot-configs to turn off unittests on linux mozilla-central

Comment 11

[[e-mail] (:bhearsum)]

Comment on attachment 410234 [details] [diff] [review]
Allow unittests to be toggled on per-platform basis

>@@ -4255,22 +4255,23 @@ class WinmoBuildFactory(MobileBuildFacto
>             haltOnFailure=True
>         )
> 
> class UnittestPackagedBuildFactory(MozillaBuildFactory):
>     def __init__(self, platform, test_suites, env=None,
>             mochitest_leak_threshold=None, crashtest_leak_threshold=None,
>             totalChunks=None, thisChunk=None, chunkByDir=None,
>             **kwargs):
>+        self.platform = platform.split('-')[0]
>+
>         if env is None:
>-            self.env = MozillaEnvironments['%s-unittest' % platform].copy()
>+            self.env = MozillaEnvironments['%s-unittest' % self.platform].copy()
>         else:
>             self.env = env


This hunk is going to conflict with one of the patches in bug 515436. Should be easy enough to merge, just a heads up though!

Looks good otherwise.

Comment 12

[[e-mail] (:bhearsum)]

Comment on attachment 410235 [details] [diff] [review]
buildbot-configs to turn off unittests on linux mozilla-central

yay

Comment 13

[Chris AtLee [:catlee]]

Comment on attachment 410234 [details] [diff] [review]
Allow unittests to be toggled on per-platform basis

changeset:   478:3927145d3dbe

Comment 14

[Chris AtLee [:catlee]]

Comment on attachment 410235 [details] [diff] [review]
buildbot-configs to turn off unittests on linux mozilla-central

changeset:   1696:55e9f1f70d15

Comment 15

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 410234 [details] [diff] [review]
Allow unittests to be toggled on per-platform basis

The builder names changed with this patch, eg from
  OS X 10.5.2 mozilla-central test debug mochitests-1/5
to
  OS X 10.5.2 mozilla-central debug test mochitests-1/5
Same for opt. Caused some confusion.

Comment 16

[[e-mail] (:bhearsum)]

sdwilsh told me today that he believes we're ready to turn off refcnt builds on m-c Windows.

Dbaron, you've heavily informed here, can you confirm that?,

Comment 17

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

(In reply to comment #16)
> sdwilsh told me today that he believes we're ready to turn off refcnt builds on
> m-c Windows.
> 
> Dbaron, you've heavily informed here, can you confirm that?,

Yes.  Sounds fine to me.

Comment 18

[Chris AtLee [:catlee]]

Attachment: buildbot-configs to turn off unittests on win32 mozilla-central

woot!

Comment 19

[Chris AtLee [:catlee]]

How about project branches?  Can we turn off Linux and Windows unittest builds on tracemonkey, places and/or electrolysis?

Comment 20

[Nick Thomas [:nthomas] (UTC+12)]

We'd have to turn on opt and debug test there first, right ?

Comment 21

[Chris AtLee [:catlee]]

Comment on attachment 415996 [details] [diff] [review]
buildbot-configs to turn off unittests on win32 mozilla-central

changeset:   1833:0c379ff0c821

Comment 22

[Chris AtLee [:catlee]]

Attachment: turn off unittests on osx mozilla-central

dbaron: please r+ when we're ready to shut off the logrefcnt builds on osx on trunk

Comment 23

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Comment on attachment 421861 [details] [diff] [review]
turn off unittests on osx mozilla-central

I'm ok with turning them off now.

We probably want to do the same for TraceMonkey.

Comment 24

[Chris AtLee [:catlee]]

Attachment: turn off unittests on osx mozilla-central and tracemonkey

disable on tracemonkey too

Comment 25

[Chris AtLee [:catlee]]

Comment on attachment 421880 [details] [diff] [review]
turn off unittests on osx mozilla-central and tracemonkey

changeset:   1951:c5e38ec5870d