Closed Bug 1054157 Opened 7 years ago Closed 6 years ago

Shut off running tests on release builds until someone wants to look at them badly enough to make someone fix them

Categories

(Release Engineering :: Release Automation: Other, defect, P2)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: philor, Assigned: nthomas)

References

Details

(Whiteboard: [capacity])

Attachments

(1 file)

Since 2013-08-16 on betas, and 2013-09-11 on releases, all of the tests we've run on them (which nobody looks at, because there's no way to see them on tbpl and nobody's going to look at tests raw, like an animal) have failed like http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/None-candidates/buildNone/logs/release-mozilla-release_snowleopard_test-mochitest-2-bm79-tests1-macosx-build3.txt.gz and http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/None-candidates/buildNone/logs/release-mozilla-beta_mountainlion_test-mochitest-2-bm77-tests1-macosx-build92.txt.gz (and more currently like http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/None-candidates/buildNone/logs/release-mozilla-beta_ubuntu64_vm_test-reftest-bm68-tests1-linux64-build4.txt.gz), trying to download jsshell and test zips in places where they aren't and with names they don't have.

The sensible thing to do about that is *not* to fix them to download from where they are, but instead to just stop running them entirely, stop wasting the slave time we don't have to spare on test runs that are clearly of absolutely zero value, and wait until someone has committed to creating a dashboard to view them before turning them back on and making them work.
+100000.

I _think_ this can be done by setting release_tests = False in the branchConfig for mozilla-beta/release/esrs, based on https://github.com/mozilla/build-buildbotcustom/blob/master/misc.py#L2966
(In reply to Ben Hearsum [:bhearsum] from comment #1)
> +100000.

^ This. Let's not waste time on activities that provide no value.

More generally, I think it's worth discussing a tighter coupling between ci and release builds. Depending on what we can do, this may make running any of the tests that already run on ci completely redundant when running on release.
Blocks: 1085659
* removing release_tests shreds 1516 test builders (we had opt and pgo variants on platforms where we do pgo)
* emptying talosTestPlatforms removes the talos sendchange
* flipping enableUnittests removes the make package-tests call and sendchange

For the template changes, I tested the non-template version release-firefox-mozilla-release.py worked OK, but did check them all.
Attachment #8523717 - Flags: review?(bhearsum)
Assignee: nobody → nthomas
Priority: -- → P2
Attachment #8523717 - Flags: review?(bhearsum) → review+
Comment on attachment 8523717 [details] [diff] [review]
[buildbot-configs] Bye bye tests

https://hg.mozilla.org/build/buildbot-configs/rev/22019f9ea388

Will need a merge to production, and a release to take affect.
Attachment #8523717 - Flags: checked-in+
there's some code in buildbotcustom that can die as well
I'm going to not bother ripping code out of buildbotcustom at this point in it's life cycle.
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.