1276237 - Run all possible e10s test suites on the release branches for Gecko 48+

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Ryan VanderMeulen [:RyanVM]]

This came up in yesterday's e10s standup. Right now, we have Ash providing test coverage for platforms that don't run in production, but that only tells us the current state of Trunk. While it's not horribly likely, there's a possibility that we have branch-specific bustage we're missing due to lack of test coverage once a given Gecko version starts riding the trains.

Ash was mainly stood up as a way of avoiding the massive capacity hit that enabling all e10s tests in production would have. However, Aurora/Beta/Release are much lower in volume than Trunk/Try, so it's unlikely that running all tests on all platforms will have a significant effect on backlogs elsewhere.

There are multiple things that need doing in this bug:
* Assess the current state of non-default e10s test suites on Aurora48. This should be discoverable with some Try pushes.
* Backport any suitable fixes needed to green up remaining suites where possible.
* Enable whatever suites we can on whatever platforms we can, taking care that this should *ONLY* apply Aurora/Beta/Release and Gecko 48+.

Comment 1

[Ryan VanderMeulen [:RyanVM]]

Note: Due to bug 1275039, Windows XP will need to be explicitly excluded from this work.

Comment 2

[Ryan VanderMeulen [:RyanVM]]

Attachment: enable e10s tests on Linux for Gecko 48+

This is just the patch from bug 1275281 with the version number changed to 48. Like that bug, we're not at a point where debug devtools tests can be enabled yet.

http://hg.mozilla.org/build/buildbot-configs/rev/cfbef61d1b5f

Comment 3

[Ryan VanderMeulen [:RyanVM]]

Attachment: enable taskcluster-based e10s tests on Linux64 debug for Gecko 48+

This is just a folded-up version of what landed in other bugs previously for trunk. Verified on Try that it works as expected. Carrying over previous r+.

https://hg.mozilla.org/releases/mozilla-aurora/rev/c03f06909893

Comment 4

[Ryan VanderMeulen [:RyanVM]]

The Windows results on my Try push looked good too, so we should be in good shape to get the remaining jobs when I get time to put that patch together.

Comment 5

[Ryan VanderMeulen [:RyanVM]]

Attachment: Enable Windows 8 e10s tests on Gecko 48+ release branches and Try

I added Try to this as well since Win8 jobs aren't scheduled by default there anymore. Might as well make the e10s jobs available to those who want to opt into them.

Builders added:
+ Windows 8 64-bit mozilla-aurora debug test crashtest-e10s
+ Windows 8 64-bit mozilla-aurora debug test jsreftest-e10s
+ Windows 8 64-bit mozilla-aurora debug test marionette-e10s
+ Windows 8 64-bit mozilla-aurora debug test mochitest-e10s-1
+ Windows 8 64-bit mozilla-aurora debug test mochitest-e10s-2
+ Windows 8 64-bit mozilla-aurora debug test mochitest-e10s-3
+ Windows 8 64-bit mozilla-aurora debug test mochitest-e10s-4
+ Windows 8 64-bit mozilla-aurora debug test mochitest-e10s-5
+ Windows 8 64-bit mozilla-aurora debug test mochitest-e10s-browser-chrome-1
+ Windows 8 64-bit mozilla-aurora debug test mochitest-e10s-browser-chrome-2
+ Windows 8 64-bit mozilla-aurora debug test mochitest-e10s-browser-chrome-3
+ Windows 8 64-bit mozilla-aurora debug test mochitest-e10s-browser-chrome-4
+ Windows 8 64-bit mozilla-aurora debug test mochitest-e10s-browser-chrome-5
+ Windows 8 64-bit mozilla-aurora debug test mochitest-e10s-browser-chrome-6
+ Windows 8 64-bit mozilla-aurora debug test mochitest-e10s-browser-chrome-7
+ Windows 8 64-bit mozilla-aurora debug test mochitest-e10s-devtools-chrome-1
+ Windows 8 64-bit mozilla-aurora debug test mochitest-e10s-devtools-chrome-2
+ Windows 8 64-bit mozilla-aurora debug test mochitest-e10s-devtools-chrome-3
+ Windows 8 64-bit mozilla-aurora debug test mochitest-e10s-devtools-chrome-4
+ Windows 8 64-bit mozilla-aurora debug test mochitest-e10s-devtools-chrome-5
+ Windows 8 64-bit mozilla-aurora debug test mochitest-e10s-devtools-chrome-6
+ Windows 8 64-bit mozilla-aurora debug test mochitest-e10s-devtools-chrome-7
+ Windows 8 64-bit mozilla-aurora debug test mochitest-e10s-devtools-chrome-8
+ Windows 8 64-bit mozilla-aurora debug test mochitest-gl-e10s
+ Windows 8 64-bit mozilla-aurora debug test mochitest-media-e10s
+ Windows 8 64-bit mozilla-aurora debug test reftest-e10s
+ Windows 8 64-bit mozilla-aurora debug test reftest-no-accel-e10s
+ Windows 8 64-bit mozilla-aurora debug test web-platform-tests-e10s-1
+ Windows 8 64-bit mozilla-aurora debug test web-platform-tests-e10s-10
+ Windows 8 64-bit mozilla-aurora debug test web-platform-tests-e10s-2
+ Windows 8 64-bit mozilla-aurora debug test web-platform-tests-e10s-3
+ Windows 8 64-bit mozilla-aurora debug test web-platform-tests-e10s-4
+ Windows 8 64-bit mozilla-aurora debug test web-platform-tests-e10s-5
+ Windows 8 64-bit mozilla-aurora debug test web-platform-tests-e10s-6
+ Windows 8 64-bit mozilla-aurora debug test web-platform-tests-e10s-7
+ Windows 8 64-bit mozilla-aurora debug test web-platform-tests-e10s-8
+ Windows 8 64-bit mozilla-aurora debug test web-platform-tests-e10s-9
+ Windows 8 64-bit mozilla-aurora debug test web-platform-tests-reftests-e10s
+ Windows 8 64-bit mozilla-aurora opt test crashtest-e10s
+ Windows 8 64-bit mozilla-aurora opt test jsreftest-e10s
+ Windows 8 64-bit mozilla-aurora opt test marionette-e10s
+ Windows 8 64-bit mozilla-aurora opt test mochitest-e10s-1
+ Windows 8 64-bit mozilla-aurora opt test mochitest-e10s-2
+ Windows 8 64-bit mozilla-aurora opt test mochitest-e10s-3
+ Windows 8 64-bit mozilla-aurora opt test mochitest-e10s-4
+ Windows 8 64-bit mozilla-aurora opt test mochitest-e10s-5
+ Windows 8 64-bit mozilla-aurora opt test mochitest-e10s-devtools-chrome-1
+ Windows 8 64-bit mozilla-aurora opt test mochitest-e10s-devtools-chrome-2
+ Windows 8 64-bit mozilla-aurora opt test mochitest-e10s-devtools-chrome-3
+ Windows 8 64-bit mozilla-aurora opt test mochitest-e10s-devtools-chrome-4
+ Windows 8 64-bit mozilla-aurora opt test mochitest-e10s-devtools-chrome-5
+ Windows 8 64-bit mozilla-aurora opt test mochitest-e10s-devtools-chrome-6
+ Windows 8 64-bit mozilla-aurora opt test mochitest-e10s-devtools-chrome-7
+ Windows 8 64-bit mozilla-aurora opt test mochitest-e10s-devtools-chrome-8
+ Windows 8 64-bit mozilla-aurora opt test mochitest-gl-e10s
+ Windows 8 64-bit mozilla-aurora opt test mochitest-media-e10s
+ Windows 8 64-bit mozilla-aurora opt test reftest-e10s
+ Windows 8 64-bit mozilla-aurora opt test reftest-no-accel-e10s
+ Windows 8 64-bit mozilla-aurora opt test web-platform-tests-e10s-1
+ Windows 8 64-bit mozilla-aurora opt test web-platform-tests-e10s-2
+ Windows 8 64-bit mozilla-aurora opt test web-platform-tests-e10s-3
+ Windows 8 64-bit mozilla-aurora opt test web-platform-tests-e10s-4
+ Windows 8 64-bit mozilla-aurora opt test web-platform-tests-e10s-5
+ Windows 8 64-bit mozilla-aurora opt test web-platform-tests-reftests-e10s
+ Windows 8 64-bit mozilla-aurora pgo test crashtest-e10s
+ Windows 8 64-bit mozilla-aurora pgo test jsreftest-e10s
+ Windows 8 64-bit mozilla-aurora pgo test marionette-e10s
+ Windows 8 64-bit mozilla-aurora pgo test mochitest-e10s-1
+ Windows 8 64-bit mozilla-aurora pgo test mochitest-e10s-2
+ Windows 8 64-bit mozilla-aurora pgo test mochitest-e10s-3
+ Windows 8 64-bit mozilla-aurora pgo test mochitest-e10s-4
+ Windows 8 64-bit mozilla-aurora pgo test mochitest-e10s-5
+ Windows 8 64-bit mozilla-aurora pgo test mochitest-e10s-devtools-chrome-1
+ Windows 8 64-bit mozilla-aurora pgo test mochitest-e10s-devtools-chrome-2
+ Windows 8 64-bit mozilla-aurora pgo test mochitest-e10s-devtools-chrome-3
+ Windows 8 64-bit mozilla-aurora pgo test mochitest-e10s-devtools-chrome-4
+ Windows 8 64-bit mozilla-aurora pgo test mochitest-e10s-devtools-chrome-5
+ Windows 8 64-bit mozilla-aurora pgo test mochitest-e10s-devtools-chrome-6
+ Windows 8 64-bit mozilla-aurora pgo test mochitest-e10s-devtools-chrome-7
+ Windows 8 64-bit mozilla-aurora pgo test mochitest-e10s-devtools-chrome-8
+ Windows 8 64-bit mozilla-aurora pgo test mochitest-gl-e10s
+ Windows 8 64-bit mozilla-aurora pgo test mochitest-media-e10s
+ Windows 8 64-bit mozilla-aurora pgo test reftest-e10s
+ Windows 8 64-bit mozilla-aurora pgo test reftest-no-accel-e10s
+ Windows 8 64-bit mozilla-aurora pgo test web-platform-tests-e10s-1
+ Windows 8 64-bit mozilla-aurora pgo test web-platform-tests-e10s-2
+ Windows 8 64-bit mozilla-aurora pgo test web-platform-tests-e10s-3
+ Windows 8 64-bit mozilla-aurora pgo test web-platform-tests-e10s-4
+ Windows 8 64-bit mozilla-aurora pgo test web-platform-tests-e10s-5
+ Windows 8 64-bit mozilla-aurora pgo test web-platform-tests-reftests-e10s
+ Windows 8 64-bit try debug test crashtest-e10s
+ Windows 8 64-bit try debug test jsreftest-e10s
+ Windows 8 64-bit try debug test marionette-e10s
+ Windows 8 64-bit try debug test mochitest-e10s-1
+ Windows 8 64-bit try debug test mochitest-e10s-2
+ Windows 8 64-bit try debug test mochitest-e10s-3
+ Windows 8 64-bit try debug test mochitest-e10s-4
+ Windows 8 64-bit try debug test mochitest-e10s-5
+ Windows 8 64-bit try debug test mochitest-e10s-browser-chrome-1
+ Windows 8 64-bit try debug test mochitest-e10s-browser-chrome-2
+ Windows 8 64-bit try debug test mochitest-e10s-browser-chrome-3
+ Windows 8 64-bit try debug test mochitest-e10s-browser-chrome-4
+ Windows 8 64-bit try debug test mochitest-e10s-browser-chrome-5
+ Windows 8 64-bit try debug test mochitest-e10s-browser-chrome-6
+ Windows 8 64-bit try debug test mochitest-e10s-browser-chrome-7
+ Windows 8 64-bit try debug test mochitest-e10s-devtools-chrome-1
+ Windows 8 64-bit try debug test mochitest-e10s-devtools-chrome-2
+ Windows 8 64-bit try debug test mochitest-e10s-devtools-chrome-3
+ Windows 8 64-bit try debug test mochitest-e10s-devtools-chrome-4
+ Windows 8 64-bit try debug test mochitest-e10s-devtools-chrome-5
+ Windows 8 64-bit try debug test mochitest-e10s-devtools-chrome-6
+ Windows 8 64-bit try debug test mochitest-e10s-devtools-chrome-7
+ Windows 8 64-bit try debug test mochitest-e10s-devtools-chrome-8
+ Windows 8 64-bit try debug test mochitest-gl-e10s
+ Windows 8 64-bit try debug test mochitest-media-e10s
+ Windows 8 64-bit try debug test reftest-e10s
+ Windows 8 64-bit try debug test reftest-no-accel-e10s
+ Windows 8 64-bit try debug test web-platform-tests-e10s-1
+ Windows 8 64-bit try debug test web-platform-tests-e10s-10
+ Windows 8 64-bit try debug test web-platform-tests-e10s-2
+ Windows 8 64-bit try debug test web-platform-tests-e10s-3
+ Windows 8 64-bit try debug test web-platform-tests-e10s-4
+ Windows 8 64-bit try debug test web-platform-tests-e10s-5
+ Windows 8 64-bit try debug test web-platform-tests-e10s-6
+ Windows 8 64-bit try debug test web-platform-tests-e10s-7
+ Windows 8 64-bit try debug test web-platform-tests-e10s-8
+ Windows 8 64-bit try debug test web-platform-tests-e10s-9
+ Windows 8 64-bit try debug test web-platform-tests-reftests-e10s
+ Windows 8 64-bit try opt test crashtest-e10s
+ Windows 8 64-bit try opt test jsreftest-e10s
+ Windows 8 64-bit try opt test marionette-e10s
+ Windows 8 64-bit try opt test mochitest-e10s-1
+ Windows 8 64-bit try opt test mochitest-e10s-2
+ Windows 8 64-bit try opt test mochitest-e10s-3
+ Windows 8 64-bit try opt test mochitest-e10s-4
+ Windows 8 64-bit try opt test mochitest-e10s-5
+ Windows 8 64-bit try opt test mochitest-e10s-devtools-chrome-1
+ Windows 8 64-bit try opt test mochitest-e10s-devtools-chrome-2
+ Windows 8 64-bit try opt test mochitest-e10s-devtools-chrome-3
+ Windows 8 64-bit try opt test mochitest-e10s-devtools-chrome-4
+ Windows 8 64-bit try opt test mochitest-e10s-devtools-chrome-5
+ Windows 8 64-bit try opt test mochitest-e10s-devtools-chrome-6
+ Windows 8 64-bit try opt test mochitest-e10s-devtools-chrome-7
+ Windows 8 64-bit try opt test mochitest-e10s-devtools-chrome-8
+ Windows 8 64-bit try opt test mochitest-gl-e10s
+ Windows 8 64-bit try opt test mochitest-media-e10s
+ Windows 8 64-bit try opt test reftest-e10s
+ Windows 8 64-bit try opt test reftest-no-accel-e10s
+ Windows 8 64-bit try opt test web-platform-tests-e10s-1
+ Windows 8 64-bit try opt test web-platform-tests-e10s-2
+ Windows 8 64-bit try opt test web-platform-tests-e10s-3
+ Windows 8 64-bit try opt test web-platform-tests-e10s-4
+ Windows 8 64-bit try opt test web-platform-tests-e10s-5
+ Windows 8 64-bit try opt test web-platform-tests-reftests-e10s

Comment 6

[[e-mail] (:bhearsum)]

Comment on attachment 8759696 [details] [diff] [review]
Enable Windows 8 e10s tests on Gecko 48+ release branches and Try

Review of attachment 8759696 [details] [diff] [review]:
-----------------------------------------------------------------

r=me if you keep the slave_platform checks, as we discussed on irc. I think they guard against cases where we drop a slave_platform on one branch but not another.

Comment 7

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 8759696 [details] [diff] [review]
Enable Windows 8 e10s tests on Gecko 48+ release branches and Try

Thanks!
https://hg.mozilla.org/build/buildbot-configs/rev/e438c77c6498

Comment 8

[Ryan VanderMeulen [:RyanVM]]

In production.