1379661 - Run Windows Talos, HW and other unmigrated tests via BBB

Status: RESOLVED FIXED

(Release Engineering :: General, enhancement)

Description

[Chris AtLee [:catlee]]

We don't yet have Windows hardware test support in Taskcluster, which means that we can't run Talos or other hardware tests natively. We'll need to use BBB in the meanwhile.

This is all tests currently running on t-w732-ix- or t-w864-ix-.

Joel, do you know if there any VM based tests that are not ready in TC and would need to be run in BB?

Comment 1

[Chris AtLee [:catlee]]

https://hg.mozilla.org/projects/date/rev/14a74a96b5e3eaff292ba1ac3ca1efbccec2138a
Bug 1379661: Add Talos tests to Windows builds

Comment 2

[Chris AtLee [:catlee]]

https://hg.mozilla.org/projects/date/rev/441554d1452190214c8dff7a0b0497efac1658f5
Bug 1379661: Enable Talos to Windows on date

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

comparing date talos to m-c talos:
https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-central&originalRevision=6fec4855b5345eb63fef57089e61829b88f5f4eb&newProject=date&newRevision=38ce7562482886964c7def235435c65e6090dbe3&framework=1&showOnlyImportant=0

I did many retriggers on both date and m-c, we will see when the data comes in what we end up with.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

I see quantum-pageload (win talos only) missing as well as xperf (win7 only).

Comment 5

[Chris AtLee [:catlee]]

https://hg.mozilla.org/projects/date/rev/f85ffb6ebbdd54336e25aa853f660f9d596f1ca3
Bug 1379661: Stop running talos on Windows nightly builds to match Linux and OSX.

Comment 6

[Chris AtLee [:catlee]]

https://hg.mozilla.org/projects/date/rev/eded5ed51e35380e44605ac450569469cd2e4ea6
Bug 1379661: Add tests for Windows PGO builds

https://hg.mozilla.org/projects/date/rev/50b281079b2e0efae51cec67301b76dab1bf3c70
Bug 1379661: Enable tests on PGO builds

https://hg.mozilla.org/projects/date/rev/dc0db9c96eb039670436de48f905d107eec5e150
Bug 1379661: Enable CI signing and tests that require signed builds for PGO builds

https://hg.mozilla.org/projects/date/rev/a857381582e276d2a90094f0cc63d5dc950c6abc
Bug 1379661: Add quantum pageload talos tests for Windows

Comment 7

[Chris AtLee [:catlee]]

https://hg.mozilla.org/projects/date/rev/02668d70cafc882aee94e22ff0f4290e1783f3c9
Bug 1379661: Remove unused test tests and fix xperf platforms

Comment 8

[Chris AtLee [:catlee]]

https://hg.mozilla.org/projects/date/rev/af885a811fd995c4f472a549e8f63a3786431c20
Bug 1379661: Enable mochitest-chrome and mochitest-clipboard tests on win7 via BBB

https://hg.mozilla.org/projects/date/rev/2a1b67a2e51cb3eb808e6d6952cfd0029b267588
Bug 1379661: Get tests working on Win8 via BBB

Comment 9

[Chris AtLee [:catlee]]

The current plan is outlined here:
https://docs.google.com/spreadsheets/d/1e4gTV0eQDl4lc_u8N3PbxkuGk81rVRPEXOZpTJdnTjY/edit#gid=0

For 32-bit, Win7 all tests will be run on TC except for mochitest-chrome, mochitest-clipboard and talos.

For 64-bit:
* we will run awsy, cppunit, gtest, jittest, jsreftest, marionette, mochitest-a11y, mochitest-jetpack, web-platform-tests, webplatform-tests-reftests and xpcshell on Win10 in TC
* Talos will be run on Win10 in BB
* crashtest, mochitest, mochitest-e10s, mochitest-browser-chrome, mochitest-devtools-chrome, mochitest-media, reftest, reftest-no-accel, mochitest-gpu, mochitest-webgl, mochitest-chrome and mochitest-clipboard will be run on Win8 in BB.

Comment 10

[Chris AtLee [:catlee]]

https://hg.mozilla.org/projects/date/rev/f733a5e1974ce06956e481f819949b1e36459045
Bug 1379661: Fix win8 typo

Comment 11

[Chris AtLee [:catlee]]

https://hg.mozilla.org/projects/date/rev/04b19c381e4d8c231f47048d416a73a58764d90e
Bug 1379661: Enable more win8 tests using BBB

Comment 12

[Chris AtLee [:catlee]]

https://hg.mozilla.org/projects/date/rev/adcbd61370f20ac706c34d2ead71b8f10299f3c1
Bug 1379661: Enable complete set of win7 and win10 tests, and fix tiers

Comment 13

[Chris AtLee [:catlee]]

https://hg.mozilla.org/projects/date/rev/7c6a0caeaf4dccf148cf4cbd2d48af3bc6580468
Bug 1379661: Fix up treeherder names

https://hg.mozilla.org/projects/date/rev/cebdeade691b69034f3847cfd6de037ee4b35024
Bug 1379661: Make nightly windows tasks obey run-on-projects