Closed Bug 1383257 Opened 7 years ago Closed 7 years ago

Land changes to flip windows nightlies on, and to tier 1 on central

Categories

(Release Engineering :: General, enhancement)

Product:
Release Engineering
Component:
General
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: Callek, Assigned: Callek)

References

Details

Attachments

(21 files)

Bug 1383257 - DO NOT LAND - changes applied only to date branch
59 bytes, text/x-review-board-request
Details
Bug 1383257 - Disable talos and unittest sendchange from windows builds.
59 bytes, text/x-review-board-request
kmoir
: review+
Details
Bug 1383257 - Don't use tc config for newly added talos suites (to be run via BBB).
59 bytes, text/x-review-board-request
kmoir
: review+
Details
Bug 1383257 - Enable win8 testing (via BBB).
59 bytes, text/x-review-board-request
kmoir
: review+
Details
Bug 1383257 - Enable win talos testing (via BBB).
59 bytes, text/x-review-board-request
kmoir
: review+
Details
Bug 1383257 - Mark mochitest-chrome on w7 as try-only.
59 bytes, text/x-review-board-request
kmoir
: review+
Details
Bug 1383257 - XPCShell is not tier 3.
59 bytes, text/x-review-board-request
kmoir
: review+
Details
Bug 1383257 - Don't run gtest on win pgo (it permafails both in tc and buildbot).
59 bytes, text/x-review-board-request
kmoir
: review+
Details
Bug 1383257 - Marionette is green, so doesn't need to be tier 3.
59 bytes, text/x-review-board-request
kmoir
: review+
Details
Bug 1383257 - JSReftest is green (only known intermittents), so doesn't need to be tier 3.
59 bytes, text/x-review-board-request
kmoir
: review+
Details
Bug 1383257 - XPCShell is currently failing on win10, run it on win8 until we can switch. Leave running on win10 as tier 2.
59 bytes, text/x-review-board-request
kmoir
: review+
Details
Bug 1383257 - When PGO runs tests via BBB they end up mixed in with the opt variant BBB tests.
59 bytes, text/x-review-board-request
catlee
: review+
Details
Bug 1383257 - Run win talos on all expected branches.
59 bytes, text/x-review-board-request
kmoir
: review+
Details
Bug 1383257: Switch Windows debug reftest-gpu tests to use BBB
59 bytes, text/x-review-board-request
Callek
: review+
Details
Bug 1383257 - Run talos-perf-singletons on all branches.
59 bytes, text/x-review-board-request
jmaher
: review+
Details
Bug 1383257 - Schedule devedition testing too.
59 bytes, text/x-review-board-request
catlee
: review+
Details
Bug 1383257 - Actually flip builds to tier 1.
59 bytes, text/x-review-board-request
catlee
: review+
Details
Bug 1383257 - Signing is always tier 1.
59 bytes, text/x-review-board-request
kmoir
: review+
Details
Bug 1383257 - Flip windows testing to be tier 1 by default.
59 bytes, text/x-review-board-request
kmoir
: review+
Details
Bug 1383257 - Do not attempt to schedule win10 hardware testing, unless we're on try.
59 bytes, text/x-review-board-request
grenade
: review+
Details
Bug 1383257 - Schedule windows nightlies via cron.
59 bytes, text/x-review-board-request
kmoir
: review+
Details 
+++ This bug was initially created as a clone of Bug #1381577 +++

This patch set will enable tier 1 and related BBB testing.
Comment on attachment 8889939 [details]
Bug 1383257 - Disable talos and unittest sendchange from windows builds.

https://reviewboard.mozilla.org/r/160996/#review166318
Attachment #8889939 - Flags: review?(kmoir) → review+
Comment on attachment 8889940 [details]
Bug 1383257 - Don't use tc config for newly added talos suites (to be run via BBB).

https://reviewboard.mozilla.org/r/160998/#review166322
Attachment #8889940 - Flags: review?(kmoir) → review+
Comment on attachment 8889941 [details]
Bug 1383257 - Enable win8 testing (via BBB).

https://reviewboard.mozilla.org/r/161000/#review166324
Attachment #8889941 - Flags: review?(kmoir) → review+
Comment on attachment 8889942 [details]
Bug 1383257 - Enable win talos testing (via BBB).

https://reviewboard.mozilla.org/r/161002/#review166326
Attachment #8889942 - Flags: review?(kmoir) → review+
Comment on attachment 8889943 [details]
Bug 1383257 - Mark mochitest-chrome on w7 as try-only.

https://reviewboard.mozilla.org/r/161004/#review166330
Attachment #8889943 - Flags: review?(kmoir) → review+
Comment on attachment 8889944 [details]
Bug 1383257 - XPCShell is not tier 3.

https://reviewboard.mozilla.org/r/161006/#review166332
Attachment #8889944 - Flags: review?(kmoir) → review+
Comment on attachment 8889987 [details]
Bug 1383257 - Don't run gtest on win pgo (it permafails both in tc and buildbot).

https://reviewboard.mozilla.org/r/161030/#review166360
Attachment #8889987 - Flags: review?(kmoir) → review+
Comment on attachment 8889988 [details]
Bug 1383257 - Marionette is green, so doesn't need to be tier 3.

https://reviewboard.mozilla.org/r/161032/#review166362
Attachment #8889988 - Flags: review?(kmoir) → review+
Comment on attachment 8889989 [details]
Bug 1383257 - JSReftest is green (only known intermittents), so doesn't need to be tier 3.

https://reviewboard.mozilla.org/r/161034/#review166364
Attachment #8889989 - Flags: review?(kmoir) → review+
Comment on attachment 8889990 [details]
Bug 1383257 - XPCShell is currently failing on win10, run it on win8 until we can switch. Leave running on win10 as tier 2.

https://reviewboard.mozilla.org/r/161036/#review166366
Attachment #8889990 - Flags: review?(kmoir) → review+
Assignee: nobody → bugspam.Callek
Comment on attachment 8890010 [details]
Bug 1383257: Switch Windows debug reftest-gpu tests to use BBB

https://reviewboard.mozilla.org/r/161066/#review166384
Attachment #8890010 - Flags: review?(bugspam.Callek) → review+
Comment on attachment 8890018 [details]
Bug 1383257 - Run talos-perf-singletons on all branches.

https://reviewboard.mozilla.org/r/161068/#review166394

::: taskcluster/ci/test/tests.yml:1530
(Diff revision 1)
>      description: "Talos perf-reftest singletons"
>      suite: talos
>      try-name: perf-reftest-singletons
>      treeherder-symbol: tc-T(ps)
>      virtualization: hardware
> -    run-on-projects: ['mozilla-central', 'try']
> +    run-on-projects: ['mozilla-beta', 'mozilla-central', 'mozilla-inbound', 'autoland', 'try']

this shouldn't run on beta until the merge next week-  I assume that is the plan?
Attachment #8890018 - Flags: review?(jmaher) → review+
Comment on attachment 8890018 [details]
Bug 1383257 - Run talos-perf-singletons on all branches.

https://reviewboard.mozilla.org/r/161068/#review166394

> this shouldn't run on beta until the merge next week-  I assume that is the plan?

Correct, with this patch it will start running on beta as of next week's merge. But in a way that doesn't require us to land more code during the merge.
Comment on attachment 8890026 [details]
Bug 1383257 - Signing is always tier 1.

https://reviewboard.mozilla.org/r/161074/#review166408
Attachment #8890026 - Flags: review?(kmoir) → review+
Comment on attachment 8890027 [details]
Bug 1383257 - Flip windows testing to be tier 1 by default.

https://reviewboard.mozilla.org/r/161076/#review166412
Attachment #8890027 - Flags: review?(kmoir) → review+
Comment on attachment 8890044 [details]
Bug 1383257 - Schedule windows nightlies via cron.

https://reviewboard.mozilla.org/r/161096/#review166424
Attachment #8890044 - Flags: review?(kmoir) → review+
Comment on attachment 8890043 [details]
Bug 1383257 - Do not attempt to schedule win10 hardware testing, unless we're on try.

https://reviewboard.mozilla.org/r/161094/#review166426
Attachment #8890043 - Flags: review?(rthijssen) → review+
Comment on attachment 8890008 [details]
Bug 1383257 - When PGO runs tests via BBB they end up mixed in with the opt variant BBB tests.

https://reviewboard.mozilla.org/r/161062/#review166436
Attachment #8890008 - Flags: review?(catlee) → review+
Comment on attachment 8890009 [details]
Bug 1383257 - Run win talos on all expected branches.

https://reviewboard.mozilla.org/r/161064/#review166440
Attachment #8890009 - Flags: review?(kmoir) → review+
Comment on attachment 8890019 [details]
Bug 1383257 - Schedule devedition testing too.

https://reviewboard.mozilla.org/r/161070/#review166442

::: taskcluster/ci/test/test-platforms.yml:173
(Diff revision 1)
> +    build-platform: win32-nightly/opt
> +    test-sets:
> +        - awsy
> +        - desktop-screenshot-capture
> +        - headless
> +        - windows-tests

what about talos?

::: taskcluster/ci/test/test-platforms.yml:214
(Diff revision 1)
> +    build-platform: win64-nightly/opt
> +    test-sets:
> +        - awsy
> +        - desktop-screenshot-capture
> +        - headless
> +        - windows-tests

what about talos?
Comment on attachment 8890025 [details]
Bug 1383257 - Actually flip builds to tier 1.

https://reviewboard.mozilla.org/r/161072/#review166444
Attachment #8890025 - Flags: review?(catlee) → review+
Comment on attachment 8890019 [details]
Bug 1383257 - Schedule devedition testing too.

https://reviewboard.mozilla.org/r/161070/#review166446
Attachment #8890019 - Flags: review?(catlee) → review+
Comment on attachment 8890019 [details]
Bug 1383257 - Schedule devedition testing too.

Please re-review based on https://reviewboard.mozilla.org/r/161070/diff/1-2/
Attachment #8890019 - Flags: review?(catlee)
Comment on attachment 8890019 [details]
Bug 1383257 - Schedule devedition testing too.

https://reviewboard.mozilla.org/r/161070/#review166526
Attachment #8890019 - Flags: review?(catlee) → review+
Pushed by Callek@gmail.com:
https://hg.mozilla.org/mozilla-central/rev/0d4a575f389c
Disable talos and unittest sendchange from windows builds. r=kmoir
https://hg.mozilla.org/mozilla-central/rev/ab34bd7e0d63
Don't use tc config for newly added talos suites (to be run via BBB). r=kmoir
https://hg.mozilla.org/mozilla-central/rev/e47352cd8afa
Enable win8 testing (via BBB). r=kmoir
https://hg.mozilla.org/mozilla-central/rev/c34983848092
Enable win talos testing (via BBB). r=kmoir
https://hg.mozilla.org/mozilla-central/rev/1b66fdca3d4b
Mark mochitest-chrome on w7 as try-only. r=kmoir
https://hg.mozilla.org/mozilla-central/rev/5478ebe80a50
XPCShell is not tier 3. r=kmoir
https://hg.mozilla.org/mozilla-central/rev/5841063d304f
Don't run gtest on win pgo (it permafails both in tc and buildbot). r=kmoir
https://hg.mozilla.org/mozilla-central/rev/786e0e0ae2e4
Marionette is green, so doesn't need to be tier 3. r=kmoir
https://hg.mozilla.org/mozilla-central/rev/312a0be92afb
JSReftest is green (only known intermittents), so doesn't need to be tier 3. r=kmoir
https://hg.mozilla.org/mozilla-central/rev/200ed040aa1c
XPCShell is currently failing on win10, run it on win8 until we can switch. Leave running on win10 as tier 2. r=kmoir
https://hg.mozilla.org/mozilla-central/rev/935018d4d6ad
When PGO runs tests via BBB they end up mixed in with the opt variant BBB tests. r=catlee
https://hg.mozilla.org/mozilla-central/rev/030f98cf2cda
Run win talos on all expected branches. r=kmoir
https://hg.mozilla.org/mozilla-central/rev/0ccfb831945b
Switch Windows debug reftest-gpu tests to use BBB r=Callek
https://hg.mozilla.org/mozilla-central/rev/dd1be262da6f
Run talos-perf-singletons on all branches. r=jmaher
https://hg.mozilla.org/mozilla-central/rev/6c65946c19a3
Schedule devedition testing too. r=catlee
https://hg.mozilla.org/mozilla-central/rev/82a855806ae9
Actually flip builds to tier 1. r=catlee
https://hg.mozilla.org/mozilla-central/rev/44c248af1597
Signing is always tier 1. r=kmoir
https://hg.mozilla.org/mozilla-central/rev/638d48362a57
Flip windows testing to be tier 1 by default. r=kmoir
https://hg.mozilla.org/mozilla-central/rev/fd77c8ca32b7
Do not attempt to schedule win10 hardware testing, unless we're on try. r=grenade
https://hg.mozilla.org/mozilla-central/rev/388d81ed93fa
Schedule windows nightlies via cron. r=kmoir a=Tomcat
Depends on: 1384728
Looks like from treeherder m-i, m-c that no windows builds are running/starting.
I see a perf improvement from here:
== Change summary for alert #8322 (as of July 26 2017 15:10 UTC) ==

Improvements:

 17%  tp5n main_startup_fileio windows7-32 pgo e10s     82,145,881.76 -> 68,408,997.33

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=8322


this pretty much aligns the pgo with opt values:
https://treeherder.mozilla.org/perf.html#/graphs?series=%5Bautoland,ac555d733ad90cd22e1fcc0b280b9d8ef3979d0b,1,1%5D&series=%5Bautoland,f5df38aff6d94ef9ffdaea648fd465fb0249bf1f,1,1%5D

overall, I think we are probably doing pgo correctly now :)
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Depends on: 1385150
Depends on: 1387831
Depends on: 1387878
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: