1431427 - Add nightly talos profiling job for Linux64

Status: RESOLVED FIXED

(Testing :: Talos, enhancement)

Description

[Robert Wood [:rwood]]

Bug 1425850 adds a new set of talos profiling tests that run on nightly OSX. These are the regular talos tests but with geckoProfiling turned on, so we can have nightly talos profiling artifacts. They report in a new treeherder group called "T-P".

Now add the same for Windows 10 & Linux64 (which requires buildbot configs and corresponding treeherder etl buildbot data ingestion updates).

Comment 1

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/treeherder/pull/3162

Comment 2

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/14bf8c4c8eadb848bfbd464f1f768396a830dac4
Bug 1431427 - Update buildbot etl for new talos profiling suite (#3162)

Comment 3

[Robert Wood [:rwood]]

Next: The actual buildbot configs

Comment 4

[Robert Wood [:rwood]]

For taskcluster I was able to just specify using '--geckoProfile' inside the tc configs themselves. For buildbot turns out I need to add matching talos job/suite names in talos.json afterall...

Comment 5

[Robert Wood [:rwood]]

Attachment: Bug 1431427 - Add talos suite configs for nightly profiling suite;

Review commit: https://reviewboard.mozilla.org/r/215050/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/215050/

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8944885 [details]
Bug 1431427 - Add talos suite configs for nightly profiling suite;

https://reviewboard.mozilla.org/r/215050/#review220664

one issue about svg

::: testing/talos/talos.json:100
(Diff revision 1)
>          "g5-stylo-disabled-e10s": {
>              "tests": ["ts_paint_webext", "tp5o_webext"],
>              "talos_options": ["--disable-stylo"],
>              "pagesets_name": "tp5n.zip"
>          },
>          "svgr-e10s": {

what about svgr?

::: testing/talos/talos.json:162
(Diff revision 1)
>              "talos_options": [
>                  "--xperf_path",
>                  "\"c:/Program Files/Microsoft Windows Performance Toolkit/xperf.exe\""
>              ]
>          },
> +        "xperf-profiling-e10s": {

not sure if we need xperf-profiling- that is technically a profiling job- it is probably fine to leave it here though.

Comment 7

[Robert Wood [:rwood]]

Attachment: bug1431427.patch

Buildbot configs for the new talos profiling nighty job. Question: How do I make this only run on the *nightly* builds? That's where I'm stuck!

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8944887 [details] [diff] [review]
bug1431427.patch

Review of attachment 8944887 [details] [diff] [review]:
-----------------------------------------------------------------

::: mozilla-tests/config.py
@@ +2965,5 @@
>          branch['perf-reftest-singletons-stylo-disabled-e10s_tests'] = (1, False, {}, ALL_TALOS_PLATFORMS)
>  
> +# Enable talos profiling suite on 60+ for m-c and try
> +for name, branch in items_at_least(BRANCHES, 'gecko_version', 60):
> +    if branch.get('enable_talos') is False:

I think doing try and m-c is good enough to keep this limited- right now all scheduling is done via taskcluster through buildbotbridge, so having the taskcluster configs to control nightly only would be the way to limit it.

Comment 9

[Robert Wood [:rwood]]

Comment on attachment 8944885 [details]
Bug 1431427 - Add talos suite configs for nightly profiling suite;

https://reviewboard.mozilla.org/r/215050/#review220664

> what about svgr?

Ah totally missed that one thank you

> not sure if we need xperf-profiling- that is technically a profiling job- it is probably fine to leave it here though.

Oh kk, actually I'll remove it since it only runs on Win 7 anyway and that may not be around much longer for talos

Comment 10

[Robert Wood [:rwood]]

Comment on attachment 8944885 [details]
Bug 1431427 - Add talos suite configs for nightly profiling suite;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/215050/diff/1-2/

Comment 11

[Robert Wood [:rwood]]

Attachment: bug1431427_bb.patch

Same buildbot config patch (adding new talos profiling suite) but without xperf.

Comment 12

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8944885 [details]
Bug 1431427 - Add talos suite configs for nightly profiling suite;

https://reviewboard.mozilla.org/r/215050/#review220928

thanks for adding stylo in here

Comment 13

[Robert Wood [:rwood]]

Attachment: Bug 1431427 - Add tc linux configs for new talos nightly profiling suite;

Review commit: https://reviewboard.mozilla.org/r/216312/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/216312/

Comment 14

[Robert Wood [:rwood]]

Comment on attachment 8944885 [details]
Bug 1431427 - Add talos suite configs for nightly profiling suite;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/215050/diff/2-3/

Comment 15

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8946346 [details]
Bug 1431427 - Add tc linux configs for new talos nightly profiling suite;

https://reviewboard.mozilla.org/r/216312/#review222090

excited to see this happening!

Comment 16

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=02a8c55082c8d4abea0e2d24dd0eca1ce9639649

Comment 17

[Robert Wood [:rwood]]

Comment on attachment 8946346 [details]
Bug 1431427 - Add tc linux configs for new talos nightly profiling suite;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/216312/diff/1-2/

Comment 18

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=16485192b61091581a808ef0c5f2e3560d7b86bd

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0340f63e2b79
Add talos suite configs for nightly profiling suite; r=jmaher
https://hg.mozilla.org/integration/autoland/rev/f9db7e6d1229
Add tc linux configs for new talos nightly profiling suite; r=jmaher

Comment 20

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/0340f63e2b79
https://hg.mozilla.org/mozilla-central/rev/f9db7e6d1229