1714122 - [perfdocs] Add the run-on-projects settings to the perfdocs test task list

Status: RESOLVED FIXED

(Testing :: Performance, task, P1)

Description

[Greg Mierzwinski [:sparky]]

This bug is for adding the run-on-projects settings (like ["mozilla-central", "autoland", ...]) from the taskcluster task information into the test task list.

We have a couple options for how we can organize this data. Using this page as an example: https://firefox-source-docs.mozilla.org/testing/perfdocs/raptor.html#process-switch-c

Option 1:

   * **Test Task**:
      * test-linux1804-64-qr/opt-browsertime-custom-firefox-process-switch-e10s: ["mozilla-central"]
      * test-linux1804-64-shippable-qr/opt-browsertime-custom-firefox-process-switch-e10s: ["mozilla-central"]
      * test-linux1804-64-shippable-qr/opt-browsertime-custom-firefox-process-switch-fis-e10s: ["mozilla-central"]
      * test-linux1804-64-shippable/opt-browsertime-custom-firefox-process-switch-e10s: ["mozilla-central"]
      * test-linux1804-64/opt-browsertime-custom-firefox-process-switch-e10s: ["mozilla-central"]
      * test-macosx1014-64-shippable-qr/opt-browsertime-custom-firefox-process-switch-e10s: ["mozilla-central"]
      * test-macosx1014-64-shippable-qr/opt-browsertime-custom-firefox-process-switch-fis-e10s: ["mozilla-central"]
      * test-macosx1015-64-shippable-qr/opt-browsertime-custom-firefox-process-switch-e10s: ["mozilla-central"]
      * ...

Option 2:

   * **Test Task**:
      * mozilla-central:
            * test-linux1804-64-qr/opt-browsertime-custom-firefox-process-switch-e10s
            * test-linux1804-64-shippable-qr/opt-browsertime-custom-firefox-process-switch-e10s
            * test-linux1804-64-shippable-qr/opt-browsertime-custom-firefox-process-switch-fis-e10s
            * test-linux1804-64-shippable/opt-browsertime-custom-firefox-process-switch-e10s
            * test-linux1804-64/opt-browsertime-custom-firefox-process-switch-e10s
            * test-macosx1014-64-shippable-qr/opt-browsertime-custom-firefox-process-switch-e10s
            * test-macosx1014-64-shippable-qr/opt-browsertime-custom-firefox-process-switch-fis-e10s
            * test-macosx1015-64-shippable-qr/opt-browsertime-custom-firefox-process-switch-e10s
            * test-macosx1015-64-shippable-qr/opt-browsertime-custom-firefox-process-switch-fis-e10s
            * ...
      * autoland
            * ...

I prefer if we go with option 2. In this case, when we see an empty run-on-projects (run-on-projects: [] we should put the task into a list called Not running in CI.

Comment 1

[Myeongjun Go]

Hello sparky :)
I have something to ask you.
One 'run_on_projects' can have multiple tests.
For example, A test such as test-windows10-64-shippable-qr/opt-browsertime-tp6-essential-firefox-amazon-e10s
have two run_on_projects (i,e. ['mozilla-beta', 'trunk'])

The test can be duplicated as follows.

* mozilla-beta:
     * test-windows10-64-shippable-qr/opt-browsertime-tp6-essential-firefox-amazon-e10s
* trunk:
     * test-windows10-64-shippable-qr/opt-browsertime-tp6-essential-firefox-amazon-e10s

I think that can be duplicated.
Are you considering in this case?

Comment 2

[Myeongjun Go]

I'm wondering about your opinion if this is right :)

Comment 3

[Myeongjun Go]

Attachment: Bug 1714122 - [perfdocs] Add the run-on-projects settings to the perfdocs test task list r=sparky

Comment 4

[Greg Mierzwinski [:sparky]]

We discussed this in our meeting today and we'll use this format instead: https://github.com/gmierz/moz-current-tests#generating-a-test-report

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f2d23545d92d
[perfdocs] Add the run-on-projects settings to the perfdocs test task list r=sparky,perftest-reviewers

Comment 6

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/f2d23545d92d