1693051 - Update tests with suite-level values to produce replicates at the suite-level

Status: RESOLVED FIXED

(Tree Management :: Perfherder, task, P3)

Description

[Ionuț Goldan [:igoldan]]

Update the targeted test harnesses to produce an extra & special subtest per each of the suites identified in FXP-1483.
This new subtest' specification is described from [comment #4|https://bugzilla.mozilla.org/show_bug.cgi?id=1674370#c4] to [comment #9|https://bugzilla.mozilla.org/show_bug.cgi?id=1674370#c9] (includes a PERFHERDER_DATA snippet as example).

Jira link: https://jira.mozilla.com/browse/FXP-1484

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

:sparky the following question is outstanding in Jira:

Gregory Mierzwinski are the acceptance criteria reasonable?

The acceptance criteria are currently listed as:

• this issue impacts all suites identified in FXP-1483
• suites identified in FXP-1483 should not alert anymore
• all extra & special subtests should alert instead

Comment 2

[Greg Mierzwinski [:sparky]]

Yup, those are good to me.

Comment 3

[Greg Mierzwinski [:sparky]]

Instead of doing this with a new subtest, we could simply add a replicates field at the suite-level. That would also make it simpler to find and be more inline with that the suite-level value is. Using subtests can make this confusing. Bug 1929962 is for that work.

Comment 4

[Greg Mierzwinski [:sparky]]

Reworking the title of this bug to make it clearer that it's for adding the suite-level replicates to our tests.

Comment 5

[Myeongjun Go]

Attachment: Bug 1693051 - Update tests with suite-level values to produce replicates at the suite-level r=sparky

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/7d387e8f7272
https://hg.mozilla.org/integration/autoland/rev/7e03dad76b8d
Update tests with suite-level values to produce replicates at the suite-level r=sparky,perftest-reviewers

Comment 7

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/7e03dad76b8d