1520417 - Rename raptor-tp6 subtests to remove redundant test name prefix

Status: RESOLVED FIXED

(Testing :: Raptor, enhancement, P1)

Description

[Dave Hunt [:davehunt] [he/him] ⌚BST]

The subtest names for raptor-tp6 repeat the test name, contributing to issues such as bug 1517512. We should clean these up, so tests such as:

• raptor-tp6-amazon-firefox-dcf
• raptor-tp6-amazon-firefox-fnbpaint
• raptor-tp6-amazon-firefox-ttfi
• raptor-tp6-amazon-firefox-hero:hero1
• raptor-tp6-amazon-firefox-loadtime

Are renamed to:

• dcf
• fnbpaint
• ttfi
• hero:hero1
• loadtime

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

We're not currently alerting on these, so we should be safe for us to test our solution to bug 1520170 when we rename these.

Comment 2

[Ionuț Goldan [:igoldan]]

On 1st look, this doesn't look like a basic *.ini tweak. The following line from output.py generates the long version of the name:

new_subtest['name'] = test.name + "-" + measurement_name

Updating it would change names for all our Raptor tp6 subtests. Benchmarks will remain the same, but they're already ok.

Comment 3

[Ionuț Goldan [:igoldan]]

Attachment: Bug 1520417 - Do not combine test name with measurement

Comment 4

[Ionuț Goldan [:igoldan]]

:rwood could you review this PR, please?

Comment 5

[Robert Wood [:rwood]]

(In reply to Ionuț Goldan [:igoldan], Performance Sheriffing from comment #4)

:rwood could you review this PR, please?

Yes I'll have a look, thanks. Do you have a try push? Also are there any implications for this in perfherder - you know more about perfherder than I do, so I'm assuming you've already covered that. :)

Comment 6

[Robert Wood [:rwood]]

(In reply to Robert Wood [:rwood] from comment #5)

Yes I'll have a look, thanks. Do you have a try push? Also are there any implications for this in perfherder - you know more about perfherder than I do, so I'm assuming you've already covered that. :)

Ah - I found the try link you noted in phabricator, thanks

https://treeherder.mozilla.org/#/jobs?repo=try&revision=b3dcb6495f64d1790cb9b4722ca36cfa7ac53c86

Comment 7

[Ionuț Goldan [:igoldan]]

(In reply to Robert Wood [:rwood] from comment #6)

Also are there any implications for this in perfherder - you know more about perfherder than I do, so I'm assuming you've already covered that. :)

It shouldn't cause problems. I double checked the initial ingestion phase and it looks ok.
On the FE side, the Javascript shouldn't have problems with the shorter name form; it should treat it as it is, without looking for any patterns. I already found recent Try signatures with dcf/fcp/etc subtests in the Graphs view, backing that up.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9c24a1d10557
Do not combine test name with measurement r=rwood

Comment 9

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/9c24a1d10557

Comment 10

[Ionuț Goldan [:igoldan]]

Reopening as this hasn't been handled on Perfherder yet.

Comment 11

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/treeherder/pull/4627

Comment 12

[Ionuț Goldan [:igoldan]]

Joel, could you have a look over the PR from comment 11? I need someone's review, before merging & running the management script on production.

Comment 13

[Joel Maher ( :jmaher ) (UTC -8)]

nice PR- the logic looks sound

Comment 14

[Ionuț Goldan [:igoldan]]

(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #13)

nice PR- the logic looks sound

Thanks! I've just sent you the review request.

Comment 15

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/releases/mozilla-beta/rev/6285113cbb13

Comment 16

[Ionuț Goldan [:igoldan]]

The new script got deployed on production. It ran successfully. Thus, we can close this bug.