1606131 - Have the browsertime test jobs differentiate between app in the perfherder data output

Status: RESOLVED FIXED

(Tree Management :: Perfherder, task, P1)

Description

[Robert Wood [:rwood]]

The browsertime visual-metrics task PERFHERDER_DATA output is not distinguishing between app. So for example for vismetrics jobs on Fenix and GVE, the name is the same so the data is all mixed:

https://treeherder.mozilla.org/perf.html#/graphs?highlightAlerts=1&selected=2200748,618674,1321.5,350,1011158630&series=try,2200748,1,13&timerange=1209600

Comment 1

[Robert Wood [:rwood]]

Perfherder doesn't distinguish between 'app' yet, i.e. for Chrome jobs we report it, but perfherder doesn't split out the results for it yet:

INFO - perftest-output Info: PERFHERDER_DATA: {"framework": {"name": "raptor"}, "application": {"version": "76.0.3809.132", "name": "chrome"},

Bug 1593198 is for that support in Perfherder.

In the mean time for these visual metrics jobs I'll add the application name to PERFHERDER_DATA extraOptions for now - then once Bug 1593198 is fixed we can add the application name key itself and remove it from extraOptions.

Comment 2

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=64234bca58dfbe0031b79cda137ce0fa11c57d12

Comment 3

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=937af7c132f022a082880237d7e9caf219570e8f

Comment 4

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=061d8aa569ef7fa5b37cf073da3f5df59ac87c25

Comment 5

[Robert Wood [:rwood]]

(In reply to Robert Wood [:rwood] from comment #4)

https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=061d8aa569ef7fa5b37cf073da3f5df59ac87c25

From the try run you can see the app being added to perfherder extra options, i.e. for Fenix:

INFO - perftest-output Info: PERFHERDER_DATA: {"framework": {"name": "browsertime"}, "application": {"name": "fenix"}, "suites": [{"extraOptions": ["fenix"],...

So that will fix the issue with the regular browsertime jobs (ensuring the perfherder data is differentiated by browser/app) however that will not fix the vismets job issue - that will require Bug 1601414 to be fixed first, which will provide the app/browser info to the vismets task. Will submit this first patch to fix the non-vismet task jobs.

Comment 6

[Robert Wood [:rwood]]

Attachment: Bug 1606131 - Have the browsertime test jobs differentiate between app in the perfherder data output

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ef007d37a0cf
Have the browsertime test jobs differentiate between app in the perfherder data output r=perftest-reviewers,stephendonner

Comment 8

[Oana Pop-Rus]

https://hg.mozilla.org/mozilla-central/rev/ef007d37a0cf

Comment 9

[Robert Wood [:rwood]]

Bug 1601414 has landed code needed to differentiate the perfherder data by browser app for the visual metrics jobs, so this bug is now resolved.