1601414 - Add application name and version in vismet perfherder results

Status: RESOLVED FIXED

(Testing :: Raptor, enhancement, P1)

Description

[Tarek Ziadé (:tarek)]

When Raptor generates the browsertime tarball for the vismet task, it should include a small json file containing the application name and version, so we can copy it inside vismet's perfherder results.

Comment 1

[Robert Wood [:rwood]]

First patch will be adding the application.json file to the browsertime-results.tgz. Second patch will be having the visual metrics task itself grab that info and use it in it's perfherder data output.

Comment 2

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=1fd0483fa7772c7e5e0ec48d67c22c796b8095e8

Comment 3

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=1fd0483fa7772c7e5e0ec48d67c22c796b8095e8&selectedJob=282979767

In the Job Details, 'browsertime-results.tgz' tarball you'll now find an 'application.json' that contains:

{"application": {"version": "73.0a1", "name": "firefox"}}

Comment 4

[Robert Wood [:rwood]]

Attachment: Bug 1601414 - Add application info json file to browsertime results for vismet jobs

Comment 5

[Robert Wood [:rwood]]

Attachment: Bug 1601414 - Add application name and version in vismet perfherder results

Comment 6

[Robert Wood [:rwood]]

Try push with both patches:

https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=5c12c68ad998927df4a0430d7f32f5736b12df14

Comment 7

[Robert Wood [:rwood]]

Try push with both again, and updates including adding the application name to perfherder 'extraOptions' (required until Bug 1593198 is fixed):

https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=359b685e90a49a6eb0deb6e593771a680069190b

Comment 8

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=87ce2dc54be8a63beea13b5ad819ff66dcc0995e

Ok ready for review, on the try run you'll see that the tp6-c-1-vismet task PERFHERDER_DATA output now includes the 'application' key (and the application name is also in extraOptions):

PERFHERDER_DATA: {"suites": [{"name": "amazon-cold", "extraOptions": ["firefox"],...
"type": "vismet", "framework": {"name": "browsertime"}, "application": {"version": "73.0a1", "name": "firefox"}}

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/11ea65b2b31c
Add application info json file to browsertime results for vismet jobs r=perftest-reviewers,stephendonner

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6865e8d68595
Add application name and version in vismet perfherder results r=perftest-reviewers,stephendonner

Comment 11

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/11ea65b2b31c
https://hg.mozilla.org/mozilla-central/rev/6865e8d68595

Comment 12

[Alexandru Ionescu (needinfo me) [:alexandrui]]

== Change summary for alert #24608 (as of Fri, 03 Jan 2020 19:40:56 GMT) ==

Improvements:

25% raptor-webaudio-firefox macosx1014-64-shippable opt 196.00 -> 146.83
21% raptor-webaudio-firefox linux64-shippable opt 150.83 -> 119.25
21% raptor-webaudio-firefox linux64-shippable-qr opt 157.50 -> 124.67
10% raptor-webaudio-firefox windows10-64-shippable opt 160.38 -> 145.00
9% raptor-webaudio-firefox windows10-64-shippable-qr opt 159.42 -> 144.42
8% raptor-webaudio-firefox windows7-32-shippable opt 197.04 -> 180.58

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=24608

Comment 13

[Robert Wood [:rwood]]

(In reply to Alexandru Ionescu :alexandrui (needinfo me) from comment #12)

== Change summary for alert #24608 (as of Fri, 03 Jan 2020 19:40:56 GMT) ==

Improvements:

25% raptor-webaudio-firefox macosx1014-64-shippable opt 196.00 -> 146.83
21% raptor-webaudio-firefox linux64-shippable opt 150.83 -> 119.25
21% raptor-webaudio-firefox linux64-shippable-qr opt 157.50 -> 124.67
10% raptor-webaudio-firefox windows10-64-shippable opt 160.38 -> 145.00
9% raptor-webaudio-firefox windows10-64-shippable-qr opt 159.42 -> 144.42
8% raptor-webaudio-firefox windows7-32-shippable opt 197.04 -> 180.58

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=24608

This must be an error; the patches landed in this bug only added additional metadata to the browsertime results and the visual metrics tasks; the improvements listed above are on raptor-webaudio which isn't run on browsertime and is unrelated. Thanks!

Comment 14

[Dave Hunt [:davehunt] [he/him] ⌚BST]

:alexandrui this improvement appears to be associated with the wrong bug and is appearing in a query for top improvements, could you take another look at this to see if there is another cause?

Comment 15

[Alexandru Ionescu (needinfo me) [:alexandrui]]

I can't make any investigations as the jobs from this period are no longer available in CI.

Comment 16

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Alexandru Ionescu (needinfo me) [:alexandrui] from comment #15)

I can't make any investigations as the jobs from this period are no longer available in CI.

Given this is unlikely to be the cause of the improvement, even without the original jobs we should be able to speculate on a more likely cause. From looking at the pushlog, I imagine bug 1596182 is much more likely due to it being a webaudio optimisation.