Closed Bug 1601414 Opened 1 year ago Closed 11 months ago

Add application name and version in vismet perfherder results

Categories

(Testing :: Raptor, enhancement, P1)

Version 3
enhancement

Tracking

(firefox73 fixed)

RESOLVED FIXED
mozilla73
Tracking Status
firefox73 --- fixed

People

(Reporter: tarek, Assigned: rwood)

References

(Blocks 1 open bug)

Details

Attachments

(2 files)

When Raptor generates the browsertime tarball for the vismet task, it should include a small json file containing the application name and version, so we can copy it inside vismet's perfherder results.

Assignee: nobody → tarek
Priority: -- → P2
Blocks: 1604527
Assignee: tarek → rwood
Status: NEW → ASSIGNED
Priority: P2 → P1

First patch will be adding the application.json file to the browsertime-results.tgz. Second patch will be having the visual metrics task itself grab that info and use it in it's perfherder data output.

https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=1fd0483fa7772c7e5e0ec48d67c22c796b8095e8&selectedJob=282979767

In the Job Details, 'browsertime-results.tgz' tarball you'll now find an 'application.json' that contains:

{"application": {"version": "73.0a1", "name": "firefox"}}

Blocks: 1606131

Try push with both again, and updates including adding the application name to perfherder 'extraOptions' (required until Bug 1593198 is fixed):

https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=359b685e90a49a6eb0deb6e593771a680069190b

https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=87ce2dc54be8a63beea13b5ad819ff66dcc0995e

Ok ready for review, on the try run you'll see that the tp6-c-1-vismet task PERFHERDER_DATA output now includes the 'application' key (and the application name is also in extraOptions):

PERFHERDER_DATA: {"suites": [{"name": "amazon-cold", "extraOptions": ["firefox"],...
"type": "vismet", "framework": {"name": "browsertime"}, "application": {"version": "73.0a1", "name": "firefox"}}

Pushed by rwood@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/11ea65b2b31c
Add application info json file to browsertime results for vismet jobs r=perftest-reviewers,stephendonner
Pushed by rwood@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6865e8d68595
Add application name and version in vismet perfherder results r=perftest-reviewers,stephendonner
Keywords: leave-open
Status: ASSIGNED → RESOLVED
Closed: 11 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla73
Regressions: 1606826

== Change summary for alert #24608 (as of Fri, 03 Jan 2020 19:40:56 GMT) ==

Improvements:

25% raptor-webaudio-firefox macosx1014-64-shippable opt 196.00 -> 146.83
21% raptor-webaudio-firefox linux64-shippable opt 150.83 -> 119.25
21% raptor-webaudio-firefox linux64-shippable-qr opt 157.50 -> 124.67
10% raptor-webaudio-firefox windows10-64-shippable opt 160.38 -> 145.00
9% raptor-webaudio-firefox windows10-64-shippable-qr opt 159.42 -> 144.42
8% raptor-webaudio-firefox windows7-32-shippable opt 197.04 -> 180.58

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=24608

(In reply to Alexandru Ionescu :alexandrui (needinfo me) from comment #12)

== Change summary for alert #24608 (as of Fri, 03 Jan 2020 19:40:56 GMT) ==

Improvements:

25% raptor-webaudio-firefox macosx1014-64-shippable opt 196.00 -> 146.83
21% raptor-webaudio-firefox linux64-shippable opt 150.83 -> 119.25
21% raptor-webaudio-firefox linux64-shippable-qr opt 157.50 -> 124.67
10% raptor-webaudio-firefox windows10-64-shippable opt 160.38 -> 145.00
9% raptor-webaudio-firefox windows10-64-shippable-qr opt 159.42 -> 144.42
8% raptor-webaudio-firefox windows7-32-shippable opt 197.04 -> 180.58

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=24608

This must be an error; the patches landed in this bug only added additional metadata to the browsertime results and the visual metrics tasks; the improvements listed above are on raptor-webaudio which isn't run on browsertime and is unrelated. Thanks!

:alexandrui this improvement appears to be associated with the wrong bug and is appearing in a query for top improvements, could you take another look at this to see if there is another cause?

Flags: needinfo?(aionescu)

I can't make any investigations as the jobs from this period are no longer available in CI.

Flags: needinfo?(aionescu)

(In reply to Alexandru Ionescu (needinfo me) [:alexandrui] from comment #15)

I can't make any investigations as the jobs from this period are no longer available in CI.

Given this is unlikely to be the cause of the improvement, even without the original jobs we should be able to speculate on a more likely cause. From looking at the pushlog, I imagine bug 1596182 is much more likely due to it being a webaudio optimisation.

You need to log in before you can comment on or make changes to this bug.