upgrade version of browsertime and imagemagick
Categories
(Testing :: Performance, task, P1)
Tracking
(firefox93 fixed)
Tracking | Status | |
---|---|---|
firefox93 | --- | fixed |
People
(Reporter: jmaher, Assigned: jmaher)
References
Details
(Keywords: perf-alert)
Attachments
(1 file)
Assignee | ||
Comment 1•3 years ago
|
||
Updated•3 years ago
|
Assignee | ||
Comment 2•3 years ago
|
||
as a note, the upgrade was to use features for responsiveness testing, then there was a small issue with our benchmarks that do not have paintTiming metrics and that was fixed in:
https://github.com/sitespeedio/browsertime/pull/1627
Pushed by jmaher@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/2c24c285ed0a bump version of browsertime and imagemagick. r=perftest-reviewers,AlexandruIonescu
Comment 4•3 years ago
|
||
bugherder |
Comment 5•3 years ago
•
|
||
Backed out changeset 2c24c285ed0a (Bug 1727780) for breaking browsertime tasks (new yargs version requires node 12 but version 10 is used).
Backout link
Failure Log
Updated•3 years ago
|
Updated•3 years ago
|
Comment 6•3 years ago
|
||
The browsertime tasks still fail for fenix - they might pull artifacts which got built when this change landed and didn't get rebuilt for the backout because previous artifacts could be reused.
Assignee | ||
Comment 7•3 years ago
|
||
I have submitted:
https://github.com/mozilla-mobile/fenix/pull/21064
ideally that will get reviewed and merged in today, then we can see if it fixes stuff. I will be on PTO until Friday September 3rd.
If this doesn't resolve the problem I see 2 choices:
- live with no fenix results for a few more days or possibly another week (if I cannot fix this Friday)
- somebody else fixes what is broken
For the future, we need this documented better- how to run the tests, what binaries are used and how they are updated. After spending ~4 hours on this, I haven't found anybody online today that knows how this works- some great pointers and tips. I had asked almost a week ago in #perftest what is needed for upgrading browsertime and the general consensus was run everything on try- it appears we need a bit more than that- and running 2000+ jobs on try takes a lot of time from our limited pool.
:davehunt - ni you for documentation and finding someone to fix this if it isn't fixed whilst i am on PTO. Ideally if fenix is working we can reland this patch which would be a prerequisite to responsiveness tests.
Pushed by jmaher@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/c3499484e459 bump version of browsertime and imagemagick. r=perftest-reviewers,AlexandruIonescu
Comment 9•3 years ago
|
||
bugherder |
Comment 10•3 years ago
|
||
(In reply to Joel Maher ( :jmaher ) (UTC -0800) from comment #7)
:davehunt - ni you for documentation and finding someone to fix this if it isn't fixed whilst i am on PTO. Ideally if fenix is working we can reland this patch which would be a prerequisite to responsiveness tests.
Thanks :jmaher, I've opened bug 1729272 to improvement our documentation.
Comment 11•3 years ago
|
||
(In reply to Marian-Vasile Laza from comment #5)
Backed out changeset 2c24c285ed0a (Bug 1727780) for breaking browsertime tasks (new yargs version requires node 12 but version 10 is used).
Backout link
Failure Log
== Change summary for alert #31246 (as of Fri, 10 Sep 2021 08:26:37 GMT) ==
Regressions:
Ratio | Test | Platform | Options | Absolute values (old vs new) |
---|---|---|---|---|
7% | google-slides ContentfulSpeedIndex | macosx1014-64-shippable-qr | cold webrender | 1,781.58 -> 1,912.42 |
7% | google-slides ContentfulSpeedIndex | macosx1014-64-shippable-qr | cold webrender | 1,787.21 -> 1,911.33 |
6% | google-slides loadtime | macosx1014-64-shippable-qr | cold webrender | 6,140.72 -> 6,517.12 |
Improvements:
Ratio | Test | Platform | Options | Absolute values (old vs new) |
---|---|---|---|---|
4% | google-slides loadtime | macosx1014-64-shippable-qr | warm webrender | 1,452.17 -> 1,395.21 |
For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=31246
Updated•3 years ago
|
Description
•