re-record web-de to try and resolve Intermittent timeout errors
Categories
(Testing :: Raptor, defect, P2)
Tracking
(firefox108 fixed)
| Tracking | Status | |
|---|---|---|
| firefox108 | --- | fixed |
People
(Reporter: kshampur, Assigned: kshampur)
References
Details
(Keywords: perf-alert)
Attachments
(2 files)
image.png
web-de is frequently timing out during browsertime tests, and increasing the output timeout does not seem to help. Maybe re-recording the site may help address this
|
Assignee | |
Comment 1•2 years ago
•
|
||
in this Try run the screenshot shows a subscription pop up as attached
also the recording .zip file differs in naming i.e. it has gve rather than fenix like the manifests here https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tooltool-manifests/playback/mitm6-android-fenix-web-de.manifest
:sparky
- is gve OK or should it be fenix? (if so, how to get these recordings on fenix?)
- since we now by default record on mitm7, should I lower it to mitm6 so that web-de would be consistent with the rest?
- any thoughts on bypassing the subscription popup?
on the other hand (& being hopeful) I see the orange factor trending down a bit past couple of days in Bug 1784176
edit: updated Try link with web-de re-recording, previous link was invalid for some reason
|
||
Comment 2•2 years ago
|
||
:kshampur, gve is fine for the recordings. If the mitm7 re-recording fixes the issue then let's go with that, otherwise try mitm6.
For the subscription popup, lets file a follow-up bug and look into it after fixing this intermittent.
|
|
Assignee | |
Comment 3•2 years ago
•
|
||
I was curious - quickly put together, but here is a Try where web-de is (seemingly) consistently passing with a recent mitm7 p2 gve recording.
|
|
Assignee | |
Updated•2 years ago
|
|
|
Assignee | |
Comment 4•2 years ago
|
||
There is a high frequency of intermittent failures with web-de raptor-browsertime tests on the android devices.
Re-recording web-de seems to resolve this issue.
|
||
Updated•2 years ago
|
Pushed by kshampur@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/8120ef32f9de re-record web-de to try and resolve Intermittent timeout errors. r=perftest-reviewers,sparky
|
||
Comment 6•2 years ago
|
||
| bugherder | ||
|
||
Comment 7•1 year ago
|
||
(In reply to Pulsebot from comment #5)
Pushed by kshampur@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/8120ef32f9de
re-record web-de to try and resolve Intermittent timeout errors.
r=perftest-reviewers,sparky
== Change summary for alert #36040 (as of Fri, 11 Nov 2022 06:41:26 GMT) ==
Regressions:
| Ratio | Test | Platform | Options | Absolute values (old vs new) |
|---|---|---|---|---|
| 126% | web-de fcp | android-hw-a51-11-0-aarch64-shippable-qr | cold webrender | 316.29 -> 713.96 |
| 62% | web-de FirstVisualChange | android-hw-a51-11-0-aarch64-shippable-qr | cold webrender | 501.00 -> 810.08 |
| 24% | web-de SpeedIndex | android-hw-a51-11-0-aarch64-shippable-qr | cold webrender | 713.75 -> 888.08 |
Improvements:
| Ratio | Test | Platform | Options | Absolute values (old vs new) |
|---|---|---|---|---|
| 86% | web-de loadtime | android-hw-a51-11-0-aarch64-shippable-qr | warm webrender | 1,762.54 -> 239.42 |
| 67% | web-de PerceptualSpeedIndex | android-hw-a51-11-0-aarch64-shippable-qr | warm webrender | 797.08 -> 266.33 |
| 60% | web-de loadtime | android-hw-a51-11-0-aarch64-shippable-qr | cold webrender | 2,900.21 -> 1,162.83 |
| 59% | web-de fcp | android-hw-a51-11-0-aarch64-shippable-qr | warm webrender | 480.50 -> 196.75 |
| 59% | web-de LastVisualChange | android-hw-a51-11-0-aarch64-shippable-qr | warm webrender | 1,574.50 -> 647.08 |
| ... | ... | ... | ... | ... |
| 23% | web-de ContentfulSpeedIndex | android-hw-a51-11-0-aarch64-shippable-qr | cold webrender | 1,078.42 -> 827.50 |
For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=36040
|
|
||
Updated•1 year ago
|
Description
•