Closed Bug 1592918 Opened 3 months ago Closed 2 months ago

2.39 - 8.9% raptor-tp6-reddit-firefox-cold fcp / raptor-tp6-yandex-firefox-cold / raptor-tp6-yandex-firefox-cold fcp (linux64-shippable-qr, windows10-64-shippable-qr) regression on push f526a659b51ac9e46cb9c8abc1f685e5f9eaf372 (Tue October 29 2019)

Categories

(Firefox :: Address Bar, defect, P3)

defect
Points:
3

Tracking

()

RESOLVED FIXED
Firefox 73
Tracking Status
firefox-esr68 --- unaffected
firefox71 --- unaffected
firefox72 --- wontfix
firefox73 --- fixed

People

(Reporter: Bebe, Unassigned)

References

(Blocks 2 open bugs, Regression)

Details

(Keywords: perf, perf-alert, regression)

Raptor has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=f526a659b51ac9e46cb9c8abc1f685e5f9eaf372

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

9% raptor-tp6-yandex-firefox-cold fcp linux64-shippable-qr opt 310.75 -> 338.42
7% raptor-tp6-yandex-firefox-cold fcp windows10-64-shippable-qr opt 278.88 -> 298.83
7% raptor-tp6-yandex-firefox-cold fcp windows10-64-shippable-qr opt 277.92 -> 297.00
5% raptor-tp6-reddit-firefox-cold fcp windows10-64-shippable-qr opt 287.50 -> 303.08
2% raptor-tp6-yandex-firefox-cold linux64-shippable-qr opt 654.61 -> 670.26

Improvements:

12% raptor-tp6-slides-firefox-cold loadtime windows7-32-shippable opt 2,285.17 -> 2,007.75
3% raptor-tp6-slides-firefox-cold windows7-32-shippable opt 1,256.47 -> 1,215.59

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=23607

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a Treeherder page showing the Raptor jobs in a pushlog format.

To learn more about the regressing test(s) or reproducing them, please see: https://wiki.mozilla.org/TestEngineering/Performance/Raptor

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/TestEngineering/Performance/Talos/RegressionBugsHandling

Blocks: 1592626
Component: Performance → Address Bar
Flags: needinfo?(mak)
Product: Testing → Firefox
Regressed by: 1592126
Version: Version 3 → unspecified

Hm, I doubt that changing the urlbar design can have a real impact on page loads. If it does, it looks like a bug in the Raptor measurement.

Since this was a nightly only enabled feature, and we're just moving back to the Release/Beta status quo, I doubt this would block any release. It would still be interesting to understand how/why a Firefox UI change affects these Raptor Tp6 tests.

What is fcp?

Flags: needinfo?(mak) → needinfo?(rwood)

(In reply to Marco Bonardo [:mak] from comment #1)

What is fcp?

Hi, fcp = 'first-contentful-paint', one of the page-load measurements taken by Raptor.

https://developer.mozilla.org/en-US/docs/Glossary/First_contentful_paint

https://wiki.mozilla.org/TestEngineering/Performance/Raptor#Page-Load_Tests

Flags: needinfo?(rwood)

How does the load happen? How can the urlbar design influence it?

Flags: needinfo?(rwood)

(In reply to Marco Bonardo [:mak] from comment #3)

How does the load happen? How can the urlbar design influence it?

Raptor uses a web extension to load test pages (which are mitmproxy page recordings played back through a local proxy). I don't know anything about the urlbar design and don't know if it's possible that is effecting the first-contentful-paint value or not. Maybe someone on the performance team might have some input or ideas on how that can be checked? Perhaps via gecko profiling?

Flags: needinfo?(rwood)
Flags: needinfo?(rjesup)
Flags: needinfo?(acreskey)

URLbar design could in theory affect pageload if the target of the load is injected as keystrokes. Is raptor loading pages via marionette? Even if it isn't, the URLbar is updated at the start of pageload with the new URL. Depending on if this delays some other aspects this could affect pageload. Even extra paints for the bar could affect it. Not sure if these things are happening, but they could

Flags: needinfo?(rjesup)
Blocks: megabar
Priority: -- → P3

We should check what happens when we'll re-enable the megabar in Nightly, comparing the 2 moves may give us a better idea of what exactly is shifting.

Points: --- → 3

I'm collecting profiles from try with screencaptures enabled to see if we can can spot a difference.

Sorry for the delay.
Looking into this but when I push a job to try from these older revisions I see no jobs (nor can I add them).
e.g.
https://treeherder.mozilla.org/#/jobs?repo=try&revision=aeb305e65a1de9fac4874cbb6f6aed20fffa9e1e

But regardless, I don't think there's much we can do except as :mak said, see see what happens when the megabar is re-enabled.

Flags: needinfo?(acreskey)

The regressions reverted now that we re-enabled the new design in bug 1599784.

https://treeherder.mozilla.org/perf.html#/alerts?id=24254

Yandex and Reddit seem the ones most affected in both reports.
This doesn't sound like an urlbar bug anymore, the regression is actually an improvement since the new design is our target, even if it's likely there's a problem in how things are measured if the ui shouldn't influence measurements too much.

Note, you may compare profiles with browser.urlbar.update1 set to true and false, and compare those.
How do we want to proceed here?

Flags: needinfo?(acreskey)

Thank you Marco.
I personally don't see anything actionable here.

Flags: needinfo?(acreskey)

Fine, anyway the "regression" reverted.

Status: NEW → RESOLVED
Closed: 2 months ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 73
You need to log in before you can comment on or make changes to this bug.