big page load regression in Nightly 48

RESOLVED FIXED

Status

()

RESOLVED FIXED
2 years ago
2 years ago

People

(Reporter: blassey, Unassigned)

Tracking

({regression})

Trunk
regression
Points:
---

Firefox Tracking Flags

(firefox48+ wontfix, firefox49+ disabled, firefox50+ fixed, firefox51 fixed)

Details

[Tracking Requested - why for this release]: regression in 48

looking at https://mzl.la/2996XIQ it looks like we had a significant regression in page load times at the end of he 48 (April 24) nightly cycle. It also looks like we fixed most of the regression on June 17.

We should figure out what regressed and what fixed it and get that fix uplifted to 48 (assuming it isn't already).
[Tracking Requested - why for this release]:

I don't see it in the beta chart:

https://mzl.la/29hwmUV

But aurora 49 seems to show it:

https://mzl.la/29bCrPL

Maybe something thats just #ifndef RELEASE_BUILD?
tracking-firefox49: --- → ?
Keywords: regression
Sorry, I didn't use permalinks.

Beta:

https://mzl.la/296gZco

Aurora:

https://mzl.la/296hrrf
Hi Brad,
Is there any update or is there anyone can help on this?
Flags: needinfo?(blassey.bugs)
Track this as there is a possible regression in 48.
tracking-firefox48: ? → +
tracking-firefox49: ? → +
tracking-firefox50: --- → +
No I don't have an update. Some one should look at what landed in the regression range and the fix range to figure out what caused this.
Flags: needinfo?(blassey.bugs)
From comment 1 looks like this isn't a problem currently in 48 beta.
status-firefox48: --- → fix-optional
status-firefox49: --- → affected
Andrei, can someone from your team help with the regression range checking here?
Flags: needinfo?(andrei.vaida)
Keywords: qawanted
We tried to manually check for a regression based on the chart from comment 0 (https://mzl.la/2996XIQ) using app.telemetry Page Speed Monitor add-on (https://addons.mozilla.org/ro/firefox/addon/apptelemetry/) but the results were inconcusive. The difference between a good build and a bad one (ex: good=2016-04-18 and bad=2016-05-08) were either very large in favour of the bad build, good build or equal while loading different websites with high content or not.

Based on the fact that manually we could not find a clear regression, we could identify two ranges from the nightly chart:

Regression where the issue started 2016-04-22 - 2016-04-23
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=0891f0fa044cba28024849803e170ed7700e01e0&tochange=37f04460ddb76d6ef4e7c32a8a6b2fbc44cb8776

Regression where the issue was fixed 2016-06-16 - 2016-06-17
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=b9f4f38063951cd5a8b249911aea61869f40fd1f&tochange=5f95858f8ddf21ea2271a12810332efd09eff138
Flags: needinfo?(andrei.vaida)
Hi Joel,
I'm not sure if you can help for this perf regression. Might need your input here.
Flags: needinfo?(jmaher)
I don't have any big page specific things, there is svg_opacity which is a larger page and usually hits network issues:
https://treeherder.mozilla.org/perf.html#/graphs?timerange=31536000&series=%5Bmozilla-inbound,b7df5ad27d6ab0f1f5651e5ceab20471cd34dd2d,1,1%5D

there is no pattern there about regressing and getting fixed.


a possible pattern would be in tp5o:
https://treeherder.mozilla.org/perf.html#/graphs?timerange=31536000&series=%5Bmozilla-inbound,7d6d0a62f7a2126944a5e7f463ac524c6f771fd3,1,1%5D&series=%5Bmozilla-inbound,e9786d3bc11a5fff1fa013567494164638d623e9,1,1%5D&zoom=1450222787158.0518,1469410642000,186.79058542925185,283.44486052218866&selected=%5Bmozilla-inbound,e9786d3bc11a5fff1fa013567494164638d623e9,33492,30261054,1%5D

a fix on june 17:
https://treeherder.mozilla.org/perf.html#/alerts?id=1555


there are a series of issues which caused regressions, although I don't see anything specific that is a main cause here.  Feel free to poke around at the regressions and look at the bugs associated with them on the graphs.

I am unclear from this bug title/comments if this is on all platforms, e10s, non e10s, which page, local page vs remote page, etc.
Flags: needinfo?(jmaher)
status-firefox48: fix-optional → wontfix
Hi Andrei,
Since Bogdan is on PTO, can you help to find someone in your team to check if this issue is still in 49 based on comment #8 and comment #10?
Flags: needinfo?(andrei.vaida)
Hi Gerry,
I tried testing this issue manually but there are too many factors that come into play here. I installed the same add-on as Bogdan in comment #8 [ https://addons.mozilla.org/ro/firefox/addon/apptelemetry/ ] and tested a few sites for page load times using the Nightly 48.0a1 (id: 20160417030601) build as a baseline because it is not in the affected builds range. I've tried out a build of Nightly (id: 20160424030601) that was in the affected range and one that was in the fixed range (id: 20160618030212), but the results are inconclusive. I have also made a spreadsheet with the results and you can find it here: https://goo.gl/kh2hyf

I guess the only way to see if the issue is still in 49 is to check the graphs.
Please let me know if I can help with anything else.
Flags: needinfo?(andrei.vaida)
Hi Nicholas,
Based on Joel's performance results in comment #10, it seems that your fix enhance the performance. Not sure if your fix is related to this bug. Can you help to take a look at it?
Flags: needinfo?(hurley)
(In reply to Gerry Chang [:gchang] from comment #13)
> Hi Nicholas,
> Based on Joel's performance results in comment #10, it seems that your fix
> enhance the performance. Not sure if your fix is related to this bug. Can
> you help to take a look at it?

It may well be related. However, the regressing feature (bug 1016628, which landed in 48) is currently enabled only if NIGHTLY_BUILD is defined (there is an intermittent leak I'm still trying to track down before letting the feature ride the trains). Additionally, the performance fix (bug 1273882) landed in 50. So if there's still a regression, its cause is somewhere else.
Flags: needinfo?(hurley)
Keywords: qawanted
Is there still a regression in 51?  Since this is still nightly only, I'm marking 49/50 as disabled.
status-firefox49: affected → disabled
status-firefox50: affected → disabled
status-firefox51: --- → affected
Flags: needinfo?(jmaher)
Flags: needinfo?(hurley)
(In reply to Liz Henry (:lizzard) (needinfo? me) from comment #15)
> Is there still a regression in 51?  Since this is still nightly only, I'm
> marking 49/50 as disabled.

The regression caused by my feature was fixed in bug 1273882, so unless there was a separate regression caused by another landing, this should be fixed as of 50 (though also disabled there for other reasons).
Flags: needinfo?(hurley)
I would agree with :hurley.
Flags: needinfo?(jmaher)
Thank you all! It sounds like we can close this bug now since it is disabled in 49 and fixed in 50/51.
Status: NEW → RESOLVED
Last Resolved: 2 years ago
status-firefox50: disabled → fixed
status-firefox51: affected → fixed
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.