CNN live site test intermittently fails often
Categories
(Testing :: Raptor, defect, P1)
Tracking
(firefox95 fixed)
Tracking | Status | |
---|---|---|
firefox95 | --- | fixed |
People
(Reporter: sparky, Assigned: sparky)
References
(Blocks 1 open bug)
Details
Attachments
(1 file)
The CNN live site test is currently failing very often (see bug 1680214). The purpose of this site was to test (and sheriff) a page with a high number of processes. This test has been failing for some time though and at one point, we tried to switch to another website (www.foxnews.com) but it also failed intermittently quite often: https://treeherder.mozilla.org/jobs?repo=try&revision=2467b904159b57b72feb7fce0ba8488ea9d9608e
From some discussions I had with jesup a while ago, it was suggested that this is happening because there are "are 3rd-party iframes/ads that are taking forever to load": https://matrix.to/#/!ECdrESRnYfSJXBzaMX:mozilla.org/$jZtuf9nAqnlIwhHjidirsD-wnGHWKjqrrMXwgG-dK6E?via=mozilla.org&via=matrix.org&via=nordgedanken.dev
We'll need to figure out how to solve this issue so we can start sheriffing those tests again.
Assignee | ||
Updated•3 years ago
|
Assignee | ||
Comment 1•3 years ago
|
||
Based on this last try run I've made, iframes are definitely part of the problem. I can't read/access all of them in a webpage because of cross-origin issues but for those that I can access I'm seeing that they never/intermittently get into a "complete"
readyState.
Try run that shows this (note how all tests fail after a 2-minute timeout): https://treeherder.mozilla.org/jobs?repo=try&revision=f12f831516788b2a85ce748bdbde4630ec5df67e
I've made some changes to exclude the iframe checks. In this push, I have the default pageCompleteCheck method but it waits for 15s rather than 5s and I also added a window.stop()
call in the hopes that it'll stop some of the extra loading going on at the end of a test: https://treeherder.mozilla.org/jobs?repo=try&revision=764a39105906b6e25932961339d7bc030db3bd1c
Assignee | ||
Comment 2•3 years ago
|
||
Another try run here: https://treeherder.mozilla.org/jobs?repo=try&revision=256b7df69f03515b052a26b7ac5a150551674f64
I've noticed that the video on this page autplays which is not a default behaviour for Firefox so I've attempted to disable it. The prefs weren't getting set when I changed them in the common user.js profile prefs which was because browsertime was overwriting them: https://github.com/gmierz/browsertime/commit/814fa863894e4c2aeb4cbccba928864b268c125f
Assignee | ||
Comment 3•3 years ago
|
||
I've managed to make some progress on this.
Here's a try run with no changes: https://treeherder.mozilla.org/jobs?repo=try&revision=8e59b58767d25f402fbae9fdc8aa3ab4a819ba13
In this try run, I've decreased the number of cycles to 10 and disabled the media autoplay: https://treeherder.mozilla.org/jobs?repo=try&revision=256b7df69f03515b052a26b7ac5a150551674f64
The patch shows a 50-75% decrease in the occurrence of this failure.
Assignee | ||
Comment 4•3 years ago
|
||
This patch reduces the number of browser-cycles that a live-site test (cnn in this case) does from 25 down to 5. Furthermore, we also disable media autoplay for all live site tests. This change required an update to browsertime to prevent default prefs from being set and overriding our prefs.
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Pushed by gmierz2@outlook.com: https://hg.mozilla.org/integration/autoland/rev/e6df1ed94379 Reduce browser-cycles and disable media autoplay in live site tests. r=perftest-reviewers,kimberlythegeek
Comment 8•3 years ago
|
||
Backed out changeset e6df1ed94379 (Bug 1733434) for causing btime and awsy failures.
Backout link: https://hg.mozilla.org/integration/autoland/rev/d1e8ca45b22f6aa52e537cb45c28d2eb79e59794
Push with failures, btime, awsy failures.
Pushed by malexandru@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/8c15d07ec653 Reduce browser-cycles and disable media autoplay in live site tests. r=perftest-reviewers,kimberlythegeek
Comment 10•3 years ago
|
||
Re-landed changeset e6df1ed94379 since the failures were caused by another push, sorry for the inconvenience!
Re-land link: https://hg.mozilla.org/integration/autoland/rev/8c15d07ec653ac53feb382e4771799589915ed58
Comment 11•3 years ago
|
||
bugherder |
Description
•