1079675 - Measure new tab page load times with talos

Status: RESOLVED DUPLICATE of bug 1078254

(Firefox :: New Tab Page, defect)

Description

[Erik Vold [:erikvold] (please needinfo? me)]

Not sure if this already exists, please point me to this if it does.

Comment 1

[Tim Taubert [:ttaubert] (inactive)]

At the newtab perf work week last week avih worked on a pageload subtest for TART.

Comment 3

[Avi Halachmi (:avih)]

(In reply to Erik Vold [:erikvold] (work week -> pto) from comment #0)
> Not sure if this already exists, please point me to this if it does.

It's implicitly measured already using TART, and expresses mostly at the .all (overall average frame interval) and the .error (overall delay to completion) values of the TART tests which use about:newtab - https://wiki.mozilla.org/Buildbot/Talos/Tests#TART.2FCART

TART measures tab animation in various cases, and 2 of these cases are while opening the about:newtab page - one test with newtab preload which is what the users experience, and one test without preload which reflects the "pure" load time of this page.

However, since TART focuses on the animation itself (and stops measuring once the animation completes), if the newtab page takes longer to load than the tab animation to complete, then the TART results don't fully reflect the newtab load times. AFAIK, however, this doesn't happen at the current talos test machines, but can happen when running TART locally on low end hardware.

All that being said, we have a real issue right now with browsing this data (i.e. newtab related TART sub results) because graphserver averages all the sub-results into a single number, DataZilla has issues but gets no resources, and Tree Herder's talos performance data browsing is still some time away from now.

So bug 1078254 is about adding tests which will explicitly measure the load times of about:newtab, regardless of their effect on tab animation smoothness, and possibly about adding these test to talos as a new test (i.e. not TART) such that even their average would be useful.