1187081 - Make sure Talos tests are appropriate for measuring the impacts of APZ & Displayport

Status: RESOLVED WORKSFORME

(Testing :: Talos, defect)

Description

[Benoit Girard (:BenWa)]

I'd like for us to be able to track the effects of APZ, which means having a displayport, properly with Talos.

I had a quick look at TART and TResize and they don't have any overflow. If the page doesn't have any overflow it means that the viewport == displayport which gives us an un-interesting trivial case. We need to make sure we are testing with pages that have overflow (that can scroll) if we want to measure the impact of having displayport.

See also bug 1186662 which is a strategy to mitigate the expected regressions from having a displayport. This would block bug 1186662 because we'd like to have measurements for this before we tackle bug 1186662.

Comment 1

[Avi Halachmi (:avih)]

Well, the newtab page never has overflow by design - it just draws less tiles for smaller windows (at least up to bugs), and besides, TART's goal is to measure basic tab animation and not newtab rendering performance (even if newtab rendering perf might affect some of the tests implicitly).

As for tresize, while currently it also never reflows since the content is mostly empty, its goal is (vaguely, implicitly maybe) to measure chrome window resize perf, and I'd _think_ we should keep it that way.

OTOH, we do have two tests which do scrolling - tscroll and tp5o-scroll (each over several different contents) which obviously have overflow, and also more naturally fit to reflect APZ effects on perf. Shouldn't they cover APZ effect without touching anything?

Currently both scroll tests scroll using requestAnimationFrame and I'm not sure if it does trigger APZ or not, but if it doesn't, we could try adding async-scroll triggers to these tests via some API, or maybe CSSOM?

Comment 2

[Benoit Girard (:BenWa)]

Point taken. TART isn't the right test. Actually it looks like 'tps' already measures tab switch performance of the tp5 pageset which would include coverage of our displayport performance.

If the goal of tresize is just to measure how costly the chrome front-end is the resize then it might be fine.

We would however have a talos blind spot of how well we resize the browser when we have APZ enabled (and likely e10s enabled) on a web page with complex content and a displayport. Something like 'tpresize' might be a good test.

Measuring sync scrolling will be valuable so tscroll and tp5o-scroll are likely good as-is.

We would still need to measure how well APZ performs with async scrolling which would be another blind spot.

In summary here's what I think we would be missing:
- Check that tps is appropriate for APZ
- (Sync) resize performance of webcontent. Maybe consider 'tpresize'?
- Add async scrolling test. Perhaps something similar to Android checkerboard tests.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

:benwa, is there work to do here?  IIRC APZ is live.  I see the request for a new tscroll-apz, is that the remaining work item?

Comment 4

[Benoit Girard (:BenWa)]

No, we also have TPS so this is probably fine. It could be better but that's always true. I'm fine with closing.