1254121 - talos tsvgx in e10s needs mozafterpaint in some of the sub-tests

Status: RESOLVED WONTFIX

(Testing :: Talos, defect)

Description

[Avi Halachmi (:avih)]

tsvgx has two types of sub-tests: plain page load tests, and animation tests.

When comparing the results between e10s and non e10s, the expectation is that they perform similarly (bug 1250620 comment 18).

However, while the animation tests show similar results, the plain page-load tests do not (e10s is "better"). It appears that e10s needs mozafterpaint for page-load tests more than non e10s in order to produce meaningful results (probably similar to bug 1250717).

And indeed enabling mozafterpaint for tsvgx makes the plain page-load sub-tests have similar results - but now the animation tests show different results.

The mozafterpaint "mode" in the pageloader was definitely not designed for animation tests.

I've looked a bit into the pageloader and it seems non trivial to support mozafterpaint only for a subset of the sub-tests - for at least two reasons: configuration would be too complex and the code is such that this change would be risky regardless.

More importantly though, the essence of this test is measuring svg performance, and using page-load is a tool - not the goal (for the actual loading performance we have tp5o).

I suggest to keep this test without mozafterpaint, and for now disable the plain page-load sub-tests.

I think that regardless the animation tests are the better subset of this test, but as a followup we can re-add the test cases of the plain svg images to the test as a non page-load tests.

Joel, thoughts?

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

we could hack up pageloader without much trouble to support this.  We could also split this into svg_animation and svg_pageload tests (i.e. a new test).

I am fine with disabling the pageload tests as well, we should outline what makes sense to test with which parameters, then figure out how to make that happen.

Comment 2

[Avi Halachmi (:avih)]

My main point was:

> More importantly though, the essence of this test is measuring svg performance, and
> using page-load is a tool - not the goal (for the actual loading performance we have tp5o).

We can test svg better without page-loading. But it would take time to make that happen.

For now, let's disable the page-load subtests of this test, and keep it without mozafterpaint.

I'll post a patch hopefully today.