1601667 - Measure raptor performance on Pixel 2 in CI with WebRender

Status: RESOLVED INVALID

(Core :: Graphics: WebRender, task, P3)

Description

[Andrew Osmond [:aosmond] (he/him)]

Currently the raptor tests run on the Pixel 2 on treeherder, but we force disable WebRender anywhere that doesn't specifically opt into it on CI, so it tests non-WebRender performance. We don't want to switch the perf tests to just use WR, because the Pixel 2 is the only device we have in CI for measuring performance (and so is a useful proxy in general for Android performance), but it would be useful to compare the WR and non-WR results against each other and possibly create a new job that only runs on request.

Comment 1

[Andrew Osmond [:aosmond] (he/him)]

try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=856ee2adc1037fe3f9a7b9e3b1d166465f65cf74

Comment 2

[Andrew Osmond [:aosmond] (he/him)]

perfherder: https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-central&newProject=try&newRevision=856ee2adc1037fe3f9a7b9e3b1d166465f65cf74&framework=10&selectedTimeRange=604800

Comment 3

[Andrew Osmond [:aosmond] (he/him)]

The results are noisy as always, but I think the key take away here is that our cold startup has likely "regressed" by quite a bit:

https://wiki.mozilla.org/TestEngineering/Performance/Raptor#Cold_Page-Load

It appears the Android app data is deleted, so I guess any cached compiled shaders need to be rebuilt. I don't think any other results are very notable at this time. There are no motionmark benchmarks able to be run on Android in CI, so we can't really see WR's strengths present in the available test suite.

Comment 4

[Kris Taeleman (:ktaeleman)]

Hey Andrew, what is the status of this task?

Comment 5

[Andrew Osmond [:aosmond] (he/him)]

I ran all the tests that I could.