1555479 - Run wrench reftests on a device in CI

Status: RESOLVED FIXED

(Core :: Graphics: WebRender, defect, P3)

Description

[Kartikaya Gupta (email:[e-mail])]

Bug 1549776 added wrench reftests in CI, running on the Android emulator. We've since found some issues with the emulator's behaviour, and it would be good to also run the reftests on actual devices.

Comment 1

[Kartikaya Gupta (email:[e-mail])]

I have patches to do this but because of the Pixel 2 backlog it's hard to test them.

Comment 2

[Kartikaya Gupta (email:[e-mail])]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=fd2873ca34b24925eb11843f2c14cc31c278aef5

Comment 3

[Kartikaya Gupta (email:[e-mail])]

Attachment: Bug 1555479 - Allow wrench to know when it is running in the emulator. r?gw

Comment 4

[Kartikaya Gupta (email:[e-mail])]

Attachment: Bug 1555479 - Update android skip annotations for Pixel2 results. r?gw

Depends on D33404

Comment 5

[Kartikaya Gupta (email:[e-mail])]

Attachment: Bug 1555479 - Rename android_emulator_wrench to android_wrench.py. r?gbrown

This is in preparation for having the same script be used for emulator
and device runs. No functional change in this patch; it just renames
the file and class.

Depends on D33405

Comment 6

[Kartikaya Gupta (email:[e-mail])]

Attachment: Bug 1555479 - Update wrench mozharness script to support running on a device as well. r?gbrown

The presence or absence of the DEVICE_SERIAL environment variable
is sufficient to control this.

Depends on D33406

Comment 7

[Kartikaya Gupta (email:[e-mail])]

Attachment: Bug 1555479 - Update run-task to support bitbar. r?tomprince

Currently all jobs run on bitbar go through mozharness-test. However for
WebRender standalone testing (i.e. built without any Gecko stuff) we want
to run the WebRender wrench test harness on bitbar using run-task. This
adds the necessary support to run-task.

Depends on D33407

Comment 8

[Kartikaya Gupta (email:[e-mail])]

Attachment: Bug 1555479 - Run wrench reftests on a Pixel2 in bitbar. r?jrmuizel

Note that on the bitbar workers we don't get to use a gecko checkout, so
we need to fetch the mozharness.zip and wrench reftests as artifacts from
other jobs.

Depends on D33408

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/44b33fe66dc5
Allow wrench to know when it is running in the emulator. r=gw
https://hg.mozilla.org/integration/autoland/rev/a439229efd64
Update android skip annotations for Pixel2 results. r=gw
https://hg.mozilla.org/integration/autoland/rev/2a44a6ac6bd1
Rename android_emulator_wrench to android_wrench.py. r=gbrown
https://hg.mozilla.org/integration/autoland/rev/48306fd29453
Update wrench mozharness script to support running on a device as well. r=gbrown
https://hg.mozilla.org/integration/autoland/rev/e20f91f0ea08
Update run-task to support bitbar. r=tomprince
https://hg.mozilla.org/integration/autoland/rev/fd26384a1def
Run wrench reftests on a Pixel2 in bitbar. r=jrmuizel

Comment 10

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/44b33fe66dc5
https://hg.mozilla.org/mozilla-central/rev/a439229efd64
https://hg.mozilla.org/mozilla-central/rev/2a44a6ac6bd1
https://hg.mozilla.org/mozilla-central/rev/48306fd29453
https://hg.mozilla.org/mozilla-central/rev/e20f91f0ea08
https://hg.mozilla.org/mozilla-central/rev/fd26384a1def