1545186 - Raptor test definitions and taskcluster configs for cold page-load tests on GeckoView Example App

Status: RESOLVED FIXED

(Testing :: Raptor, enhancement, P1)

Description

[Robert Wood [:rwood]]

Once the Raptor cold page-load test INI's and taskcluster configs land in Bug 1545175 for Fenix, mirror those for the geckoview example app, for try and mozilla-central tier 2.

Comment 1

[Robert Wood [:rwood]]

Once cold page-load on geckoview example app is ready, we will phase these into CI (try/central/trunk) and phase-out the geckoview example app warm page-load tests (move those to central/try only).

Comment 2

[Robert Wood [:rwood]]

Roll this out a few jobs at a time. i.e. for the first 3 tp6m geckoview example cold page-load jobs: try/central/beta/trunk will be tier 1.

For the remainder (tp6m-cold-4 to 14) for geckoview example they will be: try/central only for now due to device availability. That will allow reporting to the mobile health dashboard. Eventually when we have more devices then move those also to try/central/beta/trunk.

As phase them out, Geckoview example warm page-load on try/central will be tier 2.

Comment 3

[Robert Wood [:rwood]]

Before landing these (mirroring the Fenix cold page-load tests/configs landed in Bug 1545175) make sure the failures being seen on Fenix don't happen on the geckoview example app also: Bug 1547491 and Bug 1547492.

Comment 4

[Robert Wood [:rwood]]

(In reply to Robert Wood [:rwood] from comment #3)

Before landing these (mirroring the Fenix cold page-load tests/configs landed in Bug 1545175) make sure the failures being seen on Fenix don't happen on the geckoview example app also: Bug 1547491 and Bug 1547492.

Those failures only happen when using live sites, not the mitmproxy recordings, so will go ahead with the geckoview example cold page-load tests/configs.

Comment 5

[Robert Wood [:rwood]]

Attachment: Bug 1545186 - Raptor test definitions and taskcluster configs for cold page-load tests on GeckoView Example App; #perftest

Comment 6

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=317c26fac7f713ff3b0d44ee035ed5702c66bc30

Comment 7

[Robert Wood [:rwood]]

Updated patch (fixed errors in one INI, increased taskcluster job limit since some were timing out, and re-based). Pushed to try:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=9a0bdc2e70430c72f0fe5a21cd701c87461373eb

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d0383e7940c9
Raptor test definitions and taskcluster configs for cold page-load tests on GeckoView Example App; #perftest r=davehunt

Comment 9

[Alexandru Michis [:malexandru]]

https://hg.mozilla.org/mozilla-central/rev/d0383e7940c9

Comment 10

[Dave Hunt [:davehunt] [he/him] ⌚BST]

:rwood could you file a follow-up for enabling the remaining cold page load tests on trunk?

Comment 11

[Robert Wood [:rwood]]

(In reply to Dave Hunt [:davehunt] [he/him] ⌚️UTC from comment #10)

:rwood could you file a follow-up for enabling the remaining cold page load tests on trunk?

Yes, filed Bug 1550553.