1566905 - Have raptor use the fenix release build

Status: RESOLVED INVALID

(Testing :: Raptor, enhancement, P1)

Description

[Robert Wood [:rwood]]

Now that Bug 1552519 has landed and raptor uses a config file on android devices, raptor can now use the regular fenix release build. The fenix 'performancetest' build/binary will be going away soon. Update the raptor taskcluster test configs for fenix to use the 'org.mozilla.fenix' binary (instead of 'org.mozilla.fenix.performancetest').

Comment 1

[Robert Wood [:rwood]]

Attachment: Bug 1566905 - Have raptor use the fenix release build

Comment 2

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=2d8002cc7fcf3ec4032f496fd2e592135f8f3366

Comment 3

[Robert Wood [:rwood]]

(In reply to Robert Wood [:rwood] from comment #2)

https://treeherder.mozilla.org/#/jobs?repo=try&revision=2d8002cc7fcf3ec4032f496fd2e592135f8f3366

"Exception: org.mozilla.fenix is not installed"

Ah, I need to update the indexes too (no longer performance-test) not just the binary names, i.e.:

            index: project.mobile.fenix.v2.performance-test.latest.aarch64
            name: target.apk
        default:
            index: project.mobile.fenix.v2.performance-test.latest.arm
            name: target.apk

Comment 4

[Robert Wood [:rwood]]

With updated indexes:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=49ac260e450de725d9c3ea6e1e8768d5356e1b18

Comment 5

[Robert Wood [:rwood]]

I always seem to have a hard time with the indexes/targets, hopefully I have it right this time:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=8f618b3ba3214be3c0d600174d52ad355c90fce3

Comment 6

[Mitchell Hentges [:mhentges] 🦀]

I'm missing some information: fenix doesn't have a dedicated release build: instead, it has three release builds, one for each channel (nightly/beta/production). Each of these are closely tied to their channels, using "real" crash-reporting keys and other credentials.

The performanceTest build type exists for performance testing - why don't we want to use that for raptor any more?

Comment 7

[Robert Wood [:rwood]]

I believe the performanceTest build type was only created as a temporary solution to work with raptor because we needed to pass in some arguments (like the browser profile). Now we are able to do that with release builds of Fenix using an on-device config.yaml, so there's no need for a special build anymore (so we can reduce code and overhead by removing it). Am I on the right track Christian?

Comment 8

[Mitchell Hentges [:mhentges] 🦀]

Aha, I see (I also found this associated github ticket after creating the above comment ^).
I'll follow-up in that Github ticket to confirm, but I think that the performanceTest build type provided other benefits than custom raptor arguments.
Thanks :)

Comment 9

[Robert Wood [:rwood]]

Christian dropped by - as discussed with Mitchell there is still a need for the performanceTest fenix build type. Abandoning this bug.