1665010 - [meta] Deploy 2020 mobile reference hardware in CI

Status: RESOLVED FIXED

(Infrastructure & Operations :: RelOps: General, task)

Description

[Dave Hunt [:davehunt] [he/him] ⌚BST]

The following 2019 reference hardware is currently available in CI:

• Moto G5
• Pixel 2

This bug tracks deploying the following 2020 reference mobile hardware into CI:

• Moto G5 (done)
• Samsung Galaxy S7
• Samsung Galaxy S10
• Pixel 4 XL

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

Can we outline what each device will be used for and the number of devices needed?

Comment 2

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #1)

Can we outline what each device will be used for and the number of devices needed?

I can only speak for performance testing, but ideally we'd have enough of each device type to run tier 1/2 performance tests. I understand that we're unable to increase the total number of devices, so we'd need to understand what capacity we currently have, and perhaps recalibrate our test tiers to allow for more diversity in the device hardware. I've only opened bug 1665012 so far, to start with a small number of Samsung Galaxy S7 devices and go from there.

Comment 3

[Bob Clary [:bc] (inactive)]

Remember for Raptor and unrooted devices we lose the ability to disable/enable charging, to do the performance turning and perhaps some other items.

Comment 4

[Andrew Erickson [:aerickson]]

Where are we at with this?

I need a concrete plan so I can tell Bitbar what to switch out.

Comment 5

[Greg Mierzwinski [:sparky]]

(In reply to Bob Clary [:bc] from comment #3)

Remember for Raptor and unrooted devices we lose the ability to disable/enable charging, to do the performance turning and perhaps some other items.

:bc, is there a bug or doc that outlines why these devices can't be rooted?

Comment 6

[Bob Clary [:bc] (inactive)]

No, no bug or document.

Not all devices are rootable. Some don't support unlocking the bootloader and can't be flashed with updated boot images etc. Samsung devices are notorious for being difficult to root and don't support fastboot if I understand correctly. They have their own techniques. It is not impossible to root Samsung or other devices. It just isn't guaranteed a rooting technique is or will be available in a timely fashion for whatever device you are wanting to use.

Comment 7

[Greg Mierzwinski [:sparky]]

Oh interesting, thanks!

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

The current plan is to deploy a pool of Samsung S7 in bug 1665012. There is no current plan for other pools, but we need to circle back on this topic in December.

A few options to consider:

1. g5 is being deprecated by many carriers, we should determine how long we keep it around
2. we might want a few small pools that are Fenix specific.
3. we might expand our S7 pool if we find great value from it (unique regressions we don't see on G5/P2)

So for the next 6-8 weeks it is bug 1665012, we can revisit and figure out next steps.

Comment 9

[Andrew Erickson [:aerickson]]

Do we have any future changes planned? Should we close this issue?

Comment 10

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Andrew Erickson [:aerickson] from comment #9)

Do we have any future changes planned? Should we close this issue?

The original plan was to have Samsung Galaxy S10 and Pixel 4 XL, however we're actively discussing Android hardware refreshes. We can close this bug and open a new one pending the outcome of the discussions.

Comment 11

[Joel Maher ( :jmaher ) (UTC -8)]

:davehunt, please ensure that someone from releng (myself or :mtabara, also I assume :aerickson as well) is on the new bug. I don't need to be in the decision process, but for rollout there are unittests we run on these phones and I would prefer to upgrade in 2022 to reduce the risk of disaster- most likely we will just run whatever phones are decided on for perf and use the same models for unittests.