1563074 - Lift Android device performance tuning out of `testing/raptor/raptor.py`

Status: RESOLVED FIXED

(Testing :: Raptor, enhancement, P2)

Description

[Nick Alexander :nalexander [he/him]]

Bug 1547135 added functionality to configure specific Android devices for performance tests. That code landed as part of device setup in testing/raptor/raptor.py but there is nothing Raptor-specific (indeed, nothing harness-specific) about this configuration. This ticket tracks lifting it to higher up the stack for re-use.

Exactly where to put it is less clear to me. I think this functionality is not very general -- it's tied to two reference devices and probably depends on details of how they are rooted by Mozilla's operations team -- and therefore probably doesn't want to live in mozdevice. Maybe a sibling of mozdevice?

In any case, this should be invoked by mozharness raptor.py as part of general device configuration appropriate for performance tests.

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

:bc how do you feel about having this in mozdevice?

Comment 2

[Bob Clary [:bc] (inactive)]

I guess it makes sense to lift it out of raptor if it will be used elsewhere. I don't think it should be in adb.py however.

If these are device specific perhaps we can add a tuning directory or something similar and add device specific scripts there? tuning/{motog5,pixel2}.py maybe? Are these OS version specific as well or just device specific?

Comment 3

[Nick Alexander :nalexander [he/him]]

(In reply to Bob Clary [:bc:] from comment #2)

I guess it makes sense to lift it out of raptor if it will be used elsewhere. I don't think it should be in adb.py however.

I think it's fine to not put it in adb.py. But one reason that it's hard to work with Raptor right now is that it's a kitchen sink: everything related to a huge range of performance tests lives in that cupboard. Every thing that we keep in that cupboard, specific to exactly that situation, makes it harder to both move Raptor and to build a Raptor sibling or a Raptor replacement.

If these are device specific perhaps we can add a tuning directory or something similar and add device specific scripts there? tuning/{motog5,pixel2}.py maybe? Are these OS version specific as well or just device specific?

They're device specific. They're not host OS specific. They probably will be target OS version specific if they're not now. I'm happy to extract them into some tuning/ module inside Raptor for now.

Comment 4

[Bob Clary [:bc] (inactive)]

To be clear, I wasn't vetoing putting this in a subdirectory of mozdevice. I just did not want it in the adb.py file. Device/OS versioning will be great.

Comment 5

[Nick Alexander :nalexander [he/him]]

(In reply to Bob Clary [:bc:] from comment #4)

To be clear, I wasn't vetoing putting this in a subdirectory of mozdevice. I just did not want it in the adb.py file. Device/OS versioning will be great.

Ah, thanks for clarifying. I'm happy for this to be accessible in mozdevice but not in adb.py, which I agree is Large :)

Comment 6

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1563074 - Lift Android device performance tuning out of `testing/raptor/raptor.py`. r?bc

Bug 1547135 added functionality to configure specific Android devices
for performance tests. That code landed as part of device setup in
testing/raptor/raptor.py but there is nothing Raptor-specific
(indeed, nothing harness-specific) about this configuration.

This commit moves it into a mozdevice.perf and exposes it as a
single method of mozdevice (since that's the existing pattern).

Comment 7

[Nick Alexander :nalexander [he/him]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=0770316bb7b80e97a68ba395e540e51a63b816ca

Comment 8

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1563074 - To fold: Keep performance tuning in Raptor. r?#perftest-reviewers

Comment 9

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1563074 - To fold: Address bc review feedback. r?bc

Depends on D37128

Comment 10

[Nick Alexander :nalexander [he/him]]

Folks, I finally got back to this ticket. bc, I added a follow-up commit to address what the bugs with the code as it was. Try build is percolating at

remote: https://treeherder.mozilla.org/#/jobs?repo=try&revision=b1eb63772963e621702bea7aef6c470213cc683f

To the #perftest-reviewers group, it might be best to look at the squashed commit (which is what I'll land). Everything stays in Raptor, even though the first commit moves it out and the third commit moves it back.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a632d0ec7d7d
Lift Android device performance tuning out of `testing/raptor/raptor.py`. r=rwood

Comment 12

[Alexandru Michis [:malexandru]]

https://hg.mozilla.org/mozilla-central/rev/a632d0ec7d7d