1712337 - Provide a single interface for specifying the binary to use in the test

Status: RESOLVED FIXED

(Testing :: mozperftest, task, P2)

Description

[Greg Mierzwinski [:sparky]]

Currently, we have no single spot that we can look at for the binary that we should be using. We should add this --binary option to our global args here: https://searchfox.org/mozilla-central/source/python/mozperftest/mozperftest/argparser.py#28

Comment 1

[Greg Mierzwinski [:sparky]]

This is a major blocker for the application version, and name collection. Currently, my only option is to implement a metadata method to set/get binary info, along with a mobile flag so that the metrics layer can take care of version gathering. However, there's an issue with this implementation too. When we start running chrome tests in this harness, we'll need to get the application version before the test run to figure out which chromedriver we should use. Ideally, we would implement a system layer to gather this, but we don't have the binary by that stage, and can't figure out what it should be without hardcoding checks for special CLI flags which will need to be constantly updated.

I might rework all the binaries options to take care of this now rather than pushing it down the road.

Comment 2

[Greg Mierzwinski [:sparky]]

Testing comment.

Comment 3

[Greg Mierzwinski [:sparky]]

Attachment: Bug 1712337 - Add a binary setup layer. r?#perftest

This patch adds a layer to setup the binary, and provide a single interface for specifying it and using it through the metadata. At the same time, some classes are added to gather the binary version on mobile, and desktop. These version producers are used by the classes in the BinarySetup layer that are decorated with the binary_setup function.

At the same time, some changes were made to the perfherder metrics production to gather the application name, and version from the metadata. The intermediate results are also properly passed into the perfherder blob production step as they were previously being ignored.

Comment 4

[Greg Mierzwinski [:sparky]]

Attachment: Bug 1712337 - Add helper classes for gathering binary, and binary version. r?#perftest

This patch adds a set of classes that will be used for gathering the binary path, and binary version in the VersionProducer, and BinarySetup layers.

Comment 5

[Greg Mierzwinski [:sparky]]

Attachment: Bug 1712337 - Add a layer for gathering the binary version. r?#perftest

This patch adds a layer to gather the version of a given binary. The layer makes use of the tooling from setups.py to do this.

Depends on D211666

Comment 6

[Greg Mierzwinski [:sparky]]

Attachment: Bug 1712337 - Add helper classes for gathering binary, and binary version. r?#perftest

This patch adds a set of classes that will be used for gathering the binary path, and binary version in the VersionProducer, and BinarySetup layers.

Comment 7

[Greg Mierzwinski [:sparky]]

Attachment: Bug 1712337 - Add a binary setup layer. r?#perftest

This patch adds a layer to setup the binary, and provide a single interface for specifying it and using it through the metadata. At the same time, some changes were made to the perfherder metrics production to gather the application name, and version from the metadata. The intermediate results are also properly passed into the perfherder blob production step as they were previously being ignored.

Depends on D211737

Comment 8

[Greg Mierzwinski [:sparky]]

Attachment: Bug 1712337 - Add a layer for gathering the binary version. r?#perftest

This patch adds a layer to gather the version of a given binary. The layer makes use of the tooling from setups.py to do this.

Depends on D211738

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/mozilla-central/rev/605e81f8e7e0
Add helper classes for gathering binary, and binary version. r=perftest-reviewers,aglavic
https://hg.mozilla.org/mozilla-central/rev/0bb2dcd678b2
Add a binary setup layer. r=perftest-reviewers,kshampur,aglavic
https://hg.mozilla.org/mozilla-central/rev/8e22e49cd148
Add a layer for gathering the binary version. r=perftest-reviewers,aglavic

Comment 10

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/605e81f8e7e0
https://hg.mozilla.org/mozilla-central/rev/0bb2dcd678b2
https://hg.mozilla.org/mozilla-central/rev/8e22e49cd148