1141808 - Implement ability to dynamically load a compatible python client when a test needs to run against multiple versions of gecko

Status: RESOLVED INCOMPLETE

(Remote Protocol :: Marionette, defect)

Description

[Chris Manchester (limited bugmail, email directly)]

This is primarily a concern for update tests. The scenario is we have two version of gecko, gecko A and gecko B, and the python clients that shipped along with those gecko versions, client A and client B. At some point in the development cycle of gecko B, changes are landed that mean we have a client B that can't interact with gecko A. When running a test that needs to update from A to B, we need to be able to communicate with both versions of gecko. The proposed solution is to implement a facility in the marionette python client to find a compatible python client and load it dynamically based on the gecko version it's communicating with. In order to retrieve the gecko version from marionette transport, we need to assume that we will have a version of marionette transport that's able to work with any version of gecko. It seems like we'll need to restrict any test doing an update to use only endpoints implemented in the older version of gecko and uplift new endpoints as needed.

We'll need to return the current gecko version when connecting to marionette and we'll need to make a mapping from gecko versions to client versions. We'll also need to allow connecting to browser before instantiating the Marionette object on the python side (this happens in Marionette.__init__ right now, but in the proposed solution we can't instantiate the marionette object until we know what version of gecko we need to support, which would come from MarionetteTransport when we connect to the browser).

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

When you mention compatible Python clients are you talking about the branched packages such as marionette-client-b2g34_v2_1? I suspect this model will need to be updated now that we have split the test runner and driver into separate packages.

Comment 2

[Chris Manchester (limited bugmail, email directly)]

Are we landing changes in those clients that don't end up on pypi? If so I don't know how we would end up handling that. If not I think a linear history in pypi that contains all the compatible versions will give us what we mean. The driver split helps us here, it's the driver part we want to re-load dynamically.

Comment 3

[Dave Hunt [:davehunt] [he/him] ⌚BST]

They are all released to PyPI:
https://pypi.python.org/pypi/marionette_client-mozilla-b2g28_v1_3
https://pypi.python.org/pypi/marionette_client-mozilla-b2g30_v1_4
https://pypi.python.org/pypi/marionette_client-mozilla-b2g32_v2_0
https://pypi.python.org/pypi/marionette_client-mozilla-b2g34_v2_1

When this was discussed (at length) it was decided that using versions in PyPI wasn't going to work. I don't recall the specifics, but would be happy to dig them out.

Comment 4

[Chris Manchester (limited bugmail, email directly)]

(In reply to Dave Hunt (:davehunt) from comment #3)
> They are all released to PyPI:
> https://pypi.python.org/pypi/marionette_client-mozilla-b2g28_v1_3
> https://pypi.python.org/pypi/marionette_client-mozilla-b2g30_v1_4
> https://pypi.python.org/pypi/marionette_client-mozilla-b2g32_v2_0
> https://pypi.python.org/pypi/marionette_client-mozilla-b2g34_v2_1
> 
> When this was discussed (at length) it was decided that using versions in
> PyPI wasn't going to work. I don't recall the specifics, but would be happy
> to dig them out.

Ok, thanks. I wasn't aware of that but will certainly have to take it into account or be explicit that these aren't supported.

Comment 5

[Dave Hunt [:davehunt] [he/him] ⌚BST]

IIRC part of the issue is how we would version these and handle uplifts that require subsequent PyPI releases for older branched. I think another suggestion was to version the client/driver after the gecko version, and to manage dependencies to use (for example) marionette-driver version 0.38.*

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+2]]

The initial attempt for this was for our update tests working across versions of Firefox. Given that we didn't follow this path and all works fine with backward compatible code, I'm going to close this bug for now. Once there is a need again, we can easily reopen.