994176 - Create branched versions of marionette-client and consumer packages

Status: RESOLVED FIXED

(Remote Protocol :: Marionette, defect, P1)

Description

[Malini Das [:mdas] - Away, not checking bugmail]

Filing as a bug for tracking. We need to investigate how to make the marionette client backwards compatible up to v.13, to make it easier for consumers of the marionette client (like gaiatest, b2gperf, etc.). This way, they won't need to know which version of the package to use, they can just use master, and we won't need to uplift patches to different versions of the client, we'll just maintain one.

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

The latest proposal is to release packages from the branched releases, such as marionette_client_b2g28_v1_3. This would therefore be the most suitable package to use when running against gecko 28 and gaia 1.3. In this proposal marionette_client package would commit only to supporting mozilla-central, however it may work to some extent with other branches.

Comment 2

[Malini Das [:mdas] - Away, not checking bugmail]

We've decided to release marionette client packages based on branch. For example, for mozilla-b2g28_v1_3, we will release marionette_client_mozilla-b2g28_v1_3 with initial version number 0.1.

To do this, we'll need to verify that our consumers who are using unpinned (master) marionette client against these branches can safely switch to using the new packages. We'll need to uplift any missing changes before releasing the initial packages.

From our meeting, we need to verify against:
* b2gperf, b2gpopulate, and gaiatest
* marketplace tests: https://github.com/mozilla/marketplace-tests-gaia/ contains the tests that rely on some features currently in m-c
* Eideticker. The source may not branch, we'll need to find a way to pull in the right marionette client w/o branches
* any other consumer of unpinned versions of marionette. If you want to point one out, please comment.

After we release the client packages, we'll need to update the consumer packages like b2gperf, b2gpopulate and gaiatest.

Comment 3

[Dave Hunt [:davehunt] [he/him] ⌚BST]

I've tested marionette_client from the mozilla-b2g28_v1_3 branch with gaiatest from the v1.3 branch. Everything worked fine with marketplace-tests-gaia, and only a few uplifts are needed for gaiatest to get b2gperf and b2gpopulate working. I'll submit patches for each of these.

:mdas One thing I did notice is that the scrolling isn't working, so we may need the APZ fixes uplifted in the Marionette client, but I'm not sure if we really need that fixed as we're not running the FPS tests against v1.3.

:bsilverberg As you pointed out, there may be desirable uplifts in addition to the required uplifts. I think for 0.1 of each of these new packages we should only concern ourselves with the changes required to get the tests passing. We can then tackle any other uplifts (such as improving the waits) as enhancements.

Comment 4

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Attachment: Create branched version of Marionette client for mozilla-b2g28_v1_3

Comment 5

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Attachment: Create branched version of gaiatest for mozilla-b2g28_v1_3

Comment 6

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Attachment: Create branched version of b2gpopulate for mozilla-b2g28_v1_3

Comment 7

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Attachment: Create branched version of b2gperf for mozilla-b2g28_v1_3

Comment 9

[Bob Silverberg [:bsilverberg]]

(In reply to Dave Hunt (:davehunt) from comment #3)
> 
> :bsilverberg As you pointed out, there may be desirable uplifts in addition
> to the required uplifts. I think for 0.1 of each of these new packages we
> should only concern ourselves with the changes required to get the tests
> passing. We can then tackle any other uplifts (such as improving the waits)
> as enhancements.

wfm

Comment 10

[Bob Silverberg [:bsilverberg]]

Comment on attachment 8430654 [details] [review]
Create branched version of gaiatest for mozilla-b2g28_v1_3

I added a question about how to test in the PR.

Comment 11

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Landed in:
http://hg.mozilla.org/releases/mozilla-b2g28_v1_3/rev/51cacf5faa8b

Released as:
https://pypi.python.org/pypi/marionette_client-mozilla-b2g28_v1_3/0.1

Comment 12

[Malini Das [:mdas] - Away, not checking bugmail]

Attachment: PR for gaia v1.3

This is failing Gu on mozilla-b2g28 since it was trying to use tip marionette_client, so I pushed a patch to fix it on gaia's v1.3 branch.

will r? if it passes travis

Comment 13

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Malini Das [:mdas] from comment #12)
> Created attachment 8432687 [details] [review]
> PR for gaia v1.3
> 
> This is failing Gu on mozilla-b2g28 since it was trying to use tip
> marionette_client, so I pushed a patch to fix it on gaia's v1.3 branch.
> 
> will r? if it passes travis

That's what attachment 8430654 [details] [review] is for, although I'm fine with this one landing first.

Comment 14

[Ryan VanderMeulen [:RyanVM]]

(In reply to Dave Hunt (:davehunt) from comment #13)
> That's what attachment 8430654 [details] [review] is for, although I'm fine with this
> one landing first.

Why in the world was comment 11 landed without this attachment being ready? Was the bustage unexpected? What were we expecting would happen when landing one without the other?

Comment 15

[Jonathan Griffin (:jgriffin)]

Comment on attachment 8432687 [details] [review]
PR for gaia v1.3

lgtm

Comment 16

[Ryan VanderMeulen [:RyanVM]]

v1.3: https://github.com/mozilla-b2g/gaia/commit/ca34a11e16c3583eda059a7fea7d88acc67f83a4

Thanks :)

Comment 17

[Ryan VanderMeulen [:RyanVM]]

Still busted. Backed out and reverted.
https://hg.mozilla.org/releases/mozilla-b2g28_v1_3/rev/142719898564
https://github.com/mozilla-b2g/gaia/commit/2408c2238254776b0d3716753b7bf917883c9915

Comment 18

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #14)
> (In reply to Dave Hunt (:davehunt) from comment #13)
> > That's what attachment 8430654 [details] [review] is for, although I'm fine with this
> > one landing first.
> 
> Why in the world was comment 11 landed without this attachment being ready?
> Was the bustage unexpected? What were we expecting would happen when landing
> one without the other?

This is my fault - I wasn't aware that we were running Gu on mozilla-b2g28. Could someone please link to the bustage so I can see what the failure was? Malini's patch was passing on Travis, which had a prerequisite of the package being available on PyPI.

Comment 19

[Ryan VanderMeulen [:RyanVM]]

https://tbpl.mozilla.org/?tree=Mozilla-B2g28-v1.3

Comment 20

[Bob Silverberg [:bsilverberg]]

Comment on attachment 8430654 [details] [review]
Create branched version of gaiatest for mozilla-b2g28_v1_3

marketplace-tests-gaia all pass for me on a Hamachi with a v1.3 build and this branch.

Comment 21

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Bug 1020396 reminded me that we also have the marionette-transport package. Does it make sense to also branch this? Otherwise our branched marionette-client package will depend on the latest marionette-transport package.

Comment 22

[Malini Das [:mdas] - Away, not checking bugmail]

The only thing in transport.py that's at risk of breaking across branches is if we ever change what we expect from the connect() and send() function (https://mxr.mozilla.org/mozilla-central/source/testing/marionette/transport/marionette_transport/transport.py#77), which would happen if we change how we communicate with the server. We may do that to enforce WebDriver compatibility (the other option is to keep it the same and write transport.py-like libraries for each client language).


If this happens, we'll either need to pin the branches to older versions of marionette-transport or branch. Since we don't know if this will happen, I think it's safe to keep using the latest package and not branch.

Comment 23

[Jonathan Griffin (:jgriffin)]

I agree; since we expect this to change very infrequently, I think we're ok without branching.

Comment 24

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Okay this failed because we needed to land attachment 8430654 [details] [review] at the same time as the others. I'll update this patch to include the tbpl_requirements change, and reland when I have time to watch TBPL like a hawk.

Comment 25

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 8432687 [details] [review]
PR for gaia v1.3

I've incorporated this change into attachment 8430654 [details] [review].

Comment 26

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Landed in:
http://hg.mozilla.org/releases/mozilla-b2g28_v1_3/rev/4e9be6836a3d
https://github.com/mozilla-b2g/gaia/commit/e049de79ff46fcab51f29a88d5869c5efb48fc3c

Comment 27

[Dave Hunt [:davehunt] [he/him] ⌚BST]

I've released gaiatest-v1.3 version 0.1 to PyPI:
https://pypi.python.org/pypi/gaiatest-v1.3/0.1

Comment 28

[Dave Hunt [:davehunt] [he/him] ⌚BST]

b2gpopulate-v1.3

Landed in:
https://github.com/mozilla/b2gpopulate/commit/c10b5696ff1250f4ad3daa5bc358b7fbfb96fd27

Released as:
https://pypi.python.org/pypi/b2gpopulate-v1.3/0.1

Tagged in:
https://github.com/mozilla/b2gpopulate/releases/tag/v1.3-0.1

Comment 29

[Dave Hunt [:davehunt] [he/him] ⌚BST]

b2gperf-v1.3

Landed in:
https://github.com/mozilla/b2gperf/commit/7c878a4c4047c4ce441c569ab4c170f36cd2a0e5

Released as:
https://pypi.python.org/pypi/b2gperf-v1.3/0.1

Tagged in:
https://github.com/mozilla/b2gperf/releases/tag/v1.3-0.1

Comment 30

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Attachment: Create branched version of Marionette client for mozilla-b2g30_v1_4

Comment 31

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Attachment: Create branched version of gaiatest for mozilla-b2g30_v1_4

Comment 32

[Andrew Halberstadt [:ahal]]

Comment on attachment 8445138 [details] [diff] [review]
Create branched version of Marionette client for mozilla-b2g30_v1_4

Review of attachment 8445138 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks for doing this!

Comment 33

[Zac C (:zac)]

Comment on attachment 8445139 [details] [review]
Create branched version of gaiatest for mozilla-b2g30_v1_4

r+

Comment 34

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Attachment: Create branched version of Marionette client for mozilla-b2g30_v1_4

Fixed mozrunner to be the latest from the target branch. Carried r+

Comment 35

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Zac C (:zac) from comment #33)
> Comment on attachment 8445139 [details] [review]
> Create branched version of gaiatest for mozilla-b2g30_v1_4
> 
> r+

I'm currently trying to get a clean run of tests against these new branched versions. There's a unit test fix that I'll be uplifting, but I think I've resolved a lot of the other issues I was having locally. My last run unfortunately hit a crash that wasn't possible to recover from. I'll update when I have more results.

Comment 36

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Okay, all tests are now passing for me. I'll attach the patches for b2gpopulate and b2gperf. The latter required a revert that was dependent on a more recent gaiatest release.

Comment 37

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Attachment: Create branched version of b2gpopulate for mozilla-b2g30_v1_4

Comment 38

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Attachment: Create branched version of b2gperf for mozilla-b2g30_v1_4

The revert here was from https://github.com/mozilla/b2gperf/commit/1dba265dfff7ccf9c83f4fe60bca5454c49cc311 which uses getActiveApp, which was added after the branch was created.

Comment 39

[William Lachance (:wlach)]

Comment on attachment 8445952 [details] [diff] [review]
Create branched version of b2gperf for mozilla-b2g30_v1_4

Review of attachment 8445952 [details] [diff] [review]:
-----------------------------------------------------------------

makes sense to me.

Comment 40

[William Lachance (:wlach)]

Comment on attachment 8445949 [details] [diff] [review]
Create branched version of b2gpopulate for mozilla-b2g30_v1_4

lgtm

Comment 41

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Landed marionette/gaiatest changes in:
http://hg.mozilla.org/releases/mozilla-b2g30_v1_4/rev/210037af19ef
https://github.com/mozilla-b2g/gaia/commit/01427c878b8713734f7d5c1c588f1251e757ca70

Comment 42

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Released as:
https://pypi.python.org/pypi/marionette_client-mozilla-b2g30_v1_4/0.1
https://pypi.python.org/pypi/gaiatest-v1.4/0.1

Comment 43

[Dave Hunt [:davehunt] [he/him] ⌚BST]

b2gpopulate-v1.4

Landed in:
https://github.com/mozilla/b2gpopulate/commit/3b7f565eb67aa8451945d91f45f1c317ab9f1d35

Released as:
https://pypi.python.org/pypi/b2gpopulate-v1.4/0.1

Tagged in:
https://github.com/mozilla/b2gpopulate/releases/tag/v1.4-0.1

Comment 44

[Dave Hunt [:davehunt] [he/him] ⌚BST]

b2gperf-v1.4

Landed in:
https://github.com/mozilla/b2gperf/commit/1bdf17a63e8b9a922348e59afb35253d970ba94f

Released as:
https://pypi.python.org/pypi/b2gperf-v1.4/0.1

Tagged in:
https://github.com/mozilla/b2gperf/releases/tag/v1.4-0.1