1107336 - Split marionette automation API into its own package and not have a monolithic package

Status: RESOLVED FIXED

(Testing :: Marionette Client and Harness, defect, P1)

Description

[David Burns :automatedtester]

We need to split this so that it allows us to be able to drop in packages when we need them

Comment 1

[David Burns :automatedtester]

https://tbpl.mozilla.org/?tree=Try&rev=56822afff618

Comment 2

[Jonathan Griffin (:jgriffin)]

You need to update testsuite-targets.mk to package the new marionette-driver package in tests.zip:

http://dxr.mozilla.org/mozilla-central/source/testing/testsuite-targets.mk#536

You can follow the same pattern that was used for packaging the transport.

Comment 3

[David Burns :automatedtester]

https://tbpl.mozilla.org/?tree=Try&rev=ee79f173590d

Comment 4

[David Burns :automatedtester]

https://tbpl.mozilla.org/?tree=Try&rev=3a1b4e5e7fd2

Comment 5

[David Burns :automatedtester]

https://tbpl.mozilla.org/?tree=Try&rev=60b7d51bf48a

Comment 6

[David Burns :automatedtester]

https://tbpl.mozilla.org/?tree=Try&rev=433a4655e746

Comment 7

[Chris Manchester (limited bugmail, email directly)]

There's at least 1 loop test (browser/components/loop/test/functional/test_1_browser_call.py) that doesn't run in automation but might want to have its imports updated with this bug.

Comment 8

[David Burns :automatedtester]

(In reply to Chris Manchester [:chmanchester] from comment #7)
> There's at least 1 loop test
> (browser/components/loop/test/functional/test_1_browser_call.py) that
> doesn't run in automation but might want to have its imports updated with
> this bug.

Updated in my patch queue

Comment 9

[David Burns :automatedtester]

https://tbpl.mozilla.org/?tree=Try&rev=20ae63a0a7d7

Comment 10

[David Burns :automatedtester]

https://tbpl.mozilla.org/?tree=Try&rev=6de0373a8df9

Comment 11

[David Burns :automatedtester]

Attachment: MozReview Request: bz://1107336/AutomatedTester

Comment 12

[David Burns :automatedtester]

/r/2603 - Bug 1107336: Update mach for newly created MarionetteDriver
/r/2605 - Bug 1107336: Update Marionette unit tests to use Marionette Driver modules
/r/2607 - Bug 1107336: Update layout tests that use marionette to use new Marionette Driver module
/r/2609 - Bug 1107336: Move marionette code for driving the browser out of the runner
/r/2611 - Bug 1107336: Update testsuite targets to make sure MarionetteDriver is available
/r/2613 - Bug 1107336: update tests not covered by manifest for when they are ready
/r/2615 - Bug 1107336: Update imports to new Marionette Driver from runner
/r/2617 - Bug 1107336: correct loop imports for marionette
/r/2619 - Bug 1107336: selection tests imports for marionette
/r/2621 - Bug 1107336: Final clean up of imports
/r/2623 - Bug 1107336: Update imports for loop tests
/r/2625 - Bug 1107336: fix rebase errors

Pull down these commits:

hg pull review -r 6de0373a8df9044f8c9c813a69d6e4de5527d6c7

Comment 13

[Jonathan Griffin (:jgriffin)]

Comment on attachment 8550540 [details]
MozReview Request: bz://1107336/AutomatedTester

https://reviewboard.mozilla.org/r/2601/#review1935

Try looks good, and this patch does too, although it's large so I may have missed something!

Comment 14

[Jonathan Griffin (:jgriffin)]

As a postscript, I think we may need to modify https://dxr.mozilla.org/mozilla-central/source/testing/marionette/client/requirements.txt as well.

Comment 15

[David Burns :automatedtester]

have added in item in comment 14

Comment 16

[David Burns :automatedtester]

no sheriffs around currently so will land first thing tomorrow (~9 hours from now)

Comment 17

[David Burns :automatedtester]

https://hg.mozilla.org/integration/mozilla-inbound/rev/e6d21c2fa111
https://hg.mozilla.org/integration/mozilla-inbound/rev/d1847f46d293
https://hg.mozilla.org/integration/mozilla-inbound/rev/c886cd910936
https://hg.mozilla.org/integration/mozilla-inbound/rev/3c3d31607785
https://hg.mozilla.org/integration/mozilla-inbound/rev/805dce7ae79f
https://hg.mozilla.org/integration/mozilla-inbound/rev/6aca01a96d37
https://hg.mozilla.org/integration/mozilla-inbound/rev/9aec81381396
https://hg.mozilla.org/integration/mozilla-inbound/rev/5a2e085824ae
https://hg.mozilla.org/integration/mozilla-inbound/rev/3c4e0437ace5
https://hg.mozilla.org/integration/mozilla-inbound/rev/4ac07f1f2998

Comment 18

[Carsten Book [:Tomcat]]

sorry had to back this out for failing tests like https://treeherder.mozilla.org/logviewer.html#?job_id=5709786&repo=mozilla-inbound

Comment 19

[David Burns :automatedtester]

https://tbpl.mozilla.org/?tree=Try&rev=9351779e7fc2

Comment 20

[David Burns :automatedtester]

https://tbpl.mozilla.org/?tree=Try&rev=728061ade154

Comment 21

[David Burns :automatedtester]

https://tbpl.mozilla.org/?tree=Try&rev=d28eb389f271

Comment 22

[David Burns :automatedtester]

https://tbpl.mozilla.org/?tree=Try&rev=ef6e39ebbffa

Comment 23

[David Burns :automatedtester]

https://tbpl.mozilla.org/?tree=Try&rev=2181fb59c647

Comment 24

[David Burns :automatedtester]

Comment on attachment 8550540 [details]
MozReview Request: bz://1107336/AutomatedTester

/r/2603 - Bug 1107336: Update b2g desktop imports to use marionette driver; r=ahal
/r/2605 - Bug 1107336: Update mach for newly created MarionetteDriver; r=jgriffin
/r/2607 - Bug 1107336: Update layout tests that use marionette to use new Marionette Driver module; r=jgriffin
/r/2609 - Bug 1107336: Move marionette code for driving the browser out of the runner; r=jgriffin
/r/2611 - Bug 1107336: Update testsuite targets to make sure MarionetteDriver is available; r=jgriffin
/r/2613 - Bug 1107336: update tests not covered by manifest for when they are ready; r=jgriffin
/r/2615 - Bug 1107336: Update imports to new Marionette Driver from runner; r=jgriffin
/r/2617 - Bug 1107336: correct loop imports for marionette; r=jgriffin
/r/2619 - Bug 1107336: selection tests imports for marionette; r=jgriffin
/r/2621 - Bug 1107336: Update imports for loop tests; r=jgriffin
/r/2623 - Bug 1107336: update Marionette-Client dependencies to include marionette driver; r=jgriffin
/r/2625 - Bug 1107336: Update Marionette unit tests to use Marionette Driver modules; r=jgriffin
/r/3397 - Bug 1107336: Update b2g desktop imports of marionette; r=jgriffin

Pull down these commits:

hg pull review -r b7643d78d660b06aee4e5fbe4b8e6701b99e1667

Comment 25

[David Burns :automatedtester]

for some reason my try push is not picking up my changes to WPTRunner that landed in https://github.com/w3c/wptrunner/pull/70 and have been synced to m-c :/

Comment 26

[Andrew Halberstadt [:ahal]]

https://reviewboard.mozilla.org/r/2603/#review2765

Ship It!

Comment 27

[Andrew Halberstadt [:ahal]]

Comment on attachment 8550540 [details]
MozReview Request: bz://1107336/AutomatedTester

Note: this is an r+ on the single commit I was flagged for in the review request.

Comment 28

[Jonathan Griffin (:jgriffin)]

Comment on attachment 8550540 [details]
MozReview Request: bz://1107336/AutomatedTester

https://reviewboard.mozilla.org/r/2601/#review2793

::: browser/components/loop/test/functional/test_1_browser_call.py
(Diff revision 2)
> -    def check_video(self, selector):

Did you really mean to delete content from this test?

Looks good; we should ask jgraham about the wpt failures.  For the Gip failures, we'll need push a fix to the Gaia repo, which will be a bit tricky. :(

Comment 29

[David Burns :automatedtester]

patch up in bug 1131356

Comment 30

[David Burns :automatedtester]

Comment on attachment 8550540 [details]
MozReview Request: bz://1107336/AutomatedTester

/r/2603 - Bug 1107336: Update b2g desktop imports to use marionette driver; r=ahal
/r/2605 - Bug 1107336: Update mach for newly created MarionetteDriver; r=jgriffin
/r/2607 - Bug 1107336: Update layout tests that use marionette to use new Marionette Driver module; r=jgriffin
/r/2609 - Bug 1107336: Move marionette code for driving the browser out of the runner; r=jgriffin
/r/2611 - Bug 1107336: Update testsuite targets to make sure MarionetteDriver is available; r=jgriffin
/r/2613 - Bug 1107336: update tests not covered by manifest for when they are ready; r=jgriffin
/r/2615 - Bug 1107336: Update imports to new Marionette Driver from runner; r=jgriffin
/r/2617 - Bug 1107336: correct loop imports for marionette; r=jgriffin
/r/2619 - Bug 1107336: selection tests imports for marionette; r=jgriffin
/r/2621 - Bug 1107336: update Marionette-Client dependencies to include marionette driver; r=jgriffin
/r/2623 - Bug 1107336: Update Marionette unit tests to use Marionette Driver modules; r=jgriffin
/r/2625 - Bug 1107336: Update b2g desktop imports of marionette; r=jgriffin

Pull down these commits:

hg pull review -r 9faeb4777def4f9f71cb670484cce1d9210f59ad

Comment 31

[David Burns :automatedtester]

Comment on attachment 8550540 [details]
MozReview Request: bz://1107336/AutomatedTester

/r/2603 - Bug 1107336: Update b2g desktop imports to use marionette driver; r=ahal
/r/2605 - Bug 1107336: Update mach for newly created MarionetteDriver; r=jgriffin
/r/2607 - Bug 1107336: Update layout tests that use marionette to use new Marionette Driver module; r=jgriffin
/r/2609 - Bug 1107336: Move marionette code for driving the browser out of the runner; r=jgriffin
/r/2611 - Bug 1107336: Update testsuite targets to make sure MarionetteDriver is available; r=jgriffin
/r/2613 - Bug 1107336: update tests not covered by manifest for when they are ready; r=jgriffin
/r/2615 - Bug 1107336: Update imports to new Marionette Driver from runner; r=jgriffin
/r/2617 - Bug 1107336: correct loop imports for marionette; r=jgriffin
/r/2619 - Bug 1107336: selection tests imports for marionette; r=jgriffin
/r/2621 - Bug 1107336: update Marionette-Client dependencies to include marionette driver; r=jgriffin
/r/2623 - Bug 1107336: Update Marionette unit tests to use Marionette Driver modules; r=jgriffin
/r/2625 - Bug 1107336: Update b2g desktop imports of marionette; r=jgriffin

Pull down these commits:

hg pull review -r 7e952380edfde4a4f1a4e05a0631fe3d69437953

Comment 32

[Jonathan Griffin (:jgriffin)]

Comment on attachment 8550540 [details]
MozReview Request: bz://1107336/AutomatedTester

https://reviewboard.mozilla.org/r/2601/#review2903

Thanks for the update; lgtm

Comment 33

[David Burns :automatedtester]

https://hg.mozilla.org/integration/mozilla-inbound/rev/a07bb95eb209
https://hg.mozilla.org/integration/mozilla-inbound/rev/9c4b171c23be
https://hg.mozilla.org/integration/mozilla-inbound/rev/60d9d4d20971
https://hg.mozilla.org/integration/mozilla-inbound/rev/0d04801bb2f0
https://hg.mozilla.org/integration/mozilla-inbound/rev/399a436c6e5f
https://hg.mozilla.org/integration/mozilla-inbound/rev/f92a5c6256fa
https://hg.mozilla.org/integration/mozilla-inbound/rev/4f60c437140c
https://hg.mozilla.org/integration/mozilla-inbound/rev/dad798a5e595
https://hg.mozilla.org/integration/mozilla-inbound/rev/4817d3cd3810
https://hg.mozilla.org/integration/mozilla-inbound/rev/995911340bf7
https://hg.mozilla.org/integration/mozilla-inbound/rev/9ef91d4fcf3b

Comment 34

[David Burns :automatedtester]

https://hg.mozilla.org/integration/mozilla-inbound/rev/5075f0063d70

Comment 35

[Ryan VanderMeulen [:RyanVM]]

Still busted after the follow-up. Backed out.
https://hg.mozilla.org/integration/mozilla-inbound/rev/fe79362ffaaf

https://treeherder.mozilla.org/logviewer.html#?job_id=6525459&repo=mozilla-inbound

Comment 36

[David Burns :automatedtester]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=37b260e0d8a0

Comment 37

[David Burns :automatedtester]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=8c6130f52e72

Comment 38

[David Burns :automatedtester]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=7e952380edfd

Comment 39

[David Burns :automatedtester]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=403dd4aed1e7

Comment 40

[David Burns :automatedtester]

wptrunner patch up for review https://critic.hoppipolla.co.uk/r/4035

Everything looks green! \o/

Comment 41

[David Burns :automatedtester]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=ce1eb111c04b

Comment 42

[David Burns :automatedtester]

Comment on attachment 8550540 [details]
MozReview Request: bz://1107336/AutomatedTester

/r/2603 - Bug 1107336: Update loop tests to use new Marionette Driver; r=jgriffin
/r/2605 - Bug 1107336: Update b2g desktop ref test runner to use new Marionette Driver; r=ahal
/r/2607 - Bug 1107336: Update layout tests to use new Marionette Driver; r=jgriffin
/r/2609 - Bug 1107336: update build and test targets to use new marionette driver; r=jgriffin
/r/2611 - Bug 1107336: Update marionette unit tests to use new marionette driver; r=jgriffin
/r/2613 - Bug 1107336: update marionette client runners to use the new marionette driver; r=jgriffin
/r/2615 - Bug 1107336: Create Marionette Driver containining all of the automation API code; r=jgriffin

Pull down these commits:

hg pull review -r e9ee645e6c62eca51ac7fd49a33651f90bcfe317

Comment 43

[Jonathan Griffin (:jgriffin)]

Comment on attachment 8550540 [details]
MozReview Request: bz://1107336/AutomatedTester

https://reviewboard.mozilla.org/r/2601/#review3413

Ship It!

Comment 44

[David Burns :automatedtester]

Comment on attachment 8550540 [details]
MozReview Request: bz://1107336/AutomatedTester

Carrying r+ forward

Comment 45

[David Burns :automatedtester]

https://hg.mozilla.org/integration/mozilla-inbound/rev/7dd7b9e55286
https://hg.mozilla.org/integration/mozilla-inbound/rev/cd2c67fa2f10
https://hg.mozilla.org/integration/mozilla-inbound/rev/9b848f9d0bc2
https://hg.mozilla.org/integration/mozilla-inbound/rev/91f60eb6aa27
https://hg.mozilla.org/integration/mozilla-inbound/rev/ce483404c6bf
https://hg.mozilla.org/integration/mozilla-inbound/rev/473f27ac1760
https://hg.mozilla.org/integration/mozilla-inbound/rev/2712637917a5

Comment 46

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/7dd7b9e55286
https://hg.mozilla.org/mozilla-central/rev/cd2c67fa2f10
https://hg.mozilla.org/mozilla-central/rev/9b848f9d0bc2
https://hg.mozilla.org/mozilla-central/rev/91f60eb6aa27
https://hg.mozilla.org/mozilla-central/rev/ce483404c6bf
https://hg.mozilla.org/mozilla-central/rev/473f27ac1760
https://hg.mozilla.org/mozilla-central/rev/2712637917a5

Comment 47

[Henrik Skupin [:whimboo][⌚️UTC+1]]

With that patch landed we fail to install the marionette-driver package because of the following failure:

  Running setup.py (path:/home/henrik/.virtualenvs/test/build/marionette-driver/setup.py) egg_info for package marionette-driver
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/home/henrik/.virtualenvs/test/build/marionette-driver/setup.py", line 8, in <module>
        with open('requirements.txt') as f:
    IOError: [Errno 2] No such file or directory: 'requirements.txt'

I would suggest to do a follow-up fix on this bug, which will make it easier to maintain for a backport to the aurora branch for Firefox 38.

Comment 48

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Sorry the problem here is really that marionette-driver has not been released yet. Lets get this fixed via bug 1124667 ASAP.

Comment 49

[Henrik Skupin [:whimboo][⌚️UTC+1]]

The underlying problem is that marionette-driver does not have a MANIFEST.in file and that's why requirements.txt is not getting packaged. I took care of this in my patch on bug 1124667.

Comment 50

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/8cf1b1ce73bb

Comment 51

[David Burns :automatedtester]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=b3616ff265d0

Comment 52

[David Burns :automatedtester]

uplifted to Aurora

https://treeherder.mozilla.org/#/jobs?repo=mozilla-aurora&revision=2ca434cb843b

Comment 54

[David Burns :automatedtester]

Attachment: MozReview Request: Bug 1107336: Update b2g desktop imports of marionette; r=jgriffin

Comment 55

[David Burns :automatedtester]

Attachment: MozReview Request: Bug 1107336: Update b2g desktop imports of marionette; r=jgriffin

Comment 56

[David Burns :automatedtester]

Attachment: MozReview Request: Bug 1107336: Update loop tests to use new Marionette Driver; r=jgriffin

Comment 57

[David Burns :automatedtester]

Attachment: MozReview Request: Bug 1107336: Update b2g desktop ref test runner to use new Marionette Driver; r=ahal

Comment 58

[David Burns :automatedtester]

Attachment: MozReview Request: Bug 1107336: Update layout tests to use new Marionette Driver; r=jgriffin

Comment 59

[David Burns :automatedtester]

Attachment: MozReview Request: Bug 1107336: update build and test targets to use new marionette driver; r=jgriffin

Comment 60

[David Burns :automatedtester]

Attachment: MozReview Request: Bug 1107336: Update marionette unit tests to use new marionette driver; r=jgriffin

Comment 61

[David Burns :automatedtester]

Attachment: MozReview Request: Bug 1107336: update marionette client runners to use the new marionette driver; r=jgriffin

Comment 62

[David Burns :automatedtester]

Attachment: MozReview Request: Bug 1107336: Create Marionette Driver containining all of the automation API code; r=jgriffin

Comment 63

[David Burns :automatedtester]

Attachment: MozReview Request: Bug 1107336: correct loop imports for marionette; r=jgriffin

Comment 64

[David Burns :automatedtester]

Attachment: MozReview Request: Bug 1107336: selection tests imports for marionette; r=jgriffin

Comment 65

[David Burns :automatedtester]

Attachment: MozReview Request: Bug 1107336: update Marionette-Client dependencies to include marionette driver; r=jgriffin

Comment 66

[David Burns :automatedtester]

Attachment: MozReview Request: Bug 1107336: Update Marionette unit tests to use Marionette Driver modules; r=jgriffin

Comment 67

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Moving bugs for Marionette client due to component changes.