Mozrunner installed from pypi when running setup_development.py

RESOLVED FIXED in Firefox 18

Status

Testing
Mozbase
RESOLVED FIXED
5 years ago
5 years ago

People

(Reporter: mcote, Assigned: Jeff Hammel)

Tracking

Trunk
mozilla19
Points:
---
Bug Flags:
in-testsuite -

Firefox Tracking Flags

(firefox18 fixed, firefox19 fixed)

Details

Attachments

(3 attachments)

(Reporter)

Description

5 years ago
When you run setup_development.py, it determines that marionette-client is a dependency (of mozb2g, I believe) that has to be installed from pypi. The script installs this first, but it relies on mozrunner, so the latter is also installed from pypi, even though it will be installed in development mode later on in the process. This is sure to cause confusion.

$ python setup_development.py 
Searching for marionette-client
Reading http://pypi.python.org/simple/marionette_client/
Reading https://wiki.mozilla.org/Auto-tools/Projects/Marionette
Best match: marionette-client 0.5.1
Downloading http://pypi.python.org/packages/source/m/marionette_client/marionette_client-0.5.1.tar.gz#md5=ff1dfcaf176ce79fa546cb622b48f7b2
Processing marionette_client-0.5.1.tar.gz
Running marionette_client-0.5.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-CHl_kv/marionette_client-0.5.1/egg-dist-tmp-jLFtyX
Adding marionette-client 0.5.1 to easy-install.pth file
Installing marionette script to /tmp/mozbase/bin

Installed /tmp/mozbase/lib/python2.7/site-packages/marionette_client-0.5.1-py2.7.egg
Processing dependencies for marionette-client
Searching for mozrunner==5.10
Reading http://pypi.python.org/simple/mozrunner/
Reading http://github.com/mozautomation/mozmill
Reading http://mikeal.github.com/mozrunner/
Reading http://code.google.com/p/mozrunner/
Reading https://wiki.mozilla.org/Auto-tools/Projects/MozBase
Reading https://github.com/mozilla/mozbase/tree/master/mozrunner
Best match: mozrunner 5.10
Downloading http://pypi.python.org/packages/source/m/mozrunner/mozrunner-5.10.tar.gz#md5=58c1b4121685f8ae20199b9c93c77e66
(Reporter)

Comment 1

5 years ago
It seems that when mozrunner is later installed in development mode, this overrides the previous regular egg installation, so this is probably not a critical bug.
Jeff, this looks very similar to what I have seen on bug 804886 with mozinfo on Tuesday this week. So it's just a variation from it.
(Assignee)

Comment 3

5 years ago
Its a variation in that they are both setup problems.  I don't think they are related more so than that, though the original issue in bug 804886 is hard to determine based on the output available.

This is a very particular issue of a particular class of issues. To review how https://github.com/mozilla/mozbase/blob/master/setup_development.py works, it will
1. look at the dependencies for all (given) mozbase packages
2. installs dependencies from pypi
3. unrolls these dependencies in such an order that mozbase internal dependencies are installed in an order that the dependent packages are installed following their dependencies

The problem is: because we install marionette client from pypi in step 2. and then install the local mozbase, because mozrunner is a dependency of a pypi dep, it will grab mozrunner from pypi.  This *should* be overwritten when the local mozrunner is installed.  Is this the case, :mcote?

In any case, I think we can move 2. after 3. and avoid this whole problem.
(Assignee)

Comment 4

5 years ago
> ...it will grab mozrunner from pypi.  This *should* be overwritten when the local mozrunner is installed.  Is this the case, :mcote?

Reading comment 0 again, this does seem to be the case.
(Assignee)

Comment 5

5 years ago
So comment 3 is correct.  However, there is an additional problem.  Currently, mozb2g depends on marionette client which is a pypi-based dependency (i.e. it does not live in mozbase). However, marionette_client is pegged to use mozrunner == 5.10 and mozprocess == 0.5 .  This makes it impossible to have the HEAD of mozbase installed for development due to the version requirements conflicts.

If we're going to allow this pattern of things in mozbase depending on things outside of mozbase that depend on things in mozbase, then the things outside of mozbase depend on things in mozbase, we must be lenient with the versions that we require such that we don't end up making this inconsistent as is the current state.

A solution is to make marionette client use mozrunner >= mozrunner 5.10 and mozprocess >= 0.5, bump its version, release it to pypi, and bump mozb2gs version requiring it to use the new released version of marionette_client.
(Assignee)

Comment 6

5 years ago
:jgriffin, :ahal, :mdas, could one of you either do this or provide me pypi ownership?
I can do it (probably not until tomorrow), but I've added you as owner too.
(Assignee)

Comment 8

5 years ago
Created attachment 675583 [details] [diff] [review]
bump version for pypi release
Attachment #675583 - Flags: review?(jgriffin)
Comment on attachment 675583 [details] [diff] [review]
bump version for pypi release

Thanks Jeff.
Attachment #675583 - Flags: review?(jgriffin) → review+
(Assignee)

Comment 10

5 years ago
Pushed to inbound: http://hg.mozilla.org/integration/mozilla-inbound/rev/027bbf3ee90e
and released to pypi: http://pypi.python.org/pypi/marionette_client
(Assignee)

Comment 11

5 years ago
Created attachment 675610 [details] [diff] [review]
depend on new marionette_client

Will bump version to 0.2 and re-release following this
Attachment #675610 - Flags: review?(jgriffin)
(Assignee)

Comment 12

5 years ago
Not sure if we should peg mozdevice >= some_version along with this
Comment on attachment 675610 [details] [diff] [review]
depend on new marionette_client

thanks!  yes, we should probably >= for mozdevice too.
Attachment #675610 - Flags: review?(jgriffin) → review+
(Assignee)

Comment 14

5 years ago
Will peg to >= 0.12
(Assignee)

Comment 15

5 years ago
(In reply to Jonathan Griffin (:jgriffin) from comment #13)
> Comment on attachment 675610 [details] [diff] [review]
> depend on new marionette_client
> 
> thanks!  yes, we should probably >= for mozdevice too.

pushed: https://github.com/mozilla/mozbase/commit/55f09d438709202fb6d9fec8b705fed3cc4eb63f
(Assignee)

Comment 16

5 years ago
version bumped: https://github.com/mozilla/mozbase/commit/e627ea3de6ff80cd27823e929e4ed47298306c8d
(Assignee)

Comment 17

5 years ago
Created attachment 675642 [details] [diff] [review]
install internal dependencies before external dependencies

seems to work on my system
Attachment #675642 - Flags: review?(mcote)
https://hg.mozilla.org/mozilla-central/rev/027bbf3ee90e
Assignee: nobody → jhammel
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Flags: in-testsuite-
Resolution: --- → FIXED
Target Milestone: --- → mozilla19
(Assignee)

Comment 19

5 years ago
Sorry, I should have marked this as [leave-open]
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Reporter)

Comment 20

5 years ago
Comment on attachment 675642 [details] [diff] [review]
install internal dependencies before external dependencies

wfm!
Attachment #675642 - Flags: review?(mcote) → review+
(Assignee)

Comment 21

5 years ago
pushed: https://github.com/mozilla/mozbase/commit/1f679a8ae3ff49d1e3c96cbf3c01ad4742568e1b
(Assignee)

Comment 22

5 years ago
This should actually be fixed now :)
Status: REOPENED → RESOLVED
Last Resolved: 5 years ago5 years ago
Resolution: --- → FIXED
https://hg.mozilla.org/releases/mozilla-aurora/rev/30cb831b2dc7
status-firefox18: --- → fixed
status-firefox19: --- → fixed
You need to log in before you can comment on or make changes to this bug.