Closed Bug 805440 Opened 7 years ago Closed 7 years ago

Mozrunner installed from pypi when running


(Testing :: Mozbase, defect)

Not set


(firefox18 fixed, firefox19 fixed)

Tracking Status
firefox18 --- fixed
firefox19 --- fixed


(Reporter: mcote, Assigned: k0scist)



(3 files)

When you run, it determines that marionette-client is a dependency (of mozb2g, I believe) that has to be installed from pypi. The script installs this first, but it relies on mozrunner, so the latter is also installed from pypi, even though it will be installed in development mode later on in the process. This is sure to cause confusion.

$ python 
Searching for marionette-client
Best match: marionette-client 0.5.1
Processing marionette_client-0.5.1.tar.gz
Running marionette_client-0.5.1/ -q bdist_egg --dist-dir /tmp/easy_install-CHl_kv/marionette_client-0.5.1/egg-dist-tmp-jLFtyX
Adding marionette-client 0.5.1 to easy-install.pth file
Installing marionette script to /tmp/mozbase/bin

Installed /tmp/mozbase/lib/python2.7/site-packages/marionette_client-0.5.1-py2.7.egg
Processing dependencies for marionette-client
Searching for mozrunner==5.10
Best match: mozrunner 5.10
It seems that when mozrunner is later installed in development mode, this overrides the previous regular egg installation, so this is probably not a critical bug.
Jeff, this looks very similar to what I have seen on bug 804886 with mozinfo on Tuesday this week. So it's just a variation from it.
Its a variation in that they are both setup problems.  I don't think they are related more so than that, though the original issue in bug 804886 is hard to determine based on the output available.

This is a very particular issue of a particular class of issues. To review how works, it will
1. look at the dependencies for all (given) mozbase packages
2. installs dependencies from pypi
3. unrolls these dependencies in such an order that mozbase internal dependencies are installed in an order that the dependent packages are installed following their dependencies

The problem is: because we install marionette client from pypi in step 2. and then install the local mozbase, because mozrunner is a dependency of a pypi dep, it will grab mozrunner from pypi.  This *should* be overwritten when the local mozrunner is installed.  Is this the case, :mcote?

In any case, I think we can move 2. after 3. and avoid this whole problem.
> will grab mozrunner from pypi.  This *should* be overwritten when the local mozrunner is installed.  Is this the case, :mcote?

Reading comment 0 again, this does seem to be the case.
So comment 3 is correct.  However, there is an additional problem.  Currently, mozb2g depends on marionette client which is a pypi-based dependency (i.e. it does not live in mozbase). However, marionette_client is pegged to use mozrunner == 5.10 and mozprocess == 0.5 .  This makes it impossible to have the HEAD of mozbase installed for development due to the version requirements conflicts.

If we're going to allow this pattern of things in mozbase depending on things outside of mozbase that depend on things in mozbase, then the things outside of mozbase depend on things in mozbase, we must be lenient with the versions that we require such that we don't end up making this inconsistent as is the current state.

A solution is to make marionette client use mozrunner >= mozrunner 5.10 and mozprocess >= 0.5, bump its version, release it to pypi, and bump mozb2gs version requiring it to use the new released version of marionette_client.
:jgriffin, :ahal, :mdas, could one of you either do this or provide me pypi ownership?
I can do it (probably not until tomorrow), but I've added you as owner too.
Attachment #675583 - Flags: review?(jgriffin)
Comment on attachment 675583 [details] [diff] [review]
bump version for pypi release

Thanks Jeff.
Attachment #675583 - Flags: review?(jgriffin) → review+
Will bump version to 0.2 and re-release following this
Attachment #675610 - Flags: review?(jgriffin)
Not sure if we should peg mozdevice >= some_version along with this
Comment on attachment 675610 [details] [diff] [review]
depend on new marionette_client

thanks!  yes, we should probably >= for mozdevice too.
Attachment #675610 - Flags: review?(jgriffin) → review+
Will peg to >= 0.12
(In reply to Jonathan Griffin (:jgriffin) from comment #13)
> Comment on attachment 675610 [details] [diff] [review]
> depend on new marionette_client
> thanks!  yes, we should probably >= for mozdevice too.

seems to work on my system
Attachment #675642 - Flags: review?(mcote)
Assignee: nobody → jhammel
Closed: 7 years ago
Flags: in-testsuite-
Resolution: --- → FIXED
Target Milestone: --- → mozilla19
Sorry, I should have marked this as [leave-open]
Resolution: FIXED → ---
Comment on attachment 675642 [details] [diff] [review]
install internal dependencies before external dependencies

Attachment #675642 - Flags: review?(mcote) → review+
This should actually be fixed now :)
Closed: 7 years ago7 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.