964740 - web-platform-tests mozharness script fails to install dependencies

Status: RESOLVED FIXED

(Release Engineering :: Applications: MozharnessCore, defect)

Description

[James Graham [:jgraham]]

15:26:45    ERROR - Return code: 1
15:26:45    FATAL - Error running install of package, /builds/slave/test/build/venv/bin/pip install --download-cache /builds/slave/test/build/venv/cache --no-index --find-links http://pypi.pvt.build.mozilla.org/pub --find-links http://pypi.pub.build.mozilla.org/pub /builds/slave/test/build/tests/marionette!
15:26:45    FATAL - Running post_fatal callback...
15:26:45    FATAL - Exiting -1

Going to try copying the marionette script more closely, although it is far from obvious why it should make a difference.

Comment 1

[James Graham [:jgraham]]

Attachment: Try using marionette_requirements.txt file

This reuses the marionette file directly. It should probably be changed to use a different file that specifies the same set of dependencies, but I would like to avoid changing too many variables at the same time.

Comment 2

[[e-mail] (:bhearsum)]

It looks like the script is trying to install mozinfo 0.7, which isn't available on the RelEng pypi server. IIRC we broke things last time we tried using it because some scripts didn't specify that they needed an older mozinfo.

If you can, the simplest solution here is using mozinfo 0.5.

Comment 3

[James Graham [:jgraham]]

So, why does it seem to work for Marionette tests on B2G? https://tbpl.mozilla.org/php/getParsedLog.php?id=33657374&tree=Cedar&full=1

Comment 4

[James Graham [:jgraham]]

Oh, maybe that is using mozbase from the tree, in which case I guess this patch should do the right thing.

In any case this is a sucky situation because I *can't* control what version of mozinfo I use; I depend on marionette, which has to be the latest version to match Gecko, and it in turn depends on mozinfo >= 0.7[1].

[1] https://tbpl.mozilla.org/php/getParsedLog.php?id=33657374&tree=Cedar&full=1

Comment 5

[James Graham [:jgraham]]

Sigh, wrong url:

[1] http://hg.mozilla.org/mozilla-central/file/340704b289c7/testing/marionette/client/requirements.txt

Comment 6

[[e-mail] (:bhearsum)]

(In reply to James Graham [:jgraham] from comment #4)
> Oh, maybe that is using mozbase from the tree, in which case I guess this
> patch should do the right thing.

Yeah, looks that way to me too.

> In any case this is a sucky situation because I *can't* control what version
> of mozinfo I use; I depend on marionette, which has to be the latest version
> to match Gecko, and it in turn depends on mozinfo >= 0.7[1].

I'll have to defer to jgriffin or someone else at this point...I think we have strategies for working around this, but I'm not exactly sure what they are.

Good luck!

Comment 7

[Jonathan Griffin (:jgriffin)]

So yes, we really want to use mozbase from the tree.  In the past we've experimented with using packages in releng's internal pypi instead, but found that to be problematic.  Unless we use strict versioning everywhere, we can cause unexpected breakage when uploading new packages to the internal pypi, we can't restrict different gecko trees to particular mozbase package versions, and using internal pypi means we can't test the effect of mozbase changes on try.

So, the approach you've used in this patch is the best way forward that we've found...install mozbase packages from the tree.

Comment 8

[Armen [:armenzg]]

jgraham, where are we in here?