1448113 - py3_install_modules should respect pip arguments

Status: RESOLVED DUPLICATE of bug 1453658

(Release Engineering :: Applications: MozharnessCore, enhancement)

Description

[Chris AtLee [:catlee]]

In https://treeherder.mozilla.org/logviewer.html#?job_id=169664700&repo=mozilla-inbound&lineNumber=1538 we found that Windows machines were downloading some files from public pypi. Further down, you can see that the job runs `pip install mitmproxy` without specifying any of the idnex related arguments. So all of the subsequent dependencies are being fetched from public pypi.

The code responsible for this looks like it's
https://searchfox.org/mozilla-central/source/testing/mozharness/mozharness/base/python.py#782

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

So py3_install_requirement_files() at https://searchfox.org/mozilla-central/source/testing/mozharness/mozharness/base/python.py#824 looks up the pip args, but py3_install_modules() doesn't.

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

catlee said on IRC that the issue is related to source tarballs in the requirements, where setuptools reads setup.py for dependencies and talks to pypi to resolve them. pip doesn't pass on the hosts it's using to setuptools, and there is a pip issue somewhere where adding that was WONTFIXed. An alternative approach is to set up config files for pip and setuptools, taking care of pip's changes to where it reads from (https://pip.pypa.io/en/stable/user_guide/#config-file).