965519 - add python packages to mozharness

Status: RESOLVED FIXED

(Release Engineering :: Applications: MozharnessCore, defect)

Description

[Aki Sasaki (not active)]

mozprocess (also mozinfo mozfile)
requests
boto

Normally we would create a virtualenv to run the mozharness script in, but that would require a bootstrapping script that would also need to be configurable + error detecting.  That may be the route we go later, but for now I want to investigate checking the above packages into mozharness for more robust downloads + clones.

Comment 1

[Aki Sasaki (not active)]

Attachment: mozprocess-in-mozharness

Passes my preliminary testing.  I'd want a full test run on Ash or Cypress before merging to production.

Comment 2

[Aki Sasaki (not active)]

Comment on attachment 8367634 [details] [diff] [review]
mozprocess-in-mozharness

I basically cloned mozbase from github, and copied over the mozinfo mozfile mozprocess dirs from the mozprocess-0.15 tag.

I was able to verify that a subprocess run in the venv imported the venv mozprocess, not mozharness/mozprocess.

Comment 3

[Aki Sasaki (not active)]

Attachment: add boto 2.23.0 and requests 2.2.1

Not sure if this is sufficient?
I think I was mistaken about dependencies, but I'm not sure if you need boto/bin or any other packages.

Comment 4

[Jonathan Griffin (:jgriffin)]

Comment on attachment 8367634 [details] [diff] [review]
mozprocess-in-mozharness

Review of attachment 8367634 [details] [diff] [review]:
-----------------------------------------------------------------

Cool, lgtm

Comment 5

[Aki Sasaki (not active)]

Comment on attachment 8367634 [details] [diff] [review]
mozprocess-in-mozharness

https://hg.mozilla.org/build/mozharness/rev/e4a7a35849ee

Comment 6

[Aki Sasaki (not active)]

Comment on attachment 8367651 [details] [diff] [review]
add boto 2.23.0 and requests 2.2.1

[12:25]	<catlee>	aki: I don't think our machines have the python ssl libs install for requests to work properly
[12:25]	<aki>	boo
[12:25]	<aki>	run_command("wget...") ?
[12:25]	<catlee>	yeah
[12:25]	<catlee>	heheh
[12:26]	<catlee>	it also makes me feel dirty to ship boto+requests in mozharness
[12:26]	<aki>	yeah. mozprocess is at least mozilla built

Comment 7

[Aki Sasaki (not active)]

mozprocess is in production, and boto/requests have been wontfixed, so resolving.