1143350 - Switch to using peep instead of pip locally and on Travis

Status: RESOLVED FIXED

(Tree Management :: Treeherder, defect, P2)

Description

[Ed Morley [:emorley]]

peep works by checking hashes listed in the requirements files.
We'll both need to add the hashes to the requirements files (peep errors out if any hashes are missing), and before we switch to using peep in production, it will be worthwhile switching in the local Vagrant environment and also on Travis, to ensure the hashes are all valid.

We'll either check peep.py into the repo or else install it globally; I'm leaning towards the former, but waiting on bug 1070470 comment 18 to see which we'll do.

This needs bug 1143033 to land first, to avoid conflicts.

Comment 1

[Ed Morley [:emorley]]

Attachment: Switch to using peep instead of pip locally and on Travis

This PR includes the changes from bug 1144916's PR too; they'll disappear from the commit list once that PR lands - but for now you'll need to look at the individual commits to reduce the noise.

Comment 2

[Mauro Doglio [:mdoglio]]

Comment on attachment 8579764 [details] [review]
Switch to using peep instead of pip locally and on Travis

Thanks for doing this, it works like a charm :-)

Comment 3

[Treeherder GitHub Bugbot]

Commits pushed to master at https://github.com/mozilla/treeherder-service

https://github.com/mozilla/treeherder-service/commit/29ca6732057638f96b688cab48a5e6d9cf7e8d8c
Bug 1143350 - Check in peep v2.2

We're checking this in so we have a known good starting point in the
chain of trust. It also simplifies our deployment requirements.

peep.py was taken from:
https://github.com/erikrose/peep/archive/2.2.tar.gz

The only alteration made was the addition of the licence block at the
top of the file, taken from LICENCE in the peep repo.

https://github.com/mozilla/treeherder-service/commit/f1aec89409188a7ae86f5c1e38728af9d9eca376
Bug 1143350 - Add peep hashes to the requirements files

The whole point of peep is that it errors out if (a) hashes aren't
specified for a package, or (b) the provided hash is incorrect. As
such before we can start using peep, we must add the hashes. The
requirements files are still compatible with pip, since it just
treats them like any other comment.

https://github.com/mozilla/treeherder-service/commit/8e67030a35d9e22c1e911ae62ca25835140f6344
Bug 1143350 - Use peep instead of pip locally, on Travis & in Docker

We want to start using peep in production, to alleviate security
concerns with the idea of auto-updating packages from PyPI on deploy.
As a first step, we switch to using peep in the Vagrant environment,
on Travis and in the Docker build - so we can confirm the hashes are
correct.

Close bug 1143350.

Comment 4

[Ed Morley [:emorley]]

Mauro/Cameron/Will/James: You'll need to either:
* manually update pip in your Vagrant environment's virtualenv & then vagrant provision
* |vagrant destroy && vagrant up| to start fresh
...to avoid errors (older pip doesn't support wheels, so you'll see hash mis-match warnings).

(In reply to Treeherder Bugbot from comment #3)
> Close bug 1143350.

Was hoping the IRC bot would close the bug, guess the "closes" string needs to be next to the first mention of the bug #, which is in the first line of the commit message; in which case I'm not going to bother, since it clutters it up.

Comment 5

[Ed Morley [:emorley]]

(In reply to Treeherder Bugbot from comment #3)
> The only alteration made was the addition of the licence block at the
> top of the file, taken from LICENCE in the peep repo.

Filed upstream https://github.com/erikrose/peep/issues/79 to get the licence added directly to peep.py