1467255 - verify requirements files in puppet repo as part of CI

Status: RESOLVED FIXED

(Release Engineering :: General, enhancement)

Description

[[e-mail] (:bhearsum)]

We're about to enable pyup.io for the Puppet repo, and because many of the apps we deploy don't have their own CI or tests, we're vulnerable to a couple of cases where pyup could give us bad dependencies. Specifically:
* If an updated version of a package in a requirements file adds a new transitive dependency (for example: requests recently started depending on a number of outside packages, and pyup bumped the requests version, but didn't add the newly required packages)
* If two first order dependencies have incompatible version requirements on a transitive dependency (eg: A depends on aiohttp <3.0, but B depends on aiohttp >=3.0, and A and B are both in the requirements file).

Callek, you had some ideas about how to validate this. I'm not volunteering you to do it (but feel free to!), but if you could reiterate them here that would be helpful.

Comment 1

[Justin Wood (:Callek)]

So my idea is roughly:

`pip install -r <requirements-file>` in a new venv for each requirements file. This should catch the incompatible versions issue (and we 'might' be able to convince TOX/TRAVIS/TC to only run that against changed requirements files too)

The other idea can be solved if requirements files have hashes by the first thought, alternatively we can run `hashin -r <output> <requirements-on-commandline>` to generate a requirements file with hashes that we can then run against pip.

Specifically the dependent package issue is solved by requiring hashes and the fact that some dependencies are transitively included and attempted to be installed without hashes triggers the error.

Hashes in the in-puppet requirements files is a nice to have, but I would deem it not critical since we have the packages on our own internal mirror anyway rather than continually downloading from pypi.

Comment 2

[[e-mail] (:bhearsum)]

I started working on this in https://github.com/mozilla-releng/build-puppet/compare/master...mozbhearsum:requirements-verify?expand=1 - not close to done yet, though.

Comment 3

[[e-mail] (:bhearsum)]

Attachment: verify requirements files in CI

Comment 4

[Justin Wood (:Callek)]

Comment on attachment 8985214 [details] [diff] [review]
verify requirements files in CI

Review of attachment 8985214 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good to me, thanks!