1527507 - Create a constraints file and move all non-direct dependencies there

Status: RESOLVED FIXED

(Webtools Graveyard :: Pontoon, enhancement, P3)

Description

[Jarek Śmiejczak [:jotes]]

Newer versions of Pip (7.1+) support constraints files. They constrain versions (and hashes) of dependencies that are allowed to be installed.

I think this could slightly improve the amazing work done by :adrian.

With this feature, we can move all sub-dependencies to a separate file and improve readability and maintainability of the main list of dependencies.

To keep constraints maintainable, every entry in constraints.txt has to be pinned, hashed and linked (via comment) to its parent dependency.

Related:

• bug 1395231
• bug 1407449 (shows how dependencies were refactored in Socorro)

Comment 1

[Axel Hecht [:Pike]]

I've done this for elmo, too. I ended up writing down the dependency ranges, too, see https://github.com/mozilla/elmo/blob/master/requirements/constraints.txt.

That's a major hassle the first time around, but maintaining it is actually OK so far. And it does help with situations where you have a new version via pip list -o, but it's not supported by the stuff that you're using.

Comment 2

[Matjaz Horvat [:mathjazz]]

https://github.com/mozilla/pontoon/commit/6da1f3524b093c0141f3553af185474d69c704ed

Comment 3

[Matjaz Horvat [:mathjazz]]

1527507

Comment 4

[Matjaz Horvat [:mathjazz]]

This patch fails to install requirements on Heroku prod:

...
-----> Installing requirements with pip
       You must give at least one requirement to install (see "pip help install")
-----> Running post-compile hook
...

That's because as of this patch, this is no longer true:
"To specify Python package dependencies on Heroku via pip, add a pip requirements file named requirements.txt to the root of your repository."
https://devcenter.heroku.com/articles/python-pip

The patch worked on stage, because dev requirements are installed in a post_compile script (and they include main requirements):
https://github.com/mozilla/pontoon/blob/master/bin/post_compile#L3

We should fix this by either extending the post_compile script to install main requirements on prod or restoring the requirements.txt file.

Comment 5

[[DEACTIVATED] Adrian Gaudebert]

Can we put a requirements.txt file that just includes requirements/prod.txt? And add a comment that it is there specifically for Heroku?

Comment 6

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/pontoon/pull/1225

Comment 7

[Matjaz Horvat [:mathjazz]]

Fixed in:
https://github.com/mozilla/pontoon/commit/cb50ff5d4af4b5c79d965a13ab4fd757c52a082d

And confirmed on prod.