Closed Bug 1408197 Opened 2 years ago Closed 2 years ago

57.0b8 beetmover failures - 'error in cryptography setup command: Invalid environment marker: python_version < '3''

Categories

(Release Engineering :: Release Automation: Other, defect, P1)

defect

Tracking

(firefox58 fixed)

RESOLVED FIXED
Tracking Status
firefox58 --- fixed

People

(Reporter: nthomas, Assigned: nthomas)

References

Details

Attachments

(1 file)

* mar depends on cryptography
* in the last 24 hours that had 2.1 and 2.1.1 releases
* the changelog (https://cryptography.io/en/latest/changelog/#v2-1) has 
   'Installing cryptography now requires pip 6 or newer'
* we have 
  'pip 1.5.6 from /mozharness/build/venv/local/lib/python2.7/site-packages (python 2.7)'

We end up with this error in any job that uses beetmover:

22:28:38     INFO -  Downloading/unpacking cryptography (from mar)
22:28:38     INFO -    Running setup.py (path:/tmp/pip-build-L3Fp9t/cryptography/setup.py) egg_info for package cryptography
22:28:38     INFO -      error in cryptography setup command: Invalid environment marker: python_version < '3'
22:28:38     INFO -      Complete output from command python setup.py egg_info:
22:28:38     INFO -      error in cryptography setup command: Invalid environment marker: python_version < '3'
See Also: → 1408199
Comment on attachment 8918052 [details]
Bug 1408197 - fix beetmover by using a modern pip & setuptools,  a=release

https://reviewboard.mozilla.org/r/188946/#review194254
Attachment #8918052 - Flags: review?(catlee) → review+
Assignee: nobody → nthomas
Pushed by nthomas@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/e006f8a22781
fix beetmover by using a modern pip & setuptools, r=catlee a=release
Keywords: leave-open
Turns out that upgrading pip and setuptools does not get us green beetmover tasks. We can process cryptography 2.1.1's setup.py now, but hit an error installing backports.lzma:

02:19:30     INFO -      x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/mozharness/build/venv/include -I/root/include -I/opt/local/include -I/usr/local/include -I/usr/include/python2.7 -c backports/lzma/_lzmamodule.c -o build/temp.linux-x86_64-2.7/backports/lzma/_lzmamodule.o
02:19:30     INFO -      backports/lzma/_lzmamodule.c:115:18: fatal error: lzma.h: No such file or directory
02:19:30     INFO -       #include <lzma.h>
02:19:30     INFO -                        ^
02:19:30     INFO -      compilation terminated.

It turns out we have hit this in 58.0b7 (and presumably earlier) but it's not fatal. After the initial failure to 'pip install mar' we retry, and pip says 'Requirement already satisfied'. This smells like a bug in early versions of pip - ie not rolling back unsuccessful installs, and/or not checking for all dependencies on subsequent install requests.

I contemplated providing lzma.h by adding the liblzma-dev package to taskcluster/docker/beet-mover/Dockerfile but there may be other bugs we hit after that. Instead I've reverted the pip/setuptools upgrade and pinned cryptography to 2.0.3 in https://hg.mozilla.org/releases/mozilla-beta/rev/d96198d960f3cbc4d17eb9990c801c1903d454fd

This should leave us in the same state as previous releases - mar gets installed but without all the requirements, but we get away with it because we never use them in beet_mover. It only wants BZ2MarFile at https://hg.mozilla.org/releases/mozilla-beta/file/56e25919ea0c/testing/mozharness/external_tools/extract_and_run_command.py#l41.

TODO: graft d96198d960f on other branches if it works
Pushed by nthomas@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/4fd5250dedfc
beetmover fix round 2, r=bustage a=release
https://hg.mozilla.org/mozilla-central/rev/4fd5250dedfc
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
This happened again today with 57.0b12 build1, as we somehow have pip 8.1.2 via bug 1297515 now. We're going to use bug 1410960 to add the missing header to resolve it.
See Also: → 1410960
See Also: → 1414391
You need to log in before you can comment on or make changes to this bug.