1498626 - gps docker image fails to build

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

https://queue.taskcluster.net/v1/task/ekFs0t4rRYaAqeGHiQ48BQ/runs/0/artifacts/public/logs/live_backing.log
[task 2018-10-12T16:17:29.932Z] Searching for rsa>=3.1.4
[task 2018-10-12T16:17:29.932Z] Reading https://pypi.python.org/simple/rsa/
[task 2018-10-12T16:17:29.985Z] Best match: rsa 4.0
[task 2018-10-12T16:17:29.985Z] Downloading https://files.pythonhosted.org/packages/cb/d0/8f99b91432a60ca4b1cd478fd0bdf28c1901c58e3a9f14f4ba3dba86b57f/rsa-4.0.tar.gz#sha256=1a836406405730121ae9823e19c6e806c62bbad73f890574fff50efa4122c487
[task 2018-10-12T15:28:16.474Z] Processing rsa-4.0.tar.gz
[task 2018-10-12T15:28:16.482Z] Writing /tmp/easy_install-0r0spo7d/rsa-4.0/setup.cfg
[task 2018-10-12T15:28:16.482Z] Running rsa-4.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-0r0spo7d/rsa-4.0/egg-dist-tmp-ia_6h1hy
[task 2018-10-12T15:28:16.511Z] [91mTraceback (most recent call last):
[task 2018-10-12T15:28:16.511Z]   File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 154, in save_modules
[task 2018-10-12T15:28:16.511Z]     yield saved
[task 2018-10-12T15:28:16.511Z]   File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 195, in setup_context
[task 2018-10-12T15:28:16.511Z]     yield
[task 2018-10-12T15:28:16.511Z]   File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 243, in run_setup
[task 2018-10-12T15:28:16.511Z]     DirectorySandbox(setup_dir).run(runner)
[task 2018-10-12T15:28:16.511Z]   File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 273, in run
[task 2018-10-12T15:28:16.511Z]     return func()
[task 2018-10-12T15:28:16.511Z]   File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 242, in runner
[task 2018-10-12T15:28:16.511Z]     _execfile(setup_script, ns)
[task 2018-10-12T15:28:16.511Z]   File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 46, in _execfile
[task 2018-10-12T15:28:16.511Z]     exec(code, globals, locals)
[task 2018-10-12T15:28:16.511Z]   File "/tmp/easy_install-0r0spo7d/rsa-4.0/setup.py", line 20, in <module>
[task 2018-10-12T15:28:16.511Z]     author='Mozilla Release Engineering',
[task 2018-10-12T15:28:16.511Z]   File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
[task 2018-10-12T15:28:16.511Z]     return codecs.ascii_decode(input, self.errors)[0]
[task 2018-10-12T15:28:16.511Z] UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 898: ordinal not in range(128)

reproduced:
https://taskcluster-artifacts.net/ekFs0t4rRYaAqeGHiQ48BQ/1/public/logs/live_backing.log

What's weird here is that line 20 of setup.py in https://files.pythonhosted.org/packages/cb/d0/8f99b91432a60ca4b1cd478fd0bdf28c1901c58e3a9f14f4ba3dba86b57f/rsa-4.0.tar.gz is *not* 

  author='Mozilla Release Engineering',

and in fact it's not a Mozilla package.  I guess that might be an issue with Python's line number cache and a distraction from the actual issue here.

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

Johan, it looks like this occurs when pip installing https://github.com/mozilla-releng/mozapkpublisher, pinned to
  https://github.com/mozilla-releng/mozapkpublisher/commit/50750fceb7c0d831cf61cb550af62b2c93e8e755
(0.7.2).  Any idea what the issue might be?  Would it make sense to just require rsa<4.0, since that's a semver breaking change?

Comment 2

[Johan Lorenzo [:jlorenzo] - PTO - Back April 13th]

Attachment: Bug 1498626 - Fix google-play-string docker image build r=dustin,mtabara

Fix google-play-string docker image build

Comment 3

[Johan Lorenzo [:jlorenzo] - PTO - Back April 13th]

I think I see what happened. setup.py can silently upgrade/downgrade pinned packages[1]. My patch restricts the installation to what's only listed in requirements.txt and installs mozapkpublisher itself. This fixes the root cause of the problem.

[1] https://github.com/mozilla-releng/mozapkpublisher/blob/8051c7f98959e2f041ce63a1911ade88827c3a66/tox.ini#L23

Comment 4

[Johan Lorenzo [:jlorenzo] - PTO - Back April 13th]

Attachment: Bug 1498626 - Fix google-play-string docker image build r=dustin,mtabara

Fix google-play-string docker image build

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4a81fd604696
Fix google-play-string docker image build r=dustin

Comment 6

[Daniel Varga [:dvarga]]

https://hg.mozilla.org/mozilla-central/rev/4a81fd604696

Comment 7

[Tom Prince [:tomprince]]

https://hg.mozilla.org/releases/mozilla-esr60/rev/ecc55f87f965