1400746 - checksums failure when boto3 and botocore present in our pypi mirrors

Status: RESOLVED FIXED

(Release Engineering :: Release Automation: Other, enhancement, P1)

Description

[Nick Thomas [:nthomas] (UTC+12)]

Bug 1361930 added boto3 and botocore to our mirrors, then the checksums task in the release automation makes some bad life choices:
  Using version 3-1.4.7 (newest of versions: 3-1.4.7, 3-1.4.7, 2.38.0, 2.38.0, 2.32.1, 2.32.1, 2.17.0, 2.17.0, core-1.7.7, core-1.7.7)

Notice it's just trimming the leading 'boto' off boto3-1.4.7 and botocore-1.7.7, and ends up installing boto3. This was fixed in pip 1.2 via https://github.com/pypa/pip/issues/510, but we're still on 1.1 because this is a job which runs via bbb on a linux host, and /tools/python27-virtualenv/lib/python2.7/site-packages/virtualenv_support/pip-1.1.tar.gz. Hrmph!

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: [gecko] Upgrade pip before installing boto

mozharness is using pypi.{pub,pvt}.build.mozilla.org when it create the virtualenv and we have the slightly less ancient 1.5.5 there, so this change is enough to install the correct boto. Could be explicit about the version if that would be more defensive, given recent pip upgrade issues.

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/f8c3e7d5f10d
checksums failure when boto3 and botocore present in our pypi mirrors, r=catlee DONTBUILD

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: [gecko] Upgrade pip before installing boto, with pinned version

r+ catlee on IRC.

https://hg.mozilla.org/integration/mozilla-inbound/rev/f8c3e7d5f10df3296e0547284ec97fe74852a0bd
https://hg.mozilla.org/releases/mozilla-beta/rev/6b5cce5da78594813192d46f129b6e5a012c9650
https://hg.mozilla.org/releases/mozilla-release/rev/10d1856666637305af928822c10768fa707cb4e9
https://hg.mozilla.org/releases/mozilla-esr52/rev/2c88cc5f5dabc8526352537e4a75277059509594

Comment 4

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/f8c3e7d5f10d

Comment 5

[Ryan VanderMeulen [:RyanVM]]

(In reply to Nick Thomas [:nthomas] from comment #3)
> https://hg.mozilla.org/releases/mozilla-beta/rev/
> 6b5cce5da78594813192d46f129b6e5a012c9650

Can we please avoid double-landing patches on inbound/beta until after the version bump to 58? We're syncing m-c and m-b anyway and the more things that get landed straight to Beta, the higher risk of having to rely on hg's merge resolutions to do the right thing.

Comment 6

[Nick Thomas [:nthomas] (UTC+12)]

Sorry about that, old habits.

Comment 7

[Ryan VanderMeulen [:RyanVM]]

Grafted to the FIREFOX_56b13_RELBRANCH also.
https://hg.mozilla.org/releases/mozilla-beta/rev/ab81f42e9b47

Comment 8

[Nick Thomas [:nthomas] (UTC+12)]

We could probably back this out now that we have pip 8.1.2 in tree via bug 1297515.