Closed Bug 1405177 Opened 2 years ago Closed 2 years ago

release failure when boto3 and botocore present in our pypi mirrors

Categories

(Release Engineering :: Release Automation: Other, enhancement, P1)

enhancement

Tracking

(firefox56 fixed, firefox57 fixed, firefox58 fixed)

RESOLVED FIXED
Tracking Status
firefox56 --- fixed
firefox57 --- fixed
firefox58 --- fixed

People

(Reporter: tomprince, Assigned: tomprince)

References

Details

Attachments

(1 file)

This also affects the other release scripts in the same directory when running on buildbot, which is true for thunderbird.

+++ This bug was initially created as a clone of Bug #1400746 +++

Bug 1361930 added boto3 and botocore to our mirrors, then the checksums task in the release automation makes some bad life choices:
  Using version 3-1.4.7 (newest of versions: 3-1.4.7, 3-1.4.7, 2.38.0, 2.38.0, 2.32.1, 2.32.1, 2.17.0, 2.17.0, core-1.7.7, core-1.7.7)

Notice it's just trimming the leading 'boto' off boto3-1.4.7 and botocore-1.7.7, and ends up installing boto3. This was fixed in pip 1.2 via https://github.com/pypa/pip/issues/510, but we're still on 1.1 because this is a job which runs via bbb on a linux host, and /tools/python27-virtualenv/lib/python2.7/site-packages/virtualenv_support/pip-1.1.tar.gz. Hrmph!
Comment on attachment 8914559 [details]
Bug 1405177: Install (slightly) newer pip to install boto.

https://reviewboard.mozilla.org/r/185884/#review190858

r+ if you remove the beet_mover.py change.

::: testing/mozharness/scripts/release/antivirus.py:83
(Diff revision 1)
>          BaseScript.__init__(self,
>              config_options=self.config_options,
>              require_config_file=False,
>              config={
>                  "virtualenv_modules": [
> +                    "pip==1.5.5",

This is not used by Firefox so the change is fine for TB.

::: testing/mozharness/scripts/release/beet_mover.py:151
(Diff revision 1)
>              # Default configuration
>              'config': {
>                  # base index url where to find taskcluster artifact based on taskid
>                  "artifact_base_url": 'https://queue.taskcluster.net/v1/task/{taskid}/artifacts/public/{subdir}',
>                  "virtualenv_modules": [
> +                    "pip==1.5.5",

This isn't used for TB, but is for Firefox in an environment where pip v.1.5.6 is available. Lets drop this hunk.

::: testing/mozharness/scripts/release/push-candidate-to-releases.py:74
(Diff revision 1)
>          BaseScript.__init__(self,
>              config_options=self.config_options,
>              require_config_file=False,
>              config={
>                  "virtualenv_modules": [
> +                    "pip==1.5.5",

This is shared by Firefox and Thunderbird, but Firefox runs it on a system with pip v1.5.6. I think that's OK because the pip changelog has only 
* Upgrade requests to 2.3.0 to fix an issue with proxies on Python 3.4.1 (PR #1821).
in 1.5.6, which is irrelevant to us. And mozharness should downgrade it OK, certainly 'pip install pip==1.5.5' works locally with 1.5.6 in place.
Attachment #8914559 - Flags: review?(nthomas) → review+
Keywords: checkin-needed
Pushed by ryanvm@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/706ffd585a89
Install (slightly) newer pip to install boto. r=nthomas
Keywords: checkin-needed
https://hg.mozilla.org/mozilla-central/rev/706ffd585a89
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
We could probably back this out now that we have pip 8.1.2 in tree via bug 1297515.
You need to log in before you can comment on or make changes to this bug.