Closed
Bug 1460402
Opened 7 years ago
Closed 7 years ago
codespell: in treeherder, use the pip package instead of the distro package (too old)
Categories
(Developer Infrastructure :: Lint and Formatting, enhancement)
Developer Infrastructure
Lint and Formatting
Tracking
(firefox62 fixed)
RESOLVED
FIXED
mozilla62
Tracking | Status | |
---|---|---|
firefox62 | --- | fixed |
People
(Reporter: Sylvestre, Assigned: Sylvestre)
Details
Attachments
(4 files)
For example, in https://taskcluster-artifacts.net/cUrSg7yGTIq_iP7lSO4oUQ/0/public/logs/live_backing.log
it fails because of:
[task 2018-05-09T12:04:42.050Z] Unable to match regex against output: codespell: error: no such option: --ignore-words
which shipped in codespell 1.11
I guess we are using a too old version (the Debian & Ubuntu are pretty old).
Comment hidden (mozreview-request) |
Comment hidden (mozreview-request) |
Comment hidden (mozreview-request) |
Comment hidden (mozreview-request) |
Comment 5•7 years ago
|
||
mozreview-review |
Comment on attachment 8974513 [details]
Bug 1460402 - Create a new class to manage pip install
https://reviewboard.mozilla.org/r/242850/#review248698
Code analysis found 2 defects in this patch:
- 2 defects found by mozlint
You can run this analysis locally with:
- `./mach lint path/to/file` (JS/Python)
If you see a problem in this automated review, please report it here: http://bit.ly/2y9N9Vx
::: python/mozlint/mozlint/pip.py:1
(Diff revision 1)
> +#!/usr/bin/env python
Error: Missing from __future__ import absolute_import [py2: require absolute_import]
::: python/mozlint/mozlint/pip.py:1
(Diff revision 1)
> +#!/usr/bin/env python
Error: Missing from __future__ import print_function [py2: require print_function]
Comment hidden (mozreview-request) |
Comment hidden (mozreview-request) |
Comment hidden (mozreview-request) |
Comment hidden (mozreview-request) |
Comment 10•7 years ago
|
||
mozreview-review |
Comment on attachment 8974513 [details]
Bug 1460402 - Create a new class to manage pip install
https://reviewboard.mozilla.org/r/242850/#review248700
Code analysis found 2 defects in this patch:
- 2 defects found by mozlint
You can run this analysis locally with:
- `./mach lint path/to/file` (JS/Python)
If you see a problem in this automated review, please report it here: http://bit.ly/2y9N9Vx
::: python/mozlint/mozlint/pip.py:1
(Diff revision 2)
> +# This Source Code Form is subject to the terms of the Mozilla Public
Error: Missing from __future__ import absolute_import [py2: require absolute_import]
::: python/mozlint/mozlint/pip.py:1
(Diff revision 2)
> +# This Source Code Form is subject to the terms of the Mozilla Public
Error: Missing from __future__ import print_function [py2: require print_function]
Assignee | ||
Updated•7 years ago
|
Assignee: nobody → sledru
Assignee | ||
Updated•7 years ago
|
Summary: codespell: too old version of codespell used in treeherder → codespell: in treeherder, use the pip package instead of the distro package (too old)
Comment 11•7 years ago
|
||
mozreview-review |
Comment on attachment 8974513 [details]
Bug 1460402 - Create a new class to manage pip install
https://reviewboard.mozilla.org/r/242850/#review248922
Thanks for the cleanup!
Eventually I'd like if we could use the build system's virtualenv management infrastructure, but that is definitely follow-up fodder.
::: python/mozlint/mozlint/pip.py:1
(Diff revision 2)
> +# This Source Code Form is subject to the terms of the Mozilla Public
Please move this file to `python/mozlint/mozlint/util/pip.py`, as it isn't really related to the core mozlint library at all.
Attachment #8974513 -
Flags: review?(ahalberstadt) → review+
Comment 12•7 years ago
|
||
mozreview-review |
Comment on attachment 8974514 [details]
Bug 1460402 - Update the flake8 support to use the pip class
https://reviewboard.mozilla.org/r/242852/#review248924
::: tools/lint/python/__init__.py:13
(Diff revision 2)
> from collections import defaultdict
>
> import which
> from mozprocess import ProcessHandlerMixin
>
> -from mozlint import result
> +from mozlint import pip, result
This will need to be `from mozlint.util import pip` based on the issue in the previous commit.
Attachment #8974514 -
Flags: review?(ahalberstadt) → review+
Comment 13•7 years ago
|
||
mozreview-review |
Comment on attachment 8974515 [details]
Bug 1460402 - Update the codespell support to use pip instead of the package
https://reviewboard.mozilla.org/r/242854/#review248920
::: tools/lint/spell/__init__.py:22
(Diff revision 2)
>
> -from mozlint import result
> +from mozlint import pip, result
> from mozprocess import ProcessHandlerMixin
>
> +here = os.path.abspath(os.path.dirname(__file__))
> +CODESPELL_REQUIREMENTS_PATH = os.path.join(here, 'codespell_requirements.txt')
I think you forgot to add this file.
Attachment #8974515 -
Flags: review?(ahalberstadt) → review-
Comment 14•7 years ago
|
||
mozreview-review |
Comment on attachment 8974516 [details]
Bug 1460402 - Update the CI to use codespell with pip instead of the apt packages (too old)
https://reviewboard.mozilla.org/r/242856/#review248926
Attachment #8974516 -
Flags: review?(ahalberstadt) → review+
Comment hidden (mozreview-request) |
Comment hidden (mozreview-request) |
Comment hidden (mozreview-request) |
Comment hidden (mozreview-request) |
Comment 19•7 years ago
|
||
mozreview-review |
Comment on attachment 8974515 [details]
Bug 1460402 - Update the codespell support to use pip instead of the package
https://reviewboard.mozilla.org/r/242854/#review249000
Thanks for fixing this, lgtm!
Attachment #8974515 -
Flags: review?(ahalberstadt) → review+
Comment 20•7 years ago
|
||
Pushed by sledru@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/06ceda084d69
Create a new class to manage pip install r=ahal
https://hg.mozilla.org/integration/autoland/rev/e1caff997e5a
Update the flake8 support to use the pip class r=ahal
https://hg.mozilla.org/integration/autoland/rev/17526c61b995
Update the codespell support to use pip instead of the package r=ahal
https://hg.mozilla.org/integration/autoland/rev/5b40f3f18f42
Update the CI to use codespell with pip instead of the apt packages (too old) r=ahal
Comment 21•7 years ago
|
||
Backed out 4 changesets (bug 1460402) for breaking taskcluster images. CLOSED TREE
Log:
https://treeherder.mozilla.org/logviewer.html#?job_id=177938228&repo=autoland&lineNumber=1596
[task 2018-05-10T20:25:19.075Z] Successfully built configparser
[task 2018-05-10T20:25:19.076Z] Installing collected packages: pyflakes, enum34, configparser, pycodestyle, mccabe, flake8, setuptools
[task 2018-05-10T20:25:19.190Z] Found existing installation: setuptools 20.7.0
[task 2018-05-10T20:25:19.192Z] Not uninstalling setuptools at /usr/lib/python2.7/dist-packages, outside environment /usr
[task 2018-05-10T20:25:19.383Z] Successfully installed configparser-3.5.0 enum34-1.1.6 flake8-3.5.0 mccabe-0.6.1 pycodestyle-2.3.1 pyflakes-1.5.0 setuptools-38.5.1
[task 2018-05-10T20:25:19.592Z] You are using pip version 8.1.1, however version 10.0.1 is available.
[task 2018-05-10T20:25:19.592Z] You should consider upgrading via the 'pip install --upgrade pip' command.
[task 2018-05-10T20:25:19.614Z]
[task 2018-05-10T20:25:19.615Z] ###
[task 2018-05-10T20:25:19.615Z] # codespell Setup
[task 2018-05-10T20:25:19.615Z] ###
[task 2018-05-10T20:25:19.615Z]
[task 2018-05-10T20:25:19.615Z] cd /setup
[task 2018-05-10T20:25:19.615Z]
[task 2018-05-10T20:25:19.615Z] pip install --require-hashes -r /tmp/codespell_requirements.txt
[task 2018-05-10T20:25:20.161Z] Could not open requirements file: [Errno 2] No such file or directory: '/tmp/codespell_requirements.txt'
[task 2018-05-10T20:25:20.320Z] You are using pip version 8.1.1, however version 10.0.1 is available.
[task 2018-05-10T20:25:20.320Z] You should consider upgrading via the 'pip install --upgrade pip' command.
[task 2018-05-10T20:25:21.411Z] Traceback (most recent call last):
[task 2018-05-10T20:25:21.411Z] File "/builds/worker/checkouts/gecko/taskcluster/mach_commands.py", line 447, in build_image
[task 2018-05-10T20:25:21.411Z] build_image(image_name, tag, os.environ)
[task 2018-05-10T20:25:21.411Z] File "/builds/worker/checkouts/gecko/taskcluster/taskgraph/docker.py", line 83, in build_image
[task 2018-05-10T20:25:21.411Z] docker.post_to_docker(buf.getvalue(), '/build', nocache=1, t=tag)
[task 2018-05-10T20:25:21.411Z] File "/builds/worker/checkouts/gecko/taskcluster/taskgraph/util/docker.py", line 107, in post_to_docker
[task 2018-05-10T20:25:21.411Z] raise Exception(data['error'])
[task 2018-05-10T20:25:21.411Z] Exception: The command [/bin/sh -c bash /tmp/system-setup.sh] returned a non-zero code: 1
kill 8
+ kill 8
[taskcluster 2018-05-10 20:25:21.654Z] === Task Finished ===
[taskcluster 2018-05-10 20:25:21.765Z] Artifact "public/image.tar.zst" not found at "/builds/worker/workspace/artifacts/image.tar.zst"
[taskcluster 2018-05-10 20:25:22.307Z] Unsuccessful task run with exit code: 1 completed in 342.827 seconds
Push with failures:
https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=5b40f3f18f42e292d68bad895bc08bad184e847c
Backout:
https://hg.mozilla.org/integration/autoland/rev/bca28818826ba017ad63fc50a1ce2783b0c17d0a
Flags: needinfo?(sledru)
Comment hidden (mozreview-request) |
Comment 23•7 years ago
|
||
Pushed by sledru@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/3e50885329c4
Create a new class to manage pip install r=ahal
https://hg.mozilla.org/integration/autoland/rev/bb12ffd4b96e
Update the flake8 support to use the pip class r=ahal
https://hg.mozilla.org/integration/autoland/rev/3676e913dbff
Update the codespell support to use pip instead of the package r=ahal
https://hg.mozilla.org/integration/autoland/rev/c2e8fbd72ca6
Update the CI to use codespell with pip instead of the apt packages (too old) r=ahal
Comment 24•7 years ago
|
||
Backed out 4 changesets (bug 1460402) for lint failure on intl/locales/en-US/hyphenation/hyph_en_US.dic. CLOSED TREE
Log:
https://treeherder.mozilla.org/logviewer.html#?job_id=177948127&repo=autoland&lineNumber=445
task 2018-05-10T21:38:25.321Z] Unable to match regex against output: WARNING: Binary file: /builds/worker/checkouts/gecko/browser/branding/unofficial/pbmode.ico
[task 2018-05-10T21:38:25.321Z] Unable to match regex against output: WARNING: Binary file: /builds/worker/checkouts/gecko/browser/branding/unofficial/default128.png
[task 2018-05-10T21:38:25.321Z] Unable to match regex against output: WARNING: Binary file: /builds/worker/checkouts/gecko/browser/branding/unofficial/firefox.icns
[task 2018-05-10T21:38:25.321Z] Unable to match regex against output: WARNING: Binary file: /builds/worker/checkouts/gecko/browser/branding/unofficial/newwindow.ico
[task 2018-05-10T21:38:25.321Z] Unable to match regex against output: WARNING: Binary file: /builds/worker/checkouts/gecko/browser/branding/unofficial/document.icns
[task 2018-05-10T21:38:25.321Z] Unable to match regex against output: WARNING: Binary file: /builds/worker/checkouts/gecko/browser/branding/unofficial/background.png
[task 2018-05-10T21:38:25.321Z] Unable to match regex against output: WARNING: Binary file: /builds/worker/checkouts/gecko/browser/branding/unofficial/wizWatermark.bmp
[task 2018-05-10T21:38:25.321Z] Unable to match regex against output: WARNING: Binary file: /builds/worker/checkouts/gecko/browser/branding/unofficial/default64.png
[task 2018-05-10T21:38:25.322Z] Unable to match regex against output: WARNING: Binary file: /builds/worker/checkouts/gecko/browser/branding/unofficial/default16.png
[task 2018-05-10T21:38:25.322Z] Unable to match regex against output: WARNING: Binary file: /builds/worker/checkouts/gecko/browser/branding/unofficial/disk.icns
[task 2018-05-10T21:38:25.322Z] Unable to match regex against output: WARNING: Binary file: /builds/worker/checkouts/gecko/browser/branding/unofficial/VisualElements_70.png
[task 2018-05-10T21:38:25.322Z] Unable to match regex against output: WARNING: Binary file: /builds/worker/checkouts/gecko/browser/branding/unofficial/document.ico
[task 2018-05-10T21:38:25.322Z] Unable to match regex against output: WARNING: Binary file: /builds/worker/checkouts/gecko/browser/branding/unofficial/newtab.ico
[task 2018-05-10T21:38:25.323Z] Unable to match regex against output: WARNING: Binary file: /builds/worker/checkouts/gecko/browser/branding/unofficial/dsstore
[task 2018-05-10T21:38:25.323Z] Unable to match regex against output: WARNING: Binary file: /builds/worker/checkouts/gecko/browser/branding/unofficial/wizHeader.bmp
[task 2018-05-10T21:38:25.323Z] Unable to match regex against output: WARNING: Binary file: /builds/worker/checkouts/gecko/browser/branding/unofficial/firefox.ico
[task 2018-05-10T21:38:25.323Z] Unable to match regex against output: WARNING: Binary file: /builds/worker/checkouts/gecko/browser/branding/unofficial/wizHeaderRTL.bmp
[task 2018-05-10T21:38:25.323Z] Unable to match regex against output: WARNING: Binary file: /builds/worker/checkouts/gecko/browser/branding/unofficial/content/about-logo@2x.png
[task 2018-05-10T21:38:25.323Z] Unable to match regex against output: WARNING: Binary file: /builds/worker/checkouts/gecko/browser/branding/unofficial/content/about-background.png
[task 2018-05-10T21:38:25.323Z] Unable to match regex against output: WARNING: Binary file: /builds/worker/checkouts/gecko/browser/branding/unofficial/content/about-logo.png
[task 2018-05-10T21:38:25.323Z] Unable to match regex against output: WARNING: Binary file: /builds/worker/checkouts/gecko/browser/branding/unofficial/content/about.png
[task 2018-05-10T21:38:25.339Z] TEST-UNEXPECTED-WARNING | intl/locales/en-US/hyphenation/hyph_en_US.dic:2039 | nd ==> and, 2nd (codespell)
[task 2018-05-10T21:38:25.339Z] TEST-UNEXPECTED-WARNING | intl/locales/en-US/hyphenation/hyph_en_US.dic:11040 | nd ==> and, 2nd (codespell)
[task 2018-05-10T21:38:25.340Z] TEST-UNEXPECTED-WARNING | intl/locales/en-US/hyphenation/hyph_en_US.dic:11098 | nd ==> and, 2nd (codespell)
[taskcluster 2018-05-10 21:38:25.670Z] === Task Finished ===
[taskcluster 2018-05-10 21:38:25.670Z] Unsuccessful task run with exit code: 1 completed in 292.611 seconds
Push with failures:
https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=c2e8fbd72ca649e29403c31c8cee9e36cf83ebd6
Backout:
https://hg.mozilla.org/integration/autoland/rev/5a8eea8a92bbfff1c4b324e1137f73e9956f9643
Comment hidden (mozreview-request) |
Comment hidden (mozreview-request) |
Comment hidden (mozreview-request) |
Comment hidden (mozreview-request) |
Comment 29•7 years ago
|
||
Pushed by sledru@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/c1f425788ae6
Create a new class to manage pip install r=ahal
https://hg.mozilla.org/integration/autoland/rev/b69e58c4e0fd
Update the flake8 support to use the pip class r=ahal
https://hg.mozilla.org/integration/autoland/rev/3f8ad0565db2
Update the codespell support to use pip instead of the package r=ahal
https://hg.mozilla.org/integration/autoland/rev/8def71624ff1
Update the CI to use codespell with pip instead of the apt packages (too old) r=ahal
![]() |
||
Comment 30•7 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/c1f425788ae6
https://hg.mozilla.org/mozilla-central/rev/b69e58c4e0fd
https://hg.mozilla.org/mozilla-central/rev/3f8ad0565db2
https://hg.mozilla.org/mozilla-central/rev/8def71624ff1
Status: NEW → RESOLVED
Closed: 7 years ago
status-firefox62:
--- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla62
Updated•3 years ago
|
Product: Firefox Build System → Developer Infrastructure
You need to log in
before you can comment on or make changes to this bug.
Description
•