1811850 - Replace all our Python linters with ruff

Status: RESOLVED FIXED

(Developer Infrastructure :: Lint and Formatting, task)

Description

[Andrew Halberstadt [:ahal]]

Ruff is a newish tool that's making some waves in the Python community. It's written in Rust and can replace a huge number of existing Python tooling. It has some pretty compelling selling points:

Ruff can be used to replace Flake8 (plus a variety of plugins), isort, pydocstyle, yesqa, eradicate, pyupgrade, and autoflake, all while executing tens or hundreds of times faster than any individual tool.

Autofix support, for automatic error correction

Would solve our clunky autopep8 acting as flake8's "fixer" problem

10-100x faster than existing linters

Our Python linting story is a bit of a mess, and this tool looks like a promising way to collapse all our rules into a single speedy linter. It even supports flake8's per-file-ignores feature, which means we'd have that capability for things like isort and pyupgrade as well!

Comment 1

[Sylvestre Ledru [:Sylvestre]]

ah ah, I was opening a bug for this :)
More info:
https://notes.crmarsh.com/ruff-the-first-200-releases

Comment 2

[Andrew Halberstadt [:ahal]]

Since filing this I've used ruff in a few projects and have spent a decent amount of time looking at the docs. I'm convinced this is something we should do. I'm going to see how far I get here without putting too much effort in.

Comment 3

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1811850 - [lint] Replace flake8 linter with ruff, r?#linter-reviewers!

Comment 4

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1811850 - [lint] Replace isort linter with ruff, r?#linter-reviewers!

Depends on D172313

Comment 5

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1811850 - [lint] Replace pylint linter with ruff, r?#linter-reviewers!

This is enabling the PLE and PLC rules across a much larger swatch of the code
base. The PLR and PLW are additionally enabled at the warning level (introduces
~2500 new warnings).

To avoid extraneous changes in people's patches, rules at the warning level are
excluded from --fix.

Depends on D172348

Comment 6

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1811850 - [lint] Fix ruff errors resulting from new configuration, r?#linter-reviewers!

Depends on D172358

Comment 7

[Andrew Halberstadt [:ahal]]

Attachment: WIP: Bug 1811850 - [lint] Enable pyupgrade rules in ruff, r#linter-reviewers

Depends on D172359

Comment 8

[Andrew Halberstadt [:ahal]]

Attachment: WIP: Bug 1811850 - [lint] Fix pyupgrade lint errors from new configuration, r#linter-reviewers

Depends on D172360

Comment 9

[Phabricator Automation]

Comment on attachment 9322565 [details]
WIP: Bug 1811850 - [lint] Enable pyupgrade rules in ruff, r#linter-reviewers

Revision D172360 was moved to bug 1715287. Setting attachment 9322565 [details] to obsolete.

Comment 10

[Phabricator Automation]

Comment on attachment 9322566 [details]
WIP: Bug 1811850 - [lint] Fix pyupgrade lint errors from new configuration, r#linter-reviewers

Revision D172361 was moved to bug 1715287. Setting attachment 9322566 [details] to obsolete.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c3b0f9666183
[lint] Replace flake8 linter with ruff, r=linter-reviewers,sylvestre
https://hg.mozilla.org/integration/autoland/rev/d96f90c2c58b
[lint] Replace isort linter with ruff, r=taskgraph-reviewers,linter-reviewers,marco,jcristau
https://hg.mozilla.org/integration/autoland/rev/f650123cc188
[lint] Replace pylint linter with ruff, r=linter-reviewers,marco
https://hg.mozilla.org/integration/autoland/rev/f8950d716c9e
[lint] Fix ruff errors resulting from new configuration, r=webdriver-reviewers,perftest-reviewers,whimboo,sparky,firefox-build-system-reviewers,ahochheiden
https://hg.mozilla.org/integration/autoland/rev/e8fcfc7f8108
apply code formatting via Lando

Comment 12

[Cristian Tuns]

Backed out for causing linting bustages(bugzilla)

• Backout link
• Push with failures
• Failure Log
• Failure line: FileToBugzillaMappingError: Missing Bugzilla component: pyproject.toml - Set the BUG_COMPONENT in the moz.build file to fix the issue.

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5dfcd5a1b685
[lint] Replace flake8 linter with ruff, r=linter-reviewers,sylvestre
https://hg.mozilla.org/integration/autoland/rev/1f236421121c
[lint] Replace isort linter with ruff, r=taskgraph-reviewers,linter-reviewers,marco,jcristau
https://hg.mozilla.org/integration/autoland/rev/75ff6a88978b
[lint] Replace pylint linter with ruff, r=linter-reviewers,marco
https://hg.mozilla.org/integration/autoland/rev/bd79cf7201c7
[lint] Fix ruff errors resulting from new configuration, r=webdriver-reviewers,perftest-reviewers,whimboo,sparky,firefox-build-system-reviewers,ahochheiden

Comment 14

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/5dfcd5a1b685
https://hg.mozilla.org/mozilla-central/rev/1f236421121c
https://hg.mozilla.org/mozilla-central/rev/75ff6a88978b
https://hg.mozilla.org/mozilla-central/rev/bd79cf7201c7

Comment 15

[Kagami Rosylight [:saschanaz] (they/them)]

This did not replace the formatter we use (black), do we want to replace it too?

Comment 16

[Andrew Halberstadt [:ahal]]

Yeah, at the time this was filed Ruff's formatter wasn't ready. But AIUI it is now more or less a drop-in replacement for Black as well. I'm on board with switching, but let's file a new bug.