1434337 - Enforce new linter requirements via a 'lintcheck' linter

Status: NEW

(Developer Infrastructure :: Lint and Formatting, enhancement, P3)

Description

[Andrew Halberstadt [:ahal]]

This bug only tracks documentation and policy change, though it might be a good idea to enforce some of the policy via CI in follow-up bugs.

I've been a bit too lax when reviewing new linters, and it has caused a fair bit of frustration for developers. Starting now, all new linters should meet the following guidelines before being r+ed:

1. They need to pass everywhere (this was technically already true, but just stating it again for completeness). This means ./mach lint -l <linter> needs to return 0 errors.

2. They need to run in CI. If lint failures don't cause backouts, developers will never know whether lint errors are their problem or not.

3. They need to be automatically bootstrapped. Developers run the linters with vcs hooks and seeing weird error messages about random binaries not being installed is confusing.

4. They need to be mentioned in documentation somewhere. Maybe a full page like eslint/flake8 have is unnecessary, but we should at least have a page that lists linters and provides a brief description of each.

5. They need to have tests (for the future). None of the wrapper code is tested atm, so let's defer this until there is a framework new linters can copy.

Comment 1

[Andrew Halberstadt [:ahal]]

Looks like :Standard8 took care of creating some docs in bug 1434332. I'm going to morph this one into creating a "linter" linter. Let's call it 'lintcheck'.

This lintcheck could make sure that all defined linter configs are run in a taskcluster task, have a section in the documentation, have a test directory, etc.