1398765 - py2/py3 linters don't exclude directories properly

Status: RESOLVED FIXED

(Developer Infrastructure :: Lint and Formatting, enhancement)

Description

[Andrew Halberstadt [:ahal]]

Currently the py2 and py3 linters use a big blacklist to exclude directories. For example, testing/mochitest is excluded, and if you run any of:

./mach lint -l py2 .
./mach lint -l py2 testing/mochitest

The exclusion is properly detected and the linter passes. But if you run:

./mach lint -l py2 testing

The exclusion is not picked up and mochitest files will be linted (and fail). This is happening because in FileFinder, the base is '/path/to/src/testing' but the exclusion is 'testing/mochitest'. This means we ultimately end up calling:

mozpack.match('mochitest/runtests.py', 'testing/mochitest')

This doesn't match up. To fix this, we need to preprocess the exclusions so they are relative to whatever FileFinder.base is (and discard exclusions that are outside of the base). I suspect this is something that would ideally be handled in FileFinder itself, but I'm afraid to change too much there as a ton of stuff depends on it. Plus solving it generically will involve thinking about a whole bunch of edge cases.

I'd rather just solve this in the compat linter for now.

Comment 1

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1398765 - Fix directory exclusion bug in py-compat linters,

The excluded directories aren't being properly handled in the py2/py3 compat
linters. In order for FileFinder to apply the exclusions properly they need
to either be relative to or contained by the base.

This means that currently the following will work:
./mach lint -l py2 <topsrcdir>
./mach lint -l py2 testing/mochitest

But this is broken:
./mach lint -l py2 testing

This change fixes the compat linters so exclude paths will be made relative
to the FileFinder base before passing them in. Any exclude not contained by
the base is simply discarded as it won't be relevant to that FileFinder
instance anyway.

Review commit: https://reviewboard.mozilla.org/r/178392/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/178392/

Comment 2

[Gregory Szorc [:gps]]

Comment on attachment 8906684 [details]
Bug 1398765 - Fix directory exclusion bug in py-compat linters,

https://reviewboard.mozilla.org/r/178392/#review183954

LGTM.

There may be a pre-existing bug in here where a file can occur multiple times in ``files`` due to matching multiple patterns. But that's for another patch.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/237a3f842dc1
Fix directory exclusion bug in py-compat linters, r=gps

Comment 4

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/237a3f842dc1