1570654 - Linters fail with `outgoing` if a commit series first modifies then moves a file

Status: RESOLVED FIXED

(Developer Infrastructure :: Lint and Formatting, defect, P3)

Description

[Andrew Halberstadt [:ahal]]

Pretty big edge case here, but worth having on file. For example, I have a patch series that modifies taskcluster/taskgraph/optimize.py, then in a later commit moves it such that optimize.py no longer exists.

When running ./mach lint -o, it'll pass optimize.py into the linters anyway since version control will say that it was modified (by the earlier commit). Here's an example of the failures:

[~/dev/mozilla-central](c48cb09)$ ./mach lint -o
Traceback (most recent call last):
  File "/home/ahal/dev/mozilla-central/python/mozlint/mozlint/roller.py", line 39, in _run_worker
    res = func(paths, config, **lintargs) or []
  File "/home/ahal/dev/mozilla-central/python/mozlint/mozlint/types.py", line 55, in __call__
    result = self._lint(p, config, **lintargs)
  File "/home/ahal/dev/mozilla-central/python/mozlint/mozlint/types.py", line 99, in _lint
    with open(path, 'r') as fh:
IOError: [Errno 2] No such file or directory: '/home/ahal/dev/mozilla-central/taskcluster/taskgraph/optimize.py'
Traceback (most recent call last):
  File "/home/ahal/dev/mozilla-central/python/mozlint/mozlint/roller.py", line 39, in _run_worker
    res = func(paths, config, **lintargs) or []
  File "/home/ahal/dev/mozilla-central/python/mozlint/mozlint/types.py", line 55, in __call__
    result = self._lint(p, config, **lintargs)
  File "/home/ahal/dev/mozilla-central/python/mozlint/mozlint/types.py", line 99, in _lint
    with open(path, 'r') as fh:
IOError: [Errno 2] No such file or directory: '/home/ahal/dev/mozilla-central/taskcluster/taskgraph/optimize.py'
Traceback (most recent call last):
  File "/home/ahal/dev/mozilla-central/python/mozlint/mozlint/roller.py", line 39, in _run_worker
    res = func(paths, config, **lintargs) or []
  File "/home/ahal/dev/mozilla-central/python/mozlint/mozlint/types.py", line 50, in __call__
    return self._lint(paths, config, **lintargs)
  File "/home/ahal/dev/mozilla-central/python/mozlint/mozlint/types.py", line 134, in _lint
    return func(files, config, **lintargs)
  File "/home/ahal/dev/mozilla-central/tools/lint/cpp/mingw-capitalization.py", line 37, in lint
    results.extend(m._lint(path, config, **lintargs))
  File "/home/ahal/dev/mozilla-central/python/mozlint/mozlint/types.py", line 99, in _lint
    with open(path, 'r') as fh:
IOError: [Errno 2] No such file or directory: '/home/ahal/dev/mozilla-central/taskcluster/taskgraph/optimize.py'
warning: no files to lint (eslint)
Unable to match yaml regex against output: [Errno 2] No such file or directory: '/home/ahal/dev/mozilla-central/taskcluster/taskgraph/optimize.py'
A failure occurred in the cpp-virtual-final linter.
A failure occurred in the no-comment-disable linter.
A failure occurred in the mingw-capitalization linter.
✖ 3 problems (0 errors, 0 warnings, 3 failures)

I think it would be good enough to filter out non-existent paths after doing the version control lookups.

Comment 1

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1570654 - [mozlint] Filter out non-existent paths in filterpaths, r?#linter-reviewers!

While we validate user specified paths early and only consider the 'AM' (added

• modified) diff filter from versioncontrol, it's still possible to get
  non-existent files when using --outgoing. This situation can happen when you
  have a commit series that first modifies a file in an earlier commit, and then
  moves/deletes it in a later commit.

To get around this, just silently filter out and file paths that don't exist.

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fa369371a9a2
[mozlint] Filter out non-existent paths in filterpaths, r=linter-reviewers,sylvestre

Comment 3

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/fa369371a9a2