1675806 - Permafailing tools/lint/test/test_perfdocs.py::test_perfdocs_start_and_fail[perfdocs] TEST-UNEXPECTED-FAIL

Status: RESOLVED FIXED

(Developer Infrastructure :: Lint and Formatting, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: rmaries [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=320970975&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/SHFMNeOgQ6KzGKwMN_XxGA/runs/0/artifacts/public/logs/live_backing.log

---

[task 2020-11-06T16:27:11.721Z]  0:46.29 ============================= test session starts =============================
[task 2020-11-06T16:27:11.721Z]  0:46.29 platform win32 -- Python 3.6.5, pytest-3.6.2, py-1.5.4, pluggy-0.6.0 -- Z:\task_1604679179\build\src\obj-x86_64-pc-mingw32\_virtualenvs\python-test\Scripts\python.exe
[task 2020-11-06T16:27:11.721Z]  0:46.29 rootdir: Z:\task_1604679179\build\src, inifile: Z:\task_1604679179\build\src\config\mozunit\mozunit\pytest.ini
[task 2020-11-06T16:27:11.721Z]  0:46.29 collecting ... collected 4 items
[task 2020-11-06T16:27:11.721Z]  0:46.29 
[task 2020-11-06T16:27:11.721Z]  0:46.29 tools/lint/test/test_perfdocs.py::test_perfdocs_start_and_fail[perfdocs] TEST-UNEXPECTED-FAIL
[task 2020-11-06T16:27:11.721Z]  0:46.29 tools/lint/test/test_perfdocs.py::test_perfdocs_start_and_pass[perfdocs] TEST-UNEXPECTED-FAIL
[task 2020-11-06T16:27:11.721Z]  0:46.29 tools/lint/test/test_perfdocs.py::test_perfdocs_bad_paths[perfdocs] PASSED
[task 2020-11-06T16:27:11.721Z]  0:46.29 tools/lint/test/test_perfdocs.py::test_perfdocs_logger_failure[perfdocs] PASSED
[task 2020-11-06T16:27:11.721Z]  0:46.29 
[task 2020-11-06T16:27:11.721Z]  0:46.29 ================================== FAILURES ===================================
[task 2020-11-06T16:27:11.721Z]  0:46.29 ___________________ test_perfdocs_start_and_fail[perfdocs] ____________________
[task 2020-11-06T16:27:11.722Z]  0:46.29 
[task 2020-11-06T16:27:11.722Z]  0:46.30 generator = <MagicMock name='Verifier' id='2260493580104'>
[task 2020-11-06T16:27:11.722Z]  0:46.30 verifier = <MagicMock name='Generator' id='2260620603744'>
[task 2020-11-06T16:27:11.722Z]  0:46.30 structured_logger = <mozlog.structuredlog.StructuredLogger object at 0x0000020E4FEAB3C8>
[task 2020-11-06T16:27:11.722Z]  0:46.30 config = {'description': 'Performance Documentation linter', 'exclude': [], 'extensions': ['rst', 'ini', 'yml'], 'include': ['testing/raptor', 'python/mozperftest', 'testing/talos'], ...}
[task 2020-11-06T16:27:11.722Z]  0:46.30 paths = <function paths.<locals>._inner at 0x0000020E4FF2F378>
[task 2020-11-06T16:27:11.722Z]  0:46.30 
[task 2020-11-06T16:27:11.722Z]  0:46.30     @mock.patch("perfdocs.generator.Generator")
[task 2020-11-06T16:27:11.722Z]  0:46.30     @mock.patch("perfdocs.verifier.Verifier")
[task 2020-11-06T16:27:11.722Z]  0:46.30     @mock.patch("perfdocs.logger.PerfDocLogger", new=PerfDocsLoggerMock)
[task 2020-11-06T16:27:11.722Z]  0:46.30     def test_perfdocs_start_and_fail(generator, verifier, structured_logger, config, paths):
[task 2020-11-06T16:27:11.722Z]  0:46.30         from perfdocs.perfdocs import run_perfdocs
[task 2020-11-06T16:27:11.722Z]  0:46.30 
[task 2020-11-06T16:27:11.722Z]  0:46.30         with temp_file("bad", "foo") as temp:
[task 2020-11-06T16:27:11.722Z]  0:46.30 >           run_perfdocs(config, logger=structured_logger, paths=[temp], generate=False)
[task 2020-11-06T16:27:11.722Z]  0:46.30 
[task 2020-11-06T16:27:11.722Z]  0:46.30 tools\lint\test\test_perfdocs.py:39:
[task 2020-11-06T16:27:11.722Z]  0:46.30 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _```

Comment 1

[Natalia Csoregi [:nataliaCs]]

Alexandru, could you help us out with this one?
Started once Bug 1611084 has landed.
Thank you.

Comment 2

[Intermittent Failures Robot]

10 failures in 4128 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 8
• mozilla-central: 1
• try: 1

Platform breakdown:

• windows10-64: 10

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1675806&startday=2020-11-02&endday=2020-11-08&tree=all

Comment 3

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Sparky, I did some retriggers on Windows 10 x64 opt py3 mozlint. It's possible that the fingerprint error from your try push covered this mozlint failure.
Could you please look into it?

Comment 4

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1611084

Comment 5

[Greg Mierzwinski [:sparky]]

Attachment: Bug 1675806 - Fix perfdocs unittest on Windows. r?#perftest

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e6867794b570
Fix perfdocs unittest on Windows. r=perftest-reviewers,AlexandruIonescu

Comment 7

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/e6867794b570