1691558 - Intermittent 0:18.44 tools/lint/test/test_perfdocs.py::test_perfdocs_gatherer_fetch_perfdocs_tree[perfdocs] TEST-UNEXPECTED-FAIL

Status: RESOLVED FIXED

(Testing :: Performance, defect, P5)

Description

Filed by: nbeleuzu [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=329260590&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Boj47E6hRM-Lp4-frtqoCg/runs/0/artifacts/public/logs/live_backing.log

---

[task 2021-02-08T19:10:00.765Z]  0:18.44 platform darwin -- Python 3.7.4, pytest-3.6.2, py-1.5.4, pluggy-0.6.0 -- /Users/cltbld/tasks/task_1612811089/checkouts/gecko/obj-x86_64-apple-darwin18.6.0/_virtualenvs/python-test/bin/python
[task 2021-02-08T19:10:00.765Z]  0:18.44 rootdir: /Users/cltbld/tasks/task_1612811089/checkouts/gecko, inifile: /Users/cltbld/tasks/task_1612811089/checkouts/gecko/config/mozunit/mozunit/pytest.ini
[task 2021-02-08T19:10:00.766Z]  0:18.44 collecting ... collected 19 items
[task 2021-02-08T19:10:00.766Z]  0:18.44 
[task 2021-02-08T19:10:00.766Z]  0:18.44 tools/lint/test/test_perfdocs.py::test_perfdocs_start_and_fail[perfdocs] PASSED
[task 2021-02-08T19:10:00.766Z]  0:18.44 tools/lint/test/test_perfdocs.py::test_perfdocs_start_and_pass[perfdocs] PASSED
[task 2021-02-08T19:10:00.766Z]  0:18.44 tools/lint/test/test_perfdocs.py::test_perfdocs_bad_paths[perfdocs] PASSED
[task 2021-02-08T19:10:00.766Z]  0:18.44 tools/lint/test/test_perfdocs.py::test_perfdocs_gatherer_fetch_perfdocs_tree[perfdocs] TEST-UNEXPECTED-FAIL
[task 2021-02-08T19:10:00.766Z]  0:18.44 tools/lint/test/test_perfdocs.py::test_perfdocs_gatherer_get_test_list[perfdocs] TEST-UNEXPECTED-FAIL
[task 2021-02-08T19:10:00.766Z]  0:18.44 tools/lint/test/test_perfdocs.py::test_perfdocs_verification[perfdocs] TEST-UNEXPECTED-FAIL
[task 2021-02-08T19:10:00.766Z]  0:18.44 tools/lint/test/test_perfdocs.py::test_perfdocs_verifier_validate_yaml_pass[perfdocs] PASSED
[task 2021-02-08T19:10:00.766Z]  0:18.44 tools/lint/test/test_perfdocs.py::test_perfdocs_verifier_invalid_yaml[perfdocs] PASSED
[task 2021-02-08T19:10:00.766Z]  0:18.44 tools/lint/test/test_perfdocs.py::test_perfdocs_verifier_validate_rst_pass[perfdocs] PASSED
[task 2021-02-08T19:10:00.766Z]  0:18.44 tools/lint/test/test_perfdocs.py::test_perfdocs_verifier_invalid_rst[perfdocs] PASSED
[task 2021-02-08T19:10:00.766Z]  0:18.44 tools/lint/test/test_perfdocs.py::test_perfdocs_verifier_validate_descriptions_pass[perfdocs] TEST-UNEXPECTED-FAIL
[task 2021-02-08T19:10:00.766Z]  0:18.44 tools/lint/test/test_perfdocs.py::test_perfdocs_verifier_not_existing_suite_in_test_list[perfdocs] TEST-UNEXPECTED-FAIL
[task 2021-02-08T19:10:00.766Z]  0:18.44 tools/lint/test/test_perfdocs.py::test_perfdocs_verifier_not_existing_tests_in_suites[perfdocs] TEST-UNEXPECTED-FAIL
[task 2021-02-08T19:10:00.767Z]  0:18.44 tools/lint/test/test_perfdocs.py::test_perfdocs_verifier_missing_contents_in_suite[perfdocs] TEST-UNEXPECTED-FAIL
[task 2021-02-08T19:10:00.767Z]  0:18.44 tools/lint/test/test_perfdocs.py::test_perfdocs_verifier_invalid_dir[perfdocs] PASSED
[task 2021-02-08T19:10:00.767Z]  0:18.44 tools/lint/test/test_perfdocs.py::test_perfdocs_verifier_file_invalidation[perfdocs] TEST-UNEXPECTED-FAIL
[task 2021-02-08T19:10:00.767Z]  0:18.44 tools/lint/test/test_perfdocs.py::test_perfdocs_framework_gatherers[perfdocs] PASSED
[task 2021-02-08T19:10:00.767Z]  0:18.44 tools/lint/test/test_perfdocs.py::test_perfdocs_framework_gatherers_urls[perfdocs] TEST-UNEXPECTED-FAIL
[task 2021-02-08T19:10:00.767Z]  0:18.44 tools/lint/test/test_perfdocs.py::test_perfdocs_logger_failure[perfdocs] PASSED```

Comment 1

[Greg Mierzwinski [:sparky]]

Moving to performance since we are the owners of perfdocs.

Comment 2

[Intermittent Failures Robot]

4 failures in 4322 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 4

Platform and build breakdown:

• macosx1014-64: 4
  • opt: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1691558&startday=2021-02-08&endday=2021-02-14&tree=all

Comment 6

[Greg Mierzwinski [:sparky]]

Attachment: Bug 1691558 - Disable perfdocs linting tests on windows and mac. r?#perftest

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4c90462c1a56
Disable perfdocs linting tests on windows and mac. r=sylvestre

Comment 8

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/4c90462c1a56

Comment 9

[Intermittent Failures Robot]

1 failures in 3737 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• macosx1014-64: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1691558&startday=2021-02-15&endday=2021-02-21&tree=all

Comment 10

[Myeongjun Go]

Attachment: Bug 1691558 - Strengthen a log to confirm that the file will be searched r=sparky

Comment 11

[Intermittent Failures Robot]

1 failures in 3683 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• linux1804-64: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1691558&startday=2021-03-01&endday=2021-03-07&tree=all

Comment 12

[Intermittent Failures Robot]

1 failures in 4622 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• linux1804-64: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1691558&startday=2021-03-08&endday=2021-03-14&tree=all

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/866067c51ed6
Strengthen a log to confirm that the file will be searched r=sparky

Comment 14

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/866067c51ed6

Comment 15

[Intermittent Failures Robot]

1 failures in 3967 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• macosx1014-64: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1691558&startday=2021-03-22&endday=2021-03-28&tree=all

Comment 16

[Intermittent Failures Robot]

4 failures in 3125 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 4

Platform and build breakdown:

• windows10-64: 1
  • opt: 1
• linux1804-64: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1691558&startday=2021-03-29&endday=2021-04-04&tree=all

Comment 17

[Alexandru Michis [:malexandru]]

sparky, there has been a recent appeareance of this bug, is it ok to reopen it?

Comment 18

[Myeongjun Go]

Hello Alexandru, Thanks for letting me know :)

Sparky, Could you re-open it?
I found the reason why It happens Intermittent exceptions.

When the test is started, It makes a temp directory using temp-file library.
Sometimes temp directory path includes hg path [0].

And then, We have 3 items to exclude path [1], when It tries to find perfdocs directory during making perfdocs.
The problem is here. [2]
dot(.hg ) is recognized to regex-metacharacter while re.search is working.
dot meaning is to find the any character.

Therefore, If the path includes the hg [0], It can't find the target directory and file because of continue logic.

[0]
/var/folders/cf/0jwlx5tx3x7_6dsrx38vbslm000017/T/tmp4fhgl5cj/perfdocs/config.yml

[1]
https://searchfox.org/mozilla-central/source/tools/lint/perfdocs/gatherer.py#77-79

[2]
https://searchfox.org/mozilla-central/source/tools/lint/perfdocs/gatherer.py#84-85

Comment 19

[Greg Mierzwinski [:sparky]]

Yup definitely! I thought this was a stale failure.

Comment 20

[Myeongjun Go]

Attachment: Bug 1691558 - Using an escape character to find the exact path r=sparky

Comment 21

[Intermittent Failures Robot]

3 failures in 4255 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• mozilla-central: 2

Platform and build breakdown:

• macosx1014-64: 2
  • opt: 2
• windows10-64: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1691558&startday=2021-04-12&endday=2021-04-18&tree=all

Comment 22

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e88b02f0a476
Using an escape character to find the exact path r=sparky

Comment 23

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/e88b02f0a476