Closed Bug 1517686 Opened 2 years ago Closed 3 months ago

Intermittent ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTaskSkip TEST-UNEXPECTED-FAIL

Categories

(Testing :: XPCShell Harness, defect, P5)

Version 3
defect

Tracking

(firefox71 affected)

RESOLVED INCOMPLETE
mozilla71
Tracking Status
firefox71 --- affected

People

(Reporter: intermittent-bug-filer, Unassigned)

References

Details

(Keywords: intermittent-failure, Whiteboard: [stockwell unknown])

Attachments

(1 file)

Filed by: apavel [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=219911905&repo=mozilla-inbound

https://queue.taskcluster.net/v1/task/YHDd6jV2SGCye-1T2zAQ1A/runs/0/artifacts/public/logs/live_backing.log

04:41:46     INFO - z:\build\build\src\build\compare-mozconfig\compare-mozconfigs.py
04:41:46     INFO - ============================= test session starts =============================
04:41:46     INFO - platform win32 -- Python 2.7.15, pytest-3.6.2, py-1.5.4, pluggy-0.6.0 -- z:\build\build\src\obj-firefox\_virtualenvs\obj-firefox-sNC2NmI7-2.7\Scripts\python.exe
04:41:46     INFO - rootdir: z:\build\build\src, inifile: z:\build\build\src\config\mozunit\mozunit\pytest.ini
04:41:46     INFO - collecting ... collected 1 item
04:41:46     INFO - ..\build\compare-mozconfig\compare-mozconfigs.py::TestCompareMozconfigs::test_compare_mozconfigs PASSED
04:41:46     INFO - ========================== 1 passed in 0.05 seconds ===========================
04:42:24     INFO - z:\build\build\src\testing\xpcshell\selftest.py
04:42:24     INFO - ============================= test session starts =============================
04:42:24     INFO - platform win32 -- Python 2.7.15, pytest-3.6.2, py-1.5.4, pluggy-0.6.0 -- z:\build\build\src\obj-firefox\_virtualenvs\obj-firefox-sNC2NmI7-2.7\Scripts\python.exe
04:42:24     INFO - rootdir: z:\build\build\src, inifile: z:\build\build\src\config\mozunit\mozunit\pytest.ini
04:42:24     INFO - collecting ... collected 55 items
04:42:24     INFO - ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTaskRunNextTest PASSED
04:42:24  WARNING - ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTaskSkip TEST-UNEXPECTED-FAIL
04:42:24     INFO - ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTaskSkipAll PASSED
04:42:24     INFO - ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTaskStackTrace PASSED
04:42:24     INFO - ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTaskTestFailureInside PASSED
04:42:24     INFO - ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTaskTestMultiple PASSED
04:42:24     INFO - ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTaskTestRejected PASSED
04:42:24     INFO - ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTaskTestSingle PASSED
04:42:24     INFO - ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTestFailing PASSED
04:42:24     INFO - ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTestSimple PASSED
04:42:24     INFO - ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTestUncaughtRejection PASSED
04:42:24     INFO - ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTestUncaughtRejectionJSM PASSED
04:42:24     INFO - ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAssertStack <- c:\mozilla-build\python\Lib\unittest\case.py SKIPPED
04:42:24     INFO - ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAsyncCleanup PASSED
04:42:24     INFO - ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testChild PASSED

This is an intermittent failure in selftest.py's testAddTaskSkip on windows2012-64/debug|asan. A very similar failure pattern is seen in some other selftest.py tests, always on Windows.

See Also: → 1494633, 1584100
Duplicate of this bug: 1446304

Needs work based on duplicate, and see-also's.

I have not been able to reproduce on try and do not have a clear plan.

Assignee: nobody → gbrown
Whiteboard: [stockwell needswork:owner]

selftest.py runs xpcshell with a timeout of 15 seconds -- much less than the timeout used in the xpcshell tests task. Most (all?) of the failures here are 15 second timeouts, and occur on asan and debug but not opt. I will try increasing the selftest.py timeout, hoping that the test just needs a little more time to complete.

Increase the selftest.py timeout from 15 to 30 seconds in hopes of avoiding
intermittent xpcshell timeouts seen only in selftest on slower platforms.

Pushed by gbrown@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/d204b081a5af
Increase xpcshell harness timeout for selftest.py; r=bc
Status: NEW → RESOLVED
Closed: 10 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla71

That didn't work:

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=270814993&repo=autoland&lineNumber=45983-45984

[task 2019-10-11T05:26:47.977Z] 05:26:47 INFO - check> E SUITE-START | Running 1 tests
[task 2019-10-11T05:26:47.977Z] 05:26:47 INFO - check> E profile dir is c:\users\task_1570758064\appdata\local\temp\xpcshell\xpcshellprofile
[task 2019-10-11T05:26:47.977Z] 05:26:47 INFO - check> E TEST-START | test_tasks_skip.js
[task 2019-10-11T05:26:47.977Z] 05:26:47 INFO - check> E TEST-UNEXPECTED-TIMEOUT | test_tasks_skip.js | Test timed out
[task 2019-10-11T05:26:47.977Z] 05:26:47 INFO - check> E TEST-INFO took 30000ms
[task 2019-10-11T05:26:47.978Z] 05:26:47 INFO - check> E xpcshell return code: 1

Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Status: REOPENED → RESOLVED
Closed: 10 months ago9 months ago
Resolution: --- → INCOMPLETE

Yeah, okay, we can try that. I think the increase in timeout lessened the severity of this type of failure. Comment 48 is the only known counter-example.

Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
Assignee: gbrown → nobody
Status: REOPENED → RESOLVED
Closed: 9 months ago7 months ago
Resolution: --- → INCOMPLETE

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=286287250&repo=autoland&lineNumber=118549

[task 2020-01-24T05:47:35.025Z] 05:47:35 INFO - check> z:\build\build\src\testing\xpcshell\selftest.py
[task 2020-01-24T05:47:35.026Z] 05:47:35 INFO - check> ============================= test session starts =============================
[task 2020-01-24T05:47:35.026Z] 05:47:35 INFO - check> platform win32 -- Python 2.7.15, pytest-3.6.2, py-1.5.4, pluggy-0.6.0 -- z:\build\build\src\obj-firefox_virtualenvs\obj-firefox-sNC2NmI7-2.7\Scripts\python.exe
[task 2020-01-24T05:47:35.026Z] 05:47:35 INFO - check> rootdir: z:\build\build\src, inifile: z:\build\build\src\config\mozunit\mozunit\pytest.ini
[task 2020-01-24T05:47:35.026Z] 05:47:35 INFO - check> collecting ... collected 56 items
[task 2020-01-24T05:47:35.026Z] 05:47:35 INFO - check>
[task 2020-01-24T05:47:35.026Z] 05:47:35 INFO - check> ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTaskRunNextTest PASSED
[task 2020-01-24T05:47:35.026Z] 05:47:35 INFO - check> ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTaskSkip TEST-UNEXPECTED-FAIL
[task 2020-01-24T05:47:35.026Z] 05:47:35 INFO - check> ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTaskSkipAll PASSED
[task 2020-01-24T05:47:35.027Z] 05:47:35 INFO - check> ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTaskStackTrace PASSED
[task 2020-01-24T05:47:35.027Z] 05:47:35 INFO - check> ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTaskTestFailureInside PASSED
[task 2020-01-24T05:47:35.027Z] 05:47:35 INFO - check> ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTaskTestMultiple PASSED
[task 2020-01-24T05:47:35.027Z] 05:47:35 INFO - check> ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTaskTestRejected PASSED
[task 2020-01-24T05:47:35.027Z] 05:47:35 INFO - check> ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTaskTestSingle PASSED
[task 2020-01-24T05:47:35.027Z] 05:47:35 INFO - check> ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTestFailing PASSED
[task 2020-01-24T05:47:35.027Z] 05:47:35 INFO - check> ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTestSimple PASSED
[task 2020-01-24T05:47:35.027Z] 05:47:35 INFO - check> ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTestUncaughtRejection PASSED
[task 2020-01-24T05:47:35.028Z] 05:47:35 INFO - check> ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAddTestUncaughtRejectionJSM PASSED
[task 2020-01-24T05:47:35.028Z] 05:47:35 INFO - check> ..\testing\xpcshell\selftest.py::XPCShellTestsTests::testAssertStack <- c:\mozilla-build\python\Lib\unittest\case.py SKIPPED

Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
Status: REOPENED → RESOLVED
Closed: 7 months ago6 months ago
Resolution: --- → INCOMPLETE
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
Status: REOPENED → RESOLVED
Closed: 6 months ago3 months ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.