Intermittent [tier2] Linux-noopt ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTaskSkip TEST-UNEXPECTED-FAIL

RESOLVED FIXED in Firefox 67

Status

defect
P5
critical
RESOLVED FIXED
4 months ago
4 months ago

People

(Reporter: intermittent-bug-filer, Assigned: kmag)

Tracking

({intermittent-failure, regression})

Version 3
mozilla67
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox-esr60 unaffected, firefox65 unaffected, firefox66 unaffected, firefox67 fixed)

Details

(Whiteboard: [stockwell disable-recommended])

Attachments

(1 attachment)

Reporter

Description

4 months ago
treeherder

#[markdown(off)]
Filed by: nbeleuzu [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=228309329&repo=mozilla-inbound

https://queue.taskcluster.net/v1/task/f0n_X53PTuynnToTfuXFrA/runs/0/artifacts/public/logs/live_backing.log

[task 2019-02-14T03:03:38.118Z] 03:03:38 INFO - /builds/worker/workspace/build/src/testing/xpcshell/selftest.py
[task 2019-02-14T03:03:38.119Z] 03:03:38 INFO - ============================= test session starts ==============================
[task 2019-02-14T03:03:38.119Z] 03:03:38 INFO - platform linux2 -- Python 2.7.9, pytest-3.6.2, py-1.5.4, pluggy-0.6.0 -- /builds/worker/workspace/build/src/obj-firefox/_virtualenvs/obj-firefox-8yIyzR8r-2.7/bin/python
[task 2019-02-14T03:03:38.119Z] 03:03:38 INFO - rootdir: /builds/worker/workspace/build/src, inifile: /builds/worker/workspace/build/src/config/mozunit/mozunit/pytest.ini
[task 2019-02-14T03:03:38.119Z] 03:03:38 INFO - collecting ... collected 55 items
[task 2019-02-14T03:03:38.119Z] 03:03:38 INFO - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTaskRunNextTest PASSED
[task 2019-02-14T03:03:38.119Z] 03:03:38 WARNING - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTaskSkip TEST-UNEXPECTED-FAIL
[task 2019-02-14T03:03:38.120Z] 03:03:38 WARNING - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTaskSkipAll TEST-UNEXPECTED-FAIL
[task 2019-02-14T03:03:38.120Z] 03:03:38 WARNING - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTaskStackTrace TEST-UNEXPECTED-FAIL
[task 2019-02-14T03:03:38.120Z] 03:03:38 INFO - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTaskTestFailureInside PASSED
[task 2019-02-14T03:03:38.120Z] 03:03:38 WARNING - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTaskTestMultiple TEST-UNEXPECTED-FAIL
[task 2019-02-14T03:03:38.120Z] 03:03:38 INFO - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTaskTestRejected PASSED
[task 2019-02-14T03:03:38.120Z] 03:03:38 WARNING - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTaskTestSingle TEST-UNEXPECTED-FAIL
[task 2019-02-14T03:03:38.121Z] 03:03:38 INFO - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTestFailing PASSED

Blocks: 1524688
Severity: normal → critical
Flags: needinfo?(kmaglione+bmo)
Summary: Intermittent ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTaskSkip TEST-UNEXPECTED-FAIL → Perma Linux-noopt ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTaskSkip TEST-UNEXPECTED-FAIL

Updated

4 months ago
Duplicate of this bug: 1528258

Updated

4 months ago
Duplicate of this bug: 1528208

Updated

4 months ago
Duplicate of this bug: 1528185
Comment hidden (Intermittent Failures Robot)
Comment hidden (Intermittent Failures Robot)
Whiteboard: [stockwell needswork:owner]
Comment hidden (Intermittent Failures Robot)
Comment hidden (Intermittent Failures Robot)
Comment hidden (Intermittent Failures Robot)

There are 182 total failures in the last 7 days, all on linux64-noopt debug.

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=228727581&repo=mozilla-inbound&lineNumber=39488

[task 2019-02-15T20:50:04.256Z] 20:50:04 INFO - /builds/worker/workspace/build/src/testing/xpcshell/selftest.py
[task 2019-02-15T20:50:04.257Z] 20:50:04 INFO - ============================= test session starts ==============================
[task 2019-02-15T20:50:04.257Z] 20:50:04 INFO - platform linux2 -- Python 2.7.9, pytest-3.6.2, py-1.5.4, pluggy-0.6.0 -- /builds/worker/workspace/build/src/obj-firefox/_virtualenvs/obj-firefox-8yIyzR8r-2.7/bin/python
[task 2019-02-15T20:50:04.257Z] 20:50:04 INFO - rootdir: /builds/worker/workspace/build/src, inifile: /builds/worker/workspace/build/src/config/mozunit/mozunit/pytest.ini
[task 2019-02-15T20:50:04.257Z] 20:50:04 INFO - collecting ... collected 55 items
[task 2019-02-15T20:50:04.257Z] 20:50:04 INFO - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTaskRunNextTest PASSED
[task 2019-02-15T20:50:04.258Z] 20:50:04 WARNING - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTaskSkip TEST-UNEXPECTED-FAIL
[task 2019-02-15T20:50:04.258Z] 20:50:04 WARNING - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTaskSkipAll TEST-UNEXPECTED-FAIL
[task 2019-02-15T20:50:04.258Z] 20:50:04 WARNING - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTaskStackTrace TEST-UNEXPECTED-FAIL
[task 2019-02-15T20:50:04.258Z] 20:50:04 INFO - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTaskTestFailureInside PASSED
[task 2019-02-15T20:50:04.258Z] 20:50:04 WARNING - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTaskTestMultiple TEST-UNEXPECTED-FAIL
[task 2019-02-15T20:50:04.259Z] 20:50:04 INFO - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTaskTestRejected PASSED
[task 2019-02-15T20:50:04.259Z] 20:50:04 WARNING - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTaskTestSingle TEST-UNEXPECTED-FAIL
[task 2019-02-15T20:50:04.259Z] 20:50:04 INFO - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTestFailing PASSED
[task 2019-02-15T20:50:04.259Z] 20:50:04 WARNING - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTestSimple TEST-UNEXPECTED-FAIL
[task 2019-02-15T20:50:04.260Z] 20:50:04 INFO - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTestUncaughtRejection PASSED
[task 2019-02-15T20:50:04.260Z] 20:50:04 INFO - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTestUncaughtRejectionJSM PASSED
[task 2019-02-15T20:50:04.260Z] 20:50:04 WARNING - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAssertStack TEST-UNEXPECTED-FAIL
[task 2019-02-15T20:50:04.260Z] 20:50:04 WARNING - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAsyncCleanup TEST-UNEXPECTED-FAIL
[task 2019-02-15T20:50:04.261Z] 20:50:04 WARNING - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testChild TEST-UNEXPECTED-FAIL
[task 2019-02-15T20:50:04.261Z] 20:50:04 WARNING - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testChildFail TEST-UNEXPECTED-FAIL
[task 2019-02-15T20:50:04.261Z] 20:50:04 WARNING - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testChildHang TEST-UNEXPECTED-FAIL
[task 2019-02-15T20:50:04.261Z] 20:50:04 WARNING - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testChildMozinfo TEST-UNEXPECTED-FAIL
[task 2019-02-15T20:50:04.262Z] 20:50:04 WARNING - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testChildPass TEST-UNEXPECTED-FAIL
[task 2019-02-15T20:50:04.262Z] 20:50:04 WARNING - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testDoPrintWhenVerboseExplicit TEST-UNEXPECTED-FAIL
[task 2019-02-15T20:50:04.262Z] 20:50:04 WARNING - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testDoPrintWhenVerboseInManifest TEST-UNEXPECTED-FAIL
[task 2019-02-15T20:50:04.262Z] 20:50:04 WARNING - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testDoPrintWhenVerboseNotExplicit TEST-UNEXPECTED-FAIL
[task 2019-02-15T20:50:04.263Z] 20:50:04 INFO - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testDoReportForeignObject PASSED
[task 2019-02-15T20:50:04.263Z] 20:50:04 INFO - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testDoReportNonSyntaxError PASSED
[task 2019-02-15T20:50:04.263Z] 20:50:04 INFO - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testDoReportRefError PASSED
[task 2019-02-15T20:50:04.263Z] 20:50:04 INFO - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testDoReportSyntaxError PASSED
[task 2019-02-15T20:50:04.264Z] 20:50:04 INFO - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testDoThrowForeignObject PASSED
[task 2019-02-15T20:50:04.264Z] 20:50:04 INFO - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testDoThrowString PASSED
[task 2019-02-15T20:50:04.264Z] 20:50:04 INFO - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testFail PASSED
[task 2019-02-15T20:50:04.264Z] 20:50:04 INFO - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testHangingTimeout <- ../../../../../usr/lib/python2.7/unittest/case.py SKIPPED
[task 2019-02-15T20:50:04.264Z] 20:50:04 INFO - ../testing/xpcshell/selftest.py::XPCShellTestsTests::testKnownFail PASSED

Kris can you please take a look?

Flags: needinfo?(kmaglione+bmo)
Flags: needinfo?(kmaglione+bmo)
Duplicate of this bug: 1529200
Duplicate of this bug: 1528923
Duplicate of this bug: 1528847
Duplicate of this bug: 1528924
Duplicate of this bug: 1528925
Duplicate of this bug: 1528922
Duplicate of this bug: 1528572
Duplicate of this bug: 1528991
Duplicate of this bug: 1529201
Comment hidden (Intermittent Failures Robot)
Assignee

Comment 20

4 months ago

I'm still looking into it, but since I don't get a crash stack (or any other kind of failure info), and I can't reproduce it locally or on try, I don't have much to go on. I was hoping it would be the same issue as the Windows PGO crashes, but it appears not to be.

If I could figure out how to disable a test only for noopt builds, I would do that in the meantime, but I can't...

Flags: needinfo?(kmaglione+bmo)
Summary: Perma Linux-noopt ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTaskSkip TEST-UNEXPECTED-FAIL → Intermittent Linux-noopt ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTaskSkip TEST-UNEXPECTED-FAIL
Comment hidden (Intermittent Failures Robot)
Assignee

Comment 22

4 months ago

The watchdog thread's stack is intentionally kept as small as possible, since
the work that it does is trivial and should not require much stack space.
One result of this is that the stack segment is generally too small to
instantiate a new service (and we really don't want to be instantiating
services on it anyway).

The debug service is generally instantiated before the watchdog thread tries
to touch it, but this isn't guaranteed (and, in particular, it often isn't for
xpcshell runs on some noopt debug builds). Instantiating it before starting
the watchdog thread solves this problem.

Assignee

Updated

4 months ago
Assignee: nobody → kmaglione+bmo
Summary: Intermittent Linux-noopt ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTaskSkip TEST-UNEXPECTED-FAIL → Intermittent [tier2] Linux-noopt ../testing/xpcshell/selftest.py::XPCShellTestsTests::testAddTaskSkip TEST-UNEXPECTED-FAIL
Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]
Assignee

Comment 23

4 months ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/965d985aeda572d16800fa301ca9fee410cb2fe7
Bug 1527803: Make sure debug service is instantiated before starting Watchdog thread. r=erahm

Comment 24

4 months ago
bugherder
Status: NEW → RESOLVED
Closed: 4 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla67
Comment hidden (Intermittent Failures Robot)
Comment hidden (Intermittent Failures Robot)
You need to log in before you can comment on or make changes to this bug.