test_collapse.html is perma-fail but the mochitest summary shows 0 fails

NEW
Unassigned

Status

defect
4 years ago
4 years ago

People

(Reporter: Ehsan, Unassigned)

Tracking

(Blocks 1 bug)

Trunk
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

()

This test is currently run as part of ICS Emulator B2G Debug runs in mochitest-11.  It fails but as you can see in <http://ftp.mozilla.org/pub/mozilla.org/b2g/try-builds/eakhgari@mozilla.com-67d305caa2a8/try-emulator-debug/try_ubuntu64_vm-b2g-emulator-debug_test-mochitest-debug-11-bm53-tests1-linux64-build314.txt.gz> for example, it doesn't cause the tree to go orange.
This is probably a mochitest bug; the test summary is showing no failures:

17:26:25     INFO -  351 INFO TEST-START | Shutdown
17:26:25     INFO -  352 INFO Passed:  28046
17:26:25     INFO -  353 INFO Failed:  0
17:26:25     INFO -  354 INFO Todo:    2571

This causes the mozharness script to terminate with a pass.
Component: Treeherder → Mochitest
Product: Tree Management → Testing
Version: --- → Trunk
Summary: test_collapse.html is perma-fail without turning TreeHerder orange → test_collapse.html is perma-fail but the mochitest summary shows 0 fails
This test uses its own subharness, testharness.js, and it's likely the problem lies in some interaction with that and mochitest.
Blocks: 1048775
(In reply to Jonathan Griffin (:jgriffin) from comment #2)
> This test uses its own subharness, testharness.js, and it's likely the
> problem lies in some interaction with that and mochitest.

Ms2ger may know about that.
(In reply to Martijn Wargers [:mwargers] (QA) from comment #4)
> Here is the list with expected failures in for the test_collapse.html file:
> http://mxr.mozilla.org/mozilla-central/source/dom/imptests/failures/editing/
> selecttest/test_collapse.html.json

This is the test timing out, that failure isn't listed there...
Tried to reproduce locally:

10 INFO Passed:  42
11 INFO Failed:  4
12 INFO Todo:    10
You need to log in before you can comment on or make changes to this bug.