1848799 - Report crashes during wdspec tests as TEST-UNEXPECTED-CRASH

Status: RESOLVED FIXED

(Testing :: web-platform-tests, enhancement, P1)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

As right now we do not correctly report crashes of Firefox which could happen during the execution of wdspec tests, and as such failing tests are wrongly classified because the crash details as determined by mozcrash aren't taken into account. This could lead to missing information about crashes (maybe top-crashes) for Firefox where we could help developers in reproduction and fixing.

As such we should consider to report crashes correctly. Here a try build which shows a forced crash of Firefox by loading about:crashparent

https://treeherder.mozilla.org/logviewer?job_id=426057078&repo=try&lineNumber=7466

The downside is that we won't further run remaining tests in that particular pytest file. Probably that is fine for now.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I've submitted a try build to see how it works for existing non-modified wdspec tests:
https://treeherder.mozilla.org/jobs?repo=try&revision=0f14c379ab4f415a5b7a07d8015717da90681b9c

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: Bug 1848799 - [wdspec] Change test status for wdspec test to CRASH if browser crashed.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/351c8cb39e0a
[wdspec] Change test status for wdspec test to CRASH if browser crashed. r=webdriver-reviewers,jdescottes

Comment 4

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/41536 for changes under testing/web-platform/tests

Comment 5

[Natalia Csoregi [:nataliaCs]]

Backed out for causing high frequency failures in Bug 1849416

• backout: https://hg.mozilla.org/integration/autoland/rev/95d012c9953e13d735e0983768835c96b5ad3472

Comment 6

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR was closed without merging

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2]]

All the failures here in crash_content_process.py happened on LInux opt no fission builds. It should be marked as intermittent PASS / FAIL until the underlying issue has been fixed.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d66be226a1d7
[wdspec] Change test status for wdspec test to CRASH if browser crashed. r=webdriver-reviewers,jdescottes

Comment 9

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/d66be226a1d7

Comment 10

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged by moz-wptsync-bot

Comment 11

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged by moz-wptsync-bot