1630037 - Unexpected shutdown of the browser hangs Raptor

Status: RESOLVED FIXED

(Testing :: Raptor, defect, P1)

Description

[Henrik Skupin [:whimboo][⌚️UTC+1]]

When I start any kind of Raptor job and then force kill the application via Cmd+Q Raptor hangs. As it looks like we are not correctly checking the process' exit code on a regular basis while the tests are running.

This could cause various hangs without proper reporting. Maybe even related to bug 1627867 when geckoview_example gets killed by the system.

Comment 1

[Greg Mierzwinski [:sparky]]

Where in the execution do you run Cmd+Q? Is it equivalent to Ctrl+C? I'm asking because I haven't hit this issue before and I use Ctrl+C a lot.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Just close the browser. Feel free to also use "Firefox | Quit Firefox".

Comment 3

[Greg Mierzwinski [:sparky]]

Oh ok! Thanks for clarifying.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Note that if the application quits without a generated minidump it might show up on Treeherder as Connection to Raptor webextension failed!, or even task aborted. I didn't wait long enough to check which timeout we actually hit.

Comment 5

[Florin Strugariu [:Bebe]]

Here is where we should close the browser after the timeout is finished:

https://searchfox.org/mozilla-central/search?q=wait_for_test_finish&case=true&path=

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+1]]

As just verified also with the upcoming patch for bug 1630952 - when Firefox closes itself for a content crash - Raptor hangs around for about 25 minutes.

Checking the process regularly will also have an effect for CI costs.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: Bug 1630037 - [raptor] Immediately bail out for an unexpected shutdown of the browser.

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+1]]

This works great and also enables us to see crashes of Chrome:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=a985bde0d1ef815fd52d34e6ecdcab146b3a0bc2

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/062f5214f393
[raptor] Immediately bail out for an unexpected shutdown of the browser. r=perftest-reviewers,sparky

Comment 10

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/062f5214f393