1438343 - Local mochitest run hangs for a long time after the test has finished running, before exiting

Status: NEW

(Testing :: Mochitest, enhancement)

Description

[Botond Ballo [:botond]]

Attachment: Full output of 'mach mochitest' invocation

It seems whenever I run a mochitest locally with 'mach mochitest', even though the test itself finishes in a reasonable amount of time, and produces a TEST-OK message like this:

TEST-OK | gfx/layers/apz/test/mochitest/test_group_hittest.html | took 160121ms

the 'mach mochitest' invocation still hangs for a long time, before also producing a TEST-UNEXPECTED-TIMEOUT message like this:

TEST-UNEXPECTED-TIMEOUT | gfx/layers/apz/test/mochitest/test_group_hittest.html | application timed out after 370 seconds with no output

(along with a bunch of other output) and exiting.

This happens to me every time I run a mochitest locally. It usually doesn't bother me (I just Ctrl-C it after the TEST-OK message), but it does add significant overhead if you're trying to script the 'mach mochitest' invocation and do additional things after it exits.

I've attached the full output of an example 'mach mochitest' invocation.

Comment 1

[Andrew Halberstadt [:ahal]]

This is actually on purpose when running a single mochitest, its purportedly so you can debug the browser session after the mochitest runs. This behaviour can be enabled with --keep-open or disabled with --keep-open=false.

Personally I agree that defaulting to this behaviour for a single test is confusing. Though IIRC, the last time we tried to remove this default (probably 3-4 years ago), there was some pushback from developers who liked it.

You could add the following to your ~/.mozbuild/machrc if you want to always turn this behaviour off:
[alias]
mochitest = mochitest --keep-open=false

Comment 2

[Botond Ballo [:botond]]

Oh! I see.

I actually saw --keep-open in 'mach help mochitest', but nothing there suggested to me that it was the default. Perhaps that could be made clearer?

Comment 3

[Botond Ballo [:botond]]

(It's also somewhat odd for "TEST-UNEXPECTED-FAIL" to be displayed when this is the harness's expected behaviour.)

Comment 4

[Botond Ballo [:botond]]

(In reply to Botond Ballo [:botond] from comment #3)
> (It's also somewhat odd for "TEST-UNEXPECTED-FAIL" to be displayed when this
> is the harness's expected behaviour.)

Er, "TEST-UNEXPECTED-TIMEOUT", sorry.

Comment 5

[Andrew Halberstadt [:ahal]]

(In reply to Botond Ballo [:botond] from comment #2)
> Oh! I see.
> 
> I actually saw --keep-open in 'mach help mochitest', but nothing there
> suggested to me that it was the default. Perhaps that could be made clearer?

The problem is that it's only the default for mochitest-plain (not chrome, or browser-chrome) and only when running a single test (not when running multiple tests). So basically it's super confusing. I'd be happy to have the defaults be consistent across flavors and no matter the number of tests being run.

You have a good point about the TEST-UNEXPECTED-TIMEOUT being even extra confusing. Maybe this bug could be morphed into fixing that.