1448000 - Improve --debugger experience with reftest run-by-manifest and MOZ_DEBUG_CHILD_PROCESS

Status: NEW

(Testing :: Reftest, enhancement)

Description

[Andrew Halberstadt [:ahal]]

From bug 1392391 comment 50

(In reply to Matt Woodrow (:mattwoodrow) from comment #50)
> These changes make debugging reftests (on OSX) even more unpleasant than
> they were previously.
> 
> With MOZ_DEBUG_CHILD_PROCESS (the only way to debug the content process with
> lldb as far as I know), there are now 8 processes launched, with the last
> one being the one that actually runs the reftests.
> 
> Each launch pauses the parent, so I have to attach to all of them
> individually to resume them (in lots of different Terminal tabs), and keep
> switching back to the firefox process as it waits for focus.
> 
> Even when not debugging, the extra instance launch takes quite a bit of time
> in a debug build, since it opens 3 windows.
> 
> Is there anything we can do to simplify this? Trying to debug reftests takes
> forever now.

One easy solution would be to turn off run-by-manifest if --debugger is specified. The downside is that tests could run slightly differently than they do in CI (though this wouldn't be a problem if the developer passed in the manifest containing the test).

Mochitest has also been using run-by-manifest for years now, and I haven't heard any complaints about debugging over there. Would be worth investigating whether or not this problem exists there, and if not why.

As a short term workaround, try flipping this to False:
https://searchfox.org/mozilla-central/source/layout/tools/reftest/runreftest.py#242

Comment 1

[Matt Woodrow (:mattwoodrow)]

Thanks for looking into this!

Some thoughts I had last night:

* If manifest parsing is going to use a separate .jsm, does it need to run in an actual firefox instance? Can't we just have a JS shell to do this?

* Does anyone know why we open an empty browser window (with chrome and all) briefly, before closing it again. This definitely takes up quite a bit of time in a debug build.

* Why do we start so many child processes (4/instance, even with --setpref dom.ipc.processCount=1)? Do we really need all of these for reftests?

* Can we make an alternative version of MOZ_DEBUG_CHILD_PROCESS that only pauses and waits for the 'real' child process, and not the other ones.

* Is it possible to have a mach flag that attaches a debugger to the child process automatically? I believe the answer is no with lldb, but I'd love to be proven wrong.

I realize that these cover quite a large area of the codebase (and aren't really directed at any one person in particular), just thinking out loud about what it would take to have an ideal debugging situation :)

Comment 2

[Matt Woodrow (:mattwoodrow)]

Looked into the processes a bit more, I'm only seeing 3 child processes/browser now.

One is for the initial window, we close this (and shutdown the child process) before launching the actual reftest window, so we then have to launch another child. Getting rid of the extra window here should get rid of this extra process.

The third is for an "extension" process, it seems plausible to detect this (though it doesn't appear to be passed to the child currently) and only have MOZ_DEBUG_CHILD_PROCESS stop for "web" children.

Comment 3

[Andrew Halberstadt [:ahal]]

(In reply to Matt Woodrow (:mattwoodrow) from comment #1)
> * If manifest parsing is going to use a separate .jsm, does it need to run
> in an actual firefox instance? Can't we just have a JS shell to do this?

Unfortunately no. When parsing the manifests, we pull all sorts of information from internal Gecko APIs and use it to populate the condition sandbox used for skipping/fuzzy/etc. I couldn't even parse the manifests with --headless because that has different graphics characteristics.

> * Does anyone know why we open an empty browser window (with chrome and all)
> briefly, before closing it again. This definitely takes up quite a bit of
> time in a debug build.

Afaik, this has always happened. I'm not sure why (hysterical raisins?) but we could definitely investigate removing this.

> * Why do we start so many child processes (4/instance, even with --setpref
> dom.ipc.processCount=1)? Do we really need all of these for reftests?

I'd wager this isn't intentional.

> * Can we make an alternative version of MOZ_DEBUG_CHILD_PROCESS that only
> pauses and waits for the 'real' child process, and not the other ones.
> 
> * Is it possible to have a mach flag that attaches a debugger to the child
> process automatically? I believe the answer is no with lldb, but I'd love to
> be proven wrong.
> 
> I realize that these cover quite a large area of the codebase (and aren't
> really directed at any one person in particular), just thinking out loud
> about what it would take to have an ideal debugging situation :)

Understood, these last two bullets are definitely out of my control, but good questions nonetheless. Thanks for brainstorming. As far as run-by-manifest goes, most of these points were problems before bug 1392391. Do you agree with simply disabling run-by-manifest with --debugger to solve the regression?

Comment 4

[Matt Woodrow (:mattwoodrow)]

(In reply to Andrew Halberstadt [:ahal] from comment #3)
> 
> Understood, these last two bullets are definitely out of my control, but
> good questions nonetheless. Thanks for brainstorming. As far as
> run-by-manifest goes, most of these points were problems before bug 1392391.
> Do you agree with simply disabling run-by-manifest with --debugger to solve
> the regression?

Sorry, managed to miss clicking submit. Yes, I think that'd be a great help! Thanks.