Bugzilla

(In reply to Nicolas Chevobbe [:nchevobbe] from comment https://bugzilla.mozilla.org/show_bug.cgi?id=1739115#c0)
> We don't have any automated way to check that xpcshell test can be debugged, and as a result, we often break them.
> Testing it would be hard, but we could at least check for each release that it's not broken.

Nicolas: thanks for taking steps to keep `xpcshell` debugging working each release: much appreciated.

I'd like to understand if we can do something automated, however.  Bug 1686344 will make `firefox --backgroundtask ... --jsdebugger --wait-for-jsdebugger` wait for the devtools to be connected before continuing.  The advantage of the background task is that it's easy to launch from more types of test, including `browser` tests, where-as it's Not Easy to launch a "sub xpcshell test" from a `browser` test.  The actual mechanism will be _very_ close to the mechanism for `xpcshell` -- the only real differences will be in the way that the initial breakpoints is set -- so in theory testing this mechanism will both ensure that background tasks can be debugged but also that `xpcshell` tests can be debugged.

Can you suggest how to test this mechanism using the existing devtools testing helpers?  IIUC, the existing tests such as https://searchfox.org/mozilla-central/source/devtools/client/framework/browser-toolbox/test/browser_browser_toolbox_debugger.js:

1.  launch a separate/remote browser console process
2.  connect that remote process to the process running the test harness
3.  drive the remote console process to perform the test

What we want to do is something like the following:

1.  launch a background task waiting to be debugged (`firefox --backgroundtask <TEST TASK> --jsdebugger -- wait-for-jsdebugger`)
2.  connect the test harness process to the background task process
3.  drive the test harness process to perform the test

But it's not clear that this can be done: perhaps using the devtools within the test harness process is not possible?

Reading https://searchfox.org/mozilla-central/source/devtools/client/framework/browser-toolbox/README.md, it sounds like what might be better is to have a test-specific background task that itself uses the various testing helpers to launch the browser toolbox process to connect to itself... but I don't see how to do the equivalent of the "toolbox task" when the thread of execution in the task itself should be blocked waiting for the debugger to connect.

Can you suggest a path forward?  Perhaps a brief video call with one of the devtools experts would be in order, since I don't understand what's possible in that space and y'all probably don't have a great understanding of what's possible with background tasks.

(In reply to Nicolas Chevobbe [:nchevobbe] from comment https://bugzilla.mozilla.org/show_bug.cgi?id=1739115#c0)
> We don't have any automated way to check that xpcshell test can be debugged, and as a result, we often break them.
> Testing it would be hard, but we could at least check for each release that it's not broken.

Nicolas: thanks for taking steps to keep `xpcshell` debugging working each release: much appreciated.

I'd like to understand if we can do something automated, however.  This bug will make `firefox --backgroundtask ... --jsdebugger --wait-for-jsdebugger` wait for the devtools to be connected before continuing.  The advantage of the background task is that it's easy to launch from more types of test, including `browser` tests, where-as it's Not Easy to launch a "sub xpcshell test" from a `browser` test.  The actual mechanism will be _very_ close to the mechanism for `xpcshell` -- the only real differences will be in the way that the initial breakpoints is set -- so in theory testing this mechanism will both ensure that background tasks can be debugged but also that `xpcshell` tests can be debugged.

Can you suggest how to test this mechanism using the existing devtools testing helpers?  IIUC, the existing tests such as https://searchfox.org/mozilla-central/source/devtools/client/framework/browser-toolbox/test/browser_browser_toolbox_debugger.js:

1.  launch a separate/remote browser console process
2.  connect that remote process to the process running the test harness
3.  drive the remote console process to perform the test

What we want to do is something like the following:

1.  launch a background task waiting to be debugged (`firefox --backgroundtask <TEST TASK> --jsdebugger -- wait-for-jsdebugger`)
2.  connect the test harness process to the background task process
3.  drive the test harness process to perform the test

But it's not clear that this can be done: perhaps using the devtools within the test harness process is not possible?

Reading https://searchfox.org/mozilla-central/source/devtools/client/framework/browser-toolbox/README.md, it sounds like what might be better is to have a test-specific background task that itself uses the various testing helpers to launch the browser toolbox process to connect to itself... but I don't see how to do the equivalent of the "toolbox task" when the thread of execution in the task itself should be blocked waiting for the debugger to connect.

Can you suggest a path forward?  Perhaps a brief video call with one of the devtools experts would be in order, since I don't understand what's possible in that space and y'all probably don't have a great understanding of what's possible with background tasks.