Add a "--debug-on-failure" argument to Marionette harness to allow stopping a test on an unexpected test result
Categories
(Testing :: Marionette Client and Harness, enhancement, P3)
Tracking
(Not tracked)
People
(Reporter: whimboo, Unassigned)
Details
(Whiteboard: [webdriver:backlog])
Like browser chrome tests, which provide a --debug-on-failure option to automatically pause execution when an unexpected result occurs, we could introduce a similar mechanism for Marionette. The main difference from mochitests is that Marionette runs externally and connects to Firefox over a socket rather than executing directly inside the browser.
One approach would be to evaluate a debugger statement when a failure is detected. However, this is not always reliable: it depends on the active socket connection, the stack trace of the failing command is already gone, and we must ensure that evaluating debugger does not trigger a script timeout. While timeouts are configured globally, this should not have side effects since the WebDriver session will be closing anyway.
An alternative would be to add a breakpoint on the Python side, halting the script there. To investigate the failure, the developer would then need to open the Browser Toolbox manually.
I’d like to discuss whether such an option would be useful. Recently, when running many Marionette tests, I repeatedly had to locate the failing test file, modify it to only run the affected test, and insert breakpoints manually, which is a time consuming process.
|
Reporter | |
Comment 1•23 days ago
|
||
Lets try with the JavaScript evaluate approach first and see if it works.
Description
•