1009875 - Add pause on the beginning/end of test execution

Status: RESOLVED WONTFIX

(Testing Graveyard :: Mozmill, enhancement)

Description

[Mateusz Balbus]

Attachment: Proposed solution

It would be nice to be able to pause tests before theirs execution. Before tests execution one can attach Firefox remote debugger, set breakpoints and start debugging tests.

Pause after test execution is also helpful in test deuggging. It allows to dig into running application to figure out why tests are failing.

Attached patch introduces options: --pause-on-start and --pause-on-stop. Execution is resumed on Ctrl + C

Please add this feature to the code base.

Comment 1

[Andrei Eftimie]

I like this idea as I've often had to rely on long sleep calls to be able to manually debug a problem.

While --pause-on-start and --pause-on-stop might handle some cases, I think we should also export a method call that we can insert anywhere in a test which will pause the execution and continue on Ctrl + C.

A poor man's breakpoint if you will.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+1]]

So this is a good start into the direction of using the internal debugger of e.g. Firefox to control the execution of a test. That's something what we are missing for a long time. It's great that you got started on it balbusm!

What I would propose here is that we do not start in adding those command line options but to use an option which will enable the internal debugger for the Mozmill test to be executed. With that in-place you could easily add a 'debugger' statement where-ever you want in your test. Once the line of code has been reached the execution is automatically stalled, you can do modifications or analyze current values. I don't think that this should be too hard. Maybe the snippet on that site might already be helpful.

Balbusm, if you are interested to get this implemented, I'm happy to mentor through the process and to make sure to help where-ever I can. I would indeed also love to see this feature implemented! Please let us know.

Comment 3

[Mateusz Balbus]

I'm happy to help.

Henrik, would you like to pause test execution till Firefox remote debugger is attached?
When I was developing Thunderbird add-on I was using --pause-on-start in such a way - kick off mozmill, attach Firefox remote debugger, set few breakpoints, ctrl + c to kick off tests.
We could automate it:
- wait with test execution till debugger is attached
- pause debugger on start to give user opportunity to set breakpoints
What do you think about it?

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Mozmill will reach its end of life soon. We are currently working on getting all the tests for Firefox ported to Marionette. For status updates please see bug 1080766.

Regarding this request, with Marionette it will become way easier to debug a test given that you can step through on the Python side.

Comment 5

[Mateusz Balbus]

Attachment: Wait for debugger new implementation

I implemented solution that holds python execution till debugger is attached. It works with Firefox and Thunderbird. The drawback I found is that test execution starts when any debugger is attached (eg. web debugger)
I attaching my code. Maybe it will be useful in Marionette use case.