1250480 - Improve in_app restart handling in case of slow Firefox shutdowns (sometimes with expected crashes)

Status: RESOLVED INCOMPLETE

(Remote Protocol :: Marionette, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Over the last weeks I have noticed a lot of failures with the following error details while running our Firefox UI tests and especially restart tests:

* Timed out waiting for port! (bug 1202375)
* [Errno 61] Connection refused (bug 1248479)
* [Errno 10061] No connection could be made because the target machine actively refused it  (bug 1248720)

The latter two are caused by a shutdown hang of Firefox which sometimes results in a crash those days (see bug 1243485). Usually a slow shutdown should not cause a problem with the test harness given that it should wait until the application under test has been restarted. But that doesn't seem to be the case here with Marionette. While Firefox hasn't been restarted yet Marionette continuous with the test execution. This has two big drawbacks:

1. We do not test what we want given that we expect that Firefox has been restarted!

2. Marionette cannot connect to Firefox in case its server component has already been shutdown.

I cannot remember that we ever had this problem with Mozmill. So it should be some faulty handling on driver side. While checking the following link I do not see that we wait for a shutdown of the application in case of an in_app restart:

https://dxr.mozilla.org/mozilla-central/source/testing/marionette/driver/marionette_driver/marionette.py#1087

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

I haven't seen those issues anytime longer for a longer period. We can investigate more once we see the issue bumping up again. Until then I will close this bug as incomplete.