Intermittent Linux Marionette "ERROR - timed out after 1000 seconds of no output"

RESOLVED FIXED in Firefox 44

Status

RESOLVED FIXED
4 years ago
3 years ago

People

(Reporter: RyanVM, Unassigned)

Tracking

({intermittent-failure, pi-marionette-intermittent})

unspecified
mozilla44
Unspecified
Linux
intermittent-failure, pi-marionette-intermittent
Points:
---

Firefox Tracking Flags

(firefox44 fixed)

Details

Attachments

(2 attachments)

(Reporter)

Description

4 years ago
First instance was Friday on mozilla-central. Affects both e10s and non-e10s runs.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Hey Julien, I saw a few changes of yours land on Friday, could this be related?
Flags: needinfo?(j.parkouss)
Comment hidden (Treeherder Robot)
Well, this is not obvious. My changes were on the client (and python runner only) side, so I would be surprised if that could produce intermittents.
Flags: needinfo?(j.parkouss)
(In reply to Julien Pagès from comment #16)
> Well, this is not obvious. My changes were on the client (and python runner
> only) side, so I would be surprised if that could produce intermittents.

Sorry, I should have noticed before but this started before your landings.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Created attachment 8665804 [details]
MozReview Request: Bug 1185486 - Part 1: Use a timeout when waiting on the browser process after a timed out command. r=jgriffin

Bug 1185486 - Part 1: Use a timeout when waiting on the browser process after a timed out command. r=jgriffin
Attachment #8665804 - Flags: review?(jgriffin)
Created attachment 8665805 [details]
MozReview Request: Bug 1185486 - Part 2: Prevent message listeners from a failed command from causing an out of order response when a content command hangs. r=ato

Bug 1185486 - Part 2: Prevent message listeners from a failed command from causing an out of order response when a content command hangs. r=ato
Attachment #8665805 - Flags: review?(ato)
Comment on attachment 8665805 [details]
MozReview Request: Bug 1185486 - Part 2: Prevent message listeners from a failed command from causing an out of order response when a content command hangs. r=ato

https://reviewboard.mozilla.org/r/20363/#review18303

This looks okay to me, but this code is known to be quite brittle.  Can you do a try run before you land?
Attachment #8665805 - Flags: review?(ato) → review+
Attachment #8665804 - Flags: review?(jgriffin) → review+
Comment on attachment 8665804 [details]
MozReview Request: Bug 1185486 - Part 1: Use a timeout when waiting on the browser process after a timed out command. r=jgriffin

https://reviewboard.mozilla.org/r/20361/#review18309

::: testing/marionette/driver/marionette_driver/marionette.py:691
(Diff revision 1)
> -                returncode = self.instance.runner.wait()
> +                returncode = self.instance.runner.wait(timeout=self.socket_timeout)

We could actually use a much shorter timeout here, probably something like 60s (the default socket timeout is 360 in Marionette, I believe).
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
https://hg.mozilla.org/mozilla-central/rev/9ffc3758f4ba
https://hg.mozilla.org/mozilla-central/rev/1370508d6c1e
Status: NEW → RESOLVED
Last Resolved: 3 years ago
status-firefox44: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla44
Comment hidden (Treeherder Robot)
You need to log in before you can comment on or make changes to this bug.