1640530 - Perma [tier2] Process killed after 120s because no connection to Marionette server could be established. Check gecko.log for errors

Status: RESOLVED DUPLICATE of bug 1642032

(Firefox Build System :: Task Configuration, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: archaeopteryx [at] coole-files.de
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=303552122&repo=mozilla-beta
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/U49qEfMWSBeWE3GUJ1KNIw/runs/0/artifacts/public/logs/live_backing.log

---

[task 2020-05-24T21:09:41.384Z] 21:09:41 INFO - 1590354581383 Toolkit.Telemetry TRACE TelemetrySession::delayedInit
[task 2020-05-24T21:09:41.384Z] 21:09:41 INFO - 1590354581383 Toolkit.Telemetry TRACE TelemetryStorage::_popAndPerformQueuedOperation - Performing queued operation.
[task 2020-05-24T21:09:41.404Z] 21:09:41 INFO - 1590354581393 Toolkit.Telemetry INFO TelemetryStorage::_loadSessionData - can not load session data file: {"operation":"open","path":"Z:\task_1590354356\build\tmpof47cd.mozrunner\datareporting\session-state.json","winLastError":2}
[task 2020-05-24T21:09:41.404Z] 21:09:41 INFO - 1590354581394 Toolkit.Telemetry TRACE TelemetryStorage::_popAndPerformQueuedOperation - Performing queued operation.
[task 2020-05-24T21:09:41.490Z] 21:09:41 INFO - ###!!! ERROR: Potential deadlock detected:
[task 2020-05-24T21:09:41.490Z] 21:09:41 INFO - === Cyclical dependency starts at
[task 2020-05-24T21:09:41.490Z] 21:09:41 INFO - --- Mutex : StaticMutex (currently acquired)
[task 2020-05-24T21:09:41.490Z] 21:09:41 INFO - calling context
[task 2020-05-24T21:09:41.490Z] 21:09:41 INFO - [stack trace unavailable]
[task 2020-05-24T21:09:41.490Z] 21:09:41 INFO - === Cycle completed at
[task 2020-05-24T21:09:41.490Z] 21:09:41 INFO - --- Mutex : StaticMutex (currently acquired)
[task 2020-05-24T21:09:41.490Z] 21:09:41 INFO - calling context
[task 2020-05-24T21:09:41.490Z] 21:09:41 INFO - [stack trace unavailable]
[task 2020-05-24T21:09:41.490Z] 21:09:41 INFO - ###!!! Deadlock may happen NOW!
[task 2020-05-24T21:09:41.491Z] 21:09:41 INFO - [Parent 1536, Main Thread] ###!!! ASSERTION: Potential deadlock detected:
[task 2020-05-24T21:09:41.491Z] 21:09:41 INFO - Cyclical dependency starts at
[task 2020-05-24T21:09:41.491Z] 21:09:41 INFO - Mutex : StaticMutex (currently acquired)
[task 2020-05-24T21:09:41.491Z] 21:09:41 INFO - Cycle completed at
[task 2020-05-24T21:09:41.491Z] 21:09:41 INFO - Mutex : StaticMutex (currently acquired)
[task 2020-05-24T21:09:41.491Z] 21:09:41 INFO - ###!!! Deadlock may happen NOW!
[task 2020-05-24T21:09:41.491Z] 21:09:41 INFO - : 'Error', file /builds/worker/checkouts/gecko/xpcom/threads/BlockingResourceBase.cpp, line 248
[task 2020-05-24T21:11:36.494Z] 21:11:36 ERROR - Process killed after 120s because no connection to Marionette server could be established. Check gecko.log for errors
[task 2020-05-24T21:11:36.494Z] 21:11:36 ERROR - Traceback (most recent call last):
[task 2020-05-24T21:11:36.494Z] 21:11:36 ERROR -
[task 2020-05-24T21:11:36.494Z] 21:11:36 ERROR - File "Z:\task_1590354356\build\venv\lib\site-packages\marionette_harness\runtests.py", line 91, in cli
[task 2020-05-24T21:11:36.494Z] 21:11:36 ERROR - failed = harness_instance.run()
[task 2020-05-24T21:11:36.494Z] 21:11:36 ERROR -
[task 2020-05-24T21:11:36.494Z] 21:11:36 ERROR - File "Z:\task_1590354356\build\venv\lib\site-packages\marionette_harness\runtests.py", line 71, in run
[task 2020-05-24T21:11:36.494Z] 21:11:36 ERROR - runner.run_tests(tests)
[task 2020-05-24T21:11:36.494Z] 21:11:36 ERROR -
[task 2020-05-24T21:11:36.494Z] 21:11:36 ERROR - File "Z:\task_1590354356\build\venv\lib\site-packages\marionette_harness\runner\base.py", line 848, in run_tests
[task 2020-05-24T21:11:36.494Z] 21:11:36 ERROR - self.marionette = self.driverclass(**self._build_kwargs())
[task 2020-05-24T21:11:36.494Z] 21:11:36 ERROR -
[task 2020-05-24T21:11:36.494Z] 21:11:36 ERROR - File "Z:\task_1590354356\build\venv\lib\site-packages\marionette_driver\marionette.py", line 462, in init
[task 2020-05-24T21:11:36.494Z] 21:11:36 ERROR - self.start_binary(self.startup_timeout)
[task 2020-05-24T21:11:36.494Z] 21:11:36 ERROR -
[task 2020-05-24T21:11:36.494Z] 21:11:36 ERROR - File "Z:\task_1590354356\build\venv\lib\site-packages\marionette_driver\marionette.py", line 490, in start_binary
[task 2020-05-24T21:11:36.494Z] 21:11:36 ERROR - reraise(IOError, IOError(msg.format(timeout)), sys.exc_info()[2])
[task 2020-05-24T21:11:36.495Z] 21:11:36 ERROR -
[task 2020-05-24T21:11:36.495Z] 21:11:36 ERROR - File "Z:\task_1590354356\build\venv\lib\site-packages\marionette_driver\marionette.py", line 482, in start_binary
[task 2020-05-24T21:11:36.495Z] 21:11:36 ERROR - self.raise_for_port(timeout=timeout)
[task 2020-05-24T21:11:36.495Z] 21:11:36 ERROR -
[task 2020-05-24T21:11:36.495Z] 21:11:36 ERROR - File "Z:\task_1590354356\build\venv\lib\site-packages\marionette_driver\marionette.py", line 568, in raise_for_port
[task 2020-05-24T21:11:36.495Z] 21:11:36 ERROR - self.host, self.port))
[task 2020-05-24T21:11:36.495Z] 21:11:36 ERROR -
[task 2020-05-24T21:11:36.495Z] 21:11:36 ERROR - IOError: Process killed after 120s because no connection to Marionette server could be established. Check gecko.log for errors

Comment 1

[Intermittent Failures Robot]

1 failures in 4727 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform breakdown:

• windows10-64-mingwclang: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1640530&startday=2020-05-18&endday=2020-05-24&tree=all

Comment 2

[Intermittent Failures Robot]

12 failures in 5458 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 12

Platform breakdown:

• windows10-64-mingwclang: 12

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1640530&startday=2020-05-25&endday=2020-05-31&tree=all

Comment 3

[Andreea Pavel [:apavel]]

Started permafailing from this merge https://hg.mozilla.org/mozilla-central/pushloghtml?changeset=a58cc68b0c51645d34cbc7f8f8a30edae77ab396

Retriggers: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&resultStatus=pending%2Crunning%2Csuccess%2Ctestfailed%2Cbusted%2Cexception&searchStr=windows%2C10%2Cx64%2Cmingw%2Cdebug%2Cfirefox%2Cfunctional%2Ctests%2C%28remote%29%2Ctest-windows10-64-mingwclang%2Fdebug-firefox-ui-functional-remote-e10s%2Cfxfn-r%28en-us%29&selectedTaskRun=PlhTESkBSZaP6_Ev3STClA-0

Garvan can you assign someone to take a look?

Comment 4

[:garvan]

Firefox iOS uses BuddyBuild and Bitrise for CI, not treeherder, I don't think this is in the right place.

Comment 6

[Intermittent Failures Robot]

19 failures in 4949 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 19

Platform breakdown:

• windows10-64-mingwclang: 19

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1640530&startday=2020-06-01&endday=2020-06-07&tree=all