1362901 - Intermittent hang (or crash?) in install_rust_panic_hook()

Status: RESOLVED DUPLICATE of bug 1358151

(Toolkit :: Crash Reporting, defect)

Description

[Geoff Brown [:gbrown] (pto Apr 11-18)]

In bug 1352671, there are frequent reports of intermittent test failures which occur when the marionette server does not respond. It is fairly easy to show that this is because startup does not progress to marionette startup: browser startup hangs early on, or possibly crashes (but there is no crash report or other indication of a crash). These failures seem to be most frequent on Windows 8 Debug.

With extra logging, the failure becomes ridiculously easy to reproduce:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=e9810f856183d27aa4f9c3c892e73d20bd8f816f

https://archive.mozilla.org/pub/firefox/try-builds/[e-mail]-e9810f856183d27aa4f9c3c892e73d20bd8f816f/try-win64-debug/try_win8_64-debug_test-mochitest-1-bm110-tests1-windows-build64.txt.gz

15:30:36     INFO - TEST-INFO | started process GECKO(1604)
15:30:36     INFO - GECKO(1604) | XRE_mainInit calls SetExceptionHandler
15:30:36     INFO - GECKO(1604) | SetExceptionHandler calls install_rust_panic_hook
15:32:37     INFO -  Traceback (most recent call last):
15:32:37     INFO -    File "C:\slave\test\build\tests\mochitest\runtests.py", line 2507, in doTests
15:32:37     INFO -      marionette_args=marionette_args,
15:32:37     INFO -    File "C:\slave\test\build\tests\mochitest\runtests.py", line 2107, in runApp
15:32:37     INFO -      self.marionette.start_session(timeout=port_timeout)
15:32:37     INFO -    File "C:\slave\test\build\venv\lib\site-packages\marionette_driver\decorators.py", line 28, in _
15:32:37     INFO -      m._handle_socket_failure()
15:32:37     INFO -    File "C:\slave\test\build\venv\lib\site-packages\marionette_driver\decorators.py", line 23, in _
15:32:37     INFO -      return func(*args, **kwargs)
15:32:37     INFO -    File "C:\slave\test\build\venv\lib\site-packages\marionette_driver\marionette.py", line 1319, in start_session
15:32:37     INFO -      self.protocol, _ = self.client.connect()
15:32:37     INFO -    File "C:\slave\test\build\venv\lib\site-packages\marionette_driver\transport.py", line 223, in connect
15:32:37     INFO -      self.sock.connect((self.addr, self.port))
15:32:37     INFO -    File "c:\mozilla-build\python27\Lib\socket.py", line 224, in meth
15:32:37     INFO -      return getattr(self._sock,name)(*args)
15:32:37     INFO -  error: [Errno 10061] No connection could be made because the target machine actively refused it
15:32:37    ERROR - Automation Error: Received unexpected exception while running application
15:32:37    ERROR - 

XRE_mainInit() calls CrashReporter::SetExceptionHandler(), which calls install_rust_panic_hook(); intermittently, install_rust_panic_hook() does not complete.

Comment 1

[Geoff Brown [:gbrown] (pto Apr 11-18)]

:ted -- Can you investigate?

Comment 2

[(not currently active) Ted Mielczarek]

I can do you one better--I believe this is caused by the issue described in bug 1358151, which has a patch that just landed 3 days ago.

If you look at the stack in bug 1358151 comment 0 you can see that it goes through SetExceptionHandler into the Rust standard library and deadlocks.

Comment 3

[Geoff Brown [:gbrown] (pto Apr 11-18)]

Awesome - thanks! There has been a big reduction in bug 1352671 failure frequency.