1024686 - Intermittent test_healthreporter.js | application crashed [@ mozilla::storage::Connection::~Connection()] after "Assertion failure: !mAsyncExecutionThread (AsyncClose has not been invoked on this connection!), at storage/src/mozStorageConnection.cpp:491"

Status: RESOLVED FIXED

(Core :: SQLite and Embedded Database Bindings, defect)

Description

[Ryan VanderMeulen [:RyanVM]]

https://tbpl.mozilla.org/php/getParsedLog.php?id=41608963&tree=Mozilla-Aurora

WINNT 6.2 mozilla-aurora debug test xpcshell on 2014-06-12 09:06:33 PDT for push d740cc08d966
slave: t-w864-ix-046

09:38:56     INFO -  TEST-PASS | C:\\slave\\test\\build\\tests\\xpcshell\\head.js | [do_check_eq : 762] 1 == 1
09:38:56     INFO -  Services.HealthReport.HealthReporter	INFO	Request to shut down.
09:38:56     INFO -  Services.HealthReport.HealthReporter	WARN	Initiating main shutdown procedure.
09:38:56     INFO -  Services.HealthReport.HealthReporter	INFO	Shutting down provider manager.
09:38:56     INFO -  Services.HealthReport.HealthReporter	INFO	Provider manager shut down.
09:38:56     INFO -  Services.HealthReport.HealthReporter	INFO	Shutting down storage.
09:38:56     INFO -  Services.HealthReport.HealthReporter	INFO	Initializing health reporter instance against healthreport.testing.state_create_client_id.healthreport.
09:38:56     INFO -  Services.HealthReport.HealthReporter	DEBUG	Storing state in c:\\users\\cltbld~1.t-w\\appdata\\local\\temp\\tmpxhh6jw\\healthreport\\state-state_create_client_id.json
09:38:56     INFO -  Sqlite.Connection.state_create_client_id	DEBUG	Conn #0: Stmt #60 finished.
09:38:56     INFO -  Sqlite.Connection.state_create_client_id	TRACE	Conn #0: Stmt #61 SELECT 1
09:38:56     INFO -  Sqlite.Connection.state_create_client_id	DEBUG	Conn #0: Stmt #61 finished.
09:38:56     INFO -  Sqlite.Connection.state_create_client_id	DEBUG	Conn #0: Request to close connection.
09:38:56     INFO -  Sqlite.Connection.state_create_client_id	DEBUG	Conn #0: Finalizing connection.
09:38:56     INFO -  Sqlite.Connection.state_create_client_id	DEBUG	Conn #0: Calling asyncClose().
09:38:56     INFO -  Services.HealthReport.HealthReporter	WARN	No prefs data found.
09:38:56     INFO -  Sqlite.ConnectionOpener	INFO	Opening database: c:\\users\\cltbld~1.t-w\\appdata\\local\\temp\\tmpxhh6jw\\state_create_client_id (state_create_client_id#1)
09:38:56     INFO -  [4904] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80630001: file c:/builds/moz2_slave/m-aurora-w32-d-000000000000000/build/storage/src/mozStorageConnection.cpp, line 601
09:38:56     INFO -  Sqlite.ConnectionOpener	WARN	Could not open connection: 2153971713
09:38:56     INFO -  Sqlite.ConnectionOpener	INFO	Connection opened
09:38:56     INFO -  Sqlite.ConnectionOpener	WARN	Could not open database: TypeError: connection is null (resource://gre/modules/Sqlite.jsm:190) JS Stack trace: openConnection/<@Sqlite.jsm:190:1 < _do_main@C:\\slave\\test\\build\\tests\\xpcshell\\head.js:191:5 < _execute_test@C:\\slave\\test\\build\\tests\\xpcshell\\head.js:405:5 < @-e:1:1
09:38:56     INFO -  Assertion failure: !mAsyncExecutionThread (AsyncClose has not been invoked on this connection!), at c:/builds/moz2_slave/m-aurora-w32-d-000000000000000/build/storage/src/mozStorageConnection.cpp:491
09:38:56     INFO -  NS_InvokeByIndex+0x004B6DDE [xul +0x000000000054B204]
09:38:56     INFO -  NS_InvokeByIndex+0x004B76AE [xul +0x000000000054BAD4]
09:38:56     INFO -  NS_InvokeByIndex+0x004B8859 [xul +0x000000000054CC7F]
09:38:56     INFO -  NS_InvokeByIndex+0x004B8986 [xul +0x000000000054CDAC]
09:38:56     INFO -  NS_InvokeByIndex+0x004B8B6A [xul +0x000000000054CF90]
09:38:56     INFO -  NS_InvokeByIndex+0x004ABCCD [xul +0x00000000005400F3]
09:38:56     INFO -  mozilla::services::_external_GetHistoryService+0x00007086 [xul +0x000000000001D9F0]
09:38:56     INFO -  XRE_AddJarManifestLocation+0x0000A353 [xul +0x000000000008C9DD]
09:38:56     INFO -  NS_InvokeByIndex+0x00000027 [xul +0x000000000009444D]
09:38:56     INFO -  XRE_XPCShellMain+0x0000C07F [xul +0x0000000000DD5553]
09:38:56     INFO -  XRE_XPCShellMain+0x0001066E [xul +0x0000000000DD9B42]
09:38:56     INFO -  XRE_XPCShellMain+0x00014518 [xul +0x0000000000DDD9EC]
09:38:56     INFO -  UNKNOWN 0x0000000002DE7F79
09:38:56     INFO -  UNKNOWN 0x000000000A319478
09:38:56     INFO -  UNKNOWN 0x000000003FE40981
09:38:56     INFO -  JS_GetTraceThingInfo+0x000FE830 [mozjs +0x000000000021DCB0]
09:38:56     INFO -  JS_GetTraceThingInfo+0x000FEB8C [mozjs +0x000000000021E00C]
09:38:56     INFO -  JS_DefineDebuggerObject+0x0001CD7C [mozjs +0x000000000051772C]
09:38:56     INFO -  JS_DefineDebuggerObject+0x00022CE3 [mozjs +0x000000000051D693]
09:38:56     INFO -  JS_DefineDebuggerObject+0x00022FB7 [mozjs +0x000000000051D967]
09:38:56     INFO -  JS_DefineDebuggerObject+0x00023256 [mozjs +0x000000000051DC06]
09:38:56     INFO -  JS_GetTraceThingInfo+0x000F7B1F [mozjs +0x0000000000216F9F]
09:38:56     INFO -  UNKNOWN 0x000000003FE467DA
09:38:56     INFO -  UNKNOWN 0x0000000008E28C80
09:38:56     INFO -  UNKNOWN 0x000000003FE40981
09:38:56     INFO -  JS_GetTraceThingInfo+0x000FE830 [mozjs +0x000000000021DCB0]
09:38:56     INFO -  JS_GetTraceThingInfo+0x000FEB8C [mozjs +0x000000000021E00C]
09:38:56     INFO -  JS_DefineDebuggerObject+0x0001CD7C [mozjs +0x000000000051772C]
09:38:56     INFO -  JS_DefineDebuggerObject+0x00022CE3 [mozjs +0x000000000051D693]
09:38:56     INFO -  JS_DefineDebuggerObject+0x00023877 [mozjs +0x000000000051E227]
09:38:56     INFO -  JS_ExecuteScriptVersion+0x000001D1 [mozjs +0x00000000003B4E11]
09:38:56     INFO -  JS_ExecuteScriptVersion+0x00000368 [mozjs +0x00000000003B4FA8]
09:38:56     INFO -  JS_EvaluateScript+0x0000003D [mozjs +0x00000000003B526D]
09:38:56     INFO -  imgLoader::SupportImageWithMimeType+0x005993D1 [xul +0x0000000000DC9357]
09:38:56     INFO -  XRE_XPCShellMain+0x00000BFD [xul +0x0000000000DCA0D1]
09:38:56     INFO -  UNKNOWN [xpcshell +0x000000000000136E]
09:38:56     INFO -  UNKNOWN [xpcshell +0x0000000000001528]
09:38:56     INFO -  BaseThreadInitThunk+0x0000000E [KERNEL32 +0x0000000000028543]
09:38:56     INFO -  RtlInitializeExceptionChain+0x00000085 [ntdll +0x000000000005AC69]
09:38:56     INFO -  RtlInitializeExceptionChain+0x00000058 [ntdll +0x000000000005AC3C]
09:38:56     INFO -  <<<<<<<
09:39:03  WARNING -  PROCESS-CRASH | C:\slave\test\build\tests\xpcshell\tests\services\healthreport\tests\xpcshell\test_healthreporter.js | application crashed [@ mozilla::storage::Connection::~Connection()]
09:39:03     INFO -  Crash dump filename: c:\users\cltbld~1.t-w\appdata\local\temp\tmpuwmqfg\6dd125e3-0a47-411e-9d94-edd10a95b0d0.dmp
09:39:03     INFO -  Operating system: Windows NT
09:39:03     INFO -                    6.2.9200
09:39:03     INFO -  CPU: x86
09:39:03     INFO -       GenuineIntel family 6 model 30 stepping 5
09:39:03     INFO -       8 CPUs
09:39:03     INFO -  Crash reason:  EXCEPTION_BREAKPOINT
09:39:03     INFO -  Crash address: 0x725fe531
09:39:03     INFO -  Thread 0 (crashed)
09:39:03     INFO -   0  xul.dll!mozilla::storage::Connection::~Connection() [mozStorageConnection.cpp:d740cc08d966 : 491 + 0x1d]
09:39:03     INFO -      eip = 0x725fe531   esp = 0x0105e430   ebp = 0x0105e448   ebx = 0x0ace3740
09:39:03     INFO -      esi = 0x0ace3740   edi = 0x00000000   eax = 0x00000000   ecx = 0x7482ff12
09:39:03     INFO -      edx = 0x0105c818   efl = 0x00200216
09:39:03     INFO -      Found by: given as instruction pointer in context
09:39:03     INFO -   1  xul.dll!mozilla::storage::Connection::Release() [mozStorageConnection.cpp:d740cc08d966 : 520 + 0x6]
09:39:03     INFO -      eip = 0x7260010c   esp = 0x0105e43c   ebp = 0x0105e448
09:39:03     INFO -      Found by: call frame info
09:39:03     INFO -   2  xul.dll!nsRefPtr<mozilla::storage::Connection>::`scalar deleting destructor'(unsigned int) + 0x11
09:39:03     INFO -      eip = 0x7260b204   esp = 0x0105e450   ebp = 0x0105e458
09:39:03     INFO -      Found by: call frame info
09:39:03     INFO -   3  xul.dll!nsTArray_Impl<nsRefPtr<mozilla::storage::Connection>,nsTArrayInfallibleAllocator>::DestructRange(unsigned int,unsigned int) [nsTArray.h:d740cc08d966 : 1583 + 0x8]
09:39:03     INFO -      eip = 0x7260bad4   esp = 0x0105e460   ebp = 0x0105e46c
09:39:03     INFO -      Found by: call frame info
09:39:03     INFO -   4  xul.dll!nsTArray_Impl<nsRefPtr<mozilla::storage::Connection>,nsTArrayInfallibleAllocator>::RemoveElementsAt(unsigned int,unsigned int) [nsTArray.h:d740cc08d966 : 1300 + 0x8]
09:39:03     INFO -      eip = 0x7260cc7f   esp = 0x0105e474   ebp = 0x0105e488
09:39:03     INFO -      Found by: call frame info
09:39:03     INFO -   5  xul.dll!nsTArray_Impl<nsRefPtr<mozilla::storage::Connection>,nsTArrayInfallibleAllocator>::RemoveElement<mozilla::storage::Connection *,nsDefaultComparator<nsRefPtr<mozilla::storage::Connection>,mozilla::storage::Connection *> >(mozilla::storage::Connection * const &,nsDefaultComparator<nsRefPtr<mozilla::storage::Connection>,mozilla::storage::Connection *> const &) [nsTArray.h:d740cc08d966 : 1325 + 0x7]
09:39:03     INFO -      eip = 0x7260cdac   esp = 0x0105e490   ebp = 0x0105e498
09:39:03     INFO -      Found by: call frame info
09:39:03     INFO -   6  xul.dll!mozilla::storage::Service::unregisterConnection(mozilla::storage::Connection *) [mozStorageService.cpp:d740cc08d966 : 326 + 0x15]
09:39:03     INFO -      eip = 0x7260cf90   esp = 0x0105e4a0   ebp = 0x0105e4c0
09:39:03     INFO -      Found by: call frame info
09:39:03     INFO -   7  xul.dll!mozilla::storage::Connection::Release() [mozStorageConnection.cpp:d740cc08d966 : 514 + 0xf]
09:39:03     INFO -      eip = 0x726000f3   esp = 0x0105e4c8   ebp = 0x0105e4d8
09:39:03     INFO -      Found by: call frame info
09:39:03     INFO -   8  xul.dll!nsProxyReleaseEvent::Run() [nsProxyRelease.cpp:d740cc08d966 : 19 + 0xb]
09:39:03     INFO -      eip = 0x720dd9f0   esp = 0x0105e4e0   ebp = 0x0105e4e4
09:39:03     INFO -      Found by: call frame info

Comment 1

[David Teller [:Yoric] - still alive but not very active]

Attachment: Adding missing `return`

That probably won't solve the entire issue, but it's still worth fixing.

Comment 2

[David Teller [:Yoric] - still alive but not very active]

I suspect that the issue is actually in the test suite. Bug 939072 should tell us exactly which connection fails.

Comment 3

[Legacy TBPL/Treeherder Robot]

Tomcat
https://tbpl.mozilla.org/php/getParsedLog.php?id=41863364&tree=Mozilla-Inbound
WINNT 6.2 mozilla-inbound debug test xpcshell on 2014-06-17 04:33:49
revision: 4426be8be94c
slave: t-w864-ix-088

TEST-UNEXPECTED-FAIL | C:\slave\test\build\tests\xpcshell\tests\services\healthreport\tests\xpcshell\test_healthreporter.js | test failed (with xpcshell return code: -2147483645), see following log:
Assertion failure: !mAsyncExecutionThread (AsyncClose has not been invoked on this connection!), at c:/builds/moz2_slave/m-in-w32-d-0000000000000000000/build/storage/src/mozStorageConnection.cpp:491
PROCESS-CRASH | C:\slave\test\build\tests\xpcshell\tests\services\healthreport\tests\xpcshell\test_healthreporter.js | application crashed [@ mozilla::storage::Connection::~Connection()]
Return code: 1

Comment 4

[Eitan Adler]

This is a duplicate of bug #994484

Comment 5

[Legacy TBPL/Treeherder Robot]

Tomcat
https://tbpl.mozilla.org/php/getParsedLog.php?id=43800326&tree=Mozilla-Inbound
WINNT 6.2 mozilla-inbound debug test xpcshell on 2014-07-14 22:58:20
revision: 58de0be65bc2
slave: t-w864-ix-032

TEST-UNEXPECTED-FAIL | C:\slave\test\build\tests\xpcshell\tests\services\healthreport\tests\xpcshell\test_healthreporter.js | test failed (with xpcshell return code: -2147483645), see following log:
Assertion failure: !mAsyncExecutionThread (AsyncClose has not been invoked on this connection!), at c:/builds/moz2_slave/m-in-w32-d-0000000000000000000/build/storage/src/mozStorageConnection.cpp:491
PROCESS-CRASH | C:\slave\test\build\tests\xpcshell\tests\services\healthreport\tests\xpcshell\test_healthreporter.js | application crashed [@ mozilla::storage::Connection::~Connection()]
Return code: 1

Comment 6

[Legacy TBPL/Treeherder Robot]

RyanVM
https://tbpl.mozilla.org/php/getParsedLog.php?id=44363939&tree=Mozilla-Inbound
WINNT 6.2 mozilla-inbound debug test xpcshell on 2014-07-22 09:55:27
revision: db1ceb8f8d92
slave: t-w864-ix-014

TEST-UNEXPECTED-FAIL | C:\slave\test\build\tests\xpcshell\tests\services\healthreport\tests\xpcshell\test_healthreporter.js | test failed (with xpcshell return code: -2147483645), see following log:
Assertion failure: !mAsyncExecutionThread (AsyncClose has not been invoked on this connection!), at c:/builds/moz2_slave/m-in-w32-d-0000000000000000000/build/storage/src/mozStorageConnection.cpp:491
PROCESS-CRASH | C:\slave\test\build\tests\xpcshell\tests\services\healthreport\tests\xpcshell\test_healthreporter.js | application crashed [@ mozilla::storage::Connection::~Connection()]
Return code: 1

Comment 7

[Legacy TBPL/Treeherder Robot]

:nigelb
https://tbpl.mozilla.org/php/getParsedLog.php?id=44635902&tree=Mozilla-Central
WINNT 6.2 mozilla-central debug test xpcshell on 2014-07-25 17:15:25
revision: e07264876182
slave: t-w864-ix-115

TEST-UNEXPECTED-FAIL | C:\slave\test\build\tests\xpcshell\tests\services\healthreport\tests\xpcshell\test_healthreporter.js | test failed (with xpcshell return code: -2147483645), see following log:
Assertion failure: !mAsyncExecutionThread (AsyncClose has not been invoked on this connection!), at c:/builds/moz2_slave/m-cen-w32-d-000000000000000000/build/storage/src/mozStorageConnection.cpp:491
PROCESS-CRASH | C:\slave\test\build\tests\xpcshell\tests\services\healthreport\tests\xpcshell\test_healthreporter.js | application crashed [@ mozilla::storage::Connection::~Connection()]
Return code: 1

Comment 8

[Marco Bonardo [:mak]]

the patch here never landed, I guess.

Comment 9

[Legacy TBPL/Treeherder Robot]

:nigelb
https://tbpl.mozilla.org/php/getParsedLog.php?id=45905412&tree=Mozilla-Inbound
WINNT 6.2 mozilla-inbound debug test xpcshell on 2014-08-13 20:54:20
revision: 54d7e61b8913
slave: t-w864-ix-115

<bound method DesktopUnittest.run_command of <__main__.DesktopUnittest object at 0x02712F50>> failed after 1 tries!
<bound method DesktopUnittest.run_command of <__main__.DesktopUnittest object at 0x02712F50>> failed after 1 tries!
TEST-UNEXPECTED-FAIL | C:\slave\test\build\tests\xpcshell\tests\services\healthreport\tests\xpcshell\test_healthreporter.js | test failed (with xpcshell return code: -2147483645), see following log:
Assertion failure: !mAsyncExecutionThread (AsyncClose has not been invoked on this connection!), at c:/builds/moz2_slave/m-in-w32-d-0000000000000000000/build/storage/src/mozStorageConnection.cpp:491
PROCESS-CRASH | C:\slave\test\build\tests\xpcshell\tests\services\healthreport\tests\xpcshell\test_healthreporter.js | application crashed [@ mozilla::storage::Connection::~Connection()]
Return code: 1

Comment 10

[Legacy TBPL/Treeherder Robot]

RyanVM
https://tbpl.mozilla.org/php/getParsedLog.php?id=45994509&tree=Mozilla-Central
WINNT 6.2 mozilla-central debug test xpcshell on 2014-08-14 18:44:20
revision: dfc27900a9f6
slave: t-w864-ix-048

TEST-UNEXPECTED-FAIL | C:\slave\test\build\tests\xpcshell\tests\services\healthreport\tests\xpcshell\test_healthreporter.js | test failed (with xpcshell return code: -2147483645), see following log:
Assertion failure: !mAsyncExecutionThread (AsyncClose has not been invoked on this connection!), at c:/builds/moz2_slave/m-cen-w32-d-000000000000000000/build/storage/src/mozStorageConnection.cpp:491
PROCESS-CRASH | C:\slave\test\build\tests\xpcshell\tests\services\healthreport\tests\xpcshell\test_healthreporter.js | application crashed [@ mozilla::storage::Connection::~Connection()]
Return code: 1

Comment 11

[Legacy TBPL/Treeherder Robot]

RyanVM
https://tbpl.mozilla.org/php/getParsedLog.php?id=46115317&tree=Mozilla-Central
WINNT 6.2 mozilla-central debug test xpcshell on 2014-08-16 16:50:30
revision: 94ba78a42305
slave: t-w864-ix-075

TEST-UNEXPECTED-FAIL | C:\slave\test\build\tests\xpcshell\tests\services\healthreport\tests\xpcshell\test_healthreporter.js | test failed (with xpcshell return code: -2147483645), see following log:
Assertion failure: !mAsyncExecutionThread (AsyncClose has not been invoked on this connection!), at c:/builds/moz2_slave/m-cen-w32-d-000000000000000000/build/storage/src/mozStorageConnection.cpp:491
PROCESS-CRASH | C:\slave\test\build\tests\xpcshell\tests\services\healthreport\tests\xpcshell\test_healthreporter.js | application crashed [@ mozilla::storage::Connection::~Connection()]
Return code: 1

Comment 12

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/integration/fx-team/rev/18310d618bb7

Comment 13

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/18310d618bb7

Comment 14

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/bb86cb586fa9
https://hg.mozilla.org/releases/mozilla-beta/rev/da78e23cbe3d
https://hg.mozilla.org/releases/mozilla-esr31/rev/ffdd79d07939
https://hg.mozilla.org/releases/mozilla-b2g30_v1_4/rev/e6ecf07733ff

Comment 15

[Ryan VanderMeulen [:RyanVM]]

Copy/paste error on Aurora. The proper rev was:
https://hg.mozilla.org/releases/mozilla-aurora/rev/b0b83df25858

Comment 16

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-b2g32_v2_0&job_id=123073
repository: mozilla-b2g32_v2_0
start_time: 2015-04-24T11:21:04
who: rvandermeulen[at]mozilla[dot]com
machine: t-w732-ix-052
buildname: Windows 7 32-bit mozilla-b2g32_v2_0 debug test xpcshell
revision: d9dcc636b08e

TEST-UNEXPECTED-FAIL | C:\slave\test\build\tests\xpcshell\tests\services\healthreport\tests\xpcshell\test_healthreporter.js | test failed (with xpcshell return code: -2147483645), see following log:
Assertion failure: !mAsyncExecutionThread (AsyncClose has not been invoked on this connection!), at c:/builds/moz2_slave/m-b32_20-w32-d-000000000000000/build/storage/src/mozStorageConnection.cpp:491
PROCESS-CRASH | C:\slave\test\build\tests\xpcshell\tests\services\healthreport\tests\xpcshell\test_healthreporter.js | application crashed [@ mozilla::storage::Connection::~Connection()]
Return code: 1