Intermittent browser_alarms.js,test_webgl_request_mismatch.html | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]

RESOLVED FIXED in Firefox 58

Status

()

defect
P3
normal
RESOLVED FIXED
5 years ago
Last year

People

(Reporter: cbook, Assigned: Yoric)

Tracking

(Blocks 1 bug, {crash, intermittent-failure})

Trunk
mozilla58
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox-esr52 wontfix, firefox56 wontfix, firefox57 wontfix, firefox58 fixed)

Details

(Whiteboard: [necko-backlog][stockwell fixed:other], crash signature, )

Attachments

(1 attachment)

Windows 7 32-bit mozilla-central opt test mochitest-browser-chrome-2

https://treeherder.mozilla.org/ui/logviewer.html#?job_id=729651&repo=mozilla-central

05:08:22 WARNING - PROCESS-CRASH | chrome://mochitests/content/browser/hal/tests/browser_alarms.js | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)] 
5:08:22 INFO - Crash dump filename: c:\users\cltbld\appdata\local\temp\tmppamfqt.mozrunner\minidumps\fae4ff76-b276-4974-baed-09e30a9c8aa8.dmp
05:08:22 INFO - Operating system: Windows NT
05:08:22 INFO - 6.1.7601 Service Pack 1
05:08:22 INFO - CPU: x86
05:08:22 INFO - GenuineIntel family 6 model 30 stepping 5
05:08:22 INFO - 8 CPUs
05:08:22 INFO - Crash reason: EXCEPTION_BREAKPOINT
05:08:22 INFO - Crash address: 0x62fe481b
05:08:22 INFO - Thread 42 (crashed)
05:08:22 INFO - 0 xul.dll!mozilla::`anonymous namespace'::RunWatchdog(void *) [nsTerminator.cpp:035a951fc24a : 151 + 0x0]
05:08:22 INFO - eip = 0x62fe481b esp = 0x1735ff18 ebp = 0x1735ff1c ebx = 0x002cf0d8
05:08:22 INFO - esi = 0x0000003f edi = 0x01d14d00 eax = 0x0000003f ecx = 0x63e70c80
05:08:22 INFO - edx = 0x0000001b efl = 0x00000246
05:08:22 INFO - Found by: given as instruction pointer in context
05:08:22 INFO - 1 nss3.dll!_PR_NativeRunThread [pruthr.c:035a951fc24a : 397 + 0x7]
05:08:22 INFO - eip = 0x675c6718 esp = 0x1735ff24 ebp = 0x1735ff38
05:08:22 INFO - Found by: call frame info
05:08:22 INFO - 2 nss3.dll!pr_root [w95thred.c:035a951fc24a : 90 + 0xb]
05:08:22 INFO - eip = 0x675b921f esp = 0x1735ff40 ebp = 0x1735ff44
05:08:22 INFO - Found by: call frame info
05:08:22 INFO - 3 msvcr120.dll + 0x2c01c
05:08:22 INFO - eip = 0x6764c01d esp = 0x1735ff4c ebp = 0x1735ff7c
05:08:22 INFO - Found by: call frame info
05:08:22 INFO - 4 msvcr120.dll + 0x2c000
05:08:22 INFO - eip = 0x6764c001 esp = 0x1735ff84 ebp = 0x1735ff88
05:08:22 INFO - Found by: previous frame's frame pointer
05:08:22 INFO - 5 kernel32.dll + 0x53c44
05:08:22 INFO - eip = 0x77453c45 esp = 0x1735ff90 ebp = 0x1735ff94
05:08:22 INFO - Found by: previous frame's frame pointer
05:08:22 INFO - 6 ntdll.dll + 0x637f4
05:08:22 INFO - eip = 0x779c37f5 esp = 0x1735ff9c ebp = 0x1735ffd4
05:08:22 INFO - Found by: previous frame's frame pointer
05:08:22 INFO - 7 ntdll.dll + 0x637c7
OS: Windows 7 → All
Hardware: x86 → All
Summary: Intermittent browser_alarms.js | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)] → Intermittent browser_alarms.js,test_webgl_request_mismatch.html | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
Maybe the ASAN stack sheds some light on this?
See Also: → 1113595
Component: General → Networking: Cache
See Also: → 1121145
Whiteboard: [necko-backlog]
Bulk assigning P3 to all open intermittent bugs without a priority set in Firefox components per bug 1298978.
Priority: -- → P3
this picked up, 50 failures in the last week, looks like June 5th/6th was a new rate.  Primarily on linux32/opt (non-e10s)

from this log file:
https://treeherder.mozilla.org/logviewer.html#?repo=autoland&job_id=105826337&lineNumber=3065

I see this in the log file:
[task 2017-06-09T13:41:43.056013Z] 13:41:43     INFO - TEST-START | browser/components/sessionstore/test/browser_windowStateContainer.js
[task 2017-06-09T13:41:52.232132Z] 13:41:52     INFO - GECKO(1703) | MEMORY STAT | vsize 1142MB | residentFast 377MB | heapAllocated 172MB
[task 2017-06-09T13:41:52.234193Z] 13:41:52     INFO - TEST-OK | browser/components/sessionstore/test/browser_windowStateContainer.js | took 9176ms
[task 2017-06-09T13:41:52.312716Z] 13:41:52     INFO - checking window state
[task 2017-06-09T13:41:59.565830Z] 13:41:59     INFO - GECKO(1703) | Completed ShutdownLeaks collections in process 1703
[task 2017-06-09T13:41:59.567569Z] 13:41:59     INFO - TEST-START | Shutdown
[task 2017-06-09T13:41:59.570925Z] 13:41:59     INFO - Browser Chrome Test Summary
[task 2017-06-09T13:41:59.572780Z] 13:41:59     INFO - Passed:  1164
[task 2017-06-09T13:41:59.574455Z] 13:41:59     INFO - Failed:  0
[task 2017-06-09T13:41:59.576153Z] 13:41:59     INFO - Todo:    0
[task 2017-06-09T13:41:59.577929Z] 13:41:59     INFO - Mode:    non-e10s
[task 2017-06-09T13:41:59.583430Z] 13:41:59     INFO - *** End BrowserChrome Test Results ***
[task 2017-06-09T13:42:27.029268Z] 13:42:27     INFO - GECKO(1703) | [1703] WARNING: waitpid failed pid:1769 errno:10: file /home/worker/workspace/build/src/ipc/chromium/src/base/process_util_posix.cc, line 276
[task 2017-06-09T13:43:30.897461Z] 13:43:30     INFO - GECKO(1703) | ExceptionHandler::GenerateDump cloned child 1794
[task 2017-06-09T13:43:30.900046Z] 13:43:30     INFO - GECKO(1703) | ExceptionHandler::SendContinueSignalToChild sent continue signal to child
[task 2017-06-09T13:43:30.903394Z] 13:43:30     INFO - GECKO(1703) | ExceptionHandler::WaitForContinueSignal waiting for continue signal...
[task 2017-06-09T13:43:31.443716Z] 13:43:31     INFO - TEST-INFO | Main app process: exit 11
[task 2017-06-09T13:43:31.445760Z] 13:43:31     INFO - Buffered messages finished
[task 2017-06-09T13:43:31.447364Z] 13:43:31    ERROR - TEST-UNEXPECTED-FAIL | browser/components/sessionstore/test/browser_windowStateContainer.js | application terminated with exit code 11
[task 2017-06-09T13:43:31.450202Z] 13:43:31     INFO - runtests.py | Application ran for: 0:07:58.257090
[task 2017-06-09T13:43:31.454019Z] 13:43:31     INFO - zombiecheck | Reading PID log: /tmp/tmpvsQN3opidlog
[task 2017-06-09T13:43:31.455943Z] 13:43:31     INFO - ==> process 1703 launched child process 1722
[task 2017-06-09T13:43:31.458179Z] 13:43:31     INFO - ==> process 1703 launched child process 1769
[task 2017-06-09T13:43:31.461059Z] 13:43:31     INFO - zombiecheck | Checking for orphan process with PID: 1722
[task 2017-06-09T13:43:31.463053Z] 13:43:31     INFO - zombiecheck | Checking for orphan process with PID: 1769
[task 2017-06-09T13:43:31.465148Z] 13:43:31     INFO - mozcrash Downloading symbols from: https://queue.taskcluster.net/v1/task/eD7DKr2pSzesOXbz2HeMXA/artifacts/public/build/target.crashreporter-symbols.zip
[task 2017-06-09T13:43:47.916554Z] 13:43:47     INFO - mozcrash Copy/paste: /usr/local/bin/linux64-minidump_stackwalk /tmp/tmpD_xLTd.mozrunner/minidumps/133ef873-6fa0-403d-463f-f713cdff9012.dmp /tmp/tmpaSKT7V
[task 2017-06-09T13:44:09.925072Z] 13:44:09     INFO - mozcrash Saved minidump as /home/worker/workspace/build/blobber_upload_dir/133ef873-6fa0-403d-463f-f713cdff9012.dmp
[task 2017-06-09T13:44:09.942381Z] 13:44:09     INFO - mozcrash Saved app info as /home/worker/workspace/build/blobber_upload_dir/133ef873-6fa0-403d-463f-f713cdff9012.extra
[task 2017-06-09T13:44:10.051103Z] 13:44:10     INFO - PROCESS-CRASH | browser/components/sessionstore/test/browser_windowStateContainer.js | application crashed [@ RunWatchdog]
[task 2017-06-09T13:44:10.053834Z] 13:44:10     INFO - Crash dump filename: /tmp/tmpD_xLTd.mozrunner/minidumps/133ef873-6fa0-403d-463f-f713cdff9012.dmp
[task 2017-06-09T13:44:10.058048Z] 13:44:10     INFO - Operating system: Linux
[task 2017-06-09T13:44:10.059866Z] 13:44:10     INFO -                   0.0.0 Linux 3.13.0-100-generic #147-Ubuntu SMP Tue Oct 18 16:48:51 UTC 2016 x86_64
[task 2017-06-09T13:44:10.061438Z] 13:44:10     INFO - CPU: x86
[task 2017-06-09T13:44:10.063053Z] 13:44:10     INFO -      GenuineIntel family 6 model 45 stepping 7
[task 2017-06-09T13:44:10.064814Z] 13:44:10     INFO -      1 CPU
[task 2017-06-09T13:44:10.066397Z] 13:44:10     INFO - 
[task 2017-06-09T13:44:10.067941Z] 13:44:10     INFO - GPU: UNKNOWN
[task 2017-06-09T13:44:10.069592Z] 13:44:10     INFO - 
[task 2017-06-09T13:44:10.072581Z] 13:44:10     INFO - Crash reason:  SIGSEGV
[task 2017-06-09T13:44:10.074162Z] 13:44:10     INFO - Crash address: 0x0
[task 2017-06-09T13:44:10.075716Z] 13:44:10     INFO - Process uptime: not available
[task 2017-06-09T13:44:10.078003Z] 13:44:10     INFO - 
[task 2017-06-09T13:44:10.079623Z] 13:44:10     INFO - Thread 14 (crashed)
[task 2017-06-09T13:44:10.082188Z] 13:44:10     INFO -  0  libxul.so!RunWatchdog [nsTerminator.cpp:0a3b78002ba3 : 160 + 0x2]
[task 2017-06-09T13:44:10.083905Z] 13:44:10     INFO -     eip = 0xf38654fd   esp = 0xc30fe2f0   ebp = 0xc30fe318   ebx = 0xf5615000
[task 2017-06-09T13:44:10.086096Z] 13:44:10     INFO -     esi = 0x0000003f   edi = 0xbef2f574   eax = 0x00000000   ecx = 0x00000000
[task 2017-06-09T13:44:10.088787Z] 13:44:10     INFO -     edx = 0xf47a8408   efl = 0x00010246
[task 2017-06-09T13:44:10.090974Z] 13:44:10     INFO -     Found by: given as instruction pointer in context
[task 2017-06-09T13:44:10.093279Z] 13:44:10     INFO -  1  libnspr4.so!_pt_root [ptthread.c:0a3b78002ba3 : 216 + 0x9]
[task 2017-06-09T13:44:10.098195Z] 13:44:10     INFO -     eip = 0xf73b4c44   esp = 0xc30fe320   ebp = 0xc30fe368   ebx = 0xf73c56c4
[task 2017-06-09T13:44:10.099791Z] 13:44:10     INFO -     esi = 0xbf045d40   edi = 0x000006fc
[task 2017-06-09T13:44:10.101360Z] 13:44:10     INFO -     Found by: call frame info
[task 2017-06-09T13:44:10.102954Z] 13:44:10     INFO -  2  libpthread-2.23.so + 0x6295
[task 2017-06-09T13:44:10.104620Z] 13:44:10     INFO -     eip = 0xf777f295   esp = 0xc30fe370   ebp = 0xc30fe428   ebx = 0x00000000
[task 2017-06-09T13:44:10.106366Z] 13:44:10     INFO -     esi = 0x00000000   edi = 0x003d0f00
[task 2017-06-09T13:44:10.107927Z] 13:44:10     INFO -     Found by: call frame info
[task 2017-06-09T13:44:10.109510Z] 13:44:10     INFO -  3  libc-2.23.so + 0xe6eee
[task 2017-06-09T13:44:10.111173Z] 13:44:10     INFO -     eip = 0xf74b1eee   esp = 0xc30fe430   ebp = 0x00000000
[task 2017-06-09T13:44:10.114172Z] 13:44:10     INFO -     Found by: previous frame's frame pointer
[task 2017-06-09T13:44:10.115711Z] 13:44:10     INFO - 
[task 2017-06-09T13:44:10.117257Z] 13:44:10     INFO - Thread 0
[task 2017-06-09T13:44:10.119043Z] 13:44:10     INFO -  0  libxul.so!XPCRootSetElem::RemoveFromRootSet [XPCJSRuntime.cpp:0a3b78002ba3 : 3171 + 0x5]
[task 2017-06-09T13:44:10.122247Z] 13:44:10     INFO -     eip = 0xf0c874eb   esp = 0xffbed7d0   ebp = 0xffbed7e8   ebx = 0xf5615000
[task 2017-06-09T13:44:10.123895Z] 13:44:10     INFO -     esi = 0xcee9a958   edi = 0xcee9a940   eax = 0xceee6058   ecx = 0xf71352cc
[task 2017-06-09T13:44:10.125572Z] 13:44:10     INFO -     edx = 0xdf1fc898   efl = 0x00010286
[task 2017-06-09T13:44:10.127406Z] 13:44:10     INFO -     Found by: given as instruction pointer in context
[task 2017-06-09T13:44:10.129738Z] 13:44:10     INFO -  1  libxul.so!nsXPCWrappedJS::Release [XPCWrappedJS.cpp:0a3b78002ba3 : 281 + 0xe]
[task 2017-06-09T13:44:10.133705Z] 13:44:10     INFO -     eip = 0xf0ca93a5   esp = 0xffbed7f0   ebp = 0xffbed838   ebx = 0xf5615000
[task 2017-06-09T13:44:10.135440Z] 13:44:10     INFO -     esi = 0xcee9a948   edi = 0xcee9a940
[task 2017-06-09T13:44:10.137009Z] 13:44:10     INFO -     Found by: call frame info
[task 2017-06-09T13:44:10.138695Z] 13:44:10     INFO -  2  libxul.so!nsXPTCStubBase::Release [xptcall.cpp:0a3b78002ba3 : 37 + 0xe]
[task 2017-06-09T13:44:10.140335Z] 13:44:10     INFO -     eip = 0xf031ba04   esp = 0xffbed840   ebp = 0xffbed858   ebx = 0xf5615000
[task 2017-06-09T13:44:10.141965Z] 13:44:10     INFO -     esi = 0xcc8ec220   edi = 0xcc8ec218
[task 2017-06-09T13:44:10.143689Z] 13:44:10     INFO -     Found by: call frame info
[task 2017-06-09T13:44:10.146701Z] 13:44:10     INFO -  3  libxul.so!nsTArray_Impl<ObserverRef, nsTArrayInfallibleAllocator>::RemoveElementsAt [nsCOMPtr.h:0a3b78002ba3 : 294 + 0x8]
[task 2017-06-09T13:44:10.148483Z] 13:44:10     INFO -     eip = 0xf02c626f   esp = 0xffbed860   ebp = 0xffbed888   ebx = 0xf5615000
[task 2017-06-09T13:44:10.150144Z] 13:44:10     INFO -     esi = 0xcc8ec220   edi = 0xcc8ec218
[task 2017-06-09T13:44:10.152087Z] 13:44:10     INFO -     Found by: call frame info
[task 2017-06-09T13:44:10.154216Z] 13:44:10     INFO -  4  libxul.so!nsTHashtable<nsObserverList>::s_ClearEntry [nsTArray.h:0a3b78002ba3 : 1738 + 0x12]
[task 2017-06-09T13:44:10.157955Z] 13:44:10     INFO -     eip = 0xf02c658b   esp = 0xffbed890   ebp = 0xffbed8b8   ebx = 0xf5615000
[task 2017-06-09T13:44:10.159553Z] 13:44:10     INFO -     esi = 0xe36132b4   edi = 0xe36132bc
[task 2017-06-09T13:44:10.161115Z] 13:44:10     INFO -     Found by: call frame info


so we are failing on shutdown?


:mcmanus, can you help find someone to look into this failure in the next 2 weeks?
Flags: needinfo?(mcmanus)
Whiteboard: [necko-backlog] → [necko-backlog][stockwell needswork]
Hey Joel - so I don't see why this was marked a cache bug 2 years ago. I took a look at the following

https://public-artifacts.taskcluster.net/S95MYI02S8Sg1Dn2k7vOOg/0/public/logs/live_backing.log
https://treeherder.mozilla.org/logviewer.html#?repo=autoland&job_id=105826337&lineNumber=3065
https://public-artifacts.taskcluster.net/ABSg1wEAR0agSqyDcUgXKA/0/public/logs/live_backing.log

and all of them are processing shutdown.. in all of them the networking thread has completed and I don't easily see any cache thread either. I'll NI honza to confirm the networking cache is not involved.

I'm not sure where to reassign either.. none of them look deadlocked, just usually doing some js gc cleanup at different stages.
Flags: needinfo?(mcmanus) → needinfo?(honzab.moz)
Confirming the shutdown crashes (the only UNEXPECTED-FAIL I was able to find in those 3 logs) are not related to http cache.
Flags: needinfo?(honzab.moz)
Mike- this is always occurring at the end of sessionrestore, could you help find someone on the sessionrestore team to look into why we are crashing often on linux32/opt non-e10s browser-chrome tests for sessionrestore?
Component: Networking: Cache → Session Restore
Flags: needinfo?(mdeboer)
Product: Core → Firefox
Why are you sure that this is occurring due to sessionstore? What do you mean with 'end of sessionrestore'?
When I look at the brasstacks view, it looks like bug 1373116, but in disguise. Let's ask Yoshi again!
Flags: needinfo?(mdeboer) → needinfo?(allstars.chh)
(In reply to Mike de Boer [:mikedeboer] from comment #86)
> Why are you sure that this is occurring due to sessionstore? What do you
> mean with 'end of sessionrestore'?
> When I look at the brasstacks view, it looks like bug 1373116, but in
> disguise. Let's ask Yoshi again!

Most of the linux32 opt failures reported around June 14 are nsTerminator shutdown crashes which occur at the end of the browser/components/sessionstore/test tests. That is indeed just like bug 1373116 (I think the bug 1373116 failures were reported here until bug 1373116 was opened).

Beginning around June 19, we started seeing similar failures on Windows reported here, but those do not seem to be associated with browser/components/sessionstore/test, or any other particular test directory.
this appears to be fixed
Flags: needinfo?(allstars.chh)
Whiteboard: [necko-backlog][stockwell needswork] → [necko-backlog][stockwell unknown]
August 28th, this started failing, 21 times since then:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587

primarily on win7/debug and linux64/asan configurations.
Duplicate of this bug: 1295241
Duplicate of this bug: 1349080
Duplicate of this bug: 1358568
Duplicate of this bug: 1366406
Duplicate of this bug: 1374444
Duplicate of this bug: 1237259
Duplicate of this bug: 1241473
Duplicate of this bug: 1242069
Duplicate of this bug: 1374857
Duplicate of this bug: 1381683
Duplicate of this bug: 1251429
Duplicate of this bug: 1252672
Duplicate of this bug: 1262875
Duplicate of this bug: 1350975
Crash Signature: [@ RunWatchdog]
Bulk change to priority: https://bugzilla.mozilla.org/show_bug.cgi?id=1399258
Priority: P3 → P1
Bulk change to priority: https://bugzilla.mozilla.org/show_bug.cgi?id=1399258
Priority: P1 → P3
In the last 7 days there have been 33 failures. 
Most of the failures occur on the Linux x64 platform. There are also some that occur on Windows 7 and windows10-64. A very low number occur on oher platforms like: .
OS X 10.10, windows7-32-stylo-disabled.
The failures occur mostly on the asan build type. There are also a few failures on debug and a single one on opt.

Here is an example of a recent log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=141231965&lineNumber=36298

And a relevant snippet of the log:
[task 2017-11-01T03:45:49.856Z] 03:45:49     INFO - GECKO(3102) | ASAN:DEADLYSIGNAL
36297
[task 2017-11-01T03:45:49.858Z] 03:45:49     INFO - GECKO(3102) | =================================================================
36298
[task 2017-11-01T03:45:49.859Z] 03:45:49    ERROR - GECKO(3102) | ==3102==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7efc30b950be bp 0x7efbd5c5adf0 sp 0x7efbd5c5ade0 T180)
36299
[task 2017-11-01T03:45:49.859Z] 03:45:49     INFO - GECKO(3102) | ==3102==The signal is caused by a WRITE memory access.
36300
[task 2017-11-01T03:45:49.859Z] 03:45:49     INFO - GECKO(3102) | ==3102==Hint: address points to the zero page.
36301
[task 2017-11-01T03:45:50.243Z] 03:45:50     INFO - GECKO(3102) |     #0 0x7efc30b950bd in mozilla::(anonymous namespace)::RunWatchdog(void*) /builds/worker/workspace/build/src/toolkit/components/terminator/nsTerminator.cpp:163:5
36302
[task 2017-11-01T03:45:50.245Z] 03:45:50     INFO - GECKO(3102) |     #1 0x7efc412d74d3 in _pt_root /builds/worker/workspace/build/src/nsprpub/pr/src/pthreads/ptthread.c:216:5
36303
[task 2017-11-01T03:45:50.253Z] 03:45:50     INFO - GECKO(3102) |     #2 0x7efc4564e6b9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76b9)
36304
[task 2017-11-01T03:45:50.302Z] 03:45:50     INFO - GECKO(3102) |     #3 0x7efc446d73dc in clone /build/glibc-bfm8X4/glibc-2.23/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:109
36305
[task 2017-11-01T03:45:50.303Z] 03:45:50     INFO - GECKO(3102) | AddressSanitizer can not provide additional info.
36306
[task 2017-11-01T03:45:50.304Z] 03:45:50     INFO - GECKO(3102) | SUMMARY: AddressSanitizer: SEGV /builds/worker/workspace/build/src/toolkit/components/terminator/nsTerminator.cpp:163:5 in mozilla::(anonymous namespace)::RunWatchdog(void*)



:mikedeboer, we've noticed that in the last days the failures increased. Do you think you can find anyone who can look into this?
Flags: needinfo?(mdeboer)
Whiteboard: [necko-backlog][stockwell unknown] → [necko-backlog][stockwell needswork]
Hi Tiberius, sorry I can't help you with this specific bug. In fact, Joel triaged this to be part of Session Restore, but that doesn't seem to be true at all... right?
I'm wary of being the one to ping-pong stuff around, but I can't tell anything from the logs and I don't have the necessary knowledge to gather enough useful data when I look at how low level the failure actually is.

Joel, wouldn't it be more reasonable to find a platform engineer who may be able to do a preliminary analysis from the bottom up?
Flags: needinfo?(mdeboer) → needinfo?(jmaher)
given that we seem to be failing on toolkit/components/terminator/nsTerminator.cpp, moving to toolkit::async tooling- ideally we can get more info there.
Component: Session Restore → Async Tooling
Flags: needinfo?(jmaher)
Product: Firefox → Toolkit
:yoric, could you help figure out the runWatchdog crash we are seeing in comment 121?
Flags: needinfo?(dteller)
Well, as usual with nsTerminator, this means that shutdown has timed out and we killed Firefox. Normally, nsTerminator should provide a crash report with slightly more details (i.e. which shutdown phase lasted more than 63 seconds), but this doesn't seem to show up in the log. I suppose we could patch the nsTerminator to dump this info to stderr on DEBUG builds, but I don't think that this would help much.

I'll be happy to answer any question, but I suspect that I won't be able to help much.
Flags: needinfo?(dteller)
as this is heavily weighted on linux64-asan, do you think we could adjust timing for slow configurations?  I think right now we are stuck with a dead end, so if there is any additional information we could get it would be helpful!
I suppose we could use a different constant in ASAN builds, if one single step of shutdown takes more than 1 wallclock minute. Do we have a build flag for ASAN?
Ok, MOZ_ASAN. I'll try and build a patch to make nsTerminator more ASAN-friendly. I suspect that this will just change the crashes into harness timeouts, but we can try.
I think we can do MOZ_ASAN || MOZ_DEBUG, I see these referenced here:
http://searchfox.org/mozilla-central/source/js/src/jsutil.h#334
Comment on attachment 8926895 [details]
Bug 1108587 - Extending the grace period of AsyncShutdown and the nsTerminator for ASAN builds;

https://reviewboard.mozilla.org/r/198140/#review203386

LGTM, I've seen ASAN try runs turn orange so many times that anything that makes them greener is appreciated.
Attachment #8926895 - Flags: review?(gsvelto) → review+
Pushed by dteller@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/56e1ecb8bb58
Extending the grace period of AsyncShutdown and the nsTerminator for ASAN builds;r=gsvelto
https://hg.mozilla.org/mozilla-central/rev/56e1ecb8bb58
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla58
Assignee: nobody → dteller
Whiteboard: [necko-backlog][stockwell needswork] → [necko-backlog][stockwell fixed:other]
You need to log in before you can comment on or make changes to this bug.