1108587 - Intermittent browser_alarms.js,test_webgl_request_mismatch.html | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]

Status: RESOLVED FIXED

(Toolkit :: Async Tooling, defect, P3)

Description

[Carsten Book [:Tomcat]]

Windows 7 32-bit mozilla-central opt test mochitest-browser-chrome-2

https://treeherder.mozilla.org/ui/logviewer.html#?job_id=729651&repo=mozilla-central

05:08:22 WARNING - PROCESS-CRASH | chrome://mochitests/content/browser/hal/tests/browser_alarms.js | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)] 
5:08:22 INFO - Crash dump filename: c:\users\cltbld\appdata\local\temp\tmppamfqt.mozrunner\minidumps\fae4ff76-b276-4974-baed-09e30a9c8aa8.dmp
05:08:22 INFO - Operating system: Windows NT
05:08:22 INFO - 6.1.7601 Service Pack 1
05:08:22 INFO - CPU: x86
05:08:22 INFO - GenuineIntel family 6 model 30 stepping 5
05:08:22 INFO - 8 CPUs
05:08:22 INFO - Crash reason: EXCEPTION_BREAKPOINT
05:08:22 INFO - Crash address: 0x62fe481b
05:08:22 INFO - Thread 42 (crashed)
05:08:22 INFO - 0 xul.dll!mozilla::`anonymous namespace'::RunWatchdog(void *) [nsTerminator.cpp:035a951fc24a : 151 + 0x0]
05:08:22 INFO - eip = 0x62fe481b esp = 0x1735ff18 ebp = 0x1735ff1c ebx = 0x002cf0d8
05:08:22 INFO - esi = 0x0000003f edi = 0x01d14d00 eax = 0x0000003f ecx = 0x63e70c80
05:08:22 INFO - edx = 0x0000001b efl = 0x00000246
05:08:22 INFO - Found by: given as instruction pointer in context
05:08:22 INFO - 1 nss3.dll!_PR_NativeRunThread [pruthr.c:035a951fc24a : 397 + 0x7]
05:08:22 INFO - eip = 0x675c6718 esp = 0x1735ff24 ebp = 0x1735ff38
05:08:22 INFO - Found by: call frame info
05:08:22 INFO - 2 nss3.dll!pr_root [w95thred.c:035a951fc24a : 90 + 0xb]
05:08:22 INFO - eip = 0x675b921f esp = 0x1735ff40 ebp = 0x1735ff44
05:08:22 INFO - Found by: call frame info
05:08:22 INFO - 3 msvcr120.dll + 0x2c01c
05:08:22 INFO - eip = 0x6764c01d esp = 0x1735ff4c ebp = 0x1735ff7c
05:08:22 INFO - Found by: call frame info
05:08:22 INFO - 4 msvcr120.dll + 0x2c000
05:08:22 INFO - eip = 0x6764c001 esp = 0x1735ff84 ebp = 0x1735ff88
05:08:22 INFO - Found by: previous frame's frame pointer
05:08:22 INFO - 5 kernel32.dll + 0x53c44
05:08:22 INFO - eip = 0x77453c45 esp = 0x1735ff90 ebp = 0x1735ff94
05:08:22 INFO - Found by: previous frame's frame pointer
05:08:22 INFO - 6 ntdll.dll + 0x637f4
05:08:22 INFO - eip = 0x779c37f5 esp = 0x1735ff9c ebp = 0x1735ffd4
05:08:22 INFO - Found by: previous frame's frame pointer
05:08:22 INFO - 7 ntdll.dll + 0x637c7

Comment 1

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2014-12-08T07:06:18
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=mozilla-central&job_id=729651
repository: mozilla-central
who: tomcat[at]mozilla[dot]com
machine: t-w732-ix-027
buildname: Windows 7 32-bit mozilla-central opt test mochitest-browser-chrome-2
revision: 035a951fc24a

296 ERROR TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/hal/tests/browser_alarms.js | application terminated with exit code 1
PROCESS-CRASH | chrome://mochitests/content/browser/hal/tests/browser_alarms.js | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
Return code: 1

Comment 2

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2014-12-08T12:12:31
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=mozilla-central&job_id=732361
repository: mozilla-central
who: rvandermeulen[at]mozilla[dot]com
machine: tst-linux64-spot-226
buildname: Ubuntu ASAN VM 12.04 x64 mozilla-central opt test mochitest-gl
revision: af4f2eea4e27

SUMMARY: AddressSanitizer: SEGV /builds/slave/m-cen-l64-asan-000000000000000/build/src/toolkit/components/terminator/nsTerminator.cpp:151 mozilla::(anonymous namespace)::RunWatchdog(void*)
707 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code 1
Return code: 1
command timed out: 12000 seconds elapsed running ['/tools/buildbot/bin/python', 'scripts/scripts/desktop_unittest.py', '--cfg', 'unittests/linux_unittest.py', '--mochitest-suite', 'mochitest-gl', '--blob-upload-branch', 'mozilla-central', '--download-symbols', 'ondemand'], attempting to kill

Comment 3

[Ryan VanderMeulen [:RyanVM]]

Maybe the ASAN stack sheds some light on this?

Comment 4

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2014-12-17T08:34:18
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=mozilla-central&job_id=766888
repository: mozilla-central
who: rvandermeulen[at]mozilla[dot]com
machine: t-w732-ix-027
buildname: Windows 7 32-bit mozilla-central opt test mochitest-browser-chrome-2
revision: 494f71e59583

299 ERROR TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/hal/tests/browser_alarms.js | application terminated with exit code 3221225477
PROCESS-CRASH | chrome://mochitests/content/browser/hal/tests/browser_alarms.js | application crashed [None]
Return code: 1

Comment 5

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2014-12-17T09:52:45
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=b2g-inbound&job_id=1014320
repository: b2g-inbound
who: wkocher[at]mozilla[dot]com
machine: t-w732-ix-008
buildname: Windows 7 32-bit b2g-inbound opt test mochitest-browser-chrome-2
revision: a0d08d9c9748

305 ERROR TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/hal/tests/browser_alarms.js | application terminated with exit code 1
PROCESS-CRASH | chrome://mochitests/content/browser/hal/tests/browser_alarms.js | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
Return code: 1

Comment 6

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2014-12-30T14:38:36
log: https://treeherder.mozilla.org/ui/logviewer.html#?repo=fx-team&job_id=1569654
repository: fx-team
who: wkocher[at]mozilla[dot]com
machine: t-w732-ix-009
buildname: Windows 7 32-bit fx-team opt test mochitest-e10s-browser-chrome-2
revision: 5ff375efac3c

85 ERROR TEST-UNEXPECTED-FAIL | hal/tests/browser_alarms.js | application terminated with exit code 1
PROCESS-CRASH | hal/tests/browser_alarms.js | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
Return code: 1

Comment 7

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2015-01-13T09:58:06
log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=5413229
repository: mozilla-inbound
who: rvandermeulen[at]mozilla[dot]com
machine: t-w732-ix-043
buildname: Windows 7 32-bit mozilla-inbound opt test mochitest-e10s-browser-chrome-2
revision: fda146977dfe

91 ERROR TEST-UNEXPECTED-FAIL | hal/tests/browser_alarms.js | application terminated with exit code 1
PROCESS-CRASH | hal/tests/browser_alarms.js | application crashed [@ mozilla::detail::HashKnownLength<wchar_t>(wchar_t const *,unsigned int)]
Return code: 1

Comment 8

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2015-01-14T00:43:58
log: https://treeherder.mozilla.org/logviewer.html#?repo=fx-team&job_id=1685669
repository: fx-team
who: tomcat[at]mozilla[dot]com
machine: t-w732-ix-014
buildname: Windows 7 32-bit fx-team pgo test mochitest-browser-chrome-2
revision: a88e871c1dfc

275 ERROR TEST-UNEXPECTED-FAIL | hal/tests/browser_alarms.js | application terminated with exit code 1
PROCESS-CRASH | hal/tests/browser_alarms.js | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
Return code: 1

Comment 9

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2015-01-14T13:28:36
log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=5462410
repository: mozilla-inbound
who: rvandermeulen[at]mozilla[dot]com
machine: t-w732-ix-013
buildname: Windows 7 32-bit mozilla-inbound opt test mochitest-browser-chrome-2
revision: c834d4aeee3e

275 ERROR TEST-UNEXPECTED-FAIL | hal/tests/browser_alarms.js | application terminated with exit code 1
PROCESS-CRASH | hal/tests/browser_alarms.js | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
Return code: 1

Comment 10

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2015-01-15T03:30:40
log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=5499305
repository: mozilla-inbound
who: tomcat[at]mozilla[dot]com
machine: t-w732-ix-010
buildname: Windows 7 32-bit mozilla-inbound opt test mochitest-e10s-browser-chrome-2
revision: 297f99f0ed18

97 ERROR TEST-UNEXPECTED-FAIL | hal/tests/browser_alarms.js | application terminated with exit code 1
PROCESS-CRASH | hal/tests/browser_alarms.js | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
Return code: 1

Comment 11

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2015-01-15T20:53:05
log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=5531644
repository: mozilla-inbound
who: philringnalda[at]gmail[dot]com
machine: t-snow-r4-0053
buildname: Rev4 MacOSX Snow Leopard 10.6 mozilla-inbound opt test mochitest-gl
revision: 20544516f48d

719 ERROR TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code -11
Return code: 245

Comment 12

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2015-01-18T07:17:08
log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=5604127
repository: mozilla-inbound
who: philringnalda[at]gmail[dot]com
machine: t-snow-r4-0008
buildname: Rev4 MacOSX Snow Leopard 10.6 mozilla-inbound debug test mochitest-gl
revision: d12719615903

719 ERROR TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code -11
Return code: 245

Comment 13

[Legacy TBPL/Treeherder Robot]

submit_timestamp: 2015-01-19T10:00:53
log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=5630386
repository: mozilla-inbound
who: philringnalda[at]gmail[dot]com
machine: t-w732-ix-013
buildname: Windows 7 32-bit mozilla-inbound opt test mochitest-browser-chrome-2
revision: 1ad6ede35b99

275 ERROR TEST-UNEXPECTED-FAIL | hal/tests/browser_alarms.js | application terminated with exit code 1
PROCESS-CRASH | hal/tests/browser_alarms.js | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
Return code: 1

Comment 14

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=b2g-inbound&job_id=1188164
repository: b2g-inbound
start_time: 2015-01-19T12:53:14
who: philringnalda[at]gmail[dot]com
machine: t-snow-r4-0054
buildname: Rev4 MacOSX Snow Leopard 10.6 b2g-inbound opt test mochitest-gl
revision: b76d6ff1744f

719 ERROR TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code -11
Return code: 245

Comment 15

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-beta&job_id=218798
repository: mozilla-beta
start_time: 2015-01-19T14:05:40
who: philringnalda[at]gmail[dot]com
machine: tst-linux64-spot-599
buildname: Ubuntu ASAN VM 12.04 x64 mozilla-beta opt test mochitest-gl
revision: b653be6b040a

707 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code -11
Return code: 245

Comment 16

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=cedar&job_id=93116
repository: cedar
start_time: 2015-01-19T15:40:18
who: philringnalda[at]gmail[dot]com
machine: t-snow-r4-0054
buildname: Rev4 MacOSX Snow Leopard 10.6 cedar opt test mochitest-gl
revision: 852f84996843

719 ERROR TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code -11
Return code: 245

Comment 17

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=fx-team&job_id=1734277
repository: fx-team
start_time: 2015-01-19T16:09:01
who: philringnalda[at]gmail[dot]com
machine: t-w732-ix-006
buildname: Windows 7 32-bit fx-team debug test mochitest-2
revision: 739b6d124285

2201 ERROR TEST-UNEXPECTED-FAIL | dom/media/mediasource/test/test_WaitingOnMissingData.html | application terminated with exit code 1
PROCESS-CRASH | dom/media/mediasource/test/test_WaitingOnMissingData.html | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
TEST-UNEXPECTED-FAIL | leakcheck | default process: missing output line for total leaks!
Return code: 1

Comment 18

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=fx-team&job_id=1761006
repository: fx-team
start_time: 2015-01-22T01:33:00
who: tomcat[at]mozilla[dot]com
machine: t-snow-r4-0014
buildname: Rev4 MacOSX Snow Leopard 10.6 fx-team debug test mochitest-gl
revision: e91444745291

719 ERROR TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code -11
Return code: 245

Comment 19

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=b2g-inbound&job_id=1221732
repository: b2g-inbound
start_time: 2015-01-24T18:00:23
who: philringnalda[at]gmail[dot]com
machine: t-snow-r4-0072
buildname: Rev4 MacOSX Snow Leopard 10.6 b2g-inbound opt test mochitest-gl
revision: c18776175a69

719 ERROR TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code -11
Return code: 245

Comment 20

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=5883927
repository: mozilla-inbound
start_time: 2015-01-25T09:04:18
who: philringnalda[at]gmail[dot]com
machine: t-snow-r4-0054
buildname: Rev4 MacOSX Snow Leopard 10.6 mozilla-inbound debug test mochitest-gl
revision: 1f0d37966eac

719 ERROR TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code -11
Return code: 245

Comment 21

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=fx-team&job_id=1813853
repository: fx-team
start_time: 2015-01-27T19:54:34
who: philringnalda[at]gmail[dot]com
machine: t-snow-r4-0064
buildname: Rev4 MacOSX Snow Leopard 10.6 fx-team debug test mochitest-gl
revision: 134bcefd5f7b

719 ERROR TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code -11
Return code: 245

Comment 22

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=6021709
repository: mozilla-inbound
start_time: 2015-01-28T07:01:35
who: philringnalda[at]gmail[dot]com
machine: t-snow-r4-0133
buildname: Rev4 MacOSX Snow Leopard 10.6 mozilla-inbound opt test mochitest-gl
revision: c738e5c9a061

719 ERROR TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code -11
Return code: 245

Comment 23

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=6055789
repository: mozilla-inbound
start_time: 2015-01-28T23:04:19
who: tomcat[at]mozilla[dot]com
machine: t-snow-r4-0019
buildname: Rev4 MacOSX Snow Leopard 10.6 mozilla-inbound debug test mochitest-gl
revision: 0a1c01c0854e

719 ERROR TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code -11
Return code: 245

Comment 24

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=fx-team&job_id=1830827
repository: fx-team
start_time: 2015-01-29T07:01:05
who: philringnalda[at]gmail[dot]com
machine: t-snow-r4-0065
buildname: Rev4 MacOSX Snow Leopard 10.6 fx-team debug test mochitest-gl
revision: 94368a8d00f1

719 ERROR TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code -11
Return code: 245

Comment 25

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=6136027
repository: mozilla-inbound
start_time: 2015-01-31T11:23:09
who: philringnalda[at]gmail[dot]com
machine: t-snow-r4-0079
buildname: Rev4 MacOSX Snow Leopard 10.6 mozilla-inbound debug test mochitest-gl
revision: 030744f8ef5a

719 ERROR TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code -11
Return code: 245

Comment 26

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=6142861
repository: mozilla-inbound
start_time: 2015-01-31T21:31:51
who: philringnalda[at]gmail[dot]com
machine: t-snow-r4-0037
buildname: Rev4 MacOSX Snow Leopard 10.6 mozilla-inbound debug test mochitest-gl
revision: 0038cfaf847a

719 ERROR TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code -11
Return code: 245

Comment 27

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=6171191
repository: mozilla-inbound
start_time: 2015-02-02T11:25:10
who: philringnalda[at]gmail[dot]com
machine: t-snow-r4-0095
buildname: Rev4 MacOSX Snow Leopard 10.6 mozilla-inbound opt test mochitest-gl
revision: 42715e24a005

719 ERROR TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code -11
Return code: 245

Comment 28

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-b2g37_v2_2&job_id=21095
repository: mozilla-b2g37_v2_2
start_time: 2015-02-02T15:39:36
who: wkocher[at]mozilla[dot]com
machine: t-snow-r4-0004
buildname: Rev4 MacOSX Snow Leopard 10.6 mozilla-b2g37_v2_2 debug test mochitest-gl
revision: fc988d6c2428

719 ERROR TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code -11
Return code: 245

Comment 29

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=6198288
repository: mozilla-inbound
start_time: 2015-02-02T22:48:19
who: philringnalda[at]gmail[dot]com
machine: t-snow-r4-0033
buildname: Rev4 MacOSX Snow Leopard 10.6 mozilla-inbound opt test mochitest-gl
revision: 64f3413693b9

719 ERROR TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code -11
Return code: 245

Comment 30

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-central&job_id=984601
repository: mozilla-central
start_time: 2015-02-03T07:18:09
who: philringnalda[at]gmail[dot]com
machine: t-w864-ix-060
buildname: Windows 8 64-bit mozilla-central opt test mochitest-3
revision: 58e4264903ba

5691 ERROR TEST-UNEXPECTED-FAIL | dom/tests/mochitest/gamepad/test_navigator_gamepads.html | application terminated with exit code 1
PROCESS-CRASH | dom/tests/mochitest/gamepad/test_navigator_gamepads.html | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
Return code: 1

Comment 31

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=b2g-inbound&job_id=1307088
repository: b2g-inbound
start_time: 2015-02-06T19:07:29
who: philringnalda[at]gmail[dot]com
machine: t-snow-r4-0105
buildname: Rev4 MacOSX Snow Leopard 10.6 b2g-inbound debug test mochitest-gl
revision: d3d8549885ff

719 ERROR TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code -11
Return code: 245

Comment 32

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=6378864
repository: mozilla-inbound
start_time: 2015-02-06T22:26:12
who: philringnalda[at]gmail[dot]com
machine: t-snow-r4-0004
buildname: Rev4 MacOSX Snow Leopard 10.6 mozilla-inbound opt test mochitest-gl
revision: e37cf0a00e0f

719 ERROR TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code -11
Return code: 245

Comment 33

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=fx-team&job_id=1927357
repository: fx-team
start_time: 2015-02-07T07:51:03
who: philringnalda[at]gmail[dot]com
machine: t-w864-ix-028
buildname: Windows 8 64-bit fx-team opt test mochitest-3
revision: 479491be79c5

5693 ERROR TEST-UNEXPECTED-FAIL | dom/tests/mochitest/gamepad/test_navigator_gamepads.html | application terminated with exit code 1
PROCESS-CRASH | dom/tests/mochitest/gamepad/test_navigator_gamepads.html | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
Return code: 1

Comment 34

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=6392463
repository: mozilla-inbound
start_time: 2015-02-08T06:31:22
who: philringnalda[at]gmail[dot]com
machine: t-snow-r4-0077
buildname: Rev4 MacOSX Snow Leopard 10.6 mozilla-inbound opt test mochitest-gl
revision: ee3bc49e9cf5

719 ERROR TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code -11
Return code: 245

Comment 35

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=fx-team&job_id=1951566
repository: fx-team
start_time: 2015-02-10T05:03:29
who: tomcat[at]mozilla[dot]com
machine: t-snow-r4-0134
buildname: Rev4 MacOSX Snow Leopard 10.6 fx-team opt test mochitest-gl
revision: e57f94dd5cf3

719 ERROR TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code -11
Return code: 245

Comment 36

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=6654963
repository: mozilla-inbound
start_time: 2015-02-16T10:42:57
who: nigelbabu[at]gmail[dot]com
machine: t-snow-r4-0013
buildname: Rev4 MacOSX Snow Leopard 10.6 mozilla-inbound opt test mochitest-gl
revision: 04f861bfe264

"comments": "Bug 1132337 - fix \"ReferenceError: PlacesTestUtils is not defined\" in test_history_expiration.js. r=paolo\n\ncomm-central: TEST-UNEXPECTED-FAIL | toolkit/components/downloads/test/unit/test_history_expiration.js | xpcshell return code: 0",
719 ERROR TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code -11
Return code: 245

Comment 37

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=6688137
repository: mozilla-inbound
start_time: 2015-02-17T11:11:42
who: rvandermeulen[at]mozilla[dot]com
machine: t-snow-r4-0129
buildname: Rev4 MacOSX Snow Leopard 10.6 mozilla-inbound opt test mochitest-e10s-browser-chrome-2
revision: a9a074f02927

189 INFO TEST-UNEXPECTED-FAIL | testing/mochitest/tests/browser/browser_waitForFocus.js | Test timed out - expected PASS
196 INFO TEST-UNEXPECTED-FAIL | testing/mochitest/tests/browser/browser_waitForFocus.js | Found a tab after previous test timed out: about:blank - expected PASS
204 ERROR TEST-UNEXPECTED-FAIL | testing/mochitest/tests/browser/browser_waitForFocus.js | application terminated with exit code 1
PROCESS-CRASH | testing/mochitest/tests/browser/browser_waitForFocus.js | application crashed [@ mozilla::(anonymous namespace)::RunWatchdog(void*)]

Comment 38

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-aurora&job_id=599170
repository: mozilla-aurora
start_time: 2015-02-17T23:54:49
who: rvandermeulen[at]mozilla[dot]com
machine: t-xp32-ix-074
buildname: Windows XP 32-bit mozilla-aurora pgo test mochitest-3
revision: 7917cdc37925

5770 ERROR TEST-UNEXPECTED-FAIL | dom/tests/mochitest/geolocation/test_worseAccuracyDoesNotBlockCallback.html | application terminated with exit code 1
PROCESS-CRASH | dom/tests/mochitest/geolocation/test_worseAccuracyDoesNotBlockCallback.html | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
Return code: 1

Comment 39

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=fx-team&job_id=2031881
repository: fx-team
start_time: 2015-02-18T20:29:48
who: rvandermeulen[at]mozilla[dot]com
machine: t-w864-ix-089
buildname: Windows 8 64-bit fx-team pgo test mochitest-e10s-browser-chrome-3
revision: 58f815545f09

127 ERROR TEST-UNEXPECTED-FAIL | toolkit/content/tests/browser/browser_browserDrop.js | application terminated with exit code 1
PROCESS-CRASH | toolkit/content/tests/browser/browser_browserDrop.js | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
Return code: 1

Comment 40

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=fx-team&job_id=2058234
repository: fx-team
start_time: 2015-02-20T23:41:50
who: rvandermeulen[at]mozilla[dot]com
machine: t-w864-ix-134
buildname: Windows 8 64-bit fx-team pgo test mochitest-e10s-browser-chrome-3
revision: 63da271edfa5

65 ERROR TEST-UNEXPECTED-FAIL | toolkit/components/startup/tests/browser/browser_crash_detection.js | application terminated with exit code 1
PROCESS-CRASH | toolkit/components/startup/tests/browser/browser_crash_detection.js | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
Return code: 1

Comment 41

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=6843792
repository: mozilla-inbound
start_time: 2015-02-21T04:57:53
who: rvandermeulen[at]mozilla[dot]com
machine: t-snow-r4-0026
buildname: Rev4 MacOSX Snow Leopard 10.6 mozilla-inbound debug test mochitest-4
revision: 18713000b620

398 INFO TEST-UNEXPECTED-FAIL | dom/workers/test/serviceworkers/test_match_all.html | Test timed out. - expected PASS
1032 ERROR TEST-UNEXPECTED-FAIL | intl/uconv/tests/test_utf8_overconsumption.html | application terminated with exit code 1
PROCESS-CRASH | intl/uconv/tests/test_utf8_overconsumption.html | application crashed [@ mozilla::(anonymous namespace)::RunWatchdog(void*)]
TEST-UNEXPECTED-FAIL | leakcheck | default process: missing output line for total leaks!
Return code: 1

Comment 42

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=8193865
repository: mozilla-inbound
start_time: 2015-03-28T05:41:08
who: philringnalda[at]gmail[dot]com
machine: t-snow-r4-0089
buildname: Rev4 MacOSX Snow Leopard 10.6 mozilla-inbound opt test mochitest-gl
revision: 0c030f97a04f

TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code -11
Return code: 245

Comment 43

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=8743000
repository: mozilla-inbound
start_time: 2015-04-11T06:04:10
who: jmathies[at]mozilla[dot]com
machine: t-snow-r4-0054
buildname: Rev4 MacOSX Snow Leopard 10.6 mozilla-inbound debug test mochitest-gl
revision: 522aa79beb09

TEST-UNEXPECTED-FAIL | dom/canvas/test/webgl-mochitest/test_webgl_request_mismatch.html | application terminated with exit code -11
Return code: 245

Comment 44

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=try&job_id=7384595
repository: try
start_time: 2015-05-09T01:51:29
who: philringnalda[at]gmail[dot]com
machine: tst-linux32-spot-1039
buildname: Ubuntu VM 12.04 try debug test mochitest-other
revision: 354f3219c3be

TEST-UNEXPECTED-FAIL | xpfe/appshell/test/test_hiddenPrivateWindow.xul | application terminated with exit code 11
PROCESS-CRASH | xpfe/appshell/test/test_hiddenPrivateWindow.xul | application crashed [@ mozilla::::RunWatchdog]
TEST-UNEXPECTED-FAIL | leakcheck | default process: missing output line for total leaks!
Return code: 1

Comment 45

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=9916833
repository: mozilla-inbound
start_time: 2015-05-17T01:56:51
who: philringnalda[at]gmail[dot]com
machine: t-snow-r4-0108
buildname: Rev4 MacOSX Snow Leopard 10.6 mozilla-inbound debug test mochitest-browser-chrome-3
revision: c08fea2b1d59

TEST-UNEXPECTED-FAIL | toolkit/mozapps/extensions/test/browser/test-window/browser_updateid.js | application terminated with exit code 1
PROCESS-CRASH | toolkit/mozapps/extensions/test/browser/test-window/browser_updateid.js | application crashed [@ mozilla::(anonymous namespace)::RunWatchdog(void*)]
TEST-UNEXPECTED-FAIL | leakcheck | default process: missing output line for total leaks!

Comment 46

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=10118013
repository: mozilla-inbound
start_time: 2015-05-24T14:59:33
who: philringnalda[at]gmail[dot]com
machine: t-snow-r4-0018
buildname: Rev4 MacOSX Snow Leopard 10.6 mozilla-inbound opt test mochitest-3
revision: 780f3d9c7cc3

[NPAPI 1382] ###!!! ABORT: Aborting on channel error.: file /builds/slave/m-in-m64-000000000000000000000/build/src/ipc/glue/MessageChannel.cpp, line 1662
[NPAPI 1382] ###!!! ABORT: Aborting on channel error.: file /builds/slave/m-in-m64-000000000000000000000/build/src/ipc/glue/MessageChannel.cpp, line 1662
TEST-UNEXPECTED-FAIL | dom/tests/mochitest/gamepad/test_navigator_gamepads.html | application terminated with exit code 1
PROCESS-CRASH | dom/tests/mochitest/gamepad/test_navigator_gamepads.html | application crashed [@ mozilla::(anonymous namespace)::RunWatchdog(void*)]
Return code: 1

Comment 47

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=12515329
repository: mozilla-inbound
start_time: 2015-08-05T04:35:55
who: tomcat[at]mozilla[dot]com
machine: t-w732-ix-129
buildname: Windows 7 32-bit mozilla-inbound debug test mochitest-jetpack
revision: 88b05f9375e7

Return code: 1
TEST-UNEXPECTED-FAIL | automation.py | application terminated with exit code 1
PROCESS-CRASH | automation.py | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
TEST-UNEXPECTED-FAIL | leakcheck | default process: missing output line for total leaks!
Return code: 1

Comment 48

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=12543925
repository: mozilla-inbound
start_time: 2015-08-05T16:21:33
who: wkocher[at]mozilla[dot]com
machine: t-w864-ix-011
buildname: Windows 8 64-bit mozilla-inbound opt test mochitest-browser-chrome-1
revision: d4f3f7e9d235

672 INFO TEST-UNEXPECTED-FAIL | dom/workers/test/serviceworkers/browser_force_refresh.js | Test timed out - expected PASS
674 INFO TEST-UNEXPECTED-FAIL | dom/workers/test/serviceworkers/browser_force_refresh.js | Found a tab after previous test timed out: http://mochi.test:8888/browser/dom/workers/test/serviceworkers/browser_base_force_refresh.html - expected PASS
TEST-UNEXPECTED-FAIL | dom/workers/test/serviceworkers/browser_force_refresh.js | application terminated with exit code 1
PROCESS-CRASH | dom/workers/test/serviceworkers/browser_force_refresh.js | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
Return code: 1

Comment 49

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=12566012
repository: mozilla-inbound
start_time: 2015-08-06T03:07:52
who: tomcat[at]mozilla[dot]com
machine: t-w732-ix-057
buildname: Windows 7 32-bit mozilla-inbound pgo test mochitest-browser-chrome-1
revision: b1e940e09127

672 INFO TEST-UNEXPECTED-FAIL | dom/workers/test/serviceworkers/browser_force_refresh.js | Test timed out - expected PASS
674 INFO TEST-UNEXPECTED-FAIL | dom/workers/test/serviceworkers/browser_force_refresh.js | Found a tab after previous test timed out: http://mochi.test:8888/browser/dom/workers/test/serviceworkers/browser_base_force_refresh.html - expected PASS
TEST-UNEXPECTED-FAIL | dom/workers/test/serviceworkers/browser_force_refresh.js | application terminated with exit code 1
PROCESS-CRASH | dom/workers/test/serviceworkers/browser_force_refresh.js | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
Return code: 1

Comment 50

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=12593283
repository: mozilla-inbound
start_time: 2015-08-06T14:59:11
who: wkocher[at]mozilla[dot]com
machine: t-w732-ix-035
buildname: Windows 7 32-bit mozilla-inbound pgo test jsreftest
revision: bca1ad6cc378

TEST-UNEXPECTED-FAIL | Shutdown | application terminated with exit code 1
PROCESS-CRASH | Shutdown | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
Return code: 1

Comment 51

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-central&job_id=1998891
repository: mozilla-central
start_time: 2015-08-11T17:32:35
who: philringnalda[at]gmail[dot]com
machine: t-w864-ix-015
buildname: Windows 8 64-bit mozilla-central debug test mochitest-2
revision: 7dcecc8a395d

2348 INFO TEST-UNEXPECTED-FAIL | dom/media/test/test_eme_playback.html | Test timed out.
2419 INFO TEST-UNEXPECTED-FAIL | dom/media/test/test_eme_stream_capture_blocked_case2.html | Test timed out.
2476 INFO TEST-UNEXPECTED-FAIL | dom/media/test/test_eme_stream_capture_blocked_case3.html | Test timed out.
TEST-UNEXPECTED-FAIL | dom/media/webaudio/test/test_waveShaperPassThrough.html | application terminated with exit code 1
PROCESS-CRASH | dom/media/webaudio/test/test_waveShaperPassThrough.html | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
TEST-UNEXPECTED-FAIL | leakcheck | default process: missing output line for total leaks!
Return code: 1

Comment 52

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=b2g-inbound&job_id=2594180
repository: b2g-inbound
start_time: 2015-08-23T21:09:08
who: philringnalda[at]gmail[dot]com
machine: t-w732-ix-117
buildname: Windows 7 32-bit b2g-inbound opt test jsreftest
revision: 4dcdc90adbe5

TEST-UNEXPECTED-FAIL | Shutdown | application terminated with exit code 1
PROCESS-CRASH | Shutdown | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
Return code: 1

Comment 53

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=fx-team&job_id=4427980
repository: fx-team
start_time: 2015-08-27T21:33:04
who: philringnalda[at]gmail[dot]com
machine: t-w732-ix-117
buildname: Windows 7 32-bit fx-team pgo test jsreftest
revision: 45de359919d7

TEST-UNEXPECTED-FAIL | Shutdown | application terminated with exit code 1
PROCESS-CRASH | Shutdown | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
Return code: 1

Comment 54

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=fx-team&job_id=4519176
repository: fx-team
start_time: 2015-09-02T13:53:37
who: wkocher[at]mozilla[dot]com
machine: t-xp32-ix-016
buildname: Windows XP 32-bit fx-team pgo test crashtest
revision: c5f53d07ff1b

TEST-UNEXPECTED-FAIL | Shutdown | application terminated with exit code 1
PROCESS-CRASH | Shutdown | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
Return code: 1

Comment 55

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=14397775
repository: mozilla-inbound
start_time: 2015-09-19T21:05:30
who: philringnalda[at]gmail[dot]com
machine: t-w732-ix-027
buildname: Windows 7 32-bit mozilla-inbound pgo test jsreftest
revision: c0d3f6e2a3e3

TEST-UNEXPECTED-FAIL | Shutdown | application terminated with exit code 1
PROCESS-CRASH | Shutdown | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
Return code: 1

Comment 56

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-aurora&job_id=1200172
repository: mozilla-aurora
start_time: 2015-09-22T14:32:42
who: philringnalda[at]gmail[dot]com
machine: t-w732-ix-041
buildname: Windows 7 32-bit mozilla-aurora debug test mochitest-5
revision: 9087b63b0cd2

TEST-UNEXPECTED-FAIL | toolkit/mozapps/extensions/test/mochitest/test_bug887098.html | application terminated with exit code 1
PROCESS-CRASH | toolkit/mozapps/extensions/test/mochitest/test_bug887098.html | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
TEST-UNEXPECTED-FAIL | leakcheck | default process: missing output line for total leaks!
Return code: 1

Comment 57

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=14758433
repository: mozilla-inbound
start_time: 2015-09-26T05:47:21
who: nigelbabu[at]gmail[dot]com
machine: t-xp32-ix-016
buildname: Windows XP 32-bit mozilla-inbound pgo test mochitest-1
revision: bd4a6ddd01a0

TEST-UNEXPECTED-FAIL | dom/base/test/test_xhr_withCredentials.html | application terminated with exit code 1
PROCESS-CRASH | dom/base/test/test_xhr_withCredentials.html | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
Return code: 1

Comment 58

[Legacy TBPL/Treeherder Robot]

log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-central&job_id=2298465
repository: mozilla-central
start_time: 2015-09-25T20:21:58
who: tomcat[at]mozilla[dot]com
machine: t-w732-ix-035
buildname: Windows 7 32-bit mozilla-central pgo test jsreftest
revision: 94c804ef40d8

TEST-UNEXPECTED-FAIL | Shutdown | application terminated with exit code 1
PROCESS-CRASH | Shutdown | application crashed [@ mozilla::`anonymous namespace'::RunWatchdog(void *)]
Return code: 1

Comment 59

[Intermittent Failures Robot]

9 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 5
* fx-team: 2
* mozilla-central: 1
* b2g-inbound: 1

Platform breakdown:
* linux64: 8
* osx-10-6: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2015-11-09&endday=2015-11-15&tree=all

Comment 60

[Intermittent Failures Robot]

26 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 11
* fx-team: 10
* b2g-inbound: 5

Platform breakdown:
* linux64: 21
* osx-10-6: 4
* windowsxp: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2015-11-16&endday=2015-11-22&tree=all

Comment 61

[Intermittent Failures Robot]

20 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 17
* fx-team: 3

Platform breakdown:
* linux64: 16
* windowsxp: 2
* windows7-32: 1
* osx-10-6: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2015-11-23&endday=2015-11-29&tree=all

Comment 62

[Intermittent Failures Robot]

13 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* fx-team: 6
* mozilla-inbound: 4
* mozilla-central: 2
* mozilla-beta: 1

Platform breakdown:
* linux64: 11
* windows7-32: 1
* osx-10-6: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2015-11-30&endday=2015-12-06&tree=all

Comment 63

[Intermittent Failures Robot]

14 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 11
* fx-team: 2
* b2g-inbound: 1

Platform breakdown:
* linux64: 13
* windowsxp: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2015-12-07&endday=2015-12-13&tree=all

Comment 64

[Intermittent Failures Robot]

7 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 3
* mozilla-central: 1
* mozilla-beta: 1
* mozilla-aurora: 1
* fx-team: 1

Platform breakdown:
* linux64: 5
* windowsxp: 1
* osx-10-6: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2016-01-04&endday=2016-01-10&tree=all

Comment 65

[Intermittent Failures Robot]

21 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 10
* mozilla-central: 6
* fx-team: 3
* mozilla-aurora: 1
* b2g-inbound: 1

Platform breakdown:
* windowsxp: 13
* linux64: 5
* windows8-64: 1
* windows7-32: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2016-01-18&endday=2016-01-24&tree=all

Comment 66

[BMO Automation]

Bulk assigning P3 to all open intermittent bugs without a priority set in Firefox components per bug 1298978.

Comment 67

[Intermittent Failures Robot]

6 failures in 733 pushes (0.008 failures/push) were associated with this bug in the last 7 days.  

Repository breakdown:
* mozilla-inbound: 4
* autoland: 2

Platform breakdown:
* linux64: 4
* linux32: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-01-30&endday=2017-02-05&tree=all

Comment 68

[Intermittent Failures Robot]

11 failures in 836 pushes (0.013 failures/push) were associated with this bug in the last 7 days.  
Repository breakdown:
* mozilla-inbound: 7
* autoland: 4

Platform breakdown:
* linux64: 6
* windows7-32-vm: 4
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-02-06&endday=2017-02-12&tree=all

Comment 69

[Intermittent Failures Robot]

7 failures in 812 pushes (0.009 failures/push) were associated with this bug in the last 7 days.  
Repository breakdown:
* autoland: 3
* mozilla-inbound: 2
* mozilla-central: 2

Platform breakdown:
* windows7-32-vm: 5
* linux64: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-02-20&endday=2017-02-26&tree=all

Comment 70

[Intermittent Failures Robot]

10 failures in 783 pushes (0.013 failures/push) were associated with this bug in the last 7 days.  
Repository breakdown:
* mozilla-inbound: 5
* autoland: 5

Platform breakdown:
* windows7-32-vm: 6
* windows8-64: 2
* linux64: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-02-27&endday=2017-03-05&tree=all

Comment 71

[Intermittent Failures Robot]

6 failures in 898 pushes (0.007 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 4
* mozilla-esr45: 1
* autoland: 1

Platform breakdown:
* linux64: 3
* windows8-64: 2
* osx-10-6: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-03-20&endday=2017-03-26&tree=all

Comment 72

[Intermittent Failures Robot]

9 failures in 867 pushes (0.01 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 5
* mozilla-inbound: 2
* mozilla-central: 1
* mozilla-aurora: 1

Platform breakdown:
* windows7-32-vm: 3
* windows7-32: 3
* windows8-64: 2
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-04-03&endday=2017-04-09&tree=all

Comment 73

[Intermittent Failures Robot]

5 failures in 894 pushes (0.006 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 3
* autoland: 2

Platform breakdown:
* windows7-32: 2
* windows8-64: 1
* osx-10-10: 1
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-04-10&endday=2017-04-16&tree=all

Comment 74

[Intermittent Failures Robot]

5 failures in 883 pushes (0.006 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-esr45: 2
* mozilla-inbound: 1
* mozilla-beta: 1
* autoland: 1

Platform breakdown:
* linux64: 2
* windows7-32-vm: 1
* windows7-32: 1
* osx-10-6: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-04-24&endday=2017-04-30&tree=all

Comment 75

[Intermittent Failures Robot]

9 failures in 879 pushes (0.01 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 4
* mozilla-inbound: 3
* mozilla-central: 2

Platform breakdown:
* linux64: 7
* linux64-qr: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-05-08&endday=2017-05-14&tree=all

Comment 76

[Intermittent Failures Robot]

21 failures in 777 pushes (0.027 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 10
* mozilla-inbound: 5
* mozilla-central: 5
* try: 1

Platform breakdown:
* linux64: 14
* linux64-qr: 3
* linux64-nightly: 2
* linux32-nightly: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-05-15&endday=2017-05-21&tree=all

Comment 77

[Intermittent Failures Robot]

10 failures in 891 pushes (0.011 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 6
* mozilla-central: 3
* mozilla-inbound: 1

Platform breakdown:
* linux64: 9
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-05-22&endday=2017-05-28&tree=all

Comment 78

[Intermittent Failures Robot]

16 failures in 182 pushes (0.088 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* autoland: 8
* mozilla-inbound: 6
* mozilla-central: 2

Platform breakdown:
* linux32: 16

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-06-07&endday=2017-06-07&tree=all

Comment 79

[Joel Maher ( :jmaher ) (UTC -8)]

this picked up, 50 failures in the last week, looks like June 5th/6th was a new rate.  Primarily on linux32/opt (non-e10s)

from this log file:
https://treeherder.mozilla.org/logviewer.html#?repo=autoland&job_id=105826337&lineNumber=3065

I see this in the log file:
[task 2017-06-09T13:41:43.056013Z] 13:41:43     INFO - TEST-START | browser/components/sessionstore/test/browser_windowStateContainer.js
[task 2017-06-09T13:41:52.232132Z] 13:41:52     INFO - GECKO(1703) | MEMORY STAT | vsize 1142MB | residentFast 377MB | heapAllocated 172MB
[task 2017-06-09T13:41:52.234193Z] 13:41:52     INFO - TEST-OK | browser/components/sessionstore/test/browser_windowStateContainer.js | took 9176ms
[task 2017-06-09T13:41:52.312716Z] 13:41:52     INFO - checking window state
[task 2017-06-09T13:41:59.565830Z] 13:41:59     INFO - GECKO(1703) | Completed ShutdownLeaks collections in process 1703
[task 2017-06-09T13:41:59.567569Z] 13:41:59     INFO - TEST-START | Shutdown
[task 2017-06-09T13:41:59.570925Z] 13:41:59     INFO - Browser Chrome Test Summary
[task 2017-06-09T13:41:59.572780Z] 13:41:59     INFO - Passed:  1164
[task 2017-06-09T13:41:59.574455Z] 13:41:59     INFO - Failed:  0
[task 2017-06-09T13:41:59.576153Z] 13:41:59     INFO - Todo:    0
[task 2017-06-09T13:41:59.577929Z] 13:41:59     INFO - Mode:    non-e10s
[task 2017-06-09T13:41:59.583430Z] 13:41:59     INFO - *** End BrowserChrome Test Results ***
[task 2017-06-09T13:42:27.029268Z] 13:42:27     INFO - GECKO(1703) | [1703] WARNING: waitpid failed pid:1769 errno:10: file /home/worker/workspace/build/src/ipc/chromium/src/base/process_util_posix.cc, line 276
[task 2017-06-09T13:43:30.897461Z] 13:43:30     INFO - GECKO(1703) | ExceptionHandler::GenerateDump cloned child 1794
[task 2017-06-09T13:43:30.900046Z] 13:43:30     INFO - GECKO(1703) | ExceptionHandler::SendContinueSignalToChild sent continue signal to child
[task 2017-06-09T13:43:30.903394Z] 13:43:30     INFO - GECKO(1703) | ExceptionHandler::WaitForContinueSignal waiting for continue signal...
[task 2017-06-09T13:43:31.443716Z] 13:43:31     INFO - TEST-INFO | Main app process: exit 11
[task 2017-06-09T13:43:31.445760Z] 13:43:31     INFO - Buffered messages finished
[task 2017-06-09T13:43:31.447364Z] 13:43:31    ERROR - TEST-UNEXPECTED-FAIL | browser/components/sessionstore/test/browser_windowStateContainer.js | application terminated with exit code 11
[task 2017-06-09T13:43:31.450202Z] 13:43:31     INFO - runtests.py | Application ran for: 0:07:58.257090
[task 2017-06-09T13:43:31.454019Z] 13:43:31     INFO - zombiecheck | Reading PID log: /tmp/tmpvsQN3opidlog
[task 2017-06-09T13:43:31.455943Z] 13:43:31     INFO - ==> process 1703 launched child process 1722
[task 2017-06-09T13:43:31.458179Z] 13:43:31     INFO - ==> process 1703 launched child process 1769
[task 2017-06-09T13:43:31.461059Z] 13:43:31     INFO - zombiecheck | Checking for orphan process with PID: 1722
[task 2017-06-09T13:43:31.463053Z] 13:43:31     INFO - zombiecheck | Checking for orphan process with PID: 1769
[task 2017-06-09T13:43:31.465148Z] 13:43:31     INFO - mozcrash Downloading symbols from: https://queue.taskcluster.net/v1/task/eD7DKr2pSzesOXbz2HeMXA/artifacts/public/build/target.crashreporter-symbols.zip
[task 2017-06-09T13:43:47.916554Z] 13:43:47     INFO - mozcrash Copy/paste: /usr/local/bin/linux64-minidump_stackwalk /tmp/tmpD_xLTd.mozrunner/minidumps/133ef873-6fa0-403d-463f-f713cdff9012.dmp /tmp/tmpaSKT7V
[task 2017-06-09T13:44:09.925072Z] 13:44:09     INFO - mozcrash Saved minidump as /home/worker/workspace/build/blobber_upload_dir/133ef873-6fa0-403d-463f-f713cdff9012.dmp
[task 2017-06-09T13:44:09.942381Z] 13:44:09     INFO - mozcrash Saved app info as /home/worker/workspace/build/blobber_upload_dir/133ef873-6fa0-403d-463f-f713cdff9012.extra
[task 2017-06-09T13:44:10.051103Z] 13:44:10     INFO - PROCESS-CRASH | browser/components/sessionstore/test/browser_windowStateContainer.js | application crashed [@ RunWatchdog]
[task 2017-06-09T13:44:10.053834Z] 13:44:10     INFO - Crash dump filename: /tmp/tmpD_xLTd.mozrunner/minidumps/133ef873-6fa0-403d-463f-f713cdff9012.dmp
[task 2017-06-09T13:44:10.058048Z] 13:44:10     INFO - Operating system: Linux
[task 2017-06-09T13:44:10.059866Z] 13:44:10     INFO -                   0.0.0 Linux 3.13.0-100-generic #147-Ubuntu SMP Tue Oct 18 16:48:51 UTC 2016 x86_64
[task 2017-06-09T13:44:10.061438Z] 13:44:10     INFO - CPU: x86
[task 2017-06-09T13:44:10.063053Z] 13:44:10     INFO -      GenuineIntel family 6 model 45 stepping 7
[task 2017-06-09T13:44:10.064814Z] 13:44:10     INFO -      1 CPU
[task 2017-06-09T13:44:10.066397Z] 13:44:10     INFO - 
[task 2017-06-09T13:44:10.067941Z] 13:44:10     INFO - GPU: UNKNOWN
[task 2017-06-09T13:44:10.069592Z] 13:44:10     INFO - 
[task 2017-06-09T13:44:10.072581Z] 13:44:10     INFO - Crash reason:  SIGSEGV
[task 2017-06-09T13:44:10.074162Z] 13:44:10     INFO - Crash address: 0x0
[task 2017-06-09T13:44:10.075716Z] 13:44:10     INFO - Process uptime: not available
[task 2017-06-09T13:44:10.078003Z] 13:44:10     INFO - 
[task 2017-06-09T13:44:10.079623Z] 13:44:10     INFO - Thread 14 (crashed)
[task 2017-06-09T13:44:10.082188Z] 13:44:10     INFO -  0  libxul.so!RunWatchdog [nsTerminator.cpp:0a3b78002ba3 : 160 + 0x2]
[task 2017-06-09T13:44:10.083905Z] 13:44:10     INFO -     eip = 0xf38654fd   esp = 0xc30fe2f0   ebp = 0xc30fe318   ebx = 0xf5615000
[task 2017-06-09T13:44:10.086096Z] 13:44:10     INFO -     esi = 0x0000003f   edi = 0xbef2f574   eax = 0x00000000   ecx = 0x00000000
[task 2017-06-09T13:44:10.088787Z] 13:44:10     INFO -     edx = 0xf47a8408   efl = 0x00010246
[task 2017-06-09T13:44:10.090974Z] 13:44:10     INFO -     Found by: given as instruction pointer in context
[task 2017-06-09T13:44:10.093279Z] 13:44:10     INFO -  1  libnspr4.so!_pt_root [ptthread.c:0a3b78002ba3 : 216 + 0x9]
[task 2017-06-09T13:44:10.098195Z] 13:44:10     INFO -     eip = 0xf73b4c44   esp = 0xc30fe320   ebp = 0xc30fe368   ebx = 0xf73c56c4
[task 2017-06-09T13:44:10.099791Z] 13:44:10     INFO -     esi = 0xbf045d40   edi = 0x000006fc
[task 2017-06-09T13:44:10.101360Z] 13:44:10     INFO -     Found by: call frame info
[task 2017-06-09T13:44:10.102954Z] 13:44:10     INFO -  2  libpthread-2.23.so + 0x6295
[task 2017-06-09T13:44:10.104620Z] 13:44:10     INFO -     eip = 0xf777f295   esp = 0xc30fe370   ebp = 0xc30fe428   ebx = 0x00000000
[task 2017-06-09T13:44:10.106366Z] 13:44:10     INFO -     esi = 0x00000000   edi = 0x003d0f00
[task 2017-06-09T13:44:10.107927Z] 13:44:10     INFO -     Found by: call frame info
[task 2017-06-09T13:44:10.109510Z] 13:44:10     INFO -  3  libc-2.23.so + 0xe6eee
[task 2017-06-09T13:44:10.111173Z] 13:44:10     INFO -     eip = 0xf74b1eee   esp = 0xc30fe430   ebp = 0x00000000
[task 2017-06-09T13:44:10.114172Z] 13:44:10     INFO -     Found by: previous frame's frame pointer
[task 2017-06-09T13:44:10.115711Z] 13:44:10     INFO - 
[task 2017-06-09T13:44:10.117257Z] 13:44:10     INFO - Thread 0
[task 2017-06-09T13:44:10.119043Z] 13:44:10     INFO -  0  libxul.so!XPCRootSetElem::RemoveFromRootSet [XPCJSRuntime.cpp:0a3b78002ba3 : 3171 + 0x5]
[task 2017-06-09T13:44:10.122247Z] 13:44:10     INFO -     eip = 0xf0c874eb   esp = 0xffbed7d0   ebp = 0xffbed7e8   ebx = 0xf5615000
[task 2017-06-09T13:44:10.123895Z] 13:44:10     INFO -     esi = 0xcee9a958   edi = 0xcee9a940   eax = 0xceee6058   ecx = 0xf71352cc
[task 2017-06-09T13:44:10.125572Z] 13:44:10     INFO -     edx = 0xdf1fc898   efl = 0x00010286
[task 2017-06-09T13:44:10.127406Z] 13:44:10     INFO -     Found by: given as instruction pointer in context
[task 2017-06-09T13:44:10.129738Z] 13:44:10     INFO -  1  libxul.so!nsXPCWrappedJS::Release [XPCWrappedJS.cpp:0a3b78002ba3 : 281 + 0xe]
[task 2017-06-09T13:44:10.133705Z] 13:44:10     INFO -     eip = 0xf0ca93a5   esp = 0xffbed7f0   ebp = 0xffbed838   ebx = 0xf5615000
[task 2017-06-09T13:44:10.135440Z] 13:44:10     INFO -     esi = 0xcee9a948   edi = 0xcee9a940
[task 2017-06-09T13:44:10.137009Z] 13:44:10     INFO -     Found by: call frame info
[task 2017-06-09T13:44:10.138695Z] 13:44:10     INFO -  2  libxul.so!nsXPTCStubBase::Release [xptcall.cpp:0a3b78002ba3 : 37 + 0xe]
[task 2017-06-09T13:44:10.140335Z] 13:44:10     INFO -     eip = 0xf031ba04   esp = 0xffbed840   ebp = 0xffbed858   ebx = 0xf5615000
[task 2017-06-09T13:44:10.141965Z] 13:44:10     INFO -     esi = 0xcc8ec220   edi = 0xcc8ec218
[task 2017-06-09T13:44:10.143689Z] 13:44:10     INFO -     Found by: call frame info
[task 2017-06-09T13:44:10.146701Z] 13:44:10     INFO -  3  libxul.so!nsTArray_Impl<ObserverRef, nsTArrayInfallibleAllocator>::RemoveElementsAt [nsCOMPtr.h:0a3b78002ba3 : 294 + 0x8]
[task 2017-06-09T13:44:10.148483Z] 13:44:10     INFO -     eip = 0xf02c626f   esp = 0xffbed860   ebp = 0xffbed888   ebx = 0xf5615000
[task 2017-06-09T13:44:10.150144Z] 13:44:10     INFO -     esi = 0xcc8ec220   edi = 0xcc8ec218
[task 2017-06-09T13:44:10.152087Z] 13:44:10     INFO -     Found by: call frame info
[task 2017-06-09T13:44:10.154216Z] 13:44:10     INFO -  4  libxul.so!nsTHashtable<nsObserverList>::s_ClearEntry [nsTArray.h:0a3b78002ba3 : 1738 + 0x12]
[task 2017-06-09T13:44:10.157955Z] 13:44:10     INFO -     eip = 0xf02c658b   esp = 0xffbed890   ebp = 0xffbed8b8   ebx = 0xf5615000
[task 2017-06-09T13:44:10.159553Z] 13:44:10     INFO -     esi = 0xe36132b4   edi = 0xe36132bc
[task 2017-06-09T13:44:10.161115Z] 13:44:10     INFO -     Found by: call frame info


so we are failing on shutdown?


:mcmanus, can you help find someone to look into this failure in the next 2 weeks?

Comment 80

[Patrick McManus [:mcmanus]]

Hey Joel - so I don't see why this was marked a cache bug 2 years ago. I took a look at the following

https://public-artifacts.taskcluster.net/S95MYI02S8Sg1Dn2k7vOOg/0/public/logs/live_backing.log
https://treeherder.mozilla.org/logviewer.html#?repo=autoland&job_id=105826337&lineNumber=3065
https://public-artifacts.taskcluster.net/ABSg1wEAR0agSqyDcUgXKA/0/public/logs/live_backing.log

and all of them are processing shutdown.. in all of them the networking thread has completed and I don't easily see any cache thread either. I'll NI honza to confirm the networking cache is not involved.

I'm not sure where to reassign either.. none of them look deadlocked, just usually doing some js gc cleanup at different stages.

Comment 81

[Intermittent Failures Robot]

49 failures in 864 pushes (0.057 failures/push) were associated with this bug in the last 7 days. 

This is the #28 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* autoland: 21
* mozilla-inbound: 19
* mozilla-central: 9

Platform breakdown:
* linux32: 48
* windows8-64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-06-05&endday=2017-06-11&tree=all

Comment 82

[Honza Bambas (:mayhemer)]

Confirming the shutdown crashes (the only UNEXPECTED-FAIL I was able to find in those 3 logs) are not related to http cache.

Comment 83

[Intermittent Failures Robot]

21 failures in 168 pushes (0.125 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* mozilla-inbound: 16
* autoland: 3
* mozilla-central: 2

Platform breakdown:
* linux32: 19
* windows8-64: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-06-14&endday=2017-06-14&tree=all

Comment 84

[Intermittent Failures Robot]

34 failures in 814 pushes (0.042 failures/push) were associated with this bug in the last 7 days.   

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* mozilla-inbound: 22
* autoland: 9
* mozilla-central: 3

Platform breakdown:
* linux32: 32
* windows8-64: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-06-12&endday=2017-06-18&tree=all

Comment 85

[Joel Maher ( :jmaher ) (UTC -8)]

Mike- this is always occurring at the end of sessionrestore, could you help find someone on the sessionrestore team to look into why we are crashing often on linux32/opt non-e10s browser-chrome tests for sessionrestore?

Comment 86

[Mike de Boer [:mikedeboer]]

Why are you sure that this is occurring due to sessionstore? What do you mean with 'end of sessionrestore'?
When I look at the brasstacks view, it looks like bug 1373116, but in disguise. Let's ask Yoshi again!

Comment 87

[Intermittent Failures Robot]

22 failures in 892 pushes (0.025 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 8
* autoland: 8
* mozilla-central: 6

Platform breakdown:
* windows8-64: 21
* windows7-32-vm: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-06-19&endday=2017-06-25&tree=all

Comment 88

[Geoff Brown [:gbrown]]

(In reply to Mike de Boer [:mikedeboer] from comment #86)
> Why are you sure that this is occurring due to sessionstore? What do you
> mean with 'end of sessionrestore'?
> When I look at the brasstacks view, it looks like bug 1373116, but in
> disguise. Let's ask Yoshi again!

Most of the linux32 opt failures reported around June 14 are nsTerminator shutdown crashes which occur at the end of the browser/components/sessionstore/test tests. That is indeed just like bug 1373116 (I think the bug 1373116 failures were reported here until bug 1373116 was opened).

Beginning around June 19, we started seeing similar failures on Windows reported here, but those do not seem to be associated with browser/components/sessionstore/test, or any other particular test directory.

Comment 89

[Intermittent Failures Robot]

33 failures in 718 pushes (0.046 failures/push) were associated with this bug in the last 7 days. 

This is the #50 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* autoland: 17
* mozilla-inbound: 11
* mozilla-central: 5

Platform breakdown:
* windows8-64: 31
* windows7-32-vm: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-06-26&endday=2017-07-02&tree=all

Comment 90

[Intermittent Failures Robot]

1 failures in 656 pushes (0.002 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 1

Platform breakdown:
* windows7-32-vm: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-07-03&endday=2017-07-09&tree=all

Comment 91

[Joel Maher ( :jmaher ) (UTC -8)]

this appears to be fixed

Comment 92

[Intermittent Failures Robot]

8 failures in 901 pushes (0.009 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 4
* autoland: 4

Platform breakdown:
* windows7-32: 7
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-08-07&endday=2017-08-13&tree=all

Comment 93

[Intermittent Failures Robot]

11 failures in 949 pushes (0.012 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 9
* mozilla-inbound: 1
* mozilla-central: 1

Platform breakdown:
* windows7-32: 10
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-08-14&endday=2017-08-20&tree=all

Comment 94

[Intermittent Failures Robot]

4 failures in 908 pushes (0.004 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-central: 3
* mozilla-inbound: 1

Platform breakdown:
* windows7-32: 4

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-08-21&endday=2017-08-27&tree=all

Comment 95

[Joel Maher ( :jmaher ) (UTC -8)]

August 28th, this started failing, 21 times since then:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587

primarily on win7/debug and linux64/asan configurations.

Comment 110

[Intermittent Failures Robot]

25 failures in 939 pushes (0.027 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 18
* mozilla-inbound: 4
* mozilla-central: 3

Platform breakdown:
* linux64: 15
* windows7-32: 8
* windows7-32-nightly: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-08-28&endday=2017-09-03&tree=all

Comment 111

[Intermittent Failures Robot]

24 failures in 924 pushes (0.026 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 16
* mozilla-inbound: 8

Platform breakdown:
* linux64: 13
* osx-10-10: 8
* windows7-32: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-09-04&endday=2017-09-10&tree=all

Comment 112

[Firefox Bug Husbandry Bot]

Bulk change to priority: https://bugzilla.mozilla.org/show_bug.cgi?id=1399258

Comment 113

[Firefox Bug Husbandry Bot]

Bulk change to priority: https://bugzilla.mozilla.org/show_bug.cgi?id=1399258

Comment 114

[Intermittent Failures Robot]

9 failures in 1032 pushes (0.009 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 4
* mozilla-inbound: 3
* mozilla-central: 2

Platform breakdown:
* linux64: 8
* windows7-32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-09-11&endday=2017-09-17&tree=all

Comment 115

[Intermittent Failures Robot]

16 failures in 943 pushes (0.017 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 8
* mozilla-inbound: 4
* mozilla-central: 4

Platform breakdown:
* linux64: 15
* windows7-32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-09-18&endday=2017-09-24&tree=all

Comment 116

[Intermittent Failures Robot]

16 failures in 885 pushes (0.018 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 8
* autoland: 6
* mozilla-central: 1
* mozilla-beta: 1

Platform breakdown:
* linux64: 12
* windows7-32-stylo-disabled: 2
* linux32-stylo-disabled: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-09-25&endday=2017-10-01&tree=all

Comment 117

[Intermittent Failures Robot]

13 failures in 824 pushes (0.016 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 8
* mozilla-inbound: 4
* mozilla-central: 1

Platform breakdown:
* linux64: 11
* windows7-32: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-10-02&endday=2017-10-08&tree=all

Comment 118

[Intermittent Failures Robot]

18 failures in 947 pushes (0.019 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 7
* mozilla-inbound: 6
* mozilla-release: 3
* mozilla-central: 2

Platform breakdown:
* linux64: 15
* windows7-32: 2
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-10-09&endday=2017-10-15&tree=all

Comment 119

[Intermittent Failures Robot]

6 failures in 864 pushes (0.007 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 2
* mozilla-central: 2
* autoland: 2

Platform breakdown:
* linux64: 5
* windows7-32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-10-16&endday=2017-10-22&tree=all

Comment 120

[Intermittent Failures Robot]

16 failures in 912 pushes (0.018 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 8
* autoland: 5
* mozilla-central: 3

Platform breakdown:
* linux64: 9
* windows7-32: 3
* windows10-64: 2
* windows7-32-stylo-disabled: 1
* osx-10-10: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-10-23&endday=2017-10-29&tree=all

Comment 121

[Tiberius Oros[:tiberius_oros]]

In the last 7 days there have been 33 failures. 
Most of the failures occur on the Linux x64 platform. There are also some that occur on Windows 7 and windows10-64. A very low number occur on oher platforms like: .
OS X 10.10, windows7-32-stylo-disabled.
The failures occur mostly on the asan build type. There are also a few failures on debug and a single one on opt.

Here is an example of a recent log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=141231965&lineNumber=36298

And a relevant snippet of the log:
[task 2017-11-01T03:45:49.856Z] 03:45:49     INFO - GECKO(3102) | ASAN:DEADLYSIGNAL
36297
[task 2017-11-01T03:45:49.858Z] 03:45:49     INFO - GECKO(3102) | =================================================================
36298
[task 2017-11-01T03:45:49.859Z] 03:45:49    ERROR - GECKO(3102) | ==3102==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7efc30b950be bp 0x7efbd5c5adf0 sp 0x7efbd5c5ade0 T180)
36299
[task 2017-11-01T03:45:49.859Z] 03:45:49     INFO - GECKO(3102) | ==3102==The signal is caused by a WRITE memory access.
36300
[task 2017-11-01T03:45:49.859Z] 03:45:49     INFO - GECKO(3102) | ==3102==Hint: address points to the zero page.
36301
[task 2017-11-01T03:45:50.243Z] 03:45:50     INFO - GECKO(3102) |     #0 0x7efc30b950bd in mozilla::(anonymous namespace)::RunWatchdog(void*) /builds/worker/workspace/build/src/toolkit/components/terminator/nsTerminator.cpp:163:5
36302
[task 2017-11-01T03:45:50.245Z] 03:45:50     INFO - GECKO(3102) |     #1 0x7efc412d74d3 in _pt_root /builds/worker/workspace/build/src/nsprpub/pr/src/pthreads/ptthread.c:216:5
36303
[task 2017-11-01T03:45:50.253Z] 03:45:50     INFO - GECKO(3102) |     #2 0x7efc4564e6b9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76b9)
36304
[task 2017-11-01T03:45:50.302Z] 03:45:50     INFO - GECKO(3102) |     #3 0x7efc446d73dc in clone /build/glibc-bfm8X4/glibc-2.23/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:109
36305
[task 2017-11-01T03:45:50.303Z] 03:45:50     INFO - GECKO(3102) | AddressSanitizer can not provide additional info.
36306
[task 2017-11-01T03:45:50.304Z] 03:45:50     INFO - GECKO(3102) | SUMMARY: AddressSanitizer: SEGV /builds/worker/workspace/build/src/toolkit/components/terminator/nsTerminator.cpp:163:5 in mozilla::(anonymous namespace)::RunWatchdog(void*)



:mikedeboer, we've noticed that in the last days the failures increased. Do you think you can find anyone who can look into this?

Comment 122

[Intermittent Failures Robot]

38 failures in 857 pushes (0.044 failures/push) were associated with this bug in the last 7 days. 

This is the #44 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* autoland: 18
* mozilla-inbound: 16
* mozilla-central: 4

Platform breakdown:
* linux64: 26
* windows7-32: 7
* windows10-64: 3
* windows7-32-stylo-disabled: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-10-30&endday=2017-11-05&tree=all

Comment 123

[Mike de Boer [:mikedeboer]]

Hi Tiberius, sorry I can't help you with this specific bug. In fact, Joel triaged this to be part of Session Restore, but that doesn't seem to be true at all... right?
I'm wary of being the one to ping-pong stuff around, but I can't tell anything from the logs and I don't have the necessary knowledge to gather enough useful data when I look at how low level the failure actually is.

Joel, wouldn't it be more reasonable to find a platform engineer who may be able to do a preliminary analysis from the bottom up?

Comment 124

[Joel Maher ( :jmaher ) (UTC -8)]

given that we seem to be failing on toolkit/components/terminator/nsTerminator.cpp, moving to toolkit::async tooling- ideally we can get more info there.

Comment 125

[Joel Maher ( :jmaher ) (UTC -8)]

:yoric, could you help figure out the runWatchdog crash we are seeing in comment 121?

Comment 126

[David Teller [:Yoric] - still alive but not very active]

Well, as usual with nsTerminator, this means that shutdown has timed out and we killed Firefox. Normally, nsTerminator should provide a crash report with slightly more details (i.e. which shutdown phase lasted more than 63 seconds), but this doesn't seem to show up in the log. I suppose we could patch the nsTerminator to dump this info to stderr on DEBUG builds, but I don't think that this would help much.

I'll be happy to answer any question, but I suspect that I won't be able to help much.

Comment 127

[Joel Maher ( :jmaher ) (UTC -8)]

as this is heavily weighted on linux64-asan, do you think we could adjust timing for slow configurations?  I think right now we are stuck with a dead end, so if there is any additional information we could get it would be helpful!

Comment 128

[David Teller [:Yoric] - still alive but not very active]

I suppose we could use a different constant in ASAN builds, if one single step of shutdown takes more than 1 wallclock minute. Do we have a build flag for ASAN?

Comment 129

[David Teller [:Yoric] - still alive but not very active]

Ok, MOZ_ASAN. I'll try and build a patch to make nsTerminator more ASAN-friendly. I suspect that this will just change the crashes into harness timeouts, but we can try.

Comment 130

[Joel Maher ( :jmaher ) (UTC -8)]

I think we can do MOZ_ASAN || MOZ_DEBUG, I see these referenced here:
http://searchfox.org/mozilla-central/source/js/src/jsutil.h#334

Comment 131

[David Teller [:Yoric] - still alive but not very active]

Attachment: Bug 1108587 - Extending the grace period of AsyncShutdown and the nsTerminator for ASAN builds;

Bug 1108587 seems to suggest that the grace period for nsTerminator is
too short for ASAN builds, which take much longer to shutdown. This
patch changes the grace period from 1 minute to 3 minutes, hoping that
this will be sufficient. Somewhere along the way, we also extend the
duration of AsyncShutdown, because that's the simplest way to do both
at once.

Review commit: https://reviewboard.mozilla.org/r/198140/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/198140/

Comment 132

[Gabriele Svelto [:gsvelto]]

Comment on attachment 8926895 [details]
Bug 1108587 - Extending the grace period of AsyncShutdown and the nsTerminator for ASAN builds;

https://reviewboard.mozilla.org/r/198140/#review203386

LGTM, I've seen ASAN try runs turn orange so many times that anything that makes them greener is appreciated.

Comment 133

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/56e1ecb8bb58
Extending the grace period of AsyncShutdown and the nsTerminator for ASAN builds;r=gsvelto

Comment 134

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/56e1ecb8bb58

Comment 135

[Intermittent Failures Robot]

44 failures in 849 pushes (0.052 failures/push) were associated with this bug in the last 7 days. 

This is the #42 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* mozilla-inbound: 16
* mozilla-central: 15
* autoland: 13

Platform breakdown:
* linux64: 30
* windows7-32: 6
* windows7-32-stylo-disabled: 3
* windows10-64-stylo-disabled: 3
* windows10-64: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1108587&startday=2017-11-06&endday=2017-11-12&tree=all