Intermittent test_refresh_firefox.py TestFirefoxRefresh.testReset | application crashed [@ RunWatchdog]

RESOLVED DUPLICATE of bug 1425323

Status

()

Firefox
Migration
P3
critical
RESOLVED DUPLICATE of bug 1425323
7 months ago
18 days ago

People

(Reporter: Treeherder Bug Filer, Unassigned)

Tracking

(Blocks: 1 bug, 4 keywords)

Trunk
crash, intermittent-failure, regression, regressionwindow-wanted
Points:
---

Firefox Tracking Flags

(firefox56 unaffected, firefox57+ fix-optional, firefox58 wontfix, firefox59 ?)

Details

(crash signature)

(Reporter)

Description

7 months ago
treeherder
Filed by: hskupin [at] gmail.com

https://treeherder.mozilla.org/logviewer.html#?job_id=130580283&repo=try

https://queue.taskcluster.net/v1/task/HXxEKU5oSySyl0JxLAHB-A/runs/0/artifacts/public/logs/live_backing.log

[task 2017-09-13T08:51:32.790132Z] 08:51:32     INFO - Crash reason:  SIGSEGV
[task 2017-09-13T08:51:32.790853Z] 08:51:32     INFO - Crash address: 0x0
[task 2017-09-13T08:51:32.791383Z] 08:51:32     INFO - Process uptime: not available
[task 2017-09-13T08:51:32.791673Z] 08:51:32     INFO - 
[task 2017-09-13T08:51:32.792249Z] 08:51:32     INFO - Thread 27 (crashed)
[task 2017-09-13T08:51:32.792934Z] 08:51:32     INFO -  0  libxul.so!RunWatchdog [nsTerminator.cpp:a73cc4e08bf5 : 160 + 0x0]
[task 2017-09-13T08:51:32.793289Z] 08:51:32     INFO -     rax = 0x0000000000000000   rdx = 0x0000000000000000
[task 2017-09-13T08:51:32.793860Z] 08:51:32     INFO -     rcx = 0x00007ff28e0692ad   rbx = 0x00007ff24f2674e8
[task 2017-09-13T08:51:32.796891Z] 08:51:32     INFO -     rsi = 0x00007ff28e338770   rdi = 0x00007ff28e337540
[task 2017-09-13T08:51:32.796959Z] 08:51:32     INFO -     rbp = 0x00007ff24a465ec0   rsp = 0x00007ff24a465eb0
[task 2017-09-13T08:51:32.797064Z] 08:51:32     INFO -      r8 = 0x00007ff28e338770    r9 = 0x00007ff24a466700
[task 2017-09-13T08:51:32.797125Z] 08:51:32     INFO -     r10 = 0x0000000000000012   r11 = 0x0000000000000000
[task 2017-09-13T08:51:32.797232Z] 08:51:32     INFO -     r12 = 0x000000000000003f   r13 = 0x00000000000014c2
[task 2017-09-13T08:51:32.798058Z] 08:51:32     INFO -     r14 = 0x00007ff24a466700   r15 = 0x00007ff24a466670
[task 2017-09-13T08:51:32.798109Z] 08:51:32     INFO -     rip = 0x00007ff27f753590
[task 2017-09-13T08:51:32.798202Z] 08:51:32     INFO -     Found by: given as instruction pointer in context
[task 2017-09-13T08:51:32.798258Z] 08:51:32     INFO -  1  libnspr4.so!_pt_root [ptthread.c:a73cc4e08bf5 : 216 + 0x7]
[task 2017-09-13T08:51:32.798312Z] 08:51:32     INFO -     rbx = 0x00007ff250094800   rbp = 0x00007ff24a465f10
[task 2017-09-13T08:51:32.799102Z] 08:51:32     INFO -     rsp = 0x00007ff24a465ed0   r12 = 0x0000000000000001
[task 2017-09-13T08:51:32.799891Z] 08:51:32     INFO -     r13 = 0x00000000000014c2   r14 = 0x00007ff24a466700
[task 2017-09-13T08:51:32.800712Z] 08:51:32     INFO -     r15 = 0x00007ff24a466670   rip = 0x00007ff28d8d16e6
[task 2017-09-13T08:51:32.801419Z] 08:51:32     INFO -     Found by: call frame info
[task 2017-09-13T08:51:32.802104Z] 08:51:32     INFO -  2  libpthread-2.23.so + 0x76ba
[task 2017-09-13T08:51:32.802868Z] 08:51:32     INFO -     rbx = 0x0000000000000000   rbp = 0x0000000000000000
[task 2017-09-13T08:51:32.803407Z] 08:51:32     INFO -     rsp = 0x00007ff24a465f20   r12 = 0x0000000000000000
[task 2017-09-13T08:51:32.804057Z] 08:51:32     INFO -     r13 = 0x00007ffd1916420f   r14 = 0x00007ff24a4669c0
[task 2017-09-13T08:51:32.804703Z] 08:51:32     INFO -     r15 = 0x00007ffd191642a0   rip = 0x00007ff28eff06ba
[task 2017-09-13T08:51:32.805379Z] 08:51:32     INFO -     Found by: call frame info
[task 2017-09-13T08:51:32.806027Z] 08:51:32     INFO -  3  libc-2.23.so + 0x1073dd
[task 2017-09-13T08:51:32.806717Z] 08:51:32     INFO -     rsp = 0x00007ff24a465fc0   rip = 0x00007ff28e0793dd
[task 2017-09-13T08:51:32.806915Z] 08:51:32     INFO -     Found by: stack scanning

Looks like there is something blocking us from shutdown, and Firefox gets killed.
Blocks: 1358898
I see the following in the log:

[task 2017-09-15T08:17:53.934Z] 08:17:53     INFO -  1505463473929	Marionette	DEBUG	Received observer notification "xpcom-shutdown"
[task 2017-09-15T08:17:53.955Z] 08:17:53     INFO -  [Parent 4712, Main Thread] WARNING: 'NS_FAILED(rr->RetargetDeliveryTo(sts))', file /builds/worker/workspace/build/src/dom/fetch/FetchDriver.cpp, line 661
[task 2017-09-15T08:17:53.956Z] 08:17:53     INFO -  [Parent 4712, Main Thread] WARNING: 'NS_FAILED(rv)', file /builds/worker/workspace/build/src/dom/fetch/FetchConsumer.cpp, line 516
[task 2017-09-15T08:17:53.957Z] 08:17:53     INFO -  [Parent 4712, Main Thread] WARNING: Retargeting failed: file /builds/worker/workspace/build/src/dom/fetch/FetchConsumer.cpp, line 517
[task 2017-09-15T08:17:54.038Z] 08:17:54     INFO -  [Parent 4712, Main Thread] WARNING: A runnable was posted to a worker that is already shutting down!: file /builds/worker/workspace/build/src/dom/workers/WorkerPrivate.cpp, line 2958
[task 2017-09-15T08:17:54.040Z] 08:17:54     INFO -  [Parent 4712, Main Thread] WARNING: Could not dispatch ConsumeBodyRunnable: file /builds/worker/workspace/build/src/dom/fetch/FetchConsumer.cpp, line 224
[task 2017-09-15T08:18:21.499Z] 08:18:21     INFO -  JavaScript error: resource://gre/modules/osfile/osfile_async_front.jsm, line 410: Error: OS.File has been shut down. Rejecting post to stat
[task 2017-09-15T08:18:57.334Z] 08:18:57     INFO -  Hit MOZ_CRASH(Shutdown too long, probably frozen, causing a crash.) at /builds/worker/workspace/build/src/toolkit/components/terminator/nsTerminator.cpp:160
[task 2017-09-15T08:18:57.336Z] 08:18:57     INFO -  #01: ???[/builds/worker/workspace/build/application/firefox/libnspr4.so +0x286e6]
[task 2017-09-15T08:18:57.337Z] 08:18:57     INFO -  #02: ???[/lib/x86_64-linux-gnu/libpthread.so.0 +0x76ba]
[task 2017-09-15T08:18:57.338Z] 08:18:57     INFO -  #03: clone[/lib/x86_64-linux-gnu/libc.so.6 +0x1073dd]
[task 2017-09-15T08:18:57.340Z] 08:18:57     INFO -  #04: ??? (???:???)

Andrea, could this be one more issue with FetchConsumer and which is causing a shutdown hang?
Flags: needinfo?(amarchesini)
Those hangs started on September 13th on autoland if it helps.
status-firefox56: --- → unaffected
status-firefox57: --- → affected
Keywords: regression, regressionwindow-wanted
tracking-firefox57: --- → ?

Comment 3

7 months ago
20 failures in 1032 pushes (0.019 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 10
* mozilla-inbound: 6
* try: 2
* mozilla-central: 2

Platform breakdown:
* linux64: 12
* linux32: 8

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1399601&startday=2017-09-11&endday=2017-09-17&tree=all
tracking-firefox57: ? → +

Comment 4

7 months ago
9 failures in 943 pushes (0.01 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 4
* autoland: 3
* mozilla-central: 1
* mozilla-beta: 1

Platform breakdown:
* linux32: 8
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1399601&startday=2017-09-18&endday=2017-09-24&tree=all
This seems more related to worker shutting down. I'm planning to change this part soon.
Flags: needinfo?(amarchesini)

Comment 6

7 months ago
Hey baku, this is currently a P5 but triage managers felt it might be more important. Can you update the priority and let us know when you expect to get to this?
Flags: needinfo?(amarchesini)
I would say P3. overholt?
Flags: needinfo?(amarchesini) → needinfo?(overholt)
Priority: P5 → P3
Since it's something baku's likely to work on in the next few months, let's go with P2. (The component here seems odd)
Flags: needinfo?(overholt)
Priority: P3 → P2

Comment 9

7 months ago
11 failures in 885 pushes (0.012 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 6
* mozilla-inbound: 4
* try: 1

Platform breakdown:
* linux32: 11

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1399601&startday=2017-09-25&endday=2017-10-01&tree=all

Updated

7 months ago
status-firefox57: affected → fix-optional
OS: Unspecified → All
Priority: P2 → P3
Hardware: Unspecified → All
Version: unspecified → Trunk

Comment 10

7 months ago
2 failures in 824 pushes (0.002 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 2

Platform breakdown:
* linux32: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1399601&startday=2017-10-02&endday=2017-10-08&tree=all

Comment 11

6 months ago
11 failures in 947 pushes (0.012 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-central: 4
* mozilla-inbound: 3
* autoland: 3
* mozilla-beta: 1

Platform breakdown:
* linux32: 9
* linux64: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1399601&startday=2017-10-09&endday=2017-10-15&tree=all

Comment 12

6 months ago
9 failures in 864 pushes (0.01 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 7
* mozilla-inbound: 1
* mozilla-central: 1

Platform breakdown:
* linux32: 9

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1399601&startday=2017-10-16&endday=2017-10-22&tree=all

Comment 13

6 months ago
14 failures in 912 pushes (0.015 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 8
* mozilla-inbound: 4
* mozilla-central: 2

Platform breakdown:
* linux32: 8
* windows7-32: 5
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1399601&startday=2017-10-23&endday=2017-10-29&tree=all

Comment 14

6 months ago
5 failures in 857 pushes (0.006 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-central: 2
* autoland: 2
* mozilla-inbound: 1

Platform breakdown:
* linux32: 3
* windows7-32: 1
* windows10-64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1399601&startday=2017-10-30&endday=2017-11-05&tree=all
status-firefox58: --- → fix-optional

Comment 15

5 months ago
9 failures in 849 pushes (0.011 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 7
* mozilla-inbound: 2

Platform breakdown:
* linux64: 7
* windows7-32: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1399601&startday=2017-11-06&endday=2017-11-12&tree=all

Comment 16

5 months ago
1 failures in 762 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 1

Platform breakdown:
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1399601&startday=2017-11-13&endday=2017-11-19&tree=all

Comment 17

5 months ago
3 failures in 744 pushes (0.004 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 3

Platform breakdown:
* windows7-32: 2
* windows10-64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1399601&startday=2017-11-20&endday=2017-11-26&tree=all

Comment 18

5 months ago
7 failures in 792 pushes (0.009 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 4
* mozilla-inbound: 2
* mozilla-central: 1

Platform breakdown:
* linux32: 5
* windows10-64: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1399601&startday=2017-11-27&endday=2017-12-03&tree=all

Comment 19

4 months ago
4 failures in 889 pushes (0.004 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* try: 2
* mozilla-inbound: 2

Platform breakdown:
* windows10-64-ccov: 2
* windows10-64: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1399601&startday=2017-12-04&endday=2017-12-10&tree=all

Comment 20

4 months ago
3 failures in 423 pushes (0.007 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 1
* mozilla-central: 1
* mozilla-beta: 1

Platform breakdown:
* windows7-32-devedition: 1
* windows10-64-ccov: 1
* windows10-64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1399601&startday=2017-12-11&endday=2017-12-17&tree=all

Comment 21

3 months ago
1 failures in 462 pushes (0.002 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* try: 1

Platform breakdown:
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1399601&startday=2018-01-01&endday=2018-01-07&tree=all
This bug looks like a dupe of bug 1425323 now. Here a link to the stack:

https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=151425042&lineNumber=47965

Andrea, I assume sometimes in December the quota manager got moved from the main thread to it's own thread?
Flags: needinfo?(amarchesini)
> Andrea, I assume sometimes in December the quota manager got moved from the
> main thread to it's own thread?

Yes, but the shutdown notifications are received on the main-thread. I agree. it's a dup of bug 1425323.
Flags: needinfo?(amarchesini)
Status: NEW → RESOLVED
Last Resolved: 2 months ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1425323

Comment 26

2 months ago
4 failures in 685 pushes (0.006 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 4

Platform breakdown:
* windows7-32: 4

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1399601&startday=2018-02-12&endday=2018-02-18&tree=all
You need to log in before you can comment on or make changes to this bug.