1300467 - Crash in AsyncShutdownTimeout | profile-before-change | OS.File: flush I/O queued before profileBeforeChange

Status: RESOLVED WONTFIX

(Toolkit Graveyard :: OS.File, defect)

Description

[Nicholas Nethercote [inactive]]

This bug was filed from the Socorro interface and is 
report bp-f6f77ec4-7074-4914-a3e6-7d2f42160903.
=============================================================

This crash has been around for a while, going back to at least 44.0b1. In the past seven days there have been 700+ crashes across all these versions. Surprisingly, there doesn't appear to be a bug filed for it.

I confess to not understanding how the crash signature is created for this crash; the signature bears no relation to the stack trace. But it looks like it involves OS.File shutdown.

Yoric, can you please take a look?

Comment 1

[David Teller [:Yoric] - still alive but not very active]

Yeah, it's normal that the signature has no relation with the stacks: the C++ stack only confirms that the crash has been triggered voluntarily (by design of AsyncShutdown), while the real stacks are hidden somewhere in AsyncShutdownTimeout.

Here, we have a blocker "CrashMonitor: Writing notifications to file after receiving profile-before-change" that's never lifted. The blocker is created with the following stack:
  resource://gre/modules/CrashMonitor.jsm:this.CrashMonitor.init:173
  resource://gre/components/nsCrashMonitor.js:CrashMonitor.prototype.observe:24

However, there are very good chances that it's OS.File's fault anyway. I notice:
      "messagesSent": 16,
      "messagesReceived": 15,
      "messagesQueued": 27,
which pretty much means that OS.File has stopped responding after the 15th message, which was a call to
      "writeAtomic"

This probably means one of three things: 
- there's a bug in writeAtomic, and that somehow killed the ChromeWorker's ability to respond; or
- there's a bug in the operation immediately before writeAtomic, and that somehow killed the ChromeWorker thread's ability to respond; or
- we have attempted to reset the ChromeWorker thread (this was turned on in bug 1279389) and something failed.

bkelly, do you know if there is a correlation between the number of these crashes and bug 1279389?

Comment 2

[Ben Kelly [:bkelly, not reviewing]]

(In reply to Nicholas Nethercote [:njn] from comment #0)
> This crash has been around for a while, going back to at least 44.0b1. In
> the past seven days there have been 700+ crashes across all these versions.

Bug 1279389 only changed the os.file pref for non-release builds.  The other changes in that bug were to a test and to call iterator.close() on more os.file iterators.  It was landed in FF50.

I don't think it explains these crashes in release (48) and beta (49).

Comment 3

[Ben Kelly [:bkelly, not reviewing]]

It seems all of these crashes have an extension with a version number matching the FF version:

 	{972ce4c6-7e08-4474-a285-3208198ce6fd} 	49.0
        {972ce4c6-7e08-4474-a285-3208198ce6fd} 	48.0.2
        {972ce4c6-7e08-4474-a285-3208198ce6fd} 	47.0.1

Do we know what this thing is?  I don't recall seeing it before.

Comment 4

[Andrew McCreight [:mccr8]]

> Do we know what this thing is?  I don't recall seeing it before.

There's always been some generic system-y extension that shows up. I assume this is the same thing.

Comment 5

[Nicholas Nethercote [inactive]]

>  	{972ce4c6-7e08-4474-a285-3208198ce6fd} 	49.0

I documented this at https://developer.mozilla.org/en-US/docs/Understanding_crash_reports. It's the Firefox theme, so it's always present.

Comment 6

[Nicholas Nethercote [inactive]]

I filed bug 1301222 to better identify the Firefox default theme in crash reports.

Comment 7

[BugBot [:suhaib / :marco/ :calixte]]

Crash volume for signature 'AsyncShutdownTimeout | profile-before-change | OS.File: flush I/O queued before profileBeforeChange':
 - nightly (version 52): 12 crashes from 2016-09-19.
 - aurora  (version 51): 22 crashes from 2016-09-19.
 - beta    (version 50): 99 crashes from 2016-09-20.
 - release (version 49): 463 crashes from 2016-09-05.
 - esr     (version 45): 157 crashes from 2016-06-01.

Crash volume on the last weeks (Week N is from 10-03 to 10-09):
            W. N-1  W. N-2
 - nightly      10       2
 - aurora       20       2
 - beta         89      10
 - release     360     103
 - esr          39      29

Affected platforms: Windows, Mac OS X, Linux

Crash rank on the last 7 days:
           Browser     Content   Plugin
 - nightly #88
 - aurora  #49
 - beta    #162
 - release #137
 - esr     #279

Comment 8

[BugBot [:suhaib / :marco/ :calixte]]

Crash volume for signature 'AsyncShutdownTimeout | profile-before-change | OS.File: flush I/O queued before profileBeforeChange':
 - nightly (version 53): 50 crashes from 2016-11-14.
 - aurora  (version 52): 142 crashes from 2016-11-14.
 - beta    (version 51): 706 crashes from 2016-11-14.
 - release (version 50): 2426 crashes from 2016-11-01.
 - esr     (version 45): 768 crashes from 2016-07-13.

Crash volume on the last weeks (Week N is from 01-09 to 01-15):
            W. N-1  W. N-2  W. N-3  W. N-4  W. N-5  W. N-6  W. N-7
 - nightly      10       4       4       7       6       4       7
 - aurora       21      12      17      24      20      19      23
 - beta         62      74      91     101     104     104     100
 - release     323     300     307     383     329     289     296
 - esr          37      25      42      54      33      47      49

Affected platforms: Windows, Mac OS X, Linux

Crash rank on the last 7 days:
           Browser   Content   Plugin
 - nightly #102
 - aurora  #47
 - beta    #122
 - release #170
 - esr     #341

Comment 9

[BugBot [:suhaib / :marco/ :calixte]]

Crash volume for signature 'AsyncShutdownTimeout | profile-before-change | OS.File: flush I/O queued before profileBeforeChange':
 - nightly (version 54): 15 crashes from 2017-01-23.
 - aurora  (version 53): 10 crashes from 2017-01-23.
 - beta    (version 52): 97 crashes from 2017-01-23.
 - release (version 51): 268 crashes from 2017-01-16.
 - esr     (version 45): 930 crashes from 2016-08-03.

Crash volume on the last weeks (Week N is from 01-30 to 02-05):
            W. N-1  W. N-2  W. N-3  W. N-4  W. N-5  W. N-6  W. N-7
 - nightly       8
 - aurora        3
 - beta         52
 - release     133       0
 - esr          65      54      45      37      25      42      54

Affected platforms: Windows, Mac OS X, Linux

Crash rank on the last 7 days:
           Browser   Content   Plugin
 - nightly #42
 - aurora  #29
 - beta    #78
 - release #85
 - esr     #282

Comment 10

[Virtual_ManPL [:Virtual] 🇵🇱 - (please needinfo? me - so I will see your comment/reply/question/etc.)]

It crashed for me on shutdown - https://crash-stats.mozilla.com/report/index/b532ba0d-9f15-48af-9836-2b43c2170303

Comment 11

[Julien Cristau [:jcristau]]

Too late for firefox 52, mass-wontfix.

Comment 12

[Liz Henry (:lizzard) (relman/hg->git project)]

A lot of wontfixes in a row, no clear owner, no regression range, not very high crash rate. 
I think we should not keep triaging this.

Comment 13

[Trevor Skywalker]

Top Crashers for Firefox 67.0.1 (Release) - 7 days ago
#36 0.25% -0.07% AsyncShutdownTimeout | profile-before-change | OS.File: flush I/O queued before profileBeforeChange 207 206 0 0 193 0 2016-08-31

Signature report for AsyncShutdownTimeout | profile-before-change | OS.File: flush I/O queued before profileBeforeChange
Showing results from 7 days ago
786 Results

Windows 10 430 54.7%
Windows 7 250 31.8%
Windows 8.1 51 6.5%
Windows Vista 36 4.6%
Windows 8 11 1.4%
OS X 10.14 3 0.4%
OS X 10.13 2 0.3%
Linux 1 0.1%
OS X 10.10 1 0.1%

Firefox 69.0a1 35 4.5% 32
Firefox 68.0b8 17 2.2% 21
Firefox 68.0b7 9 1.1% 9
Firefox 68.0b9 7 0.9% 8
Firefox 68.0b4 2 0.3% 2
Firefox 67.0.1 207 26.3% 259
Firefox 67.0 66 8.4% 66
Firefox 67.0.2 36 4.6% 25
Firefox 67.0rc2 3 0.4% 1
Firefox 66.0.5 12 1.5% 12
Firefox 66.0.4 3 0.4% 3

Firefox 60.7.0esr 51 6.5% 54
Thunderbird 60.7.0 152 19.3% 152
Thunderbird 60.7.1 1 0.1% 1
Firefox 60.6.3esr 8 1.0% 9
Firefox 60.6.1esr 3 0.4% 3
Firefox 60.6.2esr 3 0.4% 3
Thunderbird 60.6.1 11 1.4% 13

Firefox 52.9.0esr 47 6.0% 35
Thunderbird 52.9.1 3 0.4% 3
Firefox 52.8.0esr 20 2.5% 22
Firefox 52.8.1esr 1 0.1% 1
Firefox 52.7.1esr 2 0.3% 2
Firefox 52.7.0esr 1 0.1% 1
Firefox 52.7.3esr 1 0.1% 1
Firefox 52.6.0esr 1 0.1% 1
Firefox 52.5.0esr 1 0.1% 1

Firefox 45.5.1esr 1 0.1% 1
Thunderbird 45.5.1 1 0.1% 1
Thunderbird 45.2.0 1 0.1% 1
Firefox 45.1.1esr 1 0.1% 1
Thunderbird 45.1.0 3 0.4% 1

x86 415 52.8%
amd64 370 47.1%

Comment 14

[alex_mayorga]

¡Hola!

Just crashed like bp-4c2cd603-b470-4a60-a3e2-854010200528 on Firefox Nighlty built from https://hg.mozilla.org/mozilla-central/rev/50d550c87f07e3fec8e2128f69b749a07b2cf18f

Setting flags per crash data FWIW.

¡Gracias!
Alex

Comment 15

[Beth Rennie [:beth] (she/her)]

Mass closure: OSFIle is being replaced with IOUtils and PathUtils. If you think this bug was closed in error, please re-open.