1057190 - Firefox consistently hangs on exit

Status: RESOLVED WORKSFORME

(Firefox :: General, defect)

Description

[Katelyn Gadd (:kael)]

For a couple weeks now, Aurora channel has been hanging on exit. I'm using the Exit menu item on the Firefox menu to quit.

In each case all the windows go away and the process seemingly just idles in the background with lots of threads active. I have to terminate it with a task manager. I seem to have a bunch of crash reports from this, but I'm not sure if they're from terminating it or if the hang is a result of a crash on shutdown. I didn't get a dialog box or prompt to submit these reports.

https://crash-stats.mozilla.com/report/index/9a8245a0-3630-4e04-8da1-5642d2140822
https://crash-stats.mozilla.com/report/index/6cd9b71c-bfc2-4c2c-a3d0-0199f2140822
https://crash-stats.mozilla.com/report/index/9fe12359-c8f4-49d8-80df-e9fef2140822

When I tried to attach a debugger to a firefox.exe in this hung state it was somehow in an undebuggable state I've never seen before. Screenshot:
https://www.dropbox.com/s/rw8k60sf8260yr5/Screenshot%202014-08-21%2019.57.50.png

Comment 1

[Katelyn Gadd (:kael)]

about:support
-------
Application Basics
------------------

Name: Firefox
Version: 33.0a2
User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:33.0) Gecko/20100101 Firefox/33.0
Multiprocess Windows: 0/2

Crash Reports for the Last 3 Days
---------------------------------

Report ID: bp-9fe12359-c8f4-49d8-80df-e9fef2140822
Submitted: 2 minutes ago

Report ID: bp-6cd9b71c-bfc2-4c2c-a3d0-0199f2140822
Submitted: 6 minutes ago

Report ID: bp-9a8245a0-3630-4e04-8da1-5642d2140822
Submitted: 7 minutes ago

All Crash Reports (including 34 pending crashes in the given time range)

Extensions
----------

Name: Download Manager (S3)
Version: 3.02
Enabled: true
ID: s3download@statusbar

Name: Lazarus: Form Recovery
Version: 2.3
Enabled: true
ID: [e-mail]

Name: NoScript
Version: 2.6.8.36
Enabled: true
ID: {73a6fe31-595d-460b-a920-fcc0f8843232}

Name: Rikaichan
Version: 2.07
Enabled: true
ID: {0AA9101C-D3C1-4129-A9B7-D778C6A17F82}

Name: Rikaichan Japanese-English Dictionary File
Version: 2.01.130701
Enabled: true
ID: [e-mail]

Name: about:telemetry
Version: 0.14
Enabled: false
ID: [e-mail]

Name: Cookies Manager+
Version: 1.5.2
Enabled: false
ID: {bb6bc1bb-f824-4702-90cd-35e2fb24f25d}

Name: Default Full Zoom Level
Version: 7.0.5
Enabled: false
ID: {D9A7CBEC-DE1A-444f-A092-844461596C4D}

Name: Firebug
Version: 2.0.3
Enabled: false
ID: [e-mail]

Name: Force CORS
Version: 1.4
Enabled: false
ID: [e-mail]

Name: Gecko Profiler
Version: 1.13.5
Enabled: false
ID: jid0-edalmuivkozlouyij0lpdx548bc@jetpack

Name: JIT Inspector
Version: 0.5.0.0
Enabled: false
ID: [e-mail]

Name: Logitech SetPoint
Version: 6.5
Enabled: false
ID: {F003DA68-8256-4b37-A6C4-350FA04494DF}

Name: Suspend background tabs
Version: 1.0.1
Enabled: false
ID: [e-mail]

Name: Theme Font & Size Changer
Version: 31.2
Enabled: false
ID: {f69e22c7-bc50-414a-9269-0f5c344cd94c}

Name: WebPG for Mozilla
Version: 0.9.4
Enabled: false
ID: [e-mail]

Graphics
--------

Adapter Description: NVIDIA GeForce GTX 670
Adapter Drivers: nvd3dumx,nvwgf2umx,nvwgf2umx nvd3dum,nvwgf2um,nvwgf2um
Adapter RAM: 2048
Device ID: 0x1189
Direct2D Enabled: true
DirectWrite Enabled: true (6.2.9200.16571)
Driver Date: 7-2-2014
Driver Version: 9.18.13.4052
GPU #2 Active: false
GPU Accelerated Windows: 2/2 Direct3D 11 (OMTC)
Vendor ID: 0x10de
WebGL Renderer: Google Inc. -- ANGLE (NVIDIA GeForce GTX 670 Direct3D9Ex vs_3_0 ps_3_0)
windowLayerManagerRemote: true
AzureCanvasBackend: direct2d
AzureContentBackend: direct2d
AzureFallbackCanvasBackend: cairo
AzureSkiaAccelerated: 0

Important Modified Preferences
------------------------------

accessibility.typeaheadfind.flashBar: 0
browser.cache.disk.capacity: 358400
browser.cache.disk.smart_size_cached_value: 358400
browser.cache.disk.smart_size.first_run: false
browser.cache.disk.smart_size.use_old_max: false
browser.cache.frecency_experiment: 1
browser.places.smartBookmarksVersion: 7
browser.sessionstore.upgradeBackup.latestBuildID: 20140821004002
browser.startup.homepage_override.buildID: 20140821004002
browser.startup.homepage_override.mstone: 33.0a2
browser.tabs.onTop: true
browser.tabs.warnOnClose: false
dom.max_chrome_script_run_time: 40
dom.max_script_run_time: 0
dom.mozApps.maxLocalId: 1001
dom.mozApps.used: true
dom.w3c_touch_events.expose: false
extensions.checkCompatibility.29.0a: true
extensions.checkCompatibility.nightly: true
extensions.lastAppVersion: 33.0a2
font.internaluseonly.changed: true
gfx.direct3d.last_used_feature_level_idx: 0
media.gmp-gmpopenh264.lastUpdate: 1406398731
media.gmp-gmpopenh264.path: C:\Users\Kate\AppData\Roaming\Mozilla\Firefox\Profiles\u7i5phcn.default\gmp-gmpopenh264
media.gmp-gmpopenh264.version: 1.0
media.gmp-manager.lastCheck: 1408657473
network.cookie.prefsMigrated: true
places.database.lastMaintenance: 1408660666
places.history.expiration.transient_current_max_pages: 104858
plugin.disable_full_page_plugin_for_types: application/pdf
plugin.importedState: true
plugin.state.java: 0
plugin.state.npauthz: 0
plugin.state.npdeployjava: 0
plugin.state.npgoogletalk: 0
plugin.state.npgtpo3dautoplugin: 0
plugin.state.npitunes: 0
plugin.state.npnv3dv: 0
plugin.state.npnv3dvstreaming: 0
plugin.state.npo1d: 0
plugin.state.nppdf: 0
plugin.state.nprcplugin: 0
plugin.state.nprun3d: 0
plugin.state.npspwrap: 0
plugin.state.npwebpg-ext-v0.6.5-winnt_x86-msvc: 2
plugin.state.npwpidetector: 0
print.printer_Canon_MP495_series_Printer.print_bgcolor: false
print.printer_Canon_MP495_series_Printer.print_bgimages: false
print.printer_Canon_MP495_series_Printer.print_colorspace:
print.printer_Canon_MP495_series_Printer.print_command:
print.printer_Canon_MP495_series_Printer.print_downloadfonts: false
print.printer_Canon_MP495_series_Printer.print_duplex: 7602290
print.printer_Canon_MP495_series_Printer.print_edge_bottom: 0
print.printer_Canon_MP495_series_Printer.print_edge_left: 0
print.printer_Canon_MP495_series_Printer.print_edge_right: 0
print.printer_Canon_MP495_series_Printer.print_edge_top: 0
print.printer_Canon_MP495_series_Printer.print_evenpages: true
print.printer_Canon_MP495_series_Printer.print_footercenter:
print.printer_Canon_MP495_series_Printer.print_footerleft: &PT
print.printer_Canon_MP495_series_Printer.print_footerright: &D
print.printer_Canon_MP495_series_Printer.print_headercenter:
print.printer_Canon_MP495_series_Printer.print_headerleft: &T
print.printer_Canon_MP495_series_Printer.print_headerright: &U
print.printer_Canon_MP495_series_Printer.print_in_color: true
print.printer_Canon_MP495_series_Printer.print_margin_bottom: 0.5
print.printer_Canon_MP495_series_Printer.print_margin_left: 0.5
print.printer_Canon_MP495_series_Printer.print_margin_right: 0.5
print.printer_Canon_MP495_series_Printer.print_margin_top: 0.5
print.printer_Canon_MP495_series_Printer.print_oddpages: true
print.printer_Canon_MP495_series_Printer.print_orientation: 0
print.printer_Canon_MP495_series_Printer.print_page_delay: 50
print.printer_Canon_MP495_series_Printer.print_paper_data: 1
print.printer_Canon_MP495_series_Printer.print_paper_height: 11.00
print.printer_Canon_MP495_series_Printer.print_paper_name:
print.printer_Canon_MP495_series_Printer.print_paper_size_type: 0
print.printer_Canon_MP495_series_Printer.print_paper_size_unit: 0
print.printer_Canon_MP495_series_Printer.print_paper_width: 8.50
print.printer_Canon_MP495_series_Printer.print_plex_name:
print.printer_Canon_MP495_series_Printer.print_resolution: 6619254
print.printer_Canon_MP495_series_Printer.print_resolution_name:
print.printer_Canon_MP495_series_Printer.print_reversed: false
print.printer_Canon_MP495_series_Printer.print_scaling: 1.00
print.printer_Canon_MP495_series_Printer.print_shrink_to_fit: true
print.printer_Canon_MP495_series_Printer.print_to_file: false
print.printer_Canon_MP495_series_Printer.print_unwriteable_margin_bottom: 0
print.printer_Canon_MP495_series_Printer.print_unwriteable_margin_left: 0
print.printer_Canon_MP495_series_Printer.print_unwriteable_margin_right: 0
print.printer_Canon_MP495_series_Printer.print_unwriteable_margin_top: 0
privacy.cpd.cookies: false
privacy.cpd.downloads: false
privacy.cpd.formdata: false
privacy.cpd.history: false
privacy.cpd.sessions: false
privacy.sanitize.migrateFx3Prefs: true
privacy.sanitize.timeSpan: 0
security.tls.version.max: 1
security.warn_viewing_mixed: false
storage.vacuum.last.index: 1
storage.vacuum.last.places.sqlite: 1407366114
webgl.verbose: true

Important Locked Preferences
----------------------------

JavaScript
----------

Incremental GC: true

Accessibility
-------------

Activated: false
Prevent Accessibility: 0

Library Versions
----------------

NSPR
Expected minimum version: 4.10.7
Version in use: 4.10.7

NSS
Expected minimum version: 3.17 Basic ECC
Version in use: 3.17 Basic ECC

NSSSMIME
Expected minimum version: 3.17 Basic ECC
Version in use: 3.17 Basic ECC

NSSSSL
Expected minimum version: 3.17 Basic ECC
Version in use: 3.17 Basic ECC

NSSUTIL
Expected minimum version: 3.17
Version in use: 3.17

Experimental Features
---------------------

Comment 2

[:Gijs (he/him)]

That's a lot of add-ons. Can you reproduce this without them? I'm wondering if it's correlated with any of them...

The stacks from the crashes all look like the browser is just starting, which makes me wonder if the add-ons or something else are reinitializing stuff when we're in the middle of shutting down (which, um, is a bad thing...).

Comment 3

[Katelyn Gadd (:kael)]

Most of those addons in the list are disabled. Or can disabled addons cause problems? That would be confusing...

It doesn't reproduce in my clean profile, but it's possible it has something to do with open tabs. It's not 100% so I'm having a hard time narrowing it down.

Comment 4

[:Gijs (he/him)]

(In reply to K. Gadd (:kael) from comment #3)
> Most of those addons in the list are disabled. Or can disabled addons cause
> problems? That would be confusing...

They shouldn't; sorry for not noticing that earlier. But noscript, lazarus and download managers are all equally intricate add-ons that they could be responsible here. I'm not familiar enough with Rikaichan to know whether it would be likely to have a hand in this.

> It doesn't reproduce in my clean profile, but it's possible it has something
> to do with open tabs. It's not 100% so I'm having a hard time narrowing it
> down.

Makes sense. Please let us know if you get any further in figuring this out, and we can take it from there...

(In reply to K. Gadd (:kael) from comment #0)
> In each case all the windows go away and the process seemingly just idles in
> the background with lots of threads active. I have to terminate it with a
> task manager. I seem to have a bunch of crash reports from this, but I'm not
> sure if they're from terminating it or if the hang is a result of a crash on
> shutdown. I didn't get a dialog box or prompt to submit these reports.
> 
> https://crash-stats.mozilla.com/report/index/9a8245a0-3630-4e04-8da1-
> 5642d2140822
> https://crash-stats.mozilla.com/report/index/6cd9b71c-bfc2-4c2c-a3d0-
> 0199f2140822
> https://crash-stats.mozilla.com/report/index/9fe12359-c8f4-49d8-80df-
> e9fef2140822


Actually, all of these report an uptime of '0', so they look like startup crashes, which is odd - but perhaps the uptime data is similarly corrupted if we reinitialize things during shutdown? Not sure what's going on there. :-\

Comment 5

[Katelyn Gadd (:kael)]

Ah, you know what? Those are probably the new session from *after* the hang, showing the 'profile locked' message. Not sure why that session would crash, but it explains the short uptime. (Would the uptime stay at 0 while the profile chooser is open?)

Comment 6

[Katelyn Gadd (:kael)]

Attachment: Main thread hang stacktrace

I think I identified the key to reproducing this. After leaving Firefox (Aurora v33 specifically) idle on my home desktop for a while, the compositor gets into a (reproducible, but not easily) bad state where it's interacting poorly with windows' compositor. The UI chrome is busted - for example, the minimize/maximize/close buttons don't handle mouse input anymore and draw wrong, and the tab bar is way more transparent than it should be.

If I shut firefox down with the compositor in this state (with a debugger attached beforehand, so I can debug), I see some cryptic error messages and an unhandled exception in the debug output, and the main thread hangs during shutdown with the attached stacktrace.

Comment 7

[Katelyn Gadd (:kael)]

OK, I got an empty profile to get into the bad compositor state and it didn't hang on shutdown. The main session I had open *did* hang when closed. 

So, it might be the one extension I still have enabled (noscript; can't really browse without it) or the tabs I have open. Either way, the hang doesn't seem to reproduce on a fresh process. I've tried basic stuff (like loading each of my tabs once before exiting) to reproduce and it seems to be more complex than that.

The weird compositing glitch along with seeing a hang in the compositor does make me think composition is probably involved, though.

Comment 8

[Katelyn Gadd (:kael)]

Reproed again after a long session, with noscript as the only enabled addon.

Hang is in the same place, inside the compositor, blocked inside PR_MD_WAIT_CV.
>	xul.dll!mozilla::layers::CompositorParent::ShutDown()  Line 197 + 0x9 bytes	C++

Comment 9

[Katelyn Gadd (:kael)]

I tested disabling OMTC (via about:config) and still get the compositor hang at shutdown with it off. This still happens for me almost every time I exit FF (more often for long sessions; sometimes even when the browser has only been open for 30 seconds).

I've ruled out every addon other than noscript, and I've been unable to repro this by leaving a clean profile running forever. Does it make sense to try and load all the tabs from this profile inside a new clean profile and see if leaving them idle reproduces it? I'm not sure how much state I will have to copy over to actually be able to load all these tabs - cookies, passwords, sessions, etc (gmail, task-tracking software for work, and so on)

Comment 10

[Katelyn Gadd (:kael)]

Still have to task-kill firefox.exe any time I want to restart it or apply updates. The message from the latest Aurora builds asking me to close it has a button to close the old firefox process, but the button doesn't work.

I've tried everything I can think of here and I've provided the consistent stack trace I get every time. I've tried different driver versions, a different GPU (same mfr, though - don't have an AMD handy), and tried displaying OMTC and D2D 1.1.

What can I do to help narrow this down so it can be fixed? I'm sure it affects other people in the wild.

Comment 11

[Katelyn Gadd (:kael)]

Attachment: Screenshot 2014-11-23 12.16.14.png

Brand new windows 8.1 x64 install with no files copied over. Fresh Firefox Developer Edition profile (with sync enabled), noscript & rikaichan installed. Profile is about a day old.

GTX 970 with latest drivers.

Somewhat consistently hangs on shutdown, sometimes when the process is only a few minutes old.

Comment 12

[:Gijs (he/him)]

Yoric, can you help get the right people involved to debug/diagnose this issue further? I'm not sure who besides you knows about shutdown hangs and/or what can be done to diagnose them any more than they have been so far...

Comment 13

[David Teller [:Yoric] - still alive but not very active]

So, for what it's worth, we have activated a few days ago the Terminator, which will automatically crash Firefox if it hangs on exit.

However, it seems that this is a case of the dreaded compositor-hangs-on-exit bug. Nicolas, I believe that this is one of yours.

Comment 14

[Katelyn Gadd (:kael)]

Attachment: Compositor thread stack

Bas asked me to try and get the stack of the Compositor thread during this hang. Sadly, identifying the compositor thread means I have to run FF under the debugger from start, since thread names aren't recorded unless a debugger is attached.

This is the stack I got out of the debugger in a shutdown hang scenario. Sadly, I am not sure this is the compositor hang I've been seeing, since the symbols (somehow) seemed to be garbled when I looked at other stacks - the main thread was shown as hung inside line 3700 of some telemetry function and all the argument values were noise. :(

I will attempt to repro inside the debugger again and see if I get something better.

Comment 15

[Nicolas Silva [:nical]]

(In reply to David Rajchenbach-Teller [:Yoric] (away until November 17th - use "needinfo") from comment #13)
> However, it seems that this is a case of the dreaded
> compositor-hangs-on-exit bug. Nicolas, I believe that this is one of yours.

I don't have shutdown stuff on the work at the moment

Comment 16

[Wayne Mery (:wsmwk)]

kael, does this still reproduce for you?

Comment 17

[Katelyn Gadd (:kael)]

Nope

Comment 18

[Wayne Mery (:wsmwk)]

Thanks for the update