1333805 - Some content process crashes not generating a minidump

Status: NEW

(Toolkit :: Crash Reporting, defect)

Description

[Mike Conley (:mconley) (:⚙️)]

I saw this Reddit post recently in /r/firefox:

https://www.reddit.com/r/firefox/comments/5q18iu/help_firefox_keeps_doing_this_whenever_i_enter/

This user is experiencing a tab crash every time they open a new tab. They're also not getting the opportunity to submit a crash report. The user sent me their about:support, and it looks like there are no crash reports to submit.

Something is going really wrong here.

We collect Telemetry for the number of times that about:tabcrashed is presented to the user (FX_CONTENT_CRASH_PRESENTED), and also collect Telemetry for how many times the a crash dump is not available (FX_CONTENT_CRASH_DUMP_UNAVAILABLE). This should allow us to get a sense of how many crashes we're getting no crash data from.

We should really get a sense of how widespread this problem is. If we're basing our stability measurements off of how many crash reports are coming in, they might be off because we're missing all of this data.

Comment 1

[Mike Conley (:mconley) (:⚙️)]

Updated summary because I'm unclear on how widespread of an issue this is.

Also note that I recently filed a bug at bug 1333223 where we weren't collecting certain types of crashes.

Comment 2

[Chris H-C :chutten]

Current content process crash figures are obtained from SUBPROCESS_CRASHES_WITH_DUMP/content, so it's possible we're missing cases like this.

+gsvelto - do content crash pings get sent if no dump is generated?

Comment 3

[(not currently active) Ted Mielczarek]

Guessing from the screenshot--is that user on Windows?

Comment 4

[Mike Conley (:mconley) (:⚙️)]

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #3)
> Guessing from the screenshot--is that user on Windows?

Yes. I'm going to try to get their OK to post their about:support here publicly.

Comment 5

[(not currently active) Ted Mielczarek]

OK, so it's not likely to be the same as bug 1333223 or the known issues we have on OS X 10.12, at least.

Comment 6

[Gabriele Svelto [:gsvelto] (PTO)]

(In reply to Chris H-C :chutten from comment #2)
> Current content process crash figures are obtained from
> SUBPROCESS_CRASHES_WITH_DUMP/content, so it's possible we're missing cases
> like this.
> 
> +gsvelto - do content crash pings get sent if no dump is generated?

Nope, they're sent only if we've generated a minidump. If minidump generation is disabled or if something gets in the way and it fails no ping will be sent.

Comment 7

[Mike Conley (:mconley) (:⚙️)]

The user gave me their consent to post their about:support:


Application Basics

Name: Firefox Version: 51.0
Build ID: 20170118123726
Update Channel: release
User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0 OS: Windows_NT 6.1
Multiprocess Windows: 1/1 (Enabled by default)
Safe Mode: false


Crash Reports for the Last 3 Days

All Crash Reports


Extensions

Name: Application Update Service Helper Version: 1.0 Enabled: true ID: [e-mail]

Name: Multi-process staged rollout Version: 1.7 Enabled: true ID: [e-mail]

Name: Pocket Version: 1.0.5 Enabled: true ID: [e-mail]

Name: Web Compat Version: 1.0 Enabled: true ID: [e-mail]

Name: Babylon Translation Activation Version: 1.1 Enabled: false ID: [e-mail]

Name: McAfee ScriptScan for Firefox Version: 15.1.0 Enabled: false ID: {D19CA586-DD6C-4a0a-96F8-14644F340D60}

Name: McAfee WebAdvisor Version: 5.0.360.0 Enabled: false ID: {4ED1F68A-5463-4931-9384-8FFF5ED91D92}

Name: RealDownloader Version: 17.0.13 Enabled: false ID: {9D2AA73B-6049-4799-B8AC-925723370070}

Name: Video DownloadHelper Version: 6.2.0 Enabled: false ID: {b9db16a4-6edc-47ec-a1f4-b86292ed211d}

Name: YTD Toolbar Version: 24.3 Enabled: false ID: [e-mail]


Graphics

Features

Compositing: Direct3D 11
Asynchronous Pan/Zoom: wheel input enabled
WebGL Renderer: Google Inc. -- ANGLE (AMD RADEON HD 6350 Direct3D11 vs_5_0 ps_5_0)
WebGL2 Renderer: Google Inc. -- ANGLE (AMD RADEON HD 6350 Direct3D11 vs_5_0 ps_5_0)
Hardware H264 Decoding: No; Hardware video decoding disabled or blacklisted
Audio Backend: wasapi Direct2D: true DirectWrite: true (6.2.9200.17568)

GPU #1
Active: Yes
Description: AMD RADEON HD 6350
Vendor ID: 0x1002
Device ID: 0x68f9
Driver Version: 8.830.0.0
Driver Date: 2-18-2011
Drivers: aticfx64 aticfx64 aticfx64 aticfx32 aticfx32 aticfx32 atiumd64 atidxx64 atidxx64 atiumdag atidxx32 atidxx32 atiumdva atiumd6a atitmm64
Subsys ID: 21261028
RAM: 512

Diagnostics

AzureCanvasAccelerated: 0
AzureCanvasBackend: direct2d 1.1
AzureContentBackend: direct2d 1.1
AzureFallbackCanvasBackend: cairo
Decision Log D3D9_COMPOSITING: disabled by default: Disabled by default


Important Modified Preferences

browser.cache.disk.capacity: 358400
browser.cache.disk.filesystem_reported: 1
browser.cache.disk.smart_size_cached_value: 358400
browser.cache.disk.smart_size.first_run: false
browser.cache.disk.smart_size.use_old_max: false
browser.cache.frecency_experiment: 1
browser.download.importedFromSqlite: true
browser.download.manager.alertOnEXEOpen: true
browser.download.useDownloadDir: false
browser.places.smartBookmarksVersion: 8
browser.sessionstore.upgradeBackup.latestBuildID: 20170118123726
browser.startup.homepage: about:home
browser.startup.homepage_override.buildID: 20170118123726
browser.startup.homepage_override.mstone: 51.0
browser.tabs.remote.autostart.2: true
browser.urlbar.daysBeforeHidingSuggestionsPrompt: 3
browser.urlbar.lastSuggestionsPromptDate: 20170125
dom.apps.reset-permissions: true
dom.mozApps.used: true
extensions.lastAppVersion: 51.0
general.useragent.extra.btrs: BTRS99472
gfx.crash-guard.d3d11layers.appVersion: 51.0
gfx.crash-guard.d3d11layers.deviceID: 0x68f9
gfx.crash-guard.d3d11layers.driverVersion: 8.830.0.0
gfx.crash-guard.d3d11layers.feature-d2d: true
gfx.crash-guard.d3d11layers.feature-d3d11: true
gfx.crash-guard.status.d3d11layers: 2
gfx.crash-guard.status.d3d9video: 2
gfx.direct3d.last_used_feature_level_idx: 0
keyword.URL: https://search.yahoo.com/search?fr=greentree_ff1&ei=utf-8&ilc=12&type=937811&p=
media.gmp-eme-adobe.abi: x86-msvc-x64
media.gmp-eme-adobe.lastUpdate: 1485318294
media.gmp-eme-adobe.version: 17
media.gmp-gmpopenh264.abi: x86-msvc-x64
media.gmp-gmpopenh264.lastUpdate: 1485318296
media.gmp-gmpopenh264.version: 1.6
media.gmp-manager.buildID: 20170118123726
media.gmp-manager.lastCheck: 1485318289
media.gmp-widevinecdm.abi: x86-msvc-x64
media.gmp-widevinecdm.lastUpdate: 1485318300
media.gmp-widevinecdm.version: 1.4.8.903
media.gmp.storage.version.observed: 1
media.hardware-video-decoding.failed: true
network.cookie.prefsMigrated: true
network.predictor.cleaned-up: true
places.database.lastMaintenance: 1479702072
places.history.expiration.transient_current_max_pages: 122334
plugin.disable_full_page_plugin_for_types: application/pdf
plugin.importedState: true
print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_bgcolor: false print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_bgimages: false print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_colorspace: print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_command: print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_downloadfonts: false print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_duplex: 1515870810 print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_edge_bottom: 0 print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_edge_left: 0 print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_edge_right: 0 print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_edge_top: 0 print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_evenpages: true print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_in_color: true print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_margin_bottom: 0.5 print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_margin_left: 0.5 print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_margin_right: 0.5 print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_margin_top: 0.5 print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_oddpages: true print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_orientation: 0 print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_page_delay: 50 print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_paper_data: 1 print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_paper_height: 11.00 print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_paper_name: print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_paper_size_type: 0 print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_paper_size_unit: 0 print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_paper_width: 8.50 print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_plex_name: print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_resolution: 1515870810 print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_resolution_name: print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_reversed: false print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_scaling: 1.00 print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_shrink_to_fit: true print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_to_file: false print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_unwriteable_margin_bottom: 0 print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_unwriteable_margin_left: 0 print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_unwriteable_margin_right: 0 print.printer_HP_LaserJet_M1522_MFP_Series_PCL_6.print_unwriteable_margin_top: 0
privacy.sanitize.migrateFx3Prefs: true
security.sandbox.content.tempDirSuffix: {3a1c7ae7-b69a-4a77-b03d-1fafdedd67d3}
services.sync.declinedEngines:
storage.vacuum.last.index: 0
storage.vacuum.last.places.sqlite: 1479702072

Important Locked Preferences
Places Database
JavaScript

Incremental GC: true
Accessibility

Activated: false Prevent Accessibility: 0
Library Versions

NSPR Expected minimum version: 4.13.1 Version in use: 4.13.1

NSS Expected minimum version: 3.28.1 Version in use: 3.28.1

NSSSMIME Expected minimum version: 3.28.1 Version in use: 3.28.1

NSSSSL Expected minimum version: 3.28.1 Version in use: 3.28.1

NSSUTIL Expected minimum version: 3.28.1 Version in use: 3.28.1
Experimental Features
Sandbox

Content Process Sandbox Level: 1

Comment 8

[Mike Conley (:mconley) (:⚙️)]

I've found two probes that we probably want to use in an analysis here:

FX_CONTENT_CRASH_DUMP_UNAVAILABLE: "Counts the number of times that about:tabcrashed was unable to find a crash dump."

and

FX_CONTENT_CRASH_PRESENTED: "Counts the number of times that about:tabcrashed appeared and found a crash dump."

We should see if we can get a sense of proportion of how many tab crashes result in the first case or the second.

After that, we might want to see if the distribution is uniform across reporting clients.

Comment 9

[Mike Conley (:mconley) (:⚙️)]

chutten crunched some numbers, and the news is (thankfully) better than I feared.

Quoting chutten from #e10s:

"On 2017-01-10 (Tuesday), for users running any subversion of release 50, there were 596 accumulations into FX_CONTENT_CRASH_DUMP_UNAVAILABLE which accounts for less than 0.02% of content crashes reported.

Sources: https://mzl.la/2kcg7wu, https://sql.telemetry.mozilla.org/queries/2426/source"

Comment 10

[Brad Lassey [:blassey] (use needinfo?)]

Adding Mike Taylor, who saw a class of crashes come through webcompat.com about youtube crashing and not producing a crash report.

Comment 11

[(not currently active) Ted Mielczarek]

Resummarizing, since this doesn't seem to be widespread. This shouldn't be accountable to the usual factors we see in browser process crashes (OOM, etc) since we write the minidump from the browser process.