1230226 - JIT OOM when using Google Maps (due to hwaccel)

Status: NEW

(Core :: Graphics, defect, P3)

Description

[Aldo Abad]

Attachment: Example

User Agent: Mozilla/5.0 (Windows NT 6.1; rv:42.0) Gecko/20100101 Firefox/42.0
Build ID: 20151029151421

Steps to reproduce:

I am use a PC with AMD Radeon X1300 graphic card. Use firefox with two tabs open, for example, use Youtube in one tab and Google Maps in the other tab. Next, use Google Maps using street view. 


Actual results:

After some minutes using Google Maps you can see that Firefox crashes.


Expected results:

Firefox should display Google Mapa normally

Comment 1

[Michelle Funches - QA]

@Reporter: I am unable to reproduce in Windows 7 Firefox 42. (YouTube playing and Google Maps street view throughout my neighborhood.
Can you provide more detailed steps if this is still occurring? Thanks.

Comment 2

[Aldo Abad]

Attachment: This is other screenshot. You can see the problem

Comment 3

[Aldo Abad]

I can give you some extra steps to reproduce: 

- Use Google Maps street view and go back to map view, move the map to another location and use street view again.
- On street view move throughout that neighborhood, move the camera and advance over the streets, try to make more clics to advance.
- Repeat those steps for about 5 minutes. After that, Firefox seems slow.
- Minimize the application, again maximize the window, Firefox crashes, it display a black square.

I forgot to comment that it happen in my other PC too, it has an Intel graphic card. This bug happen again on a PC with AMD X1300, which is that I am using.

Comment 4

[Anika Nawar]

I couldn't reproduce the bug in my PC which has Intel Graphics. can u plz tell which verson of windows u r using n which bit it is??
I used the following platform

Firefox version: 42.0a1
Build ID: 20150801030211
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:42.0) Gecko/20100101 Firefox/42.0


Firefox version 45.0a1
Build ID :  20151206030448
User Agent : Mozilla/5.0 (Windows NT 10.0;Win64; x64; rv:45.0


Please try to reproduce the bug with a fresh profile[1]

[1] http://mzl.la/1BAQGnB

Comment 5

[Aldo Abad]

I am using Windows 7 x86 in both PC's. One has an Intel G41 graphic card and the other one has an AMD Radeon X1300. I can reproduce this bug on a PC with an Intel Q35 Express Chipset video card, Windows 7 x86.  As I mentioned before, try to make more clics to advance throughout that neighborhood. Some minutes later Firefox seems slow. Then minimize the window and maximize the window again. The problem appears when you maximize the window. 

Maybe if you install uBlock origin? That is the only addon I am using. I send a screenshot of the PC with an Intel Q35.

Comment 6

[Aldo Abad]

Attachment: Intel Q35 problem.jpg

Comment 7

[Milan Sreckovic [:milan] (needinfo for best results)]

Can you send us a pointer to these crashes (about:crashes should list them)?  That should give us the exact driver versions you're using, etc.  Graphics section of about:support would also be useful.  Thanks!

Comment 8

[Aldo Abad]

Configuración básica de la aplicación
-------------------------------------

Nombre: Firefox
Versión: 43.0.3
Id. de compilación: 20151223140742
Canal de actualizaciones: release
Agente de usuario: Mozilla/5.0 (Windows NT 6.1; rv:43.0) Gecko/20100101 Firefox/43.0
Ventanas multiproceso: 0/1 (por defecto: false)
Modo Seguro: false

Informe de fallos de los últimos 3 días
---------------------------------------

ID del reporte: bp-67e217a1-97d8-45f6-9fba-6b7162151231
Enviado: hace 5 minutos

ID del reporte: bp-8200c0f3-f42d-463c-928e-2337d2151230
Enviado: hace 8 horas

Todos los informes de fallos

Extensiones
-----------

Nombre: uBlock Origin
Versión: 1.5.1
Activada: true
ID: [e-mail]

Gráficas
--------

Adaptador de la RAM: Unknown
Controladores del Adaptador: atiumdag atiumdva atitmmxx
Descripción de Adaptador: Radeon X1300/X1550 Series
Direct2D Habilitado: Bloqueado para la versión de tu controlador gráfico. Prueba actualizando tu controlador gráfico a la versión 9.6 o más moderna.
DirectWrite Habilitado: false (6.2.9200.16492)
Encuadro/zoom asíncrono: ninguno
Fecha del dispositivo: 2-10-2010
GPU #2 Activa: false
GPU acelerada de Windows: 0/1 Basic (OMTC) Bloqueado para la versión de tu controlador gráfico. Prueba actualizando tu controlador gráfico a la versión 9.6 o más moderna.
ID de Vendor: 0x1002
ID del dispositivo: 0x7183
ID del subsys: 0d021028
Procesador WebGL: Bloqueado para la versión de tu controlador gráfico. Prueba actualizando tu controlador gráfico a la versión 9.6 o más moderna.
Soporta Decodificación de Hardware H264: No; Hardware video decoding disabled or blacklisted
Versión del dispositivo: 8.593.100.0
windowLayerManagerRemote: true
AzureCanvasBackend: skia
AzureContentBackend: cairo
AzureFallbackCanvasBackend: cairo
AzureSkiaAccelerated: 0

Preferencias importantes modificadas
------------------------------------

browser.cache.disk.capacity: 358400
browser.cache.disk.filesystem_reported: 1
browser.cache.disk.smart_size.first_run: false
browser.cache.frecency_experiment: 1
browser.download.importedFromSqlite: true
browser.download.useDownloadDir: false
browser.places.smartBookmarksVersion: 7
browser.sessionstore.upgradeBackup.latestBuildID: 20151223140742
browser.startup.homepage: www.google.com.mx
browser.startup.homepage_override.buildID: 20151223140742
browser.startup.homepage_override.mstone: 43.0.3
browser.urlbar.suggest.bookmark: false
browser.urlbar.suggest.openpage: false
browser.urlbar.suggest.searches: true
browser.urlbar.userMadeSearchSuggestionsChoice: true
dom.apps.reset-permissions: true
dom.disable_open_during_load: false
dom.mozApps.used: true
extensions.lastAppVersion: 43.0.3
font.internaluseonly.changed: true
gfx.crash-guard.d3d11layers.appVersion: 43.0.3
gfx.crash-guard.d3d11layers.deviceID: 0x7183
gfx.crash-guard.d3d11layers.driverVersion: 8.593.100.0
gfx.crash-guard.d3d11layers.feature-d2d: false
gfx.crash-guard.d3d11layers.feature-d3d11: false
gfx.crash-guard.status.d3d11layers: 2
gfx.driver-init.appVersion: 42.0
gfx.driver-init.deviceID: 0x7183
gfx.driver-init.driverVersion: 8.593.100.0
gfx.driver-init.feature-d2d: false
gfx.driver-init.feature-d3d11: false
gfx.driver-init.status: 2
media.gmp-eme-adobe.abi: x86-msvc
media.gmp-eme-adobe.lastUpdate: 1446574983
media.gmp-eme-adobe.version: 15
media.gmp-gmpopenh264.abi: x86-msvc
media.gmp-gmpopenh264.lastUpdate: 1451505064
media.gmp-gmpopenh264.version: 1.5.3
media.gmp-manager.buildID: 20151223140742
media.gmp-manager.lastCheck: 1451505063
media.hardware-video-decoding.failed: false
network.cookie.lifetimePolicy: 2
network.cookie.prefsMigrated: true
network.http.speculative-parallel-limit: 0
network.predictor.cleaned-up: true
network.prefetch-next: false
places.database.lastMaintenance: 1451246926
places.history.enabled: false
places.history.expiration.transient_current_max_pages: 87201
plugin.disable_full_page_plugin_for_types: application/pdf
plugin.importedState: true
plugin.state.npauthz: 0
plugin.state.npctrl: 2
plugin.state.npgoogleupdate: 0
plugin.state.npitunes: 0
plugin.state.npspwrap: 0
plugin.state.npwlpg: 0
print.printer_Brother_MFC-J6720DW.print_bgcolor: false
print.printer_Brother_MFC-J6720DW.print_bgimages: false
print.printer_Brother_MFC-J6720DW.print_duplex: -437918235
print.printer_Brother_MFC-J6720DW.print_edge_bottom: 0
print.printer_Brother_MFC-J6720DW.print_edge_left: 0
print.printer_Brother_MFC-J6720DW.print_edge_right: 0
print.printer_Brother_MFC-J6720DW.print_edge_top: 0
print.printer_Brother_MFC-J6720DW.print_evenpages: true
print.printer_Brother_MFC-J6720DW.print_footercenter:
print.printer_Brother_MFC-J6720DW.print_footerleft: &PT
print.printer_Brother_MFC-J6720DW.print_footerright: &D
print.printer_Brother_MFC-J6720DW.print_headercenter:
print.printer_Brother_MFC-J6720DW.print_headerleft: &T
print.printer_Brother_MFC-J6720DW.print_headerright: &U
print.printer_Brother_MFC-J6720DW.print_in_color: true
print.printer_Brother_MFC-J6720DW.print_margin_bottom: 0.5
print.printer_Brother_MFC-J6720DW.print_margin_left: 0.5
print.printer_Brother_MFC-J6720DW.print_margin_right: 0.5
print.printer_Brother_MFC-J6720DW.print_margin_top: 0.5
print.printer_Brother_MFC-J6720DW.print_oddpages: true
print.printer_Brother_MFC-J6720DW.print_orientation: 0
print.printer_Brother_MFC-J6720DW.print_page_delay: 50
print.printer_Brother_MFC-J6720DW.print_paper_data: 1
print.printer_Brother_MFC-J6720DW.print_paper_height: 11.00
print.printer_Brother_MFC-J6720DW.print_paper_name:
print.printer_Brother_MFC-J6720DW.print_paper_size_type: 0
print.printer_Brother_MFC-J6720DW.print_paper_size_unit: 0
print.printer_Brother_MFC-J6720DW.print_paper_width: 8.50
print.printer_Brother_MFC-J6720DW.print_resolution: -437918235
print.printer_Brother_MFC-J6720DW.print_reversed: false
print.printer_Brother_MFC-J6720DW.print_scaling: 0.80
print.printer_Brother_MFC-J6720DW.print_shrink_to_fit: true
print.printer_Brother_MFC-J6720DW.print_to_file: false
print.printer_Brother_MFC-J6720DW.print_unwriteable_margin_bottom: 0
print.printer_Brother_MFC-J6720DW.print_unwriteable_margin_left: 0
print.printer_Brother_MFC-J6720DW.print_unwriteable_margin_right: 0
print.printer_Brother_MFC-J6720DW.print_unwriteable_margin_top: 0
privacy.cpd.cookies: false
privacy.cpd.offlineApps: true
privacy.cpd.siteSettings: true
privacy.donottrackheader.enabled: true
privacy.sanitize.migrateClearSavedPwdsOnExit: true
privacy.sanitize.migrateFx3Prefs: true
privacy.sanitize.sanitizeOnShutdown: true
privacy.sanitize.timeSpan: 0
storage.vacuum.last.index: 1
storage.vacuum.last.places.sqlite: 1450565432

Preferencias importantes bloqueadas
-----------------------------------

JavaScript
----------

Recogida de basura incremental: true

Accesibilidad
-------------

Activado: false
Prevenir accesibilidad: 0

Versiones de bibliotecas
------------------------

NSPR
Versión mínima esperada: 4.10.10
Versión en uso: 4.10.10

NSS
Versión mínima esperada: 3.20.2 Basic ECC
Versión en uso: 3.20.2 Basic ECC

NSSSMIME
Versión mínima esperada: 3.20.2 Basic ECC
Versión en uso: 3.20.2 Basic ECC

NSSSSL
Versión mínima esperada: 3.20.2 Basic ECC
Versión en uso: 3.20.2 Basic ECC

NSSUTIL
Versión mínima esperada: 3.20.2
Versión en uso: 3.20.2

Características experimentales
------------------------------





https://crash-stats.mozilla.com/report/index/67e217a1-97d8-45f6-9fba-6b7162151231

Comment 9

[Milan Sreckovic [:milan] (needinfo for best results)]

Moving to JS based on crash signatures.  These are tagged as out of memory, but the crash reports show 66% usage.

Aldo, have you tried using the 64-bit version of Firefox?

Comment 10

[Milan Sreckovic [:milan] (needinfo for best results)]

Not a lot of these crashes, but all Windows, spread out over all versions (XP, Vista, 7, 8, 8.1, 10.)

Comment 11

[Aldo Abad]

(In reply to Milan Sreckovic [:milan] (PTO 1/29) from comment #9)
> Moving to JS based on crash signatures.  These are tagged as out of memory,
> but the crash reports show 66% usage.
> 
> Aldo, have you tried using the 64-bit version of Firefox?

I am not able to reproduce this issue on 64-bit Firefox. All of my computers have 32 bit version

Comment 12

[Benjamin Bouvier [:bbouvier] (inactive)]

cc'ing other JIT folks. If we create too many MBox here, it might be a missed perf opportunity as well.

Comment 13

[Nicolas B. Pierron [:nbp]]

> https://crash-stats.mozilla.com/report/index/67e217a1-97d8-45f6-9fba-
> 6b7162151231

This stack trace is reported by our fuzzers in Bug 1244824 comment 0, I will fix it as soon as possible.

Still if you are seeing a lot of these, this implies that you are running out of memory and the next unhandled OOM (out-of-memory) will trigger a crash.  I guess this also explains the black tiles that you are seeing.

So I think the problem might be elsewhere.  Can you attach the result of an anonymized "about:memory" report?

In the mean time, you could enable electrolysis [1], such that firefox uses multiple processes and only the process which handles the tabs will crash after running out of memory.

[1] https://wiki.mozilla.org/Electrolysis#Enabling_and_Disabling_Electrolysis

Comment 14

[Aldo Abad]

Attachment: memory-report.json.gz

Comment 15

[Nicolas B. Pierron [:nbp]]

I do not see any issue in the memory report. Were you having black tiles when you requested the memory report?

Comment 16

[Aldo Abad]

Attachment: memory-report.json.gz

Comment 17

[Nicolas B. Pierron [:nbp]]

Comment on attachment 8714992 [details]
memory-report.json.gz

Nicholas, do you have any idea how we could investigate this bug, and potentially where it might belong?

I do not know what might cause us to have 60% of the memory in heap-unclassified.

The only thing that I can note is that the  "canvas-2d-pixels" at the end of the report is roughly about the same size.

Milan, would it make sense to have a half-stored (degraded) canvas-2d at the cost of not respecting the spec, instead of having black tiles?

Comment 18

[Nicolas B. Pierron [:nbp]]

I discussed with Nicolas Silva (nical) about this issue.  Apparently website such as street view are making paths which are out-side the canvas, which cause our backend (cairo) to allocate a lot of memory, even if the canvas are small.

He suggested that this might be the reason why you have black tiles, and suggested that you could try to switch the backend used by canvas from "cairo" to "skia".

Can you try the following, and see if you can still reproduce this issue:
 - Visit the page "about:config".
 - Promess that you would be careful.
 - Search for "gfx.canvas.azure.backends".
 - Double click on the value "cairo".
 - Replace it by "skia" and validate.

Comment 19

[Nicolas Silva [:nical]]

To be more precise: Black tiles happen when we run out of memory and fail to allocate a tile to draw web content into (at this point we can either crash or try to make do with with some glitches, so we do the latter).

I saw in the past that streetview had a tendency to create enormous clip paths which cairo (the library we used to draw on Linux) doesn't deal with very well (it will allocate huge mask surfaces for the clip which caused us to run out of memory at some point). I landed a workaround for the streetview issue a while ago, but it's possible that we are still having badness related to clips in cairo (as another bug with MapQuest suggests). But I haven't checked that it's the cause of this bug.
That said, if the bug doesn't reproduce with skia, it's good to know (and it'd be encouraging because we'll enable skia for canvases on linux soon).

Comment 20

[Nicholas Nethercote [inactive]]

DMD is the tool for investigating high "heap-unclassified" values. Instructions on using it are here: https://developer.mozilla.org/en-US/docs/Mozilla/Performance/DMD

Cairo memory usage falls into "heap-unclassified", so nical's theory is plausible.

The memory reports have this:

>   902.61 MB ── private
>   931.47 MB ── resident
>   914.61 MB ── resident-unique
>     1.81 MB ── system-heap-allocated
> 1,765.05 MB ── vsize
>     3.10 MB ── vsize-max-contiguous

The "vsize-max-contiguous" number is very low, which shows that virtual memory is either close to full or badly fragmented, which confirms it's a (virtual) OOM problem. Switching to a 64-bit Firefox (see https://www.mozilla.org/en-US/firefox/all/) should fix that.

There was also this, though:

> 52.48 MB (06.27%) -- top(none)/detached

Detached windows often indicate a memory leak. Usually the cause of this is an extension, and we often see it with users that have lots of extensions installed. But Aldo only has uBlock Origin. I'd be surprised if that was the cause, but it might be worth disabling it temporarily just to check.

Comment 21

[Michelle Funches - QA]

Due to the amount of comments, we will be changing the bug from Unconfirmed to New so that items with a component no longer display unconfirmed

Comment 22

[Derek]

Also seeing this bug, using Google Maps - particularly street view - causes rapid growth in the memory consumption of firefox. quickly growing from ~500megs (a dozen tabs open) to over 2000mb within 5 minutes forcing termination of firefox.  

i7-6700HQ, Intel HD Graphics 530 switchable with GeForce GTX 960M, 16GB RAM.  Windows 10 Pro 64bit. Firefox 44.0.2

Comment 23

[Derek]

Note: turning off hardware acceleration appears to resolve it

Comment 24

[Derek]

Turning hardware acceleration back on, but taking Cairo out of the list in gfx.canvas.azure.backends also seems to significantly help with the memory issue.

On a side note: Overall google maps performance is dismal no matter what if you try to use full version.  It appears to just flat out be slow fetching the data from google - an issue that Microsoft Edge, Google Chrome, etc do not exhibit.