Closed Bug 682625 Opened 9 years ago Closed 8 years ago

Crash, null pointer deref [@ mozilla::plugins::PluginInstanceParent::BackgroundDescriptor ]


(Core :: Plug-ins, defect, critical)

Not set





(Reporter: bjacob, Assigned: karlt)



(Keywords: crash)

Crash Data


(5 files, 1 obsolete file)

This is one of the topcrashers on Firefox 7.0b1 / Linux, however all the reports last week seem to originate from a single user:

Here's one crash:

We're dereferencing null here:

We have:

NS_ABORT_IF_FALSE(mBackground, "Need a background here");
gfxXlibSurface* xsurf = static_cast<gfxXlibSurface*>(mBackground.get());
return SurfaceDescriptorX11(xsurf->XDrawable(), xsurf->XRenderFormat()->id, xsurf->GetSize()); // <-- null deref here

Looks like mBackground was null and this wasn't caught by the NS_ABORT_IF_FALSE because it's a release build.

Last touched by cjones in bug 626602,
Severity: normal → critical
Keywords: crash, topcrash
Could be the same as bug 646150, where xsurf->XRenderFormat() is null.

Build id 20110816154714 is consistent with
so it doesn't seem to be an Ubuntu build.
Depends on: 646150
The zero offset from the null pointer confirms that it is xsurf->XRenderFormat() that is null.
Found no crashes with that signature within a week, but looking beyond, there's a couple:

The ones on 7.0 smell very much like a single user, given the install times matches to the second, the one on 7.0a2 and the one on 8.0a1 seems to be two other users that have seen it.
20 on 8.0* over 4 weeks and few on other versions, one even on 11.0a1, so still valid, but not a topcrash.
Keywords: topcrash
I confirm that this bug is still there, as described, in ff901 and sm261.

It is triggered by plugins (I rarely use them, but do need to use a couple of sites which require flash); as soon as the plugin loads the browser invariably crashes.

I use sm and ff on amd64 Linux boxen.
I forgot to write:

Given when this started occurring, it seems likely to be related to the switch to sandbox plugins.

Is there and quick and easy way to back out the plugin sandboxing so as to confirm whether the bug still exists w/o that?
And from a core dump, showing that in this cases xsurf->msize has valid height and width values.  I tried this once w/ ff8 compiled -O0 and, IIRC, the hight and width were 0....

(gdb) up
#3  mozilla::plugins::PluginInstanceParent::BackgroundDescriptor (this=<optimized out>)
    at /usr/src/debug/www-client/seamonkey-2.6.1/comm-release/mozilla/dom/plugins/ipc/PluginInstanceParent.cpp:874
874	                                xsurf->GetSize());
(gdb) p xsurf
$1 = (gfxXlibSurface *) 0x7f9bdc391c40
(gdb) p *xsurf
$2 = {<gfxASurface> = {_vptr.gfxASurface = 0x7f9c0f2d0c50, mSurface = 0x7f9bdcba6cc0, 
    mOpaqueRect = {mRawPtr = 0x0}, mFloatingRefs = 0, mBytesRecorded = 304800, 
    mSurfaceValid = 1 '\001', mAllowUseAsSource = 1 '\001'}, mPixmapTaken = 1, 
  mDisplay = 0x7f9c01649000, mDrawable = 41954458, 
  mSize = {<mozilla::gfx::BaseSize<int, nsIntSize>> = {width = 400, 
      height = 254}, <No data fields>}, mGLXPixmap = 0}
Bug 714695 seems related, but the crash seems different.
cloos: would you be able to print "xsurf->XRenderFormat()" as well as 
and attach output from xdpyinfo -ext RENDER please?

Are you building with --enable-system-cairo or the default disable-system-cairo?
Duplicate of this bug: 714695
I re-compiled ff with -O0 to get a better debug.

I turns out that some swf content displays OK.  Eg the GIS site:

while others (such as a finance site I must access) triggers the segv.

I use Gentoo’s ebuild to build; so --enable-system-cairo.  My cairo is git master.

I’ll kept the gdb session around for a while for further analysis, if needed.

Program received signal SIGSEGV, Segmentation fault.
mozilla::plugins::PluginInstanceParent::BackgroundDescriptor (this=0x7fffcd8ed6c0) at /usr/src/debug/www-client/firefox-9.0/mozilla-release/dom/plugins/ipc/PluginInstanceParent.cpp:874
874                                     xsurf->GetSize());

(gdb) print "xsurf->XRenderFormat()"
$1 = "xsurf->XRenderFormat()"

(gdb) print xsurf->XRenderFormat()
$2 = (XRenderPictFormat *) 0x0

(gdb) p *((Visual*(*)())cairo_xlib_surface_get_visual)(xsurf->mSurface)
$3 = {ext_data = 0x0, visualid = 33, c_class = 4, red_mask = 16711680, green_mask = 65280, blue_mask = 255, bits_per_rgb = 8, map_entries = 256}

(gdb) p *xsurf
$4 = {<gfxASurface> = {_vptr.gfxASurface = 0x7ffff6753330, mSurface = 0x7fffcd254320, mOpaqueRect = {mRawPtr = 0x0}, mFloatingRefs = 0, mBytesRecorded = 304800, mSurfaceValid = 1 '\001',
    mAllowUseAsSource = 1 '\001'}, mPixmapTaken = 1, mDisplay = 0x7ffff6da8000, mDrawable = 41944439, mSize = {<mozilla::gfx::BaseSize<int, nsIntSize>> = {width = 400, height = 254}, <No data fields>},
  mGLXPixmap = 0}

so msize has valid width and height, but XRenderFormat() returns NULL.
cloos, i have a Gentoo build but haven't seen the crash.
Can you attach output from "xdpyinfo -ext RENDER", please?
name of display:    :1.0
version number:    11.0
vendor string:    AT&T Laboratories Cambridge
vendor release number:    3332
maximum request size:  4194300 bytes
motion buffer size:  256
bitmap unit, bit order, padding:    32, LSBFirst, 32
image byte order:    LSBFirst
number of supported pixmap formats:    2
supported pixmap formats:
    depth 1, bits_per_pixel 1, scanline_pad 32
    depth 24, bits_per_pixel 32, scanline_pad 32
keycode range:    minimum 8, maximum 255
focus:  window 0x2800002, revert to Parent
number of extensions:    7
default screen number:    0
number of screens:    1

screen #0:
  dimensions:    1920x1200 pixels (650x406 millimeters)
  resolution:    75x75 dots per inch
  depths (1):    24
  root window id:    0x25
  depth of root window:    24 planes
  number of colormaps:    minimum 1, maximum 1
  default colormap:    0x21
  default number of colormap cells:    256
  preallocated pixels:    black 0, white 16777215
  options:    backing-store YES, save-unders YES
  largest cursor:    1920x1200
  current input event mask:    0xda3f3f
    KeyPressMask             KeyReleaseMask           ButtonPressMask          
    ButtonReleaseMask        EnterWindowMask          LeaveWindowMask          
    Button1MotionMask        Button2MotionMask        Button3MotionMask        
    Button4MotionMask        Button5MotionMask        ButtonMotionMask         
    StructureNotifyMask      SubstructureNotifyMask   SubstructureRedirectMask 
    PropertyChangeMask       ColormapChangeMask       
  number of visuals:    1
  default visual id:  0x22
    visual id:    0x22
    class:    TrueColor
    depth:    24 planes
    available colormap entries:    256 per subfield
    red, green, blue masks:    0xff0000, 0xff00, 0xff
    significant bits in color specification:    8 bits

RENDER extension not supported by server
(In reply to Ren Ren-Juan from comment #13)


> vendor string:    AT&T Laboratories Cambridge

Is this the X server shipped with Debian Squeeze?
Or is this a VNC connection?

> RENDER extension not supported by server

Do you know why this is?
Yes that's apparently it. I start FF in fvwm session which is started at the commmand line via fvwm being set as the VNC wm. I then access that either thru KDE or locally but only locally for some time. Running in just KDE I was unable to reproduce it.
Attached file xdpyinfo -ext RENDER
Sorry, forgot that.  Here it is.

Given its size and my limited net bw, I had to compress and attach it.
Assignee: nobody → karlt
Comment 5 to 7 are due to a system cairo built with --enable-xlib_xcb.  With this backend, cairo_xlib_surface_get_xrender_format returns a format only if the surface was created with a format.
The cairo_xcb_surface_t may well have a PictFormat, but this is not translated
to a XRenderPictFormat, a struct that is not typically used with xcb.

Migrating from Xlib/libXrender to xcb is something we'll want to do at some stage, and using this cairo backend may be a sensible first step.  The fix for bug 714695 will also fix this situation.
We can remove some duplicated code and fix this in one place instead of two.
Attachment #586903 - Flags: review?(jones.chris.g)
Simpler code and necessary to have a single fix in SurfaceDescriptorX11.
Attachment #586904 - Flags: review?(jones.chris.g)
This method is a bit tidier than XMatchVisualInfo and doesn't require a memory allocation, so I'd like to use it in SurfaceDescriptorX11.
Attachment #586906 - Flags: review?(jones.chris.g)
This is the actual fix for this bug.  The visual describes the Drawable format when there is no render format readily available.  I don't know for sure that X resources of different types will always have different XIDs, but the current implementation is that all resources get their XIDs from one pool.

There is some reordering of struct member variables here, which should pack better on 32-bit systems.  (On 64-bit systems, Xlib uses longs for XIDs even though they are 32-bit integers.)
Attachment #586908 - Flags: review?(jones.chris.g)
Attachment #586908 - Flags: review?(jones.chris.g)
This version is more robust against situations that shouldn't happen.
Attachment #586908 - Attachment is obsolete: true
Attachment #586911 - Flags: review?(jones.chris.g)
No longer depends on: 646150
Duplicate of this bug: 646150
Comment on attachment 586903 [details] [diff] [review]
share code for layers::SurfaceDescriptorX11 with plugins

>diff --git a/dom/plugins/ipc/PPluginInstance.ipdl b/dom/plugins/ipc/PPluginInstance.ipdl
>-struct SurfaceDescriptorX11 {
>-  int XID;
>-  int xrenderPictID;
>-  gfxIntSize size;

This makes me a little sad, but it's my fault for not fixing bug

>diff --git a/dom/plugins/ipc/PluginMessageUtils.h b/dom/plugins/ipc/PluginMessageUtils.h

>+#include "gfxipc/ShadowLayerUtils.h"

Huh I had no idea we had "gfxipc".  I wouldn't have r+'d that,
but not your problem :).

>+using mozilla::layers::SurfaceDescriptorX11;

Can you pull this into mozilla::plugins to avoid polluting the
global namespace?

r=me with that fixed.  Nice cleanup!
Attachment #586903 - Flags: review?(jones.chris.g) → review+
Attachment #586904 - Flags: review?(jones.chris.g) → review+
Comment on attachment 586906 [details] [diff] [review]
move XVisualIDToInfo to X11Util

>diff --git a/gfx/src/X11Util.h b/gfx/src/X11Util.h

>+XVisualIDToInfo(Display* aDisplay, VisualID aVisualID,
>+                Visual** aVisual, unsigned int* aDepth);

Can you add a short comment describing this function?  Also the
ownership semantics of |aVisual|.

r=me with that.
Attachment #586906 - Flags: review?(jones.chris.g) → review+
Comment on attachment 586911 [details] [diff] [review]
pass VisualID when PictFormat is not available

>diff --git a/gfx/layers/ipc/ShadowLayerUtilsX11.h b/gfx/layers/ipc/ShadowLayerUtilsX11.h

>-  SurfaceDescriptorX11(const int aXid, const int aXrenderPictID, const gfxIntSize& aSize);
>+  SurfaceDescriptorX11(const int aXid, const int aFormatID, const gfxIntSize& aSize);

I don't recall why these use |int| and not |XID|, but if you're
happy with it I'm happy with it :).  Looks like I r+'d this in
bug 687373.

Looks good.
Attachment #586911 - Flags: review?(jones.chris.g) → review+
Duplicate of this bug: 717179
You need to log in before you can comment on or make changes to this bug.