Closed Bug 855221 Opened 7 years ago Closed 7 years ago

crash in _cairo_quartz_draw_image @ CGAccessSessionRewind

Categories

(Core :: Graphics, defect, critical)

22 Branch
All
macOS
defect
Not set
critical

Tracking

()

VERIFIED FIXED
mozilla24
Tracking Status
firefox21 --- unaffected
firefox22 + verified
firefox23 --- verified
firefox24 --- verified

People

(Reporter: scoobidiver, Assigned: joe)

References

Details

(Keywords: crash, regression, topcrash)

Crash Data

Attachments

(1 file)

Despite the stack trace, it seems to be a regression in Firefox as it first showed up in 22.0a1/20130325105600. The regression range is:
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=3acbf951b3b1&tochange=4d3250f3afea (best case)
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=0a10eca0c521&tochange=4d3250f3afea (worst case)
It might be caused by bug 716140.

Signature 	CoreGraphics@0x34cd3 More Reports Search
UUID	9614f5c4-d040-4c47-b3e6-d61c52130327
Date Processed	2013-03-27 07:07:56
Uptime	365
Last Crash	6.5 minutes before submission
Install Age	6.1 minutes since version was first installed.
Install Time	2013-03-27 07:01:34
Product	Firefox
Version	22.0a1
Build ID	20130326030941
Release Channel	nightly
OS	Mac OS X
OS Version	10.8.3 12D78
Build Architecture	amd64
Build Architecture Info	family 6 model 23 stepping 10
Crash Reason	EXC_BAD_ACCESS / KERN_INVALID_ADDRESS
Crash Address	0x10
App Notes 	
AdapterVendorID: 0x10de, AdapterDeviceID: 0x 8a0GL Context? GL Context+ GL Layers? GL Layers+ 
Processor Notes 	sp-processor10.phx1.mozilla.com_16437:2008; exploitablity tool: ERROR: unable to analyze dump
EMCheckCompatibility	True
Adapter Vendor ID	0x10de
Adapter Device ID	0x 8a0

Frame 	Module 	Signature 	Source
0 	CoreGraphics 	CoreGraphics@0x34cd3 	
1 	CoreGraphics 	CoreGraphics@0x34eeb 	
2 	CoreGraphics 	CoreGraphics@0x5fda3 	
3 	libRIP.A.dylib 	libRIP.A.dylib@0xd796

More reports at:
https://crash-stats.mozilla.com/report/list?signature=CoreGraphics%400x34cd3
It occurs also on OS X 10.7 but at a lower volume.

More reports also at:
https://crash-stats.mozilla.com/report/list?signature=CoreGraphics%400x76fef
Crash Signature: [@ CoreGraphics@0x34cd3] → [@ CoreGraphics@0x34cd3] [@ CoreGraphics@0x76fef]
Summary: [10.8.3] crash in CoreGraphics@0x34cd3 → crash in libRIP.A.dylib @ CoreGraphics
With combined signatures, it's #3 top browser crasher in 22.0a1 and #2 in 23.0a1 on Mac OS X.

Here is a better stack trace:
Frame 	Module 	Signature 	Source
0 	CoreGraphics 	CGAccessSessionRewind 	
1 	CoreGraphics 	CGAccessSessionRelease 	
2 	CoreGraphics 	CGSImageDataUnlock 	
3 	libRIP.A.dylib 	ripc_DrawImages 	
4 	libRIP.A.dylib 	ripc_TilePattern 	
5 	libRIP.A.dylib 	ripc_GetColor 	
6 	libRIP.A.dylib 	ripc_Render 	
7 	libRIP.A.dylib 	ripc_DrawRects 	
8 	CoreGraphics 	CGContextFillRects 	
9 	CoreGraphics 	CGContextFillRect 	
10 	CoreGraphics 	CGContextDrawTiledImage 	
11 	XUL 	_cairo_quartz_draw_image 	cairo-quartz-surface.c:1895
12 	XUL 	_cairo_quartz_surface_paint_cg 	cairo-quartz-surface.c:2342
13 	CoreFoundation 	___CFBasicHashFindBucket1 	
14 	XUL 	_cairo_quartz_surface_paint 	cairo-quartz-surface.c:2382 

More reports also at: https://crash-stats.mozilla.com/report/list?signature=CGAccessSessionRewind
Crash Signature: [@ CoreGraphics@0x34cd3] [@ CoreGraphics@0x76fef] → [@ CoreGraphics@0x34cd3] [@ CoreGraphics@0x76fef] [@ CGAccessSessionRewind]
Component: General → Graphics
Keywords: topcrash
Summary: crash in libRIP.A.dylib @ CoreGraphics → crash in _cairo_quartz_draw_image @ CGAccessSessionRewind
Duplicate of this bug: 856444
Milan, who might be in the best position to perform stack/change inspection for this bug? Thanks :)
Assignee: nobody → milan
Joe is not back until Friday, and if this is related to bug 716140, he or Seth (covering in the interim) are good people for it. Seth, do you have the bandwidth right now?
Blocks: 846759
Milan, I already took a look at this and didn't have much luck. I'll be happy to circle back around to it if it starts sitting for too long without getting fixed, but I lack the experience with this part of the code to solve this quickly. Joe is probably a good person to triage this. I've needinfo'd him.
Flags: needinfo?(joe)
Here are some comments:
"zooming/panning google maps and kaboom. weird."
"I was uploading pictures on iTunesConnect. I had several time this crash since yesterday evening latest update"
"using bill.com, had just looked at an expense I needed to approve."
Keywords: needURLs
I hit this crash this morning when my machine was idle. I had google maps open, but I don't believe I was doing anything at the time of the crash in maps. 

Some URLs for the [@ CGAccessSessionRewind ] crash:


3 	https://maps.google.com/
1 	https://www.facebook.com/dialog/oauth?client_id=139475280761&response_type=token
1 	http://fr.yahoo.com/
1 	https://maps.google.com/maps?oe=utf-8&rls=org.mozilla:en-US:unofficial&client=fi
1 	http://www.megazip.ru/ru/bike/yamaha/view/1124/13485/160973
1 	http://fr.geneawiki.com/index.php/L%27Allemand_Gothique
1 	http://thestiffcollar.com/index.php/edistributor?___store=default
1 	https://docs.google.com/viewer?a=v&pid=gmail&attid=0.1&thid=12d050bf68faf6aa&mt=
1 	https://maps.google.fr/
1 	https://docs.google.com/viewer?a=v&pid=gmail&attid=0.1&thid=13de9510ba0a6fcf&mt=
1 	http://www.emulroom.com/games/dendy/action/legend-of-kage-legenda-o-keydzhe/
1 	https://www.facebook.com/ajax/pagelet/generic.php/PhotoViewerInitPagelet?ajaxpip
1 	http://www.aol.com/
1 	https://images.4chan.org/pol/src/1365446781919.png
1 	https://maps.google.com/maps?oe=utf-8&client=firefox-nightly&q=27+Eastwind+St.&i
1 	http://it.volkswagen.com/it/models/nuova-golf/CC5.html
1 	http://leprosorium.ru/users/Pokras_Lampas
1 	http://www.artscow.com/photo-gifts/-160
1 	https://www.google.com/calendar/render?tab=Xc&pli=1
1 	http://www.google.com/
1 	http://www.nfl.com/
1 	http://this--is-moi.tumblr.com/post/36916980727/photoset_iframe/dying-slowly-by-
1 	http://advate.com/
1 	http://maps.google.com/
1 	http://www.weather.gov/
1 	http://www.emulroom.com/
1 	https://accounts.google.com/AddressNoLongerAvailable?service=wise&continue=https
1 	http://cantinetta.antinori.twinbitlabs.com/it/mosca/wine/vini-bianchi
1 	http://images.4chan.org/g/src/1365655944262.png
1 	https://maps.google.ca/maps?oe=utf-8&client=firefox-aurora&ie=UTF-8&q=toronto+cu
1 	javascript:false;
1 	http://zeroing.tumblr.com/tagged/sculpture
1 	http://failblog.cheezburger.com/failbook
1 	http://weeklyad.michaels.com/coupons/?storeId=9982&promotionCode=Michaels-130407
1 	https://docs.google.com/document/d/1-jaLcYBCfV-UqPGEQR8-FnJ_ekcfMs3OXCOQBQKgGTo/
1 	https://www.4chan.org/frames
1 	http://doodlestagsgraphics.proboards.com/index.cgi?board=general&action=post&thr
1 	http://m.163.com/iphone/software/31qsrj.html
Keywords: needURLs
Wonder if this is any different now that bug 857876 has landed. I hope so, but only time will tell.
Flags: needinfo?(joe)
Combined signatures would put this at #6. Will keep an eye on it.
Joe, has the patch in bug 856876 been uplifted to aurora?  Combined signatures put this at #7 on 22.0a2.
Not yet, but thanks for the reminder!
Crash Signature: [@ CoreGraphics@0x34cd3] [@ CoreGraphics@0x76fef] [@ CGAccessSessionRewind] → [@ CoreGraphics@0x34cd3] [@ CoreGraphics@0x76fef] [@ CoreGraphics@0x356b3] [@ CoreGraphics@0x34b93] [@ CGAccessSessionRewind]
Hardware: x86_64 → All
FWIW I got this crash today with Nightly 23.0a1 (2013-05-04) on Mountain Lion, so bug 857876 might not have covered it all. My crash:
https://crash-stats.mozilla.com/report/index/bp-ada2e150-4b7f-4f6f-bde1-b01d02130506

I'm almost sure it occurred when switching to a google maps tab that had been open for a long time (~1 day), across sleep/wake sessions.

I'll keep an eye if it happens again
Blocks: 870005
(In reply to :Felipe Gomes from comment #13)
> FWIW I got this crash today with Nightly 23.0a1 (2013-05-04) on Mountain
> Lion, so bug 857876 might not have covered it all. My crash:
> https://crash-stats.mozilla.com/report/index/bp-ada2e150-4b7f-4f6f-bde1-
> b01d02130506
> 
> I'm almost sure it occurred when switching to a google maps tab that had
> been open for a long time (~1 day), across sleep/wake sessions.
> 

Any non-default gfx. preferences set?
It's #1 top browser crasher and accounts for 7.5% of crashes in 22.0b1 on Mac OS X.

Here are new comments:
"lots of tabs open, crashed on tab close"
"Was going though image tabs I opened from Tumblr when it crashed."
"Tried to use the Tinkercad website it just crashed."
"Just opened a new website… for SoapUI"
"posting on ebay"
"Playing Farmville on Facebook and it just quit on me. no warning"
So, interestingly, a crash I just looked at has a decoder thread concurrently writing into a PNG. I wonder if we were drawing that image, and if so, whether OS X really dislikes that.

https://crash-stats.mozilla.com/report/index/5d6a2c94-3299-4df1-a301-c41ed2130520
Very interesting!

You presumably mean the thread stack that contains a all to mozilla::image::nsPNGDecoder::WriteInternal.  I just looked at several of these crashlogs from Socorro, and all contain such a thread stack.

By the way, *all* of these crashes (on all versions of OS X) are in CGAccessSessionRewind(), at exactly the same place in that method's code.

A few days ago I started doing some analysis of these crashes.  But I need to spend another day or two before I'll have much to say.  With luck I'll have time for that later this week.

For my own future reference, here's an example where the thread stack that contains the call to mozilla::image::nsPNGDecoder::WriteInternal contains an unusual amount of code above it:

bp-26c381e3-fe94-4364-8622-aaa252130519
Still happening in 24.0 Nightly (UX build in this case). I got it while viewing screenshots on Evernote/Skitch.
Kudos and congratulations and enormous thanks if you can figure out how to reproduce the crash.

Anything at all unusual (or even just non-vanilla) about your settings?
(In reply to Scoobidiver from comment #15)
> It's #1 top browser crasher and accounts for 7.5% of crashes in 22.0b1 on
> Mac OS X.

This is bad - who's actively investigating this by the way?
Assignee: milan → nobody
Flags: needinfo?(smichaud)
Flags: needinfo?(joe)
I'm waiting on Steven for further information. Hopefully he can shed some light!
Flags: needinfo?(joe)
I've been working on this off and on for the last week.

I'll keep working on it, but so far I haven't discovered anything actionable.
Flags: needinfo?(smichaud)
I'll talk with folks about it and see what can be done.

Scoobidiver, how much crash volume do we have on Nightly? I ask because I wonder whether we can test out speculative fixes on Nightly instead of on Beta.
Assignee: nobody → joe
Flags: needinfo?(scoobidiver)
I added up all the signatures on the trunk - looks as if the last week we have about 51 crashes.

I reproduced this crash on the latest beta by loading one of the URLs and just leaving the browser idle - https://crash-stats.mozilla.com/report/index/bp-7accfd3c-d577-4471-8fe6-3447a2130524. The URL is in the crash report.
(In reply to Joe Drew (:JOEDREW! \o/) from comment #23)
> Scoobidiver, how much crash volume do we have on Nightly?
In relative value, it's #5 top crasher in 24.0a1 on Mac OS X but the trunk is polluted by new temporary top crashers.
In absolute value, there are 0.08 crashes per 100 ADU in 22.0b1 and 0.06 crashes per 100 ADU in 24.0a1 so its volume hasn't changed.

New comment say:
"http://www.horsexpo.com/html/schedule.html"
"Tried to use the Tinkercad website it just crashed." (https://tinkercad.com/)
Flags: needinfo?(scoobidiver)
Attached patch probably fixSplinter Review
This makes us not flush in imgFrame::ImageUpdated(), which probably caused the bug. We already flush when necessary, so it's not only dangerous, it's also unnecessary!
Attachment #753675 - Flags: review?(jmuizelaar)
Comment on attachment 753675 [details] [diff] [review]
probably fix

Review of attachment 753675 [details] [diff] [review]:
-----------------------------------------------------------------

::: image/src/imgFrame.cpp
@@ +492,1 @@
>  {

ImageUpdated should have a comment that it is called on multiple threads.
Attachment #753675 - Flags: review?(jmuizelaar) → review+
(In reply to comment #24)

> The URL is in the crash report.

I don't see it, even in the raw dump.
Flags: needinfo?(mozillamarcia.knous)
Comment on attachment 753675 [details] [diff] [review]
probably fix

[Approval Request Comment]
Bug caused by (feature/regressing bug #): bug 716140
User impact if declined: crashes on OS X
Testing completed (on m-c, etc.): just pushed to m-i, passed try
Risk to taking this patch (and alternatives if risky): Very low risk. Should be strictly better. If there's a bug, could possibly cause some images to not always be drawn completely, but this is *very* unlikely.
String or IDL/UUID changes made by this patch: none
Attachment #753675 - Flags: approval-mozilla-beta?
Attachment #753675 - Flags: approval-mozilla-aurora?
http://www.comic-rocket.com/go?mark&nav=next&uri=http%3A//nonadventures.com/2009/04/04/a-broken-pumice/

(In reply to Steven Michaud from comment #29)
> (In reply to comment #24)
> 
> > The URL is in the crash report.
> 
> I don't see it, even in the raw dump.
Flags: needinfo?(mozillamarcia.knous)
Thanks Marcia.

But how long did you wait (how long did you leave the browser idle)?
Comment on attachment 753675 [details] [diff] [review]
probably fix

Will approve Mon/Tue once this has had a day to bake on m-c.
When you do, set it as checkin-needed; I'm off Monday.
https://hg.mozilla.org/mozilla-central/rev/2451cdbc2b06
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla24
Attachment #753675 - Flags: approval-mozilla-beta?
Attachment #753675 - Flags: approval-mozilla-beta+
Attachment #753675 - Flags: approval-mozilla-aurora?
Attachment #753675 - Flags: approval-mozilla-aurora+
There have been no crashes since 24.0a1/20130526.
(In reply to Manuela Muntean [:Manuela] [QA] from comment #39)
> For 3 of the signatures: [@ CoreGraphics@0x34cd3], [@ CoreGraphics@0x76fef],
> and [@ CGAccessSessionRewind], I found some crash reports in Socorro
> regarding last month, marked with 2013-05-28 date in the build IDs, on
> Firefox 23.0a2.
The contrary (no crashes in 23.0a2/20130528) would be surprising as the patch landed in 23.0a2/20130529: http://hg.mozilla.org/releases/mozilla-aurora/pushloghtml?startdate=2+day+ago&enddate=now
Looks good because no crashes since 23.0a2/20130529 and 22.0b3.
Blocks: 861100
Indeed, no more crashes since 23.0a2/20130529 and 22.0b3 in Socorro, for none of the signatures.
Status: RESOLVED → VERIFIED
Duplicate of this bug: 881663
You need to log in before you can comment on or make changes to this bug.