Last Comment Bug 900020 - graphical glitches on Nexus 10/Android 4.3 (was: crash in libutils.so@0xf74e)
: graphical glitches on Nexus 10/Android 4.3 (was: crash in libutils.so@0xf74e)
Status: RESOLVED FIXED
[native-crash]
: crash, reproducible
Product: Firefox for Android
Classification: Client Software
Component: Graphics, Panning and Zooming (show other bugs)
: Trunk
: ARM Android
: -- critical with 6 votes (vote)
: Firefox 27
Assigned To: Benoit Jacob [:bjacob] (mostly away)
:
Mentors:
: 917091 917589 (view as bug list)
Depends on: 925608
Blocks: 912522 917589 920006
  Show dependency treegraph
 
Reported: 2013-07-31 07:38 PDT by Robert Kaiser (not working on stability any more)
Modified: 2014-04-03 07:13 PDT (History)
24 users (show)
See Also:
Crash Signature:
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---
+
wontfix
+
wontfix
fixed
fixed
fixed
24+


Attachments
Logcat (30.92 KB, text/plain)
2013-08-06 13:00 PDT, Aaron Train [:aaronmt]
no flags Details
One of many crashes in CPU-Z (20.44 KB, text/plain)
2013-08-08 14:13 PDT, Kartikaya Gupta (email:kats@mozilla.com)
no flags Details
Some debugging info (4.17 KB, text/plain)
2013-09-19 15:04 PDT, Kartikaya Gupta (email:kats@mozilla.com)
no flags Details
(The actual fix) Renew the surface when we hit an incomplete default framebuffer in the compositor (1.70 KB, patch)
2013-09-21 12:36 PDT, Benoit Jacob [:bjacob] (mostly away)
ncameron: review+
Details | Diff | Review
(Additional patch) In BindRenderTarget, after initialization, don't just return (2.54 KB, patch)
2013-09-21 13:12 PDT, Benoit Jacob [:bjacob] (mostly away)
no flags Details | Diff | Review
Patch backported to mozilla-beta (1.69 KB, patch)
2013-09-22 14:02 PDT, Benoit Jacob [:bjacob] (mostly away)
lukasblakk+bugs: approval‑mozilla‑aurora+
lukasblakk+bugs: approval‑mozilla‑beta+
Details | Diff | Review

Description Robert Kaiser (not working on stability any more) 2013-07-31 07:38:30 PDT
This bug was filed from the Socorro interface and is 
report bp-79080b57-a3a2-4f16-9187-de7ce2130730 .
 ============================================================= 

This has been rising and visible in explosiveness reports for a few days now, but is pretty low in overall volume so far, probably due to only being visible on Nexus 10 with Android 4.3 so far, which is a small population.

The stacks here vary somewhat, but all of them have dalvik stuff mainly, and no frames in actual Mozilla code.

This is visible on all versions, from 22 release to 25 Nightly.


Some comments:

"shows half screen when browsing and crashes nexus 10"

"Crashs and display errors since upgrade of the nexus 10 to android 4.3 with both Firefox 22 and Firefox Beta 23."

"problems ever since upgrade to 4.3"

"sudden freezes and crash"

"logging into xda-developer forum"

"Firefox Beta is driving me crazy crashing all the time. Please fix. Thanks."
Comment 2 Aaron Train [:aaronmt] 2013-07-31 08:32:46 PDT
"Highly unstable with latest update	Android 4.3 & Nexus 10: finally had Firefox somewhat stable and useful, then updated to 4.3 Jelly Bean. Now ff is so unpredictable & unreliable I'm simply no longer using it for anything other than 2 University links that seem to only work reliably on ff (oddly enough). Back to latest chrome for now."

Ill see if we have a Nexus 10 still in Toronto.
Comment 3 Kevin Smith 2013-08-02 07:31:18 PDT
Android 4.3 and Nexus 10. There appears to be a relationship to cutting and pasting into text field, such as a search bar. I can reproduce the issue that way. Could also be a multitasking issue, since I'm cutting from a password keeper switching back to FF and pasting. Then above mentioned issues appear.
Comment 4 Robert Kaiser (not working on stability any more) 2013-08-02 07:48:36 PDT
This is a rising crash that is reproducible easily with the newest Android version at least on the wide-spread Nexus 10, so I'm nominating for the release that's upcoming in 6-7 weeks.
Comment 5 bhavana bajaj [:bajaj] 2013-08-02 14:46:48 PDT
neeedinfo'ing Mark to help assign this to someone for investigation.
Mark, looks like Aaron can help from QA side based on comment #2
Comment 6 Chris Hubick 2013-08-05 01:16:06 PDT
I am also experiencing serious instability (primarily lockups) with my Nexus 10 after the Android 4.3 update.

My problem seems primarily related to multitasking between several tabs.  IIC, you can still switch between tabs, and the title will change, but no content will be drawn (contents of the previous tab remain).  Restarting the browser and then restoring all previous tabs will bring things back for a short time, at least until it locks up again :(

On the plus side, at least bug 845944 was fixed by the 4.3 update :)
Comment 7 Aaron Train [:aaronmt] 2013-08-06 07:33:52 PDT
Using the device in office this morning I haven't hit a crash yet but I highly suspect it's related to the issues I am seeing, I tested Nightly and Beta with the same symptoms: 

* On rotation, content is severed and replaced with an area of missing content (black)
* The address-bar vanishes from view on screen and does not re-appear
* Mismatch between reported address and content on screen i.e, CNN → WSJ yields (CNN address + favicon → WSJ content)

The browser is unusable on the Nexus 10 with 4.3.

Assigning in-house to Kats to take a look at this tablet.
Comment 8 Aaron Train [:aaronmt] 2013-08-06 07:50:04 PDT
Seeing in logcat plenty of drawing aborts, on load and on rotation

* 08-06 10:51:30.025 D/GeckoLayerClient( 2708): Screen-size changed to (1600,2464)
* 08-06 10:51:30.025 D/GeckoLayerClient( 2708): Window-size changed to (1600,2414)

Also

* D/mali_winsys(  576): new_window_surface returns 0x30
* W/TextureGenerator( 2708): Clearing GL error: 0x506
* W/TextureGenerator( 2708): Clearing GL error: 0x506

GPU Vendor: ARM
GPU Renderer: Mali-T604

libGLES_mali.so from device: http://people.mozilla.com/~atrain/mobile/libGLES_mali.so
Comment 9 Aaron Train [:aaronmt] 2013-08-06 12:59:14 PDT
More findings

* These events seem to happen as fallout after a device orientation change
* Keeping the device locked in landscape after startup keeps the browser stable as far as I can see in testing thus far.

Crash in libGLES_mali.so but not libutils

https://crash-stats.mozilla.com/report/index/fa84424a-9737-4d39-aed3-2d3812130806
Comment 10 Aaron Train [:aaronmt] 2013-08-06 13:00:50 PDT
Created attachment 786459 [details]
Logcat
Comment 11 Lukas Blakk [:lsblakk] use ?needinfo 2013-08-06 13:39:59 PDT
I've added this to the known issues on the FF23 release notes and we'll track this in the hopes that a low risk fix (which can be very well tested to not cause any other regressions) might be possible for a respin if we deem it necessary (due to user feedback and devices with that update trying to use FF23).  Alternately if we can find a fix to land in Beta we might want to try and encourage users who leave 1 star ratings in GP store because of this specific issue to try Beta (once a fix is landed and built/shipped).

Anyway, I expect SUMO and QA to keep us posted on our options and advancements here.
Comment 12 Kartikaya Gupta (email:kats@mozilla.com) 2013-08-08 13:51:05 PDT
I spent the last half-hour on the Nexus 10 trying to find STR for the problems. I've run into the problem where on rotation half the screen goes black and the content area stops responding (panning doesn't move the remaining half-content, but it does slide the dynamic toolbar in and out, making me think the compositor gets wedged somehow). A number of times I saw pieces of content on CNN (e.g. the video/image boxes) disappear while other parts of the page rendered fine. These were all transient issues that went away after a few seconds, and were usually accompanied by "D/GeckoLayerClient(27817): Aborting draw due to resolution change" in the logcat, so I suspect there's a legitimate bug there.

I haven't yet run into any crashes or any other issues. I've been using 23 release, 23 beta, and Nightly (some of the Firefoxen that were already on the tablet). I'll try a bit more to consistently reproduce the rotation problem since that's the only one that I've seen that doesn't go away on its own.
Comment 13 Kartikaya Gupta (email:kats@mozilla.com) 2013-08-08 14:13:45 PDT
Created attachment 787771 [details]
One of many crashes in CPU-Z

FWIW I do think this is at least partly a graphics driver problem because the CPU-Z application installed on this tablet is also reporting segfaults in libGLES_mali.so. One such crash log is attached. Interestingly it doesn't actually kill the app; I just see this dumped in logcat. I don't know what that means.
Comment 14 Mark Finkle (:mfinkle) (use needinfo?) 2013-08-12 07:29:06 PDT
Assigned
Comment 15 Aaron Train [:aaronmt] 2013-08-12 07:37:57 PDT
Any ARM Mali contacts available for outreach?
Comment 16 Kartikaya Gupta (email:kats@mozilla.com) 2013-08-12 08:34:41 PDT
I sent an email to the Mali contact who helped us out with bug 845944. Will update this bug as I get more info.
Comment 17 Robert Kaiser (not working on stability any more) 2013-08-14 13:25:44 PDT
(In reply to Kartikaya Gupta (email:kats@mozilla.com) from comment #12)
> I've run into the problem where on rotation half the screen goes
> black and the content area stops responding (panning doesn't move the
> remaining half-content, but it does slide the dynamic toolbar in and out,
> making me think the compositor gets wedged somehow).

I just got a Nexus 10, updated it to 4.3, and can get it to run into this pretty easily as well, I haven't seen *that* crash yet so far though (have run into others though).
Should we spin off the painting issue into a new bug?
Comment 18 Kartikaya Gupta (email:kats@mozilla.com) 2013-08-15 05:36:39 PDT
(In reply to Kartikaya Gupta (email:kats@mozilla.com) from comment #16)
> I sent an email to the Mali contact who helped us out with bug 845944. Will
> update this bug as I get more info.

I got this reply:

> A brief update - we have someone looking at this now and can
> reproduce; we are not sure where in the stack the error is
> stemming from, but we are working on it.

(In reply to Robert Kaiser (:kairo@mozilla.com) from comment #17)
> I just got a Nexus 10, updated it to 4.3, and can get it to run into this
> pretty easily as well, I haven't seen *that* crash yet so far though (have
> run into others though).
> Should we spin off the painting issue into a new bug?

We could, but at the moment I'm not entirely sure which pieces are separate issues. They all appear to be related to the graphics driver and may all be fixed by the same workaround so for now I'd rather just leave it in one bug and then spin off others if we end up with fixes that don't address everything.
Comment 19 Robert Kaiser (not working on stability any more) 2013-08-18 00:05:01 PDT
bp-8e2e3509-a4ea-4702-a162-73a932130818 is a crash with a different signature that I just saw right after the screen corruption happened and I tried to get to the app switcher to kill Nightly.
Comment 20 Kartikaya Gupta (email:kats@mozilla.com) 2013-08-23 07:26:08 PDT
I got a reply back from the ARM contact. They were able to reproduce the problem and debug some of it on their end, but based on their data it is a problem in our code. The information they provide should be quite useful in tracking down the problem.

> A brief update from our investigation to date. We've managed to get our Nexus 10/Firefox into a state where we are seeing calls to eglMakeCurrent returning errors in logcat. There was a pattern that each time we rotated the tablet Firefox made 10 eglMakeCurrent calls and it was the 7th that failed on each rotation.
> 
> So I had closer look at the 7th call. The call being made to our driver is
> 
> #0  eglMakeCurrent (display=0x73be6638, draw_surface=0x2, read_surface=0x2,
>      context=0x7e168840) at vendor/arm/mali6xx/egl/src/mali_egl_thread_state.c:573
> 
> Obviously 0x2 is not really the address of a surface and so we get the EGL_BAD_SURFACE error. So the question is why are these clearly erroneous values being passed in?
> 
> The full backtrace shows how the Android wrapper layer changes the application supplied surface values
> 
> #0  eglMakeCurrent (display=0x73be6638, draw_surface=0x2, read_surface=0x2,
>      context=0x7e168840) at vendor/arm/mali6xx/egl/src/mali_egl_thread_state.c:573
> #1  0x40571862 in android::egl_display_t::makeCurrent (this=0x405b5e04,
>      c=0x8284a150, cur_c=0x8284a150, draw=0x75789800, read=0x75789800,
>      ctx=0x8284a150, impl_draw=0x2, impl_read=0x2, impl_ctx=0x7e168840)
>      at frameworks/native/opengl/libs/EGL/egl_display.cpp:340
> #2  0x40573d4c in eglMakeCurrent (dpy=<optimized out>, draw=0x75789800,
>      read=0x75789800, ctx=0x8284a150)
>      at frameworks/native/opengl/libs/EGL/eglApi.cpp:652
> #3  0x7d2b4666 in ?? ()
> 
> Android applications actually get given references to egl_surface_t objects when they create surfaces. These  contain, amongst other things, the "real" surface reference.
> 
> Casting the value passed in by firefox to an egl_surface_t object gives
> 
> (gdb) p *((egl_surface_t *)0x75789800)
> $44 = {<android::egl_object_t> = {_vptr.egl_object_t = 0x7f528608,
>      display = 0x823b2340, count = 1941014408}, surface = 0x2, config = 0x735bd008,
>    win = {m_ptr = 0x0}, cnx = 0x0}
> 
> In comparison if I take some of the successful calls to eglMakeCurrent and cast the given references from those calls to egl_surface_t objects I get the following
> 
> (gdb) p /x (*((egl_surface_t *)0x72101d48))
> $49 = {<android::egl_object_t> = {_vptr.egl_object_t = 0x405afa70,
>      display = 0x405b5e04, count = 0x3}, surface = 0x7f50f900, config = 0x721822e0,
>    win = {m_ptr = 0x735f7c38}, cnx = 0x405b7750}
> 
> (gdb) p /x (*((egl_surface_t *)0x828db518))
> $50 = {<android::egl_object_t> = {_vptr.egl_object_t = 0x405afa70,
>      display = 0x405b5e04, count = 0x1}, surface = 0x821e9b38, config = 0x721822e0,
>    win = {m_ptr = 0x0}, cnx = 0x405b7750}
> (gdb) x /16w 0x828db518
> 
> In particular the fact that the vptr value is different for the failing case strongly suggests that the pointer the application has passed in does not point to an egl_surface_t object. In the absence of any other evidence then we would put this down to an application error.
> 
> We will continue a little further to investigate some of the crash reports now, but it would be useful if you could look at the code in FireFox land which is generating the eglMakeCurrent calls.
Comment 21 Kartikaya Gupta (email:kats@mozilla.com) 2013-09-05 14:29:57 PDT
So I tried working on this a little bit today but ran into various technical difficulties and couldn't make much actual progress. Turns out run-as is broken on android 4.3, so our jimdb setup doesn't work unless the device is rooted. I tried doing that but it almost got bricked and so I had to reflash it from scratch. Now that I have it rooted jimdb still doesn't to pull the libraries from the device properly (it just hangs) so I'll need to dig into that problem. :(
Comment 22 Robert Kaiser (not working on stability any more) 2013-09-10 06:46:46 PDT
I haven't run into that crash in quite some time, but easily can run into the stuck/weird screen issue. Any news here?
Comment 23 Kartikaya Gupta (email:kats@mozilla.com) 2013-09-10 13:22:07 PDT
Not really. I tried to debug it the last time I was in the Toronto office and had access to the Nexus 10 but failed to get the debugger to work. Perhaps somebody on the gfx team in the toronto office can grab the device from Aaron and give it another shot? CC'ing bjacob since I read somewhere he's out of urgent bugs :)
Comment 24 Benoit Jacob [:bjacob] (mostly away) 2013-09-10 13:56:59 PDT
Which bug should I work on first, this one of bug 912522 ? I can start on either tomorrow.
Comment 25 Kartikaya Gupta (email:kats@mozilla.com) 2013-09-10 14:18:15 PDT
It looks like 912522 is higher on the nightly topcrash list, so maybe that one. Then again, the severity of this bug is underestimated by crash-stats because it often results in bad behaviour without crashing the browser. I'll say do 912522 first but if you get stalled or blocked on it, this bug will be waiting for you :)
Comment 26 Tim Meader 2013-09-10 15:03:51 PDT
I suppose that one affects more people, but any progress made on this one would GREATLY be appreciated. Personally I've only encountered the crash cited here once, however the graphics corruption is so bad (frequent) on my Nexus 10 that I've had to stop using FF24 Beta entirely in favor of Chrome for the time being :(  Looking forward to going back to FF when possible.

Thanks.
Comment 27 Benoit Jacob [:bjacob] (mostly away) 2013-09-10 15:08:43 PDT
Milan, putting this on your radar.
Comment 28 Robert Kaiser (not working on stability any more) 2013-09-11 06:28:41 PDT
I have a Nexus 10 and if I wouldn't be so well-entrenched in Mozilla, I would abandon Firefox on it because of this graphics corruption. OTOH, bug 912522 would probably be worse but hasn't been seen in builds after 2013-09-06 even though a decent number of people have updated to yesterday's build. Also, it's in Nightly only while this one happens across all channels.

Because of that (esp. the other one not happening on yesterday's Nightly so far), I think we should prioritize this one and I'd recommend that bjacob should work on the Nexus 10 issue for now.
Comment 29 Benoit Jacob [:bjacob] (mostly away) 2013-09-11 07:41:07 PDT
bug 912522 is apparently turning out to be a duplicate of something just fixed, so i should be free to try this bug now.
Comment 30 Benoit Jacob [:bjacob] (mostly away) 2013-09-11 07:48:59 PDT
There is no Nexus 10 device for me to use at the moment in the Toronto office. Reverting assignee to Kats.
Comment 31 Aaron Train [:aaronmt] 2013-09-13 09:58:29 PDT
(In reply to Benoit Jacob [:bjacob] from comment #30)
> There is no Nexus 10 device for me to use at the moment in the Toronto
> office. Reverting assignee to Kats.

On my desk
Comment 32 Erin Lancaster [:elan] 2013-09-13 10:00:18 PDT
We may want to relnote that this is still an issue for Fx24.
Comment 33 bhavana bajaj [:bajaj] 2013-09-13 10:28:36 PDT
(In reply to Erin Lancaster [:elancaster] from comment #32)
> We may want to relnote that this is still an issue for Fx24.

https://www-dev.allizom.org/en-US/mobile/24.0/releasenotes/, already in there:)
Comment 34 regisg 2013-09-15 09:11:38 PDT
Same problems here. Firefox is almost unusable on nexus 10. I am a big supporter of mozilla but I have to temporarily switch to chrome.
Comment 35 Benoit Jacob [:bjacob] (mostly away) 2013-09-15 12:41:42 PDT
Brad, putting this on your radar.
Comment 36 Aaron Train [:aaronmt] 2013-09-17 07:05:16 PDT
*** Bug 917091 has been marked as a duplicate of this bug. ***
Comment 37 Brad Lassey [:blassey] (use needinfo?) 2013-09-17 08:57:55 PDT
(In reply to Benoit Jacob [:bjacob] from comment #35)
> Brad, putting this on your radar.

Thanks Benoit, putting it on your plate :-)
Comment 38 Benoit Jacob [:bjacob] (mostly away) 2013-09-18 10:14:31 PDT
Kats and me looked at this last night. We had difficulty reproducing. It doesn't seem to reproduce with mozilla-central or Nightly builds; we also didn't manage to reproduce in a debug build... so does it only happen in beta/stable non-debug builds? If we have a definite way of reproducing there, then we can definitely look into this bug, but yesterday, even there, we only reproduced this once AFAIR, out of many attempts. So at this point, the most useful thing to advance debugging this would be to get up-to-date, reliable STR.
Comment 39 Kartikaya Gupta (email:kats@mozilla.com) 2013-09-18 11:40:27 PDT
Yeah I tried bisecting and the last revision I definitely saw it on was the August 17 nightly. It might still happen on more recent builds but because I don't have reliable STR I cannot confirm that.
Comment 40 Tim Meader 2013-09-18 21:07:41 PDT
Not sure how much help I can be, but I was able to reproduce this on (the painting error, not the crash) on the latest Beta (25B1) and both the 9/18 Aurora builds and the 9/18 Nightly builds. However, the situations were slightly different between the two. For the Beta, I loaded up the tabs: giantbomb.com, comicvine.com, androidpolice.com, androidcentral.com, theverge.com and just began using it for a bit. The first time, I was able to get the paint error to appear (on the androidcentral tab) within about 5 minutes of browsing. Had to force stop the app before I could get it into a usable state again (just closing all tabs and starting new ones does nothing). The second time, it took a little longer (closer to 10 minutes), and actually generated the painting issue when I opened a new tab and typed "giant bomb" into the awesome bar. It acted as if it had returned results, but the page was blank, and the painting issue had definitely been triggered. I could confirm it by rotating the tablet to landscape, at which point the repaint caused the Google results to suddenly render, however they still did not scroll properly. Again, I had to force stop the browser to get it back to a usable state.

As for Aurora and Nightly (after completely uninstalling Beta), I tried each one at a time, and they both had the same results. I was able to consistently trigger the painting issue with both of these on the first try immediately after install (but ONLY this initial time). Each subsequent usage after that so far I've been unable to produce it on Aurora or Nightly. However, it's definitely still there on launch at least. To reproduce I simply installed each one, then launched it. The tablet was in landscape mode. Immediately after the first launch post install, I went to settings and checked the option for search suggestions. Then, went back to the home tab, and typed in a search in the awesome bar to generate a suggestion. Wait for one of those to appear, then click on the Google suggested link to go to the page. Each time, this would generate the same behavior I mentioned in the second version from the Beta experience, in that it would seem like the page was done loading according to the UI, but the content area was still completely blank. Trying to scroll up would also confirm that the painting issue is present. Likewise, rotating the tablet to portrait mode would cause the Google results to suddenly render into view.

Hopefully this is reproducible for someone. Sorry if I'm unclear.
Comment 41 jluke 2013-09-19 00:25:44 PDT
After a bunch of fiddling (and the creation of a duplicate bug report, 917091), I can now more or less reliably reproduce this on my Nexus 10 with the latest Nightly.  I used a fresh profile to make this easier.

Steps:
1. Run the "ES3 File Explorer" app (tried with Google Maps and the Mail app, couldn't reproduce with either)
2. Run Nightly
3. Tap the "customize Firefox with add-ons" bookmark (one of the default bookmarks included in a fresh profile)
4. Tap the "search for add-ons" text field
5. Switch between ES3 File Explorer and Nightly until Nightly stops rendering (I always gave up and restarted from the beginning if it took less than 10 switches, which only happened a few times in the 15 or so times I reproduced the bug.  So it shouldn't take ages to reproduce.)

Other details: device is rooted using SuperSU, but otherwise running stock firmware.  Interestingly, when I enable the 'root explorer' mode in ES3 File Explorer (requests su permissions, allowing one to browse and modify the whole file system), Nightly tends to stop rendering after the first switch (say, 9/10 times).  When I remove root privileges from ES3, it typically takes several switches.

Unfortunately, I cannot reproduce the crash I described in 917091 with this method.

Speculation: I know that there were some modifications to the Nexus 10 GPU driver for 4.3.  These fixed a rather bothersome memory leak detailed here http://code.google.com/p/android/issues/detail?id=54757 .  Now I'm not a programmer, but it seems plausible that this fix introduced a new bug that Firefox is hitting.
Comment 42 jluke 2013-09-19 00:40:18 PDT
Apologies, I should have said that I can reproduce the painting bug, not the crash.
Comment 43 Robert Kaiser (not working on stability any more) 2013-09-19 06:06:28 PDT
According to crash-stats we are seeing this crash at least in all versions up to 26, I guess comment #41 means Nightly 27 has at least the graphics issue.
I myself can reproduce the graphics thing sporadically with my Nightly but don't have any good STR and couldn't trigger it right now.
Comment 44 Benoit Jacob [:bjacob] (mostly away) 2013-09-19 14:01:20 PDT
(In reply to Tim Meader from comment #40)
> As for Aurora and Nightly (after completely uninstalling Beta), I tried each
> one at a time, and they both had the same results. I was able to
> consistently trigger the painting issue with both of these on the first try
> immediately after install (but ONLY this initial time). Each subsequent
> usage after that so far I've been unable to produce it on Aurora or Nightly.
> However, it's definitely still there on launch at least. To reproduce I
> simply installed each one, then launched it. The tablet was in landscape
> mode. Immediately after the first launch post install, I went to settings
> and checked the option for search suggestions. Then, went back to the home
> tab, and typed in a search in the awesome bar to generate a suggestion. Wait
> for one of those to appear, then click on the Google suggested link to go to
> the page. Each time, this would generate the same behavior I mentioned in
> the second version from the Beta experience, in that it would seem like the
> page was done loading according to the UI, but the content area was still
> completely blank. Trying to scroll up would also confirm that the painting
> issue is present. Likewise, rotating the tablet to portrait mode would cause
> the Google results to suddenly render into view.

This allows me to reproduce immediately with a mozilla-central debug build. Thanks! \o/

And so the problem is we're hitting this error case:

09-19 16:51:02.204 I/Gecko   ( 3864): WARNING: Framebuffer not complete -- error 0x8219, aFBOTextureTarget 0x0, aRect.width 2560, aRect.height 763: file /hack/mozilla-central/gfx/layers/opengl/CompositingRenderTargetOGL.cpp, line 40

And the abnormal piece of information here is "aFBOTextureTarget 0x0".

(more debugging to follow...)
Comment 45 Kartikaya Gupta (email:kats@mozilla.com) 2013-09-19 15:04:44 PDT
Created attachment 807457 [details]
Some debugging info
Comment 46 Nick Cameron [:nrc] 2013-09-19 15:21:59 PDT
(In reply to Benoit Jacob [:bjacob] from comment #44)
> (In reply to Tim Meader from comment #40)

> And the abnormal piece of information here is "aFBOTextureTarget 0x0".
> 

Is that abnormal? I thought it just meant use the window FBO? (Initialised here: http://dxr.mozilla.org/mozilla-central/source/gfx/layers/opengl/CompositingRenderTargetOGL.h#l86)
Comment 47 Benoit Jacob [:bjacob] (mostly away) 2013-09-19 15:25:25 PDT
(In reply to Nick Cameron [:nrc] from comment #46)
> Is that abnormal? I thought it just meant use the window FBO? (Initialised
> here:
> http://dxr.mozilla.org/mozilla-central/source/gfx/layers/opengl/
> CompositingRenderTargetOGL.h#l86)

Well, right after that, we get a FRAMEBUFFER_INCOMPLETE GL error as we try to draw to that framebuffer, and users here are reporting graphical glitches / nothing getting drawn; so I was wondering if that 0 value was the one we actually passed to framebufferTexture2D, like we do e.g. at line 65 of the same file, in CompositingRenderTargetOGL::InitializeImpl. What do you think?
Comment 48 Nick Cameron [:nrc] 2013-09-19 15:51:00 PDT
(In reply to Benoit Jacob [:bjacob] from comment #47)
> (In reply to Nick Cameron [:nrc] from comment #46)
> > Is that abnormal? I thought it just meant use the window FBO? (Initialised
> > here:
> > http://dxr.mozilla.org/mozilla-central/source/gfx/layers/opengl/
> > CompositingRenderTargetOGL.h#l86)
> 
> Well, right after that, we get a FRAMEBUFFER_INCOMPLETE GL error as we try
> to draw to that framebuffer, and users here are reporting graphical glitches
> / nothing getting drawn; so I was wondering if that 0 value was the one we
> actually passed to framebufferTexture2D, like we do e.g. at line 65 of the
> same file, in CompositingRenderTargetOGL::InitializeImpl. What do you think?

Yes, I think that 0 will get passed to framebufferTexture2D - we save it into InitParams and then use it at line 65. I'm not sure if that is correct but I think it is ok (we do it in LayerManagerOGL too).
Comment 49 Benoit Jacob [:bjacob] (mostly away) 2013-09-19 16:31:07 PDT
(In reply to Nick Cameron [:nrc] from comment #48)
> Yes, I think that 0 will get passed to framebufferTexture2D - we save it
> into InitParams and then use it at line 65. I'm not sure if that is correct
> but I think it is ok (we do it in LayerManagerOGL too).

We do some tweaks in our GL entry point wrappers in GLContext.h for framebuffer objects (so that the 'main' framebuffer is exposed as framebuffer 0 even if it is actually a FBO) but I am not aware of any of that for the 'texture target' argument to glFramebufferTexture2D, and the entry point in GLContext.h just forwards its textarget argument to the GL, and the GL spec is saying this: (GL ES 2.0.25 spec, http://www.khronos.org/registry/gles/specs/2.0/es_full_spec_2.0.25.pdf , Section 4.4 'framebuffer objects', page 113):

> If texture is not zero, then texture must either name an existing texture object with an target of textarget, or texture must name an existing cube map texture [...] Otherwise, INVALID_OPERATION is generated.

We didn't hit an INVALID_OPERATION though... we only got a GL error as we try to draw to an incomplete framebuffer. More debugging needed...
Comment 50 Kartikaya Gupta (email:kats@mozilla.com) 2013-09-19 19:53:28 PDT
FWIW it was the zero value from http://dxr.mozilla.org/mozilla-central/source/gfx/layers/opengl/CompositingRenderTargetOGL.h#l88 that was getting used (I had changed it to 0xDEAD1 and that's what I saw in the GL error message:

WARNING: Framebuffer not complete -- error 0x8219, aFBOTextureTarget 0xdead1, aRect.width 2560, aRect.height 1454: file /Users/kats/zspace/mozilla-git/gfx/layers/opengl/CompositingRenderTargetOG...
Comment 51 Ben 2013-09-20 09:13:57 PDT
Hi. I do have the painting problem very often and really bothers me. Device is unrooted Nexus 10. BTW I have very few crashes.
I have the strong feeling that it started before upgrading to 4.3 and maybe between FF23 and 24b1 but I cannot confirm it.
I searched for a good STR but did not manage to find a precise one. Once the bug happens I can redraw by selecting "new tab" and then hiding the virtual keyboard.
I am not developping on Android but contact me if you need more information or to check a fix.
Comment 52 Benoit Jacob [:bjacob] (mostly away) 2013-09-20 15:11:14 PDT
An update on the current status of debuging.

The code returned by CheckFramebufferStatus, 0x8219, is FRAMEBUFFER_UNDEFINED. That doesn't exist in OpenGL ES 2.0, only in OpenGL ES 3.0, giving us a clue as to how this can be specific to Android 4.3.

I still don't understand everything here, but here is roughly what is going on. The framebuffer that is incomplete here, is the primary framebuffer that we composite to --- it's a 'CreateForWindow' GL Context, not an offscreen one, and it's the default framebuffer of that context.

The context is initially good (many frames get composited successfully) and eventually starts giving this error. The EGLSurface that we're making it current against is always the same.

My best hypothesis at this point is that that EGLSurface becomes bad, which is a common thing to happen and would fit well with the STR consisting in going to search and then to search results, or with STR consisting in changing the device's orientation.

So the next thing to try is to recreate our EGLSurface at the right time. Not sure yet what 'the right time' is, but since we're hitting the above-mentioned NS_WARNING about incomplete framebuffer, that is a good starting point to recreate the EGLSurface.

Stay tuned...
Comment 53 Benoit Jacob [:bjacob] (mostly away) 2013-09-21 12:36:46 PDT
Created attachment 808210 [details] [diff] [review]
(The actual fix) Renew the surface when we hit an incomplete default framebuffer in the compositor

This works here.

I hesitated between this approach and doing the same in GLContextEGL::MakeCurrentImpl() when we hit EGL_BAD_SURFACE, but somehow the present patch is the only thing that actually worked.

This patch also only warns when this attempt was unfruitful, and then it uses printf_stderr instead of NS_WARNING so that (IIUC) it will also print in release builds, so that any future such error is understood quicker.
Comment 54 Benoit Jacob [:bjacob] (mostly away) 2013-09-21 12:38:39 PDT
Comment on attachment 808210 [details] [diff] [review]
(The actual fix) Renew the surface when we hit an incomplete default framebuffer in the compositor

Whoever, of Nick or Jeff, does it first.

Also, this patch expands the logging in the error case.
Comment 55 Benoit Jacob [:bjacob] (mostly away) 2013-09-21 12:41:08 PDT
A question for Jeff:

While this patch works, I don't currently understand exactly why it works. Indeed, calling RenewSurface seems to only change the GLContext's mSurface, not the mCurSurface, but the surface actually passed to eglMakeCurrent in MakeCurrentImpl is mCurSurface, not mSurface. The only place that I can see that syncs mCurSurface with mSurface is MakeCurrent_EGLSurface, but that only seems to be called in SharedSurfaceANGLE. So, how is this working?
Comment 56 Benoit Jacob [:bjacob] (mostly away) 2013-09-21 12:48:02 PDT
Try push:

https://tbpl.mozilla.org/?tree=Try&rev=24026f72a96c

If anyone is interested in testing builds early, they will be available in a few hours there:

http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/bjacob@mozilla.com-24026f72a96c
Comment 57 Benoit Jacob [:bjacob] (mostly away) 2013-09-21 13:12:32 PDT
Created attachment 808213 [details] [diff] [review]
(Additional patch) In BindRenderTarget, after initialization, don't just return

Nick, looking at this BindRenderTarget function, its logic looks dubious. It's like:

  if (not already initialized)
  {
    inititialize
  }
  else
  {
    do the actual work
  }

Shouldn't it be instead

  if (not already initialized)
  {
    inititialize
  }
  do the actual work

?

This patch does that. It seems to work locally (but then again, things also work locally without it, so I don't know if it actually makes any difference). It just feels like the right thing to do, no? No reason why we shouldn't actually do the bindrendertarget work on the first time right after initialization?

Try push with both patches:
https://tbpl.mozilla.org/?tree=Try&rev=81c4fb50bdf0
Comment 58 Nick Cameron [:nrc] 2013-09-21 15:17:26 PDT
Comment on attachment 808213 [details] [diff] [review]
(Additional patch) In BindRenderTarget, after initialization, don't just return

Review of attachment 808213 [details] [diff] [review]:
-----------------------------------------------------------------

The 'actual work' is done in InitializeImpl (as well as in the 'else' branch), so now you are doing the work twice. I don't remember exactly why it is setup this way, your proposal certainly sounds more natural (although I don't think there is a bug here). And I don't see InitializsImpl called from anywhere else, so you could do this, but you need to refactor InitializeImpl as well.
Comment 59 Nick Cameron [:nrc] 2013-09-21 15:21:00 PDT
Comment on attachment 808210 [details] [diff] [review]
(The actual fix) Renew the surface when we hit an incomplete default framebuffer in the compositor

Review of attachment 808210 [details] [diff] [review]:
-----------------------------------------------------------------

r=me with the printf fixed

::: gfx/layers/opengl/CompositingRenderTargetOGL.cpp
@@ +33,5 @@
>      MOZ_ASSERT(mInitParams.mStatus == InitParams::INITIALIZED);
>      mGL->fBindFramebuffer(LOCAL_GL_FRAMEBUFFER, mFBO);
>      GLenum result = mGL->fCheckFramebufferStatus(LOCAL_GL_FRAMEBUFFER);
>      if (result != LOCAL_GL_FRAMEBUFFER_COMPLETE) {
> +      if (mFBO == 0) {

why only do this for the screen FBO? Can we renew other surfaces too?

@@ +38,5 @@
> +        mGL->RenewSurface();
> +        result = mGL->fCheckFramebufferStatus(LOCAL_GL_FRAMEBUFFER);
> +      }
> +      if (result != LOCAL_GL_FRAMEBUFFER_COMPLETE) {
> +        printf_stderr("Framebuffer not complete -- CheckFramebufferStatus returned 0x%x, "

revert to NS_WARNING rather than printf
Comment 60 Ben 2013-09-22 02:54:54 PDT
(In reply to Benoit Jacob [:bjacob] from comment #56)
> If anyone is interested in testing builds early, they will be available in a
> few hours there:
> 
> http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/bjacob@mozilla.com-
> 24026f72a96c

Hi, I tried this on my Nexus 10 (Android 4.3) on  and did not encounter the painting problem anymore when changing orientation or using input fields with virtual keyboard.
Comment 62 Benoit Jacob [:bjacob] (mostly away) 2013-09-22 09:21:26 PDT
(In reply to Ben from comment #61)
> But... it crashed once:
> 
> https://crash-stats.mozilla.com/report/index/e8a9c8ee-d59d-4d6c-ab5f-
> 2d1b02130922

Interesting: that is the crash originally reported here, which is apparently a completely different issue not related to graphics at all, before the discussion above drifted towards graphics issues.

It seems that more than one (perhaps as many as 3) different bugs are being discussed here. My patch fixes one graphical issue, which was causing graphics-related warnings to show up in |adb logcat|. I don't claim that it will fix everything :)

To do: check if that patch, applied to the beta channel, fixes the crashes reported there.
Comment 63 Benoit Jacob [:bjacob] (mostly away) 2013-09-22 09:26:47 PDT
(In reply to Nick Cameron [:nrc] from comment #59)
> why only do this for the screen FBO? Can we renew other surfaces too?

Other surfaces are non-default framebuffer objects, so they must be backed by textures or renderbuffers, not by EGLSurface's, so this doesn't apply to them.

> 
> @@ +38,5 @@
> > +        mGL->RenewSurface();
> > +        result = mGL->fCheckFramebufferStatus(LOCAL_GL_FRAMEBUFFER);
> > +      }
> > +      if (result != LOCAL_GL_FRAMEBUFFER_COMPLETE) {
> > +        printf_stderr("Framebuffer not complete -- CheckFramebufferStatus returned 0x%x, "
> 
> revert to NS_WARNING rather than printf

How much do you insist on that? I was saying that now that we only get there after we tried renewing the surface, this should really not happen anymore, and if it does, that is really worth logging even in release builds. So the NS_WARNING -> printf_stderr change was intentional. Is that not the right way to do that, or do you disagree with this being worth logging in release builds?
Comment 64 Benoit Jacob [:bjacob] (mostly away) 2013-09-22 13:55:07 PDT
I've now tried on beta, following these STR:

(In reply to Tim Meader from comment #40)
> Not sure how much help I can be, but I was able to reproduce this on (the
> painting error, not the crash) on the latest Beta (25B1) and both the 9/18
> Aurora builds and the 9/18 Nightly builds. However, the situations were
> slightly different between the two. For the Beta, I loaded up the tabs:
> giantbomb.com, comicvine.com, androidpolice.com, androidcentral.com,
> theverge.com and just began using it for a bit. The first time, I was able
> to get the paint error to appear (on the androidcentral tab) within about 5
> minutes of browsing. Had to force stop the app before I could get it into a
> usable state again (just closing all tabs and starting new ones does
> nothing). The second time, it took a little longer (closer to 10 minutes),
> and actually generated the painting issue when I opened a new tab and typed
> "giant bomb" into the awesome bar. It acted as if it had returned results,
> but the page was blank, and the painting issue had definitely been
> triggered. I could confirm it by rotating the tablet to landscape, at which
> point the repaint caused the Google results to suddenly render, however they
> still did not scroll properly. Again, I had to force stop the browser to get
> it back to a usable state.

I did reproduce some repainting unreliablity on beta, with these sites open in different tabs, switching between tabs and changing orientation. After a few tries, some site wouldn't correctly repaint after some combination of tab switching and orientation change.

I rebased my patch against mozilla-beta and with it, I can't reproduce these issues anymore, so I suppose that it's useful there too.
Comment 65 Benoit Jacob [:bjacob] (mostly away) 2013-09-22 14:02:02 PDT
Created attachment 808322 [details] [diff] [review]
Patch backported to mozilla-beta

Results will be displayed on TBPL as they come in:
https://tbpl.mozilla.org/?tree=Try&rev=bffd76624833

Once completed, builds and logs will be available at:
http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/bjacob@mozilla.com-bffd76624833
Comment 66 Nick Cameron [:nrc] 2013-09-22 15:05:40 PDT
(In reply to Benoit Jacob [:bjacob] from comment #63)
> (In reply to Nick Cameron [:nrc] from comment #59)
> > why only do this for the screen FBO? Can we renew other surfaces too?
> 
> Other surfaces are non-default framebuffer objects, so they must be backed
> by textures or renderbuffers, not by EGLSurface's, so this doesn't apply to
> them.
> 

Makes sense. Could you add this as a comment please?

> > 
> > @@ +38,5 @@
> > > +        mGL->RenewSurface();
> > > +        result = mGL->fCheckFramebufferStatus(LOCAL_GL_FRAMEBUFFER);
> > > +      }
> > > +      if (result != LOCAL_GL_FRAMEBUFFER_COMPLETE) {
> > > +        printf_stderr("Framebuffer not complete -- CheckFramebufferStatus returned 0x%x, "
> > 
> > revert to NS_WARNING rather than printf
> 
> How much do you insist on that? I was saying that now that we only get there
> after we tried renewing the surface, this should really not happen anymore,
> and if it does, that is really worth logging even in release builds. So the
> NS_WARNING -> printf_stderr change was intentional. Is that not the right
> way to do that, or do you disagree with this being worth logging in release
> builds?

Well, if it is deliberate, then I guess I am not strongly opposed. But is printf_stderr the right way to do it? Do we have a macro for doing this? (printfs won't be highlighted in tbpl logs, for one thing). Do we have guidelines for what we print in release builds? Our release builds are not chatty, so I guess there is a high bar for what we spew. You probably ought to check these things before landing, but modulo that I am OK with it.
Comment 67 Robert Kaiser (not working on stability any more) 2013-09-23 05:24:32 PDT
If the graphics issues and the crash are separate, we should do separate bugs for them - possibly cloning the crash into a fresh bug, given that the graphics issue work is going on here.

I had bp-e9e677be-cd1a-483f-9b61-4738b2130922 yesterday but I don't have good STR - it happened after the graphics issue had appeared, I think when I tried to close Nightly.
Comment 68 Robert Kaiser (not working on stability any more) 2013-09-23 13:27:14 PDT
Today I had bp-12faa518-c604-45a3-9105-48d6d2130923 - after the graphics issue appeared late yesterday, I did let Nightly just sit and put the tablet into standby, and when I woke it up today, the crash reporter was waiting for me.
Comment 69 Jeff Gilbert [:jgilbert] 2013-09-23 13:48:31 PDT
(In reply to Benoit Jacob [:bjacob] from comment #55)
> A question for Jeff:
> 
> While this patch works, I don't currently understand exactly why it works.
> Indeed, calling RenewSurface seems to only change the GLContext's mSurface,
> not the mCurSurface, but the surface actually passed to eglMakeCurrent in
> MakeCurrentImpl is mCurSurface, not mSurface. The only place that I can see
> that syncs mCurSurface with mSurface is MakeCurrent_EGLSurface, but that
> only seems to be called in SharedSurfaceANGLE. So, how is this working?

We have to be able to switch out backing EGLSurfaces for the ShSurfANGLE path. (And later for the EGLStream path)
I wrote that assuming that except in this case, the surface never changes. Just have RenewSurface update mCurSurface (since we MakeCurrent against mCurSurface) and everything should work. We should assert that GLContext->Screen() is null when this happens, though.
Comment 70 Tim Meader 2013-09-23 14:08:24 PDT
I've been using the beta version of the try build since its availability yesterday, and thus far haven't been able to get it to display the corruption issue again. Haven't experienced the crash either (though I never experienced that prior). Do you think there's a good chance this could get back-ported to the beta channel? Would be nice to be able to use beta for stability's sake, but if not, I can wait till the next merge if I have to. Just glad to see I'll be able to use Firefox on the Nexus 10 again in the near future :)
Comment 71 Benoit Jacob [:bjacob] (mostly away) 2013-09-23 14:10:57 PDT
(In reply to Jeff Gilbert [:jgilbert] from comment #69)
> I wrote that assuming that except in this case, the surface never changes.

Ouch. The surface changes very frequently on Android (on orientation changes, on window de-focusing)

> Just have RenewSurface update mCurSurface (since we MakeCurrent against
> mCurSurface) and everything should work.

Will put up a patch.

I still don't know why it's working at the moment (with the existing patch on this bug)!
Comment 72 Jeff Gilbert [:jgilbert] 2013-09-23 15:58:27 PDT
For the long term, unless there's a reason to keep mSurface and mCurSurface separate, we should merge them.

If they need to be kept separate, we should formalize the difference between mBackingSurface (the surface considered to be the normal default framebuffer) and mCurSurface. (which is basically just an override)
Comment 73 jluke 2013-09-23 23:57:30 PDT
One peculiar thing I have noticed is that when the crash reporter appears as a result of the crash bug, it occasionally messes with Android's "recent apps" list.  E.g., upon starting Nightly again and pressing the recent apps button, Nightly will be called "Crash Reporter" and have the wrong icon (but it really is Nightly).  Or Nightly won't appear at all in the list of recent apps.  Or it will appear the first time I press the recent apps button, but not appear the second time.  Whether this is related to this bug in particular I do not know, though given that the painting bug can be reproduced by switching back and forth between apps, it seems plausible.

Hopefully someone else has had a similar experience.
Comment 74 Benoit Jacob [:bjacob] (mostly away) 2013-09-24 04:57:11 PDT
Landed with Nick's review:

https://hg.mozilla.org/integration/mozilla-inbound/rev/5815d18461a2

Again, this fixes only the graphical issues reported here, not the crash (which is not graphics-related for all we know).

(In reply to Nick Cameron [:nrc] from comment #66)
> (In reply to Benoit Jacob [:bjacob] from comment #63)
> > (In reply to Nick Cameron [:nrc] from comment #59)
> > > why only do this for the screen FBO? Can we renew other surfaces too?
> > 
> > Other surfaces are non-default framebuffer objects, so they must be backed
> > by textures or renderbuffers, not by EGLSurface's, so this doesn't apply to
> > them.
> > 
> 
> Makes sense. Could you add this as a comment please?

Yes. I also changed it (hope that's OK) to only do this in the non-IsOffscreen() case, as it's the case we're in here and RenewSurface() assumes it.

> Well, if it is deliberate, then I guess I am not strongly opposed. But is
> printf_stderr the right way to do it?

Reverted to NS_WARNING, easier landing.
Comment 75 Benoit Jacob [:bjacob] (mostly away) 2013-09-24 04:58:48 PDT
Comment on attachment 808210 [details] [diff] [review]
(The actual fix) Renew the surface when we hit an incomplete default framebuffer in the compositor

Assuming that Nick's review was enough and that having already looked at this patch, you didn't see something wrong with it.
Comment 76 Benoit Jacob [:bjacob] (mostly away) 2013-09-24 05:03:27 PDT
Comment on attachment 808322 [details] [diff] [review]
Patch backported to mozilla-beta

[Approval Request Comment]
Bug caused by (feature/regressing bug #): Likely not a regression, but rather, we never correctly supported Android 4.3.
User impact if declined: Severe graphical glitches (pages not rendering at all in some circumstances, browser not easily recovering to normal state) on Android 4.3
Testing completed (on m-c, etc.): m-i
Risk to taking this patch (and alternatives if risky): Feels pretty low-risk to me. What this patch does is in a particular error case, it asks for a new graphical surface to render to, and operation that we do often anyways, and retries once. Given we're in early beta, I hope that this can be accepted there.
String or IDL/UUID changes made by this patch: none
Comment 77 Benoit Jacob [:bjacob] (mostly away) 2013-09-24 05:13:25 PDT
(In reply to Jeff Gilbert [:jgilbert] from comment #72)
> For the long term, unless there's a reason to keep mSurface and mCurSurface
> separate, we should merge them.
> 
> If they need to be kept separate, we should formalize the difference between
> mBackingSurface (the surface considered to be the normal default
> framebuffer) and mCurSurface. (which is basically just an override)

Not understanding this fully myself, I filed a follow-up bug 919987 and needinfo'd you on it.
Comment 78 Benoit Jacob [:bjacob] (mostly away) 2013-09-24 05:17:15 PDT
Kats, Brad: the graphical issues here are taken care of by the patch just landed on inbound and for which I just requested aurora/beta approval, but the rest of this bug (the libutils.so crash) seems unrelated, therefore this bug should stay open for now. Making sure that you understand this :-) Putting [leave open] for now, but a better idea might be this:

(In reply to Robert Kaiser (:kairo@mozilla.com) from comment #67)
> If the graphics issues and the crash are separate, we should do separate
> bugs for them - possibly cloning the crash into a fresh bug, given that the
> graphics issue work is going on here.

That sounds like a good idea to me.
Comment 79 Benoit Jacob [:bjacob] (mostly away) 2013-09-24 05:23:02 PDT
Some bustage due to the quick printf_stderr -> NS_WARNING change, should be fixed in
https://hg.mozilla.org/integration/mozilla-inbound/rev/3da78715a53b
Comment 80 Kartikaya Gupta (email:kats@mozilla.com) 2013-09-24 06:19:51 PDT
I have filed bug 920006 for the crash. We can close this one out as having fixed the graphical glitches.
Comment 81 Brad Lassey [:blassey] (use needinfo?) 2013-09-24 08:34:50 PDT
yea, let's close this out and track the crash in a different bug.
Comment 83 Lukas Blakk [:lsblakk] use ?needinfo 2013-09-25 12:15:17 PDT
Comment on attachment 808322 [details] [diff] [review]
Patch backported to mozilla-beta

Approving uplift for graphical glitches, will make sure the crash bug is tracked as well.
Comment 85 Brad Lassey [:blassey] (use needinfo?) 2013-09-26 10:37:10 PDT
*** Bug 917589 has been marked as a duplicate of this bug. ***

Note You need to log in before you can comment on or make changes to this bug.