Closed Bug 521266 Opened 16 years ago Closed 16 years ago

Using OpenGL ES for drawing on Android

Categories

(Core :: Graphics, defect)

Product:
Core
Component:
Graphics
Platform:
x86
Android
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED

People

(Reporter: joelr, Unassigned)

References

Details

How do we tackle drawing with OpenGL ES on Android? Do we make Cairo render to OpenGL surfaces or skip Cairo entirely?
Summary: Using OpenGL ES for drawing → Using OpenGL ES for drawing on Android
Component: General → Graphics
Product: Firefox → Core
QA Contact: general → thebes
Blocks: 521280
My understanding is that we already have an OpenGL backend for Cairo. If that's the case, what would we need to do to change to OpenGL ES and use the one we have?
cairo-opengl is entirely fixed-function, which could work if we're targetting OpenGL ES 1.1. If we're targetting ES 2.0, it would have to be substantially rewritten. It's also worth noting that the current Cairo OpenGL backends aren't substantially faster, overall, than just using raw pixman. If the only rendering method for Android is using OpenGL ES, a reasonable method of going about getting a quick result would be to render 100% using image surfaces and then just upload those images to a texture, and in the native nsWindow implementation, draw a textured quad. (Note that uploading an image for the whole screen every draw can be quite slow, so some optimization will be necessary to get acceptable performance.)
Android Donut is OpenGL ES 1.1. My drawing notes are here: https://wiki.mozilla.org/User:Joel_Reymont/Android_Notes#Drawing_and_input_from_native_code
According to my notes, we can create and update a screen-size texture and this should be quick to render. Do we really need the image to texture step? My OpenGL is still flaky but I'm catching up!
If the only way to draw to the screen is through OpenGL, then we will have to draw to an image and upload it as a texture. We have no native OpenGL backend (yet).
Joe, If the current Cairo OpenGL backends aren't substantially faster than just using raw pixman but uploading an image for the whole screen on every draw is quite slow, then wouldn't we gain by using the Cairo OpenGL backend directly? I figure that it will be just as fast as pixman and we won't have the image upload issue. What do you think?
I updated my notes with instructions on how to build the NDK OpenGL ES sample: https://wiki.mozilla.org/User:Joel_Reymont/Android_Notes#Building_and_running_the_OpenGL_ES_NDK_example
Where is the Cairo OpenGL ES backend in the mozilla-central tree? I looked under gfx/cairo/cairo/src but there are no files with GL in them unlike quartz, pdf, svg, win32 and xlib.
We currently have a OpenGL ES 2.0 (with some Nvidia extensions) backend that we use on wince. It's currently also tied to a DirectDraw interface and lives in cairo-ddraw-surface.c. Nvidia is working on porting this to linux, so the DirectDraw dependency should be reduced in the future. Using this backend is faster than the DirectDraw backend without OpenGL acceleration. Upstream cairo also has a fixed-function OpenGL backend that shouldn't be too difficult to port to OpenGL ES 1.1. This backend is not yet in the Mozilla tree. Also, it's worth noting that mobile devices usually have a shared memory architecture so the cost of uploading shouldn't be too high.
I've suggested that the best way forward here is to use the offscreen branch that Chris Lord is working on. That should be a simple way to get things started.
From the FOSDEM 09 presentation by Chris Lord [1]: "Started to talk about what's yet to be done; what I like to call 'native' scrolling isn't possible in Mozilla yet. Ideally, you'd be able to tell Mozilla 'render as if you were size 'X'', then you could render an arbitrarily large buffer once, then scroll around inside this buffer instead of re-rendering on scrolling. This is what we've implemented in webkit-clutter (the WebKit version of ClutterMozembed/mozilla-headless) and probably what the iPhone and the Wii Opera browser do too. If this is possible, please contact me!" It seems like another drawback to drawing to an image (pixman?) + uploading to texture rather than using the Cairo OpenGL ES backend. Am I right? [1] http://chrislord.net/files/fosdem-09-slides.odp
(In reply to comment #12) > It seems like another drawback to drawing to an image (pixman?) + uploading to > texture rather than using the Cairo OpenGL ES backend. Am I right? Perhaps, but it seems like an issue that we should be able to work around pretty easily. However I don't know any of the specifics, so I maybe missing something.
How would you work around this issue with pixman?
I don't believe this is an issue with pixman. I believe the problem is with the api we expose to for drawing to a surface. Though, it's hard to really say anything useful without more detail on what the problem actually is.
The iPhone uses a tile cache, as does fennec. We already have the ability to draw more than what's displayed, just not in the Firefox frontend.
Joe, great to hear that! I'm currently trying to figure out what's needed to mend the offscreen incubator branch into something suitable for Android. Basically, reviewing the changes Chris had to make for Moblin.
Just to echo what others have mostly said -- (In reply to comment #0) > How do we tackle drawing with OpenGL ES on Android? Ignore it, basically; we should render using pure software rendering in Cairo (image surfaces + freetype), and then blast that to the screen using Skia (or GL, depending on which is faster; likely Skia) on the Android side. Don't look at offscreen or anything of that nature. That's not a useful path forward to getting this stuff done quickly. Once we actually have a pure GL backend we can experiment with it, but none of that/offscreen/etc. is blocking getting things running (or even interesting for now).
1) Skia is not exposed via the NDK: http://groups.google.com/group/android-ndk/browse_thread/thread/e3e2ea15251a1c0e 2) The event loop of a regular Firefox build uses pull to get native events, if I recall correctly. The Android event loop is in Java so events need to be "pushed" to Firefox. I don't know how to do that yet. 3) I wish Vlad had chimed in earlier as I "wasted" a few days with the offscreen branch already.
Vlad is just reiterating what we said earlier, which is that you should be drawing using pixman and using GL to upload those images. The offscreen branch uses pixman to draw, but *also* obviates the need for a toolkit, which you don't have, since you're on Android.
(In reply to comment #19) > 1) Skia is not exposed via the NDK: > > http://groups.google.com/group/android-ndk/browse_thread/thread/e3e2ea15251a1c0e That thread actually seems to offer some promising suggestions. In any case, we might as well pretend Skia's exposed for comparison purposes. If it's a lot faster then we could advocate for freezing that API, maybe.
(In reply to comment #18) > Just to echo what others have mostly said -- > > Don't look at offscreen or anything of that nature. That's not a useful path > forward to getting this stuff done quickly. Once we actually have a pure GL > backend we can experiment with it, but none of that/offscreen/etc. is blocking > getting things running (or even interesting for now). I don't understand why offscreen is so bad? What will you use in lieu offscreen. Don't you need a widget implementation of sorts?
I filed bug 522085 for the widget implementation.
The implementation of this landed with bug 564327.
Status: NEW → RESOLVED
Closed: 16 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.