170553 - (opengl) Render using OpenGL

Status: RESOLVED INCOMPLETE

(Core :: Graphics, enhancement, P3)

Description

[Brian 'netdragon' Bober]

Using directx and/or opengl would make rendering faster on all the oses. We 
could make it option-based so that if someone doesn't have the ability to run 
directx or opengl they could use plain GDI (or linux equiv) rendering.

This is obviously something that is for the way future.

Comment 1

[Boris Zbarsky [:bzbarsky]]

I fail to see how rendering using directX or openGL will be that much faster for
2-d rendering (which is what we are doing).

Comment 2

[Hixie (not reading bugmail)]

Make it so.

Comment 3

[Robert O'Callahan (:roc) (email my personal email if necessary)]

There's no need to use DirectX. OpenGL runs fine on Windows, AND it's cross
platform.

Things that would work better in OpenGL:
alpha compositing
scaling (bilinear filtering!)
tiling
any kind of image drawing, actually (store images in texture memory)
anti-aliased anything

Well, that list pretty much sums up everything that's slow about our current
drawing code :-).

Comment 4

[ArronM (:paper)]

In theory, video cards are faster at 2D rendering using OpenGL/DirectX because
that has been the "push", the "in thing", the "fad", the "what all perf
comparison charts use to say who's hot".  So the video card manufacturers have
been pouring many manhours into developing the fastest 3D out there, usually at
the expense of speeding up their old 2D routines.

All the cool routines are in 3D mode, even the 2D ones ;)

Comment 5

[Brian 'netdragon' Bober]

Also GDI (win32) is made for basic rendering of widgets and such, not rich 
multimedia rendering as is now required by browsers. It says that if you do 
any kind of sophisticated multimedia, you should use DirectX (or opengl). 
Mozilla isn't a text-mode application. ;-)

Comment 6

[Adam D. Moss]

I love OpenGL.  However:

Things that would likely work worse in OpenGL:
. Text rendering (uh-oh)
. Drawing to an offscreen image (double-buffering -- when
  you swap buffers in OpenGL the new backbuffer's contents
  become undefined.  Single-buffering would give flicker.
  Hardware-rendering to an offscreen pixmap is nonstandard.)
. Screen-screen copies (scrolling!)

I think that an OpenGL front-end would be groovy and IS doable,
but there are some real potential design/performance/interoperability
pitfalls to be wary of.

Comment 7

[Ed Grochowski]

Actually on Windows, I think the simplest change which would help all the issues
decribed in comment #3 would be to use GDI+

Having said that, I dont have benchmarks to know how much performance would
benefit (other than MS docs which say "programmers of new applications should
use GDI+ for all their graphics needs because GDI+ optimizes many of the
capabilities of GDI and also provides additional features"). At least it has
many of the algorithms and features Mozilla needs.

Of course, this doesnt help cross-platform performance or needs.

Comment 8

[Brian 'netdragon' Bober]

I'd like to mention that its not a good idea to start saying, "We'll fix this 
with the OpenGL fix" for bugs like bug 98971 - on bilinear image scaling. This 
bug isn't a small fix and will require a lot of testing. That's why I'm adding 
the keyword "highrisk"

highrisk: Bugs whose fixes have a high level of risk associated with them, 
typically due to some major architectural changes being required (see 
the 'arch' keyword). Typically these are bugs that were deferred from the 
previous release cycle due to the high level of risk involved, and should 
therefore be fixed as soon as possible in the next cycle. Examples: "need to 
rewrite handling of HTTP headers" or "need to redesign the content sinks".

If believe this bug could be considered a major architectural change since it 
will change the way in which graphics are rendered throughout mozilla.

Comment 9

[ArronM (:paper)]

GDIPlus is probably not an option as it requires a download from Microsoft if
you are on Win95/98/ME/2k.  I believe WinXP has it.  Some documentation says 2k
has it, but it wasn't on my 2k machine.

Unless of course, it was a build option.

Comment 10

[Christian :Biesinger (don't email me, ping me on IRC)]

paper: couldn't moz ship with the gdi+ dll?

Comment 11

[Robert O'Callahan (:roc) (email my personal email if necessary)]

> Things that would likely work worse in OpenGL:
> . Text rendering (uh-oh)

We would rasterize glyphs ourselves and upload as textures. This is basically
what our existing Freetype code already does on X (both the bstell code and the
Xft code) --- except X is slower, of course.

> . Drawing to an offscreen image (double-buffering -- when
>   you swap buffers in OpenGL the new backbuffer's contents
>   become undefined.

Not a problem, we always throw the old buffer away anyway.

> Single-buffering would give flicker.
>   Hardware-rendering to an offscreen pixmap is nonstandard.)

My OpenGL books are at home but I think they describe rendering to an offscreen
pixmap to do lots of tricks. Anyway the real issue is not whether it's standard,
but whether existing cards let us do it, and I'm pretty sure they do.

> . Screen-screen copies (scrolling!)

If OpenGL rendering is fast enough, we would just rerender the entire scrolled
area. We already do this for certain complex pages (e.g., pages with
background-attachment:fixed).

------------------------------------

I totally agree with Brian that OpenGL is not a solution to any short-term
problems. We need to keep improving our existing GFX implementations.

------------------------------------

We could detect at runtime if GDI+ is available and use it if it is.

Comment 12

[Robert O'Callahan (:roc) (email my personal email if necessary)]

BTW Adam, rendering to offscreen pixmaps is necessary to do CSS/SVG transparency
properly as well as for double buffering, and one of my main motivations to use
OpenGL would be to speed up the rendering of exactly that sort of effect. So if
offscreen OpenGL rendering doesn't work well in common cases (say Linux+NVidia)
then I'm suddenly not interested in it anymore.

Comment 13

[Adam D. Moss]

You're fine with the nVidia drivers on linux and windows.

I think the recommended way to do this on windows is with
the WGL_ARB_pbuffer extension.  There's an equivilent (but
different) extension offered by recent nV drivers on both
platforms but the name escapes me.

Comment 14

[Robert O'Callahan (:roc) (email my personal email if necessary)]

One thing I should mention here is that on Linux, OpenGL with Direct Rendering
is a much faster path to the graphics hardware than the X protocol and the X
server. If there's a win here, it's probably bigger on Linux than anywhere else.

Comment 15

[nemo]

I'd like to add my vote for OpenGl for precisely the reason mentioned by Mr.
O'Callahan.

I started out by asking about Gtk/Qt use of DRI, and when I learned that only
OpenGL used DRI at the moment, searched for this bug.

It'd be nice to be able to scroll over a fixed background without stutteriness.
 Or move a sidebar menu without it jumping around.

Mozilla under linux is a bit of an embarassment in terms of responsiveness. 
People are quick to blame the mozilla code, but I bet it is just the windows
advantage of direct access to graphics card.

*note, I am in *no* way a graphics programmer or have any familiarity with it,
all the above is just the rantings of a user*  :)

Comment 16

[Adam]

Anything ever come of this?

Comment 17

[Adam D. Moss]

I think the idea is to switch to Cairo for (most?) rendering, and that gives
gecko a pretty good OpenGL back-end for free.

There don't seem to be any bugs in bugzilla about the Cairo support though,
which is mysterious.   I guess if it's happening then it's happening offline.

Comment 18

[Robert O'Callahan (:roc) (email my personal email if necessary)]

The Cairo work is happening very slowly right now. It will speed up a lot once I
start working on it in January.

Comment 19

[Robert O'Callahan (:roc) (email my personal email if necessary)]

The overall plan here is to get Mozilla working using Cairo for all rendering on
all platforms. Then users with good OpenGL support can use the "Glitz" Cairo
backend to get hardware acceleration. This will require us to do some work on
Cairo itself to make it fast enough, but at least we won't be doing the graphics
layer alone anymore.

Comment 20

[u251854]

Does this bug still apply or can it be closed since we're using Cairo?

Comment 21

[Joe Drew (not getting mail)]

Attachment: braindead opengl on OS X

Here is a patch that Vlad gave me, and I fixed up a little, that takes the output of Firefox's rendering on OS X and, on every draw, creates a texture, uploads it, and draws a textured quad.

Let me be clear: this is proof-of-concept only. It implements hardware deceleration. It is not the way we want to go. But it's at least a starting point.

Comment 22

[Guilherme Lima]

This should be duped to something??

Comment 23

[JP Rosevear [:jpr]]

We have OpenGL layers now, implemented via other bugs.