546837 - [OS X] Smoothly animate native widgets even when the main thread is blocked

Status: NEW

(Core :: Widget: Cocoa, defect, P3)

Description

[Alex Faaborg [:faaborg] (Firefox UX)]

This is the OS X equivalent to bug 392644 on Windows.  A number of native OS X widgets animate, including pulsing default dialog button, the pulsing progress bar, etc.  In Firefox we currently use a javascript timer for some of these, but that is really a hack.  This bug is to track support of native widget animation on OS X.

Comment 1

[Markus Stange [:mstange]]

(In reply to comment #0)
> This is the OS X equivalent to bug 392644 on Windows.  A number of native OS X
> widgets animate, including pulsing default dialog button, the pulsing progress
> bar, etc.

What is "etc"? I'm only aware of those two.

> In Firefox we currently use a javascript timer for some of these,
> but that is really a hack. This bug is to track support of native widget
> animation on OS X.

So this bug is really about changing the implementation not to use JavaScript for the animation? There's no such thing as "native" animation - we just have to trigger repaints at the right time, and whether those are triggered via JavaScript or something else doesn't really matter.

Comment 2

[Alex Faaborg [:faaborg] (Firefox UX)]

>What is "etc"? I'm only aware of those two.

Was filing quickly and figured I missed some.  Aside from variations of the progress bar (indeterminate, normal), I guess those are in fact the only two that have any amount of animation.

>So this bug is really about changing the implementation not to use JavaScript
>for the animation? There's no such thing as "native" animation - we just have
>to trigger repaints at the right time, and whether those are triggered via
>JavaScript or something else doesn't really matter.

Sorry, meant [firefox's animation] of [native widgets].  We need to drastically increase the frame rate of the animation in order to be imperceptibly non-native.

My impression, although I could be entirely wrong, is that instead of a top down approach where javascript forcibly animates these widgets, we would want to go with more of a bottom up approach, where the widgets can call their own repaints, and we are able to refresh the widgets in a way that doesn't block other javascript code from executing, and doesn't look like Firefox is struggling to draw a button or progress bar.

Comment 3

[Markus Stange [:mstange]]

I read up on how Cocoa does these things for native NSProgressIndicators: they use a separate so-called heartbeat thread which repeatedly redraws the control by calling lockFocusIfCanDraw, drawRect, unlockFocus. Doing that kind of multithreaded drawing in Gecko isn't possible currently.

Roc, how will that change when we have a retained scene graph and off-the-main-thread animation? Will those animations be restricted to basic layer attributes like transform and opacity, or will it be possible to draw into them without waiting for the main thread to become available?

Comment 4

[Robert O'Callahan (:roc) (email my personal email if necessary)]

We could definitely make it work once we have off-main-thread layer rendering. We should probably create a new layer type to represent animated theme backgrounds and create a thread-safe version of nsITheme::DrawWidgetBackground. This would be a very nice way to implement theme animation.

However, you'd have to put the themed display item in its own layer, which has memory overhead and can affect the display of content above it. In particular you would probably lose subpixel AA of text above the theme background. Not a problem for progress indicators I guess, maybe a problem for a throbbing dialog button?

Comment 5

[Markus Stange [:mstange]]

Sounds good. We can probably get away with disabling subpixel AA for text on default buttons.

Comment 6

[Ben Turner (not reading bugmail, use the needinfo flag!)]

Hey guys, I think we're going to need this for bug 565217. Any progress updates, target milestone?

Comment 7

[Markus Stange [:mstange]]

Off-main-thread rendering won't happen for Firefox 4, as far as I know.

Comment 8

[Robert O'Callahan (:roc) (email my personal email if necessary)]

It won't. Why do you think it's needed for bug 565217?

Comment 9

[Ben Turner (not reading bugmail, use the needinfo flag!)]

(In reply to comment #8)
> It won't. Why do you think it's needed for bug 565217?

Our plan is to run the cycle collector's traverse phase while the main thread is blocked on the native event queue. If we can build a complete model and identify cycles before the main thread wakes back up then we can avoid a blocking cycle collection. If, on the other hand, we can't identify cycles in that amount of time then we will have to throw the model away and start over again next time. If we're constantly waking up to paint buttons then we will not see much benefit.

Comment 10

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Well, the only reason we'd be waking up to paint is if either something changed, or if the OS needs something repainted that was just revealed.  The latter case is going to be rare, and the former case is going to require the main thread to be woken up no matter what -- even if the actual compositing happens off-thread.  Regardless, I don't think this specific bug is about either issue, but is instead much more narrowly scoped.

Comment 11

[Robert O'Callahan (:roc) (email my personal email if necessary)]

I see.

Comment 12

[Robert O'Callahan (:roc) (email my personal email if necessary)]

(In reply to comment #10)
> Well, the only reason we'd be waking up to paint is if either something
> changed, or if the OS needs something repainted that was just revealed.  The
> latter case is going to be rare, and the former case is going to require the
> main thread to be woken up no matter what -- even if the actual compositing
> happens off-thread.

No, the idea here is that we could animate native widgets by making them their own layers and doing the animation entirely in the layer compositing process.

But that certainly is not going to happen for FF4.

Comment 13

[Ben Turner (not reading bugmail, use the needinfo flag!)]

Well, I was more worried about the "one timer per animated button" problem where we have lots and lots of independent timers going off really frequently when they could all be rolled into one. Is that a separate issue?

Comment 14

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Yes!

Comment 15

[Ben Turner (not reading bugmail, use the needinfo flag!)]

Aha, great! Is there a bug filed for that?

Comment 16

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Not that I know of.

Comment 17

[Markus Stange [:mstange]]

I've filed bug 586216 on that.

Comment 18

[Ben Turner (not reading bugmail, use the needinfo flag!)]

Great! I'll flip the blocking flags to that bug instead as this one seems to be a much longer term solution.

Comment 19

[Markus Stange [:mstange]]

This bug probably requires the same infrastructure as bug 717872, so I'm setting a dependency on that. Specifically, what I think is missing is:
 - Ability to update some kind of RGBA Image from a secondary thread. At the moment
   it looks like Images other than PLANAR_YCBCR can only be set on the main thread.
 - Ability to use simple memory buffers for data storage instead of shmems which are
   too heavy-weight.
 - fixes for some threading / object ownership issues

Comment 20

[Markus Stange [:mstange]]

Attachment: wip

This is on top of bug 894905.

Comment 21

[Markus Stange [:mstange]]

Attachment: testcase

Comment 22

[Markus Stange [:mstange]]

Attachment: wip