1229683 - Polish the way animations running on the compositor are represented in the animation-inspector panel

Status: RESOLVED WONTFIX

(DevTools :: Inspector: Animations, defect)

Description

[Patrick Brosset <:pbro>]

From bug 1228005 comment 10:

>>> Also, what are the lightning bolts for? http://cl.ly/image/342N2b2T2Y3o It
>>> seems like all of the animations for the page I'm currently looking at have
>>> them. It seems like they house additional information about the animation;
>>> maybe we could make the entire box trigger that info?

>> This indicates that the animation is currently running on the compositor.
>> This is helpful to developers to know if they're hitting the fast track
>> (animating properties like opacity or transform which do not cause reflows).

> Hm, all right. How would you feel about it toggling on hover for the entire
> area, instead of having the lightning bolt? The bolt just doesn't feel very
> semantic, so it was kind of confusing. (I was thinking, "Is this a super fast
> animation or something????" haha)

Comment 1

[Patrick Brosset <:pbro>]

What exactly do you have in mind for showing this on hover?

Comment 2

[Helen V. Holmes (:helenvholmes) (:✨)]

Having played with this more I finally understand the confusion—what I wanted to show on hover /does/ show on hover, it just triggers slower than I thought it did. Whoops, this bug has been rendered pointless @_@

As for the lightning bolt icon: I'm not sure what a good way to display that an animation is on the compositor because I have no mental model for what a "compositor" even is. Is it something that maybe you should explain to me and I can design a custom icon for? (Lightning bolts are used in a lot of places for a lot of things which has caused them to not be semantic for, well, anything...)

Comment 3

[Patrick Brosset <:pbro>]

(In reply to Helen V. Holmes (:helenvholmes) (:✨) from comment #2)
> Having played with this more I finally understand the confusion—what I
> wanted to show on hover /does/ show on hover, it just triggers slower than I
> thought it did. Whoops, this bug has been rendered pointless @_@
No problem! We use standard (title attribute) tooltips for this, and the delay is kind of long. If we used the devtools tooltips instead (the ones we use for image preview in the inspector for instance), we could define a shorter delay, and it would be more consistent. 
Do you think that would be a good thing to do?

> As for the lightning bolt icon: I'm not sure what a good way to display that
> an animation is on the compositor because I have no mental model for what a
> "compositor" even is. Is it something that maybe you should explain to me
> and I can design a custom icon for? (Lightning bolts are used in a lot of
> places for a lot of things which has caused them to not be semantic for,
> well, anything...)
Got it. So, I could try and explain this, but other people would do a much better job than me at that.
So, needinfo'ing Brian Birtles for this.

Comment 4

[Helen V. Holmes (:helenvholmes) (:✨)]

(In reply to Patrick Brosset [:pbrosset] [:pbro] from comment #3)
> No problem! We use standard (title attribute) tooltips for this, and the
> delay is kind of long. If we used the devtools tooltips instead (the ones we
> use for image preview in the inspector for instance), we could define a
> shorter delay, and it would be more consistent. 
> Do you think that would be a good thing to do?
I think this would be worth it! I think that it's pretty low-priority, so get to it as you want I suppose?

> Got it. So, I could try and explain this, but other people would do a much
> better job than me at that.
> So, needinfo'ing Brian Birtles for this.

All right, so I've read through this: https://wiki.mozilla.org/Gecko:Overview#Graphics and my ideas so far for an icon would be:

- just displaying a little "gfx" icon, since that's nice and tidy. (Might not be completely accurate though?)
- doing something with the idea of layers — could show layers in 3d space, perhaps.
- playing off the last idea, but we could also show drag to get at the idea of painting.

Anyway that's for starts. Brian, if any of those sound nice/horrible let me know! I'll try and do some doodles today of those and attach them later.

Comment 5

[Brian Birtles (:birtles)]

(In reply to Helen V. Holmes (:helenvholmes) (:✨) from comment #4)
> (In reply to Patrick Brosset [:pbrosset] [:pbro] from comment #3)
> > No problem! We use standard (title attribute) tooltips for this, and the
> > delay is kind of long. If we used the devtools tooltips instead (the ones we
> > use for image preview in the inspector for instance), we could define a
> > shorter delay, and it would be more consistent. 
> > Do you think that would be a good thing to do?
> I think this would be worth it! I think that it's pretty low-priority, so
> get to it as you want I suppose?
> 
> > Got it. So, I could try and explain this, but other people would do a much
> > better job than me at that.
> > So, needinfo'ing Brian Birtles for this.
> 
> All right, so I've read through this:
> https://wiki.mozilla.org/Gecko:Overview#Graphics and my ideas so far for an
> icon would be:
> 
> - just displaying a little "gfx" icon, since that's nice and tidy. (Might
> not be completely accurate though?)
> - doing something with the idea of layers — could show layers in 3d space,
> perhaps.
> - playing off the last idea, but we could also show drag to get at the idea
> of painting.
> 
> Anyway that's for starts. Brian, if any of those sound nice/horrible let me
> know! I'll try and do some doodles today of those and attach them later.

Hi, it's not so much about painting or graphics or layers even. It's true that layers are a prerequisite for this, but the essential point is that the animation is performed by a separate thread (or actually a separate process in the case of Firefox OS). So the key advantage is no jank.

With animations running on the main thread, if the browser gets busy the animation will jank, but when we are able to offload the animation to the compositor the animations keep running smoothly even when the browser is busy.

I've seen some good feedback about the lightning bolt on twitter (e.g. [1]) so I think it's not so bad.

I suppose a layer icon is a fairly close approximation, but it's not quite accurate. Sometimes we'll generate a layer for content but not run the animation on the compositor (e.g. if there is an animation of the 'left' property appearing on the same element, we'll force the 'transform' animation onto the main thread too so the two keep in step.) Most users might not care about that subtle difference, but if they're debugging with paint flashing or layer borders they might be confused because they can see the content is layerized but the animation doesn't have a layer icon.

For reference, in future we hope to expand this in two ways:

* Currently a combined animation (consisting of, say, 'color' and 'opacity') will be marked as running on the compositor even if only one of its properties is being run on the compositor ('opacity' in this case). We need to update the API to expose *which* properties are running on the compositor.

* When an animation is *not* running on the compositor we would like to report the reason why (e.g. 'Not running on the compositor because geometric properties (left, top, etc.) are also being animated').

The first part we could do fairly soon if needed.

[1] https://twitter.com/rachelnabors/status/648950449471557632

Comment 6

[Helen V. Holmes (:helenvholmes) (:✨)]

(In reply to Brian Birtles (:birtles) from comment #5)
Thanks for carefully explaining all this! This fills in a lot of gaps for me.

> I've seen some good feedback about the lightning bolt on twitter (e.g. [1])
> so I think it's not so bad.
I'll stop judging then, haha, seems like everyone likes it.

> * Currently a combined animation (consisting of, say, 'color' and 'opacity')
> will be marked as running on the compositor even if only one of its
> properties is being run on the compositor ('opacity' in this case). We need
> to update the API to expose *which* properties are running on the compositor.
Hm, meaning, if you expanded the animation you'd see the lightning bolts only on properties running on the compositor?

> * When an animation is *not* running on the compositor we would like to
> report the reason why (e.g. 'Not running on the compositor because geometric
> properties (left, top, etc.) are also being animated').
This sounds really awesome!

Comment 7

[Brian Birtles (:birtles)]

(In reply to Helen V. Holmes (:helenvholmes) (:✨) from comment #6)
> (In reply to Brian Birtles (:birtles) from comment #5)
> Thanks for carefully explaining all this! This fills in a lot of gaps for me.
> 
> > I've seen some good feedback about the lightning bolt on twitter (e.g. [1])
> > so I think it's not so bad.
> I'll stop judging then, haha, seems like everyone likes it.
> 
> > * Currently a combined animation (consisting of, say, 'color' and 'opacity')
> > will be marked as running on the compositor even if only one of its
> > properties is being run on the compositor ('opacity' in this case). We need
> > to update the API to expose *which* properties are running on the compositor.
> Hm, meaning, if you expanded the animation you'd see the lightning bolts
> only on properties running on the compositor?

Right. I'm not sure if what the plan is on the DevTools side to expose individual properties (we only recently added that feature to the API) but assuming you can drill down and see what properties are being animated, then that would be the logical point to indicate which properties are being animated on the compositor.

> > * When an animation is *not* running on the compositor we would like to
> > report the reason why (e.g. 'Not running on the compositor because geometric
> > properties (left, top, etc.) are also being animated').
> This sounds really awesome!

Yeah, hopefully we can cover this in early 2016.

Comment 8

[Patrick Brosset <:pbro>]

Let's close for now. There's been more perf-related information added to the timeline (bug 1254408).