1150299 - Show optimization tier over time per frame

Status: RESOLVED FIXED

(DevTools :: Performance Tools (Profiler/Timeline), defect)

Description

[Jordan Santell [:jsantell] [@jsantell]]

Maybe as a timeline, or maybe as something else, a frame over time whether its in interpreter, baseline, or ion.

Comment 1

[Jordan Santell [:jsantell] [@jsantell]]

Need to think on UI/UX of this one

Comment 2

[Shu-yu Guo [:shu]]

Attachment: IMG_0452.JPG

Talked with several people. This is a picture of the current idea
for the UI.

For the initial UI, we should focus on displaying leaf
functions (that is, "self time" or "exclusive frames"
only). Focusing on frames with call trees underneath them should
come after we get the current view to work well.

The x axis is time. The y axis is conceptually "hotness" that
moves with time. We want to visualize the evolution of hotness of
a particular function over time. The idea is to divide the x axis
according to N buckets. Let |b| be a bucket. Each bucket has a
|startTime(b)| and |endTime(b)|. Suppose we have two helper functions:

  timeOfSamples(leafFun, startTime, endTime)

    Returns time taken by leaf samples of leafFun between startTime
    and endTime.

  timeOfAllSamples(startTime, endTime)

    Returns time taken by all samples between startTime and endTime.

The y value (height of each bucket) for a bucket |b| is then

  timeOfSamples(f, startTime(b), endTime(b))
  ------------------------------------------
  timeOfAllSamples(startTime(b), endTime(b))

To express the implementation tier, each bucket itself would be a
bar chart of 3 colors, a distinct color for each tier of
Interpreter, Baseline, and Ion. In the picture, Interpreter is
red, Baseline is blue, Ion is green. This is either a gradient or
a blurred average or whatever.

This graph is naturally discontinuous.

A quirk is that this graph is not a static picture that can be
zoomed in/out. Changing the width of the graph changes the
buckets, and thus the y values. So, "zooming" in and out of this
graph is really like pulling or squeezing an accordion. If it's
pulled out enough, each bucket on the x axis would be exactly 1
sample, and the max(y) would be 1. This is shown in the picture
with the "zoomed in" view.

Some visceral information I hope this UI gets across:

 - Phases in a program as seen by color changes over time: graph
   is tall enough and large chunks of red, then blue, then green,
   then blue, then green, etc.

 - Can't even get into Baseline: graph is tall enough and mostly
   red.

 - Failing to stay in Ion: graph is tall enough and mostly blue.

 - Oscillating and keep jumping in/out of Ion: graph is tall
   enough and is all technicolor.

Victor, would you be the one to work on this?

Comment 3

[Shu-yu Guo [:shu]]

This UI would probably live in its own tab, and functions would be selected via an inverted call tree view, the same one used by the JS call tree.

Comment 4

[Shu-yu Guo [:shu]]

I also want to stress that this view is more important to understanding performance than the current optimization info, which is only concerned with green portions of the attached whiteboard drawing.

Comment 5

[Shu-yu Guo [:shu]]

(In reply to Shu-yu Guo [:shu] from comment #3)
> This UI would probably live in its own tab, and functions would be selected
> via an inverted call tree view, the same one used by the JS call tree.

I'm going to backpedal on this. I talked with Stefan Penner, the perf guy for Ember, who said that he finds the context given by the FPS view and time markers and stuff important for identifying where to even start looking for slowness in JS.

Another timeline when "Show JIT Optimizations" is checked will mean less real screen estate for the call tree, but since looking at JIT optimizations is necessarily a deep dive into a particular frame, less space for the call tree seems okay.

Something like:

+------------+-----------------------------------------+-------------+
|            |                                         |  JIT Opts   |
| Recording1 |  Markers Timeline                       +-------------+
|            |                                         |             |
+------------------------------------------------------+             |
|            |                                         |             |
|            |  FPS Timeline                           |             |
|            |                                         |             |
|            +-----------------------------------------+             |
|            |                                         |             |
|            |                                         |             |
|            |  Tier Timeline                          |             |
|            |                                         |             |
|            +-----------------------------------------+             |
|            |                                         |             |
|            |  Call Tree                              |             |
|            |                                         |             |
|            |                                         |             |
|            |                                         |             |
|            |                                         |             |
|            |                                         |             |
+------------+-----------------------------------------+-------------+

Comment 6

[Victor Porof [:vporof][:vp]]

Comment on attachment 8589362 [details]
IMG_0452.JPG

IIRC we talked about this over vidyo and IRC. Please ping again if not.

Comment 7

[Nick Fitzgerald [:fitzgen] [⏰PST; UTC-8]]

Cool stuff!

Would it make sense to have it as its own tab *within* the perf tool instead of within the devtools? In other words, taking up both the "tier timeline" and "call tree" boxes in the diagram in comment 5. That way, you would still have the context of the markers timeline and FPS graph, but would have a little more breathing room.

Then again, if this workflow is really dependent on "select hot function from call tree" -> "look at function over time in the JIT tiers" then probably makes sense to keep them close together.

Comment 8

[Jordan Santell [:jsantell] [@jsantell]]

Attachment: 1150299-opt-graph.patch

So here's a patch where if you have jit opts on, do a recording with opts, you'll see the opt panel, select a frame, etc. Works with real data I think? But i have test data in performance/graphs for it to make it more obvious. So what's happening is the whole details view is maybe 300px high or so visually, but the box model is computing 1400px or so, and this goes all the way up the .theme-body container -- it's all 1400px high, and the browser toolbox selector looks like that's the case too: the bottom line is well off the screen. What this means is, when rendering the graph, I only see the top 300px or so of 1400px -- so only the very top. I have to open it alllllll the way to see the rest, and even so, maybe get down to 30% or so (on my MBP, which has 800px height).

I'm guessing this is some XUL style weirdness. Any ideas?

Comment 9

[Jordan Santell [:jsantell] [@jsantell]]

So first full pass at the full Optimizations View. Make sure you have the setting for "record JIT Optimizations" before you record, and if you save a profile, it must have been recorded with JIT optimizations -- it's no longer just a visual toggle.

In the Opts view, you can select the frame that you'd like to inspect (only frames with ion data are displayed), and selecting it renders the tier graph[0], as well as displays the optimizations strategies. You can also click the (i) bubble in the call tree next to a frame that has opt data to jump to the optimizations.

There are a few visual glitches and something with resizing the graph doesn't always rerender, so stretch it out a bit and it should be fine. The optimizations graph is red for interpreter, blue for baseline, green for ion.

Pinging ya'll to check this out before Whistler. Check out try and download a build[1]


[0] http://i.imgur.com/WevoW1G.png
[1] https://treeherder.mozilla.org/#/jobs?repo=try&revision=033d98faca93

Comment 10

[Jordan Santell [:jsantell] [@jsantell]]

Attachment: 1150299-opt-graph.patch

Comment 11

[Kannan Vijayan [:djvj]]

Hey Jordan!

That looks sweet.  The only suggestion I would have is to replace the sloped lines with horizontal/vertical lines (i.e. hard transitions at edges instead of sloping transitions from point to point).

Nice work though!

Comment 12

[Jordan Santell [:jsantell] [@jsantell]]

Attachment: 1150299-opt-graph.patch

Who loves team JS? I do.

Easy quick change for vertical bars, not dealing with tests right now, but looks good vertically: http://i.imgur.com/lsJhIkL.png

Some builds going on here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=5e81016ab3cb

See ya'll Tuesday for the talk on this thing

Comment 13

[Shu-yu Guo [:shu]]

Props Jordan and Victor!

Comment 14

[Kannan Vijayan [:djvj]]

You guys are awesome.  I'll download and test on my macbook today.

Comment 15

[Kannan Vijayan [:djvj]]

I'm having trouble getting the mountain-range view to render on the latest Linux nightly.  It shows a blank area where the optimization tier plot would have gone.

Is this a known issue?

Comment 16

[Jordan Santell [:jsantell] [@jsantell]]

I know Nick had that issue before on OSX, sometimes I think it just doesn't render, not sure if it's deterministic (if so, save the recording).

Pretty much there are a lot of issues with this

Comment 17

[Jordan Santell [:jsantell] [@jsantell]]

Comment on attachment 8625172 [details] [diff] [review]
1150299-opt-graph.patch

Removing review in lieu of not having this in a standalone view (for now, if the side panel is wide enough to be useful)

Comment 18

[Jordan Santell [:jsantell] [@jsantell]]

We agreed today that this should be in the sidebar UI.

Comment 19

[Jordan Santell [:jsantell] [@jsantell]]

Attachment: 1150299-opt-graph.patch

yaaaay finally. moved this back into call tree, cleaned up a few things. Shu for tree-model and correctness changes, vp for everything else

Comment 20

[Jordan Santell [:jsantell] [@jsantell]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=ae50f935a1f5

Comment 21

[Victor Porof [:vporof][:vp]]

Comment on attachment 8659594 [details] [diff] [review]
1150299-opt-graph.patch

Review of attachment 8659594 [details] [diff] [review]:
-----------------------------------------------------------------

very nice

::: browser/devtools/performance/modules/widgets/graphs.js
@@ +449,5 @@
> +OptimizationsGraph.prototype = Heritage.extend(MountainGraphWidget.prototype, {
> +
> +  render: Task.async(function *(threadNode, frameNode) {
> +    // Regardless if we draw or clear the graph, wait
> +    // until its ready.

it's

::: browser/devtools/performance/views/optimizations-list.js
@@ +382,5 @@
> +  /**
> +   * Called when `devtools.theme` changes.
> +   */
> +  _onThemeChanged: function (_, theme) {
> +    this.graph.setTheme(theme); 

ws

Comment 22

[Shu-yu Guo [:shu]]

Comment on attachment 8659594 [details] [diff] [review]
1150299-opt-graph.patch

Review of attachment 8659594 [details] [diff] [review]:
-----------------------------------------------------------------

This is more of a functionality review.

1) It'd be nice if the tier graph was vertically resizable or something, it's kind of hard to see.
2) The selection does nothing. Jordan tells me there's a bug on file to disable it already.

I'm fine with landing this and polishing it after it's landed.

Comment 23

[Pulsebot]

https://hg.mozilla.org/integration/fx-team/rev/d81b35757962

Comment 24

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/d81b35757962