671039 - batch 'o telemetry metrics for tabs, windows, awesomebar

Status: RESOLVED WONTFIX

(Firefox :: General, defect)

Description

[Justin Dolske [:Dolske]]

Distilled down from the Etherpad:

Tab/window telemetry
    Number of pinned tab (+1)
    Number of tabs?
    How often do pinned tabs glow?
    How often tab overflow arrows used
    Number of blank tabs
    Number of "redundant" tabs (same url?)
    How often switch-to-tab is used (+1)
    Time to open new tab (+1)
    Time to switch to another tab (+1)
    Number of windows
    window metrics, such as how often the browser window is 
      minimized/unfocused (webperf -- PageVisibility)

Awesomebar
    How often is first autocomplete item used?
    How many characters are typed in URL bar before autocomplete action is
      selected (+1)

Comment 1

[:Margaret Leibovic]

I think we should asses whether or not these metrics will give us useful information before we implement them. For example, if we have metrics for "Number of pinned tabs" and "How often do pinned tabs glow", what problems would these help us solve? What kind of extreme data would it take for us to change our design?

(In reply to comment #0)
>     Time to open new tab (+1)
>     Time to switch to another tab (+1)

Performance metrics seem like the most reasonable use of telemetry, especially for tracking regressions. However, I think we should think about what exactly we want to be measuring here. I played around with adding a probe to addTab in tabbrowser, but that made me question what parts of the tab opening process we want to time.

Comment 2

[Dão Gottwald [:dao]]

(In reply to Margaret Leibovic [:margaret] from comment #1)
> I think we should asses whether or not these metrics will give us useful
> information before we implement them.

And this is best done in separate bugs, such as those just filed.

Comment 3

[Drew Willcoxon :adw]

I want to try using the JS debugging API to see if I can somehow follow multiple stacks across turns of the event loop.  e.g., one addTab path goes addTab => setTimeout => tabContainer._handleNewTab => tabContainer._notifyBackgroundTab => setTimeout.  I want to time that entire multi-turn-of-the-event-loop path.  I could instrument the final setTimeout callback (and actually it's more complex since the setTimeouts are sometimes conditional), but doing that for every path sucks for me and isn't resilient to code changes.  But I'm not able to get to a point where I realize what I wrote above is naive and impossible because it looks like the debugging API wasn't designed with the ability to debug chrome JS in mind.  So I guess it's the aforementioned suckier approach.

Comment 4

[Dietrich Ayala (:dietrich)]

would it help to have the code emit some finer-grained events, such as TabBeginOpen, TabEndOpen, and telemetrize based on those?

Comment 5

[Dão Gottwald [:dao]]

(In reply to Dietrich Ayala (:dietrich) from comment #4)
> would it help to have the code emit some finer-grained events, such as
> TabBeginOpen, TabEndOpen, and telemetrize based on those?

No! (Maybe it would help, but we don't want to unleash random events, because we can't get rid of them afterwards.) We can sprinkle the measurements all across the code. But, again, it's better to discuss and review this case by case in separate bugs.

Comment 6

[Drew Willcoxon :adw]

(In reply to Dietrich Ayala (:dietrich) from comment #4)
> would it help to have the code emit some finer-grained events, such as
> TabBeginOpen, TabEndOpen, and telemetrize based on those?

That's how I'm implementing the aforementioned suckier approach.

(In reply to Dão Gottwald [:dao] from comment #5)
> No!

I don't see why we can't fire all the events we want with the understanding that they're private implementation details.

Why do both bug 708672 and bug 708788 exist?  Tab animations are part of opening a new tab.  What does bug 708672 mean otherwise?

Dão, you assigned yourself to the new bug 708788, but I was already working on it as part of the addTab telemetry.  Now I'm happy to have you do this tab-related telemetry instead of me -- in fact I can't think of a better person to do it, since you know this code so well -- but we need to clarify who's working on what.

Comment 7

[Drew Willcoxon :adw]

Anyhow, apparently this is now a meta bug so I'm unassigning me from it.

Comment 8

[Dão Gottwald [:dao]]

(In reply to Drew Willcoxon :adw from comment #6)
> I don't see why we can't fire all the events we want with the understanding
> that they're private implementation details.

Because add-on authors would have to share this understanding...

> Why do both bug 708672 and bug 708788 exist?  Tab animations are part of
> opening a new tab.  What does bug 708672 mean otherwise?

Bug 708788 is for measuring how long the animation takes. Bug 708672 wants to measure the time "from keyboard, button, or menu click to new tab visible and then usable"; I'm not sure what exactly this means.

> Dão, you assigned yourself to the new bug 708788, but I was already working
> on it as part of the addTab telemetry.  Now I'm happy to have you do this
> tab-related telemetry instead of me -- in fact I can't think of a better
> person to do it, since you know this code so well -- but we need to clarify
> who's working on what.

I haven't started yet. If you have something, please do attach it to bug 708788.

Comment 9

[Drew Willcoxon :adw]

(In reply to Dão Gottwald [:dao] from comment #8)
> Because add-on authors would have to share this understanding...

Which is why the events would be undocumented or called out as being private...

But anyway, I don't have much, and what I do have it sounds like you'd r-, fair enough.

Comment 10

[Dão Gottwald [:dao]]

(In reply to Drew Willcoxon :adw from comment #9)
> (In reply to Dão Gottwald [:dao] from comment #8)
> > Because add-on authors would have to share this understanding...
> 
> Which is why the events would be undocumented or called out as being
> private...

I'm afraid this wouldn't help. The line between public and private is blurry in chrome code and people tend to just ignore it.

Comment 11

[Dietrich Ayala (:dietrich)]

Closing this, and making the individual probe bugs block the main metabug for front-end telemetry.