1193857 - [telemetry] Measure toolbox layout usage

Status: NEW

(DevTools :: General, defect, P3)

Description

[Jeff Griffiths (:canuckistani) (:⚡︎)]

People can dock the toolbox to the bottom, right or in a window. We should measure what they're actually doing:

* capture the orientation on toolbox start
* capture the orientations switching from and to when switching?

I think I would like to know what the native resolution of the user's screen and the size of the related Firefox window for each data point.

I'm also going to log another bug for a measure to track multiple monitor usage, because I'm curious.

Comment 1

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

:clarkbw, any thoughts on this?

I think at the very least we should try to capture where the toolbox is docked.  We regularly see people file bugs about things not looking as good in side docked mode, but it's unclear which mode is most popular.  I know Chrome's tools push people towards side docked, while I feel like people on our team typically use bottom docked.  Anyway, we don't have any data about actual users on this.

There may also be some screen size stuff we should collect here, but not sure what exactly and if it should be linked to the docking choice or not.

Comment 2

[Bryan Clark (DevTools PM) [:clarkbw]]

(In reply to J. Ryan Stinnett [:jryans] (use ni?) from comment #1)
> :clarkbw, any thoughts on this?
> 
> I think at the very least we should try to capture where the toolbox is
> docked.  We regularly see people file bugs about things not looking as good
> in side docked mode, but it's unclear which mode is most popular.  I know
> Chrome's tools push people towards side docked, while I feel like people on
> our team typically use bottom docked.  Anyway, we don't have any data about
> actual users on this.

Our default choice will definitely push people in one direction or another.  If we could capture this in a way that knows the difference between the default choice and the intentional change that would be useful.  The panel opening metrics are a good example of how we don't really know if the Console panel is opened more because it is the default panel or because it is the desired choice.

I think you have it right, the question we want answered is: Which docking location is most desired?  (such that we could change our default to better align with our users)

And I think this would be a good thing to collect as we would likely make this change if we saw the numbers going in that direction.

> There may also be some screen size stuff we should collect here, but not
> sure what exactly and if it should be linked to the docking choice or not.

I think these are our followup questions, Does resolution have a causal effect on docking location?  (including other things we suspect may have a causal relationship as well)

However we should think carefully about these before trying to implement them.  If we find a causal relationship between resolution and docking layout would we implement some change to alter behaviour?  I think we might offer some kind of tooltip, but I don't think its a good idea to have the toolbox open up in different ways depending on the screen size of the user, this can have really odd side effects.  And finally are we going to commit to prioritize this work if the numbers come back as we anticipate.

Comment 3

[Jeff Griffiths (:canuckistani) (:⚡︎)]

Some answers I wanted out of this bug when I logged it:

1) do people side-dock? what ratio / size is their screen? Is it therefore worth putting more work into how the layout changes when the toolbox is side-docked, and what testing can we do to see what users actually want / have problems with?

2) same as above, but for the undocked window. TBH I would be in favour of ditching the undocked window if the ratio of low usage and maintenance overhead was right.

Intuitively I'd expect correlations like: 

* people with larger / wider monitors side dock more
* people with second or third monitors pop out the toolbox more

I bet the data is noisy. I also think if we do decide to prioritize work to improve these toolbox layouts we should do some user testing.