1295932 - Measure the times per "session fragment" a page is loaded, broken down by action

Status: NEW

(Toolkit :: Telemetry, defect, P3)

Description

[Alessio Placitelli [:Dexter]]

As reported in [1], we should measure the number of times per "session fragment" a page is loaded as a result of any action, broken out at least by:

- entering a URI directly in the address bar
- performing a search from FF ui, broken out by:
   - the awesome bar
   - the search bar
   - a right-click search
   - newtab page search
   - about:home search
   - (any other seach access point)
- using the awesome bar, broken out by:
   - awesome bar history suggestion
   - awesome bar existing tab suggestion
   - (any other awesome bar suggestion)
- clicking an item in the history UI
- clicking a bookmark
- browsing in-content from one page to another
- selecting an item from about:newtab [what about Activity Stream?]
- loading a from another application
- (any other action causing a page load [should try to enumerate all possibilities before counting something as ""other""])

This can be stored as a keyed histogram or something like that. As a sanity check, we should require that the sum of all these items matches the number of URI's loaded as captured by the new probe in bug 1271313.

[1] - https://docs.google.com/spreadsheets/d/1G33YEBL2-hSaF0-HrHPi0HVwANoTpYHsnevmMd1eZ1U/edit#gid=0

Comment 1

[brendan c]

The list from above continues to be refined as we discuss with experts from the product team. So far, we're at this:
----------
number of times per ""session fragment"" a page is loaded as a result of any action, broken out at least by:
- entering a URI directly in the address bar
- performing a search from FF ui, broken out by:
   - the awesome bar
   - the search bar
   - a right-click search
   - newtab page search
   - about:home search
   - (any other seach access point)
- using the awesome bar, broken out by:
   - awesome bar history suggestion
   - awesome bar existing tab suggestion
   - awesomebar bookmark suggestion
   - (any other awesome bar suggestion)
- history
   - history toolbar
   - history side pane (ctrl-h)
- clicking a bookmark
   - in the bookmarks menu
   - in the bookmarks side pane (ctrl-b)
   - in the bookmarks toolbar
- back/fwd
   - back button
   - fwd button
   - back/fwd dropdown
- browsing in-content from one page to another
- selecting an item from about:newtab [what about Activity Stream?]
- loading a from another application
- restoring pages from session-restore
- (any other action causing a page load [should try to enumerate all posibilites before counting something as ""other""])

This can be stored as a keyed scalars or something like that.

As a sanity check, we should require that the sum of all these items matches the number of URI's loaded as captured by the new probe in https://bugzilla.mozilla.org/show_bug.cgi?id=1271313"
----------

but we haven't yet incorporated a very detailed list of search-related actions that Javaun has compiled.

On thing that we should start thinking about is how to store and capture the data in a way that is forward compatible as we add more detail.

For example, we might want to start capturing counts of in-content navigation immediately, but (just hypothetically) suppose that in the future we want to start differentiating between in-content navigation from a plain left click vs pages opened using the menu items "open link in new tab" and "open link in new window" in the right click menu. we'll want a way to start collecting the data now while also being able to represent the sub-bins later... but hopefully in a way that partitions the data without overlaps (so that there is not double counting).

I don't know, maybe we'd start out with just:
engagement.navigation.awesomebar.notspecified
And as we add more detail add:
engagement.navigation.awesomebar.notspecified
engagement.navigation.awesomebar.search
engagement.navigation.awesomebar.bookmark
Or something like that. I guess it will still be tricky b/c people will have to know that clients from before a certain build would not have e.g. awesomebar.search, but at least you'd get the correct total for any client by just adding all of the fields under awesomebar.

(Sorry, I guess I'm kind of thinking aloud)

Comment 2

[Georg Fritzsche [:gfritzsche]]

I'm wondering if we couldn't do this as a categorical histogram now.
The potential issues that i see is:
(1) we couldn't use dynamic keys
(2) i'm not sure if we can add new buckets to categorical histograms later (and have longitudinal etc. deal fine with it)

I think (1) is fine, we should only measure known sources anyway and can just add these to the Histograms.json entry.

Roberto, is (2) a concern?

Comment 3

[Georg Fritzsche [:gfritzsche]]

Also, categorical histograms might not be convenient for analysis right now.
We should have them in longitudinal like linear histograms, so getting from there to the labels is an unsolved item (see bug 1288444).

Comment 4

[Roberto Agostino Vitillo (:rvitillo)]

(In reply to Georg Fritzsche [:gfritzsche] from comment #2)
> I'm wondering if we couldn't do this as a categorical histogram now.
> The potential issues that i see is:
> (1) we couldn't use dynamic keys
> (2) i'm not sure if we can add new buckets to categorical histograms later
> (and have longitudinal etc. deal fine with it)
> 
> I think (1) is fine, we should only measure known sources anyway and can
> just add these to the Histograms.json entry.
> 
> Roberto, is (2) a concern?

A categorical histogram should be immutable and have a fixed set of labels, just like any other histogram. This is going to be required to support categorical histograms on t.m.o.

Comment 5

[Georg Fritzsche [:gfritzsche]]

Continuing this on IRC, categorical histograms don't seem like a good option right now due to the tooling support (supporting adding new labels e.g. depends on some future bigger changes in the aggregator).
So we should go for keyed scalars now.

Comment 6

[Alessio Placitelli [:Dexter]]

Should this probe count URI loads in private mode as well?

The "browser.engagement.total_uri_count" does not do that.

Comment 7

[brendan c]

It's important that this is consistent with the total_uri_count, so let's follow the policy decision that was made in that case: we can omit private mode URI loads. Thanks Alessio.