1518624 - Add telemetry for amount of adjustments performed by scroll anchoring

Status: RESOLVED FIXED

(Core :: Layout: Scrolling and Overflow, enhancement, P3)

Description

[Ryan Hunt [:rhunt] (on leave until early May)]

Once scroll anchoring has been landed, we should add telemetry to see the how effective our implementation is. The Chrome folks said that their implementation prevented about three visual jumps per page load [1]. It would be good to know if we have a comparable experience.

[1] https://blog.chromium.org/2017/04/scroll-anchoring-for-web-developers.html

Comment 1

[Ryan Hunt [:rhunt] (on leave until early May)]

Attachment: Bug 1518624 - Add telemetry for amount and length in pixels of scroll anchor adjustments. r?dholbert

This commit adds two new telemetry probes to collect:
 1. The amount of scroll anchoring adjustments applied
 2. The total absolute length in CSS pixels of scroll anchoring adjustments

Both of these metrics are collected on a per top-level-document basis, and
reported with other use-counters.

Comment 2

[Daniel Holbert [:dholbert]]

To gather new telemetry data, it looks like need to fill out the questionnaire [1] and post it as a .txt file on this bug, per [2], and get r+ from :chutten on it.

[1] https://github.com/mozilla/data-review/blob/master/request.md
[2] https://wiki.mozilla.org/Firefox/Data_Collection#Requesting_Data_Collection

Comment 3

[Ryan Hunt [:rhunt] (on leave until early May)]

Attachment: scroll-telemetry.txt

Comment 4

[Ryan Hunt [:rhunt] (on leave until early May)]

I need to write this here so I don't forget. The telemetry bucket count and range were just guesses. I need to validate they're reasonable after review.

Comment 5

[Ryan Hunt [:rhunt] (on leave until early May)]

Attachment: scroll-telemetry.txt

Actually there's one additional probe I'd like to consider here. I've added it to the data review, but I haven't yet implemented it.

Comment 6

[Chris H-C :chutten]

Comment on attachment 9036142 [details]
scroll-telemetry.txt

Preliminary notes:

The request is for all release channels but the metrics definitions will only collect on prerelease channels. I will review the request as written but ask that you either update your patch to collect on all channels or clarify that prerelease collection is fine.

DATA COLLECTION REVIEW RESPONSE:

Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes. These collections are Telemetry so are documented in their definitions files (Histograms.json), the Probe Dictionary, and on telemetry.mozilla.org's Measurement Dashboards.

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. These collections are Telemetry so can be controlled through Firefox's Preferences.

If the request is for permanent data collection, is there someone who will monitor the data over time?

N/A, these collections expire in Firefox 70.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction. Though the presence and location of scroll anchors is a technical detail, scrolling is a product of user interaction.

Is the data collection request for default-on or default-off?

Default on, all channels.

Does the instrumentation include the addition of any new identifiers?

No.

Is the data collection covered by the existing Firefox privacy notice?

Yes.

Does there need to be a check-in in the future to determine whether to renew the data?

Yes. :rhunt is responsible for removing or renewing these collections before they expire in Firefox 70.

---

Result: datareview+

Comment 7

[Chris H-C :chutten]

Comment on attachment 9036181 [details]
scroll-telemetry.txt

Previous review response covers the third metric SCROLL_ANCHOR_ADJUSTMENT_LENGTH_AGGREGATE though it may be worth having a conversation on #telemetry about what sorts of questions you hope to answer with these collections.

And it occurs to me when I said "the presence and location of scroll anchors" bit from the response I was incorrect as I think this feature is about anchoring the scroll location during disruptive relayouts on load. These metrics may indeed be category 1 after all. (though Category 1/2 makes no policy or review difference at this time)

Comment 8

[Ryan Hunt [:rhunt] (on leave until early May)]

(In reply to Chris H-C :chutten from comment #6)

Comment on attachment 9036142 [details]
scroll-telemetry.txt

Preliminary notes:

The request is for all release channels but the metrics definitions will
only collect on prerelease channels. I will review the request as written
but ask that you either update your patch to collect on all channels or
clarify that prerelease collection is fine.

Hmm, I've always gotten a bit confused on the default for collection with histograms.

I don't actually need release data collection, so just prerelease is fine.

(In reply to Chris H-C :chutten from comment #7)

Comment on attachment 9036181 [details]
scroll-telemetry.txt

Previous review response covers the third metric
SCROLL_ANCHOR_ADJUSTMENT_LENGTH_AGGREGATE though it may be worth having a
conversation on #telemetry about what sorts of questions you hope to answer
with these collections.

And it occurs to me when I said "the presence and location of scroll
anchors" bit from the response I was incorrect as I think this feature is
about anchoring the scroll location during disruptive relayouts on load.
These metrics may indeed be category 1 after all. (though Category 1/2 makes
no policy or review difference at this time)

The probes are supposed to capture how well scroll anchoring is performing, and should generally only be affected by our implementation and web content. Not how our users browse (like via touch, mouse wheel, fast scrolling, slow scrolling, lots of scrolling, no scrolling etc).

I'm not entirely familiar with what classifies things as "Technical" vs "Interaction", so I could be very off.

Comment 9

[Ryan Hunt [:rhunt] (on leave until early May)]

(In reply to Ryan Hunt [:rhunt] from comment #5)

Created attachment 9036181 [details]
scroll-telemetry.txt

Actually there's one additional probe I'd like to consider here. I've added
it to the data review, but I haven't yet implemented it.

The additional probe, was the non-aggregate SCROLL_ANCHOR_ADJUSTMENT_LENGTH. On additional thought, I don't believe it's necessary.

I think we can just take the SCROLL_ANCHOR_ADJUSTMENT_LENGTH_AGGREGATE / SCROLL_ANCHOR_ADJUSTMENT_COUNT to find the average adjustment length. I won't implement it then.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/bd1415025d3e
Add telemetry for amount and length in pixels of scroll anchor adjustments. r=dholbert

Comment 11

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/bd1415025d3e