1687358 - Implement telemetry probes to determine the best possible threshold for <lazyload>

Status: RESOLVED FIXED

(Core :: DOM: Core & HTML, enhancement, P2)

Description

[Sean Feng [:sefeng]]

We want to learn how effective the pixel-based threshold is on the desktop.

There are five telemetry probes that we want to implement

LAZYLOAD_TOTAL_LAZYLOAD_IMAGES - reports the total number of lazyload images per document.

LAZYLOAD_STARTED - reports the number of lazyload images that were started to load per document.

LAZYLOAD_STARTED_NOT_REACH_VIEWPORT - reports the number of lazyload images that were started to load but didn't reach the viewport per document.

LAZYLOAD_REACH_VIEWPORT_LOADING - reports the number of lazyload images that reached the viewport, but was still in the loading state per document.

LAZYLOAD_REACH_VIEWPORT_FINISHED - reports the number of lazyload images that reached the viewport and the loading was finished per document.

Once we have the above threshold, we should be able to use them to learn how effective the threshold is by looking at the trends and distributions.

Comment 1

[Sean Feng [:sefeng]]

Attachment: Bug 1687358 - Implement telemetry probes to determine the best possible threshold for <lazyload> r=emilio

This patch implements five telementry probes to help us learning how
lazyload thresholds perform in the wild.

Comment 2

[Sean Feng [:sefeng]]

Attachment: data_collection_request.txt

Comment 3

[Chris H-C :chutten]

Comment on attachment 9198839 [details]
data_collection_request.txt

DATA COLLECTION REVIEW RESPONSE:

Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes.

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, Sean Feng is responsible.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 1, Technical.

Is the data collection request for default-on or default-off?

Default on for all channels.

Does the instrumentation include the addition of any new identifiers?

No.

Is the data collection covered by the existing Firefox privacy notice?

Yes.

Does there need to be a check-in in the future to determine whether to renew the data?

No. This collection is permanent.

---

Result: datareview+

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a77ccd1c1659
Implement telemetry probes to determine the best possible threshold for <lazyload> r=emilio

Comment 5

[Bogdan Tara[:bogdan_tara | bogdant]]

Backed out changeset a77ccd1c1659 (bug 1687358) for browser_lazyload_telemetry.js failures.

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&fromchange=a77ccd1c1659568183ac2c88aaf043931a31aa2c&test_paths=layout%2Fbase%2Ftests&tochange=0a80b9cb87ba29fdd6e230e362fbc051e4bb2524&selectedTaskRun=MfMihsqqRi2RuogcU530oQ.0

Backout link: https://hg.mozilla.org/integration/autoland/rev/0a80b9cb87ba29fdd6e230e362fbc051e4bb2524

Failure log: https://treeherder.mozilla.org/logviewer?job_id=327912591&repo=autoland&lineNumber=4736

[task 2021-01-27T00:34:48.618Z] 00:34:48     INFO - TEST-START | layout/base/tests/browser_lazyload_telemetry.js
[task 2021-01-27T00:34:50.732Z] 00:34:50     INFO - TEST-INFO | started process screencapture
[task 2021-01-27T00:34:50.837Z] 00:34:50     INFO - TEST-INFO | screencapture: exit 0
[task 2021-01-27T00:34:50.837Z] 00:34:50     INFO - Buffered messages logged at 00:34:48
[task 2021-01-27T00:34:50.837Z] 00:34:50     INFO - Entering test bound testTelemetryCollection
[task 2021-01-27T00:34:50.837Z] 00:34:50     INFO - Console message: [JavaScript Error: "The character encoding of the HTML document was not declared. The document will render with garbled text in some browser configurations if the document contains characters from outside the US-ASCII range. The character encoding of the page must be declared in the document or in the transfer protocol." {file: "http://example.com/browser/layout/base/tests/file_lazyload_telemetry.html" line: 0}]
[task 2021-01-27T00:34:50.837Z] 00:34:50     INFO - Buffered messages logged at 00:34:50
[task 2021-01-27T00:34:50.837Z] 00:34:50     INFO - TEST-PASS | layout/base/tests/browser_lazyload_telemetry.js | undefined assertion name - 
[task 2021-01-27T00:34:50.837Z] 00:34:50     INFO - Buffered messages finished
[task 2021-01-27T00:34:50.837Z] 00:34:50     INFO - TEST-UNEXPECTED-FAIL | layout/base/tests/browser_lazyload_telemetry.js | undefined assertion name - Got +0, expected 1
[task 2021-01-27T00:34:50.837Z] 00:34:50     INFO - Stack trace:
[task 2021-01-27T00:34:50.837Z] 00:34:50     INFO - chrome://mochikit/content/browser-test.js:test_is:1351
[task 2021-01-27T00:34:50.837Z] 00:34:50     INFO - chrome://mochitests/content/browser/layout/base/tests/browser_lazyload_telemetry.js:testTelemetryCollection:50
[task 2021-01-27T00:34:50.837Z] 00:34:50     INFO - TEST-PASS | layout/base/tests/browser_lazyload_telemetry.js | undefined assertion name - 
[task 2021-01-27T00:34:50.837Z] 00:34:50     INFO - Not taking screenshot here: see the one that was previously logged
[task 2021-01-27T00:34:50.837Z] 00:34:50     INFO - TEST-UNEXPECTED-FAIL | layout/base/tests/browser_lazyload_telemetry.js | undefined assertion name - Got 2, expected 3
[task 2021-01-27T00:34:50.837Z] 00:34:50     INFO - Stack trace:
[task 2021-01-27T00:34:50.837Z] 00:34:50     INFO - chrome://mochikit/content/browser-test.js:test_is:1351
[task 2021-01-27T00:34:50.837Z] 00:34:50     INFO - chrome://mochitests/content/browser/layout/base/tests/browser_lazyload_telemetry.js:testTelemetryCollection:56
[task 2021-01-27T00:34:50.837Z] 00:34:50     INFO - Leaving test bound testTelemetryCollection
[task 2021-01-27T00:34:50.837Z] 00:34:50     INFO - GECKO(1591) | MEMORY STAT | vsize 6992MB | residentFast 281MB | heapAllocated 98MB
[task 2021-01-27T00:34:50.837Z] 00:34:50     INFO - TEST-OK | layout/base/tests/browser_lazyload_telemetry.js | took 2141ms

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2100448273e1
Implement telemetry probes to determine the best possible threshold for <lazyload> r=emilio

Comment 7

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/2100448273e1

Comment 8

[Sean Feng [:sefeng]]

Code landed, clearing my NI