1530481 - Implement a Firefox Origin Telemetry skeleton

Status: RESOLVED FIXED

(Toolkit :: Telemetry, enhancement, P1)

Description

[Chris H-C :chutten]

This bug is about learning how well the proposed design of Firefox Origin Telemetry (bug 1522660) works when we try to turn it into a concrete implementation.

This skeleton will have:

• An API (which we hope to keep stable as we move forward with implementation)
• A data storage mechanism
• The ability to prio-encode information to a payload
• Only enough tests to convince myself that it works
• A list of fake origins

This skeleton does not need to have:

• A ping assembly, scheduling, and sending mechanism (because I'm rather practised at this part already)
• Any way to snapshot the plaintext or encoded data (because that's only needed by tests, about:telemetry, and ping assembly)
• The rest of the tests
• about:telemetry support
• The actual list of supported origins

Questions we hope this exercise will help answer:

• In what language should the API and data storage portions be written?
• Are any aspects of the design proposal under-specified? Turn out to be wrong?

Comment 1

[Chris H-C :chutten]

Attachment: Bug 1530481 - Sketch out Firefox Origin Telemetry's impl. r?janerik

Comment 2

[Chris H-C :chutten]

Attachment: Bug 1530481 - Test Firefox Origin Telemetry. r?janerik

Depends on D21756

Comment 3

[Chris H-C :chutten]

Attachment: Bug 1530481 - Refactor PrioEncoder to be usable from C++. r?rhelmer

For some reason Telemetry's gtests doesn't like one or more of the dependencies
of mprio.h, so I had to move it into PrioEncoder's cpp and forward declare the
PublicKey typedefs.

It isn't the cleanest, but I'm surprised C++ let me off with just that level
of nonsensery.

Depends on D22119

Comment 4

[Chris H-C :chutten]

Attachment: Bug 1530481 - Support encoded snapshots in Firefox Origin Telemetry. r?janerik

The threading and promise handling is kept at the TelemetryImpl level
deliberately to facilitate testing. It also turns out that hangs/modules do
more-or-less the same split for their off-thread collection, so in
retrospect it is the obvious choice.

For the batch ID we're using the app build id as discussed.

Depends on D22605

Comment 5

[Chris H-C :chutten]

Try run's looking fairly green: https://treeherder.mozilla.org/#/jobs?repo=try&revision=9b4584eb545c737e68f1cfbee2b64540732dfd99

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0ada21a3af93
Sketch out Firefox Origin Telemetry's impl. r=janerik
https://hg.mozilla.org/integration/autoland/rev/1dab9495422d
Test Firefox Origin Telemetry. r=janerik
https://hg.mozilla.org/integration/autoland/rev/171275cc24fb
Refactor PrioEncoder to be usable from C++. r=rhelmer
https://hg.mozilla.org/integration/autoland/rev/304549224738
Support encoded snapshots in Firefox Origin Telemetry. r=janerik

Comment 7

[Oana Pop-Rus]

Backed out for causing leaks

Push with failure: https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=linux%2Cx64%2Cdebug%2Cmochitests%2Cwith%2Ce10s%2Ctest-linux64%2Fdebug-mochitest-devtools-chrome-e10s-3%2Cm-e10s%28dt3%29&selectedJob=235237671&revision=30454922473818f48661f81b29b097b2e718e0d0

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=235237671&repo=autoland&lineNumber=3728

Backout: https://hg.mozilla.org/integration/autoland/rev/7eb8e627961c4b54041da26c73bd5ee2d65fee3c

Comment 8

[Chris H-C :chutten]

Ack, missed gBatchID's buffer and gOriginsList's nsTArray_base.

New patches put them on the heap and manage them explicitly. I guess I'll work on moving them to definitions files next so I can generate them at build time instead of putting them on the heap.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a032ef105979
Sketch out Firefox Origin Telemetry's impl. r=janerik
https://hg.mozilla.org/integration/autoland/rev/9cdc1c4c026b
Test Firefox Origin Telemetry. r=janerik
https://hg.mozilla.org/integration/autoland/rev/70d73a31d22a
Refactor PrioEncoder to be usable from C++. r=rhelmer
https://hg.mozilla.org/integration/autoland/rev/91a4fa9c15a8
Support encoded snapshots in Firefox Origin Telemetry. r=janerik

Comment 10

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/a032ef105979
https://hg.mozilla.org/mozilla-central/rev/9cdc1c4c026b
https://hg.mozilla.org/mozilla-central/rev/70d73a31d22a
https://hg.mozilla.org/mozilla-central/rev/91a4fa9c15a8

Comment 11

[Peter Van der Beken [:peterv]]

https://hg.mozilla.org/projects/ash/rev/a032ef105979c860d0dd563ebd834869a300a0f8
Bug 1530481 - Sketch out Firefox Origin Telemetry's impl. r=janerik

https://hg.mozilla.org/projects/ash/rev/9cdc1c4c026b780856a0169864051b4a151fc94d
Bug 1530481 - Test Firefox Origin Telemetry. r=janerik

https://hg.mozilla.org/projects/ash/rev/70d73a31d22a0a9409d30bfcee9c38b71ada56e6
Bug 1530481 - Refactor PrioEncoder to be usable from C++. r=rhelmer

https://hg.mozilla.org/projects/ash/rev/91a4fa9c15a838309e4abf7250f4e7267a603eb6
Bug 1530481 - Support encoded snapshots in Firefox Origin Telemetry. r=janerik