1995796 - How much of our applink startup telemetry is coming from our applink startup CI tests?

Status: NEW

(Testing :: Performance, task)

Description

[Markus Stange [:mstange]]

We're probably submitting telemetry data from the Android startup tests that we're running in CI. Is that a problem?

Comment 1

[Andrew Creskey [:acreskey]]

Marc suggested that we could potentially capture this with glam debug pings, which could be useful.

Comment 2

[Dave Hunt [:davehunt] [he/him] ⌚BST]

@bas @sparky are you able to help answer this question? do we disable telemetry pings for our applink startup tests in CI, or is there a way to identify them after the fact?

Comment 3

[Bas Schouten (:bas.schouten)]

I believe in various situations we disable telemetry pings. But the startup tests are kinda special so I bet we're not? We should though, IMHO.

Comment 4

[Greg Mierzwinski [:sparky]]

Last time we found this issue, we found a periodic signal coming from them in the telemetry data. We use a special startup activity to disable them in Raptor, but we don't use that with the startup tests so we're likely submitting telemetry data from them: https://searchfox.org/firefox-main/source/testing/raptor/raptor/browsertime/android.py#114

We also have a special tag for any telemetry that does get produced by our tests: https://searchfox.org/firefox-main/source/testing/raptor/raptor/browsertime/android.py#159

Ideally, we should be able to set a pref to disable these instead of having to use a separate startup activity since I imagine that changing the startup activity might impact our tests.