1995796 - How much of our applink startup telemetry is coming from our applink startup CI tests?

Status: NEW

(Testing :: Performance, task)

Description

[Markus Stange [:mstange]]

We're probably submitting telemetry data from the Android startup tests that we're running in CI. Is that a problem?

Comment 1

[Andrew Creskey [:acreskey]]

Marc suggested that we could potentially capture this with glam debug pings, which could be useful.

Comment 2

[Dave Hunt [:davehunt] [he/him] ⌚BST]

@bas @sparky are you able to help answer this question? do we disable telemetry pings for our applink startup tests in CI, or is there a way to identify them after the fact?

Comment 3

[Bas Schouten (:bas.schouten)]

I believe in various situations we disable telemetry pings. But the startup tests are kinda special so I bet we're not? We should though, IMHO.

Comment 4

[Greg Mierzwinski [:sparky]]

Last time we found this issue, we found a periodic signal coming from them in the telemetry data. We use a special startup activity to disable them in Raptor, but we don't use that with the startup tests so we're likely submitting telemetry data from them: https://searchfox.org/firefox-main/source/testing/raptor/raptor/browsertime/android.py#114

We also have a special tag for any telemetry that does get produced by our tests: https://searchfox.org/firefox-main/source/testing/raptor/raptor/browsertime/android.py#159

Ideally, we should be able to set a pref to disable these instead of having to use a separate startup activity since I imagine that changing the startup activity might impact our tests.

Comment 5

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Based on comment 2, are you aware of a way to capture how much telemetry pings are being sent from our tests using glam debug pings? If not, do you have any ideas on who could help move this forwards?