1868582 - Glean testGetValue API returns `null` for all metrics when run in mozregression (because `datareporting.healthreport.uploadEnabled` is set to false)

Status: NEW

(Testing :: mozregression, defect, P3)

Description

[:Gijs (he/him)]

mozregression either explicitly sets or implicitly leads to datareporting.healthreport.uploadEnabled being set to false.

This means Glean's test/debugging API testGetValue returns null.

This is unfortunate when you're trying to work out when a given metric stopped/started working a particular way. Also because I obviously don't actually want to submit data to prod from my mozregression run, but AIUI there is no other way to look at the data.

(I tried to use about:glean and set a debug ping identifier, but this does not appear to have made a difference to the test API on its own. Also, there's no feedback when clicking the "apply settings and submit ping" button, so unclear if there's an error or that's just expected.)

Comment 1

[Chris H-C :chutten]

(Sorry about the lack of feedback. Tracked in bug 1818352 )

Well that kinda sucks. I'd have thought that mozregression would supply a default new profile (ie, one that enabled data upload). The amount of data we'd get from it would be miniscule, so there'd have been no meaningful harm in it just acting normal.

If we're worried about mozregression-supplied information, we can choose whether to use an environment variable to tell the pipeline to treat these as coming from automation (FOG also listens to MOZ_AUTOMATION if we want to go that route) or use a negative value for telemetry.fog.test.localhost_port to lie to the SDK and have it think that data upload is enabled (when really we're just dropping all pings on the floor).

:Gijs, I don't suppose you know which Firefox might prefer mozregression use? (Or who'd like to make that decision?). If not, I suppose I could just try my best here -- though the data upload pref is listened to by more than just Glean and Telemetry these days.

Comment 2

[:Gijs (he/him)]

(In reply to Chris H-C :chutten from comment #1)

If we're worried about mozregression-supplied information, we can choose whether to use an environment variable to tell the pipeline to treat these as coming from automation (FOG also listens to MOZ_AUTOMATION if we want to go that route) or use a negative value for telemetry.fog.test.localhost_port to lie to the SDK and have it think that data upload is enabled (when really we're just dropping all pings on the floor).

:Gijs, I don't suppose you know which Firefox might prefer mozregression use? (Or who'd like to make that decision?). If not, I suppose I could just try my best here -- though the data upload pref is listened to by more than just Glean and Telemetry these days.

I don't know what's easier on mozregression. On the Fx side, maybe prefs are slightly faster than env vars from a performance PoV because you can use staticprefs and shared memory for child procs and so on, so the check is very cheap, whereas looking up env vars continuously is potentially more expensive (though you could decide to cache it on startup or whatever, I think you'd have to architect it yourself). Looks like Zeid has contributed to the mozregression github repo somewhat recently, maybe he can comment from the mozregression PoV?

Comment 3

[Zeid [:zeid]]

mozregression either explicitly sets or implicitly leads to datareporting.healthreport.uploadEnabled being set to false

It is indeed set explicitly to false. You can set it to true on a case-by-case basis by passing --pref "datareporting.healthreport.uploadEnabled:true" when launching mozregression.

If we're worried about mozregression-supplied information, we can choose whether to use an environment variable to tell the pipeline to treat these as coming from automation (FOG also listens to MOZ_AUTOMATION if we want to go that route) or use a negative value for telemetry.fog.test.localhost_port to lie to the SDK and have it think that data upload is enabled (when really we're just dropping all pings on the floor).

If needed, you can also set telemetry.fog.test.localhost_port:-1 as a pref.

Does this help?

Edit: not sure why this was decided originally, but if this is an ongoing problem or affecting things more broadly, it is certainly possible to make these changes permanent, if there is no negative effect on telemetry data OR if we can somehow namespace them.

Comment 4

[:Gijs (he/him)]

(In reply to Zeid Zabaneh [:zeid] from comment #3)

Edit: not sure why this was decided originally, but if this is an ongoing problem or affecting things more broadly,

Right, it confused me for a while because I didn't understand why the API was returning null - I filed the bug after finding the root cause. :-)

I think it would be nice to make this "do what I mean" by default, ie add the pref change and/or update glean so the API call "works" when called from mozregression, without requiring the mozregression user to (a) know about this issue and (b) manually pass or set a pref, or set an env var. :-)

it is certainly possible to make these changes permanent, if there is no negative effect on telemetry data OR if we can somehow namespace them.

From what :chutten said, I think this is possible. Not 100% sure if it's solely a mozregression fix to set the pref or if there would need to be further changes to glean as well. Chris?

Comment 5

[Chris H-C :chutten]

All we need to do is remove this line to turn datareporting.healthreport.uploadEnabled on. This may have downstream ramifications as this will turn on a few things. But seeing as mozregression's only used tens of times a day by a dozen or so clients, it ought not trouble even the narrowest of analyses unless someone goes looking for the data.

Comment 6

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:zeid, could you have a look please?

For more information, please visit BugBot documentation.