Why isn't a failing `test_no_expired_metrics` failing builds?
Categories
(Toolkit :: Telemetry, defect)
Tracking
()
People
(Reporter: chutten, Unassigned)
References
Details
bug 1832454 and bug 1832459: neither removed their expired metrics. Prior to bug 1696916 that would fail the build. Since bug 1696916, that fails the python test test_no_expired_metrics.
So how have expired metrics persisted in the tree? Is test_no_expired_metrics not run at the appropriate Tier? Is this expected/desired?
For me it isn't expected or desired as it is the Glean Team's opinion that expired metrics should be forbidden from remaining in the tree. Prompt cleanup is part of good data hygiene. But maybe there are reasons of tree greenliness that take precedence?
|
||
Comment 1•2 years ago
|
||
source-test-python-fog which includes test_no_expired_metrics only run if toolkit/components/glean/** has been modified. Therefore it doesn't run when the version gets increased and its failure will be noticed with delay.
The task has been added to the command for the version increase simulation today, and issues with future version increases should be detected starting with the version increase simulation for version 117 at the end of this week.
Expired probes remain in tree because the bugs to extend or remove them might not get prioritized. Automatic removal will fail if they are covered by a test - see the mda failures in bug 1832459 as example.
|
Reporter | |
Comment 2•2 years ago
|
||
This is excellently informative, thank you. With the addition of this test to the version increase simulations, I don't think there's much we'll be able to do here to encourage the metrics owners to tidy up (or at least nothing that won't be better done after the version increase simulation) so I'm leaning towards calling this solved.
Description
•