1415316 - Permaorange test-verify toolkit/components/telemetry/tests/browser/browser_DynamicScalars.js | The dynamic scalar must not be updated after being expired. - Got undefined, expected 3

Status: RESOLVED WONTFIX

(Toolkit :: Telemetry, defect, P1)

Description

[Treeherder Bug Filer]

Filed by: ebalazs [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=142761571&repo=autoland

https://queue.taskcluster.net/v1/task/NLQjrueuR32rq5hedy_s5A/runs/0/artifacts/public/logs/live_backing.log

Comment 1

[Alessio Placitelli [:Dexter]]

This is interesting :) On a first look, this doesn't seem like a problem. It's a "feature". The TV test suite runs the test, multiple times, within the same browsing session.

This test works like this:

- register a dynamic test scalar;
- accumulate some data to it;
- expire it;
- try to accumulate again;
- verify that we don't accumulate anymore after expiring.

After the first half of the test runs, a test scalar is dynamically expired. So, if the same test runs again in the test-suite, it won't be able to update the test scalar and the test will fail.

Aryx, given the behaviour of this test and how the TV suite runs, can I close this bug after waiting a few days to make sure that my theory still holds?

Comment 2

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Sure, alternatively is it possible to reset the behavior after "- verify that we don't accumulate anymore after expiring."?

If you set it to WONTFIX, sheriffs will star against the closed bug. We need bugs to classify these common issues so we don't have to spend time investigating each occurrence of this.

Comment 3

[Alessio Placitelli [:Dexter]]

(In reply to Sebastian Hengst [:aryx][:archaeopteryx] (needinfo on intermittent or backout) from comment #2)
> Sure, alternatively is it possible to reset the behavior after "- verify
> that we don't accumulate anymore after expiring."?

Thanks Sebastian!

That would require some changes to the Telemetry core and might be too involving for this particular case. Closing this as WONTFIX.

Comment 4

[Intermittent Failures Robot]

8 failures in 849 pushes (0.009 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 8

Platform breakdown:
* windows7-32: 3
* windows10-64: 3
* osx-10-10: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1415316&startday=2017-11-06&endday=2017-11-12&tree=all

Comment 5

[Narcis Beleuzu [:NarcisB]]

Fail reappeared on autoland

Log: https://treeherder.mozilla.org/logviewer.html#?job_id=164343384&repo=autoland&lineNumber=1434
Push: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=0dfc8b69311128b0b4cf356a6b7151daf77fa986

Comment 6

[Intermittent Failures Robot]

36 failures in 145 pushes (0.248 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* mozilla-inbound: 12
* mozilla-central: 12
* autoland: 12

Platform breakdown:
* linux64: 12
* windows7-32: 9
* windows10-64: 9
* osx-10-10: 6

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1415316&startday=2018-02-26&endday=2018-02-26&tree=all

Comment 7

[Intermittent Failures Robot]

31 failures in 831 pushes (0.037 failures/push) were associated with this bug in the last 7 days.   

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* autoland: 11
* mozilla-inbound: 10
* mozilla-central: 10

Platform breakdown:
* linux64: 11
* windows10-64: 8
* windows7-32: 7
* osx-10-10: 5

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1415316&startday=2018-02-26&endday=2018-03-04&tree=all