Intermittent visual-metrics failure: ZeroDivisionError: float division by zero
Categories
(Testing :: Raptor, defect, P1)
Tracking
(firefox76 fixed)
| Tracking | Status | |
|---|---|---|
| firefox76 | --- | fixed |
People
(Reporter: sparky, Assigned: Bebe)
Details
(Whiteboard: [fenix-browsertime])
Attachments
(1 file)
[task 2020-03-21T01:28:23.131Z] 2020-03-21T01:28:23.131773Z Command succeeded result=b'{"SpeedIndex": 3720, "FirstVisualChange": 3720, "LastVisualChange": 6680, "VisualProgress": "0=0, 3720=100, 4080=100, 4520=100, 6680=100", "ContentfulSpeedIndex": 3729, "videoRecordingStart": 2040, "ContentfulSpeedIndexProgress": "0=0, 3720=99, 4080=99, 4520=99, 6680=100", "PerceptualSpeedIndex": 4088, "PerceptualSpeedIndexProgress": "0=0, 3720=90, 4080=91, 4520=93, 6680=100"}\n'
[task 2020-03-21T01:28:23.134Z] 2020-03-21T01:28:23.134409Z Unhandled exception: float division by zero
[task 2020-03-21T01:28:23.134Z] Traceback (most recent call last):
[task 2020-03-21T01:28:23.134Z] File "/builds/worker/bin/run-visual-metrics.py", line 339, in <module>
[task 2020-03-21T01:28:23.134Z] sys.exit(main(log, args))
[task 2020-03-21T01:28:23.134Z] File "/builds/worker/bin/run-visual-metrics.py", line 267, in main
[task 2020-03-21T01:28:23.134Z] suites = [get_suite(suite) for suite in suites.values()]
[task 2020-03-21T01:28:23.134Z] File "/builds/worker/bin/run-visual-metrics.py", line 267, in <listcomp>
[task 2020-03-21T01:28:23.134Z] suites = [get_suite(suite) for suite in suites.values()]
[task 2020-03-21T01:28:23.134Z] File "/builds/worker/bin/run-visual-metrics.py", line 141, in get_suite
[task 2020-03-21T01:28:23.134Z] compute_median(subtest) for subtest in suite["subtests"].values()
[task 2020-03-21T01:28:23.134Z] File "/builds/worker/bin/run-visual-metrics.py", line 141, in <listcomp>
[task 2020-03-21T01:28:23.134Z] compute_median(subtest) for subtest in suite["subtests"].values()
[task 2020-03-21T01:28:23.134Z] File "/builds/worker/bin/run-visual-metrics.py", line 127, in compute_median
[task 2020-03-21T01:28:23.134Z] subtest["value"] = float(sum(series)) / float(len(series))
[task 2020-03-21T01:28:23.134Z] ZeroDivisionError: float division by zero
|
||
Comment 1•4 years ago
|
||
When we have an empty series, we should bail out before doing the above division. Florin, can someone from your team fix that?
| Comment hidden (Intermittent Failures Robot) |
|
Assignee | |
Comment 3•4 years ago
|
||
what sould we do if we get a 0 metric?
A. Stop execution and throw error or
B. Return 0 and continue with the next metric
|
|
Assignee | |
Updated•4 years ago
|
|
Reporter | |
Comment 4•4 years ago
|
||
Redirecting ni? to :barret, would you know why we might be hitting this error?
|
|
||
Comment 5•4 years ago
|
||
So the failure is here:
https://searchfox.org/mozilla-central/rev/3fd53c47864fedb916f0ed8f002f15456324f729/taskcluster/docker/visual-metrics/run-visual-metrics.py#127
Has this anything to do with Raptor at all? If not what's the right component?
|
|
Reporter | |
Comment 6•4 years ago
|
||
It's related to raptor-browsertime and this is the right component. But rather than just throw an error when the series is empty (which we'll have to add), I want to know why it might be empty and fix that issue.
|
||
Comment 7•4 years ago
|
||
Looking at the code, the issue is specifically
series = subtest["replicates"][1:]
subtest["value"] = float(sum(series)) / float(len(series))
so the issue comes about when we only have a single replicate, not 0 replicates. From what I gather, this will happen if we only have a single video in a given browsertime.json.
:tarek, why are we excluding the first result from subtest["replicates"] to find the median? Also, looking closely, this is computing the mean of subtest[replicates][1:] and not the median.
|
||
Comment 8•4 years ago
|
||
yeah this looks wrong, I don't remember why we did this.
Since this script uses Python 3, we can drop that custom code and use https://docs.python.org/3/library/statistics.html#statistics.median
|
|
Assignee | |
Comment 9•4 years ago
|
||
|
||
Comment 10•4 years ago
|
||
Pushed by fstrugariu@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/55adb80c65b0 Intermittent visual-metrics failure: ZeroDivisionError: float division by zero r=tarek,sparky,barret
|
||
Comment 11•4 years ago
|
||
| bugherder | ||
Description
•