Closed Bug 1677896 Opened 10 months ago Closed 10 months ago

Perma Android browsertime KeyError: 'speedometer'

Categories

(Testing :: Raptor, defect, P5)

defect

Tracking

(firefox85 fixed)

RESOLVED FIXED
85 Branch
Tracking Status
firefox85 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: sparky)

References

(Regression)

Details

(Keywords: intermittent-failure)

Attachments

(1 file)

Filed by: csabou [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=322069365&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/M7HMD7jRSg2xcwPCNFoJIQ/runs/0/artifacts/public/logs/live_backing.log


[task 2020-11-17T19:29:53.562Z] 19:29:53     INFO -  raptor-browsertime Info: Wrote data to blobber_upload_dir/browsertime-results/speedometer
[task 2020-11-17T19:29:53.562Z] 19:29:53     INFO -  raptor-browsertime-android Info: removing reverse socket connections
[task 2020-11-17T19:29:53.562Z] 19:29:53     INFO -  adb command_output: adb -s HT7A21A02882 wait-for-device reverse --remove-all, timeout: None, timedout: None, exitcode: 0, output:
[task 2020-11-17T19:29:53.562Z] 19:29:53     INFO -  perftest-results-handler Info: retrieving browsertime test results
[task 2020-11-17T19:29:53.562Z] 19:29:53     INFO -  perftest-results-handler Info: found browsertime results at /builds/task_160564119676176/workspace/build/blobber_upload_dir/browsertime-results/speedometer/browsertime.json
[task 2020-11-17T19:29:53.562Z] 19:29:53     INFO -  perftest-results-handler Info: parsing results from browsertime json
[task 2020-11-17T19:29:53.562Z] 19:29:53     INFO -  perftest-results-handler Info: parsed new benchmark result: {'statistics': {'dcf': {u'p99': 133, u'mdev': 0, u'min': 133, u'p90': 133, u'max': 133, u'median': 133, u'p10': 133, u'stddev': 0, u'mean': 133}, 'fnbpaint': {u'p99': 133, u'mdev': 0, u'min': 133, u'p90': 133, u'max': 133, u'median': 133, u'p10': 133, u'stddev': 0, u'mean': 133}, 'loadtime': {u'p99': 132, u'mdev': 0, u'min': 132, u'p90': 132, u'max': 132, u'median': 132, u'p10': 132, u'stddev': 0, u'mean': 132}}, 'name': u'speedometer', 'url': (u'http://127.0.0.1:39477/Speedometer/index.html?raptor',), 'lower_is_better': False, 'measurements': {'dcf': [133], 'fnbpaint': [133], 'loadtime': [132]}, 'subtest_unit': u'ms', 'extra_options': ['nocondprof', 'fenix'], 'bt_ver': u'9.4.1', 'alert_threshold': u'2.0', 'subtest_lower_is_better': True, 'cold': False, 'type': 'benchmark', 'unit': u'score', 'browser': {u'userAgent': u'Mozilla/5.0 (Android 8.0.0; Mobile; rv:84.0) Gecko/84.0 Firefox/84.0', u'windowSize': u'414x664'}}
[task 2020-11-17T19:29:53.563Z] 19:29:53     INFO -  perftest-output Info: preparing browsertime results for output
[task 2020-11-17T19:29:53.563Z] 19:29:53    ERROR -  Traceback (most recent call last):
[task 2020-11-17T19:29:53.563Z] 19:29:53     INFO -    File "/builds/task_160564119676176/workspace/build/tests/raptor/raptor/raptor.py", line 205, in <module>
[task 2020-11-17T19:29:53.563Z] 19:29:53     INFO -      main()
[task 2020-11-17T19:29:53.563Z] 19:29:53     INFO -    File "/builds/task_160564119676176/workspace/build/tests/raptor/raptor/raptor.py", line 151, in main
[task 2020-11-17T19:29:53.563Z] 19:29:53     INFO -      success = raptor.run_tests(raptor_test_list, raptor_test_names)
[task 2020-11-17T19:29:53.563Z] 19:29:53     INFO -    File "/builds/task_160564119676176/workspace/build/tests/raptor/raptor/browsertime/android.py", line 195, in run_tests
[task 2020-11-17T19:29:53.563Z] 19:29:53     INFO -      return super(BrowsertimeAndroid, self).run_tests(tests, test_names)
[task 2020-11-17T19:29:53.563Z] 19:29:53     INFO -    File "/builds/task_160564119676176/workspace/build/tests/raptor/raptor/perftest.py", line 399, in run_tests
[task 2020-11-17T19:29:53.563Z] 19:29:53     INFO -      return self.process_results(tests, test_names)
[task 2020-11-17T19:29:53.563Z] 19:29:53     INFO -    File "/builds/task_160564119676176/workspace/build/tests/raptor/raptor/perftest.py", line 427, in process_results
[task 2020-11-17T19:29:53.563Z] 19:29:53     INFO -      return self.results_handler.summarize_and_output(self.config, tests, test_names)
[task 2020-11-17T19:29:53.563Z] 19:29:53     INFO -    File "/builds/task_160564119676176/workspace/build/tests/raptor/raptor/results.py", line 810, in summarize_and_output
[task 2020-11-17T19:29:53.563Z] 19:29:53     INFO -      output.summarize(test_names)
[task 2020-11-17T19:29:53.563Z] 19:29:53     INFO -    File "/builds/task_160564119676176/workspace/build/tests/raptor/raptor/output.py", line 1533, in summarize
[task 2020-11-17T19:29:53.563Z] 19:29:53     INFO -      subtests, vals = self.parseSpeedometerOutput(test)
[task 2020-11-17T19:29:53.563Z] 19:29:53     INFO -    File "/builds/task_160564119676176/workspace/build/tests/raptor/raptor/output.py", line 455, in parseSpeedometerOutput
[task 2020-11-17T19:29:53.563Z] 19:29:53     INFO -      data = test["measurements"]["speedometer"]
[task 2020-11-17T19:30:04.579Z] 19:29:53     INFO -  KeyError: 'speedometer'
[task 2020-11-17T19:30:04.579Z] 19:29:53    ERROR - Return code: 1
[task 2020-11-17T19:30:04.579Z] 19:29:53  WARNING - setting return code to 1
[task 2020-11-17T19:30:04.579Z] 19:29:53     INFO - Killing logcat pid 795.
[task 2020-11-17T19:30:04.579Z] 19:29:53     INFO - Copying Raptor results to upload dir:
[task 2020-11-17T19:30:04.579Z] 19:29:53     INFO - /builds/task_160564119676176/workspace/build/blobber_upload_dir/perfherder-data.json
[task 2020-11-17T19:30:04.579Z] 19:29:53     INFO - Copying raptor results from /builds/task_160564119676176/workspace/build/raptor.json to /builds/task_160564119676176/workspace/build/blobber_upload_dir/perfherder-data.json
[task 2020-11-17T19:30:04.579Z] 19:29:53 CRITICAL - Error copying results /builds/task_160564119676176/workspace/build/raptor.json to upload dir /builds/task_160564119676176/workspace/build/blobber_upload_dir/perfherder-data.json
[task 2020-11-17T19:30:04.579Z] 19:29:53     INFO - [Errno 2] No such file or directory: u'/builds/task_160564119676176/workspace/build/raptor.json'
[task 2020-11-17T19:30:04.579Z] 19:29:53     INFO - Running post-action listener: _package_coverage_data```
Summary: Perma Android 8.0 Pixel2 AArch64 browsertime KeyError: 'speedometer' → Perma Android browsertime KeyError: 'speedometer'
See Also: → 1677982
Assignee: nobody → gmierz2
Status: NEW → ASSIGNED
Regressed by: 1562642
Pushed by gmierz2@outlook.com:
https://hg.mozilla.org/integration/autoland/rev/561b182ae314
Remove license header from raptor benchmark script. r=perftest-reviewers,tarek
Duplicate of this bug: 1677982
Status: ASSIGNED → RESOLVED
Closed: 10 months ago
Resolution: --- → FIXED
Target Milestone: --- → 85 Branch
You need to log in before you can comment on or make changes to this bug.