Fix `KeyError: suites` when a tp6 page times out

RESOLVED FIXED in Firefox 68

Status

defect
P1
normal
RESOLVED FIXED
5 months ago
5 months ago

People

(Reporter: rwood, Assigned: rwood)

Tracking

(Blocks 1 bug)

Version 3
mozilla68
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox68 fixed)

Details

Attachments

(1 attachment, 1 obsolete attachment)

The cold page-load work (landed in Bug 1533423) has introduced a new error when a tp6 page times out:

23:16:42 INFO - raptor-control-server received webext_raptor-page-timeout: [u'raptor-tp6m-google-maps-geckoview', u'https://www.google.com/maps?force=pwa', {u'fcp': True, u'hero': False, u'dcf': False, u'fnb paint': False, u'ttfi': False, u'load time': False}]
...
23:16:43 INFO - results-handler summarizing raptor test results
23:16:43 INFO - raptor-output error: no raptor test results found for raptor-tp6m-google-maps-geckoview
23:16:43 ERROR - Traceback (most recent call last):
23:16:43 INFO - File "/builds/worker/workspace/build/tests/raptor/raptor/raptor.py", line 869, in <module>
23:16:43 INFO - main()
23:16:43 INFO - File "/builds/worker/workspace/build/tests/raptor/raptor/raptor.py", line 837, in main
23:16:43 INFO - success = raptor.process_results(raptor_test_names)
23:16:43 INFO - File "/builds/worker/workspace/build/tests/raptor/raptor/raptor.py", line 342, in process_results
23:16:43 INFO - return self.results_handler.summarize_and_output(self.config, test_names)
23:16:43 INFO - File "/builds/worker/workspace/build/tests/raptor/raptor/results.py", line 80, in summarize_and_output
23:16:43 INFO - output.combine_browser_cycles()
23:16:43 INFO - File "/builds/worker/workspace/build/tests/raptor/raptor/output.py", line 214, in combine_browser_cycles
23:16:43 INFO - for _index, suite in enumerate(self.summarized_results['suites']):
23:16:43 INFO - KeyError: 'suites'

Blocks: 1504757

Also as part of this patch I am slipping in a fix - I noticed when providing --browser-cycles on the cmd line it didn't override the browser-cycles INI setting.

Backed out changeset 030cb27161ee (Bug 1538786) for raptor failures in testing/raptor/test/test_manifest.py::test_get_raptor_test_list_firefox CLOSED TREE

Failure: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=unclassified&fromchange=f5dd833c3b60b3365d2c49d930748e33a22651b6&selectedJob=236013041

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=236013041&repo=autoland&lineNumber=364

[task 2019-03-26T02:47:03.962Z] 0:23.97 ===================== 2 passed, 1 skipped in 0.28 seconds ======================
[task 2019-03-26T02:47:04.153Z] 0:24.16 /builds/worker/checkouts/gecko/testing/raptor/test/test_manifest.py
[task 2019-03-26T02:47:04.153Z] 0:24.16 ============================= test session starts ==============================
[task 2019-03-26T02:47:04.153Z] 0:24.16 platform linux2 -- Python 2.7.12, pytest-3.6.2, py-1.5.4, pluggy-0.6.0 -- /builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_virtualenvs/gecko-QUm8IANb-2/bin/python
[task 2019-03-26T02:47:04.153Z] 0:24.16 rootdir: /builds/worker/checkouts/gecko, inifile: /builds/worker/checkouts/gecko/config/mozunit/mozunit/pytest.ini
[task 2019-03-26T02:47:04.154Z] 0:24.16 collecting ... collected 19 items
[task 2019-03-26T02:47:04.154Z] 0:24.16
[task 2019-03-26T02:47:04.154Z] 0:24.16 testing/raptor/test/test_manifest.py::test_get_browser_test_list[firefox] PASSED
[task 2019-03-26T02:47:04.154Z] 0:24.16 testing/raptor/test/test_manifest.py::test_get_browser_test_list[chrome] PASSED
[task 2019-03-26T02:47:04.154Z] 0:24.16 testing/raptor/test/test_manifest.py::test_get_browser_test_list[geckoview] PASSED
[task 2019-03-26T02:47:04.155Z] 0:24.16 testing/raptor/test/test_manifest.py::test_get_browser_test_list[refbrow] PASSED
[task 2019-03-26T02:47:04.155Z] 0:24.16 testing/raptor/test/test_manifest.py::test_get_browser_test_list[fenix] PASSED
[task 2019-03-26T02:47:04.156Z] 0:24.16 testing/raptor/test/test_manifest.py::test_validate_test_ini_valid[test_details0] PASSED
[task 2019-03-26T02:47:04.156Z] 0:24.16 testing/raptor/test/test_manifest.py::test_validate_test_ini_valid[test_details1] PASSED
[task 2019-03-26T02:47:04.156Z] 0:24.16 testing/raptor/test/test_manifest.py::test_validate_test_ini_invalid[test_details0] PASSED
[task 2019-03-26T02:47:04.156Z] 0:24.16 testing/raptor/test/test_manifest.py::test_validate_test_ini_invalid[test_details1] PASSED
[task 2019-03-26T02:47:04.156Z] 0:24.16 testing/raptor/test/test_manifest.py::test_validate_test_ini_invalid[test_details2] PASSED
[task 2019-03-26T02:47:04.156Z] 0:24.16 testing/raptor/test/test_manifest.py::test_get_raptor_test_list_firefox TEST-UNEXPECTED-FAIL
[task 2019-03-26T02:47:04.156Z] 0:24.16 testing/raptor/test/test_manifest.py::test_get_raptor_test_list_chrome TEST-UNEXPECTED-FAIL
[task 2019-03-26T02:47:04.156Z] 0:24.16 testing/raptor/test/test_manifest.py::test_get_raptor_test_list_geckoview TEST-UNEXPECTED-FAIL
[task 2019-03-26T02:47:04.156Z] 0:24.16 testing/raptor/test/test_manifest.py::test_get_raptor_test_list_gecko_profiling TEST-UNEXPECTED-FAIL
[task 2019-03-26T02:47:04.156Z] 0:24.16 testing/raptor/test/test_manifest.py::test_get_raptor_test_list_debug_mode TEST-UNEXPECTED-FAIL
[task 2019-03-26T02:47:04.156Z] 0:24.16 testing/raptor/test/test_manifest.py::test_get_raptor_test_list_override_page_cycles TEST-UNEXPECTED-FAIL
[task 2019-03-26T02:47:04.156Z] 0:24.16 testing/raptor/test/test_manifest.py::test_get_raptor_test_list_override_page_timeout TEST-UNEXPECTED-FAIL
[task 2019-03-26T02:47:04.157Z] 0:24.16 testing/raptor/test/test_manifest.py::test_get_raptor_test_list_refbrow TEST-UNEXPECTED-FAIL
[task 2019-03-26T02:47:04.157Z] 0:24.16 testing/raptor/test/test_manifest.py::test_get_raptor_test_list_fenix PASSED

Flags: needinfo?(rwood)
Backout by rgurzau@mozilla.com:
https://hg.mozilla.org/mozilla-central/rev/24f40f4b59ac
Backed out changeset 030cb27161ee for raptor failures in testing/raptor/test/test_manifest.py::test_get_raptor_test_list_firefox CLOSED TREE

(In reply to Stefan Hindli [:stefan_hindli] from comment #5)

Backed out changeset 030cb27161ee (Bug 1538786) for raptor failures in testing/raptor/test/test_manifest.py::test_get_raptor_test_list_firefox CLOSED TREE

Apologies for the breakage, thank you for the backout Stefan.

Flags: needinfo?(rwood)
Attachment #9053340 - Attachment is obsolete: true
Pushed by dhunt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/526c37cbcb1f
Fix 'KeyError: suites' when a tp6 page times out; r=davehunt
Status: ASSIGNED → RESOLVED
Closed: 5 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla68
You need to log in before you can comment on or make changes to this bug.