1548407 - Intermittent raptor-main TEST-UNEXPECTED-FAIL: test 'raptor-tp6-amazon-firefox-cold' timed out loading test page: https://www.amazon.com/s/url=search-alias%3Daps&field-keywords=laptop pending metrics: fcp, fnb paint, dcf, load time

Status: RESOLVED INCOMPLETE

(Testing :: Raptor, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: csabou [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=244030568&repo=autoland
Full log: https://queue.taskcluster.net/v1/task/RAeXjHSsTfOi_QOgk2pcKQ/runs/0/artifacts/public/logs/live_backing.log

---

23:00:23 INFO - raptor-output combining results from browser cycle 2 for raptor-tp6-amazon-firefox-cold
23:00:23 INFO - raptor-output adding replicates for dcf
23:00:23 INFO - raptor-output adding replicates for fcp
23:00:23 INFO - raptor-output adding replicates for fnbpaint
23:00:23 INFO - raptor-output adding replicates for loadtime
23:00:23 INFO - raptor-output screen captures can be found locally at: C:\Users\task_1556747733\build\screenshots.html
23:00:23 INFO - raptor-output PERFHERDER_DATA: {"framework": {"name": "raptor"}, "suites": [{"extraOptions": [], "name": "raptor-tp6-amazon-firefox-cold", "lowerIsBetter": true, "alertThreshold": 2.0, "value": 718.1, "subtests": [{"name": "dcf", "lowerIsBetter": true, "alertThreshold": 2.0, "value": 661.0, "replicates": [1205, 752, 630, 559, 1135, 929, 889, 936, 1118, 567, 587, 566, 586, 560, 969, 668, 829, 598, 582, 570, 694, 600, 654, 873], "unit": "ms"}, {"name": "fcp", "lowerIsBetter": true, "alertThreshold": 2.0, "value": 575.0, "shouldAlert": true, "replicates": [1412, 607, 575, 787, 1022, 765, 764, 537, 1094, 810, 526, 522, 551, 534, 790, 549, 811, 534, 546, 514, 575, 535, 535, 737], "unit": "ms"}, {"name": "fnbpaint", "lowerIsBetter": true, "alertThreshold": 2.0, "value": 520.0, "replicates": [1199, 544, 530, 553, 779, 514, 718, 490, 840, 562, 491, 483, 486, 487, 528, 502, 758, 488, 496, 477, 526, 486, 479, 694], "unit": "ms"}, {"name": "loadtime", "lowerIsBetter": true, "alertThreshold": 2.0, "value": 1345.0, "shouldAlert": true, "replicates": [2137, 1346, 1374, 1288, 1586, 1340, 1589, 1344, 1596, 1343, 1306, 1335, 1384, 1260, 1334, 1309, 1487, 1407, 1346, 1300, 1313, 1391, 1343, 1578], "unit": "ms"}], "type": "pageload", "unit": "ms"}]}
23:00:23 INFO - raptor-output results can also be found locally at: C:\Users\task_1556747733\build\raptor.json
23:00:23 INFO - raptor-control-server shutting down control server
23:00:23 INFO - raptor-main finished
23:00:23 INFO - raptor-main TEST-UNEXPECTED-FAIL: test 'raptor-tp6-amazon-firefox-cold' timed out loading test page: https://www.amazon.com/s/url=search-alias%3Daps&field-keywords=laptop pending metrics: fcp, fnb paint, dcf, load time
23:00:23 ERROR - Return code: 1
23:00:23 WARNING - setting return code to 1
23:00:23 INFO - Validating PERFHERDER_DATA against C:\Users\task_1556747733\mozharness\external_tools\performance-artifact-schema.json
23:00:23 INFO - copying raptor results to upload dir:
23:00:23 INFO - C:\Users\task_1556747733\build\blobber_upload_dir\perfherder-data.json
23:00:23 INFO - copying raptor results from C:\Users\task_1556747733\build\raptor.json to C:\Users\task_1556747733\build\blobber_upload_dir\perfherder-data.json
23:00:23 INFO - C:\Users\task_1556747733\build\blobber_upload_dir\screenshots.html
23:00:23 INFO - copying raptor results from C:\Users\task_1556747733\build\screenshots.html to C:\Users\task_1556747733\build\blobber_upload_dir\screenshots.html
23:00:23 INFO - Running post-action listener: _package_coverage_data
23:00:23 INFO - Running post-action listener: _resource_record_post_action
23:00:23 INFO - Running post-action listener: process_java_coverage_data
23:00:23 INFO - Running post-action listener: stop_device
23:00:23 INFO - [mozharness: 2019-05-01 23:00:23.389000Z] Finished run-tests step (success)

Comment 1

[Intermittent Failures Robot]

32 failures in 933 pushes (0.034 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 22
• mozilla-central: 5
• mozilla-inbound: 4
• try: 1

Platform breakdown:

• linux64-shippable-qr: 7
• windows10-64-qr: 1
• windows7-32-shippable: 8
• windows10-64-shippable: 3
• osx-10-10-shippable: 4
• windows10-64-shippable-qr: 3
• linux64-shippable: 5
• windows10-aarch64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1548407&startday=2019-05-02&endday=2019-05-02&tree=all

Comment 2

[Oana Pop-Rus]

Since bug 1522972, which added "raptor-tp6-amazon-firefox-cold", landed 2 days ago : https://hg.mozilla.org/integration/autoland/rev/918567a07dbb60dafc2b971afc4ce1610992d069, this has 45 failures.

Treeherder range: https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=windows%2C10%2Cx64%2Cquantumrender%2Cshippable%2Copt%2Craptor%2Cperformance%2Ctests%2Con%2Cfirefox%2Ctest-windows10-64-shippable-qr%2Fopt-raptor-tp6-1-firefox-cold-e10s%2Crap%28tp6-c-1%29&tochange=f08eb5f2a017499fc2d1b8704fc1c62ffdca5f35&fromchange=64a0de86aa876d39b49985f947895620bbc99052&selectedJob=244030568&group_state=expanded

Could you please take a look?

Comment 4

[Andreea Pavel [:apavel]]

Reopen based on https://bugzilla.mozilla.org/show_bug.cgi?id=1539184#c55

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=244772762&repo=mozilla-inbound&lineNumber=1643

23:57:11 INFO - raptor-main starting firefox
23:57:11 INFO - Application command: C:\Users\task_1557093586\build\application\firefox\firefox.exe --wait-for-browser -profile c:\users\task_1557093586\appdata\local\temp\tmprtdkid.mozrunner
23:57:12 INFO - raptor-control-server received webext_status: raptor runner.js is loaded!
23:57:12 INFO - raptor-control-server reading test settings from raptor-tp6-amazon-firefox-cold.json

23:58:53 INFO - raptor-output combining results from browser cycle 2 for raptor-tp6-amazon-firefox-cold
23:58:53 INFO - raptor-output adding replicates for dcf
23:58:53 INFO - raptor-output adding replicates for fcp
23:58:53 INFO - raptor-output adding replicates for fnbpaint
23:58:53 INFO - raptor-output adding replicates for loadtime
23:58:53 INFO - raptor-output screen captures can be found locally at: C:\Users\task_1557093586\build\screenshots.html
23:58:53 INFO - raptor-output PERFHERDER_DATA: {"framework": {"name": "raptor"}, "suites": [{"extraOptions": [], "name": "raptor-tp6-amazon-firefox-cold", "lowerIsBetter": true, "alertThreshold": 2.0, "value": 723.82, "subtests": [{"name": "dcf", "lowerIsBetter": true, "alertThreshold": 2.0, "value": 633.5, "replicates": [1001, 615, 592, 596, 637, 611, 608, 594, 653, 684, 694, 635, 588, 600, 636, 721, 683, 677, 685, 660, 607, 607, 594, 632], "unit": "ms"}, {"name": "fcp", "lowerIsBetter": true, "alertThreshold": 2.0, "value": 603.0, "shouldAlert": true, "replicates": [1015, 572, 599, 598, 599, 619, 572, 552, 659, 692, 629, 597, 532, 612, 594, 679, 628, 676, 617, 676, 568, 545, 607, 597], "unit": "ms"}, {"name": "fnbpaint", "lowerIsBetter": true, "alertThreshold": 2.0, "value": 547.5, "replicates": [950, 524, 541, 531, 543, 558, 522, 504, 604, 628, 581, 549, 484, 549, 543, 628, 579, 626, 569, 612, 520, 498, 542, 546], "unit": "ms"}, {"name": "loadtime", "lowerIsBetter": true, "alertThreshold": 2.0, "value": 1312.0, "shouldAlert": true, "replicates": [1876, 1284, 1292, 1259, 1364, 1323, 1338, 1350, 1362, 1384, 1360, 1206, 1189, 1251, 1173, 1309, 1376, 1340, 1315, 1386, 1215, 1205, 1222, 1222], "unit": "ms"}], "type": "pageload", "unit": "ms"}]}
23:58:53 INFO - raptor-output results can also be found locally at: C:\Users\task_1557093586\build\raptor.json
23:58:53 INFO - raptor-control-server shutting down control server
23:58:53 INFO - raptor-main finished
23:58:53 INFO - raptor-main TEST-UNEXPECTED-FAIL: test 'raptor-tp6-amazon-firefox-cold' timed out loading test page: https://www.amazon.com/s/url=search-alias%3Daps&field-keywords=laptop pending metrics: fcp, fnb paint, dcf, load time
23:58:54 ERROR - Return code: 1

Comment 5

[Intermittent Failures Robot]

1 failures in 5197 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform breakdown:

• windows10-64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1548407&startday=2019-06-03&endday=2019-06-09&tree=all

Comment 6

[Intermittent Failures Robot]

3 failures in 4753 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1
• mozilla-central: 2

Platform breakdown:

• linux64-shippable: 1
• windows10-64-ux: 1
• windows10-aarch64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1548407&startday=2019-06-10&endday=2019-06-16&tree=all

Comment 7

[Intermittent Failures Robot]

1 failures in 2581 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-inbound: 1

Platform breakdown:

• macosx1014-64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1548407&startday=2019-06-17&endday=2019-06-23&tree=all

Comment 8

[Intermittent Failures Robot]

6 failures in 4409 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 6

Platform breakdown:

• windows10-64-shippable-qr: 1
• linux64-shippable-qr: 1
• linux64-shippable: 2
• windows10-64-shippable: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1548407&startday=2019-06-24&endday=2019-06-30&tree=all

Comment 9

[Firefox Bug Husbandry Bot]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup