1541023 - Intermittent Reftest Automation Error: mozprocess timed out after 1000 seconds running ['/Users/cltbld/tasks/task_1554206825/build/venv/bin/python', '-u', '/Users/cltbld/tasks/task_1554206825/build/tests/reftest/runreftest.py', '--total-chunks', '3', '--t

Status: RESOLVED DUPLICATE of bug 1559729

(Testing :: Reftest, defect, P5)

Description

[Treeherder Bug Filer]

#[markdown(off)]
Filed by: nbeleuzu [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=237572998&repo=autoland

https://queue.taskcluster.net/v1/task/RybpnjVzRAap1phvasLzNw/runs/0/artifacts/public/logs/live_backing.log

https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/RybpnjVzRAap1phvasLzNw/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

12:17:41 INFO - Calling ['/Users/cltbld/tasks/task_1554206825/build/venv/bin/python', '-u', '/Users/cltbld/tasks/task_1554206825/build/tests/reftest/runreftest.py', '--total-chunks', '3', '--this-chunk', '2', '--appname=/Users/cltbld/tasks/task_1554206825/build/application/Firefox NightlyDebug.app/Contents/MacOS/firefox', '--utility-path=tests/bin', '--extra-profile-file=tests/bin/plugins', '--symbols-path=/Users/cltbld/tasks/task_1554206825/build/symbols', '--log-raw=/Users/cltbld/tasks/task_1554206825/build/blobber_upload_dir/reftest_raw.log', '--log-errorsummary=/Users/cltbld/tasks/task_1554206825/build/blobber_upload_dir/reftest_errorsummary.log', '--cleanup-crashes', '--marionette-startup-timeout=180', '--sandbox-read-whitelist=/Users/cltbld/tasks/task_1554206825/build', '--log-raw=-', '--suite=reftest', '--', 'tests/reftest/tests/layout/reftests/reftest.list'] with output_timeout 1000
12:17:41 INFO - REFTEST INFO | Removed pending crash reports at '/Users/cltbld/Library/Application Support/firefox/Crash Reports'
12:17:41 INFO - REFTEST INFO | Running with e10s: True
12:17:41 INFO - REFTEST INFO | Application command: /Users/cltbld/tasks/task_1554206825/build/application/Firefox NightlyDebug.app/Contents/MacOS/firefox -marionette -foreground -profile /var/folders/lt/dxhfr33n251br7y2br2845j400000x/T/tmptPW_CZ.mozrunner
12:34:21 INFO - Automation Error: mozprocess timed out after 1000 seconds running ['/Users/cltbld/tasks/task_1554206825/build/venv/bin/python', '-u', '/Users/cltbld/tasks/task_1554206825/build/tests/reftest/runreftest.py', '--total-chunks', '3', '--this-chunk', '2', '--appname=/Users/cltbld/tasks/task_1554206825/build/application/Firefox NightlyDebug.app/Contents/MacOS/firefox', '--utility-path=tests/bin', '--extra-profile-file=tests/bin/plugins', '--symbols-path=/Users/cltbld/tasks/task_1554206825/build/symbols', '--log-raw=/Users/cltbld/tasks/task_1554206825/build/blobber_upload_dir/reftest_raw.log', '--log-errorsummary=/Users/cltbld/tasks/task_1554206825/build/blobber_upload_dir/reftest_errorsummary.log', '--cleanup-crashes', '--marionette-startup-timeout=180', '--sandbox-read-whitelist=/Users/cltbld/tasks/task_1554206825/build', '--log-raw=-', '--suite=reftest', '--', 'tests/reftest/tests/layout/reftests/reftest.list']
12:34:21 ERROR - timed out after 1000 seconds of no output
12:34:21 ERROR - Return code: -15
12:34:21 ERROR - No checks run.
12:34:21 ERROR - No suite end message was emitted by this harness.
12:34:21 INFO - TinderboxPrint: reftest-reftest<br/><em class="testfail">T-FAIL</em>
12:34:21 ERROR - # TBPL FAILURE #
12:34:21 WARNING - setting return code to 2
12:34:21 ERROR - The reftest suite: reftest ran with return status: FAILURE
12:34:21 INFO - Running post-action listener: _package_coverage_data
12:34:21 INFO - Running post-action listener: _resource_record_post_action
12:34:21 INFO - Running post-action listener: process_java_coverage_data
12:34:21 INFO - [mozharness: 2019-04-02 12:34:21.887913Z] Finished run-tests step (success)
12:34:21 INFO - Running post-run listener: _resource_record_post_run
12:34:22 INFO - Validating Perfherder data against /Users/cltbld/tasks/task_1554206825/mozharness/external_tools/performance-artifact-schema.json
12:34:22 INFO - PERFHERDER_DATA: {"framework": {"name": "job_resource_usage"}, "suites": [{"subtests": [{"name": "cpu_percent", "value": 35.571473735408574}, {"name": "io_write_bytes", "value": 304854016}, {"name": "io.read_bytes", "value": 14950400}, {"name": "io_write_time", "value": 764}, {"name": "io_read_time", "value": 90}], "extraOptions": ["e10s", "buildbot-unknown"], "name": "reftest.reftest.reftest.2.overall"}, {"subtests": [{"name": "time", "value": 26.393692016601562}, {"name": "cpu_percent", "value": 53.8048076923077}], "name": "reftest.reftest.reftest.2.install"}, {"subtests": [{"name": "time", "value": 0.0002849102020263672}], "name": "reftest.reftest.reftest.2.stage-files"}, {"subtests": [{"name": "time", "value": 1002.8299400806427}, {"name": "cpu_percent", "value": 35.07922077922078}], "name": "reftest.reftest.reftest.2.run-tests"}]}
12:34:22 INFO - Total resource usage - Wall time: 1028s; CPU: 35.0%; Read bytes: 14950400; Write bytes: 304854016; Read time: 90; Write time: 764
12:34:22 INFO - TinderboxPrint: CPU usage<br/>35.1%
12:34:22 INFO - TinderboxPrint: I/O read bytes / time<br/>14,950,400 / 90
12:34:22 INFO - TinderboxPrint: I/O write bytes / time<br/>304,854,016 / 764
12:34:22 INFO - TinderboxPrint: CPU idle<br/>2,650.1 (64.4%)
12:34:22 INFO - TinderboxPrint: CPU system<br/>935.6 (22.7%)
12:34:22 INFO - TinderboxPrint: CPU user<br/>527.6 (12.8%)
12:34:22 INFO - TinderboxPrint: Swap in / out<br/>241,524,736 / 0
12:34:22 INFO - install - Wall time: 26s; CPU: 54.0%; Read bytes: 222078976; Write bytes: 216364544; Read time: 20813; Write time: 434
12:34:22 INFO - stage-files - Wall time: 0s; CPU: Can't collect data; Read bytes: 0; Write bytes: 0; Read time: 0; Write time: 0
12:34:22 INFO - run-tests - Wall time: 1003s; CPU: 35.0%; Read bytes: 569344; Write bytes: 61230592; Read time: 11; Write time: 269
12:34:22 WARNING - returning nonzero exit status 2
[taskcluster 2019-04-02T12:34:22.442Z] Exit Code: 2
[taskcluster 2019-04-02T12:34:22.442Z] User Time: 1m33.658805s
[taskcluster 2019-04-02T12:34:22.442Z] Kernel Time: 29.481495s
[taskcluster 2019-04-02T12:34:22.442Z] Wall Time: 19m56.526035195s
[taskcluster 2019-04-02T12:34:22.442Z] Result: FAILED
[taskcluster 2019-04-02T12:34:22.442Z] === Task Finished ===
[taskcluster 2019-04-02T12:34:22.442Z] Task Duration: 19m56.52621271s
[taskcluster 2019-04-02T12:34:23.096Z] Uploading artifact public/logs/localconfig.json from file logs/localconfig.json with content encoding "gzip", mime type "application/json" and expiry 2020-04-01T11:55:20.349Z
[taskcluster 2019-04-02T12:34:23.620Z] Uploading artifact public/test_info/reftest_errorsummary.log from file build/blobber_upload_dir/reftest_errorsummary.log with content encoding "gzip", mime type "text/plain" and expiry 2020-04-01T11:55:20.349Z
[taskcluster 2019-04-02T12:34:24.020Z] Uploading artifact public/test_info/reftest_raw.log from file build/blobber_upload_dir/reftest_raw.log with content encoding "gzip", mime type "text/plain" and expiry 2020-04-01T11:55:20.349Z
[taskcluster 2019-04-02T12:34:24.417Z] Uploading artifact public/test_info/resource-usage.json from file build/blobber_upload_dir/resource-usage.json with content encoding "gzip", mime type "application/json" and expiry 2020-04-01T11:55:20.349Z
[taskcluster 2019-04-02T12:34:25.014Z] Uploading artifact public/test_info/system-info.log from file build/blobber_upload_dir/system-info.log with content encoding "gzip", mime type "text/plain" and expiry 2020-04-01T11:55:20.349Z
[taskcluster:error] exit status 2

Comment 1

[Intermittent Failures Robot]

1 failures in 2843 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• macosx64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1541023&startday=2019-04-01&endday=2019-04-07&tree=all

Comment 2

[Intermittent Failures Robot]

2 failures in 3025 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1
• autoland: 1

Platform breakdown:

• windows10-64: 1
• windows7-32: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1541023&startday=2019-04-08&endday=2019-04-14&tree=all

Comment 3

[Intermittent Failures Robot]

1 failures in 3031 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform breakdown:

• windows10-64-ccov: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1541023&startday=2019-04-22&endday=2019-04-28&tree=all

Comment 4

[Intermittent Failures Robot]

1 failures in 4705 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-inbound: 1

Platform breakdown:

• osx-10-10: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1541023&startday=2019-05-06&endday=2019-05-12&tree=all

Comment 5

[Intermittent Failures Robot]

1 failures in 4730 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• windows10-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1541023&startday=2019-05-20&endday=2019-05-26&tree=all

Comment 6

[Intermittent Failures Robot]

1 failures in 4337 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-inbound: 1

Platform breakdown:

• windows10-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1541023&startday=2019-05-27&endday=2019-06-02&tree=all

Comment 7

[Intermittent Failures Robot]

2 failures in 4753 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• mozilla-central: 1

Platform breakdown:

• linux64: 1
• windows10-64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1541023&startday=2019-06-10&endday=2019-06-16&tree=all

Comment 8

[Cosmin Sabou [:CosminS]]

Let's follow these failures in Bug 1559729 cause it has more failures starred against it. Will reclassify the ones here.