Intermittent Error copying results /builds/task_"id#"/workspace/build/raptor.json to upload dir /builds/task_"id#"/workspace/build/blobber_upload_dir/perfherder-data.json | [Errno 2] No such file or directory: '/builds/task_"id#"/<...>/raptor.json'
Categories
(Testing :: Raptor, defect, P5)
Tracking
(Not tracked)
People
(Reporter: intermittent-bug-filer, Unassigned)
Details
(Keywords: intermittent-failure)
Filed by: imoraru [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=364070866&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/WmrIjbX3RZOx1hXQMLI3fw/runs/0/artifacts/public/logs/live_backing.log
[task 2022-01-14T03:55:52.538Z] 03:55:48 INFO - raptor-perftest Info: starting test: imdb
[task 2022-01-14T03:55:52.538Z] 03:55:48 INFO - raptor-perftest Info: test uses playback tool: mitmproxy-android
[task 2022-01-14T03:55:52.538Z] 03:55:48 INFO - raptor-mitmproxy Info: mozproxy_dir used for mitmproxy downloads and exe files: /builds/task_164213225211120/workspace/testing/mozproxy
[task 2022-01-14T03:55:52.538Z] 03:55:48 INFO - raptor-mitmproxy Info: Playback tool: mitmproxy-android
[task 2022-01-14T03:55:52.538Z] 03:55:48 INFO - raptor-mitmproxy Info: Playback tool version: 6.0.2
[task 2022-01-14T03:55:52.538Z] 03:55:48 INFO - raptor-mitmproxy Info: create mitmproxy 6.0.2 dir
[task 2022-01-14T03:55:52.538Z] 03:55:48 INFO - raptor-mitmproxy Info: downloading mitmproxy binary
[task 2022-01-14T03:55:52.538Z] 03:55:48 INFO - raptor-mitmproxy Info: b'INFO - File mitmproxy-6.0.2-linux.tar.gz not present in local cache folder /builds/tooltool_cache'
[task 2022-01-14T03:55:52.538Z] 03:55:48 INFO - raptor-mitmproxy Info: b"INFO - Attempting to fetch from 'http://localhost:8099/tooltool.mozilla-releng.net/'..."
[task 2022-01-14T03:55:52.538Z] 03:55:50 INFO - raptor-mitmproxy Info: b'INFO - File mitmproxy-6.0.2-linux.tar.gz fetched from http://localhost:8099/tooltool.mozilla-releng.net/ as /builds/task_164213225211120/workspace/testing/mozproxy/mitmdump-6.0.2/tmpx8n3ibtj'
[task 2022-01-14T03:55:52.538Z] 03:55:51 INFO - raptor-mitmproxy Info: b'INFO - File integrity verified, renaming tmpx8n3ibtj to mitmproxy-6.0.2-linux.tar.gz'
[task 2022-01-14T03:55:52.538Z] 03:55:51 INFO - raptor-mitmproxy Info: b'INFO - Updating local cache /builds/tooltool_cache...'
[task 2022-01-14T03:55:52.538Z] 03:55:51 INFO - raptor-mitmproxy Info: b'INFO - Creating cache in /builds/tooltool_cache...'
[task 2022-01-14T03:55:52.538Z] 03:55:51 INFO - raptor-mitmproxy Info: b'INFO - Local cache /builds/tooltool_cache updated with mitmproxy-6.0.2-linux.tar.gz'
[task 2022-01-14T03:55:52.538Z] 03:55:51 INFO - raptor-mitmproxy Info: b'INFO - untarring "mitmproxy-6.0.2-linux.tar.gz"'
[task 2022-01-14T03:55:52.538Z] 03:55:51 INFO - raptor-mitmproxy Info: downloading mitmproxy pageset
[task 2022-01-14T03:55:52.538Z] 03:55:51 INFO - raptor-mitmproxy Info: b'INFO - File mitm6-android-fenix-imdb.zip not present in local cache folder /builds/tooltool_cache'
[task 2022-01-14T03:55:52.538Z] 03:55:51 INFO - raptor-mitmproxy Info: b"INFO - Attempting to fetch from 'http://localhost:8099/tooltool.mozilla-releng.net/'..."
[task 2022-01-14T03:55:52.538Z] 03:55:52 INFO - raptor-mitmproxy Info: b'INFO - File mitm6-android-fenix-imdb.zip fetched from http://localhost:8099/tooltool.mozilla-releng.net/ as /builds/task_164213225211120/workspace/testing/mozproxy/tmppib6d76y'
[task 2022-01-14T03:55:52.538Z] 03:55:52 INFO - raptor-mitmproxy Info: b'INFO - File integrity verified, renaming tmppib6d76y to mitm6-android-fenix-imdb.zip'
[task 2022-01-14T03:55:52.538Z] 03:55:52 INFO - raptor-mitmproxy Info: b'INFO - Updating local cache /builds/tooltool_cache...'
[task 2022-01-14T03:55:52.538Z] 03:55:52 INFO - raptor-mitmproxy Info: b'INFO - Local cache /builds/tooltool_cache updated with mitm6-android-fenix-imdb.zip'
[task 2022-01-14T03:55:52.538Z] 03:55:52 INFO - raptor-mitmproxy Info: Adding /builds/task_164213225211120/workspace/testing/mozproxy/mitm6-android-fenix-imdb.zip to recording list
[task 2022-01-14T03:55:52.538Z] 03:55:52 INFO - raptor-mitmproxy Info: Loaded recoording generated on 2021-04-19 10:32:42.317642
[task 2022-01-14T03:55:52.538Z] 03:55:52 INFO - raptor-mitmproxy Info: mitmdump path: /builds/task_164213225211120/workspace/testing/mozproxy/mitmdump-6.0.2/mitmdump
[task 2022-01-14T03:55:52.538Z] 03:55:52 INFO - raptor-mitmproxy Info: browser path: org.mozilla.geckoview_example
[task 2022-01-14T03:55:52.538Z] 03:55:52 INFO - raptor-mitmproxy Info: Starting mitmproxy playback using env path: /builds/task_164213225211120/workspace/build/venv/bin:/builds/task_164213225211120/workspace/build/venv/bin:/builds/task_164213225211120/workspace/build/venv/bin:/builds/worker/.pyenv/shims:/builds/worker/.pyenv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/builds/worker/bin:/usr/local/bin:/builds/worker/android-sdk-linux/platform-tools:/builds/worker/android-sdk-linux/platform-tools/bin:/builds/worker/android-sdk-linux/tools:/builds/worker/android-sdk-linux/tools/bin:/builds/worker/android-sdk-linux/build-tools/27.0.3:/builds/worker/android-sdk-linux/tools/bin:/builds/worker/android-sdk-linux/platform-tools
[task 2022-01-14T04:00:13.097Z] 03:55:52 INFO - raptor-mitmproxy Info: Starting mitmproxy playback using command: /builds/task_164213225211120/workspace/testing/mozproxy/mitmdump-6.0.2/mitmdump --listen-host 127.0.0.1 --listen-port 53989 --set upstream_cert=false --set upload_dir=/builds/task_164213225211120/workspace/build/blobber_upload_dir --set websocket=false --set server_replay_files=/builds/task_164213225211120/workspace/testing/mozproxy/mitm6-android-fenix-imdb/dump.mp --scripts /builds/task_164213225211120/workspace/build/venv/lib/python3.6/site-packages/mozproxy/backends/mitm/scripts/alternate-server-replay.py
[task 2022-01-14T04:00:13.143Z] 03:55:53 INFO - raptor-mitmproxy Info: Mitmproxy playback successfully started on 127.0.0.1:53989 as pid 1495
[task 2022-01-14T04:00:13.151Z] 03:55:53 INFO - raptor-mitmproxy Info: downloading certutil binary (hostutils)
[task 2022-01-14T04:00:13.178Z] 03:55:53 INFO - raptor-mitmproxy Info: downloading: https://hg.mozilla.org/integration/autoland/raw-file/344f14b85f84dabbc7709b5fc5d23e9471060d75/testing/config/tooltool-manifests/linux64/hostutils.manifest to /builds/task_164213225211120/workspace/testing/mozproxy/hostutils.manifest
[task 2022-01-14T04:00:13.194Z] 03:55:54 INFO - raptor-mitmproxy Info: b'INFO - File host-utils-93.0a1.en-US.linux-x86_64.tar.gz not present in local cache folder /builds/tooltool_cache'
[task 2022-01-14T04:00:13.194Z] 03:55:54 INFO - raptor-mitmproxy Info: b"INFO - Attempting to fetch from 'http://localhost:8099/tooltool.mozilla-releng.net/'..."
[task 2022-01-14T04:00:13.202Z] 03:56:00 INFO - raptor-mitmproxy Info: b'INFO - File host-utils-93.0a1.en-US.linux-x86_64.tar.gz fetched from http://localhost:8099/tooltool.mozilla-releng.net/ as /builds/task_164213225211120/workspace/testing/mozproxy/tmpfawcz38v'
[task 2022-01-14T04:00:13.202Z] 03:56:02 INFO - raptor-mitmproxy Info: b'INFO - File integrity verified, renaming tmpfawcz38v to host-utils-93.0a1.en-US.linux-x86_64.tar.gz'
[task 2022-01-14T04:00:13.202Z] 03:56:02 INFO - raptor-mitmproxy Info: b'INFO - Updating local cache /builds/tooltool_cache...'
[task 2022-01-14T04:00:13.202Z] 03:56:02 INFO - raptor-mitmproxy Info: b'INFO - Local cache /builds/tooltool_cache updated with host-utils-93.0a1.en-US.linux-x86_64.tar.gz'
[task 2022-01-14T04:00:13.202Z] 03:56:02 INFO - raptor-mitmproxy Info: b'INFO - untarring "host-utils-93.0a1.en-US.linux-x86_64.tar.gz"'
[task 2022-01-14T04:00:13.202Z] 03:56:04 INFO - raptor-mitmproxy Info: certutil is found at: /builds/task_164213225211120/workspace/testing/mozproxy/host-utils-93.0a1.en-US.linux-x86_64/certutil
[task 2022-01-14T04:00:13.202Z] 03:56:04 INFO - raptor-mitmproxy Info: checking if the nss cert db already exists in the android browser profile
[task 2022-01-14T04:00:13.203Z] 03:56:04 INFO - raptor-mitmproxy Info: Certutil: Running command: /builds/task_164213225211120/workspace/testing/mozproxy/host-utils-93.0a1.en-US.linux-x86_64/certutil -d sql:/tmp/tmpu5e9ty7e/profile/ -L
[task 2022-01-14T04:00:13.208Z] 03:56:04 INFO - raptor-mitmproxy Info: nss cert db doesn't exist yet.
[task 2022-01-14T04:00:13.208Z] 03:56:04 INFO - raptor-mitmproxy Info: creating nss cert database
[task 2022-01-14T04:00:13.208Z] 03:56:04 INFO - raptor-mitmproxy Info: Certutil: Running command: /builds/task_164213225211120/workspace/testing/mozproxy/host-utils-93.0a1.en-US.linux-x86_64/certutil -d sql:/tmp/tmpu5e9ty7e/profile/ -N --empty-password
[task 2022-01-14T04:00:13.209Z] 03:59:27 ERROR - Return code: -9
[task 2022-01-14T04:00:13.209Z] 03:59:27 WARNING - setting return code to -9
[task 2022-01-14T04:00:13.209Z] 03:59:27 INFO - Killing logcat pid 934.
[task 2022-01-14T04:00:13.209Z] 03:59:27 INFO - Copying Raptor results to upload dir:
[task 2022-01-14T04:00:13.209Z] 03:59:27 INFO - /builds/task_164213225211120/workspace/build/blobber_upload_dir/perfherder-data.json
[task 2022-01-14T04:00:13.209Z] 03:59:27 INFO - Copying raptor results from /builds/task_164213225211120/workspace/build/raptor.json to /builds/task_164213225211120/workspace/build/blobber_upload_dir/perfherder-data.json
[task 2022-01-14T04:00:13.209Z] 03:59:28 CRITICAL - Error copying results /builds/task_164213225211120/workspace/build/raptor.json to upload dir /builds/task_164213225211120/workspace/build/blobber_upload_dir/perfherder-data.json
[task 2022-01-14T04:00:13.209Z] 03:59:28 INFO - [Errno 2] No such file or directory: '/builds/task_164213225211120/workspace/build/raptor.json'
[task 2022-01-14T04:00:13.209Z] 03:59:28 INFO - Running post-action listener: _package_coverage_data
[task 2022-01-14T04:00:13.209Z] 03:59:28 INFO - Running post-action listener: _resource_record_post_action
[task 2022-01-14T04:00:13.209Z] 03:59:28 INFO - Running post-action listener: process_java_coverage_data
[task 2022-01-14T04:00:13.209Z] 03:59:28 INFO - Running post-action listener: stop_device
[task 2022-01-14T04:00:13.209Z] 03:59:43 INFO - /data/anr/traces.txt deleted
[task 2022-01-14T04:00:13.209Z] 03:59:48 INFO - /data/anr/traces_SystemServer_WDT.txt deleted
[task 2022-01-14T04:00:13.209Z] 04:00:01 INFO - Killing logcat pid 934.
[task 2022-01-14T04:00:13.209Z] 04:00:01 INFO - [mozharness: 2022-01-14 04:00:01.054166Z] Finished run-tests step (success)
[task 2022-01-14T04:00:13.209Z] 04:00:01 INFO - Running post-run listener: _resource_record_post_run
[task 2022-01-14T04:00:13.209Z] 04:00:05 INFO - Total resource usage - Wall time: 449s; CPU: 22%; Read bytes: 27755880448; Write bytes: 4098371584; Read time: 466848; Write time: 9856348
[task 2022-01-14T04:00:13.209Z] 04:00:05 INFO - TinderboxPrint: CPU usage<br/>21.8%
[task 2022-01-14T04:00:13.209Z] 04:00:05 INFO - TinderboxPrint: I/O read bytes / time<br/>27,755,880,448 / 466,848
[task 2022-01-14T04:00:13.209Z] 04:00:05 INFO - TinderboxPrint: I/O write bytes / time<br/>4,098,371,584 / 9,856,348
[task 2022-01-14T04:00:13.209Z] 04:00:05 INFO - TinderboxPrint: CPU idle<br/>1,116.6 (62.7%)
[task 2022-01-14T04:00:13.209Z] 04:00:05 INFO - TinderboxPrint: CPU iowait<br/>275.0 (15.4%)
[task 2022-01-14T04:00:13.209Z] 04:00:05 INFO - TinderboxPrint: CPU softirq<br/>18.5 (1.0%)
[task 2022-01-14T04:00:13.209Z] 04:00:05 INFO - TinderboxPrint: CPU system<br/>196.6 (11.0%)
[task 2022-01-14T04:00:13.209Z] 04:00:05 INFO - TinderboxPrint: CPU user<br/>173.8 (9.8%)
[task 2022-01-14T04:00:13.209Z] 04:00:05 INFO - TinderboxPrint: Swap in / out<br/>17,461,248 / 1,107,316,736
[task 2022-01-14T04:00:13.209Z] 04:00:05 INFO - install-chrome-android - Wall time: 0s; CPU: Can't collect data; Read bytes: 0; Write bytes: 0; Read time: 0; Write time: 0
[task 2022-01-14T04:00:13.209Z] 04:00:05 INFO - install-chromium-distribution - Wall time: 0s; CPU: Can't collect data; Read bytes: 0; Write bytes: 0; Read time: 0; Write time: 0
[task 2022-01-14T04:00:13.209Z] 04:00:05 INFO - install - Wall time: 26s; CPU: 21%; Read bytes: 0; Write bytes: 63901696; Read time: 0; Write time: 58056
[task 2022-01-14T04:00:13.209Z] 04:00:05 INFO - run-tests - Wall time: 391s; CPU: 22%; Read bytes: 27750092800; Write bytes: 4023189504; Read time: 465348; Write time: 9795504
[task 2022-01-14T04:00:13.209Z] 04:00:05 WARNING - returning nonzero exit status -9
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 20•3 years ago
|
||
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.
Reporter | ||
Comment 21•3 years ago
|
||
treeherder |
New failure instance: https://treeherder.mozilla.org/logviewer?job_id=383684761&repo=autoland
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 24•3 years ago
|
||
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.
Updated•3 years ago
|
Reporter | ||
Comment 25•3 years ago
|
||
treeherder |
New failure instance: https://treeherder.mozilla.org/logviewer?job_id=390625115&repo=autoland
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 28•2 years ago
|
||
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.
Reporter | ||
Comment 29•2 years ago
|
||
treeherder |
New failure instance: https://treeherder.mozilla.org/logviewer?job_id=398368347&repo=autoland
Comment hidden (Intermittent Failures Robot) |
Comment 31•2 years ago
|
||
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.
Reporter | ||
Comment 32•2 years ago
|
||
treeherder |
New failure instance: https://treeherder.mozilla.org/logviewer?job_id=403069668&repo=mozilla-central
Comment hidden (Intermittent Failures Robot) |
Comment 34•2 years ago
|
||
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.
Reporter | ||
Comment 35•2 years ago
|
||
treeherder |
New failure instance: https://treeherder.mozilla.org/logviewer?job_id=406535249&repo=autoland
Comment hidden (Intermittent Failures Robot) |
Comment 37•2 years ago
|
||
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.
Reporter | ||
Comment 38•2 years ago
|
||
treeherder |
New failure instance: https://treeherder.mozilla.org/logviewer?job_id=411354127&repo=mozilla-central
Comment hidden (Intermittent Failures Robot) |
Comment 40•2 years ago
|
||
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.
Reporter | ||
Comment 41•2 years ago
|
||
treeherder |
New failure instance: https://treeherder.mozilla.org/logviewer?job_id=418941874&repo=mozilla-central
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 45•2 years ago
|
||
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.
Reporter | ||
Comment 46•2 years ago
|
||
treeherder |
New failure instance: https://treeherder.mozilla.org/logviewer?job_id=424738355&repo=autoland
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 51•2 years ago
|
||
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.
Reporter | ||
Comment 52•1 years ago
|
||
treeherder |
New failure instance: https://treeherder.mozilla.org/logviewer?job_id=430388978&repo=mozilla-beta
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 62•1 year ago
|
||
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.
Reporter | ||
Comment 63•11 months ago
|
||
treeherder |
New failure instance: https://treeherder.mozilla.org/logviewer?job_id=456470535&repo=autoland
Comment hidden (Intermittent Failures Robot) |
Comment 65•10 months ago
|
||
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.
Reporter | ||
Comment 66•9 months ago
|
||
treeherder |
New failure instance: https://treeherder.mozilla.org/logviewer?job_id=465271175&repo=mozilla-central
Comment hidden (Intermittent Failures Robot) |
Comment 68•8 months ago
|
||
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.
Description
•