1782798 - Intermittent mozproxy Error while downloading /opt/worker/tasks/task_165946337216384/build/tests/awsy/tp6-pageset.manifest from tooltool:Command failed

Status: RESOLVED INCOMPLETE

(Testing :: AWSY, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: mlaza [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=386153485&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/FfZotLP4QNe0MSwNI8UUqA/runs/0/artifacts/public/logs/live_backing.log

---

[task 2022-08-02T18:04:40.243Z] 18:04:40     INFO - TEST-START | awsy/test_memory_usage.py TestMemoryUsage.test_open_tabs
[task 2022-08-02T18:04:40.252Z] 18:04:40     INFO - areweslimyet run by 0 pages, 1 iterations, 15 perTabPause, 30 settleWaitTime
[task 2022-08-02T18:04:40.266Z] 18:04:40     INFO - setting up
[task 2022-08-02T18:04:40.306Z] 18:04:40     INFO - mozproxy mozproxy_dir used for mitmproxy downloads and exe files: /opt/worker/tasks/task_165946337216384/build/tests/html/testing/mozproxy
[task 2022-08-02T18:04:40.307Z] 18:04:40     INFO - mozproxy Playback tool: mitmproxy
[task 2022-08-02T18:04:40.307Z] 18:04:40     INFO - mozproxy Playback tool version: 5.1.1
[task 2022-08-02T18:04:40.308Z] 18:04:40     INFO - mozproxy create mitmproxy 5.1.1 dir
[task 2022-08-02T18:04:40.308Z] 18:04:40     INFO - mozproxy downloading mitmproxy binary
...
[task 2022-08-02T18:05:05.999Z] 18:05:05     INFO - mozproxy b'WARNING - Impossible to add file mitm5-linux-firefox-youtube.zip to cache folder /builds/tooltool_cache'
[task 2022-08-02T18:05:06.000Z] 18:05:06     INFO - mozproxy b'Traceback (most recent call last):'
[task 2022-08-02T18:05:06.000Z] 18:05:06     INFO - mozproxy b'  File "/opt/worker/tasks/task_165946337216384/mozharness/external_tools/tooltool.py", line 1159, in fetch_files'
[task 2022-08-02T18:05:06.001Z] 18:05:06     INFO - mozproxy b'    os.makedirs(cache_folder, 0o0700)'
[task 2022-08-02T18:05:06.001Z] 18:05:06     INFO - mozproxy b'  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/os.py", line 213, in makedirs'
[task 2022-08-02T18:05:06.001Z] 18:05:06     INFO - mozproxy b'    makedirs(head, exist_ok=exist_ok)'
[task 2022-08-02T18:05:06.002Z] 18:05:06     INFO - mozproxy b'  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/os.py", line 223, in makedirs'
[task 2022-08-02T18:05:06.002Z] 18:05:06     INFO - mozproxy b'    mkdir(name, mode)'
[task 2022-08-02T18:05:06.002Z] 18:05:06     INFO - mozproxy b"OSError: [Errno 30] Read-only file system: '/builds'"
[task 2022-08-02T18:05:06.002Z] 18:05:06     INFO - mozproxy b"ERROR - The following files failed: 'mitm5-linux-firefox-wikia.zip'"
[task 2022-08-02T18:05:06.060Z] 18:05:06 CRITICAL - mozproxy Error while downloading /opt/worker/tasks/task_165946337216384/build/tests/awsy/tp6-pageset.manifest from tooltool:Command failed
[task 2022-08-02T18:05:09.323Z] 18:05:09     INFO - mozproxy b'INFO - File mitm5-linux-firefox-wikia.zip not present in local cache folder /builds/tooltool_cache'
[task 2022-08-02T18:05:09.324Z] 18:05:09     INFO - mozproxy b"INFO - Attempting to fetch from 'https://tooltool.mozilla-releng.net/'..."
[task 2022-08-02T18:05:11.409Z] 18:05:11     INFO - mozproxy b'INFO - File mitm5-linux-firefox-wikia.zip fetched from https://tooltool.mozilla-releng.net/ as /opt/worker/tasks/task_165946337216384/build/tests/html/testing/mozproxy/tmphd_wii2b'
[task 2022-08-02T18:05:11.462Z] 18:05:11     INFO - mozproxy b'INFO - File integrity verified, renaming tmphd_wii2b to mitm5-linux-firefox-wikia.zip'
[task 2022-08-02T18:05:11.462Z] 18:05:11     INFO - mozproxy b'INFO - Updating local cache /builds/tooltool_cache...'
[task 2022-08-02T18:05:11.463Z] 18:05:11     INFO - mozproxy b'INFO - Creating cache in /builds/tooltool_cache...'
[task 2022-08-02T18:05:11.463Z] 18:05:11     INFO - mozproxy b'WARNING - Impossible to add file mitm5-linux-firefox-wikia.zip to cache folder /builds/tooltool_cache'

Comment 1

[Intermittent Failures Robot]

1 failures in 3057 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• macosx1015-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1782798&startday=2022-08-01&endday=2022-08-07&tree=all

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Comment 3

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=391080353&repo=autoland

Comment 4

[Intermittent Failures Robot]

4 failures in 2909 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2
• mozilla-central: 1
• mozilla-beta: 1

Platform and build breakdown:

• macosx1015-64-shippable-qr: 4
  • opt: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1782798&startday=2022-09-19&endday=2022-09-25&tree=all

Comment 5

[Intermittent Failures Robot]

2 failures in 2041 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2

Platform and build breakdown:

• macosx1015-64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1782798&startday=2022-09-26&endday=2022-10-02&tree=all

Comment 6

[Intermittent Failures Robot]

1 failures in 2449 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform and build breakdown:

• macosx1015-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1782798&startday=2022-10-03&endday=2022-10-09&tree=all

Comment 7

[Intermittent Failures Robot]

4 failures in 3034 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1
• autoland: 3

Platform and build breakdown:

• macosx1015-64-shippable-qr: 3
  • opt: 3
• windows10-64-2004-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1782798&startday=2022-10-10&endday=2022-10-16&tree=all

Comment 8

[Intermittent Failures Robot]

1 failures in 2793 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1

Platform and build breakdown:

• macosx1015-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1782798&startday=2022-10-17&endday=2022-10-23&tree=all

Comment 9

[Intermittent Failures Robot]

4 failures in 3743 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 4

Platform and build breakdown:

• macosx1015-64-shippable-qr: 4
  • opt: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1782798&startday=2022-10-24&endday=2022-10-30&tree=all

Comment 10

[Intermittent Failures Robot]

1 failures in 3589 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• macosx1015-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1782798&startday=2022-11-14&endday=2022-11-20&tree=all

Comment 11

[Intermittent Failures Robot]

1 failures in 3510 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• macosx1015-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1782798&startday=2022-12-05&endday=2022-12-11&tree=all

Comment 12

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.