1443978 - Intermittent Automation Error: mozprocess timed out after 600 seconds running ['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace

Status: RESOLVED INCOMPLETE

(Release Engineering :: Applications: MozharnessCore, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: csabou [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=166568443&repo=mozilla-inbound

https://queue.taskcluster.net/v1/task/WOC2hq9FRECtJ_pDnBlqgA/runs/0/artifacts/public/logs/live_backing.log

[task 2018-03-07T19:53:17.980Z] 19:53:17     INFO -  INFO - Attempting to fetch from 'http://relengapi/tooltool/'...
[task 2018-03-07T20:03:18.002Z] 20:03:18     INFO - Automation Error: mozprocess timed out after 600 seconds running ['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace/build/.android/releng.manifest', '-o', '-c', '/builds/worker/tooltool_cache']
[task 2018-03-07T20:03:18.010Z] 20:03:18    ERROR - timed out after 600 seconds of no output
[task 2018-03-07T20:03:18.011Z] 20:03:18    ERROR - Return code: -15
[task 2018-03-07T20:03:18.011Z] 20:03:18     INFO - retry: Failed, sleeping 60 seconds before retrying
[task 2018-03-07T20:04:18.021Z] 20:04:18     INFO - retry: Calling run_command with args: (['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace/build/.android/releng.manifest', '-o', '-c', '/builds/worker/tooltool_cache'],), kwargs: {'output_timeout': 600, 'error_list': [{'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x7fade24d2408>, 'level': 'warning'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x7fade3f61c48>, 'level': 'critical'}, {'regex': <_sre.SRE_Pattern object at 0x7fade24f53d8>, 'level': 'critical'}, {'substr': 'ERROR - ', 'level': 'error'}], 'cwd': '/builds/worker/workspace/build/.android', 'privileged': False}, attempt #2
[task 2018-03-07T20:04:18.021Z] 20:04:18     INFO - Running command: ['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace/build/.android/releng.manifest', '-o', '-c', '/builds/worker/tooltool_cache'] in /builds/worker/workspace/build/.android
[task 2018-03-07T20:04:18.021Z] 20:04:18     INFO - Copy/paste: /usr/bin/python2.7 -u /builds/worker/workspace/mozharness/external_tools/tooltool.py --url http://relengapi/tooltool/ fetch -m /builds/worker/workspace/build/.android/releng.manifest -o -c /builds/worker/tooltool_cache
[task 2018-03-07T20:04:18.021Z] 20:04:18     INFO - Calling ['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace/build/.android/releng.manifest', '-o', '-c', '/builds/worker/tooltool_cache'] with output_timeout 600
[task 2018-03-07T20:04:18.069Z] 20:04:18     INFO -  INFO - File AVDs-armv7a-android-4.3.1_r1-build-2016-08-02.tar.gz not present in local cache folder /builds/worker/tooltool_cache
[task 2018-03-07T20:04:18.069Z] 20:04:18     INFO -  INFO - Attempting to fetch from 'http://relengapi/tooltool/'...
[task 2018-03-07T20:04:25.006Z] 20:04:25     INFO -  INFO - File AVDs-armv7a-android-4.3.1_r1-build-2016-08-02.tar.gz fetched from http://relengapi/tooltool/ as /builds/worker/workspace/build/.android/tmpVWG5rH
[task 2018-03-07T20:04:26.154Z] 20:04:26     INFO -  INFO - File integrity verified, renaming tmpVWG5rH to AVDs-armv7a-android-4.3.1_r1-build-2016-08-02.tar.gz
[task 2018-03-07T20:04:26.155Z] 20:04:26     INFO -  INFO - Updating local cache /builds/worker/tooltool_cache...
[task 2018-03-07T20:04:26.299Z] 20:04:26     INFO -  INFO - Local cache /builds/worker/tooltool_cache updated with AVDs-armv7a-android-4.3.1_r1-build-2016-08-02.tar.gz
[task 2018-03-07T20:04:26.303Z] 20:04:26     INFO -  INFO - untarring "AVDs-armv7a-android-4.3.1_r1-build-2016-08-02.tar.gz"
[task 2018-03-07T20:04:31.692Z] 20:04:31     INFO - Return code: 0
[task 2018-03-07T20:04:31.728Z] 20:04:31     INFO - Running post-action listener: _resource_record_post_action
[task 2018-03-07T20:04:31.728Z] 20:04:31     INFO - [mozharness: 2018-03-07 20:04:31.728765Z] Finished setup-avds step (success)
[task 2018-03-07T20:04:31.729Z] 20:04:31     INFO - [mozharness: 2018-03-07 20:04:31.728917Z] Running start-emulator step.
[task 2018-03-07T20:04:31.729Z] 20:04:31     INFO - Running pre-action listener: _resource_record_pre_action
[task 2018-03-07T20:04:31.729Z] 20:04:31     INFO - Running main action method: start_emulator
[task 2018-03-07T20:04:31.729Z] 20:04:31     INFO - Writing to file /builds/worker/workspace/build/tooltool.tt
[task 2018-03-07T20:04:31.730Z] 20:04:31     INFO - Contents:
[task 2018-03-07T20:04:31.730Z] 20:04:31     INFO -  
[task 2018-03-07T20:04:31.730Z] 20:04:31     INFO -          [
[task 2018-03-07T20:04:31.730Z] 20:04:31     INFO -          {
[task 2018-03-07T20:04:31.730Z] 20:04:31     INFO -          "size": 38451805,
[task 2018-03-07T20:04:31.730Z] 20:04:31     INFO -          "digest": "ac3000aa6846dd1b09a420a1ba84727e393036f49da1419d6c9e5ec2490fc6105c74ca18b616c465dbe693992e2939afd88bc2042d961a9050a3cafd2a673ff4",
[task 2018-03-07T20:04:31.731Z] 20:04:31     INFO -          "algorithm": "sha512",
[task 2018-03-07T20:04:31.731Z] 20:04:31     INFO -          "filename": "android-sdk_r24.0.2a-linux.tar.gz",
[task 2018-03-07T20:04:31.731Z] 20:04:31     INFO -          "unpack": "True"
[task 2018-03-07T20:04:31.731Z] 20:04:31     INFO -          }
[task 2018-03-07T20:04:31.731Z] 20:04:31     INFO -          ]

Comment 1

[Intermittent Failures Robot]

1 failures in 747 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 1

Platform breakdown:
* android-4-3-armv7-api16: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1443978&startday=2018-03-05&endday=2018-03-11&tree=all

Comment 2

[Firefox Bug Husbandry Bot]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup

Comment 3

[Andreea Pavel [:apavel]]

Recent failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=175197376&repo=autoland&lineNumber=677

[task 2018-04-23T22:11:35.449Z] 22:11:35     INFO -  INFO - File AVDs-armv7a-android-4.3.1_r1-build-2016-08-02.tar.gz not present in local cache folder /builds/worker/tooltool_cache
[task 2018-04-23T22:11:35.450Z] 22:11:35     INFO -  INFO - Attempting to fetch from 'http://relengapi/tooltool/'...
[task 2018-04-23T22:11:38.938Z] compiz (core) - Warn: Attempted to restack relative to 0x1400006 which is not a child of the root window or a window compiz owns
[task 2018-04-23T22:21:35.472Z] 22:21:35     INFO - Automation Error: mozprocess timed out after 600 seconds running ['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace/build/.android/releng.manifest', '-o', '-c', '/builds/worker/tooltool_cache']
[task 2018-04-23T22:21:35.474Z] 22:21:35    ERROR - timed out after 600 seconds of no output
[task 2018-04-23T22:21:35.474Z] 22:21:35    ERROR - Return code: -15
[task 2018-04-23T22:21:35.474Z] 22:21:35     INFO - retry: Failed, sleeping 60 seconds before retrying
[task 2018-04-23T22:22:35.493Z] 22:22:35     INFO - retry: Calling run_command with args: (['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace/build/.android/releng.manifest', '-o', '-c', '/builds/worker/tooltool_cache'],), kwargs: {'output_timeout': 600, 'error_list': [{'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x7fa2b9c8a408>, 'level': 'warning'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x7fa2bb719c48>, 'level': 'critical'}, {'regex': <_sre.SRE_Pattern object at 0x7fa2b9cac510>, 'level': 'critical'}, {'substr': 'ERROR - ', 'level': 'error'}], 'cwd': '/builds/worker/workspace/build/.android', 'privileged': False}, attempt #2
[task 2018-04-23T22:22:35.494Z] 22:22:35     INFO - Running command: ['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace/build/.android/releng.manifest', '-o', '-c', '/builds/worker/tooltool_cache'] in /builds/worker/workspace/build/.android
[task 2018-04-23T22:22:35.494Z] 22:22:35     INFO - Copy/paste: /usr/bin/python2.7 -u /builds/worker/workspace/mozharness/external_tools/tooltool.py --url http://relengapi/tooltool/ fetch -m /builds/worker/workspace/build/.android/releng.manifest -o -c /builds/worker/tooltool_cache
[task 2018-04-23T22:22:35.494Z] 22:22:35     INFO - Calling ['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace/build/.android/releng.manifest', '-o', '-c', '/builds/worker/tooltool_cache'] with output_timeout 600
[task 2018-04-23T22:22:35.545Z] 22:22:35     INFO -  INFO - File AVDs-armv7a-android-4.3.1_r1-build-2016-08-02.tar.gz not present in local cache folder /builds/worker/tooltool_cache
[task 2018-04-23T22:22:35.545Z] 22:22:35     INFO -  INFO - Attempting to fetch from 'http://relengapi/tooltool/'...
[task 2018-04-23T22:32:35.567Z] 22:32:35     INFO - Automation Error: mozprocess timed out after 600 seconds running ['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace/build/.android/releng.manifest', '-o', '-c', '/builds/worker/tooltool_cache']
[task 2018-04-23T22:32:35.568Z] 22:32:35    ERROR - timed out after 600 seconds of no output
[task 2018-04-23T22:32:35.568Z] 22:32:35    ERROR - Return code: -15
[task 2018-04-23T22:32:35.568Z] 22:32:35     INFO - retry: Failed, sleeping 120 seconds before retrying
[task 2018-04-23T22:34:35.571Z] 22:34:35     INFO - retry: Calling run_command with args: (['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace/build/.android/releng.manifest', '-o', '-c', '/builds/worker/tooltool_cache'],), kwargs: {'output_timeout': 600, 'error_list': [{'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x7fa2b9c8a408>, 'level': 'warning'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x7fa2bb719c48>, 'level': 'critical'}, {'regex': <_sre.SRE_Pattern object at 0x7fa2b9cac510>, 'level': 'critical'}, {'substr': 'ERROR - ', 'level': 'error'}], 'cwd': '/builds/worker/workspace/build/.android', 'privileged': False}, attempt #3
[task 2018-04-23T22:34:35.571Z] 22:34:35     INFO - Running command: ['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace/build/.android/releng.manifest', '-o', '-c', '/builds/worker/tooltool_cache'] in /builds/worker/workspace/build/.android
[task 2018-04-23T22:34:35.571Z] 22:34:35     INFO - Copy/paste: /usr/bin/python2.7 -u /builds/worker/workspace/mozharness/external_tools/tooltool.py --url http://relengapi/tooltool/ fetch -m /builds/worker/workspace/build/.android/releng.manifest -o -c /builds/worker/tooltool_cache
[task 2018-04-23T22:34:35.571Z] 22:34:35     INFO - Calling ['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace/build/.android/releng.manifest', '-o', '-c', '/builds/worker/tooltool_cache'] with output_timeout 600
[task 2018-04-23T22:34:35.618Z] 22:34:35     INFO -  INFO - File AVDs-armv7a-android-4.3.1_r1-build-2016-08-02.tar.gz not present in local cache folder /builds/worker/tooltool_cache
[task 2018-04-23T22:34:35.619Z] 22:34:35     INFO -  INFO - Attempting to fetch from 'http://relengapi/tooltool/'...
[task 2018-04-23T22:44:35.658Z] 22:44:35     INFO - Automation Error: mozprocess timed out after 600 seconds running ['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace/build/.android/releng.manifest', '-o', '-c', '/builds/worker/tooltool_cache']
[task 2018-04-23T22:44:35.670Z] 22:44:35    ERROR - timed out after 600 seconds of no output
[task 2018-04-23T22:44:35.670Z] 22:44:35    ERROR - Return code: -15
[task 2018-04-23T22:44:35.670Z] 22:44:35     INFO - retry: Failed, sleeping 240 seconds before retrying
[task 2018-04-23T22:48:35.685Z] 22:48:35     INFO - retry: Calling run_command with args: (['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace/build/.android/releng.manifest', '-o', '-c', '/builds/worker/tooltool_cache'],), kwargs: {'output_timeout': 600, 'error_list': [{'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x7fa2b9c8a408>, 'level': 'warning'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x7fa2bb719c48>, 'level': 'critical'}, {'regex': <_sre.SRE_Pattern object at 0x7fa2b9cac510>, 'level': 'critical'}, {'substr': 'ERROR - ', 'level': 'error'}], 'cwd': '/builds/worker/workspace/build/.android', 'privileged': False}, attempt #4
[task 2018-04-23T22:48:35.685Z] 22:48:35     INFO - Running command: ['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace/build/.android/releng.manifest', '-o', '-c', '/builds/worker/tooltool_cache'] in /builds/worker/workspace/build/.android
[task 2018-04-23T22:48:35.686Z] 22:48:35     INFO - Copy/paste: /usr/bin/python2.7 -u /builds/worker/workspace/mozharness/external_tools/tooltool.py --url http://relengapi/tooltool/ fetch -m /builds/worker/workspace/build/.android/releng.manifest -o -c /builds/worker/tooltool_cache
[task 2018-04-23T22:48:35.686Z] 22:48:35     INFO - Calling ['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace/build/.android/releng.manifest', '-o', '-c', '/builds/worker/tooltool_cache'] with output_timeout 600
[task 2018-04-23T22:48:35.737Z] 22:48:35     INFO -  INFO - File AVDs-armv7a-android-4.3.1_r1-build-2016-08-02.tar.gz not present in local cache folder /builds/worker/tooltool_cache
[task 2018-04-23T22:48:35.737Z] 22:48:35     INFO -  INFO - Attempting to fetch from 'http://relengapi/tooltool/'...
[task 2018-04-23T22:58:35.764Z] 22:58:35     INFO - Automation Error: mozprocess timed out after 600 seconds running ['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace/build/.android/releng.manifest', '-o', '-c', '/builds/worker/tooltool_cache']
[task 2018-04-23T22:58:35.788Z] 22:58:35    ERROR - timed out after 600 seconds of no output
[task 2018-04-23T22:58:35.788Z] 22:58:35    ERROR - Return code: -15
[task 2018-04-23T22:58:35.788Z] 22:58:35     INFO - retry: Failed, sleeping 300 seconds before retrying
[task 2018-04-23T23:03:35.804Z] 23:03:35     INFO - retry: Calling run_command with args: (['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace/build/.android/releng.manifest', '-o', '-c', '/builds/worker/tooltool_cache'],), kwargs: {'output_timeout': 600, 'error_list': [{'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x7fa2b9c8a408>, 'level': 'warning'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x7fa2bb719c48>, 'level': 'critical'}, {'regex': <_sre.SRE_Pattern object at 0x7fa2b9cac510>, 'level': 'critical'}, {'substr': 'ERROR - ', 'level': 'error'}], 'cwd': '/builds/worker/workspace/build/.android', 'privileged': False}, attempt #5
[task 2018-04-23T23:03:35.805Z] 23:03:35     INFO - Running command: ['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace/build/.android/releng.manifest', '-o', '-c', '/builds/worker/tooltool_cache'] in /builds/worker/workspace/build/.android
[task 2018-04-23T23:03:35.805Z] 23:03:35     INFO - Copy/paste: /usr/bin/python2.7 -u /builds/worker/workspace/mozharness/external_tools/tooltool.py --url http://relengapi/tooltool/ fetch -m /builds/worker/workspace/build/.android/releng.manifest -o -c /builds/worker/tooltool_cache
[task 2018-04-23T23:03:35.805Z] 23:03:35     INFO - Calling ['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace/build/.android/releng.manifest', '-o', '-c', '/builds/worker/tooltool_cache'] with output_timeout 600
[task 2018-04-23T23:03:35.853Z] 23:03:35     INFO -  INFO - File AVDs-armv7a-android-4.3.1_r1-build-2016-08-02.tar.gz not present in local cache folder /builds/worker/tooltool_cache
[task 2018-04-23T23:03:35.853Z] 23:03:35     INFO -  INFO - Attempting to fetch from 'http://relengapi/tooltool/'...

[taskcluster:error] Task timeout after 3600 seconds. Force killing container.
[taskcluster 2018-04-23 23:11:15.527Z] === Task Finished ===
[taskcluster 2018-04-23 23:11:15.528Z] Unsuccessful task run with exit code: -1 completed in 3928.075 seconds

Comment 4

[Intermittent Failures Robot]

6 failures in 780 pushes (0.008 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 4
* mozilla-inbound: 2

Platform breakdown:
* windows7-32: 3
* android-4-3-armv7-api16: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1443978&startday=2018-04-23&endday=2018-04-29&tree=trunk

Comment 5

[Intermittent Failures Robot]

1 failures in 551 pushes (0.002 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 1

Platform breakdown:
* android-4-2-x86: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1443978&startday=2018-04-30&endday=2018-05-06&tree=trunk

Comment 6

[Intermittent Failures Robot]

5 failures in 639 pushes (0.008 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 3
* mozilla-beta: 1
* autoland: 1

Platform breakdown:
* android-4-3-armv7-api16: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1443978&startday=2018-05-14&endday=2018-05-20&tree=trunk

Comment 7

[Intermittent Failures Robot]

1 failures in 632 pushes (0.002 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 1

Platform breakdown:
* android-4-3-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1443978&startday=2018-05-21&endday=2018-05-27&tree=trunk

Comment 8

[Intermittent Failures Robot]

1 failures in 655 pushes (0.002 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 1

Platform breakdown:
* linux64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1443978&startday=2018-05-28&endday=2018-06-03&tree=trunk

Comment 9

[Intermittent Failures Robot]

1 failures in 408 pushes (0.002 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 1

Platform breakdown:
* android-4-3-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1443978&startday=2018-06-11&endday=2018-06-17&tree=trunk

Comment 10

[Rok Garbas [:garbas]]

:apavel looks like this are intermittent errors which are happening very rarely.

Comment 11

[Andreea Pavel [:apavel]]

(In reply to Rok Garbas [:garbas] from comment #10)
> :apavel looks like this are intermittent errors which are happening very
> rarely.

Yes, it appears so.

Comment 12

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://treeherder.mozilla.org/logviewer.html#?job_id=187423169&repo=mozilla-central

Comment 13

[Intermittent Failures Robot]

1 failures in 623 pushes (0.002 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-central: 1

Platform breakdown:
* android-em-4-3-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1443978&startday=2018-07-09&endday=2018-07-15&tree=trunk

Comment 14

[Firefox Bug Husbandry Bot]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup

Comment 15

[Intermittent Failures Robot]

1 failures in 2759 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* try: 1

Platform breakdown:
* android-4-0-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1443978&startday=2018-10-15&endday=2018-10-21&tree=all

Comment 16

[Andreea Pavel [:apavel]]

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=222133727&repo=autoland&lineNumber=916

[task 2019-01-16T03:56:17.410Z] 03:56:17 INFO - retry: Calling run_command with args: (['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace/build/hostutils/releng.manifest', '-o', '-c', '/builds/worker/tooltool-cache'],), kwargs: {'output_timeout': 600, 'error_list': [{'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x7f9c8b5f9408>, 'level': 'warning'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x7f9c8c2ffc48>, 'level': 'critical'}, {'regex': <_sre.SRE_Pattern object at 0x7f9c8c26a510>, 'level': 'critical'}, {'substr': 'ERROR - ', 'level': 'error'}], 'cwd': '/builds/worker/workspace/build/hostutils', 'privileged': False}, attempt #1
[task 2019-01-16T03:56:17.410Z] 03:56:17 INFO - Running command: ['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace/build/hostutils/releng.manifest', '-o', '-c', '/builds/worker/tooltool-cache'] in /builds/worker/workspace/build/hostutils
[task 2019-01-16T03:56:17.410Z] 03:56:17 INFO - Copy/paste: /usr/bin/python2.7 -u /builds/worker/workspace/mozharness/external_tools/tooltool.py --url http://relengapi/tooltool/ fetch -m /builds/worker/workspace/build/hostutils/releng.manifest -o -c /builds/worker/tooltool-cache
[task 2019-01-16T03:56:17.411Z] 03:56:17 INFO - Calling ['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace/build/hostutils/releng.manifest', '-o', '-c', '/builds/worker/tooltool-cache'] with output_timeout 600
[task 2019-01-16T03:56:17.461Z] 03:56:17 INFO - INFO - File host-utils-66.0a1.en-US.linux-x86_64.tar.gz not present in local cache folder /builds/worker/tooltool-cache
[task 2019-01-16T03:56:17.461Z] 03:56:17 INFO - INFO - Attempting to fetch from 'http://relengapi/tooltool/'...
[task 2019-01-16T04:06:17.491Z] 04:06:17 INFO - Automation Error: mozprocess timed out after 600 seconds running ['/usr/bin/python2.7', '-u', '/builds/worker/workspace/mozharness/external_tools/tooltool.py', '--url', 'http://relengapi/tooltool/', 'fetch', '-m', '/builds/worker/workspace/build/hostutils/releng.manifest', '-o', '-c', '/builds/worker/tooltool-cache']
[task 2019-01-16T04:06:17.522Z] 04:06:17 ERROR - timed out after 600 seconds of no output
[task 2019-01-16T04:06:17.522Z] 04:06:17 ERROR - Return code: -15
[task 2019-01-16T04:06:17.522Z] 04:06:17 INFO - retry: Failed, sleeping 60 seconds before retrying

Comment 17

[Intermittent Failures Robot]

1 failures in 2378 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• android-em-4-3-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1443978&startday=2019-01-14&endday=2019-01-20&tree=all

Comment 18

[Firefox Bug Husbandry Bot]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup

Comment 19

[Firefox Bug Husbandry Bot]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup

Comment 20

[Cristian Brindusan [:cbrindusan]]

Recent log failure:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=241795433&repo=mozilla-beta&lineNumber=825

Comment 21

[Intermittent Failures Robot]

1 failures in 3031 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform breakdown:

• android-em-7-0-x86_64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1443978&startday=2019-04-22&endday=2019-04-28&tree=all

Comment 22

[Intermittent Failures Robot]

14 failures in 4131 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 10
• mozilla-inbound: 1
• autoland: 1
• mozilla-beta: 2

Platform breakdown:

• android-em-7-0-x86_64: 5
• android-em-7-0-x86: 8
• android-em-4-3-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1443978&startday=2019-04-29&endday=2019-05-05&tree=all

Comment 23

[Intermittent Failures Robot]

2 failures in 4705 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 2

Platform breakdown:

• android-em-7-0-x86: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1443978&startday=2019-05-06&endday=2019-05-12&tree=all

Comment 24

[Intermittent Failures Robot]

1 failures in 4658 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 1

Platform breakdown:

• android-em-7-0-x86_64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1443978&startday=2019-05-13&endday=2019-05-19&tree=all

Comment 25

[Intermittent Failures Robot]

2 failures in 5197 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 2

Platform breakdown:

• android-em-7-0-x86: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1443978&startday=2019-06-03&endday=2019-06-09&tree=all

Comment 26

[Firefox Bug Husbandry Bot]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup

Comment 27

[Cosmin Sabou [:CosminS]]

https://treeherder.mozilla.org/logviewer.html#?job_id=262408540&repo=mozilla-inbound

Comment 28

[Intermittent Failures Robot]

25 failures in 5073 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• mozilla-inbound: 1
• mozilla-central: 17
• mozilla-beta: 6

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16: 25

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1443978&startday=2019-08-19&endday=2019-08-25&tree=all

Comment 29

[Firefox Bug Husbandry Bot]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup

Comment 30

[Stefan Hindli [:stefan_hindli]]

New failure: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=unclassified&selectedJob=274656706

Comment 31

[Intermittent Failures Robot]

1 failures in 4850 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1443978&startday=2019-11-04&endday=2019-11-10&tree=all

Comment 32

[Intermittent Failures Robot]

1 failures in 4472 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 1

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1443978&startday=2020-01-06&endday=2020-01-12&tree=all

Comment 33

[Intermittent Failures Robot]

14 failures in 5582 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 3
• autoland: 9
• mozilla-beta: 2

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 7
• android-hw-p2-8-0-android-aarch64-qr: 7

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1443978&startday=2020-04-20&endday=2020-04-26&tree=all

Comment 34

[Intermittent Failures Robot]

1 failures in 5276 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• android-em-7-0-x86_64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1443978&startday=2020-05-04&endday=2020-05-10&tree=all

Comment 35

[Intermittent Failures Robot]

4 failures in 5488 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 4

Platform breakdown:

• windows10-64-shippable-qr: 1
• macosx1014-64-shippable: 2
• android-em-7-0-x86_64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1443978&startday=2020-05-11&endday=2020-05-17&tree=all

Comment 36

[Intermittent Failures Robot]

4 failures in 5458 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 4

Platform breakdown:

• android-em-7-0-x86_64: 3
• android-em-7-0-x86_64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1443978&startday=2020-05-25&endday=2020-05-31&tree=all

Comment 37

[Intermittent Failures Robot]

3 failures in 4230 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 3

Platform breakdown:

• android-em-7-0-x86_64: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1443978&startday=2020-10-19&endday=2020-10-25&tree=all

Comment 38

[Intermittent Failures Robot]

2 failures in 4327 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2

Platform and build breakdown:

• windows10-64-shippable: 1
  • opt: 1
• macosx1014-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1443978&startday=2021-02-22&endday=2021-02-28&tree=all

Comment 39

[Cosmin Sabou [:CosminS]]

Looks like this failure is present also on green runs: https://treeherder.mozilla.org/logviewer?job_id=332646144&repo=autoland&lineNumber=964 so it's not relevant for the classified failures here. Gonna re-classify. Failures were all Bug 1589796.

Comment 40

[Cosmin Sabou [:CosminS]]

Andrew, looks like these timeouts started occurring after Bug 1369784 and Bug 1646161 landed here. Could you have a look over these?

Comment 41

[Andrew Halberstadt [:ahal]]

It should be impossible for a change in mozlint (lint infrastructure) to affect tests like mochitest. Likely further retriggers will show another push is the culprit or the error is due to some external factor and that push happened to be at the wrong place at the wrong time.

Comment 42

[Intermittent Failures Robot]

19 failures in 4622 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 10
• autoland: 6
• mozilla-release: 2
• mozilla-central: 1

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable-qr: 8
  • opt: 8
• android-hw-p2-8-0-arm7-api-16-shippable: 4
  • opt: 4
• android-hw-p2-8-0-android-aarch64: 1
  • opt: 1
• android-hw-p2-8-0-arm7-api-16: 1
  • debug: 1
• android-hw-p2-8-0-android-aarch64-qr: 5
  • debug: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1443978&startday=2021-03-08&endday=2021-03-14&tree=all

Comment 43

[Intermittent Failures Robot]

3 failures in 3967 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 3

Platform and build breakdown:

• android-em-7-0-x86_64: 2
  • opt: 2
• android-em-7-0-x86_64-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1443978&startday=2021-03-22&endday=2021-03-28&tree=all

Comment 44

[Intermittent Failures Robot]

1 failures in 3626 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1443978&startday=2021-10-04&endday=2021-10-10&tree=all

Comment 45

[Norisz Fay [:noriszfay]]

These failures have started appearing recently starting with this merge . So far they are only failing on Android 8 as tier 2.

Comment 46

[Intermittent Failures Robot]

47 failures in 401 pushes (0.117 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 20
• autoland: 26
• mozilla-beta: 1

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-qr: 43
  • debug: 32
  • opt: 11
• android-hw-p2-8-0-android-aarch64-shippable-qr: 4
  • opt: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1443978&startday=2022-04-15&endday=2022-04-15&tree=all

Comment 47

[Intermittent Failures Robot]

47 failures in 2948 pushes (0.016 failures/push) were associated with this bug in the last 7 days.

This is the #13 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 20
• autoland: 26
• mozilla-beta: 1

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-qr: 43
  • debug: 32
  • opt: 11
• android-hw-p2-8-0-android-aarch64-shippable-qr: 4
  • opt: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1443978&startday=2022-04-11&endday=2022-04-17&tree=all

Comment 48

[Greg Mierzwinski [:sparky]]

This is also happening on our tier-3 G5 and Pixel 2 perf tests: https://treeherder.mozilla.org/jobs?repo=mozilla-central&tier=1%2C2%2C3&searchStr=browsertime%2Cchrome%2Candroid&revision=0d591d3bc99786bdb3cb057203a3831110d00800&selectedTaskRun=TB4zOTdQQFGMS4JT3dThGQ.1

Comment 49

[Intermittent Failures Robot]

16 failures in 3159 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 3
• try: 3
• mozilla-release: 6
• mozilla-central: 4

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-qr: 7
  • opt: 1
  • debug: 6
• android-hw-p2-8-0-android-aarch64-shippable-qr: 9
  • opt: 9

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1443978&startday=2022-04-18&endday=2022-04-24&tree=all

Comment 50

[Intermittent Failures Robot]

1 failures in 3540 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1443978&startday=2022-05-02&endday=2022-05-08&tree=all

Comment 51

[Intermittent Failures Robot]

4 failures in 3170 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 4

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-qr: 4
  • debug: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1443978&startday=2022-06-13&endday=2022-06-19&tree=all

Comment 52

[Intermittent Failures Robot]

1 failures in 3497 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1443978&startday=2022-07-18&endday=2022-07-24&tree=all

Comment 53

[Intermittent Failures Robot]

17 failures in 722 pushes (0.024 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 11
• try: 1
• autoland: 5

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable-qr: 9
  • opt: 9
• windows10-64-2004-qr: 1
  • debug: 1
• android-hw-p2-8-0-android-aarch64-qr: 7
  • debug: 7

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1443978&startday=2022-08-23&endday=2022-08-23&tree=all

Comment 54

[Intermittent Failures Robot]

29 failures in 3293 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

This is the #40 most frequent failure this week.

Repository breakdown:

• mozilla-central: 13
• try: 1
• autoland: 15

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable-qr: 9
  • opt: 9
• windows10-64-2004-qr: 1
  • debug: 1
• android-hw-p2-8-0-android-aarch64-qr: 19
  • debug: 15
  • opt: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1443978&startday=2022-08-22&endday=2022-08-28&tree=all

Comment 55

[Geoff Brown [:gbrown]]

These failures are almost exclusively on android-hw, when downloading host-utils from tooltool. Most android-hw and android-em test tasks download host-utils this way.

host-utils is about 100 MB in size. For android-em tasks, host-utils is typically downloaded and extracted in a few seconds. On android-hw, host-utils downloads typically take 4 to 6 minutes. In many of the failure logs for this bug, tooltool runs for 10 minutes, times out, retries, might run tooltool and timeout again after 10 minutes, eventually succeeds, but then the task exceeds its max-run-time (having run perhaps 30 minutes extra downloading host-utils).

At first I thought a shorter tooltool timeout might be helpful: retry sooner, might succeed sooner if tooltool is hung. But, as the timeout is an output timeout, and tooltool doesn't produce any progress output, and typical android-hw downloads take 4-6 minutes anyway, there's not much room to reduce the timeout, and doing so might cause more retries / more failures.

Increasing the tooltool timeout might be helpful, if those timed-out downloads are actually progressing but just needed more time. But if those tooltool sessions are actually hung, increasing the timeout will make things worse.

Overall, I think the 10 minute tooltool timeout is sensible in this case.

Another solution would be to increase the android-hw mochitest and reftest (and maybe other) max-run-time: In case it takes several attempts and 30+ minutes to download host-utils, allow the tasks to run that much longer.

Ideally, it would be nice to remove this reliance on tooltool; can we use a fetch instead; would that work any better?

https://searchfox.org/mozilla-central/rev/380fc5571b039fd453b45bbb64ed13146fe9b066/testing/mozharness/mozharness/mozilla/tooltool.py#63

Comment 56

[Intermittent Failures Robot]

1 failures in 3537 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• android-hw-p2-8-0-android-aarch64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1443978&startday=2022-08-29&endday=2022-09-04&tree=all

Comment 57

[BugBot [:suhaib / :marco]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.