1030082 - (panda-0643) panda-0643 problem tracking

Status: RESOLVED WONTFIX

(Infrastructure & Operations Graveyard :: CIDuty, task, P3)

Description

[Ed Morley [:emorley]]

perma-retrying; disabled

12:18:16     INFO - ##### Running request-device step.
12:18:16     INFO - #####
12:18:16     INFO - Running pre-action listener: _resource_record_pre_action
12:18:16     INFO - Running main action method: request_device
12:18:16     INFO - Getting output from command: ['/builds/panda-0643/test/build/venv/bin/python', '-c', 'from distutils.sysconfig import get_python_lib; print(get_python_lib())']
12:18:16     INFO - Copy/paste: /builds/panda-0643/test/build/venv/bin/python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())"
12:18:16     INFO - Reading from file tmpfile_stdout
12:18:16     INFO - Output received:
12:18:16     INFO -  /builds/panda-0643/test/build/venv/lib/python2.7/site-packages
12:18:16     INFO - Got request, url=http://mobile-imaging-007.p7.releng.scl1.mozilla.com/api/request/1596098/
12:18:16     INFO - Waiting for request 'ready' stage.  Current state: 'contact_lifeguard'
12:19:17     INFO - Waiting for request 'ready' stage.  Current state: 'pending'
12:20:17     INFO - Waiting for request 'ready' stage.  Current state: 'pending'
12:21:17     INFO - Waiting for request 'ready' stage.  Current state: 'pending'
12:22:17     INFO - Waiting for request 'ready' stage.  Current state: 'pending'
12:23:17     INFO - Waiting for request 'ready' stage.  Current state: 'pending'
12:24:17     INFO - Waiting for request 'ready' stage.  Current state: 'pending'
12:25:17     INFO - Waiting for request 'ready' stage.  Current state: 'pending'
12:26:17     INFO - Waiting for request 'ready' stage.  Current state: 'pending'
12:27:17     INFO - Waiting for request 'ready' stage.  Current state: 'pending'
12:28:17     INFO - Waiting for request 'ready' stage.  Current state: 'pending'
12:29:17     INFO - Waiting for request 'ready' stage.  Current state: 'pending'
12:30:17     INFO - Waiting for request 'ready' stage.  Current state: 'pending'
12:31:17     INFO - Waiting for request 'ready' stage.  Current state: 'pending'
12:32:18     INFO - Waiting for request 'ready' stage.  Current state: 'pending'
12:33:18     INFO - Waiting for request 'ready' stage.  Current state: 'pending'
12:34:18     INFO - Waiting for request 'ready' stage.  Current state: 'pending'
12:35:18     INFO - Waiting for request 'ready' stage.  Current state: 'pending'
12:36:18     INFO - Waiting for request 'ready' stage.  Current state: 'pending'
12:37:18     INFO - Waiting for request 'ready' stage.  Current state: 'pending'
12:38:18    ERROR - INFRA-ERROR: Request did not become ready in time
12:38:18    ERROR - # TBPL EXCEPTION #
12:38:18     INFO - Aborting mozpool request.
12:38:18     INFO - Request 'http://mobile-imaging-007.p7.releng.scl1.mozilla.com/api/request/1596098/' deleted on cleanup
12:38:18    FATAL - Retries limit exceeded
12:38:18    FATAL - Running post_fatal callback...
12:38:18     INFO - Running post-action listener: _resource_record_post_action
12:38:18    FATAL - Uncaught exception: Traceback (most recent call last):
12:38:18    FATAL -   File "/builds/panda-0643/test/scripts/mozharness/base/script.py", line 1230, in run
12:38:18    FATAL -     self.run_action(action)
12:38:18    FATAL -   File "/builds/panda-0643/test/scripts/mozharness/base/script.py", line 1172, in run_action
12:38:18    FATAL -     self._possibly_run_method(method_name, error_if_missing=True)
12:38:18    FATAL -   File "/builds/panda-0643/test/scripts/mozharness/base/script.py", line 1113, in _possibly_run_method
12:38:18    FATAL -     return getattr(self, method_name)()
12:38:18    FATAL -   File "scripts/scripts/android_panda_talos.py", line 131, in request_device
12:38:18    FATAL -     self.retrieve_android_device(b2gbase="")
12:38:18    FATAL -   File "/builds/panda-0643/test/scripts/mozharness/mozilla/testing/mozpool.py", line 96, in retrieve_android_device
12:38:18    FATAL -     self._wait_for_request_ready()
12:38:18    FATAL -   File "/builds/panda-0643/test/scripts/mozharness/mozilla/testing/mozpool.py", line 129, in _wait_for_request_ready
12:38:18    FATAL -     tbpl_status=TBPL_EXCEPTION, fail_cb=on_fail):
12:38:18    FATAL -   File "/builds/panda-0643/test/scripts/mozharness/mozilla/testing/mozpool.py", line 119, in _retry_sleep
12:38:18    FATAL -     self.fatal('Retries limit exceeded')
12:38:18    FATAL -   File "/builds/panda-0643/test/scripts/mozharness/base/log.py", line 112, in fatal
12:38:18    FATAL -     self.log(message, level=FATAL, exit_code=exit_code)
12:38:18    FATAL -   File "/builds/panda-0643/test/scripts/mozharness/base/log.py", line 55, in log
12:38:18    FATAL -     post_fatal_callback=self._post_fatal,
12:38:18    FATAL -   File "/builds/panda-0643/test/scripts/mozharness/base/log.py", line 338, in log_message
12:38:18    FATAL -     post_fatal_callback(message=message, exit_code=exit_code)
12:38:18    FATAL -   File "scripts/scripts/android_panda_talos.py", line 392, in _post_fatal
12:38:18    FATAL -     self.close_request()
12:38:18    FATAL -   File "scripts/scripts/android_panda_talos.py", line 396, in close_request
12:38:18    FATAL -     mph.close_request(self.request_url)
12:38:18    FATAL -   File "/builds/panda-0643/test/build/venv/lib/python2.7/site-packages/mozpoolclient.py", line 255, in close_request
12:38:18    FATAL -     request_url = request_url + 'return/'
12:38:18    FATAL - TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
12:38:18    FATAL - Running post_fatal callback...
12:38:18     INFO - Running post-run listener: _resource_record_post_run
12:38:18     INFO - Running post-run listener: _upload_blobber_files
12:38:18     INFO - Blob upload gear active.
12:38:18  WARNING - Blob upload directory does not exist!
Traceback (most recent call last):
  File "scripts/scripts/android_panda_talos.py", line 410, in <module>
    pandaTalosTest.run_and_exit()
  File "/builds/panda-0643/test/scripts/mozharness/base/script.py", line 1254, in run_and_exit
    sys.exit(self.run())
  File "/builds/panda-0643/test/scripts/mozharness/base/script.py", line 1232, in run
    self.fatal("Uncaught exception: %s" % traceback.format_exc())
  File "/builds/panda-0643/test/scripts/mozharness/base/log.py", line 112, in fatal
    self.log(message, level=FATAL, exit_code=exit_code)
  File "/builds/panda-0643/test/scripts/mozharness/base/log.py", line 55, in log
    post_fatal_callback=self._post_fatal,
  File "/builds/panda-0643/test/scripts/mozharness/base/log.py", line 338, in log_message
    post_fatal_callback(message=message, exit_code=exit_code)
  File "scripts/scripts/android_panda_talos.py", line 392, in _post_fatal
    self.close_request()
  File "scripts/scripts/android_panda_talos.py", line 396, in close_request
    mph.close_request(self.request_url)
  File "/builds/panda-0643/test/build/venv/lib/python2.7/site-packages/mozpoolclient.py", line 255, in close_request
    request_url = request_url + 'return/'
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
program finished with exit code 1
elapsedTime=1246.754234

Comment 1

[Chris Cooper [:coop] (he/him)]

Due to the problematic nature of this panda, we've decided to decommission it.