934410 - (panda-0796) panda-0796 problem tracking

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task, P3)

Description

[Carsten Book [:Tomcat]]

slave failed 2 times, disabled in slave alloc

https://tbpl.mozilla.org/php/getParsedLog.php?id=30064229&tree=Mozilla-Inbound

https://tbpl.mozilla.org/php/getParsedLog.php?id=30063701&tree=Mozilla-Inbound



Automation Error: Unable to connect to device after 5 attempts
Return code: 1
Preparing to abort run due to failed verify check.
Dying due to failing verification
Running post_fatal callback...
Uncaught exception: Traceback (most recent call last):
 File "/builds/panda-0796/test/scripts/mozharness/base/script.py", line 1112, in run
   self.run_action(action)
 File "/builds/panda-0796/test/scripts/mozharness/base/script.py", line 1054, in run_action
   self._possibly_run_method(method_name, error_if_missing=True)
 File "/builds/panda-0796/test/scripts/mozharness/base/script.py", line 995, in _possibly_run_method
   return getattr(self, method_name)()
 File "scripts/scripts/android_panda_talos.py", line 137, in request_device
   self.fatal("Dying due to failing verification")
 File "/builds/panda-0796/test/scripts/mozharness/base/log.py", line 112, in fatal
   self.log(message, level=FATAL, exit_code=exit_code)
 File "/builds/panda-0796/test/scripts/mozharness/base/log.py", line 55, in log
   post_fatal_callback=self._post_fatal,
 File "/builds/panda-0796/test/scripts/mozharness/base/log.py", line 340, in log_message
   post_fatal_callback(message=message, exit_code=exit_code)
 File "scripts/scripts/android_panda_talos.py", line 392, in _post_fatal
   self.close_request()
 File "scripts/scripts/android_panda_talos.py", line 396, in close_request
   mph.close_request(self.request_url)
 File "/builds/panda-0796/test/build/venv/lib/python2.7/site-packages/mozpoolclient/mozpoolclient.py", line 248, in close_request
   request_url = request_url + 'return/'
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
Running post_fatal callback...
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'

Comment 1

[Ed Morley [:emorley]]

Was about to do the same - thank you :-)

Poorly panda:
https://secure.pub.build.mozilla.org/builddata/reports/slave_health/slave.html?name=panda-0796

Comment 2

[Ryan VanderMeulen [:RyanVM]]

Looks like it's still taking and torching jobs. John, can you please kill this harder?

Comment 3

[John Hopkins (:jhopkins)]

Ran:
$ python manage_foopies.py  -f ../mobile/devices.json stop -D panda-0796

Comment 4

[Justin Wood (:Callek)]

recovered by "panda-recovery" bug 902657

Comment 5

[Vinh Hua [:vinh]]

Replaced SD card and reran self test.  Panda is now in ready state.

Comment 6

[Chris Cooper [:coop] (he/him)]

Re-enabled in production.