912312 - MozpoolException: Can't decode json: No JSON object could be decoded: line 1 column 0 (char 0)!

Status: RESOLVED WONTFIX

(Release Engineering :: General, defect)

Description

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Got this one several times this afternoon:

16:33:47     INFO - Running post-action listener: _resource_record_post_action
16:33:47    FATAL - Uncaught exception: Traceback (most recent call last):
16:33:47    FATAL -   File "/builds/panda-0739/test/scripts/mozharness/base/script.py", line 1070, in run
16:33:47    FATAL -     self.run_action(action)
16:33:47    FATAL -   File "/builds/panda-0739/test/scripts/mozharness/base/script.py", line 1012, in run_action
16:33:47    FATAL -     self._possibly_run_method(method_name, error_if_missing=True)
16:33:47    FATAL -   File "/builds/panda-0739/test/scripts/mozharness/base/script.py", line 953, in _possibly_run_method
16:33:47    FATAL -     return getattr(self, method_name)()
16:33:47    FATAL -   File "scripts/scripts/android_panda.py", line 159, in request_device
16:33:47    FATAL -     self.retrieve_android_device(b2gbase="")
16:33:47    FATAL -   File "/builds/panda-0739/test/scripts/mozharness/mozilla/testing/mozpool.py", line 99, in retrieve_android_device
16:33:47    FATAL -     self._wait_for_request_ready()
16:33:47    FATAL -   File "/builds/panda-0739/test/scripts/mozharness/mozilla/testing/mozpool.py", line 133, in _wait_for_request_ready
16:33:47    FATAL -     response = mph.query_request_status(self.request_url)
16:33:47    FATAL -   File "/builds/panda-0739/test/build/venv/lib/python2.7/site-packages/mozpoolclient/mozpoolclient.py", line 301, in query_request_status
16:33:47    FATAL -     response, status = self.url_get(request_url, **kwargs)
16:33:47    FATAL -   File "/builds/panda-0739/test/build/venv/lib/python2.7/site-packages/mozpoolclient/mozpoolclient.py", line 88, in url_get
16:33:47    FATAL -     j = self.decode_json(r.text)
16:33:47    FATAL -   File "/builds/panda-0739/test/build/venv/lib/python2.7/site-packages/mozpoolclient/mozpoolclient.py", line 106, in decode_json
16:33:47    FATAL -     raise MozpoolException("Can't decode json: %s!" % str(e))
16:33:47    FATAL - MozpoolException: Can't decode json: No JSON object could be decoded: line 1 column 0 (char 0)!
16:33:47    FATAL - Running post_fatal callback...
16:33:47    FATAL - Exiting -1
16:33:47     INFO - Running post-run listener: _resource_record_post_run

https://tbpl.mozilla.org/php/getParsedLog.php?id=27344570&tree=Mozilla-Inbound#error0
https://tbpl.mozilla.org/php/getParsedLog.php?id=27344977&tree=Fx-Team
https://tbpl.mozilla.org/php/getParsedLog.php?id=27344963&tree=Mozilla-Aurora
https://tbpl.mozilla.org/php/getParsedLog.php?id=27344971&tree=Mozilla-Aurora
https://tbpl.mozilla.org/php/getParsedLog.php?id=27344973&tree=Mozilla-Aurora

These were on pandas 0739, 801, 0129, 0130 and 0131, respectively.


Possibly related to bug 912310, since they all started happening around the same time?

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

This is a MozpoolClient error hiding the true error.

John, do you want mozpoolclient bugs here or in a Release Engineering component?

Comment 2

[John Hopkins (:jhopkins)]

Can you please file MozpoolClient bugs under Release Engineering::General Automation?  Thanks.

Comment 3

[John Hopkins (:jhopkins)]

dustin: can you clarify what the expected behaviour is?

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

Some message closer to the actual problem.  Invalid JSON was not the problem.

I expect there was a non-200 HTTP status here, which is why there was no JSON body.  But if there was more context than just the HTTP status code, that should be included as well.  In this particular case, I think the problem was a *client* timeout after 10 minutes, so the error message should reflect that.

Comment 5

[John Hopkins (:jhopkins)]

I won't be able to get to this for at least another week.

Comment 6

[John Hopkins (:jhopkins)]

Unforunately, I haven't had time to work on this.  Returning to the pool.

Comment 7

[Justin Wood (:Callek)]

Mozpool is no longer a thing for us