837134 - Intermittent Datazilla "socket.error: [Errno 60] Operation timed out" in dzclient.py

Status: RESOLVED WORKSFORME

(Testing :: Talos, defect)

Description

[Ryan VanderMeulen [:RyanVM]]

https://tbpl.mozilla.org/php/getParsedLog.php?id=19351334&tree=Mozilla-Inbound

Android Tegra 250 mozilla-inbound talos remote-tp4m_nochrome on 2013-02-01 01:59:33 PST for push 9069c3f568f7
slave: tegra-041

NOISE: Posting result 0 of 3 to http://graphs.mozilla.org/server/collect.cgi, attempt 0
NOISE: Posting result 1 of 3 to http://graphs.mozilla.org/server/collect.cgi, attempt 0
NOISE: Posting result 2 of 3 to http://graphs.mozilla.org/server/collect.cgi, attempt 0
RETURN: <a href='http://graphs.mozilla.org/graph.html#tests=[[84,63,20]]'>tp4m_nochrome: 720.10</a>
RETURN: <a href='http://graphs.mozilla.org/graph.html#tests=[[136,63,20]]'>tp4m_main_rss_nochrome: 91.1MB</a>
RETURN: <a href='http://graphs.mozilla.org/graph.html#tests=[[86,63,20]]'>tp4m_shutdown_nochrome: 28810931.00</a>
NOISE: Outputting datazilla results to https://datazilla.mozilla.org/talos
NOISE: datazilla: https//datazilla.mozilla.org/talos; oauth=True
Traceback (most recent call last):
  File "run_tests.py", line 331, in <module>
    main()
  File "run_tests.py", line 328, in main
    run_tests(parser)
  File "run_tests.py", line 304, in run_tests
    talos_results.output(results_urls, **results_options)
  File "/builds/tegra-041/talos-data/talos/results.py", line 84, in output
    _output.output(results, url, tbpl_output)
  File "/builds/tegra-041/talos-data/talos/output.py", line 410, in output
    self.post(results, results_server, results_path, results_scheme, tbpl_output)
  File "/builds/tegra-041/talos-data/talos/output.py", line 491, in post
    responses = req.submit()
  File "/builds/tegra-041/talos-data/talos/dzclient.py", line 196, in submit
    responses.append(self.send(dataset))
  File "/builds/tegra-041/talos-data/talos/dzclient.py", line 253, in send
    conn.request("POST", path, body, header)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 914, in request
    self._send_request(method, url, body, headers)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 951, in _send_request
    self.endheaders()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 908, in endheaders
    self._send_output()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 780, in _send_output
    self.send(msg)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 739, in send
    self.connect()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 1112, in connect
    sock = socket.create_connection((self.host, self.port), self.timeout)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/socket.py", line 561, in create_connection
    raise error, msg
socket.error: [Errno 60] Operation timed out
program finished with exit code 1
elapsedTime=293.483082
TinderboxPrint: <a href='http://graphs.mozilla.org/graph.html#tests=[[84,63,20]]'>tp4m_nochrome: 720.10</a>

TinderboxPrint: <a href='http://graphs.mozilla.org/graph.html#tests=[[136,63,20]]'>tp4m_main_rss_nochrome: 91.1MB</a>

TinderboxPrint: <a href='http://graphs.mozilla.org/graph.html#tests=[[86,63,20]]'>tp4m_shutdown_nochrome: 28810931.00</a>
========= Finished 'python run_tests.py ...' failed (results: 2, elapsed: 4 mins, 53 secs) (at 2013-02-01 02:09:24.902857) =========

Comment 1

[Ed Morley [:emorley]]

We should catch this exception and output a "Automation Error: foo".

Comment 2

[Jeff Hammel]

ticketed in datazilla_client: https://github.com/mozilla/datazilla_client/issues/35

Comment 3

[Ed Morley [:emorley]]

Resolving WFM keyword:intermittent-failure bugs last modified >3 months ago, whose whiteboard contains none of:
{random,disabled,marked,fuzzy,todo,fails,failing,annotated,time-bomb,leave open}

There will inevitably be some false positives; for that (and the bugspam) I apologise. Filter on orangewfm.