Closed Bug 1301807 Opened 9 years ago Closed 9 years ago

Intermittent BadStatusLine: ''

Categories

(Release Engineering :: Applications: MozharnessCore, defect)

Product:
Release Engineering
Component:
Applications: MozharnessCore
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(firefox52 fixed, firefox53 fixed, firefox54 fixed)

Status:
RESOLVED FIXED
Tracking Flags:
Tracking Status
firefox52 --- fixed
firefox53 --- fixed
firefox54 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: gbrown)

References

Details

(Keywords: intermittent-failure, Whiteboard: [stockwell fixed])

Attachments

(1 file)

handle all exceptions
951 bytes, patch
mozilla
: review+
Details | Diff | Splinter Review
Bug 1300812 should have fixed this.
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
this seems to have become a problem again, but it comes in spurts
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Assignee: nobody → gbrown
See Also: → 1303668
This stack seems typical: 04:11:03 FATAL - Uncaught exception: Traceback (most recent call last): 04:11:03 FATAL - File "/builds/slave/test/scripts/mozharness/base/script.py", line 2028, in run 04:11:03 FATAL - self.run_action(action) 04:11:03 FATAL - File "/builds/slave/test/scripts/mozharness/base/script.py", line 1967, in run_action 04:11:03 FATAL - self._possibly_run_method(method_name, error_if_missing=True) 04:11:03 FATAL - File "/builds/slave/test/scripts/mozharness/base/script.py", line 1907, in _possibly_run_method 04:11:03 FATAL - return getattr(self, method_name)() 04:11:03 FATAL - File "scripts/scripts/desktop_unittest.py", line 550, in download_and_extract 04:11:03 FATAL - suite_categories=target_categories) 04:11:03 FATAL - File "/builds/slave/test/scripts/mozharness/mozilla/testing/testbase.py", line 580, in download_and_extract 04:11:03 FATAL - self._download_and_extract_symbols() 04:11:03 FATAL - File "/builds/slave/test/scripts/mozharness/mozilla/testing/testbase.py", line 520, in _download_and_extract_symbols 04:11:03 FATAL - self.symbols_url = self.query_symbols_url() 04:11:03 FATAL - File "/builds/slave/test/scripts/mozharness/mozilla/testing/testbase.py", line 207, in query_symbols_url 04:11:03 FATAL - self._urlopen(symbols_url, timeout=120) 04:11:03 FATAL - File "/builds/slave/test/scripts/mozharness/mozilla/testing/testbase.py", line 302, in _urlopen 04:11:03 FATAL - return urllib2.urlopen(url, **kwargs) 04:11:03 FATAL - File "/tools/python27/lib/python2.7/urllib2.py", line 126, in urlopen 04:11:03 FATAL - return _opener.open(url, data, timeout) 04:11:03 FATAL - File "/tools/python27/lib/python2.7/urllib2.py", line 406, in open 04:11:03 FATAL - response = meth(req, response) 04:11:03 FATAL - File "/tools/python27/lib/python2.7/urllib2.py", line 519, in http_response 04:11:03 FATAL - 'http', request, response, code, msg, hdrs) 04:11:03 FATAL - File "/tools/python27/lib/python2.7/urllib2.py", line 438, in error 04:11:03 FATAL - result = self._call_chain(*args) 04:11:03 FATAL - File "/tools/python27/lib/python2.7/urllib2.py", line 378, in _call_chain 04:11:03 FATAL - result = func(*args) 04:11:03 FATAL - File "/tools/python27/lib/python2.7/urllib2.py", line 625, in http_error_302 04:11:03 FATAL - return self.parent.open(new, timeout=req.timeout) 04:11:03 FATAL - File "/tools/python27/lib/python2.7/urllib2.py", line 400, in open 04:11:03 FATAL - response = self._open(req, data) 04:11:03 FATAL - File "/tools/python27/lib/python2.7/urllib2.py", line 418, in _open 04:11:03 FATAL - '_open', req) 04:11:03 FATAL - File "/tools/python27/lib/python2.7/urllib2.py", line 378, in _call_chain 04:11:03 FATAL - result = func(*args) 04:11:03 FATAL - File "/tools/python27/lib/python2.7/urllib2.py", line 1215, in https_open 04:11:03 FATAL - return self.do_open(httplib.HTTPSConnection, req) 04:11:03 FATAL - File "/tools/python27/lib/python2.7/urllib2.py", line 1180, in do_open 04:11:03 FATAL - r = h.getresponse(buffering=True) 04:11:03 FATAL - File "/tools/python27/lib/python2.7/httplib.py", line 1030, in getresponse 04:11:03 FATAL - response.begin() 04:11:03 FATAL - File "/tools/python27/lib/python2.7/httplib.py", line 407, in begin 04:11:03 FATAL - version, status, reason = self._read_status() 04:11:03 FATAL - File "/tools/python27/lib/python2.7/httplib.py", line 371, in _read_status 04:11:03 FATAL - raise BadStatusLine(line) 04:11:03 FATAL - BadStatusLine: '' 04:11:03 FATAL - Running post_fatal callback... 04:11:03 FATAL - Exiting -1 query_symbols_url() expects and handles various exceptions from httplib, but not this one.
Everything should work out if we just handle this exception like the other exceptions. I'm not sure what else httplib can raise, so let's catch Exception.
Attachment #8830949 - Flags: review?(aki)
Comment on attachment 8830949 [details] [diff] [review] handle all exceptions This works, but is a bit heavy handed. If we're mainly concerned about httplib errors, all httplib exceptions seem to be based off httplib.HTTPException, which means we'd catch all of them if we add that to the list. https://docs.python.org/2/library/httplib.html#httplib.HTTPException Also, socket.timeout inherits socket.error which inherits IOError, and urllib2.HTTPError inherits urllib2.URLError, which inherits IOError. So this line could be rewritten as except (IOError, httplib.HTTPException) as ex: to catch all the current exceptions and add all httplib exceptions to the list. (I'm getting this by > 855$ python2 > Python 2.7.12 (default, Nov 24 2016, 18:40:05) > [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin > Type "help", "copyright", "credits" or "license" for more information. > >>> import urllib2 > >>> help(urllib2.URLError) > 855$ python2 > Python 2.7.12 (default, Nov 24 2016, 18:40:05) > [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin > Type "help", "copyright", "credits" or "license" for more information. > >>> import urllib2 > >>> help(urllib2.URLError) )
Attachment #8830949 - Flags: review?(aki) → review+
(In reply to Aki Sasaki [:aki] from comment #12) > except (IOError, httplib.HTTPException) as ex: We'd need to import httplib to do this without erroring. Also, I'm fine with either approach (either your patch or my suggested approach). Also also, sorry for the double paste; I got hit by bug 1334240 today.
Thanks aki. You make good points and I'm tempted to limit the exceptions as you suggest. I worry that non-httplib code might raise in _urlopen(), or the _urlopen() implementation may change over time, so I'm going to keep the heavy handed approach.
Pushed by gbrown@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/3424385712cf Guard against all exceptions mozharness query_symbols_url(); r=aki
https://hg.mozilla.org/mozilla-central/rev/3424385712cf
Status: REOPENED → RESOLVED
Closed: 9 years ago → 9 years ago
status-firefox54: --- → fixed
Resolution: --- → FIXED
Whiteboard: [stockwell fixed]
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: