Intermittent AttributeError: 'NoneType' object has no attribute 'find' when fallback for not finding symbols URL doesn't work

RESOLVED FIXED

Status

Release Engineering
Applications: MozharnessCore
RESOLVED FIXED
2 years ago
2 years ago

People

(Reporter: Treeherder Bug Filer, Assigned: gbrown)

Tracking

({intermittent-failure})

unspecified
intermittent-failure
Dependency tree / graph

Firefox Tracking Flags

(firefox50 wontfix, firefox51 fixed, firefox52 fixed, firefox53 fixed)

Details

MozReview Requests

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(2 attachments, 1 obsolete attachment)

(Reporter)

Description

2 years ago
treeherder
Filed by: philringnalda [at] gmail.com

https://treeherder.mozilla.org/logviewer.html#?job_id=3671062&repo=autoland

https://archive.mozilla.org/pub/firefox/tinderbox-builds/autoland-linux-debug/1474251137/autoland_ubuntu32_vm-debug_test-mochitest-4-bm08-tests1-linux32-build68.txt.gz

21:34:06  WARNING - Can't figure out symbols_url from installer_url: https://queue.taskcluster.net/v1/task/c5zgKXI2QQmKduigjtiIcg/artifacts/public/build/firefox-51.0a1.en-US.linux-i686.tar.bz2!
(traceback from a connection reset while querying the symbols_url)
WARNING - No symbols_url found. Let minidump_stackwalk query for symbols.
 21:34:06     INFO - Setting buildbot property symbols_url to None
 21:34:06     INFO - Writing buildbot properties ['symbols_url'] to /builds/slave/test/properties/symbols_url
 21:34:06     INFO - Writing to file /builds/slave/test/properties/symbols_url
 21:34:06     INFO - Contents:
 21:34:06     INFO -  symbols_url:None
 21:34:06     INFO - Downloading and extracting to /builds/slave/test/build/symbols these dirs * from None
 21:34:06     INFO - retry: Calling fetch_url_into_memory with args: (), kwargs: {'url': None}, attempt #1
 21:34:06     INFO - Fetch None into memory
 21:34:06     INFO - Running post-action listener: _resource_record_post_action
 21:34:06     INFO - Running post-action listener: set_extra_try_arguments
 21:34:06     INFO - [mozharness: 2016-09-19 04:34:06.136733Z] Finished download-and-extract step (failed)
 21:34:06    FATAL - Uncaught exception: Traceback (most recent call last):
 21:34:06    FATAL -   File "/builds/slave/test/scripts/mozharness/base/script.py", line 2001, in run
 21:34:06    FATAL -     self.run_action(action)
 21:34:06    FATAL -   File "/builds/slave/test/scripts/mozharness/base/script.py", line 1940, in run_action
 21:34:06    FATAL -     self._possibly_run_method(method_name, error_if_missing=True)
 21:34:06    FATAL -   File "/builds/slave/test/scripts/mozharness/base/script.py", line 1880, in _possibly_run_method
 21:34:06    FATAL -     return getattr(self, method_name)()
 21:34:06    FATAL -   File "scripts/scripts/desktop_unittest.py", line 505, in download_and_extract
 21:34:06    FATAL -     suite_categories=target_categories)
 21:34:06    FATAL -   File "/builds/slave/test/scripts/mozharness/mozilla/testing/testbase.py", line 557, in download_and_extract
 21:34:06    FATAL -     self._download_and_extract_symbols()
 21:34:06    FATAL -   File "/builds/slave/test/scripts/mozharness/mozilla/testing/testbase.py", line 518, in _download_and_extract_symbols
 21:34:06    FATAL -     self.download_unpack(self.symbols_url, self.symbols_path)
 21:34:06    FATAL -   File "/builds/slave/test/scripts/mozharness/base/script.py", line 675, in download_unpack
 21:34:06    FATAL -     **retry_args
 21:34:06    FATAL -   File "/builds/slave/test/scripts/mozharness/base/script.py", line 1099, in retry
 21:34:06    FATAL -     status = action(*args, **kwargs)
 21:34:06    FATAL -   File "/builds/slave/test/scripts/mozharness/base/script.py", line 366, in fetch_url_into_memory
 21:34:06    FATAL -     parsed_url = urlparse.urlparse(url)
 21:34:06    FATAL -   File "/usr/lib/python2.7/urlparse.py", line 135, in urlparse
 21:34:06    FATAL -     tuple = urlsplit(url, scheme, allow_fragments)
 21:34:06    FATAL -   File "/usr/lib/python2.7/urlparse.py", line 174, in urlsplit
 21:34:06    FATAL -     i = url.find(':')
 21:34:06    FATAL - AttributeError: 'NoneType' object has no attribute 'find'
 21:34:06    FATAL - Running post_fatal callback...
 21:34:06    FATAL - Exiting -1

hahahaha, "Fetch None into memory"

Updated

2 years ago
Depends on: 1303759
Comment hidden (mozreview-request)

Updated

2 years ago
Assignee: nobody → armenzg

Updated

2 years ago
Attachment #8794331 - Flags: review?(jlund)

Comment 3

2 years ago
13 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-central: 6
* autoland: 5
* mozilla-inbound: 1
* mozilla-beta: 1

Platform breakdown:
* linux32: 7
* windows7-32-vm: 2
* linux64: 2
* android-4-3-armv7-api15: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1303668&startday=2016-09-19&endday=2016-09-25&tree=all

Comment 4

2 years ago
mozreview-review
Comment on attachment 8794331 [details]
Bug 1303668 - Failing to find a symbols url should be a warning instead of a exception.

https://reviewboard.mozilla.org/r/80836/#review79652

::: testing/mozharness/mozharness/mozilla/testing/testbase.py
(Diff revision 1)
>              return self.symbols_url
>  
>          elif self.installer_url:
> -            symbols_url = self.query_prefixed_build_dir_url('.crashreporter-symbols.zip')
> +            self.symbols_url = self.query_prefixed_build_dir_url('.crashreporter-symbols.zip')
>  
> -            # Check if the URL exists. If not, use none to allow mozcrash to auto-check for symbols

We should not remove that part. As the comment below states minidump-stackwalk can only auto-detect the symbols for Nightly and release builds but not for CI builds.

I would suggest to maybe reduce the failure from exception to warning.

Comment 5

2 years ago
mozreview-review
Comment on attachment 8794331 [details]
Bug 1303668 - Failing to find a symbols url should be a warning instead of a exception.

https://reviewboard.mozilla.org/r/80836/#review79744

I am less familiar with current state than you and whimboo are. as this is not part of core mh(it's just a subclass of our test scripts), I'll defer to whimboo to give the final r+ once his conerns are addressed.
Attachment #8794331 - Flags: review?(jlund)
Comment hidden (mozreview-request)

Comment 7

2 years ago
mozreview-review
Comment on attachment 8794331 [details]
Bug 1303668 - Failing to find a symbols url should be a warning instead of a exception.

https://reviewboard.mozilla.org/r/80836/#review80420
Attachment #8794331 - Flags: review?(hskupin) → review+

Comment 8

2 years ago
Pushed by armenzg@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/1418ec10eda2
Failing to find a symbols url should be a warning instead of a exception. r=whimboo

Comment 9

2 years ago
Backout by gszorc@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a008ff0d08b4
Backed out changeset 1418ec10eda2 for mozharness bustage
Comment hidden (mozreview-request)

Comment 11

2 years ago
Pushed by armenzg@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a7c3db5394f2
Failing to find a symbols url should be a warning instead of a exception. r=whimboo

Comment 12

2 years ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/a7c3db5394f2
Status: NEW → RESOLVED
Last Resolved: 2 years ago
status-firefox52: --- → fixed
Resolution: --- → FIXED
If I understand https://treeherder.mozilla.org/logviewer.html#?job_id=11788227&repo=fx-team correctly, that's not enough, because the FATAL problem is that after failing to find the symbols URL, we just merrily go ahead and "INFO - Fetch None into memory" which doesn't go well for us.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---

Comment 14

2 years ago
12 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 9
* mozilla-aurora: 2
* fx-team: 1

Platform breakdown:
* linux64: 11
* windows8-64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1303668&startday=2016-09-26&endday=2016-10-02&tree=all

Comment 15

2 years ago
For that log, we obsiously have symbols avalaible:
https://queue.taskcluster.net/v1/task/TeCsg2htQYeMN5JVX8RkEQ/runs/0/artifacts/public/build/target.crashreporter-symbols.zip

I believe if we retry when we expect symbols (e.g. debug tests) we would be able to reduce this.
Comment hidden (mozreview-request)

Updated

2 years ago
Attachment #8794331 - Attachment is obsolete: true

Updated

2 years ago
Attachment #8797171 - Flags: review?(hskupin)
Comment on attachment 8797171 [details]
Bug 1303668 - Retry downloading symbols if we're expecting them.

https://reviewboard.mozilla.org/r/82772/#review82794

I'm not an official peer of mozharness. So you might want to run this review request through Jordan first.

Updated

2 years ago
Attachment #8797171 - Flags: review?(jlund) → review?(aki)

Updated

2 years ago
Attachment #8797171 - Flags: review?(hskupin)

Comment 18

2 years ago
mozreview-review
Comment on attachment 8797171 [details]
Bug 1303668 - Retry downloading symbols if we're expecting them.

https://reviewboard.mozilla.org/r/82772/#review82978
Attachment #8797171 - Flags: review?(aki) → review+

Comment 19

2 years ago
Pushed by armenzg@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/df471131e1b6
Retry downloading symbols if we're expecting them. r=aki

Comment 20

2 years ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/df471131e1b6
Status: REOPENED → RESOLVED
Last Resolved: 2 years ago2 years ago
Resolution: --- → FIXED

Comment 21

2 years ago
6 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* fx-team: 3
* mozilla-inbound: 1
* mozilla-aurora: 1
* autoland: 1

Platform breakdown:
* windows7-32-vm: 3
* linux64: 2
* android-4-3-armv7-api15: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1303668&startday=2016-10-03&endday=2016-10-09&tree=all
status-firefox50: --- → affected
status-firefox51: --- → affected
https://treeherder.mozilla.org/logviewer.html#?job_id=4945394&repo=autoland
Status: RESOLVED → REOPENED
Resolution: FIXED → ---

Comment 23

2 years ago
8 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 3
* mozilla-inbound: 2
* fx-team: 2
* mozilla-central: 1

Platform breakdown:
* windows7-32-vm: 3
* android-4-3-armv7-api15: 3
* linux64: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1303668&startday=2016-10-10&endday=2016-10-16&tree=all

Comment 24

2 years ago
8 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* try: 5
* mozilla-inbound: 2
* autoland: 1

Platform breakdown:
* linux64: 7
* android-4-3-armv7-api15: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1303668&startday=2016-10-17&endday=2016-10-23&tree=all

Comment 25

2 years ago
13 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 8
* fx-team: 2
* mozilla-release: 1
* mozilla-inbound: 1
* mozilla-aurora: 1

Platform breakdown:
* windows7-32-vm: 5
* linux64: 5
* linux32: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1303668&startday=2016-10-24&endday=2016-10-30&tree=all

Comment 26

2 years ago
7 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* try: 4
* mozilla-inbound: 1
* mozilla-aurora: 1
* autoland: 1

Platform breakdown:
* linux64: 5
* linux32: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1303668&startday=2016-10-31&endday=2016-11-06&tree=all

Comment 27

2 years ago
16 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 12
* autoland: 4

Platform breakdown:
* windows7-32-vm: 12
* linux32: 2
* linux64: 1
* android-4-3-armv7-api15: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1303668&startday=2016-11-07&endday=2016-11-13&tree=all

Updated

2 years ago
Assignee: armenzg → nobody

Comment 28

2 years ago
29 failures in 715 pushes (0.041 failures/push) were associated with this bug in the last 7 days.  

Repository breakdown:
* autoland: 12
* mozilla-inbound: 7
* mozilla-aurora: 6
* mozilla-central: 3
* try: 1

Platform breakdown:
* android-4-3-armv7-api15: 13
* linux64: 12
* windows7-32-vm: 4

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1303668&startday=2016-11-14&endday=2016-11-20&tree=all

Comment 29

2 years ago
5 failures in 623 pushes (0.008 failures/push) were associated with this bug in the last 7 days.  

Repository breakdown:
* mozilla-inbound: 2
* try: 1
* mozilla-aurora: 1
* autoland: 1

Platform breakdown:
* android-4-3-armv7-api15: 2
* osx-10-10: 1
* linux64: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1303668&startday=2016-11-21&endday=2016-11-27&tree=all

Comment 30

2 years ago
6 failures in 694 pushes (0.009 failures/push) were associated with this bug in the last 7 days.  

Repository breakdown:
* autoland: 3
* mozilla-inbound: 2
* mozilla-aurora: 1

Platform breakdown:
* linux64: 2
* windows8-64: 1
* windows7-32-vm: 1
* osx-10-10: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1303668&startday=2016-11-28&endday=2016-12-04&tree=all

Comment 31

2 years ago
5 failures in 526 pushes (0.01 failures/push) were associated with this bug in the last 7 days.  

Repository breakdown:
* mozilla-inbound: 3
* mozilla-aurora: 2

Platform breakdown:
* linux32: 2
* windows7-32-vm: 1
* linux64: 1
* android-4-3-armv7-api15: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1303668&startday=2016-12-12&endday=2016-12-18&tree=all

Comment 32

2 years ago
8 failures in 609 pushes (0.013 failures/push) were associated with this bug in the last 7 days.  

Repository breakdown:
* autoland: 4
* mozilla-inbound: 2
* try: 1
* mozilla-aurora: 1

Platform breakdown:
* android-4-3-armv7-api15: 4
* linux64: 2
* linux32: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1303668&startday=2016-12-19&endday=2016-12-25&tree=all

Comment 33

2 years ago
9 failures in 563 pushes (0.016 failures/push) were associated with this bug in the last 7 days.  

Repository breakdown:
* mozilla-central: 5
* autoland: 3
* mozilla-inbound: 1

Platform breakdown:
* linux32: 6
* android-4-3-armv7-api15: 2
* osx-10-10: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1303668&startday=2017-01-02&endday=2017-01-08&tree=all
The retry() added in comment 20 generally doesn't work (doesn't retry) because query_symbols_url() catches exceptions -- it may return None, but it never fails from the retry() perspective. 

I think the retry might work without the change landed in comment 12 (change the warning() back to exception()).

I would also like to report the exception causing query_symbols_url() to fail.

For the failures that I checked, the symbols artifact is present, so I am hopeful that the retry will work. Still, I wonder why the failed jobs are not finding the symbols when they check.
Assignee: nobody → gbrown
Created attachment 8826411 [details] [diff] [review]
fix retry of  query_symbols_url

With this patch, query_symbols_url() takes a new parameter, raise_on_failure, which is set to true when called from Armen's retry(); if raise_on_failure is set and the url lookup fails, query_symbols_url() will raise an exception, triggering a retry.

Other callers of query_symbols_url(), which don't retry and appear to be more tolerant of query_symbols_url() failures do not set retry_on_failure; I anticipate no change in behavior for those callers.

I also updated the query_symbols_url() warning wording to give us more information.

Finally, _download_and_extract_symbols() now does not call download_unpack() if the symbols_url is not set; this should not be necessary since retry() should be fatal on failure, but is logical and provides further protection from the exception in this bug.

Normal run without failure: https://treeherder.mozilla.org/#/jobs?repo=try&revision=7312358584ce878a5b4f2e4de5fd07ad649ee86b

Forced permanent error: https://treeherder.mozilla.org/#/jobs?repo=try&revision=aa02a72deb8d42b63dc3c70f297dc099c5a06cf1

https://public-artifacts.taskcluster.net/TF_RZWR1RjKr0H6CY3ZYBw/0/public/logs/live_backing.log

[task 2017-01-12T22:06:09.713180Z] 22:06:09     INFO - retry: Calling query_symbols_url with args: (True,), kwargs: {}, attempt #1
[task 2017-01-12T22:06:15.353281Z] 22:06:15  WARNING - Cannot open symbols url https://queue.taskcluster.net/v1/task/cAlOB_FTRDeRDBsxGBYO4g/artifacts/public/build/target.crashreporter-symbols.zip (installer url: https://queue.taskcluster.net/v1/task/cAlOB_FTRDeRDBsxGBYO4g/artifacts/public/build/target.tar.bz2): 42
[task 2017-01-12T22:06:15.354712Z] 22:06:15     INFO - retry: attempt #1 caught exception: 42
[task 2017-01-12T22:06:15.355387Z] 22:06:15     INFO - retry: Failed, sleeping 20 seconds before retrying
[task 2017-01-12T22:06:35.377218Z] 22:06:35     INFO - retry: Calling query_symbols_url with args: (True,), kwargs: {}, attempt #2
[task 2017-01-12T22:06:35.824206Z] 22:06:35  WARNING - Cannot open symbols url https://queue.taskcluster.net/v1/task/cAlOB_FTRDeRDBsxGBYO4g/artifacts/public/build/target.crashreporter-symbols.zip (installer url: https://queue.taskcluster.net/v1/task/cAlOB_FTRDeRDBsxGBYO4g/artifacts/public/build/target.tar.bz2): 42
[task 2017-01-12T22:06:35.826373Z] 22:06:35     INFO - retry: attempt #2 caught exception: 42
[task 2017-01-12T22:06:35.827786Z] 22:06:35     INFO - retry: Failed, sleeping 40 seconds before retrying
[task 2017-01-12T22:07:15.859127Z] 22:07:15     INFO - retry: Calling query_symbols_url with args: (True,), kwargs: {}, attempt #3
[task 2017-01-12T22:07:16.255247Z] 22:07:16  WARNING - Cannot open symbols url https://queue.taskcluster.net/v1/task/cAlOB_FTRDeRDBsxGBYO4g/artifacts/public/build/target.crashreporter-symbols.zip (installer url: https://queue.taskcluster.net/v1/task/cAlOB_FTRDeRDBsxGBYO4g/artifacts/public/build/target.tar.bz2): 42
[task 2017-01-12T22:07:16.257462Z] 22:07:16     INFO - retry: attempt #3 caught exception: 42
[task 2017-01-12T22:07:16.258832Z] 22:07:16     INFO - retry: Failed, sleeping 80 seconds before retrying
[task 2017-01-12T22:08:36.340457Z] 22:08:36     INFO - retry: Calling query_symbols_url with args: (True,), kwargs: {}, attempt #4
[task 2017-01-12T22:08:36.785815Z] 22:08:36  WARNING - Cannot open symbols url https://queue.taskcluster.net/v1/task/cAlOB_FTRDeRDBsxGBYO4g/artifacts/public/build/target.crashreporter-symbols.zip (installer url: https://queue.taskcluster.net/v1/task/cAlOB_FTRDeRDBsxGBYO4g/artifacts/public/build/target.tar.bz2): 42
[task 2017-01-12T22:08:36.788011Z] 22:08:36     INFO - retry: attempt #4 caught exception: 42
[task 2017-01-12T22:08:36.789372Z] 22:08:36     INFO - retry: Failed, sleeping 160 seconds before retrying
[task 2017-01-12T22:11:16.857755Z] 22:11:16     INFO - retry: Calling query_symbols_url with args: (True,), kwargs: {}, attempt #5
[task 2017-01-12T22:11:17.219013Z] 22:11:17  WARNING - Cannot open symbols url https://queue.taskcluster.net/v1/task/cAlOB_FTRDeRDBsxGBYO4g/artifacts/public/build/target.crashreporter-symbols.zip (installer url: https://queue.taskcluster.net/v1/task/cAlOB_FTRDeRDBsxGBYO4g/artifacts/public/build/target.tar.bz2): 42
[task 2017-01-12T22:11:17.221217Z] 22:11:17     INFO - retry: attempt #5 caught exception: 42
[task 2017-01-12T22:11:17.222591Z] 22:11:17    FATAL - We can't proceed without downloading symbols.
[task 2017-01-12T22:11:17.224379Z] 22:11:17    FATAL - Caught exception: 42
[task 2017-01-12T22:11:17.225981Z] 22:11:17    FATAL - Caught exception: 42
[task 2017-01-12T22:11:17.227562Z] 22:11:17    FATAL - Caught exception: 42
[task 2017-01-12T22:11:17.229180Z] 22:11:17    FATAL - Caught exception: 42
[task 2017-01-12T22:11:17.230765Z] 22:11:17    FATAL - Caught exception: 42
[task 2017-01-12T22:11:17.232422Z] 22:11:17    FATAL - Running post_fatal callback...
[task 2017-01-12T22:11:17.233995Z] 22:11:17    FATAL - Exiting -1
Attachment #8826411 - Flags: review?(aki)

Comment 36

2 years ago
Comment on attachment 8826411 [details] [diff] [review]
fix retry of  query_symbols_url

>-    def query_symbols_url(self):
>+    def query_symbols_url(self, raise_on_failure=False):
>         if self.symbols_url:
>             return self.symbols_url
> 
>@@ -206,8 +206,11 @@ class TestingMixin(VirtualenvMixin, Buil
>                 if symbols_url:
>                     self._urlopen(symbols_url, timeout=120)
>                     self.symbols_url = symbols_url
>-            except (urllib2.HTTPError, urllib2.URLError, socket.error, socket.timeout):
>-                self.warning("Can't figure out symbols_url from installer_url: %s!" % self.installer_url)
>+            except (urllib2.HTTPError, urllib2.URLError, socket.error, socket.timeout) as ex:
>+                self.warning("Cannot open symbols url %s (installer url: %s): %s" %
>+                    (symbols_url, self.installer_url, ex))
>+                if raise_on_failure:
>+                    raise

Good catch! We retry on exceptions.

> 
>         # If no symbols URL can be determined let minidump_stackwalk query the symbols.
>         # As of now this only works for Nightly and release builds.
>@@ -524,6 +527,7 @@ 2. running via buildbot and running the 
>             # before being unable to proceed (e.g. debug tests need symbols)
>             self.symbols_url = self.retry(
>                 action=self.query_symbols_url,
>+                args=(True,),

This will work because raise_on_failure is the first kwarg for query_symbols_url.  Relying on that order is more implicit than explicit, and can break if someone introduces another kwarg in the wrong place.

kwargs={'raise_on_failure': True} would be more explicit.
Attachment #8826411 - Flags: review?(aki) → review+
(In reply to Aki Sasaki [:aki] from comment #36)
> kwargs={'raise_on_failure': True} would be more explicit.

Good idea - will do. Thanks!

Comment 38

2 years ago
Pushed by gbrown@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/40e0595028bf
Actually retry when mozharness query_symbols_url() fails; r=aki

Comment 39

2 years ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/40e0595028bf
Status: REOPENED → RESOLVED
Last Resolved: 2 years ago2 years ago
status-firefox53: --- → fixed
Resolution: --- → FIXED

Comment 40

2 years ago
14 failures in 722 pushes (0.019 failures/push) were associated with this bug in the last 7 days.  

Repository breakdown:
* mozilla-inbound: 7
* autoland: 6
* mozilla-central: 1

Platform breakdown:
* android-4-3-armv7-api15: 5
* linux64: 4
* windows10-64-vm: 2
* linux32: 2
* windows7-32-vm: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1303668&startday=2017-01-09&endday=2017-01-15&tree=all
status-firefox50: affected → wontfix
status-firefox52: fixed → affected

Comment 41

2 years ago
bugherderuplift
https://hg.mozilla.org/releases/mozilla-aurora/rev/d31636b6dbbc
status-firefox52: affected → fixed
See Also: → bug 1301807
You need to log in before you can comment on or make changes to this bug.