1127774 - Pushlog ingestion exceptions should include the full json-pushes URL

Status: RESOLVED FIXED

(Tree Management :: Treeherder: Data Ingestion, defect, P2)

Description

[Ed Morley [:emorley]]

This would have made things like:
https://rpm.newrelic.com/accounts/677903/applications/4180461/traced_errors/2998874433

Traceback (most recent call last):
File "/usr/lib64/python2.6/site-packages/newrelic-2.40.0.34/newrelic/hooks/application_celery.py", line 66, in wrapper return wrapped(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/celery-3.1.16-py2.6.egg/celery/app/trace.py", line 437, in __protected_call__ return self.run(*args, **kwargs)
File "/data/www/treeherder.mozilla.org/treeherder-service/treeherder/etl/tasks/buildapi_tasks.py", line 69, in fetch_hg_push_log process.run(repo_url + '/json-pushes/?full=1', repo_name)
File "/data/www/treeherder.mozilla.org/treeherder-service/treeherder/etl/pushlog.py", line 103, in run raise e
HTTPError: 500 Server Error: Internal Server Error

Much easier to debug. 

The logs didn't have anything more useful either:

--
[2015-01-30 02:47:00,207: ERROR/MainProcess] Task fetch-hg-push-logs[e750ab75-757b-477f-916a-5a35f17771f3] raised unexpected: HTTPError('500 Server Error: Internal Server Error',)
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 240, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/usr/lib64/python2.6/site-packages/newrelic-2.40.0.34/newrelic/hooks/application_celery.py", line 66, in wrapper

    return wrapped(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in __protected_call__
    return self.run(*args, **kwargs)
  File "/data/www/treeherder.mozilla.org/treeherder-service/treeherder/etl/tasks/buildapi_tasks.py", line 69, in fetch_hg_push_log
    process.run(repo_url + '/json-pushes/?full=1', repo_name)
  File "/data/www/treeherder.mozilla.org/treeherder-service/treeherder/etl/pushlog.py", line 103, in run
--
HTTPError: 500 Server Error: Internal Server Error

https://github.com/mozilla/treeherder-service/blob/45824d8507d3cdc154ff452b1abefa3b6f3d75d7/treeherder/etl/pushlog.py#L103

Or alternatively there's bug 1124278, but that wouldn't improve the quality of the logs, just new relic.

Comment 1

[Ed Morley [:emorley]]

Attachment: Include the URL in the logs for pushlog HTTPErrors

Comment 2

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/b1a2462f1ee25eacfcbeee26504d1aaa8734f020
Bug 1127774 - Include the URL in the logs for pushlog HTTPErrors

At some point as we move more things to using the requests library, we
should probably add some wrappers around requests to common.py and use
those instead.