Closed
Bug 1386007
Opened 8 years ago
Closed 8 years ago
Release runner graph submission no longer works properly
Categories
(Release Engineering :: Release Automation, enhancement, P1)
Release Engineering
Release Automation
Tracking
(Not tracked)
RESOLVED
WORKSFORME
People
(Reporter: mtabara, Unassigned)
Details
(Whiteboard: [releaseduty])
Something is fishy. We've had these issues last week as well but I thought they were intermittent. We need to dive in more for these.
Behavior:
* the graph is being generated and submitted
* the emails no longer arrive so there must be some new error thrown that makes the release runner script barf before submitting the actual emails notifying release-drivers
Example:
2017-07-31 12:24:29,684 - DEBUG - Making attempt 7
2017-07-31 12:24:29,684 - DEBUG - Making a PUT request to https://scheduler.taskcluster.net/v1/task-graph/0rKw_NbcTjKuroEEWveJmg
2017-07-31 12:24:29,684 - DEBUG - HTTP Headers: {'Content-Type': 'application/json', 'Authorization': 'Hawk id=".....", ts="......", , ext="e30=", mac="....."'}
2017-07-31 12:24:29,684 - DEBUG - HTTP Payload: {"scopes":["project:releng:signing:format:gpg","queue:define-task:buildbot-bridge/buildbot-bridge"," (limit 100 char)
2017-07-31 12:24:29,685 - INFO - Starting new HTTPS connection (1): scheduler.taskcluster.net
/builds/releaserunner/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:79: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
2017-07-31 12:25:02,307 - DEBUG - "PUT /v1/task-graph/0rKw_NbcTjKuroEEWveJmg HTTP/1.1" 503 506
2017-07-31 12:25:02,309 - DEBUG - Received HTTP Status: 503
2017-07-31 12:25:02,310 - DEBUG - Received HTTP Headers: {'content-length': '506', 'server': 'Cowboy', 'connection': 'keep-alive', 'cache-control': 'no-cache, no-store', 'date': 'Mon, 31 Jul 2017 19:25:02 GMT', 'content-type': 'text/html; charset=utf-8'}
2017-07-31 12:25:02,310 - DEBUG - Received HTTP Payload: <!DOCTYPE html>
<html>
<head>
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta charset="utf-8">
<title>Application Error</title>
<style media="screen">
html,body,iframe {
margin: 0;
padding: 0;
}
html,body {
height: 100%;
overflow: hidden;
}
iframe {
width: 100%;
height: 100%;
border: 0;
}
</style>
</head>
<body>
<iframe src="//www.herokucdn.com/error-pages/application-error.html"></iframe>
</body>
</html> (limit 1024 char)
2017-07-31 12:25:02,310 - WARNING - Retrying because of: 503 Server Error: Service Unavailable
2017-07-31 12:25:02,310 - INFO - Sleeping 6.40 seconds for exponential backoff
2017-07-31 12:25:08,717 - DEBUG - parsed URL parts: {'hostname': u'scheduler.taskcluster.net',
'path': u'/v1/task-graph/0rKw_NbcTjKuroEEWveJmg',
'port': 443,
'query': '',
'resource': u'/v1/task-graph/0rKw_NbcTjKuroEEWveJmg',
'scheme': u'https'}
2017-07-31 12:25:08,718 - DEBUG - artifacts={'app': None,
'dlg': None,
'ext': 'e30=',
'hash': None,
'host': u'scheduler.taskcluster.net',
'method': u'put',
'nonce': 'JKrMRj',
'port': 443,
'resource': u'/v1/task-graph/0rKw_NbcTjKuroEEWveJmg',
'ts': 1501529108}
2017-07-31 12:25:08,718 - DEBUG - Making attempt 8
|
Reporter | |
Comment 1•8 years ago
|
||
Hm, sounds like both graphs were generated in the end, unlike last week when no emails arrived. Might be a scheduler TC thing. I'll close for now and will reopen if it happens again this week.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → WORKSFORME
You need to log in
before you can comment on or make changes to this bug.
Description
•