Fix upload_symbols.py's retry logic and log more information when upload_symbols.py gets an error from the symbol upload API

RESOLVED FIXED in Firefox 39, Firefox OS v2.0

Status

RESOLVED FIXED
4 years ago
9 months ago

People

(Reporter: ted, Assigned: ted)

Tracking

Trunk
mozilla41
Dependency tree / graph

Firefox Tracking Flags

(firefox39 fixed, firefox40 fixed, firefox41 fixed, firefox-esr31 fixed, firefox-esr38 fixed, b2g-v2.0 fixed, b2g-v2.0M fixed, b2g-v2.1 fixed, b2g-v2.1S fixed, b2g-v2.2 fixed, b2g-master fixed)

Details

Attachments

(2 attachments, 1 obsolete attachment)

Currently when upload_symbols.py gets an error from the symbol upload API it just prints the error status like:
04:52:23     INFO -  Error: got HTTP response 400

This is not super helpful. It should dump the full response headers + body for diagnostics.
Created attachment 8611322 [details]
MozReview Request: bz://1168535/ted

/r/9465 - bug 1168535 - print error response body in upload_symbols.py. r?gps

Pull down this commit:

hg pull -r 3d70a156280c91e4934ec0da60f2d8db2638064c https://reviewboard-hg.mozilla.org/gecko/
Attachment #8611322 - Flags: review?(gps)
I tested this locally by switching the upload URL to point at a local python httpd that served error codes. Running `./mach build uploadsymbols` it looks like:
```
 0:01.23 Error: got HTTP response 500: Error
 0:01.23 Response body:
 0:01.23 ====================
 0:01.23 <head>
 0:01.23 <title>Error response</title>
 0:01.23 </head>
 0:01.23 <body>
 0:01.23 <h1>Error response</h1>
 0:01.23 <p>Error code 500.
 0:01.23 <p>Message: Error.
 0:01.23 <p>Error code explanation: 500 = Server got itself in trouble.
 0:01.23 </body>
 0:01.23 
 0:01.23 ====================
 0:01.23 
 0:01.25 make: *** [uploadsymbols] Error 1
```

Also I stuck a -u in the Python invocation to unbuffer the output, since the errors were going to stderr and thus the log looked weird because it would print the error and *then* the "Retrying".
Comment on attachment 8611322 [details]
MozReview Request: bz://1168535/ted

/r/9465 - bug 1168535 - print error response body in upload_symbols.py. r?gps
/r/9467 - bug 1168535 - re-open the zip file for each upload_symbols retry. r?gps

Pull down these commits:

hg pull -r 1f01910a42fd33cfb5d316f17c44fc940475134d https://reviewboard-hg.mozilla.org/gecko/
While I was writing that patch I figured out bug 1168368 so I wrote the patch to fix that too, but that bug is marked moco-confidential so I put the patch here.

Apparently I didn't fix bug 1165347 properly the first time.
Blocks: 1165347

Comment 7

4 years ago
Comment on attachment 8611322 [details]
MozReview Request: bz://1168535/ted

https://reviewboard.mozilla.org/r/9463/#review8251

Ship It!
Attachment #8611322 - Flags: review?(gps) → review+
Summary: Log more information when upload_symbols.py gets an error from the symbol upload API → Fix upload_symbols.py's retry logic and log more information when upload_symbols.py gets an error from the symbol upload API
https://hg.mozilla.org/mozilla-central/rev/accf471c90ce
https://hg.mozilla.org/mozilla-central/rev/87c37eb7ba36
Status: NEW → RESOLVED
Last Resolved: 4 years ago
status-firefox41: affected → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla41
Blocks: 1171527
status-b2g-v2.1S: --- → fixed
status-b2g-master: --- → fixed
Comment on attachment 8611322 [details]
MozReview Request: bz://1168535/ted
Attachment #8611322 - Attachment is obsolete: true
Attachment #8620362 - Flags: review+
Attachment #8620363 - Flags: review+
Created attachment 8620362 [details]
MozReview Request: bug 1168535 - print error response body in upload_symbols.py. r?gps
Created attachment 8620363 [details]
MozReview Request: bug 1168535 - re-open the zip file for each upload_symbols retry. r?gps

Updated

9 months ago
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.