Closed
Bug 1213782
Opened 9 years ago
Closed 8 years ago
Upload buildbot logs via the new upload hosts
Categories
(Release Engineering :: General, defect)
Release Engineering
General
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: nthomas, Unassigned)
References
Details
Attachments
(2 files)
2.52 KB,
patch
|
rail
:
review+
nthomas
:
checked-in+
|
Details | Diff | Splinter Review |
1.12 KB,
patch
|
hwine
:
review+
nthomas
:
checked-in+
|
Details | Diff | Splinter Review |
No description provided.
Reporter | ||
Comment 1•9 years ago
|
||
* swap to new upload hosts * drops dead gaia-try and shadow-central, xulrunner * for thunderbird try, makes a guess which way bug 1213723 will go Holding on review until we know more.
Reporter | ||
Comment 2•9 years ago
|
||
Comment on attachment 8672551 [details] [diff] [review] [puppet] New hosts & tidy up Bug 1213723 is still pending, but this looks good in staging for everything except try-comm-central. Deployment of this might be an interesting problem. It's going to go something like this * oremj deploys cloudfront change, directing requests for firefox, mobile etc to S3 instead of SCL3. Propagation time is ~20 min * we land this patch in puppet, and masters pick it up within 30 minutes (unless we force). All logs from that point will be uploaded to S3 instead of SCL3 * logs that get uploaded to stage will report locations with ftp.m.o in them, which will point to SCL3 before the transition and S3 afterwards. After we swap they'll be archive.mozilla.org * we're keep syncing SCL3 -> S3 until all the jobs with old config finish So there's a couple of things that will go wrong here * upload to stage after cloudfront switches - log is missing until sync picks it up * upload to S3 before cloudfront switches - log is missing until propagation is done We might just have to put up with a period where treeherder can't get logs. emorley, what's the best bet here ?
Flags: needinfo?(emorley)
Attachment #8672551 -
Flags: review?(rail)
Comment 3•9 years ago
|
||
I'm not sure I understand what "uploaded to stage" means? Do you meaning a staging instance or is that the name of another downloads area? Treeherder doesn't automatically retry 404 or 403 logs (https://github.com/mozilla/treeherder/blob/master/treeherder/log_parser/utils.py#L101) since it commonly happens (at least in the past, not sure now) with TaskCluster. However when users click the job in the UI it should initiate a repeat parse, so the impact shouldn't be too bad.
Flags: needinfo?(emorley)
Reporter | ||
Comment 4•9 years ago
|
||
Sorry, I meant stage.mozilla.org, the soon-to-be-deprecated upload host. Sounds like repeat parse will save the day, thanks.
Updated•9 years ago
|
Attachment #8672551 -
Flags: review?(rail) → review+
Reporter | ||
Comment 5•9 years ago
|
||
Comment on attachment 8672551 [details] [diff] [review] [puppet] New hosts & tidy up https://hg.mozilla.org/build/puppet/rev/0a5caeb217b5 https://hg.mozilla.org/build/puppet/rev/cc97b416245f
Attachment #8672551 -
Flags: checked-in+
Reporter | ||
Comment 6•9 years ago
|
||
We still need to upload b2g logs via stage.mozilla.org. It will transition later.
Attachment #8676505 -
Flags: review?(hwine)
Comment 7•9 years ago
|
||
Comment on attachment 8676505 [details] [diff] [review] [puppet] Fix b2g lgtm!
Attachment #8676505 -
Flags: review?(hwine) → review+
Reporter | ||
Comment 8•9 years ago
|
||
Comment on attachment 8676505 [details] [diff] [review] [puppet] Fix b2g https://hg.mozilla.org/build/puppet/rev/ec151da42a58 https://hg.mozilla.org/build/puppet/rev/07c547efcba4
Attachment #8676505 -
Flags: checked-in+
Reporter | ||
Comment 9•9 years ago
|
||
We also had some problems uploading logs for mobile builds on try, when s3://net-mozaws-prod-delivery-archive/pub/mobile/try-builds/ wasn't writeable from upload.trybld.productdelivery.prod.mozaws.net. oremj added new policy to allow that, but we'll have to loop around and figure out the proper fix. On stage.mozilla.org we had a symlink from mobile/trybuilds to ../firefox/try-builds.
Reporter | ||
Comment 10•8 years ago
|
||
Bug 1217742 and bug 1217223 were also relevant here.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Assignee | ||
Updated•6 years ago
|
Component: General Automation → General
You need to log in
before you can comment on or make changes to this bug.
Description
•