Tracking bug for migration from ftp.m.o to S3 (firefox, mobile, thunderbird)

RESOLVED FIXED

Status

defect
RESOLVED FIXED
4 years ago
11 months ago

People

(Reporter: nthomas, Assigned: nthomas)

Tracking

(Depends on 2 bugs)

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(4 attachments, 1 obsolete attachment)

This is the get-off-the-netapp move from ftp cluster in SCL3, moving onto AWS S3. Cloud Ops is driving this, we need to adjust where we upload things.

We are going to migrate firefox/, mobile/, thunderbird/ and seamonkey/ on Oct 13.

Bug 1186297 has been tracking the CI side, but I'll add other bugs here.
Bug 1165405 has all the bugs for release automation fixes.
Depends on: 1213723
tl;dr stage.mozilla.org --> upload.ffxbld.productdelivery.prod.mozaws.net. This mainly affects branches like beta, release, and esr38 where we're not using mozharness.

We'll do b2g a little later, need to sort out things like bug 1211371 and bug 1211374 first. Fuzzing moves to pvtbuilds2 because it has private stuff, that gets sorted out later too. Drops mozilla.org in paths because that's being deprecated (but will still work via a 302), and swaps from ftp.m.o to archive.m.o (the new canonical domain).
Attachment #8672505 - Flags: review?(rail)
Attachment #8672505 - Attachment description: [buildbot-configs] Switch firefox, mobile, thunderbird but not b2g → [buildbot-configs] Switch CI for firefox, mobile, thunderbird but not b2g
Depends on: 1213772
Between a local python doing uploads to the server, a python call to post_upload on the server, and probably buildbot stdin/stdout handling, we routinely end up with output mangled all together. This seems to be worse with the new system, but this patch solves the problem.
Attachment #8672532 - Flags: review?(mh+mozilla)
The most important parts of this are configs/builds/build_pool_specifics.py and configs/single_locale/production.py, where the upload host is changing for Firefox desktop and mobile for all the branches except beta/release/esr38.

Other notable things
* the new upload host is going to respond with archive.m.o urls after post_upload, so translate that to the CDN for updates (mobile only given funsize for desktop)
* a bunch of noise setting up ash l10n for mobile
* s,(stage|ftp).mozilla.org/pub/mozilla.org,archive.mozilla.org/pub,g

This will need to land on a lot of branches when we transition over.
Attachment #8672536 - Flags: review?(rail)
Depends on: 1213782
Comment on attachment 8672536 [details] [diff] [review]
[gecko] mozharness config changes

Review of attachment 8672536 [details] [diff] [review]:
-----------------------------------------------------------------

OOOOOH!

::: testing/mozharness/configs/single_locale/ash_android-api-11.py
@@ +2,4 @@
>  MOZ_UPDATE_CHANNEL = "nightly"
>  MOZILLA_DIR = BRANCH
>  OBJDIR = "obj-l10n"
> +EN_US_BINARY_URL = "http://archive.mozilla.org/pub/mobile/nightly/latest-%s-android-api-11/en-US" % (BRANCH)

A nit. Can you either remove the parenthesis around BRANCH or add a trailing coma to make it a tuple. Something like:
  ... % (BRANCH,)
Attachment #8672536 - Flags: review?(rail) → review+
Attachment #8672505 - Flags: review?(rail) → review+
Comment on attachment 8672532 [details] [diff] [review]
[gecko] Unbuffered output for make upload

Review of attachment 8672532 [details] [diff] [review]:
-----------------------------------------------------------------

Kind of gross, but meh. I guess at some point, we should force make to do the right thing, since make 4 now supports that.
Attachment #8672532 - Flags: review?(mh+mozilla) → review+
Keywords: leave-open
With '% (BRANCH)'  --> '% BRANCH' changes. Carrying rail's review.
Attachment #8672536 - Attachment is obsolete: true
Attachment #8675927 - Flags: review+
Depends on: 1216859
Depends on: 1216883
Depends on: 1216907
Comment on attachment 8672505 [details] [diff] [review]
[buildbot-configs] Switch CI for firefox, mobile, thunderbird but not b2g

Landed yesterday http://hg.mozilla.org/build/buildbot-configs/rev/f500b8d4b883
Attachment #8672505 - Flags: checked-in+
Comment on attachment 8675927 [details] [diff] [review]
[gecko] mozharness config changes, v2

Landed yesterday on inbound at c7cedbb6e3fb, as well fx-team, b2g-inbound and aurora.

TODO - try to merge to beta/release/esr38, although we don't use mozharness for ci builds there. Maybe people push to try occasionally, although I'm not sure that works.
Attachment #8675927 - Flags: checked-in+
Blocks: 1217223
Depends on: 1217742
Should land on the project branches too, at least date is still uploading to scl3.
Depends on: 1217947
Depends on: 1217486
Depends on: 1218570
I forgot to cover the Thunderbird production case, so that's trying to use to upload.ffxbld instead of upload.tbirdbird. Also update download_base_url to use the new domain and drop mozilla.org from the path. I updated the try values, but I'm pretty sure it's not used there.
Attachment #8679738 - Flags: review?(jlund)
Attachment #8679738 - Flags: review?(jlund) → review+
Depends on: 1219588
Depends on: 1219592
Depends on: 1219607
No longer blocks: 1217223
Depends on: 1217223
Depends on: 1219626
Depends on: 1221391
Depends on: 1221742
Depends on: 1221917
Depends on: 1222872
Depends on: 1226460
Depends on: 1226991
Depends on: 1226992
Depends on: 1211732
Depends on: 1229532
Depends on: 1230144
Depends on: 1227945
Depends on: 1233623
Depends on: 1245581
No longer depends on: 1245581
Depends on: 1222878
Depends on: 1222693
Blocks: 1223633
Depends on: 1322344
Depends on: 1325283
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Removing leave-open keyword from resolved bugs, per :sylvestre.
Keywords: leave-open
Component: General Automation → General
Assignee: nobody → nthomas
You need to log in before you can comment on or make changes to this bug.