1203907 - Lightning assumes upload and download hosts are the same

Status: RESOLVED FIXED

(Calendar :: Build Config, defect)

Description

[Nick Thomas [:nthomas] (UTC+12)]

... and that won't be true once we move to S3. 

eg 
make -C ../../calendar/lightning LOCALE_MERGEDIR=/builds/slave/tb-rel-c-esr38-m64_rpk_1-00000/comm-esr38/obj-l10n/mail/locales/merged langpack-zh-TW
(cd ../../dist/xpi-stage && wget -nv -N http://upload.tbirdbld.productdelivery.stage.mozaws.net/pub/mozilla.org/calendar/lightning/nightly/4.0.8-candidates/build2/mac/lightning-4.0.8.en-US.mac.xpi)

We need to allow for a different host than upload.tbirdbld.productdelivery.stage.mozaws.net. I'm thinking we'll just have a DOWNLOAD_HOST variable in the environment and use that in calendar/lightning/lightning-packager.mk. Sound OK ?

Comment 1

[Philipp Kewisch [:Fallen] [:📆][:🧩]]

Yes, DOWNLOAD_HOST seems fine for me, it should be pretty easy to adjust for that. Thanks for keeping Lightning in mind!

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Use DOWNLOAD_HOST

This should land at the same time as attachment 8662901 [details] [diff] [review], on both comm-beta and comm-esr38.

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

By the way, it swaps from nightly/ to candidates/ because we won't have symlinks after we transition.

Comment 4

[Philipp Kewisch [:Fallen] [:📆][:🧩]]

Comment on attachment 8670666 [details] [diff] [review]
Use DOWNLOAD_HOST

Review of attachment 8670666 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks for the patch, r=philipp

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

We'll need to get this landed. Do we have EN_US_BINARY_URL set in central & aurora nightlies ? I can't see that anywhere in dxr so I think I'll need a buildbot patch to set DOWNLOAD_HOST for those. For release automation it's set by this new code 
   http://hg.mozilla.org/build/tools/rev/b2d4cac21f15#l8.34

Comment 6

[Philipp Kewisch [:Fallen] [:📆][:🧩]]

EN_US_BINARY_URL is set for l10n nightlies, see for example
http://ftp.mozilla.org/pub/mozilla.org/thunderbird/tinderbox-builds/comm-central-l10n/comm-central-win32-l10n-dep-ru-bm86-build1-build19.txt.gz

From reading the comments in lightning-packager.mk, it seems that in some cases release repacks didn't set EN_US_BINARY_URL correctly.

Comment 7

[Nick Thomas [:nthomas] (UTC+12)]

Nice, that must be via
  https://dxr.mozilla.org/build-central/source/buildbotcustom/process/factory.py#3349
and
  https://dxr.mozilla.org/build-central/source/buildbot-configs/mozilla/thunderbird_config.py#755

So we can land this on central and aurora without waiting to see if there's any bustage, and get it onto beta and esr38 in time for the 13th.

Comment 8

[Philipp Kewisch [:Fallen] [:📆][:🧩]]

Comment on attachment 8670666 [details] [diff] [review]
Use DOWNLOAD_HOST

Sounds good to me, let's do it.

Comment 9

[Nick Thomas [:nthomas] (UTC+12)]

We're migrating tomorrow afternoon, and this will need to land for releases which start after that.

Comment 10

[aleth [:aleth]]

https://hg.mozilla.org/comm-central/rev/5d79bfdedcc534d6d1a1da41b97ed3d1f55b4cce
Bug 1203907 - Lightning assumes upload and download hosts are the same. r=Fallen a=aleth CLOSED TREE

Comment 11

[aleth [:aleth]]

Note: unbitrotted before landing on c-c.

Comment 12

[Philipp Kewisch [:Fallen] [:📆][:🧩]]

This still needs backporting to to all branches.

Comment 13

[Nick Thomas [:nthomas] (UTC+12)]

38.4.0 build is failing because the backporting didn't happen yet.

Comment 14

[Kent James (:rkent)]

(In reply to Philipp Kewisch [:Fallen] from comment #12)
> This still needs backporting to to all branches.

I don't track approval-calendar as part of my release prep. Is that normally something Fallen would do?

Comment 15

[Philipp Kewisch [:Fallen] [:📆][:🧩]]

In calendar the patch author usually takes care of backporting, although this isn't something I expect releng folks need to adhere to or know.

This took the train to aurora, here is the checkin for beta and esr38:

https://hg.mozilla.org/releases/comm-beta/rev/7688c970ea1c
https://hg.mozilla.org/releases/comm-esr38/rev/502ef538ce9b

Comment 16

[Nick Thomas [:nthomas] (UTC+12)]

This is still failing, because I forgot to update the path from calendar/nightly to calendar/candidates.

Comment 17

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Use candidates path

The main fix here is s/nightly/candidates/. Previously we had symlinks from nightly back to candidates, but no longer since migrating to S3. Also drops mozilla.org from the path, since that's no longer used.

Failing
$ curl -I http://archive.mozilla.org/pub/mozilla.org/calendar/lightning/nightly/4.0.4-candidates/build2/win32/lightning-4.0.4.en-US.win32.xpi
HTTP/1.1 404 Not Found
becomes
$ curl -I http://archive.mozilla.org/pub/calendar/lightning/candidates/4.0.4-candidates/build2/win32/lightning-4.0.4.en-US.win32.xpi
HTTP/1.1 200 OK

Comment 18

[Philipp Kewisch [:Fallen] [:📆][:🧩]]

Comment on attachment 8687717 [details] [diff] [review]
Use candidates path

Review of attachment 8687717 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks for the updated patch, r=philipp

Comment 19

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8687717 [details] [diff] [review]
Use candidates path

https://hg.mozilla.org/comm-central/rev/49a5d80772d9
https://hg.mozilla.org/releases/comm-aurora/rev/0257a4aa8b81
https://hg.mozilla.org/releases/comm-beta/rev/7a62db58ff85
https://hg.mozilla.org/releases/comm-esr38/rev/86aaf7d1e621

Comment 20

[Nick Thomas [:nthomas] (UTC+12)]

We should be good to build 38.4.0 build3 from default now.

Comment 21

[:aceman]

Looks like this is landed.

What about the DOWNLOAD_HOST patch? Is it to be landed, or was that an abandoned alternative?

Comment 22

[Nick Thomas [:nthomas] (UTC+12)]

Attachment 8670666 [details] [diff] landed at comment #10 and comment #15. We're done here.

Comment 23

[:aceman]

Comment on attachment 8670666 [details] [diff] [review]
Use DOWNLOAD_HOST

OK, thanks.