Closed
Bug 1387622
Opened 7 years ago
Closed 7 years ago
Add latest-ssl aliases for bedrock to use
Categories
(Release Engineering :: Release Automation: Other, enhancement, P1)
Release Engineering
Release Automation: Other
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: mtabara, Assigned: nthomas)
References
Details
(Whiteboard: [releaseduty])
Attachments
(3 files)
59 bytes,
text/x-review-board-request
|
mtabara
:
review+
|
Details |
2.61 KB,
patch
|
mtabara
:
review+
nthomas
:
checked-in+
|
Details | Diff | Splinter Review |
969 bytes,
patch
|
rail
:
review+
|
Details | Diff | Splinter Review |
So this is quite an interesting thing I discovered today while talking to :catlee about the 55.0 release. Context: === 1. Among the first tasks to run in the beta/release graph, there is the bouncer submission one. This task lies here[1] and uses this config file[2] (e.g. beta). For each of those products, per each platform it submits all those paths (paths to mirrors future location of the artifacts) to bouncer[3]. 2. "release-localtest" updates are served directly from the candidates directory so this is easy. 3. "release-cdntest" updates are served via downloads page[4]. The downloads page gets its information from bouncer. If we haven't pushed to mirrors yet, all these updates, even though retrieved as valid urls from Balrog, would result in 404. That is because bouncer checks (well, technically Sentry) if files exist at the locations before redirecting to them. Since no push-to-mirrors happened, there's gonna be 404. So as soon as we complete this step (push-to-mirrors), updates to "release-cdntest" become available. 4. After we publish to "release" channel, there are a couple of builders which get unblocked, out of which are: 4.1 mark the release as shipped in Ship-it Once we mark a release as shipped, there's a chain of tools that eventually get updated: "ship-it -> product-details -> product-details-json -> bedrock -> www". This usually takes about 30mins or less. 4.2 update bouncer aliases We've got a small script[5] that talks to bouncer aliases based on this[2] configs, and tells it to update the `firefox-latest` and `firefox-stub` to the current release. This means that once this builder is completed, all urls like this[6], regardless of the platform, will serve the latest version. Then what is the problem then? === If we glance at the source of the downloads page[7], we notice that except the Win32 installer which gets served by querying the `firefox-stub`, none of the other platforms do so, but instead have versioned urls: https://download.mozilla.org/?product=firefox-stub&os=win&lang=en-US https://download.mozilla.org/?product=firefox-54.0.1-SSL&os=win64&lang=en-US https://download.mozilla.org/?product=firefox-54.0.1-SSL&os=osx&lang=en-US https://download.mozilla.org/?product=firefox-54.0.1-SSL&os=linux&lang=en-US https://download.mozilla.org/?product=firefox-54.0.1-SSL&os=linux64&lang=en-US Historically, the bouncer aliases were added specifically to support the `firefox-stub` so that we can offer the latest stub installer to win32 users. But question is, now that we have the context and logic to support this operation for all the other platforms, why are we still having this urls versioned? Solution If we move these urls to point to "firefox-latest", they will be available as soon as the bouncer aliases builder job runs in our graph. That is ~30 mins faster than waiting for "ship-it -> product-details -> product-details-json -> bedrock -> www" chain. Resolution * we need to investigate more to make sure we're not misunderstanding something here * we need to rope in all the teams involved along the line * see if we can do this and if this makes sense. Observation: * bedrock is just for releases, not for nightly. nightlies version from downloads page[7] talks directly to bouncer and is part of the reason why we manually update[8] these at each nightly => beta version bump. This is the reason why we update bouncer directly during mergeduty[8] each time we bump the version in nightly. [1]: https://dxr.mozilla.org/mozilla-central/rev/52285ea5e54c73d3ed824544cef2ee3f195f05e6/testing/mozharness/scripts/bouncer_submitter.py [2]: https://dxr.mozilla.org/mozilla-central/rev/52285ea5e54c73d3ed824544cef2ee3f195f05e6/testing/mozharness/configs/releases/bouncer_firefox_release.py [3]: http://bounceradmin.mozilla.com/ [4]: http://download.mozilla.org/?product=firefox-55.0.... [5]: https://dxr.mozilla.org/mozilla-central/rev/52285ea5e54c73d3ed824544cef2ee3f195f05e6/testing/mozharness/scripts/release/postrelease_bouncer_aliases.py [6]: https://download.mozilla.org/?product=firefox-54.0.1-SSL&os=OS_TO_FILL&lang=en-US [7]: view-source:https://www.mozilla.org/en-US/firefox/new/?scene=2 [8]: https://github.com/mozilla/releasewarrior/blob/master/how-tos/mergeduty.md#bump-bouncer-versions
Reporter | ||
Comment 1•7 years ago
|
||
I tested today during the 55.0 and it does seem like https://download.mozilla.org/?product=firefox-latest&os=osx&lang=en-GB was updated as soon as the `bouncer-aliases` builder run successfully. This means that if we can have a broader conversation with downloads.page website admins, we can change those urls to point to latest-firefox or alike so that we go live right away and we no longer need to wait for all these changes to propagate via ship-it => product details => bedrock => www, etc.
Reporter | ||
Comment 2•7 years ago
|
||
Note to self: pages we investigate while waiting for propagation after marking releases as shipped in Ship-it. https://ship-it.mozilla.org/json/1.0/firefox_versions.json https://ship-it.mozilla.org/json/1.0/mobile_versions.json https://product-details.mozilla.org/1.0/firefox_versions.json https://product-details.mozilla.org/1.0/mobile_versions.json | v bedrock | v downloads page www.
Assignee | ||
Comment 3•7 years ago
|
||
Dupe of bug 937865 ?
Reporter | ||
Comment 4•7 years ago
|
||
(In reply to Nick Thomas [:nthomas] from comment #3) > Dupe of bug 937865 ? Holy smokes, a six-digits bug to reference! :) Reading your comment https://bugzilla.mozilla.org/show_bug.cgi?id=937865#c13 I think we can keep this and rename it once we have some RelEng work to do.
See Also: → 937865
Reporter | ||
Comment 5•7 years ago
|
||
Sounds like we'll soon be reaching a consensus in bug 937865 comment 14 so we should pay a bit more attention do this. With a bit of luck, we may be shipping 56 faster in September!
Priority: -- → P1
Whiteboard: [releaseduty]
Updated•7 years ago
|
Assignee: mtabara → nobody
Priority: P1 → P3
Assignee | ||
Comment 6•7 years ago
|
||
bug 937865 comment 14 asks for * firefox-beta-latest-ssl * firefox-latest-ssl * firefox-esr-latest-ssl Should be easy to add.
Assignee | ||
Comment 7•7 years ago
|
||
I've manually added those 3 products in comment #6 as well as firefox-sha1-ssl, and set them to point at the current -SSL release. And removed devedition-latest, devedition-latest-ssl, and devedition-stub, and firefox-latest-euballot.
Comment hidden (mozreview-request) |
Assignee | ||
Comment 9•7 years ago
|
||
TODO - update https://hg.mozilla.org/build/nagios-tools/file/tip/nagios_tools/scripts/check_bouncer.py#l64
Assignee | ||
Comment 10•7 years ago
|
||
TODO - make sure people know how www.mozilla.org will behave differently after this goes live
Comment 11•7 years ago
|
||
Thanks Nick! Can't wait to update bedrock to use this!
Reporter | ||
Comment 12•7 years ago
|
||
mozreview-review |
Comment on attachment 8903101 [details] Bug 1387622 - add latest-ssl aliases for bedrock, https://reviewboard.mozilla.org/r/174884/#review180984 This is looking good! Excuse the n00b question, but please remind me again, why are we killing devedition bouncer entries?
Attachment #8903101 -
Flags: review?(mtabara) → review+
Assignee | ||
Comment 13•7 years ago
|
||
The www folks wanted to use firefox-devedition-* ones, see bug 937865 comment #14.
Comment 14•7 years ago
|
||
Pushed by nthomas@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/b5cbf8e3f776 add latest-ssl aliases for bedrock, r=mtabara
Assignee | ||
Comment 15•7 years ago
|
||
https://hg.mozilla.org/releases/mozilla-beta/rev/837b0dc3e6ee86e60f00111d244146d708d012a5 https://hg.mozilla.org/releases/mozilla-release/rev/90025483ae5ab1c0f750346ce8cc0cf439b5b9b8 https://hg.mozilla.org/releases/mozilla-esr52/rev/593b3587b73265742c9afd99a580e008f690a287
Keywords: leave-open
Assignee | ||
Comment 16•7 years ago
|
||
No rush on this, please leave it until you get back. If you haven't seen this before it's a nagios check that runs on bm81 to make sure the important bouncer products are working. The code lives at https://hg.mozilla.org/build/nagios-tools/. I've added checks for firefox-latest-ssl and firefox-beta-latest-ssl, in anticipation of bedrock using them. Noticed we don't have anything for esr so corrected that with the new aliases. For nightly I'm just tidying up based on what we'll want after bug 1377842 (one day!). And a version bump because this is deployed as a python package via puppet.
Attachment #8904389 -
Flags: review?(mtabara)
Reporter | ||
Comment 17•7 years ago
|
||
(In reply to Nick Thomas [:nthomas] from comment #13) > The www folks wanted to use firefox-devedition-* ones, see bug 937865 > comment #14. Ah, makes sense. Misread the bouncer names at first glance.
Reporter | ||
Comment 18•7 years ago
|
||
Comment on attachment 8904389 [details] [diff] [review] [nagios-tools] Update bouncer checks Lgtm! I broke this script before ^_^ we've made our acquaintances when I last changed some bouncer entries and used " " instead of "%20".
Attachment #8904389 -
Flags: review?(mtabara) → review+
Comment 19•7 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/b5cbf8e3f776
Assignee | ||
Comment 20•7 years ago
|
||
Comment on attachment 8904389 [details] [diff] [review] [nagios-tools] Update bouncer checks https://hg.mozilla.org/build/nagios-tools/rev/e05e5a010588c99a539a343d4eca10f07c5bf030
Attachment #8904389 -
Flags: checked-in+
Assignee | ||
Comment 21•7 years ago
|
||
Added nagios-tools 0.6 to the puppet server and deployed it with https://hg.mozilla.org/build/puppet/rev/b8ddcebb1cde8a820f4b9ff5ed374ab0af7cdc03 https://hg.mozilla.org/build/puppet/rev/615a2af805b3a1d6c840e89eb1c140c1754ddad6 All done here.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Assignee | ||
Comment 22•7 years ago
|
||
Forgot to say that all future releases will update the aliases in an automated fashion. I'll do that manually for 56.0b9 when it ships in the next 12-24 hours.
Comment 23•7 years ago
|
||
I'm implementing this in bedrock now. I have a question: What will firefox-esr-latest-ssl return when there are 2 current ESR releases. In product-details we get a version for ESR and one for ESR_NEXT during the overlapping time. Will firefox-esr-latest-ssl return the current ESR or the NEXT one during that time? Given an option I'd prefer for it to stick with the previous release until ESR_NEXT becomes the only active release so that I can always use the "latest" product for the current ESR and only bring out the version specific URLs for the "next" version when it's active. I should be able to work with it either way, but I just need to know for implementation purposes.
Updated•7 years ago
|
Flags: needinfo?(nthomas)
Comment 24•7 years ago
|
||
Reopening this, because we stopped updating "firefox-sha1" by https://hg.mozilla.org/mozilla-central/rev/b5cbf8e3f776#l2.30, which is used by bouncer: https://github.com/rail/go-bouncer/blob/master/handlers_test.go#L183 to redirect XP users to latest ESR build. I don't think we use firefox-sha1-ssl anywhere publicly available, so probably we can just revert that line.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Reporter | ||
Comment 25•7 years ago
|
||
(In reply to Rail Aliiev [:rail] ⌚️ET from comment #24) > Reopening this, because we stopped updating "firefox-sha1" by > https://hg.mozilla.org/mozilla-central/rev/b5cbf8e3f776#l2.30, which is used > by bouncer: > https://github.com/rail/go-bouncer/blob/master/handlers_test.go#L183 to > redirect XP users to latest ESR build. > > I don't think we use firefox-sha1-ssl anywhere publicly available, so > probably we can just revert that line. Manually updated firefox-sha to point to latest ESR for both production and staging instances. Had no idea staging is in such a good shape, not sure who's updating it but it's almost a production mirror.
Assignee | ||
Comment 26•7 years ago
|
||
(In reply to Paul [:pmac] McLanahan ⏰ET needinfo? me from comment #23) > What will firefox-esr-latest-ssl return when there are 2 current ESR > releases. In product-details we get a version for ESR and one for ESR_NEXT > during the overlapping time. Will firefox-esr-latest-ssl return the current > ESR or the NEXT one during that time? Given an option I'd prefer for it to > stick with the previous release until ESR_NEXT becomes the only active > release so that I can always use the "latest" product for the current ESR > and only bring out the version specific URLs for the "next" version when > it's active. The short answer is: it's not well defined in our process. When starting up ESR52 we had bug 1347754, which disabled the aliasing on ESR45 about the time of 52.0.1. I have a memory that we were offering 52.x.y on www.mozilla.org during the overlap with 45, but perhaps that's faulty based on your comments. We can pretty much do whatever we want here though, same amount of work either way.
Flags: needinfo?(nthomas)
Comment 27•7 years ago
|
||
The way it's handled on the site is that both versions are listed on the download page when FIREFOX_ESR_NEXT is defined in product-details[0]. You're right that there is a planned 6 week overlap after a new ESR release and they're both available at: https://www.mozilla.org/en-US/firefox/organizations/all/ Which is the only page from which ESR is downloadable. So either way will work for me I think. What I believe will be easiest is for the firefox-esr-latest-ssl alias to stay on the old version during the transitional period, and we can use the value of ESR_NEXT in product-details to produce those download buttons. A really-really easy thing would be for there to exist a firefox-esr-next-latest-ssl alias when a new ESR is released so that we can use aliases for both, but that is really just me wishing out loud (no idea how difficult these things are to do). Hope that's clear and helpful. [0] https://product-details.mozilla.org/1.0/firefox_versions.json
Assignee | ||
Comment 28•7 years ago
|
||
Attachment #8915742 -
Flags: review?(rail)
Updated•7 years ago
|
Attachment #8915742 -
Flags: review?(rail) → review+
Assignee | ||
Comment 29•7 years ago
|
||
Comment on attachment 8915742 [details] [diff] [review] [gecko] Revert firefox-sha1 alias change https://hg.mozilla.org/releases/mozilla-esr52/rev/31949068fe2c9cf147f00df0b8ccda57d74a337f TODO: landed on central/beta/release when they reopen.
Reporter | ||
Comment 30•7 years ago
|
||
(In reply to Nick Thomas [:nthomas] (PTO 9-11 Oct) from comment #29) > Comment on attachment 8915742 [details] [diff] [review] > [gecko] Revert firefox-sha1 alias change > > https://hg.mozilla.org/releases/mozilla-esr52/rev/ > 31949068fe2c9cf147f00df0b8ccda57d74a337f > > TODO: landed on central/beta/release when they reopen. Found in triaging: did we land this already or still need to cherry-pick that?
Comment 31•7 years ago
|
||
Pushed by nthomas@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/c49fa16162cb revert firefox-sha1 alias change, r=rail a=release DONTBUILD
Assignee | ||
Comment 32•7 years ago
|
||
https://hg.mozilla.org/releases/mozilla-release/rev/9b143216bc1b54dabc11011ac2bccf3218587f14 https://hg.mozilla.org/releases/mozilla-beta/rev/ea66f4c8766c9fde44f15ae53ea438377cb78283 https://hg.mozilla.org/integration/mozilla-inbound/rev/c49fa16162cb7161e942421f0b7d2f428e46e38c (m-r and m-b for consistency only). Spun the ESR59 discussion out to bug 1408868.
Status: REOPENED → RESOLVED
Closed: 7 years ago → 7 years ago
Resolution: --- → FIXED
See Also: → 1408868
Comment 33•7 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/c49fa16162cb
Comment 34•6 years ago
|
||
Removing leave-open keyword from resolved bugs, per :sylvestre.
Keywords: leave-open
You need to log in
before you can comment on or make changes to this bug.
Description
•