Closed Bug 1256063 Opened 8 years ago Closed 8 years ago

Add "update bouncer aliases" builder

Categories

(Release Engineering :: Release Automation: Other, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: rail, Assigned: mtabara)

References

Details

Attachments

(6 files, 3 obsolete files)

58 bytes, text/x-review-board-request
rail
: review+
Details
58 bytes, text/x-review-board-request
rail
: review+
Details
58 bytes, text/x-review-board-request
rail
: review+
Details
58 bytes, text/x-review-board-request
rail
: review+
Details
58 bytes, text/x-review-board-request
rail
: review+
Details
58 bytes, text/x-review-board-request
rail
: review+
Details
The old process handles this in http://hg.mozilla.org/build/tools/file/tip/scripts/release/post-release.py and the corresponding configs are in http://hg.mozilla.org/build/buildbot-configs/file/tip/mozilla/release-firefox-mozilla-beta.py.template#l165

Basically, we need to update the *-latest aliases and point them to $product-$version.
First step to move the logic from tools to mozharness scripts.
Following-up will be patches to add builder and schedule this via BBB.
Attachment #8731819 - Flags: review?(rail)
Couldn't find mozharneess/configs/releases files equivalent for (fennec,thunderbird)-beta-latest. Should I add some?
Flags: needinfo?(rail)
(In reply to Mihai Tabara [:mtabara] from comment #2)
> Couldn't find mozharneess/configs/releases files equivalent for
> (fennec,thunderbird)-beta-latest. Should I add some?

Let's address those when we actually need them.
Flags: needinfo?(rail)
Comment on attachment 8731819 [details] [diff] [review]
Move post-release.py script from tools to intree mozharness scripts

Review of attachment 8731819 [details] [diff] [review]:
-----------------------------------------------------------------

In overall it looks good to me!

Just some nits:

::: testing/mozharness/scripts/release/postrelease_bouncer_aliases.py
@@ +26,5 @@
> +    config_options = [
> +        [["--version"], {
> +            "dest": "version",
> +            "help": "Version of release, eg: 39.0b5",
> +        }],

I don't see the integration part of this script yet, but I assume we are going to use buildbot for this, because we don't have a way to talk to bouncer outside of scl3.

In this case you need a way to pass the version and the url runtime (we don't have release configs where you can get your version from). https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/scripts/release/updates.py#90 may give you some ideas how to have args (so you can run it locally) and be able to override them using buildbot properties.

@@ +63,5 @@
> +                              related_product_template, alias):
> +        from redo import retry
> +
> +        url = "%s/create_update_alias" % tuxedoServerUrl
> +        related_product = related_product_template % {"version": version}

I'd probably move this interpolation logic to the caller.

@@ +72,5 @@
> +                                                              url))
> +
> +        # Wrap the real call to hide credentials from retry's logging
> +        def do_update_bouncer_alias():
> +            r = requests.post(url, data=data, auth=auth, verify=False)

can you add some timeout here? 60s?
Attachment #8731819 - Flags: review?(rail)
First of an upcoming series of patches - the mozharness script based upon rail's review.
Attachment #8731819 - Attachment is obsolete: true
Attachment #8734210 - Flags: review?(rail)
Comment on attachment 8734210 [details] [diff] [review]
Move post-release.py script from tools to intree mozharness scripts

Review of attachment 8734210 [details] [diff] [review]:
-----------------------------------------------------------------

Almost! :)

::: testing/mozharness/scripts/release/postrelease_bouncer_aliases.py
@@ +12,5 @@
> +"""
> +
> +import os
> +import sys
> +import requests

I think you cannot import requests here. It should be imported after you run virtualenv setup, like you did for redo.

@@ +67,5 @@
> +            else:
> +                self.warning("%s could not be found within buildprops" % prop)
> +                return
> +
> +    def _update_bouncer_alias(self, tuxedoServerUrl, auth, version,

version is unused in this function, kill it with fire! :)

@@ +86,5 @@
> +
> +        retry(do_update_bouncer_alias)
> +
> +    def update_bouncer_aliases(self):
> +        tuxedoServerUrl = self.config['tuxedo_server_url']

A nit. Everywhere in the script you use variable names lowercase_separated_by_underscores, except tuxedoServerUrl. Can you use tuxedo_server_url instead?

@@ +89,5 @@
> +    def update_bouncer_aliases(self):
> +        tuxedoServerUrl = self.config['tuxedo_server_url']
> +        credentials_file = os.path.join(os.getcwd(),
> +                                        self.config['credentials_file'])
> +        # TODO anyway readConfig from release.info under tools?

Feel free to remove this TODO line, there is no need to depend on a external repo for a single simple function.
Attachment #8734210 - Flags: review?(rail)
Refactoring.
Attachment #8734210 - Attachment is obsolete: true
Attachment #8734333 - Flags: review?(rail)
Comment on attachment 8734333 [details] [diff] [review]
Move post-release.py script from tools to intree mozharness scripts


       _~
    _~ )_)_~
    )_))_))_)
    _!__!__!_
    \______t/
  ~~~~~~~~~~~~~ it!
Attachment #8734333 - Flags: review?(rail) → review+
Comment on attachment 8734362 [details]
MozReview Request: Bug 1256063 - Add bouncer aliases for dev branch too. r=rail a=release DONTBUILD

https://reviewboard.mozilla.org/r/42183/#review38749
Attachment #8734362 - Flags: review?(rail) → review+
Attachment #8734333 - Attachment is obsolete: true
Keywords: leave-open
Note to self: to investigate how and if I can test my buildbotcustom and buildbot-configs changes
Flags: needinfo?(rail)
Comment on attachment 8735193 [details]
MozReview Request: Bug 1256063 - Add update bouncer aliases builder. r=rail

https://reviewboard.mozilla.org/r/42645/#review39145

::: process/release.py:1941
(Diff revision 1)
> +        "extra_args": [
> +             "-c",  branch_config['postrelease_bouncer_aliases_config'][product],
> +        ]
> +    }
> +    # TODO: should we use release-{branch}-{product}_postrelease to keep naming
> +    # convention with buildbot?

I think the current name is better than vague "postrelease". No need to cargo cult old stuff. :)

Feel free to remove the TODO.
Attachment #8735193 - Flags: review?(rail) → review+
Comment on attachment 8735194 [details]
MozReview Request: Bug 1256063 - Add update bouncer aliases builder. r=rail

https://reviewboard.mozilla.org/r/42649/#review39147
Attachment #8735194 - Flags: review?(rail) → review+
Comment on attachment 8735198 [details]
MozReview Request: Bug 1256063 - Enrich release graph with bouncer aliases context. r=rail

https://reviewboard.mozilla.org/r/42653/#review39149
Attachment #8735198 - Flags: review?(rail) → review+
https://reviewboard.mozilla.org/r/42649/#review39151

::: mozilla/config.py:2713
(Diff revision 1)
>  }
> +BRANCHES['mozilla-beta']['postrelease_bouncer_aliases_config'] = {
> +    "firefox": 'releases/bouncer_firefox_beta.py',
> +    # TODO - fennec
> +    "fennec": "",
> +}

Can you add this config variable to "date" too http://hg.mozilla.org/build/buildbot-configs/file/tip/mozilla/project_branches.py#l114
Comment on attachment 8735199 [details]
MozReview Request: Bug 1256063 - Enrich config with bouncer aliases flag. r=rail

https://reviewboard.mozilla.org/r/42655/#review39153

::: mozilla/config.py:2733
(Diff revision 1)
>  BRANCHES['mozilla-beta']['platforms']['win32']['dep_signing_servers'] = 'release-signing'
>  BRANCHES['mozilla-beta']['platforms']['win64']['dep_signing_servers'] = 'release-signing'
>  # used by releasetasks
>  BRANCHES['mozilla-beta']['bouncer_enabled'] = True
>  BRANCHES['mozilla-beta']['postrelease_version_bump_enabled'] = True
> +BRANCHES['mozilla-beta']['postrelease_bouncer_aliases_enabled'] = True

Can you add this variabel to "date" too http://hg.mozilla.org/build/buildbot-configs/file/tip/mozilla/project_branches.py#l114
Attachment #8735199 - Flags: review?(rail) → review+
(In reply to Mihai Tabara [:mtabara] from comment #16)
> Note to self: to investigate how and if I can test my buildbotcustom and
> buildbot-configs changes

After you address the comments above and enable the missing variables on "date", you can run CI tests and see it they pass.

A short summary of other steps:

* Land the patches
* reconfig
* Fix the releasetasks PR (rebase + test fixes)
* checkout your branch of releasetasks on bm83
* start a release process on date
Flags: needinfo?(rail)
BTW, bug 1259526 is going to split the graph and may exclude some postrelease steps from the first graph. JFYI.
See Also: → 1259526
Comment on attachment 8734362 [details]
MozReview Request: Bug 1256063 - Add bouncer aliases for dev branch too. r=rail a=release DONTBUILD

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/42183/diff/1-2/
Attachment #8734362 - Attachment description: MozReview Request: Bug 1256063 - Add update bouncer aliases builder. r=rail a=release DONTBUILD → MozReview Request: Bug 1256063 - Add bouncer aliases for dev branch too. r=rail a=release DONTBUILD
Comment on attachment 8735964 [details]
MozReview Request: Bug 1256063 - Add bouncer aliases params for date too. r=rail

https://reviewboard.mozilla.org/r/43015/#review39559
Attachment #8735964 - Flags: review?(rail) → review+
FWIW https://github.com/mozilla/releasetasks/pull/155 is the PR tracking the changes in releasetasks for this bug.
We have successfully started using this with 46.0b7 - https://wiki.mozilla.org/Releases/Firefox_46.0b7/BuildNotes.
I'm closing this bug, will reopen if any issues come by.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Removing leave-open keyword from resolved bugs, per :sylvestre.
Keywords: leave-open
You need to log in before you can comment on or make changes to this bug.