replace TuxedoEntrySubmitterFactory with a mozharness script

RESOLVED FIXED

Status

P3
normal
RESOLVED FIXED
6 years ago
4 years ago

People

(Reporter: bhearsum, Assigned: rail)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [mozharness])

Attachments

(5 attachments, 9 obsolete attachments)

18.88 KB, patch
rail
: review+
Details | Diff | Splinter Review
7.27 KB, patch
rail
: review+
Details | Diff | Splinter Review
26.25 KB, patch
rail
: review+
Details | Diff | Splinter Review
1.03 KB, patch
nthomas
: review+
Details | Diff | Splinter Review
1.23 KB, patch
nthomas
: review+
Details | Diff | Splinter Review
(Reporter)

Description

6 years ago
Needed for reconfigless releases.
Whiteboard: [mozharness]
(Reporter)

Updated

6 years ago
Priority: -- → P3
Product: mozilla.org → Release Engineering
(Assignee)

Updated

5 years ago
Assignee: nobody → rail
Blocks: 916181
(Assignee)

Comment 1

5 years ago
I'm going to look at this this week since I'm on release duty
(Assignee)

Updated

5 years ago
Depends on: 976101
(Assignee)

Comment 2

5 years ago
Created attachment 8380703 [details] [diff] [review]
tuxedo-mozharness.diff

* Once the blocker is fixed I can test it in staging. The generated URLs look OK to me. 
* SSL-only product API call added.
* WCPGW? :)
Attachment #8380703 - Flags: review?(nthomas)
(Assignee)

Comment 3

5 years ago
Created attachment 8380705 [details] [diff] [review]
tuxedo-buildbotcustom.diff
Attachment #8380705 - Flags: review?(nthomas)
(Assignee)

Comment 4

5 years ago
Created attachment 8380706 [details] [diff] [review]
tuxedo-buildbot-configs.diff
Attachment #8380706 - Flags: review?(nthomas)
(Assignee)

Comment 5

5 years ago
Created attachment 8380708 [details] [diff] [review]
tuxedo-mozharness-1.diff

* removed a TODO comment which is not a TODO anymore :)
Attachment #8380703 - Attachment is obsolete: true
Attachment #8380703 - Flags: review?(nthomas)
Attachment #8380708 - Flags: review?(nthomas)
(Assignee)

Comment 6

5 years ago
Created attachment 8380936 [details] [diff] [review]
tuxedo-buildbot-configs-1.diff

* no solaris for betas
* no euballot for betas
Attachment #8380706 - Attachment is obsolete: true
Attachment #8380706 - Flags: review?(nthomas)
Attachment #8380936 - Flags: review?(nthomas)
(Assignee)

Comment 7

5 years ago
Created attachment 8380937 [details] [diff] [review]
tuxedo-buildbotcustom-1.diff

Use bouncer_add_euballot to avoid submitting euballot for betas
Attachment #8380705 - Attachment is obsolete: true
Attachment #8380705 - Flags: review?(nthomas)
Attachment #8380937 - Flags: review?(nthomas)
(Assignee)

Comment 8

5 years ago
Created attachment 8380948 [details] [diff] [review]
tuxedo-buildbotcustom-2.diff

oopsie, syntax fix
Attachment #8380937 - Attachment is obsolete: true
Attachment #8380937 - Flags: review?(nthomas)
Attachment #8380948 - Flags: review?(nthomas)
(Assignee)

Comment 9

5 years ago
Created attachment 8380951 [details] [diff] [review]
tuxedo-buildbot-configs-2.diff

Updated *.py files as well to make test-masters.sh happier.
Attachment #8380936 - Attachment is obsolete: true
Attachment #8380936 - Flags: review?(nthomas)
Attachment #8380951 - Flags: review?(nthomas)
(Assignee)

Comment 10

5 years ago
Created attachment 8380959 [details] [diff] [review]
tuxedo-buildbotcustom-3.diff

forgot to update the second entry of bouncer_add_euballot
Attachment #8380948 - Attachment is obsolete: true
Attachment #8380948 - Flags: review?(nthomas)
Attachment #8380959 - Flags: review?(nthomas)
Comment on attachment 8380708 [details] [diff] [review]
tuxedo-mozharness-1.diff

Review of attachment 8380708 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good, r+ with some tweaks. I didn't check the configs in Super Pedantic Man mode, so I suggest we compare locations against a prior release (either in staging before this lands, or in prod after) to make sure no little details changed in the transfer.

::: scripts/bouncer_submitter.py
@@ +47,5 @@
> +            "help": "Specify buildbot platforms",
> +        }],
> +        [["--bouncer-api-prefix"], {
> +            "dest": "bouncer-api-prefix",
> +            "help": "Specify Bouncer API prefix",

Some of these help messages just repeat the argument rather than adding any extra information. Could you make them more human friendly ? eg 'URL of bouncer admin host"

@@ +72,5 @@
> +                                'download-shipped-locales',
> +                                'add-product',
> +                                'add-ssl-only-product',
> +                                'add-complete-upadtes',
> +                                'add-partial-upadtes',

s/upadtes/updates/g.

@@ +113,5 @@
> +        if not self.config.get("shipped-locales-url"):
> +            self.info("Not downloading shipped-locales")
> +            return
> +        if self.config["no-locales"]:
> +            self.info("No need downloading shipped-locales")

nit, s/downloading/to download/

@@ +198,5 @@
> +        }
> +        for p in platforms:
> +            c = self.config["platform-config"][p]
> +            bouncer_platform = c["bouncer-platform"]
> +            path = c[path_type] % replace_dict

Might get KeyError's here with incomplete configs.

@@ +209,5 @@
> +        self.add_locations(product_name, "installer")
> +
> +    def add_ssl_only_product(self):
> +        if not self.config.get("add-ssl-only-product"):
> +            self.info("SSL-only product disaled. Skipping...")

nit, typo in disabled
Attachment #8380708 - Flags: review?(nthomas) → review+
Comment on attachment 8380951 [details] [diff] [review]
tuxedo-buildbot-configs-2.diff

Review of attachment 8380951 [details] [diff] [review]:
-----------------------------------------------------------------

r+ with tweaks on landing.

::: mozilla/release-thunderbird-comm-beta.py
@@ +136,5 @@
>  releaseConfig['tuxedoServerUrl']     = 'https://bounceradmin.mozilla.com/api/'
>  releaseConfig['extraBouncerPlatforms'] = ('solaris-sparc', 'solaris-i386',
>                                            'opensolaris-sparc',
>                                            'opensolaris-i386')
> +releaseConfig['bouncer_submitter_config'] = 'releases/bouncer_firefox.py'

Wrong config file - s/firefox/thunderbird/. There are several Thunderbird configs that need this fix.

::: mozilla/release-thunderbird-comm-beta.py.template
@@ -135,5 @@
> -releaseConfig['tuxedoConfig']        = 'firefox-tuxedo.ini'
> -releaseConfig['tuxedoServerUrl']     = 'https://bounceradmin.mozilla.com/api/'
> -releaseConfig['extraBouncerPlatforms'] = ('solaris-sparc', 'solaris-i386',
> -                                          'opensolaris-sparc',
> -                                          'opensolaris-i386')

Apparently the Solaris elves out there are still doing Thunderbird betas, just not Firefox ones, so should add extraBouncerPlatforms back for them.

::: mozilla/staging_release-firefox-mozilla-beta.py.template
@@ +141,3 @@
>  releaseConfig['extraBouncerPlatforms'] = ('solaris-sparc', 'solaris-i386',
>                                            'opensolaris-sparc',
>                                            'opensolaris-i386')

Please remove extraBouncerPlatforms so the staging config is closer to prod.

::: mozilla/staging_release-firefox-mozilla-release.py.template
@@ +142,4 @@
>  releaseConfig['extraBouncerPlatforms'] = ('solaris-sparc', 'solaris-i386',
>                                            'opensolaris-sparc',
>                                            'opensolaris-i386')
> +releaseConfig['bouncer_submitter_config'] = 'releases/bouncer_firefox.py'

Missing the new EU ballot pref here and in non-template file.
Attachment #8380951 - Flags: review?(nthomas) → review+
Comment on attachment 8380959 [details] [diff] [review]
tuxedo-buildbotcustom-3.diff

Review of attachment 8380959 [details] [diff] [review]:
-----------------------------------------------------------------

r+ with tweak.

::: process/release.py
@@ +1610,5 @@
> +                          releaseConfig['tuxedoServerUrl'],
> +                          "--platform", "win32-EUBallot",
> +                          "--no-add-ssl-only-product",
> +                          "--no-add-complete-upadtes",
> +                          "--no-add-partial-upadtes",

s/upadtes/updates/g to match mozharness fix.
Attachment #8380959 - Flags: review?(nthomas) → review+
(Assignee)

Comment 14

5 years ago
Created attachment 8381113 [details] [diff] [review]
tuxedo-mozharness-2.diff

* addressed comments
* the KeyError issue is made more obvious by checking and calling self.fatal(). I didn't want to make the initial verification code (_pre_config_lock) too complex.
* carry on r=nthomas

I don't feel like landing this tonight. If I won't be able to test this in staging I'll land this before the next beta.
Attachment #8380708 - Attachment is obsolete: true
Attachment #8381113 - Flags: review+
(Assignee)

Comment 15

5 years ago
Created attachment 8381115 [details] [diff] [review]
tuxedo-buildbotcustom-4.diff
Attachment #8380959 - Attachment is obsolete: true
Attachment #8381115 - Flags: review+
(Assignee)

Comment 16

5 years ago
Created attachment 8381116 [details] [diff] [review]
tuxedo-buildbot-configs-3.diff
Attachment #8380951 - Attachment is obsolete: true
Attachment #8381116 - Flags: review+
(Assignee)

Comment 17

5 years ago
Comment on attachment 8381113 [details] [diff] [review]
tuxedo-mozharness-2.diff

https://hg.mozilla.org/build/mozharness/rev/ab3e42e05a4f

with one change (s/browser/mail/) in the Thunderbird config
Attachment #8381113 - Flags: checked-in+
(Assignee)

Comment 20

5 years ago
I tested the patches in staging with disabled self.fatal(). Everything looked fine to me (except 401s). I'm going to watch this on Thursday.
(Assignee)

Comment 21

5 years ago
:Callek, you may want to create a SeaMonkey config, so we can retire tuxedo-add.py soon.
In production.
(Assignee)

Comment 23

5 years ago
Created attachment 8383433 [details] [diff] [review]
[custom] fix --revision
Attachment #8383433 - Flags: review?(nthomas)
Attachment #8383433 - Flags: review?(nthomas) → review+
(Assignee)

Comment 25

5 years ago
Created attachment 8383447 [details] [diff] [review]
[mozharness] fail early

A small tweak to make the script exit if it fails to download shipped-locales and avoid running the next steps.
Attachment #8383447 - Flags: review?(nthomas)
Attachment #8383447 - Flags: review?(nthomas) → review+
(Assignee)

Updated

5 years ago
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
buildbotcustom patch is merged and live in production! :)
this might have been already in production as a hot fix. please ignore if that is the case
Component: Release Automation → Mozharness
You need to log in before you can comment on or make changes to this bug.