1236951 - Replace PartnerRepackFactory with a mozharness script

Status: RESOLVED FIXED

(Release Engineering :: Release Requests, defect)

Description

[Chris Cooper [:coop] (he/him)]

This amounts to "re-integrate partner repacks into the release process in a way that makes it easy to move them forward into TaskCluster."

I have a working mozharness script that encapsulates the changes, along with the requisite suite of buildbotcustom, configs, and puppet patches to make it work.

Patches coming up.

Comment 1

[Chris Cooper [:coop] (he/him)]

Attachment: [mozilla-inbound] New mozharness script for desktop partner repacks, and associated config files

I used the other scripts in the mozharness/scripts dir as templates and tried to keep the new script as simple as possible. The new script is a thin wrapper around the partner repo sync steps[1] and the subsequent call to partner-repacks.py[2] with some basic release config vars. This is the same sequence I've been running for months on the standalone partner repacks machine.

1. https://github.com/mozilla-partners/repack-manifests/blob/master/README.md
2. https://github.com/mozilla-partners/repack-scripts/blob/master/partner-repacks.py

The config files aren't really used at present, but it felt important to keep a config source in-tree just in case.

Here is output of the script running in my staging env against a clone of mozilla-beta against all the release platforms: 

http://dev-master2.bb.releng.use1.mozilla.com:8044/builders/release-mozilla-beta-linux_partner_repack/builds/0
http://dev-master2.bb.releng.use1.mozilla.com:8044/builders/release-mozilla-beta-linux64_partner_repack/builds/0
http://dev-master2.bb.releng.use1.mozilla.com:8044/builders/release-mozilla-beta-macosx64_partner_repack/builds/11
http://dev-master2.bb.releng.use1.mozilla.com:8044/builders/release-mozilla-beta-win32_partner_repack/builds/3
http://dev-master2.bb.releng.use1.mozilla.com:8044/builders/release-mozilla-beta-win64_partner_repack/builds/0

Once we're done with reviews, I'll need to uplift the script all the way up to mozilla-release, hopefully before the next release cycle.

Comment 2

[Chris Cooper [:coop] (he/him)]

Attachment: [puppet] s3cfg files for release and partners

This patch adds two new configs files used by the s3cmd tool for uploading partner repacks to the appropriate partner/mozilla buckets in S3. The contents of the files are already in the hiera cache. 

At present, we only do partner repacks on Mac, so I only install the config files there.

Since these are new secrets, they'll need to be added the cleanup instructions for slave loan:

https://wiki.mozilla.org/ReleaseEngineering/How_To/Loan_a_Slave#bld-lion-r5.2C_talos-mtnlion-r5.2C_t-yosemite-r5.2C_t-yosemite-r7

Comment 3

[Chris Cooper [:coop] (he/him)]

Attachment: [buildbot-configs] Re-enable partner repacks on beta and release

In addition to re-enabling the partner repacks for beta and release, this patch also:

* switches teh repacks to use the mozharness script
* removes the old partner repo from the list of tagged repos
* adds win64 to the list of platforms for staging releases, since it's in production now

Comment 4

[Chris Cooper [:coop] (he/him)]

Attachment: [buildbotcustom] Replace PartnerRepackFactory with a SigningScriptFactory instance

The bulk of this patch is removing the old PartnerRepackFactory class. Yay for code removal. There is some other stuff going on, though.

In SigningScriptFactory, I've switched the toolsdir SetProperty step to use the tools_repo_cache variable when set. It seems silly that we have a canonical tools checkout on each machine that we keep up-to-date but never use. It also avoids the problem where you can't use a SigningScriptFactory with a mozharness archive because it can't find signtool.py where it expects to. See my staging master run[1] for an example of this failure.

You can work around this by cloning tools within your mozharness script, but this seems cleaner to me. I did a quick survey of other SigningScriptFactory invocations, and none of them seem to set tools_repo_cache, so this shouldn't affect them anyway.

The actual factory creation is pretty straightforward. We pass our releaseConfig variables through to the new mozharness script.

I am planning to add a separate, standalone partner repack builds like the l10n one (bug 1236954).

1. http://dev-master2.bb.releng.use1.mozilla.com:8044/builders/release-mozilla-beta-macosx64_partner_repack/builds/6/steps/run_script/logs/stdio

Comment 5

[Jordan Lund (:jlund)]

Comment on attachment 8704794 [details] [diff] [review]
[mozilla-inbound] New mozharness script for desktop partner repacks, and associated config files

Review of attachment 8704794 [details] [diff] [review]:
-----------------------------------------------------------------

script looks great! I'm going to r- just because of the unused imports and variables I point out at the top that may have been copied from somewhere else. All the other comments are nits so feel free to argue back :)

::: testing/mozharness/scripts/desktop_partner_repacks.py
@@ +9,5 @@
> +This script manages Desktop partner repacks for beta/release builds.
> +"""
> +import multiprocessing
> +import os
> +import re

I think from the `re` import to `FATAL` there are many modules that are not actually used. Maybe just leftover from copy/paste?

@@ +47,5 @@
> +
> +# some other values such as "%(version)s", "%(buildid)s", ...
> +# are defined at run time and they cannot be enforced in the _pre_config_lock
> +# phase
> +runtime_config_tokens = ('version', 'buildnumber', 'platform', 'partner')

likewise, `SUCCESS` `FAILURE` `configuration_tokens` and `runtime_config_tokens` I don't think are needed for this script

@@ +50,5 @@
> +# phase
> +runtime_config_tokens = ('version', 'buildnumber', 'platform', 'partner')
> +
> +# DesktopPartnerRepacks {{{1
> +class DesktopPartnerRepacks(ReleaseMixin, BuildbotMixin,

are all these subclasses needed? Looks like you are running most of your actions via subprocess (self.run_command). Ones in question would be VCSMixin, SigningMixin, and TransferMixin.

@@ +101,5 @@
> +            'default_actions': DesktopPartnerRepacks.actions,
> +            'config': {
> +                "buildbot_json_path": "buildprops.json",
> +                "log_name": "partner-repacks",
> +                "appName": "Firefox",

would this ever change? Wondering if this is worthy of putting into your already existing in-tree config: release_mozilla-release_desktop.py

@@ +111,5 @@
> +                    's3cmd==1.6.0',
> +                ],
> +                'virtualenv_path': 'venv',
> +                'workdir': 'partner-repacks',
> +                'repack_manifests_url': '[e-mail]:mozilla-partners/repack-manifests.git',

likewise, would this value ever change?

@@ +134,5 @@
> +        self.s3cfg = self.config.get('s3cfg', None)
> +        self.hgroot = self.config.get('hgroot', None)
> +        self.repo = self.config.get('repo', None)
> +
> +        self.workdir = os.path.abspath(self.config['work_dir'])

fyi - for these paths you can use or subclass: https://dxr.mozilla.org/build-central/source/mozharness/mozharness/base/script.py#1729

e.g. the above assigned self.workdir could be replaced by: self.query_abs_dirs()['abs_work_dir']

@@ +143,5 @@
> +
> +    # Actions {{{
> +    def setup(self):
> +        """setup step"""
> +        self.download_file("https://raw.githubusercontent.com/mozilla/git-repo/master/repo",

since you are not sanity checking the return value here of download_file: which will either == file_name if complete, or `failure_status`( which I think in this case equates to None since you don't overwrite _retry_download's default), you may want to put error_level=FATAL so we halt here after exhausting retries. just so we don't cause spurious results in subsequent lines

@@ +147,5 @@
> +        self.download_file("https://raw.githubusercontent.com/mozilla/git-repo/master/repo",
> +                           file_name="repo", parent_dir=self.workdir)
> +        repo = os.path.join(self.workdir, "repo")
> +        self.chmod(repo, 0755)
> +        self.mkdir_p(self.workdir)

iiuc the self.chmod() call would require self.workdir to already exist. iirc, parent_dir in download_file will create it for you every time if it doesn't exist.

@@ +152,5 @@
> +        self.run_command([repo, "init", "--no-repo-verify",
> +                          "-u", self.config['repack_manifests_url']],
> +                         cwd=self.workdir,
> +                         halt_on_failure=True)
> +        self.run_command([repo, "sync"],

do either of these run_command calls require try/catch or retries?

@@ +163,5 @@
> +        repack_cmd = [python, "partner-repacks.py",
> +                      "-v", self.version,
> +                      "-n", self.buildnumber]
> +        if self.platform:
> +            repack_cmd.extend(["--platform", self.platform])

nit: aki preferred we reference config items only from self.config because self.config is immutable unlike all the self attrs you defined in the above __init__. I don't always stick to this idiom so I won't block you here but the mh norm would be more like:

if self.config.get('platform'):
    repack_cmd.extend(["--platform", self.config['platform'])

Comment 6

[Chris Cooper [:coop] (he/him)]

(In reply to Jordan Lund (:jlund) from comment #5)
> script looks great! I'm going to r- just because of the unused imports and
> variables I point out at the top that may have been copied from somewhere
> else. All the other comments are nits so feel free to argue back :)

Thanks for the thorough review. Can you tell this is my first mh script? ;)

> @@ +101,5 @@
> > +            'default_actions': DesktopPartnerRepacks.actions,
> > +            'config': {
> > +                "buildbot_json_path": "buildprops.json",
> > +                "log_name": "partner-repacks",
> > +                "appName": "Firefox",
> 
> would this ever change? Wondering if this is worthy of putting into your
> already existing in-tree config: release_mozilla-release_desktop.py

Unlikely now that we won't be worried about Thunderbird, and Fennec has it's own script.
 
> @@ +111,5 @@
> > +                    's3cmd==1.6.0',
> > +                ],
> > +                'virtualenv_path': 'venv',
> > +                'workdir': 'partner-repacks',
> > +                'repack_manifests_url': '[e-mail]:mozilla-partners/repack-manifests.git',
> 
> likewise, would this value ever change?

Might be useful to be able to specify a user repo here, but we'd want to do that in our staging configs. Maybe this should be default with a option to override?

> @@ +134,5 @@
> > +        self.s3cfg = self.config.get('s3cfg', None)
> > +        self.hgroot = self.config.get('hgroot', None)
> > +        self.repo = self.config.get('repo', None)
> > +
> > +        self.workdir = os.path.abspath(self.config['work_dir'])
> 
> fyi - for these paths you can use or subclass:
> https://dxr.mozilla.org/build-central/source/mozharness/mozharness/base/
> script.py#1729
> 
> e.g. the above assigned self.workdir could be replaced by:
> self.query_abs_dirs()['abs_work_dir']

OK, good to know. I'll make the change.

> @@ +143,5 @@
> > +
> > +    # Actions {{{
> > +    def setup(self):
> > +        """setup step"""
> > +        self.download_file("https://raw.githubusercontent.com/mozilla/git-repo/master/repo",
> 
> since you are not sanity checking the return value here of download_file:
> which will either == file_name if complete, or `failure_status`( which I
> think in this case equates to None since you don't overwrite
> _retry_download's default), you may want to put error_level=FATAL so we halt
> here after exhausting retries. just so we don't cause spurious results in
> subsequent lines

You're right, I should be checking the return value here.

> @@ +147,5 @@
> > +        self.download_file("https://raw.githubusercontent.com/mozilla/git-repo/master/repo",
> > +                           file_name="repo", parent_dir=self.workdir)
> > +        repo = os.path.join(self.workdir, "repo")
> > +        self.chmod(repo, 0755)
> > +        self.mkdir_p(self.workdir)
> 
> iiuc the self.chmod() call would require self.workdir to already exist.
> iirc, parent_dir in download_file will create it for you every time if it
> doesn't exist.

Are you suggesting I create the workdir myself before the download just to be safe?

> @@ +152,5 @@
> > +        self.run_command([repo, "init", "--no-repo-verify",
> > +                          "-u", self.config['repack_manifests_url']],
> > +                         cwd=self.workdir,
> > +                         halt_on_failure=True)
> > +        self.run_command([repo, "sync"],
> 
> do either of these run_command calls require try/catch or retries?

It wouldn't hurt, certainly. The only failure case I've run into here is when running the sync with parallelism (-j). Sometimes we end up in a logjam and need to remove the repos and start over. I'll add some extra checking and recovery here.

This does raise the question of what is a better mozharness script to use as a template for learning, because the ones I was copying from (desktop_l10n.py in particular) seem to play fast and loose here.

> nit: aki preferred we reference config items only from self.config because
> self.config is immutable unlike all the self attrs you defined in the above
> __init__. I don't always stick to this idiom so I won't block you here but
> the mh norm would be more like:
> 
> if self.config.get('platform'):
>     repack_cmd.extend(["--platform", self.config['platform'])

Well, that's why I pinged you for review - I want to learn good mh habits before I start reinforcing bad ones.

Comment 7

[Chris Cooper [:coop] (he/him)]

Attachment: [mozilla-inbound] New mozharness script for desktop partner repacks, and associated config files, v2

Incorporates Jordan's feedback from comment #5, namely:

* removes unused variables and imports
* moved some variables that might possibly change (i.e. in staging) into the config files
* subclassed query_abs_dir() for this script
* wrapped repo actions in retry, with cleanup
* access config vars via self.config

Tested it against bing repacks (mac, win) on bld-lion-r5-002 and it seems to work fine.

Comment 8

[Jordan Lund (:jlund)]

(In reply to Chris Cooper [:coop] from comment #6)

> 
> > @@ +147,5 @@
> > > +        self.download_file("https://raw.githubusercontent.com/mozilla/git-repo/master/repo",
> > > +                           file_name="repo", parent_dir=self.workdir)
> > > +        repo = os.path.join(self.workdir, "repo")
> > > +        self.chmod(repo, 0755)
> > > +        self.mkdir_p(self.workdir)
> > 
> > iiuc the self.chmod() call would require self.workdir to already exist.
> > iirc, parent_dir in download_file will create it for you every time if it
> > doesn't exist.
> 
> Are you suggesting I create the workdir myself before the download just to
> be safe?
> 

sorry for not being clear - I meant that 1) self.workdir would have to exist here because download_file call would have created it. 2) you call self.mkdir_p(path) _after_ self.chmod(path)

> 
> This does raise the question of what is a better mozharness script to use as
> a template for learning, because the ones I was copying from
> (desktop_l10n.py in particular) seem to play fast and loose here

I care more about mozharness/{base,mozilla} than I do about the end scripts. Saying that, for some example scripts, you can poke at https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/examples

> 
> > nit: aki preferred we reference config items only from self.config because
> > self.config is immutable unlike all the self attrs you defined in the above
> > __init__. I don't always stick to this idiom so I won't block you here but
> > the mh norm would be more like:
> > 
> > if self.config.get('platform'):
> >     repack_cmd.extend(["--platform", self.config['platform'])
> 
> Well, that's why I pinged you for review - I want to learn good mh habits
> before I start reinforcing bad ones.

:)

Comment 9

[Jordan Lund (:jlund)]

Comment on attachment 8705734 [details] [diff] [review]
[mozilla-inbound] New mozharness script for desktop partner repacks, and associated config files, v2

Review of attachment 8705734 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/mozharness/scripts/desktop_partner_repacks.py
@@ +110,5 @@
> +        self.platform = self.config.get('platform', None)
> +        self.partner = self.config.get('partner', None)
> +        self.s3cfg = self.config.get('s3cfg', None)
> +        self.hgroot = self.config.get('hgroot', None)
> +        self.hgrepo = self.config.get('hgrepo', None)

iiuc, most (all?) these attr assignments be deleted now that you reference self.config directly?

@@ +112,5 @@
> +        self.s3cfg = self.config.get('s3cfg', None)
> +        self.hgroot = self.config.get('hgroot', None)
> +        self.hgrepo = self.config.get('hgrepo', None)
> +
> +    def query_abs_dirs(self):

++

@@ +140,5 @@
> +        status = self.run_command([repo, "init", "--no-repo-verify",
> +                                   "-u", self.config['repack_manifests_url']],
> +                                  cwd=self.query_abs_dirs()['abs_work_dir'],
> +                                  halt_on_failure=True)
> +        if status != 0:

this could be simply `if status`

but I think I have steered you wrong. looking at this again, you have halt_on_failure=True for your run_command calls. run_command automatically evaluates the return value for you and if it it's not in success_codes (defaults to [0]), will throw an error and in your case, halt: https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/mozharness/base/script.py#1158

which means, 1) you don't have to check `if status` at all. 2) halt_on_failure won't even let self.retry retry.

so you could remove the retry and status condition checks on these if you are happy without retrying.

Or, if you do want retry, you will need to remove the halt_on_failure in run_command, and instead add, good_statuses=[0], error_level=FATAL to the self.retry()[1] call. Unfortunately, self.retry and self.run_command have different param names that do similar things. good_statuses == success_codes except it's defaulted to None. and self.retry doesn't have halt_on_failure but I believe error_level=FATAL will achieve the same thing after retries are exhausted. Does this make sense?

[1] https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/mozharness/base/script.py#800

@@ +154,5 @@
> +                                  parent_dir=self.query_abs_dirs()['abs_work_dir'],
> +                                  error_level=FATAL)
> +        if not os.path.exists(repo):
> +            self.fatal("Unable to download repo tool.")
> +        #repo = os.path.join(self.query_abs_dirs()['abs_work_dir'], "repo")

can this line be deleted?

Comment 10

[Chris Cooper [:coop] (he/him)]

(In reply to Jordan Lund (:jlund) from comment #9)
> Comment on attachment 8705734 [details] [diff] [review]
> [mozilla-inbound] New mozharness script for desktop partner repacks, and
> associated config files, v2
> 
> Review of attachment 8705734 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: testing/mozharness/scripts/desktop_partner_repacks.py
> @@ +110,5 @@
> > +        self.platform = self.config.get('platform', None)
> > +        self.partner = self.config.get('partner', None)
> > +        self.s3cfg = self.config.get('s3cfg', None)
> > +        self.hgroot = self.config.get('hgroot', None)
> > +        self.hgrepo = self.config.get('hgrepo', None)
> 
> iiuc, most (all?) these attr assignments be deleted now that you reference
> self.config directly?

You're right. I'll fix it up.
 
> which means, 1) you don't have to check `if status` at all. 2)
> halt_on_failure won't even let self.retry retry.
> 
> so you could remove the retry and status condition checks on these if you
> are happy without retrying.

For simplicity's sake, I think I'll just revert to a the halt_on_failure without retry.

> > +        #repo = os.path.join(self.query_abs_dirs()['abs_work_dir'], "repo")
> 
> can this line be deleted?

Yep, will remove.

Comment 11

[Chris Cooper [:coop] (he/him)]

Attachment: [mozilla-inbound] New mozharness script for desktop partner repacks, and associated config files, v3

(In reply to Chris Cooper [:coop] from comment #10)
> For simplicity's sake, I think I'll just revert to a the halt_on_failure
> without retry.

Second-guessing myself here. I think it's more valuable to have a retry built-in in case there's a transitory failure. I'd rather spend a few minutes of machine time retrying than need to run the repacks by hand.

The other suggestions from comment #9 have also been incorporated.

Comment 12

[Chris Cooper [:coop] (he/him)]

Comment on attachment 8704796 [details] [diff] [review]
[puppet] s3cfg files for release and partners

Review of attachment 8704796 [details] [diff] [review]:
-----------------------------------------------------------------

https://hg.mozilla.org/build/puppet/rev/ee145a659339

Comment 13

[Chris Cooper [:coop] (he/him)]

(In reply to Chris Cooper [:coop] from comment #2)
> Since these are new secrets, they'll need to be added the cleanup
> instructions for slave loan:
> 
> https://wiki.mozilla.org/ReleaseEngineering/How_To/Loan_a_Slave#bld-lion-r5.
> 2C_talos-mtnlion-r5.2C_t-yosemite-r5.2C_t-yosemite-r7

Instructions have been updated.

Comment 14

[Jordan Lund (:jlund)]

Comment on attachment 8706949 [details] [diff] [review]
[mozilla-inbound] New mozharness script for desktop partner repacks, and associated config files, v3

Review of attachment 8706949 [details] [diff] [review]:
-----------------------------------------------------------------

looks like we are pretty much ready to go :D

::: testing/mozharness/scripts/desktop_partner_repacks.py
@@ +98,5 @@
> +        )
> +
> +        if 'version' not in self.config:
> +            self.fatal("Version (-v) not supplied.")
> +        self.version = self.config.get('version')

nit: I think this can be removed too

@@ +152,5 @@
> +                      args=(repo,),
> +                      error_level=FATAL,
> +                      cleanup=self._repo_cleanup(),
> +                      good_statuses=[0],
> +                      sleeptime=5) != 0:

because error_level is set to FATAL and you supplied good_statuses, you don't need to put this as a condition or the self.fatal line below because iiuc self.retry() will fatal internally.

Comment 15

[Shane Caraveo (:mixedpuppy)]

I haven't looked over this yet, but wanted to point out bug 1239032 I just posted.

Comment 16

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/ec4ff5db9559

Comment 17

[Chris Cooper [:coop] (he/him)]

Comment on attachment 8706949 [details] [diff] [review]
[mozilla-inbound] New mozharness script for desktop partner repacks, and associated config files, v3

Review of attachment 8706949 [details] [diff] [review]:
-----------------------------------------------------------------

https://hg.mozilla.org/integration/mozilla-inbound/rev/ec4ff5db9559

...and now we start the uplift dance.

Comment 18

[Chris Cooper [:coop] (he/him)]

(In reply to Chris Cooper [:coop] from comment #17)
> Comment on attachment 8706949 [details] [diff] [review]
> [mozilla-inbound] New mozharness script for desktop partner repacks, and
> associated config files, v3
> 
> Review of attachment 8706949 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> https://hg.mozilla.org/integration/mozilla-inbound/rev/ec4ff5db9559
> 
> ...and now we start the uplift dance.

Hey Ritu,

Given the uplift lockdown we're in, I wanted to ping you directly about this.

This is a mozharness script that releng wants to use in the release process for beta and release, but to get it there for 44 will require uplifts. It's a self-contained script with its own config files that will only be used in the release process, so it's pretty much the definition of NPOTB.

Do you have any issues with me uplifting this all the way to mozilla-beta?

Comment 19

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

(In reply to Chris Cooper [:coop] from comment #18)
> (In reply to Chris Cooper [:coop] from comment #17)
> > Comment on attachment 8706949 [details] [diff] [review]
> > [mozilla-inbound] New mozharness script for desktop partner repacks, and
> > associated config files, v3
> > 
> > Review of attachment 8706949 [details] [diff] [review]:
> > -----------------------------------------------------------------
> > 
> > https://hg.mozilla.org/integration/mozilla-inbound/rev/ec4ff5db9559
> > 
> > ...and now we start the uplift dance.
> 
> Hey Ritu,
> 
> Given the uplift lockdown we're in, I wanted to ping you directly about this.
> 
> This is a mozharness script that releng wants to use in the release process
> for beta and release, but to get it there for 44 will require uplifts. It's
> a self-contained script with its own config files that will only be used in
> the release process, so it's pretty much the definition of NPOTB.
> 
> Do you have any issues with me uplifting this all the way to mozilla-beta?

Hi Chris, thanks for checking with me. If this is NPOTB, we are good. Also hoping this will not affect our ability to get Beta9 build ready and pushed out to Beta channel.

Comment 20

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/ec4ff5db9559

Comment 21

[Chris Cooper [:coop] (he/him)]

Uplifted to aurora and beta:

https://hg.mozilla.org/releases/mozilla-aurora/rev/47c244a99493
https://hg.mozilla.org/releases/mozilla-beta/rev/6512ec26d8c1

Re-opening because I still need to land the buildbot patches.

Comment 22

[Chris Cooper [:coop] (he/him)]

Comment on attachment 8704807 [details] [diff] [review]
[buildbotcustom] Replace PartnerRepackFactory with a SigningScriptFactory instance

Review of attachment 8704807 [details] [diff] [review]:
-----------------------------------------------------------------

https://hg.mozilla.org/build/buildbotcustom/rev/089b94bab05c

Comment 23

[Chris Cooper [:coop] (he/him)]

Comment on attachment 8704803 [details] [diff] [review]
[buildbot-configs] Re-enable partner repacks on beta and release

Review of attachment 8704803 [details] [diff] [review]:
-----------------------------------------------------------------

https://hg.mozilla.org/build/buildbot-configs/rev/5bdbf1ac3f5f

Comment 24

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/5bdbf1ac3f5f

Comment 25

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbotcustom/rev/089b94bab05c

Comment 26

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/releases/mozilla-b2g44_v2_5/rev/6512ec26d8c1

Comment 27

[Chris Cooper [:coop] (he/him)]

Attachment: [buildbotcustom] Set script_repo_revision for ScriptFactory when archiver is used

I made a mistake in my testing. I thought I needed to pass 'script_repo_revision' manually because that was something that the release process would normally do, and I was simply doing one-offs. However, it turns out that when you use a ScriptFactory with an archiver mozharness archive, the script_repo_revision property is never set, so the build fails later when it tries to echo it out for display. I guess I'm the first person to try this permutation of ScriptFactory options. :/

This patch attempts to set script_repo_revision for archiver-based scripts using data that the factory already has, namely (in preferential order):
* relengapi_archiver_release_tag
* relengapi_archiver_rev
* revision

We're unlikely to have a revision in the release context, but we will always have a release tag.

I cancelled the job early once I was convinced it was working, but here's an example of script_repo_revision being set to FIREFOX_44_0b4_RELEASE automatically in a staging run:

http://dev-master2.bb.releng.use1.mozilla.com:8044/builders/release-mozilla-beta-macosx64_partner_repack/builds/12

Comment 28

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbotcustom/rev/aa885f5cdbc3

Comment 29

[Chris Cooper [:coop] (he/him)]

Attachment: [buildbot-configs] Update partner repack stanza in release templates

Release configs got overwritten by automation because I didn't update the templates.

Comment 30

[Chris Cooper [:coop] (he/him)]

Comment on attachment 8707860 [details] [diff] [review]
[buildbotcustom] Set script_repo_revision for ScriptFactory when archiver is used

Review of attachment 8707860 [details] [diff] [review]:
-----------------------------------------------------------------

https://hg.mozilla.org/build/buildbotcustom/rev/aa885f5cdbc3

Comment 31

[Chris Cooper [:coop] (he/him)]

Comment on attachment 8709047 [details] [diff] [review]
[buildbot-configs] Update partner repack stanza in release templates

Review of attachment 8709047 [details] [diff] [review]:
-----------------------------------------------------------------

https://hg.mozilla.org/build/buildbot-configs/rev/2f6ad8dd5b7e

Comment 32

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/2f6ad8dd5b7e

Comment 33

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbotcustom/rev/6368247d8c7a

Comment 34

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbotcustom/rev/b049ab7ef9a7

Comment 35

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbotcustom/rev/c06e740a09e4

Comment 36

[Chris Cooper [:coop] (he/him)]

Partner repacks for Firefox 44.0 build 3 were automatically generated.