port b2g branching script to mozharness, with revision locking

RESOLVED FIXED

Status

Release Engineering
Mozharness
RESOLVED FIXED
4 years ago
3 years ago

People

(Reporter: aki, Assigned: aki)

Tracking

(Blocks: 1 bug)

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [mozharness])

Attachments

(1 attachment)

(Assignee)

Description

4 years ago
Bug 917433 is an example of how we're going to have to treat merge day very carefully for gecko.git.

However, for gecko l10n, we may have to switch the branch names to also be versioned (rather than train-names).

For example, https://git.mozilla.org/?p=releases/l10n/es-ES/gecko.git;a=summary currently has a mozilla-beta branch.  This is ok for now, since there is only one branch for partners to use for their localizations.  When we add the other trains, they either need to know which train corresponds to which release, or we need to name the branches differently, e.g. v1.2, and we hit the exact same timing issues that we need to resolve in bugs like bug 917433... before merge day, l10n needs to come from https://hg.mozilla.org/releases/l10n/mozilla-aurora ; after merge day it needs to come from https://hg.mozilla.org/releases/l10n/mozilla-beta , and waiting til *after* the merge means we have an ugly non-fastforward issue in the git mirrors.

Once we switch over to mozharness for hg-git, which I'm hoping is before the next 6 week cycle, we will need to deal with this freezing-and-bumping every 6 weeks for merge day^Wweek.  I kind of want it to be a script, hence this bug.  Then again, it might be a simple enough edit that it can be a line or two in docs.
(Assignee)

Updated

3 years ago
Blocks: 1003346
Summary: b2g merge day script → port b2g branching script to mozharness
(Assignee)

Updated

3 years ago
Blocks: 988613
(Assignee)

Updated

3 years ago
Blocks: 910745
(Assignee)

Updated

3 years ago
Summary: port b2g branching script to mozharness → port b2g branching script to mozharness, with revision locking
(Assignee)

Comment 1

3 years ago
Per our discussion with the sheriffs and the b2g team, we want to lock down all 3rd-party repos to a specific revision when we create the git branches.

The <include> statements in various manifests.  I don't see any conflicts here yet (the <default> entries for including manifests are the same), but there's the possibility for divergence.  If we're going to be locking these down, I prefer to flatten the manifests like we do with b2g_bumper.  (Except keep gecko in there, and don't change the <remote>s.)

The question here is scope.  We can:

* only lock down the manifests we build, or
* lock down all the non-included manifests (flattened).

With the manifests we don't lock down, we can

* leave them, or
* nuke them.

I'm currently leaning towards either

a) only lock down the manifests we build, and leave the un-locked-down manifests as tier 3, or
b) lock down all the manifests, and nuke the included base-____.xml files once all the others are flattened.

Fabrice, do you have an opinion here?
Flags: needinfo?(fabrice.desre)
(Assignee)

Updated

3 years ago
Depends on: 971741
(Assignee)

Comment 2

3 years ago
Going with locking down all the non-included manifests, and deleting the others.
Flags: needinfo?(fabrice.desre)
(Assignee)

Updated

3 years ago
Assignee: nobody → aki
(Assignee)

Updated

3 years ago
Duplicate of this bug: 971741
(Assignee)

Updated

3 years ago
No longer depends on: 799719
(Assignee)

Comment 4

3 years ago
Created attachment 8441067 [details] [diff] [review]
b2g_branch_repos.diff

Ready for review.

I'm running this on my laptop sans --push and the diffs look ok.  I'm both able to branch the repos and lock the 3rd party revisions, and I'm able to lock 3rd party revisions without branching for existing branches (v2.0, v1.4; I'm working on v1.3t and v1.3 now).

I have logs and b2g-manifest diffs if you'd like.
Attachment #8441067 - Flags: review?(rail)
Comment on attachment 8441067 [details] [diff] [review]
b2g_branch_repos.diff

Review of attachment 8441067 [details] [diff] [review]:
-----------------------------------------------------------------

Great job! I can't believe we can obsolete the initial script now!

::: mozharness/base/script.py
@@ +734,5 @@
>                                  silent=False, log_level=INFO,
>                                  tmpfile_base_path='tmpfile',
>                                  return_type='output', save_tmpfiles=False,
> +                                throw_exception=False, fatal_exit_code=2,
> +                                success_codes=None):

2 hunks failed to apply, but it shouldn't be hard to unbitrot these.

::: scripts/merge_day/b2g_branch_repos.py
@@ +386,5 @@
> +        git = self.query_exe("git", return_type="list")
> +        new_branch = self.config['branch_name']
> +        # b2g-manifest is special
> +        if os.path.exists(dirs['abs_manifest_dir']):
> +            for cmd in (git + ["reset", "--hard"], git + ["checkout", self.config["manifest_repo_revision"]]):

maybe also add "git clean -fdx"?

@@ +400,5 @@
> +            if branch_status[0] and not branch_status[1]:
> +                # We have a local branch that doesn't exist on origin
> +                self.info("Cleaning up existing %s branch from %s." % (new_branch, name))
> +                self.run_command(
> +                    git + ['branch', '-d', new_branch],

Should it be capital D (-D) here? -d deletes merged branches only?

@@ +416,5 @@
> +        new_branch = self.config['branch_name']
> +        for name, r in repos.iteritems():
> +            revision = r["revision"]
> +            cwd = os.path.join(dirs['abs_work_dir'], name)
> +            branch_status = self.check_existing_branch(new_branch, cwd)

branch_status is a bit cryptic here. maybe unpack on the fly?

local_status, remote_status = self.check_existing_branch(new_branch, cwd)
Attachment #8441067 - Flags: review?(rail) → review+
(Assignee)

Comment 6

3 years ago
Comment on attachment 8441067 [details] [diff] [review]
b2g_branch_repos.diff

http://hg.mozilla.org/build/mozharness/rev/3230c9fbf560
With review comments: http://hg.mozilla.org/build/mozharness/rev/90e793f7c2aa
Attachment #8441067 - Flags: checked-in+
(Assignee)

Comment 7

3 years ago
Todo:

_ update docs
_ verify (possibly when branching 1.4d)
_ remove http://hg.mozilla.org/build/braindump/file/ac27869b637b/git-related/b2g-branching/b2g_branch_repos.py
_ resolve bug
(Assignee)

Comment 8

3 years ago
Cypress hamachi build running to verify I didn't bork anything obvious:
http://buildbot-master85.srv.releng.scl3.mozilla.com:8001/builders/b2g_cypress_hamachi_periodic/builds/0

Comment 9

3 years ago
3 mozharness patches are live in production :)
Blocks: 1028111
(Assignee)

Comment 10

3 years ago
Docs: https://wiki.mozilla.org/ReleaseEngineering/Merge_Duty/Steps#Create_new_B2G_git_branches
(Assignee)

Comment 11

3 years ago
I filed bug 1029860 for cleanup.  We're not going to create a dolphin branch.  Closing, for now.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED

Updated

3 years ago
Component: Tools → Mozharness
You need to log in before you can comment on or make changes to this bug.