1361153 - Add beta uplift simulation repo to treeherder

Status: RESOLVED WONTFIX

(Release Engineering :: General, enhancement)

Description

[Chris Cooper [:coop] (he/him)]

This bug comes out of a conversation I had with Ryan last week.

One manual part of the current sheriff workload is running uplift simulations in advance of the next merge to beta. Since the uplift simulation patch doesn't change much (if at all) over the course of a given release cycle, this seems like something we could automate.

My proposal is to setup a new repo on treeherder where people who care about uplift simulation (sheriffs and release managers) can view the results of build and test results. We could schedule the builds every second day during the early part of a release cycle, and then more frequently near the end. Someone from releng can help us get the scheduling setup here.

Ideally these builds would have the same signing context as beta. This is one of the things that trips us up now with manual uplift simulation runs on try is the need to perform two separate runs in order to properly test talos.

Not sure about the logistics on the hg side...should this be a separate bookmark in the unified repo?

Comment 1

[Ryan VanderMeulen [:RyanVM]]

I was thinking we would just run the release automation on top of whatever the tip revision of Beta is at the time and push a new head to this branch. Thanks so much for filing this!

Comment 2

[Phil Ringnalda (:philor)]

Yeah. The Treeherder step is just a tiny one at the very end, after we know the name of the target repo, to add an entry to the repos menu.

Basically what we need is:

1. Bug 1360542 fixed so that a central->beta gecko_migration.py action exists.
2. https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/configs/merge_day/staging_beta_migration.py updated to use central->beta, and perhaps to use a different target repo
3. The bug (or awareness of the existence of a bug, dunno if it's filed, but the awareness exists because of the need to migrate the hsts/hpkp/blocklist updates to taskcluster) to make it possible to have a taskcluster job which creates a push fixed
4. A taskcluster cronjob that runs gecko_migration.py -c central_to_beta.py -c staging_beta_migration.py
5. Both Taskcluster and Buildbot automation set up to do the same builds/tests as beta on whatever target repo we use (insert massive handwaving here about how we can handle both early_beta_or_earlier and late-beta, since I don't have any idea)

I could be wrong, but I doubt a useful half-measure of using a patch exists - I don't think we have any existing automation that would automatically rebase a patch against mozilla-central, and if we instead wind up adding yet another step onto every merge to m-c, to have to merge m-c to the patched repo, we're going to wind up with more total work than just having the person who is responsible that cycle pull/rebase/push-to-try.

Comment 3

[Ryan VanderMeulen [:RyanVM]]

I think the value of this proposal is that it would allow us to test our release automation beforehand as well, not just Gecko itself. So yeah, IMO the win is directly tied to dropping the fixed patches in favor of automated simulated merge day commits instead.

EARLY_BETA_OR_EARLIER is an interesting edge case I hadn't given any thought to. I guess there's no reason we *couldn't* run two pushes with and without it set...

Comment 4

[Phil Ringnalda (:philor)]

People have finally caught on to using early_beta, and many more will pile on to it now that they can't have both nightly and aurora for the things they aren't ready to ship. I used to just unset it for all my simulations and not bother testing the first few weeks case, but in the last couple of rounds we've had early-only test failures, and (horrors) late-only build bustage, so we really do need to test it.

Comment 5

[Phil Ringnalda (:philor)]

Looks like bug 1171193 morphed from being about porting periodic-update, which would have had a dependency on "Be able to push from a taskcluster job" to being "Be able to push from a taskcluster job."

Comment 6

[Tom Prince [:tomprince]]

(cross posted from bug 1372697 comment #59)

In bug 1497575, I've added a tool for doing try pushes that configure the tree as various release type builds, which will (when https://phabricator.services.mozilla.com/D8452 lands) correctly pick the mozconfig variant that is used for that type of release build. It is currently optimized for testing the release process, so only builds nightly (i.e. shippable) builds, and doesn't run any tests, but I think it could easily be extended to have configurations for things useful for other teams.

Comment 7

[[e-mail] (:bhearsum)]

I think this is irrelevant now due to us running merge day simulations on Try.