594930 - Release automation should pushsnip automatically

Status: RESOLVED WONTFIX

(Release Engineering :: Release Automation: Other, defect, P4)

Description

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

As part of release automation, once we get a formal "go to pushsnip" from rel-drivers, automation should:

> - wait for formal "go"
> - pushsnip to "beta" if needed
> - pushsnip to "release" if needed
...as well as handle pushsnip to "betatest" and "releasetest"

Comment 1

[Chris Cooper [:coop] (he/him)]

(In reply to comment #0)
> > - wait for formal "go"
> > - pushsnip to "beta" if needed
> > - pushsnip to "release" if needed
> ...as well as handle pushsnip to "betatest" and "releasetest"

I'd like to see the automation do the following:

* backupsnip (need to modify script to provide some output?)
* push to the test channel (betatest or releasetest)
* run automated Mozmill update tests against test channel (bug 498425)
* push to appropriate channel (beta or release)

Comment 2

[[e-mail] (:bhearsum)]

test channels are already automatically pushed as part of the 'updates' builder

Comment 3

[Rail Aliiev [:rail]]

Attachment: buildbotcustom changes

Not tested in staging, a first draft.

* ReleaseUpdatesFactory.getSnippetDir() and MajorUpdateFactory.getSnippetDir() logic moved to tools/lib/python
* Used Firefox-$version-build$n (and similar for MU) instead of fuzzy $date-Firefox-$version
* Added standalone builders for pushing release, beta, mu, mu-beta snippets

Comment 4

[Rail Aliiev [:rail]]

Attachment: tools

Comment 5

[[e-mail] (:bhearsum)]

Some comments:
- I'd like to see a bit of protection against somebody accidentally hitting "Force Build". Something like, bailing out unless a certain property name/value exists would be sufficient.
- ffxbld won't work for backupsnip/pushsnip, you should use ausUser from the release config
- To be superduper safe, it would be good to check that the directory exists prior to trying to run any commands
- For consistency, I'd like to see ReleaseUpdatesFactory use this for pushing test snippets rather than its current steps.
- What do you think of having a more generic builder for this, and using a property to define "action"? If you did that, you could even forego my first point.
- Mail should be sent to release-drivers 5min after the "pushsnip" completes successfully.

I assume the echo_run_remote_cmd() will be a real run_remote_cmd() at some point?

Comment 6

[[e-mail] (:bhearsum)]

Comment on attachment 505796 [details] [diff] [review]
buildbotcustom changes

feedback- to this one, see previous comment.

Comment 7

[[e-mail] (:bhearsum)]

Just realized another important thing: separating backupsnip/pushsnip -- we need to be able to run backupsnip well ahead of when we're going to push.

Comment 9

[Rail Aliiev [:rail]]

Not working on this bug actively. Back to the pool.

Comment 10

[[e-mail] (:bhearsum)]

Mass move of bugs to Release Automation component.

Comment 11

[[e-mail] (:bhearsum)]

We'll need this for rapid betas.

Comment 12

[Lukas Blakk [:lsblakk] use ?needinfo]

With regards to the rapid betas, this trigger should be based on QA automation completing the sign-offs. Possibly use Rabbit queue here or something else, tbd.

Comment 13

[Lukas Blakk [:lsblakk] use ?needinfo]

(In reply to Lukas Blakk [:lsblakk] from comment #12)
> With regards to the rapid betas, this trigger should be based on QA
> automation completing the sign-offs. Possibly use Rabbit queue here or
> something else, tbd.

And by based on, I mean that as one option since while this will be true for rapid betas it will not be for all types of releases. Some will required different triggers (email to rel-drivers, for example).

Comment 14

[[e-mail] (:bhearsum)]

It's notable that this goes away when Balrog is in production, too, where "pushsnip" is replaced with a web ui.

Comment 15

[[e-mail] (:bhearsum)]

I don't think it's worth doing this at this point because the Balrog is coming. (https://wiki.mozilla.org/Balrog)