1537713 - improve beetmover resilience by switching Thunderbird and Devedition to declarative artifacts; cleanup afterwards

Status: RESOLVED FIXED

(Release Engineering :: Release Automation: Uploading, enhancement)

Description

[Mihai Tabara [:mtabara]⌚️GMT]

In Q1 we are undergoing changes that move the beetmover templates in-tree. This work makes beetmover more ligthweight and reduces our operations when an urgent fix is needed.

One of the long-time running problems we have been having with beetmover is our tests coverage. We often made mistakes both in-tree and in beetmoverscript that weren't caught, despite having 100% code coverage. That is because we have a dozen types of beetmover types but the tests only cover a tiny fraction of those.

This OKR should track work to improve this in order to catch potential errors early. This includes:

• rewrite of the beetmoverscript tests following the declarative artifacts work in Q1

• support integration tests for all types of beetmover

• add tests coverage in-tree for templates as well

Comment 1

[Mihai Tabara [:mtabara]⌚️GMT]

FWIW, operational tests/staging testing has taken 60% of declarative artifacts world. Mostly because there wasn't a clear way of ensuring destinations will match and automate that process verification.

Moreover, there's various weird things that beetmoverscript is consuming from a task, things like task.extra{} and task.tags{}. I'd really love for those to go away and have beetmoverscript solely rely on payload.

Integration tests should also ensure that when an in-tree manifest change happens, that doesn't or does affect the manifest.json (that's consumed by balrogworkers downstream). I had to go over that process manually.

Comment 2

[Mihai Tabara [:mtabara]⌚️GMT]

Attachment: Bug 1537713 - switch Devedition to declarative artifacts.r=sfraser a=release

Switch Devedition to declarative artifacts

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/83d73c85529a
switch Devedition to declarative artifacts.r=sfraser a=release

Comment 4

[Mihai Tabara [:mtabara]⌚️GMT]

Attachment: [beetmoverscript] remove useless templates from beetmoverscript as we switched to DA

Comment 5

[Mihai Tabara [:mtabara]⌚️GMT]

Attachment: [beetmoverscript] remove zip support in mobile world as we transition to declarative artifacts

Comment 6

[Mihai Tabara [:mtabara]⌚️GMT]

Attachment: [beetmoverscript] WIP - initial cleanup in beetmover now that all mobile world has switched to declarative artifacts

Comment 7

[Mihai Tabara [:mtabara]⌚️GMT]

Once https://hg.mozilla.org/integration/autoland/rev/83d73c85529a gets to central & beta, we can close this bug as follows:

• Devedition + Thunderbird are now in declarative-artifacts world as well.
• large cleanup in transforms to remove conflicting artifacts is now done
• some initial cleanup in beetmoverscript is now completed, courtesy to the PRs enumerated in this bug.

More cleanup in beetmoverscript is tracked in bug 1567124 but it's due to happen in Q4 only because we need to wait until we EOL esr60.

Comment 8

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/83d73c85529a

Comment 9

[Mihai Tabara [:mtabara]⌚️GMT]

Grafted to beta too so that tomorrow's Devedition picks it up.
https://hg.mozilla.org/releases/mozilla-beta/rev/e7f3f12d59e3b95b63bef535fa022ac9646afc9f

Comment 10

[Mihai Tabara [:mtabara]⌚️GMT]

Attachment: [beetmoverscript] Remove even more templates following switch to declarative artifacts