1481916 - migrate l10n bumper to treescript

Status: RESOLVED FIXED

(Release Engineering :: Release Automation, enhancement)

Description

[Aki Sasaki (not active)]

Currently, we run l10n-bumper via cron, through a puppetized mozharness workdir, on two use1 buildbot masters. These can get stuck (bug 1455355) and are less visible/testable than other automation in taskcluster. We also want to retire the buildbot masters in September. These two masters are in use1, so we don't need to retire them when scl3 goes away, but it would be preferable to not need to keep them around indefinitely.

We're currently thinking:

- migrate l10n-bumper behavior to treescript
- add the appropriate scopes to control that behavior and restrict it
- create a cron task and hook to trigger it on a schedule. The hook should be manually triggerable by releaseduty and probably ciduty; we may want to allow the l10n team to trigger it as well.

This task should ideally be visible on treeherder, so errors are more noticeable.

Comment 1

[Aki Sasaki (not active)]

At some point, rather than landing directly, this task would push to phabricator and land via lando.

Comment 2

[Aki Sasaki (not active)]

Attachment: Bug 1481916 - add l10n-bumper task. r=Callek

Comment 3

[Aki Sasaki (not active)]

Attachment: treescript PR

Comment 4

[Aki Sasaki (not active)]

https://firefoxci.taskcluster-artifacts.net/Qy9g939aT-2YnIbh1lhdRQ/0/public/logs/live_backing.log seems to show I need to fix the purge call (or the mercurial extensions/deps).

Comment 5

[Aki Sasaki (not active)]

sheehan 12:30
@aki: I haven't seen that before but I took a look and I think it's a bug in robustcheckout
purge used to import util when that line was introduced, now imports scmutil and cmdutil instead

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/11f48aaae955
add l10n-bumper task. r=Callek

Comment 7

[Aki Sasaki (not active)]

Followup scriptworker-scripts PR: https://github.com/mozilla-releng/scriptworker-scripts/pull/77

Comment 8

[Aki Sasaki (not active)]

Attachment: puppet - remove l10n bumper support

Comment 9

[Daniel Varga [:dvarga]]

https://hg.mozilla.org/mozilla-central/rev/11f48aaae955

Comment 10

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/mozilla-central/rev/4bc97ee973d2
Backed out changeset 11f48aaae955 as per aki's request. a=backout

Comment 11

[Andrei Ciure[:aciure]]

Backed out changeset 11f48aaae955 (bug 1481916) for causing l10 bumper exceptions

Comment 12

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9fe3bf81d49d
Backed out changeset 11f48aaae955 for causing l10 bumper exceptions CLOSED TREE

Comment 13

[Aki Sasaki (not active)]

In https://treeherder.mozilla.org/#/jobs?repo=try&revision=f31831e913874e69cda1143b1853753ef7341437&selectedJob=277493566 , we appear to not end up in an esr68 head. https://firefoxci.taskcluster-artifacts.net/fotQEh4WSde367cteULElA/0/public/logs/live_backing.log

We could try landing and fixing if it's a problem on esr68 itself, since I don't think we'll push the wrong thing anymore. Alternately, we could block uplifting on https://github.com/mozilla-releng/scriptworker-scripts/issues/88 .

The beta-esque try push looks good after I fixed the release-type -> release_type issue. https://treeherder.mozilla.org/#/jobs?repo=try&revision=7750c5d0e8168d08244646d5fc3a05cf4dd94be6&selectedJob=277489964 and https://firefoxci.taskcluster-artifacts.net/cvCDoob-RmG8aU3vDHMs8g/0/public/logs/live_backing.log

Comment 14

[Aki Sasaki (not active)]

Tom, any thoughts?

Comment 15

[Tom Prince [:tomprince]]

I think if you hacked taskgraph to generate a metadata that pointed at esr68, it would test what you want.

Comment 16

[Aki Sasaki (not active)]

Attachment: treescript - source_repo and target_repo support

Comment 17

[Aki Sasaki (not active)]

The metadata hack didn't work: it broke CoT.

I:

• created a PR to add source_repo and target_repo support, and populated source_repo in-tree
• fixed the esr68 l10n dashboard link
• removed the use of push scopes, so we can try to push to try in the future
• fixed flake8
• tested a bunch

I'm pretty confident in this now. Thanks for your help!

I was wondering if we would need a way to override ignore-closed-tree and force the cron hook, so mergeduty can run l10n-bumper. Then I realized that in the near future, we'll be kicking off both Firefox and Devedition b1 without a week of beta closure, so we may not need to force it (we just need to reopen beta to approval-only after merging, then wait for the l10n bumper to run before gtb).

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/98bbf5137827
add l10n-bumper task. r=Callek,tomprince

Comment 19

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/98bbf5137827

Comment 20

[Aki Sasaki (not active)]

Comment on attachment 9108892 [details]
Bug 1481916 - add l10n-bumper task. r=Callek

ESR Uplift Approval Request

• If this is not a sec:{high,crit} bug, please state case for ESR consideration: This will enable the l10n bumper for Fennec releases on esr68.
• User impact if declined: When we turn off buildbot-master01, where the retiring l10n bumper has been running, we'll no longer pick up new Fennec locale bumps. If we no longer need Fennec releases off esr68, this may be the desirable outcome.
• Fix Landed on Version: 72
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): I've tested quite a bit on Try. We now count the number of outgoing changes and make sure they match the expected number of outgoing patches.
• String or UUID changes made by this patch:

Comment 21

[Axel Hecht [:Pike]]

(In reply to Aki Sasaki [:aki] (he/him) (UTC-7) from comment #20)

• If this is not a sec:{high,crit} bug, please state case for ESR consideration: This will enable the l10n bumper for Fennec releases on esr68.
• User impact if declined: When we turn off buildbot-master01, where the retiring l10n bumper has been running, we'll no longer pick up new Fennec locale bumps. If we no longer need Fennec releases off esr68, this may be the desirable outcome.

From an l10n POV, I'd prefer to be on the safe side and continue to ship string updates on Fennec. Well, I consider this landing to the safe side than not landing it.

Comment 22

[Aki Sasaki (not active)]

Attachment: Bug 1481916 - add l10n-bumper force cron hooks. r=callek

Comment 23

[Mihai Tabara [:mtabara]⌚️GMT]

Worked like a charm on beta earleir today as part of mergeduty! \o/

Comment 24

[Aki Sasaki (not active)]

:jcristau, do you have an opinion about whether we should keep running l10n bumper on esr68 for Fennec? We're going to shut down the instance that's running l10n bumper soon, and we'll need to uplift https://bugzilla.mozilla.org/attachment.cgi?id=9108892 to keep running l10n bumper on esr68.

Comment 25

[Julien Cristau [:jcristau]]

Sounds like we should uplift then. I suspect the EOL for fennec on esr68 is a few months away still.

flod fyi.

Comment 26

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/releases/mozilla-esr68/rev/f5013b2a8949

Comment 27

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9108892 [details]
Bug 1481916 - add l10n-bumper task. r=Callek

For posterity.

Comment 28

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Every time the L10n bumper script runs, 15 lint and source test tasks are getting executed, e.g. https://firefox-ci-tc.services.mozilla.com/tasks/groups/bmNZADS2QQO30c6j8Y5Tqw
This looks like this: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&selectedJob=280274456&searchStr=python&revision=053b0bb00fedc42ea54e7a6b8e32843d7d6b7849

It made bug 1500321 more visible. Aki, does this need a follow-up action?

Comment 29

[Aki Sasaki (not active)]

There's a patch in bug 1500321 now.