1572316 - Add l10n-bumper task to Thunderbird

Status: RESOLVED FIXED

(Thunderbird :: Build Config, task)

Description

[Rob Lemley [:rjl]]

Attachment: l10n_bumper_configs

Bug for getting the l10n_bumper configs into Thunderbird and documenting how to do it.

The attached patch is a WIP and only the esr68 one has been tested.

Comment 1

[Rob Lemley [:rjl]]

Attachment: l10n-bumper.patch

Automatic l10n changeset bumper for Thunderbird. It mostly works; there's
some Treestatus problems to sort out

Firefox runs this every hour. I think we can safely trigger it to run on
comm-beta on merge day somehow and that will be sufficient.

Comment 2

[Rob Lemley [:rjl]]

When checking to see if the tree is closed via Treestatus, an error occurs:

2020-01-20 20:16:04,877 - scriptworker_client.aio - INFO - Downloading https://treestatus.mozilla-releng.net/trees/comm-central
2020-01-20 20:16:05,066 - scriptworker_client.aio - DEBUG - Failed to download https://treestatus.mozilla-releng.net/trees/comm-central: 400; body={
  "detail": "No tree comm-central found.",
  "status": 400,
  "title": "Bad Request",
  "type": "about:blank"
}

The URLs should be:
https://treestatus.mozilla-releng.net/trees/comm-central-thunderbird
https://treestatus.mozilla-releng.net/trees/comm-beta-thunderbird
https://treestatus.mozilla-releng.net/trees/comm-esr68-thunderbird
but of course try-c-c is different:
https://treestatus.mozilla-releng.net/trees/try-comm-central

I suspect that -thunderbird suffix has something to do with Seamonkey. It doesn't make sense to me to add logic to Treescript to support this case. Not sure what the best way forward might be. Aki?

Side note: The treestatus check can be bypassed altogether by setting the "ignore-closed-tree" flag in the task description. That's a little dangerous for my taste, personally.

Comment 3

[Aki Sasaki (not active)]

Hm.

I can think of two solutions off the top of my head:

1. rename comm-*-thunderbird to remove the -thunderbird suffix. If Seamonkey isn't still a concern, this makes sense, but requires some treestatus config work. I'm also not sure what looks at that: hg.m.o hooks, treeherder, etc may need updates.
2. Create a map in treescript. If tree doesn't exist in the map, default to tree. If it does exist in the map, use the associated value. So the map could look like

{
    "comm-central": "comm-central-thunderbird",
    "comm-beta": "comm-beta-thunderbird",
    "comm-esr68": "comm-esr68-thunderbird",
}

and https://github.com/mozilla-releng/scriptworker-scripts/blob/master/treescript/src/treescript/l10n.py#L154-L155 might look like

    tree = get_short_source_repo(task)
    treestatus_tree = TREESTATUS_MAP.get(tree, tree)
    url = "%s/trees/%s" % (config["treestatus_base_url"], treestatus_tree)

Comment 4

[Magnus Melin [:mkmelin]]

Comment on attachment 9121933 [details] [diff] [review]
l10n-bumper.patch

Review of attachment 9121933 [details] [diff] [review]:
-----------------------------------------------------------------

(not really my cup of tea)

Comment 5

[Rob Lemley [:rjl]]

Submitted a pull request for Treescript at https://github.com/mozilla-releng/scriptworker-scripts/pull/145.

I wound up writing a small function that matches the repo name with the prefix 'comm-' and appends '-thunderbird' or not accordingly. It's just as much of a hack, but I find it preferable to creating a map of repo names to treestatus names that will require maintenance whenever a new esr repo is created.

Comment 6

[Rob Lemley [:rjl]]

Feedback on the pull request in comment 5. It's preferable to make changes to Treestatus so that the tree names there match the repo names in Mercurial, so no more 'comm-central-thunderbird' and 'comm-central-seamonkey'.
That would mean changes for Seamonkey and when they can push. I've been in contact with Edmund and he reports back that this is okay to do.
I'll be opening bugs to make changes to other systems affected by such a change: (this list should be everything)

• TreeStatus itself: First to create a new tree with the name we want, then after the other changes are done the old trees can be removed. (I think this is all database changes, no code?)
• Changes to hg.m.o push-receive hooks to refer to the new TreeStatus URLs for comm-* repos
• Changes to Treeherder for the same.

Comment 7

[Philipp Kewisch [:Fallen] [:📆][:🧩]]

Comment on attachment 9121933 [details] [diff] [review]
l10n-bumper.patch

Review of attachment 9121933 [details] [diff] [review]:
-----------------------------------------------------------------

From what I can tell this looks fine to me. Glad you are getting the repo names resolved, but I'd also be fine checking in with the Seamonkey team and removing the `-thunderbird` suffix.

Comment 8

[Rob Lemley [:rjl]]

I checked with the Seamonkey team last week and they are okay with the changes, so that is moving forward.

(In reply to Philipp Kewisch [:Fallen] [:📆][:🧩] from comment #7)

From what I can tell this looks fine to me. Glad you are getting the repo
names resolved, but I'd also be fine checking in with the Seamonkey team and
removing the -thunderbird suffix.

Comment 9

[Rob Lemley [:rjl]]

Attachment: Add l10n_bumper task

Runs the l10n pinning script that I've been doing manually in automation.
This much is copied from Firefox where they run the pinning as a cron task
once an hour.

Comment 10

[Rob Lemley [:rjl]]

Attachment: Add l10n-bumper-beta task

Thunderbird does not do as many beta releases as Firefox, so it's not really
necessary to run the l10n-bump job that often. It's really only needs to run
when preparing a new release. The first push to beta on merge day will trigger
a l10n-bump with this configuration as the version.txt file and .gecko_rev.yml
both get updated at that time. Any additional betas will also trigger l10n-bump
assuming it's pinned to a different Gecko version.

Comment 11

[Rob Lemley [:rjl]]

Attachment: Add l10n-bumper action

This really isn't 100% necessary, I mostly wanted to see if actions could be
defined in our tree. This does work, it will show up in Treeherder in the
"Custom Push Action" dialog. It will be useful to have as we roll the next
major release out this summer. Anyone will level 3 should be able to trigger
it.

Comment 12

[Geoff Lankow (:darktrojan)]

Comment on attachment 9134932 [details] [diff] [review]
Add l10n_bumper task

Review of attachment 9134932 [details] [diff] [review]:
-----------------------------------------------------------------

::: taskcluster/ci/config.yml
@@ +21,5 @@
>          'Deb10': 'Packages for Debian 10'
>          'Fetch': 'Fetch and store content'
>          'L10n': 'Localised Repacks'
>          'L10n-Rpk': 'Localized Repackaged Repacks'
> +        'l10n-bump': 'L10n Bumper'

This probably should have an uppercase L. But m-c has a lowercase one. Hmm.

::: taskcluster/ci/l10n-bump/kind.yml
@@ +27,5 @@
> +                beta: https://hg.mozilla.org/releases/comm-beta
> +                default: https://hg.mozilla.org/comm-central
> +        l10n-bump-info:
> +            by-release-type:
> +                beta:

I think it might be a good idea to specify that ESR does nothing here. If for some weird reason this does get run on ESR it would fall through to the default section and we don't want that.

Comment 13

[Rob Lemley [:rjl]]

(In reply to Geoff Lankow (:darktrojan) from comment #12)

Comment on attachment 9134932 [details] [diff] [review]
::: taskcluster/ci/l10n-bump/kind.yml

•    l10n-bump-info:
  

•        by-release-type:
  

•            beta:
  

I think it might be a good idea to specify that ESR does nothing here. If
for some weird reason this does get run on ESR it would fall through to the
default section and we don't want that.

Yep should be beta|release there. It's not that it won't run on esr ever, it most likely will in the early 78s.

Comment 14

[Rob Lemley [:rjl]]

Attachment: Add l10n_bumper task

Sets up the l10n-bumper task to run in automation for Thunderbird.
Basic config here could run as a cron task or via "Add new jobs" action.

Comment 15

[Rob Lemley [:rjl]]

Comment on attachment 9135194 [details] [diff] [review]
Add l10n_bumper task

Review of attachment 9135194 [details] [diff] [review]:
-----------------------------------------------------------------

darktrojan had r+'d a previous version, this addresses the comment about not running properly on esr.

Since this job is really meant to run on beta requesting uplift. It will never run on comm-central.

Comment 16

[Wayne Mery (:wsmwk)]

Comment on attachment 9134934 [details] [diff] [review]
Add l10n-bumper action

[Triage Comment]

Comment 17

[Wayne Mery (:wsmwk)]

Comment on attachment 9135194 [details] [diff] [review]
Add l10n_bumper task

[Triage Comment]

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/be12a117692a
Add l10n_bumper task. r=darktrojan
https://hg.mozilla.org/comm-central/rev/0b44865abda1
Add l10n-bumper-beta task. r=darktrojan
https://hg.mozilla.org/comm-central/rev/8471c521e820
Add l10n-bumper action. r=darktrojan

Comment 19

[Rob Lemley [:rjl]]

Thunderbird 75.0b2:
https://hg.mozilla.org/releases/comm-beta/rev/4f52ed76a6a0
https://hg.mozilla.org/releases/comm-beta/rev/fc8fa71d32c2
https://hg.mozilla.org/releases/comm-beta/rev/7e75d447441c

Comment 20

[Rob Lemley [:rjl]]

Attachment: Follow-up: Fix paths to check for changes in l10n-bump-beta task

This should work better.

Comment 21

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/30bd179df0f5
Follow-up: Fix paths to check for changes in l10n-bump-beta task. r=darktrojan