Closed Bug 758149 Opened 7 years ago Closed 4 years ago

create-release-repacks.py (l10n repacks) should get mozconfigs from source, not from buildbot-configs

Categories

(Release Engineering :: Release Automation: Other, defect, P3)

defect

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: standard8, Unassigned)

References

Details

Attachments

(4 files, 2 obsolete files)

Currently create-release-repacks.py is obtaining the mozconfig to use for l10n repacks from buildbot-configs, rather than from the source tree.
We should add these to the source repositories, create a list in the release configs (like https://github.com/mozilla/buildbot-configs/blob/master/mozilla/release-firefox-mozilla-beta.py#L107), and then change the logic to find it based on the location in the release config. Should be pretty easy, not high priority though.
Priority: -- → P3
Attached patch Tools fix v1 (obsolete) — Splinter Review
This patch updates the tools scripts which is where the repacks come from. Specifically wrt mozconfigs it:

- Looks for a new key in releaseConfig called 'l10n-mozconfigs'
- If that is found, then it will attempt to use the source mozconfig specified, falling back to the buildbot-configs one if that is not found
- If the 'l10n-mozconfigs' key isn't present, it will just fall back to the old method of getting mozconfigs from buildbot-configs.

It also adds a 'cat .mozconfig' step so the result can be identified.

I've tested this locally as far as the copying of the mozconfig with a couple of extra patches that I'll attach in a moment, and it seemed to work fine.

Not sure if you guys will want a staging run on it though.

I thought we could start rolling this out with the next Thunderbird beta in a couple of weeks time, and then move onto other products/channels later.
Assignee: nobody → mbanner
Attachment #730161 - Flags: review?(bhearsum)
This patch tells the comm-beta release scripts to use the in source tree version, and removes the old buildbot-configs mozconfigs for comm-beta.

I've also verified according to the last comm-beta release that the 'release' mozconfigs in buildbot-configs are no longer required afaict.
Attachment #730164 - Flags: review?(bhearsum)
This is the patch that adds the mozconfigs to comm-beta. I think the client.py-args changes will work, but I'm just pushed to try to verify (https://tbpl.mozilla.org/?tree=Thunderbird-Try&rev=1f5f9f1c992e).

When I land this on branches, I'll make sure to update our scripts to update the l10n path name (for now, I believe there's a separate bug on that somewhere).
Attachment #730168 - Flags: review?(bhearsum)
Comment on attachment 730168 [details] [diff] [review]
Add configs to comm-central and co

This doesn't quite work on Windows wrt the client.py args. I'll think about that tomorrow.
Attachment #730168 - Flags: review?(bhearsum)
Comment on attachment 730161 [details] [diff] [review]
Tools fix v1

Review of attachment 730161 [details] [diff] [review]:
-----------------------------------------------------------------

::: lib/python/build/l10n.py
@@ +56,5 @@
> +                    cp %(src_mozconfig)s .mozconfig;
> +                else
> +                    echo Downloading mozconfig;
> +                    cp ../%(hg_mozconfig)s .mozconfig;
> +                fi'''.replace("\n", "") % {'src_mozconfig': srcMozconfigPath, 'hg_mozconfig': mozconfigPath }]

Is there any reason this bash to be used? We do it in Buildbot because there's no alternative, but this should be translated to Python if possible. You should be able to get rid of the extra fallback in the "else" block if you do.

Can you file a bug to kill the fallback logic too? We should keep it until we ship a final release/esr with the new mozconfigs (in case we need to chemspill from a relbranch), but I don't want to support it longer than we have to.
Attachment #730161 - Flags: review?(bhearsum) → review-
Comment on attachment 730164 [details] [diff] [review]
buildbot-configs - Thunderbird comm-beta

Review of attachment 730164 [details] [diff] [review]:
-----------------------------------------------------------------

Rolling out on Thunderbird first seems like a good idea as long as that beta is happening in the first half of the cycle.
Attachment #730164 - Flags: review?(bhearsum) → review+
I just remembered that client.py picks up MOZILLA_REV and COMM_REV from the environment and buildbot already sets those for l10n builds.

I'd forgotten it with the previous version of this patch, so this will work now.
Attachment #730168 - Attachment is obsolete: true
Attachment #752169 - Flags: review?(bhearsum)
Attached patch Tools fix v2Splinter Review
Fixes the review comments.
Attachment #752172 - Flags: review?(bhearsum)
Attachment #752169 - Flags: review?(bhearsum) → review+
Comment on attachment 752172 [details] [diff] [review]
Tools fix v2

Review of attachment 752172 [details] [diff] [review]:
-----------------------------------------------------------------

::: scripts/l10n/create-release-repacks.py
@@ +239,4 @@
>          l10nBaseRepo=make_hg_url(branchConfig["hghost"],
>                                   releaseConfig["l10nRepoPath"]),
>          mozconfigPath=mozconfig,
> +        srcMozconfigPath=releaseConfig.get('l10n-mozconfigs', {}).get(options.platform),

Can you change this to l10n_mozconfigs on landing, for consistency?

Sorry I didn't get to this before beta 1!
Attachment #752172 - Flags: review?(bhearsum) → review+
Attachment #730161 - Attachment is obsolete: true
Comment on attachment 752172 [details] [diff] [review]
Tools fix v2

Landed this section:

https://hg.mozilla.org/build/tools/rev/4405017a713d
Attachment #752172 - Flags: checked-in+
Comment on attachment 730164 [details] [diff] [review]
buildbot-configs - Thunderbird comm-beta

https://hg.mozilla.org/build/buildbot-configs/rev/94098aa58ab8
Attachment #730164 - Flags: checked-in+
Live in production.
Looks like we need to update the source factory as well?

cp: cannot stat `configs/mozilla2/linux/comm-beta/release/mozconfig': No such file or directory

http://buildbot-master64.srv.releng.usw2.mozilla.com:8001/builders/release-comm-beta-thunderbird_source/builds/0
Rather than try to fix the source issue straight away, lets play it safe and put the mozconfig back that it wants. We can then follow-up later with a full fix for the source factories.
Attachment #770353 - Flags: review?(aki)
Attachment #770353 - Flags: review?(aki) → review+
Comment on attachment 730164 [details] [diff] [review]
buildbot-configs - Thunderbird comm-beta

https://hg.mozilla.org/build/buildbot-configs/rev/7f72e2475137
Attachment #730164 - Flags: checked-in+ → checked-in-
Bug 898099 has fixed some of this for all apps, but there's some tidy up that can be done once we've completed shipping with that in place.
Depends on: 898099
Product: mozilla.org → Release Engineering
Not working on this atm
Assignee: standard8 → nobody
This script is dying in favour of the new Mozharness one.
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.