1373150 - Disable compile environment on l10n builds

Status: RESOLVED FIXED

(Release Engineering :: General, enhancement, P1)

Description

[Mike Hommey [:glandium]]

In preparation for the after-bug 1313111 life where I want to start removing things from tooltool manifests, l10n jobs expecting to pull things from there (and not yet being on TC so being able to get the TC toolchain artifacts) are making things more difficult than necessary.

As it turns out, l10n builds are essentially artifact builds. They shouldn't need a toolchain. In fact, they probably only do for nsinstall (which is why they run make export in config/), but the build system can work without nsinstall.

I did a try build with --disable-compile-environment, and the l10n build that was triggered there (arguably on TC) was strill green.

So let's see if l10n buildbot builds bust if we do the same.

Comment 1

[Mike Hommey [:glandium]]

Attachment: Bug 1373150 - Disable compile environment on l10n builds.

Review commit: https://reviewboard.mozilla.org/r/149314/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/149314/

Comment 2

[Justin Wood (:Callek)]

Comment on attachment 8877885 [details]
Bug 1373150 - Disable compile environment on l10n builds.

https://reviewboard.mozilla.org/r/149314/#review154074

from the try run: https://treeherder.mozilla.org/logviewer.html#?job_id=107394380&repo=try&lineNumber=6332 --  failed configure on windows.

Comment 3

[Mike Hommey [:glandium]]

(In reply to Justin Wood (:Callek) from comment #2)
> Comment on attachment 8877885 [details]
> Bug 1373150 - Disable compile environment on l10n builds.
> 
> https://reviewboard.mozilla.org/r/149314/#review154074
> 
> from the try run:
> https://treeherder.mozilla.org/logviewer.
> html#?job_id=107394380&repo=try&lineNumber=6332 --  failed configure on
> windows.

Log is unavailable...

Comment 4

[Justin Wood (:Callek)]

https://treeherder.mozilla.org/logviewer.html#?job_id=107394380&repo=try&lineNumber=6332

Comment 5

[Mike Hommey [:glandium]]

Comment on attachment 8877885 [details]
Bug 1373150 - Disable compile environment on l10n builds.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/149314/diff/1-2/

Comment 6

[Mike Hommey [:glandium]]

Comment on attachment 8877885 [details]
Bug 1373150 - Disable compile environment on l10n builds.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/149314/diff/2-3/

Comment 7

[Mike Hommey [:glandium]]

Comment on attachment 8877885 [details]
Bug 1373150 - Disable compile environment on l10n builds.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/149314/diff/3-4/

Comment 8

[Mike Hommey [:glandium]]

Comment on attachment 8877885 [details]
Bug 1373150 - Disable compile environment on l10n builds.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/149314/diff/4-5/

Comment 9

[Mike Hommey [:glandium]]

Comment on attachment 8877885 [details]
Bug 1373150 - Disable compile environment on l10n builds.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/149314/diff/5-6/

Comment 10

[Mike Hommey [:glandium]]

Note about the try associated with the review:
- The busted SM builds are perma busted for other reasons and aren't even running on m-c/inbound/etc.
- The Linux opt l10n(l10n) build is busted because of bug 1373486 ; here's another try with the patch from that bug applied: https://treeherder.mozilla.org/#/jobs?repo=try&revision=9b6d8d358af0f13d45f501f2f5e584d30cdb4353 where it fails for a different reason (see below)
- The Linux x64 opt l10n(l10n) build is busted because it tries to download http://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central/target.tar.bz2, which doesn't exist, and that's related to EN_US_PACKAGE_NAME being set to target.tar.bz2 and en_us_binary_url to http://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central, neither of which is my due.

Comment 11

[Mike Hommey [:glandium]]

And it looks like the osx and windows builds are busted because of things like:
20:09:45  WARNING - Timed out accessing http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-central/mar-tools/macosx64/mar: timed out

which seem unrelated too.

Comment 12

[Justin Wood (:Callek)]

Comment on attachment 8877885 [details]
Bug 1373150 - Disable compile environment on l10n builds.

https://reviewboard.mozilla.org/r/149314/#review154498

Can you make sure a followup is on file about tooltool manifests for l10n jobs. Specifically many mozharness runs download a bunch of tooltool things to support compiling, (e.g. windows does msvc), with this patch its not needed.

I should note that your toolchain support could eliminate some of the pain there, but even ahead of that the tooltool cleanup(s) could benefit us in end to end.

::: build/mozconfig.no-compile:11
(Diff revision 6)
> +unset HOST_CC
> +unset HOST_CXX
> +unset RUSTC
> +unset CARGO
> +unset MAKECAB
> +unset TOOLCHAIN_PREFIX

I'm less confident on why we need (or should need) the `unset`'s here, since imho that means --disable-compile-environment doesn't properly block that stuff from "doing things" even in a local environment where those values are defined.

That said, I'm no expert on what they are needed for, and in practice our mozconfigs are "for automation" not local dev so, not blocking.

Comment 13

[Justin Wood (:Callek)]

Comment on attachment 8877885 [details]
Bug 1373150 - Disable compile environment on l10n builds.

https://reviewboard.mozilla.org/r/149314/#review154500

Additionally, I am *slightly* concerned with the windows results, but I do think its merely a case of "too big locales lists break on Buildbot based try l10n jobs..."  (the linux BB l10n, I'm not worried about)

Given that can you do a new try push before landing, with a *much* reduced https://dxr.mozilla.org/mozilla-central/source/browser/locales/all-locales -- specifically languages of `ru, ja{,-jp-mac},fr,en-GB,zh-TW}` will be plenty enough for my purposes

Comment 14

[Mike Hommey [:glandium]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=f849f43acf873834ba82b5ca69e29374976c3931&filter-searchStr=l10n All green

Comment 15

[Mike Hommey [:glandium]]

(In reply to Justin Wood (:Callek) from comment #12)
> ::: build/mozconfig.no-compile:11
> (Diff revision 6)
> > +unset HOST_CC
> > +unset HOST_CXX
> > +unset RUSTC
> > +unset CARGO
> > +unset MAKECAB
> > +unset TOOLCHAIN_PREFIX
> 
> I'm less confident on why we need (or should need) the `unset`'s here, since
> imho that means --disable-compile-environment doesn't properly block that
> stuff from "doing things" even in a local environment where those values are
> defined.

Actually, configure rejects them being set. And mozconfigs are a mess, it's easier to unset them.

Comment 16

[Mike Hommey [:glandium]]

(In reply to Justin Wood (:Callek) from comment #12)
> Comment on attachment 8877885 [details]
> Bug 1373150 - Disable compile environment on l10n builds.
> 
> https://reviewboard.mozilla.org/r/149314/#review154498
> 
> Can you make sure a followup is on file about tooltool manifests for l10n
> jobs. Specifically many mozharness runs download a bunch of tooltool things
> to support compiling, (e.g. windows does msvc), with this patch its not
> needed.

I'll do that once it's confirmed this sticks.

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/943dc0fef263
Disable compile environment on l10n builds. r=Callek

Comment 18

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/943dc0fef263