685166 - Port the work to enable mozconfigs to live in the source tree

Status: RESOLVED FIXED

(Thunderbird :: Build Config, defect)

Description

[Mark Banner (:standard8)]

Firefox have been working on putting their mozconfigs in the source tree. This will make it simpler at merge time, as any differences from central to aurora (for example) will just be automatically merged across, and we won't need to do a separate analysis in the build-config repository for what needs to be merged across mozconfigs.

Comment 1

[John Hopkins (:jhopkins)]

Here is the Firefox bug:

Bug 558180 - Move mozconfigs into source tree

Comment 2

[John Hopkins (:jhopkins)]

If we could start by copying the mozconfigs into comm-central, that would help me with bug 698843: I need to tweak the mozconfigs to ALWAYS_RUN_CLIENT_PY (currently only done on tryserver).

Comment 3

[John Hopkins (:jhopkins)]

Attachment: copy of buildbot-configs/thunderbird/* mozconfigs

Comment 4

[Mark Banner (:standard8)]

Comment on attachment 572893 [details] [diff] [review]
copy of buildbot-configs/thunderbird/* mozconfigs

Actually, now I look at what Firefox is doing, I believe this wants to be:

- You land only the mozconfigs for a particular version on that branch.
- The mozconfigs are renamed and structured thus:
-- <platform>/<type>
--- where type is one of debug, release, nightly, l10n-mozconfig etc.

Comment 5

[John Hopkins (:jhopkins)]

<Standard8> jhopkins: the point is that if central has a change in mozconfig, it'll automatically get merged across to aurora etc at the right times
<jhopkins> Standard8: ah, i had thought debug flags might be an issue
<Standard8> jhopkins: shouldn't be, all our builds should have separate configs

Comment 6

[John Hopkins (:jhopkins)]

Attachment: comm-aurora mozconfigs

Comment 7

[John Hopkins (:jhopkins)]

Attachment: comm-beta mozconfigs

Comment 8

[John Hopkins (:jhopkins)]

Attachment: comm-central mozconfigs

Comment 9

[John Hopkins (:jhopkins)]

Attachment: comm-release mozconfigs

Comment 10

[Mark Banner (:standard8)]

Comment on attachment 600175 [details] [diff] [review]
comm-central mozconfigs




>+mk_add_options CLIENT_PY_ARGS="--mozilla-repo=http://hg.mozilla.org/mozilla-central --skip-inspector --skip-venkman --skip-chatzilla --hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --tinderbox-print"

Can we drop the mozilla-repo argument here and rely on the client.py defaults? This would make it much easier for each merge every 6 weeks, as we only need to update client.py.

Also, please ensure the order of arguments is consistent across the files.

>diff --git a/mail/config/mozconfigs/linux64/debug b/mail/config/mozconfigs/linux64/debug

>+mk_add_options ALWAYS_RUN_CLIENT_PY=1
>\ No newline at end of file

Please fix these (there's several of them).


>diff --git a/mail/config/mozconfigs/macosx-universal/unittest b/mail/config/mozconfigs/macosx-universal/unittest
>diff --git a/mail/config/mozconfigs/win32/unittest b/mail/config/mozconfigs/win32/unittest

I don't believe we need these files, if we do then I'd expect to have Linux versions as well.

I'd also expect release mozconfigs, but we can add those later if you want.

Comment 11

[Mark Banner (:standard8)]

Comment on attachment 600173 [details] [diff] [review]
comm-aurora mozconfigs

I think the same comments apply here, cancelling reviews until there's updated patches.

Comment 12

[John Hopkins (:jhopkins)]

Attachment: comm-central mozconfigs

Mark: I'll get your review on the comm-central mozconfigs and do the others if/when these are approved.

Comment 13

[Mark Banner (:standard8)]

Comment on attachment 604463 [details] [diff] [review]
comm-central mozconfigs

Review of attachment 604463 [details] [diff] [review]:
-----------------------------------------------------------------

This is fine, with the minor nits fixed. We will need release mozconfigs to go on central as well, but I know you're still getting that part of automation running, so I'm happy for this to land with the nits fixed and the release mozconfigs part as follow-up.

::: mail/config/mozconfigs/linux64/debug
@@ +27,5 @@
> +CONCURRENCY=$(( `grep processor /proc/cpuinfo | wc -l` + 2 ))
> +mk_add_options MOZ_MAKE_FLAGS="-j$CONCURRENCY"
> +
> +# Run client.py
> +mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-chatzilla --skip-inspector --skip-venkman --tinderbox-print"

nit: You've got --skip-chatzilla in here twice.

::: mail/config/mozconfigs/linux64/nightly
@@ +28,5 @@
> +CONCURRENCY=$(( `grep processor /proc/cpuinfo | wc -l` + 2 ))
> +mk_add_options MOZ_MAKE_FLAGS="-j$CONCURRENCY"
> +
> +# Run client.py
> +mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print"

nit: --skip-chatzilla twice

::: mail/config/mozconfigs/win32/debug
@@ +15,5 @@
> +export MOZ_DEBUG_SYMBOLS=1
> +
> +
> +# Run client.py
> +mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-chatzilla --skip-inspector --skip-venkman --tinderbox-print"

nit: --skip-chatzilla is here twice

::: mail/config/mozconfigs/win32/nightly
@@ +14,5 @@
> +# Needed to enable breakpad in application.ini
> +export MOZILLA_OFFICIAL=1
> +
> +# Run client.py
> +mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print"

nit: --skip-chatzilla twice

Comment 14

[John Hopkins (:jhopkins)]

Committed comm-central mozconfigs in http://hg.mozilla.org/comm-central/rev/fe140018319f

Comment 15

[Mark Banner (:standard8)]

Comment on attachment 604463 [details] [diff] [review]
comm-central mozconfigs

[Triage Comment]
a=me for landing this on comm-beta.

Comment 16

[John Hopkins (:jhopkins)]

Landed in comm-beta at these changesets:

http://hg.mozilla.org/releases/comm-beta/rev/c0d7b0d6c42a
http://hg.mozilla.org/releases/comm-beta/rev/63a11393de40

Comment 17

[John Hopkins (:jhopkins)]

TODO: We still need release mozconfigs.

Comment 18

[Mark Banner (:standard8)]

Attachment: release mozconfigs

Release mozconfigs with s/release/${MOZ_UPDATE_CHANNEL}/ in the --enable-update-channel line (which I believe FF automation sets correctly), and also the necessary lines for client.py calls.

Comment 19

[John Hopkins (:jhopkins)]

Comment on attachment 615895 [details] [diff] [review]
release mozconfigs

Review of attachment 615895 [details] [diff] [review]:
-----------------------------------------------------------------

The only discrepancy I noticed were these lines missing from the linux32 mozconfig (compared to the current mozconfig in buildbot-configs):

< # XXX disable pedantic until we get MOZ_IPC or we resolve the build issues
< ac_add_options --disable-pedantic

Looks good - thanks, Mark!

Comment 20

[Mark Banner (:standard8)]

(In reply to John Hopkins (:jhopkins) from comment #19)
> The only discrepancy I noticed were these lines missing from the linux32
> mozconfig (compared to the current mozconfig in buildbot-configs):
> 
> < # XXX disable pedantic until we get MOZ_IPC or we resolve the build issues
> < ac_add_options --disable-pedantic

Yes, that's expected, see the review request on bug 737028!

Comment 21

[Mark Banner (:standard8)]

Checked in:

http://hg.mozilla.org/comm-central/rev/56ea15c0e0a2

As far as I know we don't need mozconfigs for l10n, although admittedly I'm confused by the ones Firefox has in-tree - the l10n base seem to all reference l10n-central regardless of what tree they are on.

If we do need those, then we can either reopen or file a new bug.

Comment 22

[Mark Banner (:standard8)]

Comment on attachment 615895 [details] [diff] [review]
release mozconfigs

[Triage Comment]
Going to take these forward so that we've got them on aurora & beta hence ready for the switch.

Comment 23

[Mark Banner (:standard8)]

Checked into branches:

http://hg.mozilla.org/releases/comm-aurora/rev/4ab069b426ba
http://hg.mozilla.org/releases/comm-beta/rev/122d6253f4e8

Comment 24

[Mark Banner (:standard8)]

Comment on attachment 604463 [details] [diff] [review]
comm-central mozconfigs

[Triage Comment]
Taking this onto esr as well, as we'll need it there, and currently its NPOTB.

Comment 25

[Mark Banner (:standard8)]

Checked in:

http://hg.mozilla.org/releases/comm-esr10/rev/40679e585d2a
http://hg.mozilla.org/releases/comm-esr10/rev/24648526a812