1053761 - L10n nightly builds failing post build changes

Status: RESOLVED FIXED

(Thunderbird :: Build Config, defect)

Description

[Mark Banner (:standard8)]

Now we've got Win32 nightly builds running again, we're seeing L10n failures (Linux are running but failed today due to a different bug that I've fixed).

http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/2014-08-14-03-02-02-comm-central-l10n/comm-central-win32-l10n-nightly-ro-bm86-build1-build3025.txt.gz

'python' 'c:/builds/moz2_slave/tb-c-cen-w32-l10n-ntly-0000000/tools/buildfarm/utils/retry.py' '-s' '1' '-r' '5' '-t' '1260' 'python' 'c:/builds/moz2_slave/tb-c-cen-w32-l10n-ntly-0000000/build/comm-central/build/pymake/make.py' 'wget-en-US'
 in dir c:\builds\moz2_slave\tb-c-cen-w32-l10n-ntly-0000000\build/comm-central/objdir-tb/mail/locales (timeout 1320 secs)
 watching logfiles {}
 argv: ['python', 'c:/builds/moz2_slave/tb-c-cen-w32-l10n-ntly-0000000/tools/buildfarm/utils/retry.py', '-s', '1', '-r', '5', '-t', '1260', 'python', 'c:/builds/moz2_slave/tb-c-cen-w32-l10n-ntly-0000000/build/comm-central/build/pymake/make.py', 'wget-en-US']
...
Executing: ['python.exe', 'c:/builds/moz2_slave/tb-c-cen-w32-l10n-ntly-0000000/build/comm-central/build/pymake/make.py', 'wget-en-US']
pymake\..\..\mozmake.exe: *** No rule to make target 'wget-en-US'.  Stop.
retry: Failed, sleeping 1 seconds before retrying

Comment 1

[Mark Banner (:standard8)]

This is most likely the issue:

"in dir c:\builds\moz2_slave\tb-c-cen-w32-l10n-ntly-0000000\build/comm-central/objdir-tb/mail/locales"

So that's a result of combining the objdir.

Comment 2

[Joshua Cranmer [:jcranmer]]

(In reply to Mark Banner (:standard8) from comment #1)
> This is most likely the issue:
> 
> "in dir
> c:\builds\moz2_slave\tb-c-cen-w32-l10n-ntly-0000000\build/comm-central/
> objdir-tb/mail/locales"
> 
> So that's a result of combining the objdir.

Yeah, for the current objdir setup, it should be objdir-tb/mozilla/mail/locales.

Of course, if we finish pseudo-rework part2, objdir-tb/mail/locales is correct. :-)

Comment 3

[Mark Banner (:standard8)]

Following the latest patches on bug 1059511, we're now working again on Aurora on Linux. Windows is still an issue:

http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/2014-09-05-00-40-05-comm-aurora-l10n/comm-aurora-win32-l10n-nightly-bn-BD-bm85-build1-build4106.txt.gz

'python' 'c:/builds/moz2_slave/tb-c-aurora-w32-l10n-ntly-0000/build/comm-aurora/build/pymake/make.py' 'ident'
 in dir c:\builds\moz2_slave\tb-c-aurora-w32-l10n-ntly-0000\build/comm-aurora/objdir-tb/mail/locales (timeout 1200 secs)
...
buildid: '20140905004005'
moz_revision: '92a6bbdfd945'
comm_revision: '92a6bbdfd945'

Those revs should be different...

Comment 4

[Mark Banner (:standard8)]

I checked the builds, and the issue is the application.ini files - the Linux versions are using the comm repo & revision, but the Windows versions are using the mozilla repo & revision...

Joshua, any ideas?

Comment 5

[Joshua Cranmer [:jcranmer]]

http://dxr.mozilla.org/comm-central/source/mail/locales/Makefile.in#137 is where we print out... I'm guessing that MOZ_SOURCE_STAMP is being defined improperly?

Comment 7

[Mark Banner (:standard8)]

(In reply to Joshua Cranmer [:jcranmer] from comment #5)
> http://dxr.mozilla.org/comm-central/source/mail/locales/Makefile.in#137 is
> where we print out... I'm guessing that MOZ_SOURCE_STAMP is being defined
> improperly?

Yeah, that's my guess. Though I've no idea why it is just windows. It is either something to do with the new build system, or the fact that we've switched how application.ini is generated.

Comment 9

[Joshua Cranmer [:jcranmer]]

I honestly don't have the time to look at this in the near future.

Comment 10

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

From the build log:
-DMOZ_SOURCE_REPO='https://hg.mozilla.org/comm-central'
changed to
-DMOZ_SOURCE_REPO='https://hg.mozilla.org/mozilla-central'

So http://mxr.mozilla.org/comm-central/source/mail/app/Makefile.in#22
SOURCE_REPO := $(shell hg -R $(topsrcdir) showconfig paths.default 2>/dev/null | sed -e "s/^ssh:/http:/")
either has a wrong topsrcdir or a messed up config. If it would be the latter one, we would see more issues. So does something overwrite topsrcdir / @top_srcdir@?

Comment 11

[Alice0775 White]

No progress? 
So, It need to backed out Bug 1040009 before mearge happens Aurora to Beta.

Comment 12

[Mark Banner (:standard8)]

I wonder if this might be fixed by bug 1084997

Comment 13

[Mark Banner (:standard8)]

Looks like bug 1084997 may have fixed this - as of today we've got Windows L10n builds on central/daily (as well as Mac & Linux).

I've asked for it to be rolled out to branches, so let's hope it fixes those as well.

Comment 14

[Mark Banner (:standard8)]

I've respun the aurora nightly builds now that bug 1084997 is on aurora and Windows L10n builds have started appearing on ftp :-)

They are still running though, so it may be an hour or two more before we get a full set.