Open Bug 902876 Opened 6 years ago Updated 11 months ago

[Tracking Bug] Fix SeaMonkey l10n dep nightly builds

Categories

(SeaMonkey :: Build Config, defect, major)

defect
Not set
major

Tracking

(seamonkey2.21 affected, seamonkey2.22 affected, seamonkey2.23 affected)

ASSIGNED
Tracking Status
seamonkey2.21 --- affected
seamonkey2.22 --- affected
seamonkey2.23 --- affected

People

(Reporter: tonymec, Assigned: ewong)

References

(Depends on 1 open bug)

Details

(Keywords: intl, Whiteboard: [workaround see comment #26])

Attachments

(3 files, 3 obsolete files)

No SeaMonkey langpacks later than 26 July. The most recent trunk builds are as follows:

- 26-Jul: 2.22a1 be, de, es-AR, es-ES, ga, lt, hu, nl, ro, ru, sk, uk, zh-CN, zh-TW
- 09-Jul: 2.22a1 en-GB, fr, ja, ja-JP-mac, pl 
- 2.19a1 cz, nb-NO
- 2.17a1 pt-PT
- 2.14a1 it
- 2.12a1 sv-SE
- 2.6a1 fi
- 2.1b1pre ka
So regarding the build problems on aurora, I saw a build log yesterday about stdint.h missing. Firefox also had the same problem on their beta channel, see Bug 902091. They solved it by using some rather complicated patches IMHO. I think for us the easiest/best solution would be to fix the include/lib/path used during the build process. The problem is that the l10n process does call "make" directly and does not use client.mk to do the various work. But "make" does not take the env vars from this mozconfig file (http://mxr.mozilla.org/mozilla-central/source/build/win32/mozconfig.vs2010?raw=1&ctype=, it's included in the l10n mozconfig) into account. I would just set those path on those machines (via buildbot) during the build process. Firefox can't do that as they still use VS 2008 for the ESR builds. We only use VS 2010 on Windows afaik, so there's no point in keeping the old VS 2008 INCLUDE/LIB/PATH/LIBPATH/WIN32_REDIST_DIR env vars around.
And I suppose there are similar mozconfigs for linux-i686 and mac. Strange that make does not obey them; normally it takes env vars as make vars, but if defined also in the Makefile without a test that they already exist you need "make -e" to give envvars precedence over makevars.
July 26 is also the latest build date for linux-i686 _full_ Suite builds; and, for some time prior to that, new langpacks appeared only in https://ftp.mozilla.org/pub/mozilla.org/seamonkey/nightly/latest-comm-central-trunk-l10n/linux-i686/xpi/ but not in ../../win32/xpi/ or ../../mac/xpi/
BTW: Bug 903118 is working on exporting the env vars from .mozconfig files into the build environment. But not sure how and when and on which branches this patch will land.
Now that Linux builds are being made again, langpacks have also started being produced again (but only from linux-i686 builds, not win32 or mac) for languages be, de, es-AR, es-ES, gl, hu, lt, nl, pl, ro, ru, sk, uk, zh-CN, zh-TW, i.e., the same ones that were being built just before building stopped altogether on Linux (ga in comment #0 is probably a typo).

No full localized builds either except on Linux; on win32 and mac they are being attempted but fail: IIUC the .txt.gz files in http://ftp.mozilla.org/pub/mozilla.org/seamonkey/nightly/latest-comm-central-trunk-l10n/ are the build logs.

IOW non-Linux testers who want a localized build should grab an en-US build for their platform and a linux-i686 langpack for their language (one among the 15 above).
So what is stopping the ones from the 09-Jul-13 from being built now?
(In reply to Ian Neal from comment #6)
> So what is stopping the ones from the 09-Jul-13 from being built now?

I don't know. I guess the same thing that prevented them being built from 10 to 26 July.

Let's have a look at http://ftp.mozilla.org/pub/mozilla.org/seamonkey/nightly/latest-comm-central-trunk-l10n/comm-central-trunk-linux-l10n-nightly-en-GB-bm01-universal-build714.txt.gz (which SeaMonkey unzips for me, no need to download it first):

Comparing en-GB to en-US
Properties in ./chrome/chatzilla.properties don't match:
  In ../../../extensions/irc/locales/en-US: (add these to your localization)
    cmd.clear-view.key

make[2]: *** [libs] Error 1
There is also:
Bug 901338 - creation of langpack for for locale "de" fails
(In reply to Philip Chee from comment #8)
> There is also:
> Bug 901338 - creation of langpack for for locale "de" fails

Yes, but that's for SeaMonkey 2.20. German (and 13 other) langpacks are now being built again on trunk.
(In reply to Philip Chee from comment #8)
> There is also:
> Bug 901338 - creation of langpack for for locale "de" fails

That's different, it's not about our build systems but about someone trying to create it himself, without our L10n merging that runs on our build systems.
BTW: I looked at some random OS X build log for trunk from today (http://ftp.mozilla.org/pub/mozilla.org/seamonkey/nightly/latest-comm-central-trunk-l10n/comm-central-trunk-macosx64-l10n-nightly-es-ES-bm01-universal-build2078.txt.gz), the error message says:
========= Started configure done failed (results: 2, elapsed: 30 secs) (at 2013-09-16 23:38:56.702796) ==========
[...]
checking Python environment is Mozilla virtualenv... yes
configure: error: Invalid value --with-l10n-base, ../../l10n-central doesn't exist
*** Fix above errors and then restart with               "make -f client.mk build"
make[1]: *** [configure] Error 1
Summary: No SeaMonkey langpacks (or, I presume, localized builds) later than 26 July → No SeaMonkey langpacks (or localized builds) later than 26 July
Justin: Am I correct that probably Bug 852553 needs to be ported to our buildbot version and also the mozconfig changes need to be ported (to get nightly l10n builds working again)? Not sure why exactly this needs to be done now, but ewong told me you basically applied the patch from Bug 852553 locally to the buildbot (not committed to mercurial yet) to get beta builds working.
Flags: needinfo?(bugspam.Callek)
(In reply to Frank Wein [:mcsmurf] from comment #12)
> Justin: Am I correct that probably Bug 852553 needs to be ported to our
> buildbot version and also the mozconfig changes need to be ported (to get
> nightly l10n builds working again)? Not sure why exactly this needs to be
> done now, but ewong told me you basically applied the patch from Bug 852553
> locally to the buildbot (not committed to mercurial yet) to get beta builds
> working.

The buildbotcustom changes from bug 852553 has been incorporated into
our buildbotcustom code (from bug 1039139) so this part is done.  We're
now waiting for changes from bug 840427 to be reviewed and pushed to
the repo (which then requires a subsequent master reconfig).  But this
is waiting for the resolution of bug 795354.

So what this bug needs now is the suite/config changes, which I'll provide
asap.
Flags: needinfo?(bugspam.Callek)
Assignee: nobody → ewong
Status: NEW → ASSIGNED
Attachment #8458558 - Flags: review?(bugspam.Callek)
Attachment #8458558 - Attachment is obsolete: true
Attachment #8458558 - Flags: review?(bugspam.Callek)
Attachment #8459081 - Flags: review?(bugspam.Callek)
Comment on attachment 8458558 [details] [diff] [review]
config/mozconfig changes patch (v1)

Review of attachment 8458558 [details] [diff] [review]:
-----------------------------------------------------------------

::: suite/config/mozconfigs/macosx-universal/l10n-mozconfig
@@ +9,5 @@
>  
>  . $topsrcdir/build/macosx/universal/mozconfig
>  
>  ac_add_options --enable-application=suite
> +ac_add_options --with-l10n-base=../../l10n

Actually, just noticed that this should have been a ../../../l10n,
so I updated the patch.
Summary: No SeaMonkey langpacks (or localized builds) later than 26 July → No SeaMonkey langpacks (or localized builds) later than 26 July 2013
Comment on attachment 8459081 [details] [diff] [review]
config/mozconfig changes patch (v2)

Review of attachment 8459081 [details] [diff] [review]:
-----------------------------------------------------------------

not looking closely at non-beta atm, but stamp=me and a=me for needed branches
Attachment #8459081 - Flags: review?(bugspam.Callek) → review+
(In reply to Edmund Wong (:ewong) from comment #16)
> Comment on attachment 8458558 [details] [diff] [review]
> config/mozconfig changes patch (v1)
> 
> Review of attachment 8458558 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: suite/config/mozconfigs/macosx-universal/l10n-mozconfig
> @@ +9,5 @@
> >  
> >  . $topsrcdir/build/macosx/universal/mozconfig
> >  
> >  ac_add_options --enable-application=suite
> > +ac_add_options --with-l10n-base=../../l10n
> 
> Actually, just noticed that this should have been a ../../../l10n,
> so I updated the patch.

Just wondering why this needs to have the extra ../
(In reply to Ian Neal from comment #19)
> (In reply to Edmund Wong (:ewong) from comment #16)
> > Comment on attachment 8458558 [details] [diff] [review]
> > config/mozconfig changes patch (v1)
> > 
> > Review of attachment 8458558 [details] [diff] [review]:
> > -----------------------------------------------------------------
> > 
> > ::: suite/config/mozconfigs/macosx-universal/l10n-mozconfig
> > @@ +9,5 @@
> > >  
> > >  . $topsrcdir/build/macosx/universal/mozconfig
> > >  
> > >  ac_add_options --enable-application=suite
> > > +ac_add_options --with-l10n-base=../../l10n
> > 
> > Actually, just noticed that this should have been a ../../../l10n,
> > so I updated the patch.
> 
> Just wondering why this needs to have the extra ../

(aside for mail/  having the same thing) I tried it with only
../../ and it failed due to path issues.  I haven't looked into
it too deeply (since I'm currently fixing the buildbotcustom code),
but once I get it done.. I'll figure out why.
Comment on attachment 8459081 [details] [diff] [review]
config/mozconfig changes patch (v2)

[Approval Request Comment]
Regression caused by (bug #): 
User impact if declined: no langpacks created. 
Testing completed (on m-c, etc.): 
Risk to taking this patch (and alternatives if risky):
String changes made by this patch:
Attachment #8459081 - Flags: approval-comm-beta?
Attachment #8459081 - Flags: approval-comm-aurora?
Comment on attachment 8459081 [details] [diff] [review]
config/mozconfig changes patch (v2)

Got a+ from Callek over IRC:

Pushed to comm-aurora:
https://hg.mozilla.org/releases/comm-aurora/rev/4013101a7d2e

Pushed to comm-beta:
https://hg.mozilla.org/releases/comm-beta/rev/b52ded91ca91
Comment on attachment 8459081 [details] [diff] [review]
config/mozconfig changes patch (v2)

Marking as a= due to comment 22. Should this be marked as fixed now it is pushed?
Attachment #8459081 - Flags: approval-comm-beta?
Attachment #8459081 - Flags: approval-comm-beta+
Attachment #8459081 - Flags: approval-comm-aurora?
Attachment #8459081 - Flags: approval-comm-aurora+
Flags: needinfo?(ewong)
(In reply to Ian Neal from comment #23)
> Comment on attachment 8459081 [details] [diff] [review]
> config/mozconfig changes patch (v2)
> 
> Marking as a= due to comment 22. Should this be marked as fixed now it is
> pushed?

No.  OSX langpacks aren't being generated.  I'm not sure why because we
have the 'same' path as mail/; but mail/ doesn't choke whereas we do.
I haven't figured out why it's not liking the ../../../l10n path.
Flags: needinfo?(ewong)
Attached patch OSX64 fix. (obsolete) — Splinter Review
In the previous patch, I set the --with-l10n-basedir=../../../l10n because
mail/ had it.  I'm not sure if they even use that for their l10n langpacks;
haven't had the chance to check out mail/'s infra logs.
This patch is to make it run with ../../l10n.  (I mean, if it works with
linux* and WinNT, who am I to say it won't work with OSX64? ;P )
Attachment #8506536 - Flags: review?(bugspam.Callek)
As a workaround, langpacks (and full Linux localized builds) for some languages can be downloaded from the subdirectories of http://l10n.mozilla-community.org/~akalla/unofficial/seamonkey/nightly/

For langpacks (but of course not for full builds), you may disregard the platform but not the branch.

These langpacks and builds are "unofficial" because they are built out of Mozilla control, but (Adrian, please correct me if I'm wrong) they are built from official Mozilla sources.
Whiteboard: [workaround see comment #26]
(In reply to Tony Mechelynck [:tonymec] from comment #26)
> As a workaround, langpacks (and full Linux localized builds) for some
> languages can be downloaded from the subdirectories of
> http://l10n.mozilla-community.org/~akalla/unofficial/seamonkey/nightly/

This is not really a workaround for the bug - more a way to get language packs without the need to build them itself ;)

> These langpacks and builds are "unofficial" because they are built out of
> Mozilla control, but (Adrian, please correct me if I'm wrong) they are built
> from official Mozilla sources.

Yep - they are always being built from the latest Mozilla sources (see the Python script I do use to create this builds: https://bitbucket.org/adrianer/seamonkey-build/src/default/build-sm.py ).


Nevertheless: why isn't this bug fixed nearly 1,5 years later...? :(
(In reply to Adrian Kalla [:adriank] from comment #27)
> (In reply to Tony Mechelynck [:tonymec] from comment #26)
> > As a workaround, langpacks (and full Linux localized builds) for some
> > languages can be downloaded from the subdirectories of
> > http://l10n.mozilla-community.org/~akalla/unofficial/seamonkey/nightly/
> 
> This is not really a workaround for the bug - more a way to get language
> packs without the need to build them itself ;)

Depends on what one calls a workaround. It is a workaround for the user who cannot get langpacks from Mozilla. It is not a workaround for the RelEng and l10n teams trying to produce langpacks and offer them at ftp.mozilla.org, and it is certainly not a fix.
Comment on attachment 8506536 [details] [diff] [review]
OSX64 fix.

Pushed to comm-central: (bustage fix)
https://hg.mozilla.org/comm-central/rev/c31c999efa37
(In reply to Edmund Wong (:ewong) from comment #29)
> Comment on attachment 8506536 [details] [diff] [review]
> OSX64 fix.
> 
> Pushed to comm-central: (bustage fix)
> https://hg.mozilla.org/comm-central/rev/c31c999efa37

This was backed out as it's actually wrong.  The original value is
correct, i.e. ../../../l10n.

The reason is because the platform OBJDIR is objdir/i386, whereas
for the other platforms, they're just objdir.
Attached patch osx64 fix (v3) (obsolete) — Splinter Review
Attachment #8506536 - Attachment is obsolete: true
Attachment #8506536 - Flags: review?(bugspam.Callek)
Attachment #8574404 - Flags: review?(bugspam.Callek)
Attached patch osx64 fix (v4)Splinter Review
Previous patch removed the mozconfig include.  Instead, we should skip
the build/macosx/universal/mozconfig (which includes the mozconfig.common file)
and use the mozconfig.common file directly.
Attachment #8574404 - Attachment is obsolete: true
Attachment #8574404 - Flags: review?(bugspam.Callek)
Attachment #8574581 - Flags: review?(bugspam.Callek)
Comment on attachment 8574581 [details] [diff] [review]
osx64 fix (v4)

Rationale for this patch:

$topsrcdir/build/macosx/universal/mozconfig defines build projects which,
for macosx64 = i386 and x86_64.  What this does is during the client.mk 
configure process, it creates under the objdir (in osx64 case, it's 
objdir/i386) two subdirectories (i386 and x86_64) as the new objdir.  However, 
all of the osx64 building assumes the objdir = objdir/i386.   So when the build 
process goes into objdir/i386/i386,  the configure process chokes because it's 
not finding the l10n dir (which is normally ../../../l10n, but with the added 
level, it should be ../../../../l10n).

(Another solution was to use --with-l10n-base=../../../../l10n, but that's
clunky.)

However, since build/macosx/universal/mozconfig also calls the mozconfig.common
file from the same directory, we might as well use the mozconfig.common file.
Comment on attachment 8574581 [details] [diff] [review]
osx64 fix (v4)

Since only our osx64 nightlies are building, and triggering the l10n
changes,  I can only hope to garner some info from the osx64 nightly l10n
repacks.

I'm going to push this for a post-land review and trigger a repack.
(In reply to Edmund Wong (:ewong) from comment #35)
> Comment on attachment 8574581 [details] [diff] [review]
> osx64 fix (v4)
> 
> Pushed to comm-central:
>   https://hg.mozilla.org/comm-central/rev/c462becfd612

This managed to unhork it from the configure but it fails at the repack_installer
step (which is possibly fixed in the backend..)

So this needs to be pushed to comm-aurora.
Comment on attachment 8574581 [details] [diff] [review]
osx64 fix (v4)

[Triage Comment]
a=me for c-a
Attachment #8574581 - Flags: approval-comm-aurora+
Duplicate of this bug: 1086553
https://hg.mozilla.org/SeaMonkey/puppet/rev/70968214f45fae3d5566700c85ff875985ecabbc
Whitelist c-cen-t-lnx-l10n-ntly temporarily to help fix bug 902876
repacks now need to use the tooltool-enabled mozconfigs.
Attachment #8778791 - Flags: review?(philip.chee)
Attachment #8778791 - Flags: review?(bugspam.Callek)
https://hg.mozilla.org/comm-central/rev/887ac30fffb3e349d8774d56c5ea9d65697b249a
Bug 902876 - Remove no_tooltool from Linux* l10n mozconfigs. r+a=bustagefix
Depends on: 1184085
Attachment #8778791 - Flags: review?(philip.chee)
Attachment #8778791 - Flags: review?(bugspam.Callek)
Attachment #8778791 - Flags: review+
Do we want to port Bug 1283438? (Set AUTOCLOBBER to empty for l10n)
Flags: needinfo?(ewong)
(In reply to Philip Chee from comment #42)
> Do we want to port Bug 1283438? (Set AUTOCLOBBER to empty for l10n)

Possibly something we might need.
Flags: needinfo?(ewong)
Morphing this to a tracking bug.
Summary: No SeaMonkey langpacks (or localized builds) later than 26 July 2013 → [Tracking Bug] Fix SeaMonkey l10n dep nightly builds
Attachment #8574581 - Flags: review?(bugspam.Callek)
You need to log in before you can comment on or make changes to this bug.