523856 - Localized builds on WinCE

Status: RESOLVED FIXED

(Firefox Build System :: General, defect, P2)

Description

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

What's involved in creating localized builds for WinCE?  We had a request come in for an Indonesian build (id), and I'm just looking for a rough time schedule.

Comment 1

[Armen [:armenzg]]

If I was to start today I would say a couple of weeks if the "make installers-locale" stuff is already in place and working properly.

Comment 2

[Axel Hecht [:Pike]]

We don't have Makefile logic to repack cab's.

For fennec, folks are not really fond of cabs anyway, so it really depends on:

What's a Firefox build on WinCE?

If it's cabs, doing cab will require someone that knows cabs to do it, basically, nobody in the l10n nor in the releng team have the background knowledge on that.

Comment 3

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Creating a CAB file is not an issue; if we can get zip files created, we can turn those into CABs as needed.  The same bits end up on the disk, so updates are not affected.

Comment 4

[Axel Hecht [:Pike]]

If it's just for testing purposes, the langpacks on ftp should work fine.

Comment 5

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Any update on this?

Armen, what do I need to do to test and see whether the current code works correctly?  Just run make installers-locale?  Do I need to check out the locales anywhere?

Comment 6

[Armen [:armenzg]]

Vlad this script should give you an idea more or less of what is needed:
https://wiki.mozilla.org/User:Armenzg:l10n_scripts
It might be slightly out-of-date but it pretty much works like that.

If you don't use --disable-compile-environment you will have make few more things. Let me know.

make installers-locale is the step that will do the work. You should find the installers under "dist".

Ping me if you have any questions.

Comment 7

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Script seems to work just fine, though I'm not quite getting the right branch from the locale dir or something.  Like it's complaining about missing chrome/global/aboutRights.dtd, aboutRights.properties, and aboutSupport.dtd in l10n-central/id/toolkit (I'm using the 1.9.2 tag in the id locale).  When I manually put those files back in, it happily generated firefox-3.6b2pre.id.wince-arm.zip.

Comment 8

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

I should say, for clarity -- it correctly downloaded the zip build and repackaged it into a .id zip build without problems.

Comment 9

[Armen [:armenzg]]

If you look at the dashboard you could see that depending which locale you choose it will work smoothly and if not you should use compare-locale to generate a merged folder.
http://l10n.mozilla.org/dashboard/?tree=fx37x

If the locale doesn't have everything that en-US it creates a folder called "merged" (which will contain no missing entities or files):
python ../../../compare-locales/scripts/compare-locales -m merged l10n.ini ../../../l10n-central or
(in dir /builds/moz2_slave/mozilla-central-linux-l10n-nightly/build/mozilla-central/browser/locales)

Make installers will check to see if the LOCALE_MERGEDIR exists and then uses that directory instead of the locale's checkout:
make installers-or LOCALE_MERGEDIR=$PWD/merged
(in dir /builds/moz2_slave/mozilla-central-linux-l10n-nightly/build/mozilla-central/browser/locales)

Comment 10

[Chris Cooper [:coop] (he/him)]

There was some discussion about this in #build today in ref. to a requested Indonesian repack. 

Vlad: should we be re-prioritizing this out of Future? What's the horizon on this? There are some other priorities here if it's helpful:

https://wiki.mozilla.org/ReleaseEngineering:L10nMeeting/2009-11-02#Priorities

Comment 11

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

I don't know, tbh -- it all depends on a real estimate for time to get this set up.  It seems like our current zip repack infrastructure "just works", so I don't know if it's a matter of twiddling a config file, or something more involved than that.  Ideally I need to produce an Indonesian l10n build today, which I can do (and planned to do) manually, if need be.

Comment 12

[Justin Dolske [:Dolske]]

I think we should reprioritize this up, because (1) we need to ship a localized 3.6 build one WinCE and (2) this should be a small project.

As vlad noted, producing a localized .zip build is already working when run manually. I just got it working too, no changes needed. So I think the only notable tasks here should be:

1) Modify build config to trigger creation of L10N builds for WinCE, like other platforms (just a matter of adding "wince" to "l10nPlatforms"?

2) Add a repackage-cab-% target to browser/locales/Makefile.in, and glue to feed it an existing .cab. [Basically, a .cab specific version of the existing repackage-zip-% target / .zip glue].

The latter of these is just mozilla-central code, so I'll go ahead and work on fixing that. I think once that's done, this should all Just Work.

Comment 13

[Justin Dolske [:Dolske]]

Attachment: Patch v.1

This seems to be the easiest, if slightly hacky, fix to cause "make installers-AB_CD" to repack cabs, yielding an appropriately named .CAB in $DIST. It's basically just leveraging the existing ZIP repack, since the CAB generation wants the exact same directory tree as input.

I tried making a repackage-cab target, but that would require some rejiggering of repackage-zip so that we could ensure the same dir tree is being fed to both (ie, lest someone make changes to repackage-zip but not repackage-cab). Calling "make package MOZ_PKG_FORMAT=CAB" also suffers from wanting to do more work than is needed, so it seemed easier to just call the make_wince_cab.py script directly.

Hopefully this is a good enough start that someone more familiar with the build and releng bits can easily complete this.

Comment 14

[Axel Hecht [:Pike]]

Justin, do you know if the name you pass for the .inf file should be the localized display name or the non-localized appname? Like, on desktop, we install into the Firefox dir, but the desktop links are localized names.

Comment 15

[Justin Dolske [:Dolske]]

Pike: Looks like it currently should be the non-localized name, as the generated inf in my tree gets:

[CEStrings]
AppName = "Namoroka"
InstallDir = %CE1%\%AppName%

We could modify the make_wince_cab.py script to take both, though, and use the localized one for "AppName" and the non-localized one for "InstallDir".

Does "Mozilla" get localized? Right now that's hardcoded in the script, and could also become an argument if needed...

Comment 16

[Armen [:armenzg]]

I will look into this next week. I have a working patch to add a nightly builder which needs a day or two of proper testing and debugging. Adding the nightly builder will help me get the right idea of how much work is needed for the release builder.

Justin can we get the patch attached reviewed and landed? The release side needs those targets to call them.

Comment 17

[Justin Dolske [:Dolske]]

Attachment: Patch v.2

Reworked version of last patch; I didn't want to split the CAB creation into two places because they'd inevitably get out of sync.

Comment 18

[Armen [:armenzg]]

Changing ownership and product/component of the bug after filing bug 531275 which will tackle the Release Engineering specific tasks to make this happen.

Comment 19

[Axel Hecht [:Pike]]

Comment on attachment 414689 [details] [diff] [review]
Patch v.2

This patch looks good to me.

Comment 20

[Mike Beltzner [:beltzner, not reading bugmail]]

Comment on attachment 414689 [details] [diff] [review]
Patch v.2

a192=beltzner

Comment 21

[Justin Dolske [:Dolske]]

Pushed http://hg.mozilla.org/mozilla-central/rev/11a9c2e01f58
Pushed to 192: http://hg.mozilla.org/releases/mozilla-1.9.2/rev/ac78276d3189

Followup work in bug 531275 to actually start producing localized nightlies.

Comment 22

[Nick Thomas [:nthomas] (UTC+12)]

>diff --git a/toolkit/mozapps/installer/packager.mk b/toolkit/mozapps/installer/packager.mk
>+	"$(MOZ_PKG_DIR)" "$(MOZ_APP_DISPLAYNAME)" "$(PKG_BASENAME).cab"
>-	"$(MOZ_PKG_DIR)" "$(MOZ_APP_DISPLAYNAME)" "$(PACKAGE)"

This change breaks en-US release builds by leaving the cab in the wrong place for uploading, see bug 535019 comment 6. The problem is that $(PACKAGE) includes $(PKG_PATH), which is set for the release case. 

Do you recall why this change was made ?

Comment 23

[Justin Dolske [:Dolske]]

Yeah, because this code path is only run once, and generates both ZIP and CAB in one shot. So $(PACKAGE) here is ".../mumble.zip", whereas we want ".../mumble.cab"

I suppose the alternative would be to so something like $(PACKAGE:.zip=.cab)?, or $(SOMETHING_FULL_PATH)$(PKG_BASENAME)?

Comment 24

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Fix release case

I think this should work because $(PKG_PATH) is empty except in the release case. It's fragile w.r.t 
  PACKAGE       = $(PKG_PATH)$(PKG_BASENAME)$(PKG_SUFFIX)
in toolkit/mozapps/installer/packager.mk changing. Perhaps a sed -e 's/zip$/cab/' on $(PACKAGE) instead ?

Comment 25

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 418776 [details] [diff] [review]
Fix release case

Ted, this will unbreak the location release cab's are placed, so we get them uploaded automatically. We can work around this for 3.6rc1 if it's too late but will need it to make localised releases work (coming as soon as dolske and armen get that figured out). Please see comment #24 for more comments.

I'd like to ask for approval1.9.2 but that seems to be disabled.

Comment 26

[Mike Beltzner [:beltzner, not reading bugmail]]

Comment on attachment 418776 [details] [diff] [review]
Fix release case

a192=beltzner

Comment 27

[Robert O'Callahan (:roc) (email my personal email if necessary)]

http://hg.mozilla.org/releases/mozilla-1.9.2/rev/602777a62abb

Comment 28

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 418776 [details] [diff] [review]
Fix release case

http://hg.mozilla.org/mozilla-central/rev/0592043479c3