490482 - Create deb packages for fennec l10n builds

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: General, defect)

Description

[Axel Hecht [:Pike]]

There are some strings inside the deb that we need to expose to l10n and then create deb packages out of those for l10n builds.

This likely involves adding some to fennec's defines.inc, and then to use those in preprocessing some generic deb .in files and use those for the packaging.

Comment 1

[Axel Hecht [:Pike]]

Mark, you wanted to CC someone here that could help, right?

Comment 2

[Mark Finkle (:mfinkle) (use needinfo?)]

Brad, Antonio - We are interested in the parts of the .deb file system that can be localized. What's the specification for those files?

Comment 3

[Mark Finkle (:mfinkle) (use needinfo?)]

http://standards.freedesktop.org/desktop-entry-spec/latest/ar01s04.html

seems to be the spec for localizing the desktop stuff:

http://mxr.mozilla.org/mobile-browser/source/installer/debian/fennec.desktop.in

Comment 4

[Aki Sasaki (not active)]

Coop's going to give this a shot.
Thanks Coop!

Comment 5

[Axel Hecht [:Pike]]

There are two breeds of l10n strings that I found in the deb files.

Firstly, Fennec-specific strings. We don't really like to localize these, but some locales like to transcribe our product names, and there are languages like khmer that don't have an 'f'. We need to poke legal here again to get a policy worked out, both for Fennec, and for Firefox, too. Seth?

Secondly, Desktop-integration strings like http://mxr.mozilla.org/mobile-browser/source/installer/debian/menu.in. No idea what they're doing in localized desktops on the device.

Comment 6

[Axel Hecht [:Pike]]

I started to dive into this a bit.

I made some progress in terms of getting a deb package for the fennec piece, but I learned a few interesting details.

One of which is that we actually need to create two deb packages, one for fennec, which depends on another one for xulrunner.

Not sure if the dependencies between the two should actually depend on the locale code? Brad?

Comment 7

[Axel Hecht [:Pike]]

Taking, as coop is on vacation anyway, and we're running short on time 'til a localized milestone per our meeting today with the mobile folks.

Comment 8

[Axel Hecht [:Pike]]

Adding Jeremiah, hopefully he can help.

Comment 9

[Jeremiah Foster]

General info on translation in debian is here: http://www.debian.org/international/l10n/po-debconf/README-trans

Debian uses PO files, which is a GNUism. Info on PO files: http://www.gnu.org/software/gettext/manual/html_node/gettext_9.html#Creating

Comment 10

[Axel Hecht [:Pike]]

Attachment: create debs for fennec, not xulrunner yet, wip

This is the patch that produced the debs on http://l10n.mozilla.org/~axel/fennec-deb/. Those require xulrunner (>=1.9.2a1pre-20090729010629) (non-localized, latest on ftp as of this posting.)

I'd love to get feedback on both the debs that's producing (do they work?) and the patch.

Not yet sure if I stick to the installer/packaging.mk. I need to check whether I can actually factor it back in with the packaging rules for the regular builds. If that works, I'd probably do.

If those work, we need to throw a dice on what to do with xulrunner.

Comment 11

[Jeremiah Foster]

I am willing to test this on my Maemo tablet. 

I can also do some diagnostics with lintian (and perhaps maemian, the maemo version of lintian.)

At some point it would be nice to have a look at the debian directory used to build the binary deb. 

Hope to reply with more info today or tomorrow.

Comment 12

[Axel Hecht [:Pike]]

Attachment: the point I'm giving up

So I'm clueless on how to do this right. Someone with better understanding of what those magic deb scripts actually do needs to take over here.

My principle problem is to get l10n up without breaking en-US, and I just don't know why things go the way they do. The latest mystery is how to get the output of the deb to have a reasonable name. I tried debian/files, but that's just making fun of me.

Comment 13

[Axel Hecht [:Pike]]

Unassigning as I can't bring this home.

Comment 14

[Mike Beltzner [:beltzner, not reading bugmail]]

Axel was saying this needs an owner - Stuart, got any ideas? John O'Duinn says that it's more of a development than release engineering thing.

Comment 15

[Jeremiah Foster]

Why doesn't Mozilla use the debian package building tools from the debian packaging? The debian maintainer of Iceweasel is Eric Dorland <[e-mail]> and Mike Hommey <[e-mail]> maintains Xulrunner-1.9. 

I realize that mozilla uses some special build tools, but I think debian doesn't use those and using the debian tools might be a short cut to building debs for fennec as well.

Comment 16

[Axel Hecht [:Pike]]

Can you give a pointer to those tools?

Comment 17

[Jeremiah Foster]

dpkg-buildpackage is a tool widely used to build packages both in debian and in maemo for example. If you can build fennec and xulrunner with dpkg-buildpackage on debian you have come a long way to getting it ready for maemo. This has the additional advantage of leaving the source code for fennec pristine; you just have to add a debian/ directory with build information and a good deal of that you can most likely just copy from upstream, i.e. iceweasel.

Comment 18

[Axel Hecht [:Pike]]

Brad, did you have a chance to look at this?

In our call today, Stuart said that we want to offer the localized builds via the package manager, that is, we need to get a distinct and descriptive name into Packages.gz, at least.

Comment 19

[Axel Hecht [:Pike]]

Update from #l10n:

If we can use the deb from ftp (1):

in debstage:
ar x path-to-fennec.deb

That creates
 control.tar.gz  data.tar.gz  debian-binary

data.tar.gz needs repackaging, control.tar.gz might need package name tweak,see comment 18

For the data.tar.gz, unpack that to data-stage, and then clobber en-US:
make clobber-zip AB_CD=en-US STAGEDIST=data-stage/usr/local/fennec/

Then sync l10n in, as in 
http://mxr.mozilla.org/mozilla-central/source/toolkit/locales/l10n.mk#133, STAGEDIST should be the same.

Package data-stage back up as data.tar.gz, and 
ar c * ../path-to-l10n.deb

That's probably easier than to try to get the tar.bz2 bits back and forth again.

Open issues remaining are the 

How to get the fennec en-US deb down as part of wget-en-US, package-names.mk doesn't know this one.
How to name the l10n debs, and where to put them in ftp? Possibly a somewhat consistent naming scheme for en-US and l10n?
Do we need to tweak control.tar.gz? That needs input on what the install experience is supposed to be. That has an impact on the ftp structure, too, I guess.

If we want a single repository with all languages to download, we need control tweaked. If we want separate repositories per locale, we don't. Then we'll need a more involved ftp structure, though, and it might be more work to create the Packages.gz file.

Comment 20

[Armen [:armenzg]]

I have been reading https://wiki.ubuntu.com/PackagingGuide/Basic?action=show&redirect=HowToBuildDebianPackagesFromScratch
and I can already see that we are doing things a little different than other people.

I am blocked on any RelEng work towards Fennec Beta 5 with L10n until we get this bug fixed because this seems to be the easiest out of the three possible release deliverables for L10n according to this page: https://wiki.mozilla.org/Mobile/Fennec/Fennec1.0/L10n

Comment 21

[Armen [:armenzg]]

I will hold off any work on this bug until any definite decision is made. Stuart said that he will figure this out sometime today or during this week.

Comment 22

[Axel Hecht [:Pike]]

Attachment: use ar and stuff

Here's what I got going based on what Brad and I talked about.

I'm patching this 80s style, I just add new rules for this. Seems like the safest thing. I'm currently hacking around the fact that we don't have a mono-locale en-US deb by doing a funky rm ??.* ??-??.* which removes ab.jar and .manifest and ab-CD.jar and manifests.

The entry point is deb-de or fr, with LOCALE_MERGEDIR as full path if wanted.

The dependency logic closely matches what I do with STAGEDIST for the regular repacks, partly in toolkit/locales/l10n.mk.

The --exclude=crashreporter.app in the tar move is more for my local testing here on the mac. Which works just fine, funny enough. I also copy over update.locale and updater.ini, not sure if we need those at some point, but better safe than sorry.

Comment 23

[Brad Lassey [:blassey] (use needinfo?)]

Comment on attachment 410274 [details] [diff] [review]
use ar and stuff


>+DEB_PKG_NAME = fennec_$(MOZ_APP_VERSION)_armel.deb
we can't hard code armel here

>+DATASTAGE = $(CURDIR)/data-stage
nit, we try to set all variables before targets, please move this up (with DEB_PKG_NAME)

>+deb-%: AB_CD=$*
>+deb-%: clobber-% langpack-%
nit, is there a reason to not put all the deps on one line?

r- just for the armel thing, fix that and the nit(s) and we should be good to go

Comment 24

[Axel Hecht [:Pike]]

Attachment: update for review comments

Updated, per irc I'm using dpkg-architecture -qDEB_BUILD_ARCH to get armel.

The other comment I had was re AB_CD=$* being on a single line, that's not a dependency but a variable definition, I think those have to be on a single line, and in our code base, they are. So I left that that way.

Comment 25

[Axel Hecht [:Pike]]

http://hg.mozilla.org/mobile-browser/rev/1fbb7f70fddb, marking FIXED.

There need to be bugs on repos, build automation, and fennec.install creation.

Comment 26

[Joel Maher ( :jmaher ) (UTC -8)]

how can I verify this?  Are these builds posted somewhere?

Comment 27

[Axel Hecht [:Pike]]

You can build one yourself :-)

As I mentioned in bug 25, bugs for creating debs automated are needed to get the output on ftp.