Closed Bug 563382 Opened 12 years ago Closed 11 years ago

Android multilocale

Categories

(Release Engineering :: General, defect, P2)

ARM
Android
defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: aki, Assigned: aki)

References

Details

(Whiteboard: [android][l10n])

Attachments

(4 files, 1 obsolete file)

Once we know how we want these repacked and which locales.
Assigning to Stuart for info; throw this back once we know how to repack.
Assignee: nobody → pavlov
We can look at unzipping/rezipping, and creating a multi-locale repack factory -- we don't necessarily need to block on the list of locales.
Assignee: pavlov → aki
Whiteboard: [android][l10n][needinfo] → [android][l10n]
For Q3.
Priority: P5 → P4
Status: NEW → ASSIGNED
Status: ASSIGNED → NEW
Depends on: 594017
Status: NEW → ASSIGNED
Per Stuart, no single locale repacks.
Priority: P4 → P2
Summary: Android l10n repacks + multilocale → Android multilocale
Android repacks should be easy to do now that bug 567873 is fixed. bug 575751 might also be needed but maybe not. I've only tested single locale repacks, however.
Axel: are you aware we're not planning to do single locale repacks? Does that cause issues?
I had no idea.

Not sure if that's raising any problems. The only interesting question I have ad-hoc is the size of the resulting bundle due to all locales being in there.
Currently investigating whether bug 575751 has broken our multilocale hack.
Depends on: 575751
I'm overloading options.builddir, which is currently Try-only.

If that's set in a non-try make upload, use that as a subdirectory in tinderbox-builds, nightly latest/dated dirs, and candidates.

I kind of want to s,builddir,subdir,g but that would require touching all Try uploads, so I'm very much inclined to leave it alone at the moment.

Tested on staging-stage.
Attachment #485151 - Flags: review?(coop)
Comment on attachment 485151 [details] [diff] [review]
post_upload.py subdirs

(In reply to comment #9)
> I kind of want to s,builddir,subdir,g but that would require touching all Try
> uploads, so I'm very much inclined to leave it alone at the moment.

I'm fine with it as-is for now, but file a follow-up?
Attachment #485151 - Flags: review?(coop) → review+
Duplicate of this bug: 527928
This is pretty small, since I've got configs in mozharness and some defaults in the factories.
Attachment #485235 - Flags: review?(coop)
Ok, this one is much less small.

This depends on attachment 485151 [details] [diff] [review] (I hacked running aki_post_upload.py on staging-stage in my test runs).  It also depends on http://hg.mozilla.org/users/asasaki_mozilla.com/mozharness/ default; the patch as is requires that we move the PRODUCTION tag to revision ed05270b708c (default tip as of this writing).

One piece of luck: this will be the first multilocale build in the 0.8.0 branch, so I can't break Maemo multilocale with this specific patch.  (Maemo multilocale builds are all still on the default/07x branch of buildbotcustom).

This patch also fixes the doStepIf=previousApkExists steps.

This is the first successful nightly test:
http://staging-master.build.mozilla.org:8012/builders/Android%20R7%20mozilla-central%20nightly/builds/10

This is my currently-still-running depend build test, which is not multilocale (as designed):
http://staging-master.build.mozilla.org:8012/builders/Android%20R7%20mozilla-central%20build/builds/3

I'm trying to kick off some more Android builds to try to test this further.

Ideally I'm hoping to get this reviewed and landed today, so mobile&qa can look at it over the weekend.  Let me know if you have any questions about this.
Attachment #485238 - Flags: review?(coop)
So that's
http://hg.mozilla.org/users/asasaki_mozilla.com/mozharness/file/ed05270b708c .

As far as potentially landing this in an official location, I was thinking build/tools/mozharness/..., but now I'm wondering if the layout isn't kind of leaning towards being its own repo.

I'm open, and I'm still not completely sold on its current directory layout, either.  It'll take a little work to reorganize it but I still know it intimately enough that I think I can do it without too much trouble.
Once this is reviewed and landed, I need to write a corresponding set of patches to get multilocale Android into the release configs/factories over in 07x land.
The apks generated by the above nightly staging multilocale android build are here:

http://people.mozilla.com/~asasaki/android_multilocale/

Should be signed with the official nightly key.

K, going to try to sleep.
Comment on attachment 485235 [details] [diff] [review]
android m-c multil10n configs

+BRANCHES['mozilla-central']['enable_multi_locale'] = True

looks like it's enabling multi-locale builds for Desktop Firefox? We wouldn't want that.
(In reply to comment #17)
> Comment on attachment 485235 [details] [diff] [review]
> android m-c multil10n configs
> 
> +BRANCHES['mozilla-central']['enable_multi_locale'] = True
> 
> looks like it's enabling multi-locale builds for Desktop Firefox? We wouldn't
> want that.

nope, check misc.py. gotta enable on branch+platform.
(In reply to comment #14)
> As far as potentially landing this in an official location, I was thinking
> build/tools/mozharness/..., but now I'm wondering if the layout isn't kind of
> leaning towards being its own repo.
> 
I vote for own repo please! If we want mozharness to be standalone and have people contribute to it we might want to start it by its own repo. Unless, there are many dependencies with other things inside of tools (maybe move some of these pieces into mozharness).
You know this better so your call! (just giving my opinion just in case :D)
No dependencies inside of tools, just wanted to limit the number of repos we had to check out to do a build. But I'd be more than fine with build/mozharness.
One thing I've noticed in later nightlies on sm:8012 -- downloading the previous apk for en-US looks in latest-mozilla-central-android-r7/, rather than latest-mozilla-central-android-r7/en-US/.

I'm going to fix that; should hopefully be a small change.
Comment on attachment 485238 [details] [diff] [review]
factory changes for android multil10n

>diff --git a/process/factory.py b/process/factory.py
>+                 multiLocale=False,
>+                 mozharnessRepoPath="users/asasaki_mozilla.com/mozharness",
>+                 mozharnessRevision="PRODUCTION",

Don't block on this, but since we *are* using mozharness for mobile releases now (and Armen plans to work with his students on it too), let's get mozharness into a proper build repo soon.

>+        self.multiLocale = multiLocale
>+        if multiLocale:
>+            assert mozharnessConfig
>+            self.mozharnessRepoPath = mozharnessRepoPath
>+            self.mozharnessRevision = mozharnessRevision
>+            self.mozharnessConfig = mozharnessConfig
>+            self.mozharnessRepository = self.getRepository(mozharnessRepoPath)
>+            self.mozharnessBranchName = self.getRepoName(self.mozharnessRepository)
>+            self.mergeLocales = mergeLocales
>+

Should we assert on more of these vars?

>-            command=['ls', 'previous.apk'],
>+            command='test -s previous.apk && ls previous.apk',

I like the explicit test here.

I have no way to verify the resulting build, but the logs from your staging run look OK.
Attachment #485238 - Flags: review?(coop) → review+
Comment on attachment 485235 [details] [diff] [review]
android m-c multil10n configs

*stamp*
Attachment #485235 - Flags: review?(coop) → review+
Awesome.

(In reply to comment #22)
> Don't block on this, but since we *are* using mozharness for mobile releases
> now (and Armen plans to work with his students on it too), let's get mozharness
> into a proper build repo soon.

Agreed.  I'll use bug 574473 for that, and file an IT bug to create a build/mozharness repo.

> >+        self.multiLocale = multiLocale
> >+        if multiLocale:
> >+            assert mozharnessConfig
> >+            self.mozharnessRepoPath = mozharnessRepoPath
> >+            self.mozharnessRevision = mozharnessRevision
> >+            self.mozharnessConfig = mozharnessConfig
> >+            self.mozharnessRepository = self.getRepository(mozharnessRepoPath)
> >+            self.mozharnessBranchName = self.getRepoName(self.mozharnessRepository)
> >+            self.mergeLocales = mergeLocales
> >+
> 
> Should we assert on more of these vars?

The others have defaults specified, so they'll always be set.
If we decide to force config.py entries by not specifying useful defaults, we should add to the assert command.

> >-            command=['ls', 'previous.apk'],
> >+            command='test -s previous.apk && ls previous.apk',
> 
> I like the explicit test here.

Yeah, this was specifically to fix the fact that |wget -O previous.apk URL| was creating a size-0 previous.apk when it got a 404.

> I have no way to verify the resulting build, but the logs from your staging run
> look OK.

Thanks Coop!  I'll try to land this ASAP.

(In reply to comment #10)
> > I kind of want to s,builddir,subdir,g but that would require touching all Try
> > uploads, so I'm very much inclined to leave it alone at the moment.
> 
> I'm fine with it as-is for now, but file a follow-up?

Sure. I'll be touching post_upload.py over the weekend for other mobile upload bugs, so I'll either handle that then, or file a followup bug.

(In reply to comment #21)
> One thing I've noticed in later nightlies on sm:8012 -- downloading the
> previous apk for en-US looks in latest-mozilla-central-android-r7/, rather than
> latest-mozilla-central-android-r7/en-US/.
> 
> I'm going to fix that; should hopefully be a small change.

I'm going to cut my losses and file a followup.
The workaround for the weekend will be to |ln -s en-US/fennec*.apk .| inside of latest-mozilla-central-android-r7/ til I get this fixed.
Attached patch fixes to android multilocale (obsolete) — Splinter Review
Running this in staging.
This patch:
1) fixes make package-tests (oops)
2) should hopefully fix the en-US previous apk download for multilocale builds.
Attachment #485480 - Flags: review?(coop) → review+
Comment on attachment 485480 [details] [diff] [review]
same thing, but %(completeMarFilename)s instead of %%(completeMarFilename)s

http://hg.mozilla.org/build/buildbotcustom/rev/370bc1182899
Attachment #485480 - Flags: checked-in+
Status: ASSIGNED → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.