923950 - Beta & Nightly Android l10n repacks broken, "IOError: [Errno 2] No such file or directory: '/builds/slave/m-cen-and-l10n_1-0000000000000/build/mozilla-central/obj-l10n/dist/l10n-stage/fennec/res/drawable-xlarge-xhdpi-v11/ic_menu_bookmark_remove.png'

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Phil Ringnalda (:philor)]

Turns out I was wrong about it being the same thing as bug 920637, and single-locale repacks remain broken:

https://tbpl.mozilla.org/php/getParsedLog.php?id=28785473&tree=Mozilla-Central

04:23:46     INFO -  /builds/slave/m-cen-and-l10n_1-0000000000000/build/mozilla-central/obj-l10n/_virtualenv/bin/python /builds/slave/m-cen-and-l10n_1-0000000000000/build/mozilla-central/toolkit/mozapps/installer/l10n-repack.py /builds/slave/m-cen-and-l10n_1-0000000000000/build/mozilla-central/obj-l10n/mobile/android/locales/../../../dist/l10n-stage/fennec ../../../dist/xpi-stage/locale-ar \
04:23:50    ERROR -  Traceback (most recent call last):
04:23:50     INFO -    File "/builds/slave/m-cen-and-l10n_1-0000000000000/build/mozilla-central/toolkit/mozapps/installer/l10n-repack.py", line 48, in <module>
04:23:50     INFO -      main()
04:23:50     INFO -    File "/builds/slave/m-cen-and-l10n_1-0000000000000/build/mozilla-central/toolkit/mozapps/installer/l10n-repack.py", line 44, in main
04:23:50     INFO -      l10n.repack(args.build, args.l10n, args.non_resource, NON_CHROME)
04:23:50     INFO -    File "/builds/slave/m-cen-and-l10n_1-0000000000000/build/mozilla-central/python/mozbuild/mozpack/packager/l10n.py", line 173, in repack
04:23:50     INFO -      copier.copy(source, skip_if_older=False)
04:23:50     INFO -    File "/builds/slave/m-cen-and-l10n_1-0000000000000/build/mozilla-central/python/mozbuild/mozpack/copier.py", line 293, in copy
04:23:50     INFO -      if f.copy(destfile, skip_if_older):
04:23:50     INFO -    File "/builds/slave/m-cen-and-l10n_1-0000000000000/build/mozilla-central/python/mozbuild/mozpack/copier.py", line 438, in copy
04:23:50     INFO -      file.copy(deflater, skip_if_older)
04:23:50     INFO -    File "/builds/slave/m-cen-and-l10n_1-0000000000000/build/mozilla-central/python/mozbuild/mozpack/files.py", line 105, in copy
04:23:50     INFO -      shutil.copyfileobj(self.open(), dest)
04:23:50     INFO -    File "/builds/slave/m-cen-and-l10n_1-0000000000000/build/mozilla-central/python/mozbuild/mozpack/files.py", line 133, in open
04:23:50     INFO -      return open(self.path, 'rb')
04:23:50     INFO -  IOError: [Errno 2] No such file or directory: '/builds/slave/m-cen-and-l10n_1-0000000000000/build/mozilla-central/obj-l10n/mobile/android/locales/../../../dist/l10n-stage/fennec/res/drawable-xxhdpi/icon.png'
04:23:50    ERROR -  make[1]: *** [repackage-zip] Error 1

Comment 1

[Gregory Szorc [:gps]]

For reference, a log from a working job:

https://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/mozilla-central-l10n/mozilla-central-android-l10n_3-unknown-bm57-build1-build3.txt.gz

Comment 2

[Gregory Szorc [:gps]]

Regression range is 1fda74e33e06:68d279364a8b.

Good: https://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/mozilla-central-l10n/mozilla-central-android-l10n_5-unknown-bm64-build1-build34.txt.gz
Bad: https://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/mozilla-central-l10n/mozilla-central-android-l10n_3-unknown-bm57-build1-build3.txt.gz

Bugs touching mobile/fennec in that range that might be relevant:

815202, 906039

Other possible bugs:

918392

I /really/ wish we had the ability to test l10n repacks via try.

Any volunteers to look into this deeper?

Comment 3

[Nick Alexander :nalexander [he/him]]

I strongly suspect that this is a race.  We have special (shitty) rules for copying the drawable-*/icon.png files into $(OBJDIR)/m/a/b/res, and my guess is they're not in place for l10n-repacks.  I have WIP-code to install these resources using manifests, which should eliminate that class of bug, but that won't be soon enough for this bug.

I'll look into this.

Comment 5

[Axel Hecht [:Pike]]

I've reproduced this locally, and I don't think there's a race involved.

All files are there in l10n-stage/fennec/res/ during the repack, up to the point where I'm actually calling into the python repack code:

/src/central/mozilla-central/obj-android/_virtualenv/bin/python /src/central/mozilla-central/toolkit/mozapps/installer/l10n-repack.py /src/central/mozilla-central/obj-android/dist/l10n-stage/fennec ../../../dist/xpi-stage/locale-de

That in some way cleans out the files in res/*, which I haven't dug up yet.

step by step instructions to reproduce:

make wget-en-US EN_US_BINARY_URL=ftp://ftp.mozilla.org/pub/mobile/nightly/latest-mozilla-central-android-x86/en-US/

make merge-de LOCALE_MERGEDIR=$PWD/merge-de

make clobber-stage
make unpack
make libs-de LOCALE_MERGEDIR=$PWD/merge-de  # files in res are still there


kochbuch:locales ahecht$ /src/central/mozilla-central/obj-android/_virtualenv/bin/python /src/central/mozilla-central/toolkit/mozapps/installer/l10n-repack.py /src/central/mozilla-central/obj-android/dist/l10n-stage/fennec ../../../dist/xpi-stage/locale-de
Traceback (most recent call last):
  File "/src/central/mozilla-central/toolkit/mozapps/installer/l10n-repack.py", line 48, in <module>
    main()
  File "/src/central/mozilla-central/toolkit/mozapps/installer/l10n-repack.py", line 44, in main
    l10n.repack(args.build, args.l10n, args.non_resource, NON_CHROME)
  File "/src/central/mozilla-central/python/mozbuild/mozpack/packager/l10n.py", line 173, in repack
    copier.copy(source, skip_if_older=False)
  File "/src/central/mozilla-central/python/mozbuild/mozpack/copier.py", line 293, in copy
    if f.copy(destfile, skip_if_older):
  File "/src/central/mozilla-central/python/mozbuild/mozpack/copier.py", line 438, in copy
    file.copy(deflater, skip_if_older)
  File "/src/central/mozilla-central/python/mozbuild/mozpack/files.py", line 105, in copy
    shutil.copyfileobj(self.open(), dest)
  File "/src/central/mozilla-central/python/mozbuild/mozpack/files.py", line 133, in open
    return open(self.path, 'rb')
IOError: [Errno 2] No such file or directory: '/src/central/mozilla-central/obj-android/dist/l10n-stage/fennec/res/anim/grow_fade_in.xml'

Comment 6

[Nick Alexander :nalexander [he/him]]

(In reply to Axel Hecht [:Pike] from comment #5)
> I've reproduced this locally, and I don't think there's a race involved.

Axel, thanks for this valuable leg work.  I'm quite confident we're hitting the "delete res/*" path through m/a/b/Makefile.in. (http://hg.mozilla.org/mozilla-central/annotate/a41ce04d632d/mobile/android/base/Makefile.in#l207)

This part of the build is a mess.  My plan for fixing it is to get rid of this delete path altogether.

First, we separate Android resources into types: static resources in m/a/b/resources, and generated resources in $OBJDIR/m/a/b/res.  Second, we get rid of the delete path for $OBJDIR/m/a/b/res.  There are tradeoffs to this approach that I will enumerate in Bug 934646.

Comment 7

[Axel Hecht [:Pike]]

Shouldn't that also nuke the directories?

kochbuch:dist ahecht$ find l10n-stage/fennec/res
l10n-stage/fennec/res
l10n-stage/fennec/res/anim
l10n-stage/fennec/res/color
l10n-stage/fennec/res/drawable
l10n-stage/fennec/res/drawable-hdpi
l10n-stage/fennec/res/drawable-hdpi-v11
l10n-stage/fennec/res/drawable-large-hdpi-v11
l10n-stage/fennec/res/drawable-large-land-v11
l10n-stage/fennec/res/drawable-large-mdpi-v11
l10n-stage/fennec/res/drawable-large-xhdpi-v11
l10n-stage/fennec/res/drawable-mdpi
l10n-stage/fennec/res/drawable-mdpi-v11
l10n-stage/fennec/res/drawable-xhdpi
l10n-stage/fennec/res/drawable-xhdpi-v11
l10n-stage/fennec/res/drawable-xlarge-hdpi-v11
l10n-stage/fennec/res/drawable-xlarge-mdpi-v11
l10n-stage/fennec/res/drawable-xlarge-v11
l10n-stage/fennec/res/drawable-xlarge-xhdpi-v11
l10n-stage/fennec/res/drawable-xxhdpi
l10n-stage/fennec/res/layout
l10n-stage/fennec/res/layout-large-land-v11
l10n-stage/fennec/res/layout-large-v11
l10n-stage/fennec/res/layout-xlarge-v11
l10n-stage/fennec/res/menu
l10n-stage/fennec/res/menu-large-v11
l10n-stage/fennec/res/menu-v11
l10n-stage/fennec/res/menu-xlarge-v11
l10n-stage/fennec/res/xml
l10n-stage/fennec/res/xml-v11

Comment 8

[Alex Keybl [:akeybl]]

Tracking for FF28, but we'd untrack as of Beta since single locale is only enabled for Nightly/Aurora. Hopefully this will be resolved long before.

Comment 9

[Aki Sasaki (not active)]

We do fail in nightlies/aurora:
https://tbpl.mozilla.org/php/getParsedLog.php?id=31740198&tree=Mozilla-Central&full=1#error0

The script hits the make error 2 during make installer-%s, and skips the create update snippets and upload steps.

Comment 10

[Nick Alexander :nalexander [he/him]]

Okay, Bug 934646 improves this (?) and I can reproduce it locally, following the instructions at https://wiki.mozilla.org/Mobile/Fennec/Android#Single-locale_language_repacks.  Digging in.

Comment 11

[Nick Alexander :nalexander [he/him]]

Attachment: Fix Android single locale repacks.

There are two parts to this.  The first is to add AndroidManifest.xml
as a dependency to the "no dependencies" ap_ built during packaging.
The aapt call requires it.  So "no dependencies" is more accurately
"no *resource* dependencies".

The second is to avoid including the Android res/ directory in the
language repack step.  What happens is that the l10n.py script sees
the Android res/ files left in the dist/ directory after unpacking and
expects to find them in the objdir.  They're not there, so the script
fails.  To avoid this, we delete them after unpacking.  See the
comments in packager.mk describing this process.

Comment 12

[Nick Alexander :nalexander [he/him]]

Comment on attachment 8347671 [details] [diff] [review]
Fix Android single locale repacks.

Builds and single-locale repacks are working well for me locally.  I'll make sure this doesn't foul up multi-locale builds, and that the resulting APKs actually have the correct languages, before landing.

Comment 13

[Nick Alexander :nalexander [he/him]]

(In reply to Nick Alexander :nalexander from comment #12)
> Comment on attachment 8347671 [details] [diff] [review]
> Fix Android single locale repacks.
> 
> Builds and single-locale repacks are working well for me locally.  I'll make
> sure this doesn't foul up multi-locale builds, and that the resulting APKs
> actually have the correct languages, before landing.

multi-locale builds are working locally, and I can switch locales (using rnewman's locale-switcher add-on from http://160.twinql.com/building-and-testing-multi-locale-firefox-for-android/).

Comment 14

[Nick Alexander :nalexander [he/him]]

(In reply to Nick Alexander :nalexander from comment #12)
> Comment on attachment 8347671 [details] [diff] [review]
> Fix Android single locale repacks.
> 
> Builds and single-locale repacks are working well for me locally.  I'll make
> sure this doesn't foul up multi-locale builds, and that the resulting APKs
> actually have the correct languages, before landing.

single-locale repacks have the correct languages locally.

Comment 15

[Jeff Beatty [:gueroJeff]]

(In reply to Nick Alexander :nalexander from comment #14)
> (In reply to Nick Alexander :nalexander from comment #12)
> > Comment on attachment 8347671 [details] [diff] [review]
> > Fix Android single locale repacks.
> > 
> > Builds and single-locale repacks are working well for me locally.  I'll make
> > sure this doesn't foul up multi-locale builds, and that the resulting APKs
> > actually have the correct languages, before landing.
> 
> single-locale repacks have the correct languages locally.

I'm thrilled to hear that this is working locally! How soon might we be able to see this fix implemented? Before the holiday break?

Comment 16

[Nick Alexander :nalexander [he/him]]

> I'm thrilled to hear that this is working locally! How soon might we be able
> to see this fix implemented? Before the holiday break?

Hi Jeff, with a little luck glandium will agree that my solution is reasonable and we should see how it fairs on TBPL in a day or two.

Comment 17

[Jeff Beatty [:gueroJeff]]

(In reply to Nick Alexander :nalexander from comment #16)
> > I'm thrilled to hear that this is working locally! How soon might we be able
> > to see this fix implemented? Before the holiday break?
> 
> Hi Jeff, with a little luck glandium will agree that my solution is
> reasonable and we should see how it fairs on TBPL in a day or two.

Great news! Thank you for your work here Nick!

Comment 18

[Nick Alexander :nalexander [he/him]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/db5ef6c37d74

Comment 19

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/db5ef6c37d74

Comment 20

[Rail Aliiev [:rail]]

Reopening since it blocks the next Fennec release (27.0) and the patch should be uplifted to aurora and beta.

Comment 21

[Phil Ringnalda (:philor)]

Well, once we have a working patch. Right now we have 

04:09:35     INFO -  /tools/android-sdk-r16/platforms/android-16/../..//platform-tools/aapt package -f -M AndroidManifest.xml -I /tools/android-sdk-r16/platforms/android-16/android.jar --auto-add-overlay -S /builds/slave/m-cen-and-l10n_1-0000000000000/build/mozilla-central/mobile/android/base/crashreporter/res -S /builds/slave/m-cen-and-l10n_1-0000000000000/build/mozilla-central/mobile/android/base/resources -S /builds/slave/m-cen-and-l10n_1-0000000000000/build/mozilla-central/mobile/android/branding/nightly/res -S /builds/slave/m-cen-and-l10n_1-0000000000000/build/mozilla-central/obj-l10n/mobile/android/base/res --custom-package org.mozilla.gecko --non-constant-id -F gecko-nodeps.ap_ -J gecko-nodeps/ --output-text-symbols gecko-nodeps/
04:09:35     INFO -  /tools/android-sdk-r16/platforms/android-16/../..//platform-tools/aapt: /lib/libz.so.1: no version information available (required by /tools/android-sdk-r16/platforms/android-16/../..//platform-tools/aapt)
04:09:35     INFO -  ERROR: Unknown option '--output-text-symbols'

and they're still red.

Comment 22

[Nick Alexander :nalexander [he/him]]

(In reply to Phil Ringnalda (:philor) from comment #21)
> Well, once we have a working patch. Right now we have 
> 
> 04:09:35     INFO - 
> /tools/android-sdk-r16/platforms/android-16/../..//platform-tools/aapt
> package -f -M AndroidManifest.xml -I
> /tools/android-sdk-r16/platforms/android-16/android.jar --auto-add-overlay
> -S
> /builds/slave/m-cen-and-l10n_1-0000000000000/build/mozilla-central/mobile/
> android/base/crashreporter/res -S
> /builds/slave/m-cen-and-l10n_1-0000000000000/build/mozilla-central/mobile/
> android/base/resources -S
> /builds/slave/m-cen-and-l10n_1-0000000000000/build/mozilla-central/mobile/
> android/branding/nightly/res -S
> /builds/slave/m-cen-and-l10n_1-0000000000000/build/mozilla-central/obj-l10n/
> mobile/android/base/res --custom-package org.mozilla.gecko --non-constant-id
> -F gecko-nodeps.ap_ -J gecko-nodeps/ --output-text-symbols gecko-nodeps/
> 04:09:35     INFO - 
> /tools/android-sdk-r16/platforms/android-16/../..//platform-tools/aapt:
> /lib/libz.so.1: no version information available (required by
> /tools/android-sdk-r16/platforms/android-16/../..//platform-tools/aapt)
> 04:09:35     INFO -  ERROR: Unknown option '--output-text-symbols'
> 
> and they're still red.

So, this is interesting.  Filed Bug 951238 to get to the bottom of this apparent version mismatch.

Comment 23

[hwine]

This is now affecting 27.0b as well. Summary updated

Comment 24

[bhavana bajaj [:bajaj]]

As discussed in the channel meeting that this was an existing bug in earlier beta and not Firefox27 specific, so removing tracking. Happy to uplift any patches if it helps on aurora/beta though

Comment 25

[Aki Sasaki (not active)]

(In reply to bhavana bajaj [:bajaj] from comment #24)
> As discussed in the channel meeting that this was an existing bug in earlier
> beta and not Firefox27 specific, so removing tracking. Happy to uplift any
> patches if it helps on aurora/beta though

This will prevent us from shipping to the T-Store once fx27 hits release.

Comment 26

[bhavana bajaj [:bajaj]]

(In reply to Aki Sasaki [:aki] from comment #25)
> (In reply to bhavana bajaj [:bajaj] from comment #24)
> > As discussed in the channel meeting that this was an existing bug in earlier
> > beta and not Firefox27 specific, so removing tracking. Happy to uplift any
> > patches if it helps on aurora/beta though
> 
> This will prevent us from shipping to the T-Store once fx27 hits release.

Hmm, as per the channel meeting discussion this existed throughtout the course of Fx26 as well, so wonder what differently did we do there ?

 If this is bug starting firefox 27 and prevents us from shipping to T-store, No on nick to help with next steps here ?

Comment 27

[Aki Sasaki (not active)]

(In reply to bhavana bajaj [:bajaj] from comment #26)
> (In reply to Aki Sasaki [:aki] from comment #25)
> > (In reply to bhavana bajaj [:bajaj] from comment #24)
> > > As discussed in the channel meeting that this was an existing bug in earlier
> > > beta and not Firefox27 specific, so removing tracking. Happy to uplift any
> > > patches if it helps on aurora/beta though
> > 
> > This will prevent us from shipping to the T-Store once fx27 hits release.
> 
> Hmm, as per the channel meeting discussion this existed throughtout the
> course of Fx26 as well, so wonder what differently did we do there ?

Looking at the Fx26 beta+release candidates directories, we did have working android l10n repacks:

http://ftp.mozilla.org/pub/mozilla.org/mobile/candidates/26.0b1-candidates/build1/android/
http://ftp.mozilla.org/pub/mozilla.org/mobile/candidates/26.0-candidates/build2/android/

This means this started breaking in Fx27.

Comment 28

[Nick Thomas [:nthomas] (UTC+12)]

Latest log for failing on 27.0b6:
http://ftp.mozilla.org/pub/mozilla.org/mobile/candidates/27.0b6-candidates/build1/logs/release-mozilla-beta-android_repack_1-bm65-build1-build4.txt.gz

23:10:24     INFO -  IOError: [Errno 2] No such file or directory: '/builds/slave/rel-m-beta-and_rpk_1-000000000/build/mozilla-beta/obj-l10n/dist/l10n-stage/fennec/res/drawable-xxhdpi/icon.png'
23:10:24    ERROR -  make[1]: *** [repackage-zip] Error 1
23:10:24     INFO -  make[1]: Leaving directory `/builds/slave/rel-m-beta-and_rpk_1-000000000/build/mozilla-beta/obj-l10n/mobile/android/locales'
23:10:24     INFO -  make: *** [repackage-zip-cs] Error 2

Comment 29

[Nick Alexander :nalexander [he/him]]

Per discussion with aki: Nightly l10n repacks appear to be working; Aurora broken; Beta broken.

nalexander's initial belief about state of world: Nightly has complete fix and Nightly build environment is correct; Aurora has incomplete fix and unknown build environment; Beta has no fixes at all and unknown build environment.

nalexander to dig into beta l10n repack code locally and update ticket by EOD.

Comment 30

[Nick Alexander :nalexander [he/him]]

Attachment: 923950.beta.patch

[Approval Request Comment]
Bug caused by (feature/regressing bug #): 

I suppose you could say this was caused by Bug 951238 and Bug 934646, but in reality l10n single locale repacks were broken before those tickets landed because of general build system problems.

User impact if declined:

This ticket blocks pushing Firefox (Beta) to some online marketplaces.

Testing completed (on m-c, etc.): single locale (de) repack built locally; installed and browsed fine.

This testing followed the steps that I documented at https://wiki.mozilla.org/Mobile/Fennec/Android#Single-locale_language_repacks.

It's not possible to test this any other way than to land it and hope.

Risk to taking this patch (and alternatives if risky):

It can't be any worse than it is right now.

String or IDL/UUID changes made by this patch: none.

Comment 31

[Nick Alexander :nalexander [he/him]]

(In reply to Nick Alexander :nalexander from comment #29)

> nalexander's initial belief about state of world: ... Beta has no fixes at all and unknown build
> environment.

I do not expect the build environment to be an issue on Beta, since the Beta code base (and proposed fix) does not use the aapt flags that caused Bug 951238.

Comment 32

[Nick Alexander :nalexander [he/him]]

I think the right thing to do here is to uplift the five commits landed by Bugs 934646 and Bug 923950 to Aurora.

The five commits apply cleanly (except for CLOBBER, which is expected to conflict).  Locally, I can build, package, and do a single locale repack.  (I succeeded with cs but failed with de and eu; it appears those languages just don't have recent mobile/android strings.  I thought compare-locales made sure missing strings didn't happen, but I'm seeing it.  In any case, one successful language is better than none.)

I anticipate that uplifting these commits will uncover build environment issues from Bug 951238, since the Aurora build process is different than the Central process; but this seems the correct way forward.

For the record, I did see if the Beta fix I propose just works on Aurora.  It does not, because AndroidManifest.xml is missing from the unpacked repository, and the mobile/android/base/Makefile.in deps don't create it properly.  It's a good deal of effort to make that work; uplifting and addressing fallout is better.

bajaj: I don't have time to attach patches right now.  They are:

https://hg.mozilla.org/mozilla-central/rev/9b0d76fafd52
https://hg.mozilla.org/mozilla-central/rev/710003215a5c
https://hg.mozilla.org/mozilla-central/rev/8490d45f1525
https://hg.mozilla.org/mozilla-central/rev/9be250f7458f
https://hg.mozilla.org/mozilla-central/rev/db5ef6c37d74

approval-mozilla-aurora?

Comment 33

[Nick Alexander :nalexander [he/him]]

glandium: you suggested I could test the patch for beta on birch.  But I don't see N1 jobs at https://tbpl.mozilla.org/?tree=Birch&showall=1 (I do see N).  And I don't know how to configure this to build the beta source code, with beta branding, or how to get the repacks to fetch l10n resources from the beta l10n repositories.  If you can give some instruction, that would help.

Comment 34

[Mike Hommey [:glandium]]

Yeah that probably wouldn't work :(

Comment 35

[Mike Hommey [:glandium]]

Comment on attachment 8359975 [details] [diff] [review]
923950.beta.patch

Review of attachment 8359975 [details] [diff] [review]:
-----------------------------------------------------------------

::: toolkit/mozapps/installer/packager.mk
@@ +453,5 @@
>    ( cd $(MOZ_PKG_DIR) && \
>      $(UNZIP) $(UNPACKAGE) && \
>      mv lib/$(ABI_DIR)/libmozglue.so . && \
>      mv lib/$(ABI_DIR)/*plugin-container* $(MOZ_CHILD_PROCESS_NAME) && \
> +    rm -rf lib/$(ABI_DIR) && \

btw, this && is missing on central.

Comment 36

[Nick Alexander :nalexander [he/him]]

(In reply to Mike Hommey [:glandium] from comment #35)
> Comment on attachment 8359975 [details] [diff] [review]
> 923950.beta.patch
> 
> Review of attachment 8359975 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: toolkit/mozapps/installer/packager.mk
> @@ +453,5 @@
> >    ( cd $(MOZ_PKG_DIR) && \
> >      $(UNZIP) $(UNPACKAGE) && \
> >      mv lib/$(ABI_DIR)/libmozglue.so . && \
> >      mv lib/$(ABI_DIR)/*plugin-container* $(MOZ_CHILD_PROCESS_NAME) && \
> > +    rm -rf lib/$(ABI_DIR) && \
> 
> btw, this && is missing on central.

Yeah; I noticed right after I landed the original.  I'll fix it some time when I'm in the area; by random chance it works because the command gets interpreted as "rm -rf first_thing rm -rf second_thing".

Comment 37

[bhavana bajaj [:bajaj]]

(In reply to Nick Alexander :nalexander from comment #32)
> I think the right thing to do here is to uplift the five commits landed by
> Bugs 934646 and Bug 923950 to Aurora.
> 
> The five commits apply cleanly (except for CLOBBER, which is expected to
> conflict).  Locally, I can build, package, and do a single locale repack. 
> (I succeeded with cs but failed with de and eu; it appears those languages
> just don't have recent mobile/android strings.  I thought compare-locales
> made sure missing strings didn't happen, but I'm seeing it.  In any case,
> one successful language is better than none.)
> 
> I anticipate that uplifting these commits will uncover build environment
> issues from Bug 951238, since the Aurora build process is different than the
> Central process; but this seems the correct way forward.
> 
> For the record, I did see if the Beta fix I propose just works on Aurora. 
> It does not, because AndroidManifest.xml is missing from the unpacked
> repository, and the mobile/android/base/Makefile.in deps don't create it
> properly.  It's a good deal of effort to make that work; uplifting and
> addressing fallout is better.
> 
> bajaj: I don't have time to attach patches right now.  They are:
> 
> https://hg.mozilla.org/mozilla-central/rev/9b0d76fafd52
> https://hg.mozilla.org/mozilla-central/rev/710003215a5c
> https://hg.mozilla.org/mozilla-central/rev/8490d45f1525
> https://hg.mozilla.org/mozilla-central/rev/9be250f7458f
> https://hg.mozilla.org/mozilla-central/rev/db5ef6c37d74
> 
> approval-mozilla-aurora?

Discussed the risk offline with nick, we need these in any case for the repacks to be built correctly hence approving .Please land with a=bajaj

Comment 38

[Nick Alexander :nalexander [he/him]]

https://hg.mozilla.org/releases/mozilla-beta/rev/4cc7086ff425

Comment 39

[Nick Alexander :nalexander [he/him]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/5b3c27d7ce04
https://hg.mozilla.org/releases/mozilla-aurora/rev/6dc17ae36efa
https://hg.mozilla.org/releases/mozilla-aurora/rev/d51b8b51e6bc
https://hg.mozilla.org/releases/mozilla-aurora/rev/ef058ebed6fb
https://hg.mozilla.org/releases/mozilla-aurora/rev/ea80006dd4f9

Comment 40

[Nick Alexander :nalexander [he/him]]

(In reply to Nick Alexander :nalexander from comment #39)
> https://hg.mozilla.org/releases/mozilla-aurora/rev/5b3c27d7ce04
> https://hg.mozilla.org/releases/mozilla-aurora/rev/6dc17ae36efa
> https://hg.mozilla.org/releases/mozilla-aurora/rev/d51b8b51e6bc
> https://hg.mozilla.org/releases/mozilla-aurora/rev/ef058ebed6fb
> https://hg.mozilla.org/releases/mozilla-aurora/rev/ea80006dd4f9

It appears that http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/latest-mozilla-aurora-android-l10n/ is populated with builds from January 16th!  Huzzah!

Jeff: can we get QA to smoketest these, please -- just make sure one can install, is in fact translated, and can render a page?

Comment 41

[Jeff Beatty [:gueroJeff]]

(In reply to Nick Alexander :nalexander from comment #40)
> (In reply to Nick Alexander :nalexander from comment #39)
> > https://hg.mozilla.org/releases/mozilla-aurora/rev/5b3c27d7ce04
> > https://hg.mozilla.org/releases/mozilla-aurora/rev/6dc17ae36efa
> > https://hg.mozilla.org/releases/mozilla-aurora/rev/d51b8b51e6bc
> > https://hg.mozilla.org/releases/mozilla-aurora/rev/ef058ebed6fb
> > https://hg.mozilla.org/releases/mozilla-aurora/rev/ea80006dd4f9
> 
> It appears that
> http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/latest-mozilla-aurora-
> android-l10n/ is populated with builds from January 16th!  Huzzah!
> 
> Jeff: can we get QA to smoketest these, please -- just make sure one can
> install, is in fact translated, and can render a page?

I just downloaded and installed the pt-BR apk on my Samsung Tab 10.1 and it clears a basic smoke test.

Comment 42

[Julen Ruiz Aizpuru]

Tried an 'eu' build successfully. The UI is localized and missing entities fallback properly to English.

Comment 43

[Nick Alexander :nalexander [he/him]]

So, I'm seeing APKs in places like

http://ftp.mozilla.org/pub/mozilla.org/mobile/candidates/27.0b8-candidates/build1/android/ar/

Can we get l10n QA eyes on this to make sure these are looking good enough to close this ticket?

Comment 44

[Aki Sasaki (not active)]

Let's resolve so QA can see it needs action.

Comment 45

[Joshua Mitchell (Inactive)]

Issue is resolved - clearing old keywords - qa-wanted clean-up