933189 - Update builders to Android SDK platform 17

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task)

Description

[Gian-Carlo Pascutto [:gcp]]

The updated webrtc.org code has a new audio backend that's using some new features in Android 4.2/17 to detect the native sampling rate of the device and the minimal audio buffers required:
http://developer.android.com/reference/android/media/AudioManager.html#PROPERTY_OUTPUT_FRAMES_PER_BUFFER
http://developer.android.com/reference/android/media/AudioManager.html#PROPERTY_OUTPUT_SAMPLE_RATE

In order to use this, we will need to update to platform-17, i.e. ANDROID_SDK_VERSION=17 for the builders and mozconfigs.

Comment 1

[Randell Jesup [:jesup] (needinfo me)]

Blocks landing the webrtc.org update, which is almost ready.  Can we deal with this soon?

Ted/blassey: who should take this?

Comment 2

[Gian-Carlo Pascutto [:gcp]]

>Blocks landing the webrtc.org update

We could unblock it by patching out the new code, and add a seperate bug "restore low-latency support" that's blocked on this. If needed.

Comment 3

[Brad Lassey [:blassey] (use needinfo?)]

This would be a release engineering bug. Callek, will you be taking it?

Comment 4

[Brad Lassey [:blassey] (use needinfo?)]

Let's hold off on this for now and just use reflection for the required code (It's only 2 constants and one method)

Comment 5

[Trevor Saunders (:tbsaunde)]

fixing this shouldn't be all that hard, but I'm not sure I have time this week.  What needs to happen is this

1. download the linux x86_64 android sdk.
2. untar it strip out obviously useless garbage and retar it. Saddly I didn't scriptify this for the sdk, but it should more or less be the same as the ndk script in people.mozilla.org/~tsaunders/ndk.sh.txt
3. put that tarball someplace some place on the internet and get someone in releng to add it to tooltool.
4. change the mozconfigs in mobile/android/config/mozconfigs and the tooltool manifests in mobile/android/config/ to have the right check sums and file sizes.

Comment 6

[Mark Finkle (:mfinkle) (use needinfo?)]

Who can we assign this bug to? We need SDK 17 to land the RTL work for Firefox (bug 702845).

Comment 7

[Justin Wood (:Callek)]

I'm looking into whom in releng can take this work. (after being pinged by blassey due to it blocking a locale from landing)

"[11:23:46]	blassey	deadline is trunk becoming 32
[11:24:03]	blassey	which is April 19"

Comment 8

[Justin Wood (:Callek)]

For further info we have:

Android SDK16 in rpm form (+ spec) at
And Android SDK18 for Deb/Ubuntu in the same folder:
http://mxr.mozilla.org/build/source/puppet/modules/packages/manifests/mozilla/

Lastly a rough doc I found with some searching (unsure how relevant) which was created by :coop  https://wiki.mozilla.org/ReleaseEngineering/How_To/Build_An_Android_SDK_rpm


For added context to help whomever works on it:
Android SDK 16 was created in Bug 775720

Comment 9

[Trevor Saunders (:tbsaunde)]

(In reply to Justin Wood (:Callek) from comment #8)
> For further info we have:
> 
> Android SDK16 in rpm form (+ spec) at
> And Android SDK18 for Deb/Ubuntu in the same folder:
> http://mxr.mozilla.org/build/source/puppet/modules/packages/manifests/
> mozilla/
> 
> Lastly a rough doc I found with some searching (unsure how relevant) which
> was created by :coop 
> https://wiki.mozilla.org/ReleaseEngineering/How_To/Build_An_Android_SDK_rpm
> 
> 
> For added context to help whomever works on it:
> Android SDK 16 was created in Bug 775720

We've been using an sdk from tooltool since bug 859984, and while l10n builds didn't initially do the same (iirc)  they appear to today according to https://tbpl.mozilla.org/php/getParsedLog.php?id=35210003&tree=Mozilla-Central

Comment 10

[Trevor Saunders (:tbsaunde)]

I have a bit of time know, so I'm testing a tarball with sdk 17 as I write this.

Comment 11

[Trevor Saunders (:tbsaunde)]

Attachment: teach android.m4 about build tools v 19.0.2

Grabbing the latest android sdk gets me the build tools in $sdkdir/build-tools/19.0.2/ so lets teach android.m4 about that location too

Comment 12

[Trevor Saunders (:tbsaunde)]

jhopkins can you please upload 
people.mozilla.org/~tsaunders/setup.sh.ndk
and
people.mozilla.org/~tsaunders/android-sdk.tar.xz
to tooltool? thanks!

Comment 13

[John Hopkins (:jhopkins)]

android-sdk.tar.xz 69c9be39a452c202ebe99fa3a7f31cb9a8028a8413afc344ef80d0f0d41e28c25cc9ad634e24e23639792b19f07d6acc22494429789dde725d8e202fe5625e27

setup.sh.ndk 5aaec0a91da377fb1bf86ac2a28a2adea3d3f7736b71c0fb2249ae8ca0ff66731311f89c9e933f0944e8e5771a0c14d6a3055f181721f52a0cd5397f175bcb14

Comment 14

[John Hopkins (:jhopkins)]

Again, with filesize:

filename: android-sdk.tar.xz
sha512: 69c9be39a452c202ebe99fa3a7f31cb9a8028a8413afc344ef80d0f0d41e28c25cc9ad634e24e23639792b19f07d6acc22494429789dde725d8e202fe5625e27
filesize: 156663204

filename: setup.sh.ndk 
sha512: 5aaec0a91da377fb1bf86ac2a28a2adea3d3f7736b71c0fb2249ae8ca0ff66731311f89c9e933f0944e8e5771a0c14d6a3055f181721f52a0cd5397f175bcb14
filesize: 156663204

Comment 15

[John Hopkins (:jhopkins)]

*correction* filesize of setup.sh.ndk: 217 bytes

Comment 16

[Trevor Saunders (:tbsaunde)]

Attachment: 894885fixup

this code dupplicates (and overrides!) what's in the common android mozconfig

Comment 17

[Trevor Saunders (:tbsaunde)]

Attachment: sdktest

Comment 18

[Mike Hommey [:glandium]]

Comment on attachment 8385798 [details] [diff] [review]
894885fixup

Review of attachment 8385798 [details] [diff] [review]:
-----------------------------------------------------------------

r+, but i'd actually like more cleanup.

::: mobile/android/config/mozconfigs/android/debug
@@ -18,5 @@
> -fi
> -
> -ac_add_options --with-android-gnu-compiler-version=4.7
> -ac_add_options --with-android-version=9
> -ac_add_options --with-system-zlib

This is duplicated in other mozconfigs, too.

Note the common mozconfig should probably be changed to *not* include the --target, and be included in the x86 mozconfig as well.

Comment 19

[Mike Hommey [:glandium]]

Comment on attachment 8385800 [details] [diff] [review]
sdktest

Review of attachment 8385800 [details] [diff] [review]:
-----------------------------------------------------------------

With the lack of update of l10n and x86/armv6 mozconfigs in the previous patch, this will fail.

Comment 20

[Trevor Saunders (:tbsaunde)]

(In reply to Mike Hommey [:glandium] from comment #18)
> Comment on attachment 8385798 [details] [diff] [review]
> 894885fixup
> 
> Review of attachment 8385798 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> r+, but i'd actually like more cleanup.
> 
> ::: mobile/android/config/mozconfigs/android/debug
> @@ -18,5 @@
> > -fi
> > -
> > -ac_add_options --with-android-gnu-compiler-version=4.7
> > -ac_add_options --with-android-version=9
> > -ac_add_options --with-system-zlib
> 
> This is duplicated in other mozconfigs, too.

yeah, I only fixed that one because it was the only one to break on try and I forgot about l10n somehow. So I was trying to be conservative but that doesn't actually make any sense.

> Note the common mozconfig should probably be changed to *not* include the
> --target, and be included in the x86 mozconfig as well.

it looks like it *is* included in the x86 mozconfigs, presumably the second --target wins

(In reply to Mike Hommey [:glandium] from comment #19)
> With the lack of update of l10n and x86/armv6 mozconfigs in the previous
> patch, this will fail.

well, it looks like the android-armv6/debug mozconfig is unused given https://tbpl.mozilla.org/?tree=Try&rev=ec94922ca61d but I can't say anything about x86 since I can't spell :(

Comment 21

[Trevor Saunders (:tbsaunde)]

Attachment: 894885fixup

still not perfect, but this at least mostly cleans up those other mozconfigs

Comment 22

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/ea3ad950c01b

Comment 23

[Chenxia Liu [:liuche]]

The newest Android build tools version has been updated to 19.0.3; our Fennec build docs say we don't support r22 or build-tools-18+, but we could update the best-effort build-tools search[1] to include the new revision.

[1] http://hg.mozilla.org/mozilla-central/file/4f08a24e4fb5/build/autoconf/android.m4#l296

Comment 24

[Brad Lassey [:blassey] (use needinfo?)]

(In reply to Trevor Saunders (:tbsaunde) from comment #21)
> Created attachment 8387148 [details] [diff] [review]
> 894885fixup
> 
> still not perfect, but this at least mostly cleans up those other mozconfigs

glandium, review ping? or suggest another reviewer?

Comment 25

[Trevor Saunders (:tbsaunde)]

(In reply to Brad Lassey [:blassey] (use needinfo?) from comment #24)
> (In reply to Trevor Saunders (:tbsaunde) from comment #21)
> > Created attachment 8387148 [details] [diff] [review]
> > 894885fixup
> > 
> > still not perfect, but this at least mostly cleans up those other mozconfigs
> 
> glandium, review ping? or suggest another reviewer?

arguably the r+ on the previous version  could cary forward since all the changes are what I believe glandium asked for, but I'd be more comfortable with that if you make it explicit that was what you meant.

Comment 26

[Mike Hommey [:glandium]]

Comment on attachment 8387148 [details] [diff] [review]
894885fixup

Review of attachment 8387148 [details] [diff] [review]:
-----------------------------------------------------------------

Sorry for the long wait. I was heads down on something.
r=me with the following addressed.

::: mobile/android/config/mozconfigs/android-armv6/debug
@@ +10,2 @@
>  # Android
>  ac_add_options --target=arm-linux-androideabi

You now have this twice. (note it's better to keep --target and --with-arch close together, so i'd just remove the one you're adding).

::: mobile/android/config/mozconfigs/android-armv6/l10n-nightly
@@ +15,2 @@
>  # Android
>  ac_add_options --target=arm-linux-androideabi

Likewise

::: mobile/android/config/mozconfigs/android-noion/nightly
@@ -37,5 @@
> -# This will overwrite the default of stripping everything and keep the symbol table.
> -# This is useful for profiling with eideticker. See bug 788680
> -STRIP_FLAGS="--strip-debug"
> -
> -. "$topsrcdir/mobile/android/config/mozconfigs/common.override"

In case of unexpected consequences, you may want this to be in a separate patch, so that if the removal needs to be backed out for whatever reason, the rest is not backed out at the same time.

::: mobile/android/config/mozconfigs/android/debug-nonunified
@@ +1,4 @@
>  . "$topsrcdir/mobile/android/config/mozconfigs/android/debug"
>  
> +# Android
> +ac_add_options --target=arm-linux-androideabi

That's already in mobile/android/config/mozconfigs/android/debug

::: mobile/android/config/mozconfigs/android/nightly-nonunified
@@ +1,4 @@
>  . "$topsrcdir/mobile/android/config/mozconfigs/android/nightly"
>  
> +# Android
> +ac_add_options --target=arm-linux-androideabi

This is already in mobile/android/config/mozconfigs/android/nightly

Comment 27

[Trevor Saunders (:tbsaunde)]

remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/34f2538b1c38
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/7bb92c73225e
remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/518bca1c6944

Comment 28

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/34f2538b1c38
https://hg.mozilla.org/mozilla-central/rev/7bb92c73225e
https://hg.mozilla.org/mozilla-central/rev/518bca1c6944