554249 - nightly updates for Android

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P4)

Description

[Brad Lassey [:blassey] (use needinfo?)]

Once we get nightly builds, we'll want to have an update channel.

Comment 1

[Chris Cooper [:coop] (he/him)]

Assuming this happens in Q2, we'll be on it.

Comment 2

[Aki Sasaki (not active)]

Stuart is investigating whether this will be AUS-based or if there's an Android-specific update mechanism we're going to use.

Comment 3

[Aki Sasaki (not active)]

Assigning to Stuart for info; throw this back once we know how we want to do updates.

Comment 4

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Stuart: any update (groan) on this?

Comment 5

[Aki Sasaki (not active)]

We want this for Q3, but still need info.

Comment 6

[Aki Sasaki (not active)]

Bug 587384 isn't strictly blocking our end, but is needed for the updates produced to be useful to the client.

Comment 7

[Aki Sasaki (not active)]

Bug 593242 tracks server-side changes, which may or may not be needed.

Comment 8

[Aki Sasaki (not active)]

Attachment: [wip] android updates -- configs

Assuming:

1) we want a /opt/aus2/incoming/2/Fennec/
2) the update_platform is Android_arm-eabi-gcc3

These are both quite possibly bad assumptions. But it's something I can test with at first.

Comment 9

[Aki Sasaki (not active)]

Attachment: [wip] android updates -- custom (080)

For nightlies. I'll have to add another patch or two for Android release builds.
Got most of the logic/code from Coop's WinmoBuildFactory updates.

Also got rid of the dupe Android upload logic -- MobileBuildFactory's addUploadSteps() should be sufficient for everything but updates.
 
This passes checkconfig and I'm running a test m-c Android nightly in staging.

Comment 10

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #8)
> Assuming:
> 1) we want a /opt/aus2/incoming/2/Fennec/
> 2) the update_platform is Android_arm-eabi-gcc3

Looks fine to me. We'll also need to update the mozconfigs for the nightlies to set the channel, and change 
 http://mxr.mozilla.org/mozilla/source/webtools/aus/xml/inc/config-dist.php#136
to map the incoming requests to mozilla-central/mozilla-2.0 etc.

Note that if we're going to do updates for tracemonkey and e10s nightlies we need set a different update channel for them. Do the nothumb android builds need updates too ?

Comment 11

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #9)
> I'll have to add another patch or two for Android release
> builds.

You will either want to be able to send release snippets somewhere else, or not upload them, or use patcher2 for the release case. We do the latter for desktop.

Comment 12

[Aki Sasaki (not active)]

Attachment: [wip] android nightly updates -- custom (080)

With these fixes, I got

version=1
type=complete
url=http://staging-stage.build.mozilla.org/pub/mozilla.org/firefox/nightly/2010/
09/2010-09-09-04-mozilla-central-android-r7/fennec.apk
hashFunction=sha512
hashValue=a04cc685e64f1a557e13715ee41a77f3fc2ed82f72f735aec4865fc4e192520e25ae5b
88f9afabe7ab1cd988197706750eaf2808db8183e3661ec1fa8f2d74b5
size=12215274
build=20100909045846
appv=2.0b1pre
extv=2.0b1pre

I'll address Nick's comments and start addressing release builds on 0.7.x today.

Comment 13

[Aki Sasaki (not active)]

Attachment: aus patch for Fennec 2.0*

Add Fennec 2.0 to aus' config-dist.php.

Blassey said he doesn't care about tracemonkey or electrolysis updates, but does want mozilla-central + mozilla-2.0 updates when mozilla-2.0 branches.

I'll be taking care of multi-branch updates in bug 594867, but thought it might be good to prepopulate this file with mozilla-2.0. I can certainly remove that line and keep it for later if that's wisest.

I have new patches for buildbot-configs and buildbotcustom to add a create_mobile_snippet that allows us to flip that on/off independent of Desktop settings.

Comment 14

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #13)
> Blassey said he doesn't care about tracemonkey or electrolysis updates, but
> does want mozilla-central + mozilla-2.0 updates when mozilla-2.0 branches.

This nicely steps around the problem of mobile not using different versions for different gecko revisions, and having no support for doing anything with PLATFORM_VERSION in AUS.

Comment 15

[Aki Sasaki (not active)]

(In reply to comment #14)
> (In reply to comment #13)
> > Blassey said he doesn't care about tracemonkey or electrolysis updates, but
> > does want mozilla-central + mozilla-2.0 updates when mozilla-2.0 branches.
> 
> This nicely steps around the problem of mobile not using different versions for
> different gecko revisions, and having no support for doing anything with
> PLATFORM_VERSION in AUS.

Won't we still run into that? App version is set by http://hg.mozilla.org/mobile-browser/file/afa447a64bdf/confvars.sh , which sources http://hg.mozilla.org/mobile-browser/file/afa447a64bdf/default-version.txt .

Currently the plan is to build both mozilla-2.0 and mozilla-central against mobile-browser, which means both branches would have the same app version.

However, if we have mozconfigs that specify different channels we'll be ok ?
maybe?
hopefully?

Comment 16

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #15)
> However, if we have mozconfigs that specify different channels we'll be ok ?
> maybe?
> hopefully?

Yes, exactly.

Comment 17

[Aki Sasaki (not active)]

Attachment: android nightly updates -- configs (default)

Add a create_mobile_snippet flag; remove aus2_mobile_base_upload_dir* from all branches but mozilla-central and mozilla-2.0.  I'll flip the mozilla-2.0 create_mobile_snippet flag in bug 594867, along with the mozconfig shuffle etc.

Comment 18

[Aki Sasaki (not active)]

Attachment: [wip] android nightly updates -- custom (080)

Same as previous (tested) patch, checking for create_mobile_snippet in misc.py.
Passes test-masters.sh.

Comment 19

[Aki Sasaki (not active)]

(In reply to comment #11)
> (In reply to comment #9)
> > I'll have to add another patch or two for Android release
> > builds.
> 
> You will either want to be able to send release snippets somewhere else, or not
> upload them, or use patcher2 for the release case. We do the latter for
> desktop.

I took a look at patcher2 and ReleaseUpdatesFactory and I'm more than a little daunted. I'm definitely leaning towards sending release snippets somewhere else until we're actually generating complete and partial mars like Desktop.

Comment 20

[Aki Sasaki (not active)]

Attachment: [wip] android nightly updates -- custom (080)

This is the same as attachment 473792 [details] [diff] [review], except I split the call to CreateCompleteUpdateSnippet() to a _createSnippet() function that I override in AndroidReleaseBuildFactory in bug 594219.

Not strictly needed now, but will hopefully simplify merging branches later.

Comment 21

[Aki Sasaki (not active)]

Landed mozconfig updates: http://hg.mozilla.org/build/buildbot-configs/rev/38eda43c57e6

Kicked off nightly #1, which will hopefully update to nightly #2 when pointed to a patched staging aus.

Comment 22

[Aki Sasaki (not active)]

Comment on attachment 473760 [details] [diff] [review]
aus patch for Fennec 2.0*

I'll refresh the nightly configs/custom patches and r?

Comment 23

[Aki Sasaki (not active)]

Attachment: android nightly updates -- custom (080)

As above, but I pull the addStep's for uploading snippets to _uploadSnippet() so I can call it from AndroidReleaseBuildFactory in bug 594219.

Comment 24

[Aki Sasaki (not active)]

Attachment: nightly configs with mobile_download_base_url

Comment 25

[Aki Sasaki (not active)]

Attachment: nightly custom with mobile_download_base_url

Comment 26

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 473760 [details] [diff] [review]
aus patch for Fennec 2.0*

Looks good to me. I'll double check on aus2-staging, and get it deployed if all is well.

Comment 27

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 475415 [details] [diff] [review]
nightly custom with mobile_download_base_url

Looks fine, except that the snippet for the new build has to be placed in a directory named with the buildID of the previous build, and an empty file at <NEW_BUILDID>/complete.txt.

Will also need to check if we're using the right buildid, not sure which ends up in the query url.

Comment 28

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: [as checked in] aus patch for Fennec 2.0*

Also needed to update $nightlyChannels, caught that in staging.

Checking in inc/config-dist.php;
/cvsroot/mozilla/webtools/aus/xml/inc/config-dist.php,v  <--  config-dist.php
new revision: 1.98; previous revision: 1.97
done

Tagged webtools/aus/xml with AUS2_RTM_201009150339, moved AUS2_PRODUCTION to rev 1.98 of config-dist.php. See deps for deployment bug.

Comment 29

[Aki Sasaki (not active)]

Attachment: AndroidBuildFactory with previous apk download

Running a build on sm03 atm.

Comment 30

[Aki Sasaki (not active)]

Attachment: fixed up factory changes w/ previous apk

This is after several staging runs with incremental fixes.
It succeeded on sm03 (android nightly #8) though I commented out the clobber/build/package steps to make it cycle faster.

I'm now running nightly #9 to verify the steps when previous_build_id != build_id.

Comment 31

[Aki Sasaki (not active)]

Comment on attachment 475414 [details] [diff] [review]
nightly configs with mobile_download_base_url

http://hg.mozilla.org/build/buildbot-configs/rev/f747e78056da

Comment 32

[Aki Sasaki (not active)]

Comment on attachment 475657 [details] [diff] [review]
fixed up factory changes w/ previous apk

http://hg.mozilla.org/build/buildbotcustom/rev/8f11453de8b4

Comment 33

[Aki Sasaki (not active)]

This is all landed.

I attempted to test by installing the 7:20something pm nightly, to try to update to the 9:20something nightly, but after installing Fennec + going to about:config my phone has frozen while attempting to do anything.

I'm probably going to have to leave this to someone else to test.

Comment 34

[Aki Sasaki (not active)]

Attachment: add /update/4/ support in aus

The current url

https://aus2.mozilla.org/update/4/Fennec/2.0b1pre/20100916041726/Android_arm-eabi-gcc3/en-US/nightly/Linux%202.6.32.9-27227-g3c98b0d/default/default/2.0b7pre/update.xml

works when I |s,4,3,|

Adding in a |case 4:| to support update/4/ style urls.

Comment 35

[Michael Morgan [:morgamic]]

Comment on attachment 476024 [details] [diff] [review]
add /update/4/ support in aus

Looks fine, passes acceptance tests.

Comment 36

[Aki Sasaki (not active)]

Comment on attachment 476024 [details] [diff] [review]
add /update/4/ support in aus

Checked in as revision 1.29.
Tagged mozilla/webtools/aus/xml as AUS2_RTM-201009161505, and bumped AUS2_PRODUCTION to revision 1.29 of index.php.

I'll file a bug to update AUS.

Comment 37

[Aki Sasaki (not active)]

https://aus2.mozilla.org/update/4/Fennec/2.0b1pre/20100916041726/Android_arm-eabi-gcc3/en-US/nightly/Linux%202.6.32.9-27227-g3c98b0d/default/default/2.0b7pre/update.xml now gives xml:

<updates><update type="minor" version="2.0b1pre" extensionVersion="2.0b1pre" buildID="20100917040728"><patch type="complete" URL="http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/2010/09/2010-09-17-04-mozilla-central-android-r7/fennec.apk" hashFunction="sha512" hashValue="b686fa7c36d4befdd07a55eeda3f6ecb6245389e1828f3b2dbef11a23477e206f6ca46e949806835e40af45cdb6b969155f5e1adf4678ed35a8ff3924b856d15" size="12344434"/></update></updates>

Comment 38

[Aki Sasaki (not active)]

I can't test due to bug 597093 and bug 596662, but this appears to be set up properly.