releng work for dropping api-11 through api-14

RESOLVED FIXED

Status

RESOLVED FIXED
3 years ago
5 months ago

People

(Reporter: rnewman, Assigned: jlund)

Tracking

Trunk
All
Android
Dependency tree / graph

Details

(URL)

Attachments

(9 attachments)

10.31 KB, patch
camd
: review+
emorley
: checked-in+
Details | Diff | Splinter Review
31.51 KB, patch
rail
: review+
Details | Diff | Splinter Review
13.11 KB, patch
rail
: review+
Details | Diff | Splinter Review
44.74 KB, patch
rail
: review+
Details | Diff | Splinter Review
850 bytes, patch
rail
: review+
jlund
: checked-in+
Details | Diff | Splinter Review
1.05 KB, patch
rail
: review+
jlund
: checked-in+
Details | Diff | Splinter Review
3.11 KB, patch
rail
: review+
jlund
: checked-in+
Details | Diff | Splinter Review
47 bytes, text/x-github-pull-request
Details | Review | Splinter Review
689 bytes, patch
jlund
: review+
Details | Diff | Splinter Review
(Reporter)

Description

3 years ago
Part of Bug 1155801.

We'll only do this if we land the changes in that bug!

jlund, could you move this to the right component?
Flags: needinfo?(jlund)
(Reporter)

Comment 1

3 years ago
Note that this might also affect the updater -- Honeycomb Nightly and Aurora users will need to be orphaned on their current build, and 14+ Nightly and Aurora users will need to jump to the api-14 build.
(Assignee)

Updated

3 years ago
Component: Build Config & IDE Support → Platform Support
Flags: needinfo?(jlund)
Product: Firefox for Android → Release Engineering
QA Contact: coop
I can take care of the renaming parts and also the logic for how we are going to phase out api-11 and integrate api-14 assuming we ride the trains on this.

It sounds like there is a Friday meeting to discuss how we are proceeding with branches like esr, etc.

As I am not involved with the discussions or Bug 1155801, I will wait until I get pinged in this bug before doing anything.
Assignee: nobody → jlund

Comment 3

3 years ago
I wonder if it's worth using a more generic name that won't get out of date again in the future?

The problem is that with the rename now, things like perf stats will appear as a new platform, so make it hard to follow trends - when in fact other than dropping old Honeycomb code, it's the same platform.

Really we have two builds, a "old android api" and "newer android api" build. The actual version numbers used for both are somewhat academic (unless someone is debugging failures that only occur on one, where they can look up what target API is used at the time).

Updated

3 years ago
Summary: Prepare to rename API-11 builds to 14 → Prepare to rename Android API-11 builds to API-14
(In reply to Ed Morley [:emorley] from comment #3)
> I wonder if it's worth using a more generic name that won't get out of date
> again in the future?

make sense. Though, I believe in the future we would like to do more than one split. So if we could keep that in mind with the naming, wfm.
:rnewman -- Is it time to proceed with this bug?
Flags: needinfo?(rnewman)
this bug will still consist of:

1) Prepare to rename Android API-11 builds to API-14

but will also include

2) stop serving api-11 -> api-13 based nightlies to users.

rnewman: I'm assuming we will be riding the trains with this? e.g. we still want to serve updates to honeycomb aurora users until 46 reaches m-a, correct?
Summary: Prepare to rename Android API-11 builds to API-14 → releng work for dropping honeycomb support
(Reporter)

Comment 7

3 years ago
Yes. Nightly is now 14+, and that'll ride the trains to release in 46.
tracking-fennec: --- → 46+
status-firefox44: affected → unaffected
status-firefox45: --- → unaffected
status-firefox46: --- → affected
Flags: needinfo?(rnewman)
rail: can you sanity check: https://aus4-admin.mozilla.org/rules#nightlytest-honeycomb

we want to stop serving honeycomb users m-c nightly updates.

if r+, I'll change the channel name to 'nightly'.

I should note, as https://bugzilla.mozilla.org/show_bug.cgi?id=1155801#c63 finished on Jan 1st, I would imagine a large subset of honeycomb users have already received updates for a nightly that is newer than that.
Flags: needinfo?(rail)
list of versions derived from the following convo:

14:20:25 <jlund> https://en.wikipedia.org/wiki/Android_version_history suggests honeycomb is represented by 3.0–3.2.6
14:20:43 <jlund> anyone know where I can get an official explicit list of versions within that range?
14:20:57 <jlund> looking to disable updates to honeycomb users
14:21:36 <jlund> this requires a special balrog rule. e.g. for api-9 split, I use the following list: 2.3, 2.3.1, 2.3.2
14:24:33 <•mfinkle> jlund: your list sounds reasable
14:24:37 <•mfinkle> jlund: http://developer.android.com/reference/android/os/Build.VERSION_CODES.html
14:24:56 <•mfinkle> doesn't show the minor/minor release
14:25:24 <•mfinkle> jlund: but we do know that anything in the 3.x range is honeycomb only
14:25:40 <•mfinkle> the next higher release is 4.0
14:27:36 <jlund> mfinkle: hm, I think I need to specify the full version including minor dot as we don't match substrings in os version lookup
14:28:02 <•mfinkle> jlund: start with your list then


also, here is the discussion that determined we likely have had users try and fail to update on honeycomb..

14:30:21 <jlund> mfinkle: k. one more question while I have you, https://bugzilla.mozilla.org/show_bug.cgi?id=1155801#c63 suggests we dropped honeycomb on the 1st
14:30:25 <firebot> Bug 1155801 — FIXED, rnewman@mozilla.com — Drop Honeycomb support from Firefox builds
14:30:37 <jlund> our current nightly points to http://download.cdn.mozilla.net/pub/mobile/nightly/2016/01/2016-01-04-03-02-17-mozilla-central-android-api-11/
14:30:45 <jlund> with buildid 20160104030217
14:31:43 <jlund> so I am assuming there are honeycomb users who have gotten a build that is newer than bug 1155801 
14:32:04 <•mfinkle> if they did, they are not happy
14:32:57 <•mfinkle> jlund: any honeycomb user that tries to use a current nightly should get an install failure
hmm, my assumption of OS VERSION may be wrong; it might allow substring values. e.g. 'Windows' would catch 'Windows_NT'. In which case, numbered versions would be bad. e.g. '3.2' would catch '2.3.2' and '3.2.2'
(In reply to Jordan Lund (:jlund) from comment #10)
> hmm, my assumption of OS VERSION may be wrong; it might allow substring
> values. e.g. 'Windows' would catch 'Windows_NT'. In which case, numbered
> versions would be bad. e.g. '3.2' would catch '2.3.2' and '3.2.2'

yes, it looks like we accept substrings: https://github.com/mozilla/balrog/blob/078e23b660d621ae841af172b9f706231f1d5ecf/auslib/db.py#L710

this might cause spurious results as our list of possible versions grow ..
I've set update rate to 0% for all non api-9-10 fennec m-c updates while we figure this out today so we don't keep trying to serve updates to honeycomb users.
1) yeah, substring matching may bite us in the future, but probably we can deal with that later
2) please bump the priority to something >100 so it's easier to spot these watersheds
3) maybe also add the build target to narrow down the criteria? I don't know exactly what the value is, it can be found in the update URLs.

otherwise lgtm
Flags: needinfo?(rail)
ftr - yesterday evening, I reverted the update rate back to 100% for api-11 (now api-14) and put the No-Update for honeycomb rule into action on the nightly channel

I didn't include build target because I wanted to keep it in sync with the other apk splits.

Tomorrow morning I'll address the s/11/14 part.
(Assignee)

Updated

3 years ago
Blocks: 1237338
(Reporter)

Comment 15

3 years ago
jlund: I think we're going to change 14 to 15 right now; 14 has ~0 users, so we might as well make that change to avoid churn. I'll let you know when you can s/11/15 :D
(In reply to Richard Newman [:rnewman] from comment #15)
> jlund: I think we're going to change 14 to 15 right now; 14 has ~0 users, so
> we might as well make that change to avoid churn. I'll let you know when you
> can s/11/15 :D

okay sounds good. I'll hold off changing things until 15 is the new minimum and sticks
api 15+ has landed on central. I've added: 4.0, 4.0.1, and 4.0.2 to the No-Update list for m-c nightly based on versions with api-14: http://developer.android.com/guide/topics/manifest/uses-sdk-element.html

rail: looks like we are going to need to address this sooner rather than later bc '4.0' is in '4.0.3' and '4.0.3' == api-15 (we only want to block api 11-14) ..

I have a series of patches lined up that I'm hoping to land tomorrow morning.
Flags: needinfo?(rail)
You can add another rule for 4.0.3 with higher priority and point it to Fennec-mozilla-central-nightly-latest. This way balrog would catch 4.0.3 in that rule and serve updates as expected. Does it make sense?
Flags: needinfo?(rail)
(In reply to Rail Aliiev [:rail] from comment #18)
> You can add another rule for 4.0.3 with higher priority and point it to
> Fennec-mozilla-central-nightly-latest. This way balrog would catch 4.0.3 in
> that rule and serve updates as expected. Does it make sense?

done. 4.0.3 should be getting updates again
Summary: releng work for dropping honeycomb support → releng work for dropping api-11 through api-14
Created attachment 8705919 [details] [diff] [review]
160108_bug_1219094_drop_api_11_through_14-treeherder.patch

Hey cam,

we are dropping api-11 on trunk. We need to support api-11 on other gecko based branches.

How does this look?
Attachment #8705919 - Flags: review?(cdawson)
Created attachment 8705921 [details] [diff] [review]
160108_bug_1219094_drop_api_11_through_14-bbot-cfgs.patch

1st up. because this is riding the trains, we need to phase out api-11 on each gecko bump.
Attachment #8705921 - Flags: review?(rail)
Created attachment 8705923 [details] [diff] [review]
160108_bug_1219094_drop_api_11_through_14-bbot-tools.patch

iirc, the mozconfig whitelist file is used by releaserunner/sanity? in which case, I'll have to uplift this change to 'old-release-runner' branch too?

I think it's safe to remove the old non-split 'android' platform across our buildbot masters. We actually need to do this anyway because bbot test slaves complain about having too many builders attached to them
Attachment #8705923 - Flags: review?(rail)
Created attachment 8705925 [details] [diff] [review]
160108_bug_1219094_drop_api_11_through_14-gecko-tc-mh.patch

this one is a bit funnier. Because some android builds (afaik) are still scheduled through buildbot, we have to do this in two stages.

1) keep api-11 around until buildbot is reconfiged with trunk based api-15 patch
2) remove api-11 files

however we can:

1) change things wholesale at once for anything taskcluster scheduling/task based. \o/

I know for a fact that b2gdroid and dummy apk are only scheduled via TC so that's why I didn't bother even keeping the api-11 bits for those around at all.
Attachment #8705925 - Flags: review?(rail)
Created attachment 8705926 [details] [diff] [review]
160108_bug_1219094_drop_api_11_through_14-puppet.patch

simple one to keep up with tc routes
Attachment #8705926 - Flags: review?(rail)
Created attachment 8705927 [details] [diff] [review]
160108_bug_1219094_drop_api_11_through_14-cloud-tools.patch

again, need to keep api-11 around here.
Attachment #8705927 - Flags: review?(rail)
Created attachment 8705928 [details] [diff] [review]
160108_bug_1219094_drop_api_11_through_14-slave-health.patch

last one ... I think

Let's see what I forgot :)

sorry for the r? storm rail. Who would have thought there would be this much logic let alone places to change for a silly name. feel free to 302 any of the requests on; I figured it be easier to just iterate thought them all at once for most them.
Attachment #8705928 - Flags: review?(rail)
Comment on attachment 8705921 [details] [diff] [review]
160108_bug_1219094_drop_api_11_through_14-bbot-cfgs.patch

if you are going to run dump_masters/checkconfig against this patch, you will need to add this to your env: https://dxr.mozilla.org/build-central/source/buildbot-configs/mozilla-tests/config_seta.py#39

this is because the following doesn't exist yet in http://alertmanager.allizom.org/data/setadetails/:

+                  "android-4-3-armv7-api15": ("android-api-15", ["ubuntu64_vm_armv7_mobile", "ubuntu64_vm_armv7_large"])

speaking of which, jmaher: would you be able to help me add the api-15 based apk split to the above url?
Flags: needinfo?(jmaher)
Comment on attachment 8705925 [details] [diff] [review]
160108_bug_1219094_drop_api_11_through_14-gecko-tc-mh.patch

hey nalexander :)

fyi - I didn't touch any of this mozbuild stuff: https://dxr.mozilla.org/mozilla-central/search?q=api-11+path%3Amozbuild&redirect=true&case=false

Not too familiar with it. not sure if we have to keep any of the api-11 bits around or if we can just swap it to 15 immediately. seems like you have commited to these two files a fair bit.
Flags: needinfo?(nalexander)
Comment on attachment 8705921 [details] [diff] [review]
160108_bug_1219094_drop_api_11_through_14-bbot-cfgs.patch

Review of attachment 8705921 [details] [diff] [review]:
-----------------------------------------------------------------

::: mozilla-tests/mobile_config.py
@@ +3293,4 @@
>      for platform in branch['platforms']:
>          if not platform in PLATFORMS:
>              continue
> +        if platform not in ('android-api-9'):

A nit. Missing trailing coma, ('android-api-9') is converted to 'android-api-9', in which case:

 'android' in ('android-api-9')
 True

@@ +3455,5 @@
>  remove_suite_from_slave_platform(BRANCHES, PLATFORMS, 'mochitest-chrome', 'ubuntu64_vm_large', branches_to_keep=trunk_branches)
>  
>  # bug 1214362 - Enable DOM push Mochitests on Android Fennec
> +BRANCHES['cedar']['platforms']['android-api-15']['ubuntu64_vm_armv7_mobile']['opt_unittest_suites'] += ANDROID_4_3_MOCHITEST_PUSH
> +BRANCHES['cedar']['platforms']['android-api-15']['ubuntu64_vm_armv7_large']['debug_unittest_suites'] += ANDROID_4_3_MOCHITEST_PUSH

Not related to this patch, just wonder why these configs are not in the projects_branches.py file... I guess, because it's too strict?

::: mozilla/config.py
@@ +1758,5 @@
> +            'nightly_signing_servers': 'dep-signing',
> +            'dep_signing_servers': 'dep-signing',
> +            'use_mock': True,
> +            'mock_target': 'mozilla-centos6-x86_64-android',
> +            'mock_packages': ['autoconf213', 'mozilla-python27-mercurial',

I wonder if we need these mock_packages set here... I thought that we use the ones defined in mozharness.
Attachment #8705921 - Flags: review?(rail) → review+
Attachment #8705923 - Flags: review?(rail) → review+
Comment on attachment 8705925 [details] [diff] [review]
160108_bug_1219094_drop_api_11_through_14-gecko-tc-mh.patch

Review of attachment 8705925 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm
Attachment #8705925 - Flags: review?(rail) → review+
Attachment #8705926 - Flags: review?(rail) → review+
Attachment #8705927 - Flags: review?(rail) → review+
Attachment #8705928 - Flags: review?(rail) → review+
Thank you for taking care of this!
:jlund, I am not sure exactly what we need to add to SETA, I can help get it added- SETA takes existing jobs that it finds and generates data based on that, is it possible that we don't have these jobs on inbound?  I have done some preliminary work to add api15 to the list for the SETA backend.  looking on mozilla-inbound, I don't see api15 jobs.  We can preseed stuff, I just need to know what we are going to be running and I can do that.
Flags: needinfo?(jmaher) → needinfo?(jlund)
(In reply to Rail Aliiev [:rail] from comment #31)
> Thank you for taking care of this!

thank you for the reviews :)

I think as for the order in landing these, the treeherder one should go first so I'll wait for the review of that one.
Flags: needinfo?(jlund)
(In reply to Joel Maher (:jmaher) from comment #32)
> :jlund, I am not sure exactly what we need to add to SETA, I can help get it
> added- SETA takes existing jobs that it finds and generates data based on
> that, is it possible that we don't have these jobs on inbound?  I have done
> some preliminary work to add api15 to the list for the SETA backend. 
> looking on mozilla-inbound, I don't see api15 jobs.  We can preseed stuff, I
> just need to know what we are going to be running and I can do that.


I have never touched seta but here is what I discovered:

we will need to pre-seed this because buildbot won't reconfig unless api-15 exists in setadetails

the end goal:

* replace api-11 based builds and tests with api-15

which meant needing to patch the following:

diff --git a/mozilla-tests/config_seta.py b/mozilla-tests/config_seta.py
index aff4c71..c123d14 100644
--- a/mozilla-tests/config_seta.py
+++ b/mozilla-tests/config_seta.py
@@ -22,7 +22,7 @@ seta_platforms = {"Rev4 MacOSX Snow Leopard 10.6": ("macosx64", ["snowleopard"])
                   "Rev7 MacOSX Yosemite 10.10.5": ("macosx64", ["yosemite_r7"]),
                   "Ubuntu Code Coverage VM 12.04 x64": ("linux64-cc", ["ubuntu64_vm", "ubuntu64_vm_lnx_large"]),                  
                   "android-2-3-armv7-api9": ("android-api-9", ["ubuntu64_vm_mobile", "ubuntu64_vm_large"]),
-                  "android-4-3-armv7-api11": ("android-api-11", ["ubuntu64_vm_armv7_mobile", "ubuntu64_vm_armv7_large"])
+                  "android-4-3-armv7-api15": ("android-api-15", ["ubuntu64_vm_armv7_mobile", "ubuntu64_vm_armv7_large"])
                   }

which broke https://dxr.mozilla.org/build-central/source/buildbot-configs/mozilla-tests/config_seta.py?case=true&from=config_seta.py#115

because that code gets it's data from something like: http://alertmanager.allizom.org/data/setadetails/?date=2016-01-11&buildbot=1&branch=mozilla-inbound&inactive=1

but uses platforms from:https://dxr.mozilla.org/build-central/source/buildbot-configs/mozilla-tests/config_seta.py?case=true&from=config_seta.py#14

so, iiuc, we will need to add the following jobs to setadetails from today forward:
copy all android-4-3-armv7-api11 jobtypes from: https://pastebin.mozilla.org/8856403 and s/android-4-3-armv7-api11/android-4-3-armv7-api15

this all assumes that data on alertmanager.allizom.org/data/setadetails is manually entered. if it is updated via automation, there might be a way to hack this together by bumping the '11's to '15's in two stages at this line: https://dxr.mozilla.org/build-central/source/buildbot-configs/mozilla-tests/config_seta.py?case=true&from=config_seta.py#25

jmaher: does this make more sense now?
Flags: needinfo?(jmaher)
Comment on attachment 8705919 [details] [diff] [review]
160108_bug_1219094_drop_api_11_through_14-treeherder.patch

This looks good to me.  Please need-info me if you need me to land this for you.  Thanks!  :)
Attachment #8705919 - Flags: review?(cdawson) → review+
(In reply to Jordan Lund (:jlund) from comment #28)
> Comment on attachment 8705925 [details] [diff] [review]
> 160108_bug_1219094_drop_api_11_through_14-gecko-tc-mh.patch
> 
> hey nalexander :)
> 
> fyi - I didn't touch any of this mozbuild stuff:
> https://dxr.mozilla.org/mozilla-central/search?q=api-
> 11+path%3Amozbuild&redirect=true&case=false
> 
> Not too familiar with it. not sure if we have to keep any of the api-11 bits
> around or if we can just swap it to 15 immediately. seems like you have
> commited to these two files a fair bit.

I thought I responded to this.  In any case, their should be no build system changes needed, 'cuz right now we're lieing: we say we're building API 11 but in reality we're building API 15+.
Flags: needinfo?(nalexander)
Created attachment 8707034 [details] [review]
[treeherder] lundjordan:api-15 > mozilla:master
 Ibelieve my work is done here for now- clearing the needinfo.
Flags: needinfo?(jmaher)
(In reply to Cameron Dawson [:camd] from comment #35)
> Comment on attachment 8705919 [details] [diff] [review]
> 160108_bug_1219094_drop_api_11_through_14-treeherder.patch
> 
> This looks good to me.  Please need-info me if you need me to land this for
> you.  Thanks!  :)

thanks for the review. looks like I do not have write access. I created a PR. Cam, Could you land for me please: https://github.com/mozilla/treeherder/pull/1243

also, next time should I use reviewable for r? requests?
Flags: needinfo?(cdawson)
Comment hidden (obsolete)

Updated

3 years ago
Attachment #8705919 - Flags: checked-in+
(In reply to Jordan Lund (:jlund) from comment #39)
> thanks for the review. looks like I do not have write access. I created a
> PR. Cam, Could you land for me please:
> https://github.com/mozilla/treeherder/pull/1243
> 
> also, next time should I use reviewable for r? requests?

The process for contributing to Treeherder is now (we've only recently started using autolander):
1) File a bug in a Treeherder component (unless it's a one-line doc fix or something)
2) Create a pull request with that bug number in it
3) Wait for autolander to create the attachment in bugzilla that links to the PR (this will only work if the bug is in a whitelisted component; which are currently Firefox OS and Tree Management)
4) Flag r? on that attachment as normal

Even if autolander is whitelisted to allow the Release Engineering component, I still think there is an advantage of filing bugs for the Treeherder parts of these bugs -- people component watching can receive notifications of them. Had it not been by change I wouldn't have seen this bug or comment 39 etc :-)
Flags: needinfo?(cdawson)
s/change/chance/
Thanks for merging this, Ed.  You're a rock-star, man.  :)
Dang, sorry I let this sit too long.
(Assignee)

Updated

3 years ago
Depends on: 1241995
Comment on attachment 8705928 [details] [diff] [review]
160108_bug_1219094_drop_api_11_through_14-slave-health.patch

looks like we are good to go with treeherder. patches are landing now
Attachment #8705928 - Flags: checked-in+
(Assignee)

Updated

3 years ago
Attachment #8705927 - Flags: checked-in+
(Assignee)

Updated

3 years ago
Attachment #8705926 - Flags: checked-in+
Created attachment 8711163 [details] [diff] [review]
160122_bug_1219094_drop_api11_fix_base_jobs.patch

apparently I forgot at least one spot ;)

exception in task graph generation:

TemplatesException: "/Users/jlund/devel/mozilla/cleanRepos/mozilla-inbound/testing/taskcluster/tasks/builds/android_api_11_b2gdroid.yml" is not a file
Attachment #8711163 - Flags: review?(rail)
Comment on attachment 8711163 [details] [diff] [review]
160122_bug_1219094_drop_api11_fix_base_jobs.patch

r+ by garndt via irc. thanks!
Attachment #8711163 - Flags: review?(rail) → review+

Comment 51

3 years ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/3752abafdf19
https://hg.mozilla.org/mozilla-central/rev/6a250cbf8c2c
Status: NEW → RESOLVED
Last Resolved: 3 years ago
status-firefox46: affected → fixed
Resolution: --- → FIXED
Are there any plans to remove directories like mobile/android/config/mozconfigs/android-api-11/ to avoid future confusion ?
Depends on: 1244372

Updated

3 years ago
Duplicate of this bug: 1244463
(In reply to Nick Thomas [:nthomas] from comment #52)
> Are there any plans to remove directories like
> mobile/android/config/mozconfigs/android-api-11/ to avoid future confusion ?

yep. I'll remove all of these for central and aurora now. needed an overlap last week where they both existed.
(In reply to Jordan Lund (:jlund) from comment #54)
> (In reply to Nick Thomas [:nthomas] from comment #52)
> > Are there any plans to remove directories like
> > mobile/android/config/mozconfigs/android-api-11/ to avoid future confusion ?
> 
> yep. I'll remove all of these for central and aurora now. needed an overlap
> last week where they both existed.

this is done now. removed api-11 mozconfigs on inbound and aurora
Depends on: 1245796
Depends on: 1245966
(Assignee)

Updated

3 years ago
Blocks: 1262000
See Also: → bug 1384482
Component: Platform Support → Buildduty
Product: Release Engineering → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.