1414021 - Autophone finding incorrect APK -- geckoview_example.apk rather than target.apk

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: Testing, defect, P1)

Description

[Nick Alexander :nalexander [he/him] (PTO Feb 24-March 3)]

Bug 1405396 "switched" the Gradle and non-Gradle builds: the regular B builds are now --with-gradle, and the Bg (now Bng) build is --without-gradle.  Gradle builds produce geckoview_example.apk as well as target.apk.  Autophone appears to be finding the wrong APK (just like Robocop did, moons ago) -- possibly because it's enumerating candidate APKs alphabetically.

See the wall of orange and red at https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=36599de1793a3233141452e25fa73eb29129f62a.

Comment 1

[Nick Alexander :nalexander [he/him] (PTO Feb 24-March 3)]

bc: can you look at this ASAP?  I really can't make much sense of the autophone APK finding code -- it looks like it's completely independent from the Task Cluster job-dependency integration -- and I don't see how the APKs found at https://github.com/mozilla/autophone/blob/master/builds.py#L400 get ordered or chosen.

Comment 2

[Intermittent Failures Robot]

131 failures in 143 pushes (0.916 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* autoland: 131

Platform breakdown:
* android-7-1-armv8-api16: 45
* android-4-4-armv7-api16: 38
* android-6-0-armv8-api16: 36
* android-4-2-armv7-api16: 12

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1414021&startday=2017-11-02&endday=2017-11-02&tree=all

Comment 3

[Bob Clary [:bc] (inactive)]

Attachment: bug-1414021-configs-v1.patch

Stop testing the android-api-16-gradle platform (which is currently non-gradle) and consolidate configs on android-api-16.

Comment 4

[Bob Clary [:bc] (inactive)]

Attachment: bug-1414021-multiple-apps-v1.patch

Change to allow testing multiple apps on a given build. Currently restricted to fennec and geckoview_example. This could be generalized but I'd rather handle that in autophone2.

Comment 5

[Bob Clary [:bc] (inactive)]

test runs:

https://treeherder.allizom.org/#/jobs?repo=autoland&exclusion_profile=false&group_state=expanded&fromchange=0fd6f8ab2c89fe5766e1b4339a675f14037f41e6&filter-searchStr=autophone&tochange=d69df5cdaca312ca854dd3b5f691d525d3670480

https://treeherder.allizom.org/#/jobs?repo=mozilla-inbound&filter-searchStr=autophone&exclusion_profile=false&group_state=expanded&tochange=83f0e033450d1dce9a9ce96b37b465ae50dacdb6&fromchange=899a407832c09e685748779d288637a24aaae2af

https://treeherder.allizom.org/#/jobs?repo=try&revision=5b83d52a8a5693960524aa0d87075c60d130f0fd&filter-searchStr=autophone&group_state=expanded

https://treeherder.allizom.org/#/jobs?repo=try&revision=4f702d4c58b81e5b8b9765d3887e71681b3d82dd&group_state=expanded

tested with trigger_runs.py and consuming pulse.

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8925392 [details] [diff] [review]
bug-1414021-multiple-apps-v1.patch

Review of attachment 8925392 [details] [diff] [review]:
-----------------------------------------------------------------

a bit of rubber stamping involved here.

Comment 7

[Bob Clary [:bc] (inactive)]

https://github.com/mozilla/autophone/commit/83c15f092965f15c34bef6ca3b808167f31af145
https://github.com/mozilla/autophone/commit/d571d8f504b4f70974b32cf55819599d082177d6

yeah. sorry. I hope we can clean this up later. Cleaning out the buildbot and ftp stuff would be easy but not a priority atm.

Comment 8

[Intermittent Failures Robot]

134 failures in 857 pushes (0.156 failures/push) were associated with this bug in the last 7 days. 

This is the #11 most frequent failure this week. 

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **   

Repository breakdown:
* autoland: 134

Platform breakdown:
* android-7-1-armv8-api16: 48
* android-4-4-armv7-api16: 41
* android-6-0-armv8-api16: 35
* android-4-2-armv7-api16: 10

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1414021&startday=2017-10-30&endday=2017-11-05&tree=all