Periodically *some* workers will fail to get the split test zips. It seems for some reason they attempt to download the old combined tests.zip. I wonder if a temporary network issue on the server causing autophone to fall back to the old style tests? http://archive.mozilla.org/pub/mobile/tinderbox-builds/mozilla-central-android-api-15/1456830164/fennec-47.0a1.en-US.android-arm.apk: Zip file error retrieving tests: http://archive.mozilla.org/pub/mobile/tinderbox-builds/mozilla-central-android-api-15/1456830164/fennec-47.0a1.en-US.android-arm.tests.zip.
Created attachment 8725492 [details] [diff] [review] bug-1252517-v1.patch https://github.com/mozilla/autophone/blob/master/builds.py#L707 seems totally wrong: test_packages = utils.get_remote_json( urlparse.urljoin(buildurl, 'test_packages.json')) buildurl is the url to the apk for the build. See http://archive.mozilla.org/pub/mobile/tinderbox-builds/mozilla-inbound-android-api-15/1456875034/fennec-47.0a1.en-US.android-arm.test_packages.json as an example. I don't know how this ever successfully downloaded the proper tests zips. Example test runs: https://treeherder.allizom.org/#/jobs?filter-searchStr=autophone&exclusion_profile=false&repo=mozilla-inbound&filter-tier=1&filter-tier=2&filter-tier=3&fromchange=863c6740f637&tochange=93c0e8939efa
Attachment #8725492 - Flags: review?(gbrown)
Summary: Autophone - periodically Autophone will attempt to download old-style tests.zip and fail → Autophone - uses incorrect url to download test_packages.json
Comment on attachment 8725492 [details] [diff] [review] bug-1252517-v1.patch Review of attachment 8725492 [details] [diff] [review]: ----------------------------------------------------------------- Looks good to me. There are some logging statements following your change, but they are all debug. Any interest in making those info, or otherwise making the test package names more visible?
Attachment #8725492 - Flags: review?(gbrown) → review+
That's a great idea. I'll change them, run a quick test to make sure I haven't had a typo/thinko and will submit that. Thanks!
https://github.com/mozilla/autophone/commit/2472b8ba41d6dd52bfa6cc8fc171268dfd2a3408 deployed 2016-03-02 01:43
Status: ASSIGNED → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
I wasn't a complete idiot before. They are changing the way the test_packages.json is generated perhaps depending on the phase of the moon or a random number or something. http://email@example.com/try-android-api-15/ has the old style file name so we need to support both. esawin: Your try run from this morning won't run properly until I fix this completely.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Very strange. Looks like we should use something like http://hg.mozilla.org/mozilla-central/annotate/bdde3fedb45b/testing/mozharness/mozharness/mozilla/building/buildbase.py#l1524: ('testPackagesUrl', lambda m: m.endswith('test_packages.json')),
We don't iterate over the directory contents so I'm not sure that approach would help. I am testing a fall back approach now while tweaking the logger output a bit. I'll have a patch shortly.
Created attachment 8725845 [details] [diff] [review] bug-1252517-tests-packages-followup-v1.patch test runs: https://treeherder.allizom.org/#/jobs?repo=mozilla-inbound&revision=36f909ca9e23&filter-searchStr=autophone&exclusion_profile=false&filter-tier=1&filter-tier=2&filter-tier=3 https://treeherder.allizom.org/#/jobs?repo=try&revision=9cc090d26a79&exclusion_profile=false&filter-tier=1&filter-tier=2&filter-tier=3 I ran with two versions of the patch. The second modified the fallback logging and added the error return if no test files are found at all.
Attachment #8725845 - Flags: review?(gbrown)
deployed 2016-03-02 17:43-44. esawin, I'll cancel and retrigger your try jobs.
Status: REOPENED → RESOLVED
Last Resolved: 3 years ago → 3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.