1252517 - Autophone - uses incorrect url to download test_packages.json

Status: RESOLVED FIXED

(Testing Graveyard :: Autophone, defect)

Description

[Bob Clary [:bc] (inactive)]

Periodically *some* workers will fail to get the split test zips. It seems for some reason they attempt to download the old combined tests.zip. I wonder if a temporary network issue on the server causing autophone to fall back to the old style tests?

http://archive.mozilla.org/pub/mobile/tinderbox-builds/mozilla-central-android-api-15/1456830164/fennec-47.0a1.en-US.android-arm.apk: Zip file error retrieving tests: http://archive.mozilla.org/pub/mobile/tinderbox-builds/mozilla-central-android-api-15/1456830164/fennec-47.0a1.en-US.android-arm.tests.zip.

Comment 1

[Bob Clary [:bc] (inactive)]

Attachment: bug-1252517-v1.patch

https://github.com/mozilla/autophone/blob/master/builds.py#L707 seems totally wrong:

            test_packages = utils.get_remote_json(
                urlparse.urljoin(buildurl, 'test_packages.json'))

buildurl is the url to the apk for the build.

See http://archive.mozilla.org/pub/mobile/tinderbox-builds/mozilla-inbound-android-api-15/1456875034/fennec-47.0a1.en-US.android-arm.test_packages.json as an example. I don't know how this ever successfully downloaded the proper tests zips.

Example test runs:

https://treeherder.allizom.org/#/jobs?filter-searchStr=autophone&exclusion_profile=false&repo=mozilla-inbound&filter-tier=1&filter-tier=2&filter-tier=3&fromchange=863c6740f637&tochange=93c0e8939efa

Comment 2

[Geoff Brown [:gbrown]]

Comment on attachment 8725492 [details] [diff] [review]
bug-1252517-v1.patch

Review of attachment 8725492 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good to me.

There are some logging statements following your change, but they are all debug. Any interest in making those info, or otherwise making the test package names more visible?

Comment 3

[Bob Clary [:bc] (inactive)]

That's a great idea. I'll change them, run a quick test to make sure I haven't had a typo/thinko and will submit that.

Thanks!

Comment 4

[Bob Clary [:bc] (inactive)]

https://github.com/mozilla/autophone/commit/2472b8ba41d6dd52bfa6cc8fc171268dfd2a3408
deployed 2016-03-02 01:43

Comment 5

[Bob Clary [:bc] (inactive)]

I wasn't a complete idiot before. They are changing the way the test_packages.json is generated perhaps depending on the phase of the moon or a random number or something.

http://archive.mozilla.org/pub/mobile/try-builds/[e-mail]-9cc090d26a795a72679677e0f3dff83e86070568/try-android-api-15/ has the old style file name so we need to support both.

esawin: Your try run from this morning won't run properly until I fix this completely.

Comment 6

[Geoff Brown [:gbrown]]

Very strange.

Looks like we should use something like http://hg.mozilla.org/mozilla-central/annotate/bdde3fedb45b/testing/mozharness/mozharness/mozilla/building/buildbase.py#l1524:

            ('testPackagesUrl', lambda m: m.endswith('test_packages.json')),

Comment 7

[Bob Clary [:bc] (inactive)]

We don't iterate over the directory contents so I'm not sure that approach would help. I am testing a fall back approach now while tweaking the logger output a bit. I'll have a patch shortly.

Comment 8

[Bob Clary [:bc] (inactive)]

Attachment: bug-1252517-tests-packages-followup-v1.patch

test runs:

https://treeherder.allizom.org/#/jobs?repo=mozilla-inbound&revision=36f909ca9e23&filter-searchStr=autophone&exclusion_profile=false&filter-tier=1&filter-tier=2&filter-tier=3

https://treeherder.allizom.org/#/jobs?repo=try&revision=9cc090d26a79&exclusion_profile=false&filter-tier=1&filter-tier=2&filter-tier=3

I ran with two versions of the patch. The second modified the fallback logging and added the error return if no test files are found at all.

Comment 9

[Bob Clary [:bc] (inactive)]

deployed 2016-03-02 17:43-44. esawin, I'll cancel and retrigger your try jobs.

Comment 10

[Bob Clary [:bc] (inactive)]

https://github.com/mozilla/autophone/commit/bf9809382d4d8322a85c6e0fe17f8af9b48c0ed0