1298730 - Autophone - properly initialize profile during smoketests

Status: RESOLVED FIXED

(Testing Graveyard :: Autophone, defect)

Description

[Bob Clary [:bc] (inactive)]

https://treeherder.allizom.org/#/jobs?repo=mozilla-inbound&revision=bfd9274acb82b0c829f20c6d8143520ebe95d5ca&filter-searchStr=autophone&selectedJob=31993809

shows a failure during smoketest for debug Android 6.0

The root cause is seen in https://autophone-dev.s3.amazonaws.com/v1/task/XDiHdOluRU6YFm1IQdD16w/runs/0/artifacts/public/build/autophone-smoketest-smoketest-settings.ini-1-nexus-6p-6-08f216cd-acec-481b-9680-de87dce799be-autophone.log

2016-08-28 17:35:01,572|21060|MainThread|phonetest|ERROR|nexus-6p-6|20160828034206|autophone-smoketest|Attempt 1 Exception installing profile to /sdcard/tests/autophone/profile
Traceback (most recent call last):
  File "/mozilla/projects/autophone/src/bclary-autophone/phonetest.py", line 1055, in install_profile
    self.dm.chmod(profile_path_parent, root=root)
  File "/mozilla/projects/autophone/src/bclary-autophone/adb.py", line 1425, in chmod
    timeout=timeout, root=root)
  File "/mozilla/projects/autophone/src/bclary-autophone/adb.py", line 1123, in shell_output
    raise ADBError("%s" % adb_process)
ADBError: args: adb -s 84B7N15A28010563 wait-for-device shell su 0 chmod 777 /sdcard/tests/autophone; echo rc=$?, exitcode: 1, stdout: chmod: /sdcard/tests/autophone: No such file or directory, stderr: 

Autophone fails to detect the failure and continue to attempt to run the test resulting in a crash due to a non-local network access.

Comment 1

[Bob Clary [:bc] (inactive)]

Attachment: bug-1298730-v1.patch

Adds the sources to the config file, cleans up the PhoneTest initializer a bit and explicitly installs the local pages and creates the profile in the same fashion as s1s2.

example run (with some additional unrelated patches)

https://treeherder.allizom.org/#/jobs?repo=mozilla-inbound&revision=aef078d44124e895091e7341ae348fe3bb0c2255&filter-searchStr=autophone&group_state=expanded

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8785765 [details] [diff] [review]
bug-1298730-v1.patch

Review of attachment 8785765 [details] [diff] [review]:
-----------------------------------------------------------------

I just have one question- possibly if my question is that the exception status is proper, we should rename the variable.

::: tests/smoketest.py
@@ +20,4 @@
>          # Clear logcat
>          self.logcat.clear()
>  
> +        is_test_completed = True

why is this moved ahead of the test?  Will the PhoneTestResult.EXCEPTION/BUSTED do the proper error handling up the food chain?

Comment 3

[Bob Clary [:bc] (inactive)]

We only want to consider the test not completed if it was interrupted by the user/administrator so that it will be retried. In the other cases the test should be considered to have been completed although possibly as a failure.

Comment 4

[Bob Clary [:bc] (inactive)]

Attachment: bug-1298730-v2.patch

Fixes the real problem of the use of an absolute host path in

         except (ConfigParser.NoSectionError, ConfigParser.NoOptionError):
-            self._paths['sources'] = [
-                os.path.join(self.autophone_directory, 'files/base/')]
+            self._paths['sources'] = ['files/base/']

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8787079 [details] [diff] [review]
bug-1298730-v2.patch

Review of attachment 8787079 [details] [diff] [review]:
-----------------------------------------------------------------

just one small detail, otherwise this looks good.

::: tests/smoketest.py
@@ +23,5 @@
> +        is_test_completed = True
> +
> +        if not self.install_local_pages():
> +            self.test_failure(
> +                self.name, 'TEST_UNEXPECTED_FAIL',

should this be - instead of _ to make TEST-UNEXPECTED-FAIL ?

Comment 6

[Bob Clary [:bc] (inactive)]

It should be, but I left the bug in at this stage since it was going to be fixed in Bug 1298732.

Comment 7

[Bob Clary [:bc] (inactive)]

Good catch though!

Comment 8

[Bob Clary [:bc] (inactive)]

https://github.com/mozilla/autophone/commit/7dc8e3ab6b9000222a5a9124873a4a8a7f0b6720

Comment 9

[Bob Clary [:bc] (inactive)]

deployed 2016-09-04 ~13:40 PDT