1285040 - devicemanagerADB pushDir (without zip) pushes to sub-directory with latest adb

Status: RESOLVED FIXED

(Testing :: Mozbase, defect)

Description

[Geoff Brown [:gbrown]]

The adb executable in the latest Android SDK has changed the behavior of "push" with respect to directories when the remote directory exists. Consider:

OLD ADB

gbrown@mozpad:~/src$ ~/arch/adb-18-1_0_31 shell ls -lR /mnt/sdcard/tests/new
/mnt/sdcard/tests/new: No such file or directory
gbrown@mozpad:~/src$ ~/arch/adb-18-1_0_31 push zzz /mnt/sdcard/tests/new/
push: zzz/f1 -> /mnt/sdcard/tests/new/f1
1 file pushed. 0 files skipped.
0 KB/s (6 bytes in 0.046s)
gbrown@mozpad:~/src$ ~/arch/adb-18-1_0_31 shell ls -lR /mnt/sdcard/tests/new

/mnt/sdcard/tests/new:
----rwxr-x system   sdcard_rw        6 2016-07-06 14:41 f1
gbrown@mozpad:~/src$ ~/arch/adb-18-1_0_31 shell rm -r /mnt/sdcard/tests/new
gbrown@mozpad:~/src$ ~/arch/adb-18-1_0_31 shell mkdir /mnt/sdcard/tests/new
gbrown@mozpad:~/src$ ~/arch/adb-18-1_0_31 shell ls -lR /mnt/sdcard/tests/new

/mnt/sdcard/tests/new:
gbrown@mozpad:~/src$ ~/arch/adb-18-1_0_31 push zzz /mnt/sdcard/tests/new/
push: zzz/f1 -> /mnt/sdcard/tests/new/f1
1 file pushed. 0 files skipped.
0 KB/s (6 bytes in 0.042s)
gbrown@mozpad:~/src$ ~/arch/adb-18-1_0_31 shell ls -lR /mnt/sdcard/tests/new

/mnt/sdcard/tests/new:
----rwxr-x system   sdcard_rw        6 2016-07-06 14:41 f1


NEW ADB

gbrown@mozpad:~/src$ adb shell ls -lR /mnt/sdcard/tests/new
/mnt/sdcard/tests/new: No such file or directory
gbrown@mozpad:~/src$ adb push zzz /mnt/sdcard/tests/new/
/mnt/sdcard/tests/new/: 1 file pushed. 0 files skipped. 0.0 MB/s (6 bytes in 0.017s)
gbrown@mozpad:~/src$ adb shell ls -lR /mnt/sdcard/tests/new

/mnt/sdcard/tests/new:
----rwxr-x system   sdcard_rw        6 2016-07-06 14:41 f1
gbrown@mozpad:~/src$ adb shell rm -r /mnt/sdcard/tests/new
gbrown@mozpad:~/src$ adb shell mkdir /mnt/sdcard/tests/new
gbrown@mozpad:~/src$ adb shell ls -lR /mnt/sdcard/tests/new

/mnt/sdcard/tests/new:
gbrown@mozpad:~/src$ adb push zzz /mnt/sdcard/tests/new/
/mnt/sdcard/tests/new/zzz/: 1 file pushed. 0 files skipped. 0.0 MB/s (6 bytes in 0.014s)
gbrown@mozpad:~/src$ adb shell ls -lR /mnt/sdcard/tests/new

/mnt/sdcard/tests/new:
d---rwxr-x system   sdcard_rw          2016-07-06 15:19 zzz

/mnt/sdcard/tests/new/zzz:
----rwxr-x system   sdcard_rw        6 2016-07-06 14:41 f1


This changes the behavior of DeviceManagerADB's pushDir() when zip is not used. Note that pushDir always creates the destination directory before issuing "adb push". Consequently, when pushDir() is called in an environment that uses the new adb, the directory will be pushed to a sub-directory (named "tmp" because of the copytree() in pushDir) of the intended destination.


Test harnesses use pushDir to push the profile directory to the device. Pushing the profile to a sub-directory creates havoc: When Firefox is started, it finds the profile directory exists, but none of the test profile files are in the right place -- they are in a "tmp" sub-directory.

Comment 1

[Geoff Brown [:gbrown]]

I verified this behavior with NEW adb 1.0.36 and OLD adb (1.0.31 and 1.0.32).

Comment 3

[Geoff Brown [:gbrown]]

Attachment: adapt devicemanagerADB pushDir to new adb

My thinking here is that "old" and "new" adb push differ when the destination directory exists -- so make sure it doesn't exist before pushing.

Of course, deleting the destination directory in pushDir changes the behavior when there are 2 or more pushDir's to the same destination. There was a case of that in the xpcshell harness: It pushed to a /sdcard/tests/xpc/c first and then /sdcard/tests/xpc: reversing the order of pushDir's put things right again.

I expect I'll need to do something similar for autophone -- will handle in a separate bug.

Comment 4

[Geoff Brown [:gbrown]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=c2a68e094746 looks okay to me.

Comment 5

[Bob Clary [:bc] (inactive)]

Comment on attachment 8768978 [details] [diff] [review]
adapt devicemanagerADB pushDir to new adb

Review of attachment 8768978 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/mozbase/mozdevice/mozdevice/devicemanagerADB.py
@@ +270,5 @@
>                  self._useZip = False
>                  self.pushDir(localDir, remoteDir, retryLimit=retryLimit, timeout=timeout)
>          else:
> +            # If the remote directory exists, newer implementations of "adb push" will
> +            # create a sub-directory, while older versions will not!

Maybe add "See bug 1285040."

::: testing/xpcshell/remotexpcshelltests.py
@@ +270,5 @@
>  
>          if options.localAPK:
>              self.localAPKContents = ZipFile(options.localAPK)
>          if options.setup:
> +            self.setupTestDir()

Why is this needed?

Comment 6

[Geoff Brown [:gbrown]]

(In reply to Bob Clary [:bc:] from comment #5)
> ::: testing/xpcshell/remotexpcshelltests.py
> @@ +270,5 @@
> >  
> >          if options.localAPK:
> >              self.localAPKContents = ZipFile(options.localAPK)
> >          if options.setup:
> > +            self.setupTestDir()
> 
> Why is this needed?

setupTestDir() calls pushDir(xxx, ".../tests/xpc")
setupModules() calls pushDir(zzz, ".../tests/xpc/m")

Now that pushDir() deletes the destination directory, I needed to ensure that tests/xpc is populated first and its sub-directories later.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/e6dcdb2b8651
Adapt device manager pushDir to latest adb; r=bc

Comment 8

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/e6dcdb2b8651