1513078 - Unable to run web platform test in local Android build on physical device

Status: RESOLVED DUPLICATE of bug 1560033

(Testing :: web-platform-tests, defect)

Description

[Botond Ballo [:botond]]

Attachment: Output from |mach wpt|

I'm trying to run a web platform test in my local Android build. Attached is the output from the |mach wpt| command I'm trying to run.

The error seems to be:

IOError: No devices connected. Ensure the device is on and remote debugging via adb is enabled in the settings.

However, the device is connected and adb is enabled; other mach commands that use adb, like |mach install|, work fine.

Comment 1

[Botond Ballo [:botond]]

Geoff, any suggestions based on the output for what might be the issue, or how to diagnose further?

Comment 2

[Geoff Brown [:gbrown]]

I'd be interested in seeing the output from 'adb devices'. Also 'mach -v wpt ...' might give us more info.

wpt on android is pretty new and might not be as robust as some test harnesses; I'll have a closer look tomorrow.

Comment 3

[Botond Ballo [:botond]]

Attachment: Output from |mach wpt| and |adb devices|

Attached is some updated output. It shows |adb devices| successfully finding my device both before and after the |mach wpt| command. I also passed -v to mach as mentioned, although it doesn't seem to have made the output any more verbose?

Comment 4

[Geoff Brown [:gbrown]]

It turns out that running wpt is only supported on emulators: The wpt harness unconditionally creates a FennecEmulatorRunner

https://dxr.mozilla.org/mozilla-central/rev/c2593a3058afdfeaac5c990e18794ee8257afe99/testing/web-platform/tests/tools/wptrunner/wptrunner/browsers/fennec.py#221

which intentionally rejects non-emulator devices.

Marionette tests also use FennecEmulatorRunner, but conditionally. That change could probably be made easily. But honestly, I don't understand why this code makes a distinction at all. Looking into it...

(As a workaround, you should have a better experience with an emulator: Unplug your device and re-run mach wpt -- it should offer to start an appropriate emulator for you.)

Comment 5

[Botond Ballo [:botond]]

(In reply to Geoff Brown [:gbrown] from comment #4)
> (As a workaround, you should have a better experience with an emulator:
> Unplug your device and re-run mach wpt -- it should offer to start an
> appropriate emulator for you.)

Thanks - that got me further. However, I'm now running into bug 1513695.

Comment 6

[Geoff Brown [:gbrown]]

Attachment: allow connection to any device

I'm having trouble finding a functional test device, so I cannot properly test this patch, but it *might* work.

:botond - Give this a try?

Comment 7

[Geoff Brown [:gbrown]]

(In reply to Geoff Brown [:gbrown] from comment #4)
> Marionette tests also use FennecEmulatorRunner, but conditionally.

Actually, it looks like marionette-test has the same limitation: emulator only. mozrunner makes some internal allowance for non-emulator devices, but doesn't seem to expose an interface to that.

Comment 8

[Botond Ballo [:botond]]

Comment on attachment 9030943 [details] [diff] [review]
allow connection to any device

With this patch, I still get an error, but it's a slightly different error.

Comment 9

[Botond Ballo [:botond]]

Attachment: Output from |mach wpt| with the patch applied

Attached is the output with the patch applied.

The error now seems to be:

ADBError: Unable to remount device

Comment 10

[Geoff Brown [:gbrown]]

(In reply to Botond Ballo [:botond] from comment #9)
> ADBError: Unable to remount device

wpt needs to update the device's /etc/hosts so that tests can reference test host names and have those directed to the desktop host. /etc is typically on the /system partition, which is typically read-only. The wpt harness runs 'adb remount' to mount /system read-write, then pushes to /etc/hosts; on your device, 'adb remount' fails.

There are reports that 'mount -o rw,remount /system' succeeds sometimes where 'adb remount' fails; this is probably not fool-proof, but worth a try...

Comment 11

[Geoff Brown [:gbrown]]

Attachment: allow connection to any device, and use an alternative to 'adb remount'

Does this help?

Comment 12

[Botond Ballo [:botond]]

Attachment: Output from |mach wpt| with the updated patch applied

Seems to be fairly stubborn. The error is now:

adb: error: failed to copy '/tmp/tmpBpS1Xy' to '/etc/hosts': remote Read-only file system

Comment 13

[Botond Ballo [:botond]]

Comment on attachment 9032048 [details] [diff] [review]
allow connection to any device, and use an alternative to 'adb remount'

(See above)

Comment 14

[James Graham [:jgraham]]

I winder if we can avoid updating /etc/hosts at all. On desktop we're able to use a pref to override the DNS lookup for names resolving to localhost, but I guess that doesn't work here because the server isn't on localhost. But maybe there's some fancy port forwarding thing we can do that would open up ports on the device and proxy them to the host. Then we could use the Firefox prefs rather than editing the hosts file.

Comment 15

[James Graham [:jgraham]]

https://developer.android.com/studio/command-line/adb#forwardports suggests that this should be possible.

Comment 16

[Geoff Brown [:gbrown]]

Bug 1560033 will remove reliance on /etc/hosts. Physical devices will of course vary in their capabilities and configuration -- we can't guarantee trouble free operation everywhere, but device runs should generally be possible.