1617810 - Add support for multiple android devices attached

Status: RESOLVED FIXED

(Testing :: geckodriver, defect, P3)

Description

[Benjamin De Kosnik [:bdekoz]]

Using geckodriver on a host system with >1 android devices causes an internal error, even if ANDROID_SERIAL set to a specific devices.

Error looks like this in logs:

1582244217920 geckodriver::capabilities DEBUG Extracting profile to /tmp/rust_mozprofilexk7Ckk/extensions/[e-mail]
1582244217920 geckodriver::capabilities DEBUG Extracting profile to /tmp/rust_mozprofilexk7Ckk/key4.db
1582244217922 mozdevice WARN adb server response contained hexstring length 305 and message length was 305 and message was "9C081FFBA002DU device usb:1-2.1 product:coral model:Pixel_4_XL device:coral transport_id:1\nFA79G1A05075 device usb:1-2.2 product:walleye model:Pixel_2 device:walleye transport_id:3\nRF8MB1E9NHB device usb:1-2.3 product:beyond1ltexx model:SM_G973F device:beyond1 transport_id:4\n"
1582244217922 webdriver::server DEBUG <- 500 Internal Server Error {"value":{"error":"unknown error","message":"adb error: ","stacktrace":""}}
[2020-02-20 16:16:57] INFO: Browser failed to start, trying 0 more time(s): adb error:

Host has these devices (via 'adb devices -l')

List of devices attached
9C081FFBA002DU device usb:1-2.1 product:coral model:Pixel_4_XL device:coral transport_id:1
FA79G1A05075 device usb:1-2.2 product:walleye model:Pixel_2 device:walleye transport_id:3
RF8MB1E9NHB device usb:1-2.3 product:beyond1ltexx model:SM_G973F device:beyond1 transport_id:4

Invocation via browsertime, started as follows:

ANDROID_SERIAL1=9C081FFBA002DU

$browsertime_bin
--android
--browser firefox
--firefox.android.deviceSerial "$ANDROID_SERIAL1" \

Expected behavior is same as chromedriver, which is to use the specified ANDROID_SERIAL device.

Source search:
adb server response contained hexstring length

Shows
testing/mozbase/rust/mozdevice/src/lib.rs

Maybe around:

impl Host {
/// Searches for available devices, and selects the one as specified by device_serial.
///
/// If multiple devices are online, and no device has been specified,
/// the ANDROID_SERIAL environment variable can be used to select one.
pub fn device_or_default<T: AsRef<str>>(self, device_serial: Option<&T>) -> Result<Device> {
let serials: Vec<String> = self.devices::<Vec<_>>()?
.into_iter()
.map(|d| d.serial)
.collect();

    if let Some(ref serial) = device_serial
        .map(|v| v.as_ref().to_owned())
        .or_else(|| std::env::var("ANDROID_SERIAL").ok())
    {

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2] (away 10/03 - 10/13)]

Even with that working, it would not be possible to correctly forward the port. See bug 1525126 comment 52:

As I noticed it seems to be not possible to forward ports if multiple devices are attached; even the adb command line doesn't work when specifying -s serial for the adb forward <local> <remote> command. Interestingly adb forward --list works with -s. I would propose that we abort if multiple devices are connected.

Looks like I missed to file a bug for that follow-up. So we can re-use this bug for tracking, or do it as part of bug 1578424.

Comment 2

[Benjamin De Kosnik [:bdekoz]]

Hey Henrik, thanks for chiming in.

I appreciate your attention to this bug, it is a blocker for getting CI working with Fenix performance testing. I'm not in charge of triage but I'd set this as a P2 at least... even a partial solution for on-host USB and no TCP connections would be appreciated.

In the meantime, I could work around this by using one linux VM on the host per device, and only assigning that device's USB connection to the VM. This is a pain, and adds a bunch of complexity to the testing environment that is not required for other browsers (namely chrome).

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2] (away 10/03 - 10/13)]

Can I ask why multiple devices are connected to the machine running the performance tests? Maybe you can lay-out the testing setup a bit more? I would need a clear understanding first before we can do any scheduling. Thanks.

Comment 4

[Beth Rennie [:beth] (she/her)]

I compared our implementation against chromedriver's and the difference seems to be that we don't specify the host we are requesting to port forward (we dispatch host:forward:tcp:1234;tcp:4567 and they dispatch host-serial:SERIAL:forward:....

Updating the Device::execute_host_command to dispatchhost-serial:SERIAL:REQUEST and having Device::forward_port use that instead of Host::execute_host_command seems to do the trick.

e.g.

Barret@FOXYSHAZAM ~/Workspace/src/hg.mozilla.org/mozilla-central/testing/mozbase/rust/mozdevice
$ env ANDROID_SERIAL=ZY322HN7MS cargo test forward -- --test-threads 1
    Finished dev [optimized + debuginfo] target(s) in 0.37s
     Running c:\Users\Barret\Workspace\src\hg.mozilla.org\mozilla-central\target\debug\deps\mozdevice-b1d43699f2d1602e.exe

running 5 tests
test test::device_forward_port_hardcoded ... ok
test test::device_kill_forward_all_ports_no_forwarded_port ... ok
test test::device_kill_forward_all_ports_twice ... ok
test test::device_kill_forward_port_no_forwarded_port ... ok
test test::device_kill_forward_port_twice ... ok

test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 24 filtered out


Barret@FOXYSHAZAM ~/Workspace/src/hg.mozilla.org/mozilla-central/testing/mozbase/rust/mozdevice
$ env ANDROID_SERIAL=8CWX1U0W8 cargo test forward -- --test-threads 1
    Finished dev [optimized + debuginfo] target(s) in 0.37s
     Running c:\Users\Barret\Workspace\src\hg.mozilla.org\mozilla-central\target\debug\deps\mozdevice-b1d43699f2d1602e.exe

running 5 tests
test test::device_forward_port_hardcoded ... ok
test test::device_kill_forward_all_ports_no_forwarded_port ... ok
test test::device_kill_forward_all_ports_twice ... ok
test test::device_kill_forward_port_no_forwarded_port ... ok
test test::device_kill_forward_port_twice ... ok

test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 24 filtered out

I'm still waiting to verify this works in Browsertime (I cant' get it to work on Windows with my motog5), so I haven't uploaded a patch yet.

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2] (away 10/03 - 10/13)]

This is great to hear. Thanks for investigating that. I'm kinda looking forward to a patch. Thank you in advance.

Comment 6

[Benjamin De Kosnik [:bdekoz]]

As far as testing goes, we have a linux host with 3 devices, and it rolls through an android test harness such that a different device is tested every hour or so, with built-in buffer/quite time.

Barret, thanks for your work on this.

Comment 7

[Beth Rennie [:beth] (she/her)]

Attachment: Bug 1617810 - Specify host serial when dispatching adb requests r?#webdriver-reviewers

Depends on D65176

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/440429bc41ed
Specify host serial when dispatching adb requests r=webdriver-reviewers,whimboo

Comment 9

[Dorel Luca [:dluca]]

https://hg.mozilla.org/mozilla-central/rev/440429bc41ed