Closed Bug 697486 Opened 11 years ago Closed 4 years ago

mozrunner should have a variant class for running fennec remotely

Categories

(Testing :: Mozbase, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: wlach, Unassigned)

References

Details

(Whiteboard: [mozbase][mozrunner])

We'd like to use the mozprocess API in things like talos, but that requires us handling the remote case for launching a subprocess on an externally connected device. See:

https://github.com/mozilla/talos/blob/master/bcontroller.py

for an example of controlling a subprocess on a remote device using devicemanagerADB.py or devicemanagerSUT.py.
If we're going to do this properly, we should really make devicemanager and friends a mozbase module that mozprocess_remote can depend on.
Depends on: 697492
So it turns out that subclassing mozprocess is not quite what we want, as its semantics don't really cleanly map onto what we can do with the remote case (i.e. there's no way we can reliably tell immediately that a process has launched, or get its standard output). 

Instead, let's create a subclass of mozrunner -- this exposes much simpler semantics around stopping/starting/waiting for a browser to do its thing.
Summary: mozprocess should have a mozprocess_remote subclass for remote subprocesses → mozrunner should have a variant class for running fennec remotely
I've committed my work in progress on this here:

https://github.com/wlach/mozbase/tree/mozdevice

This tree also contains the work on mozdevice, which this depends on.

I believe the next step with this is to give the existing (local) mozrunner class identical semantics to my remote one (and/or rebase them both on a common abstract base class which exposes their common subset of functionality).  

I wouldn't trust anything done here without a real-world test case using both variants. I've started work on a version of talos which uses this in bug 698898.

The big differences between remote and local:

* Remote doesn't have any concept of a "logfile". If your program needs to process the standard output of your browser, then you'll need to write a firefox extension and specify where it goes (this is what talos does AFAICT).
* Remote doesn't have any concept of controlling a running process. All it can do is start a process, check if one is running (not necessarily the one it launched!), and kill the process (same limitation)
Blocks: 698898
(In reply to William Lachance (:wlach) from comment #3)

> The big differences between remote and local:
> 
> * Remote doesn't have any concept of a "logfile". If your program needs to
> process the standard output of your browser, then you'll need to write a
> firefox extension and specify where it goes (this is what talos does AFAICT).
> * Remote doesn't have any concept of controlling a running process. All it
> can do is start a process, check if one is running (not necessarily the one
> it launched!), and kill the process (same limitation)

This makes sense.  Different agents/mechanisms for talking to the device under test will allow us different capabilities.  The important thing here is that talos use the devicemanager* apis so that we can ensure that whatever the agent supports, talos can use well.  This may mean that to do this, we have to change devicemanager* apis as well as fixing talos to use them.  Does that make sense?
(In reply to Clint Talbert ( :ctalbert ) from comment #4)
> (In reply to William Lachance (:wlach) from comment #3)
> 
> > The big differences between remote and local:
> > 
> > * Remote doesn't have any concept of a "logfile". If your program needs to
> > process the standard output of your browser, then you'll need to write a
> > firefox extension and specify where it goes (this is what talos does AFAICT).
> > * Remote doesn't have any concept of controlling a running process. All it
> > can do is start a process, check if one is running (not necessarily the one
> > it launched!), and kill the process (same limitation)
> 
> This makes sense.  Different agents/mechanisms for talking to the device
> under test will allow us different capabilities.  The important thing here
> is that talos use the devicemanager* apis so that we can ensure that
> whatever the agent supports, talos can use well.  This may mean that to do
> this, we have to change devicemanager* apis as well as fixing talos to use
> them.  Does that make sense?

Talos currently uses the devicemanager* APIs directly, though I think ideally it would only use them through mozrunner (since that should provide everything we need for interacting with the device and running tests).

I think it's best if we write harnesses (including talos) against "the lowest common denominator". That way we wouldn't have to do any special casing for mobile/desktop inside the harnesses: just use the appropriate mozrunner class for the platform you want to test and everything will just work.
I've been working on this a bit for Eideticker. The code in question is here:

https://github.com/wlach/mozbase/blob/mozrunner-remote/mozrunner/mozrunner/remoterunner.py
There's been a bunch of work on 870876 to make mozrunner work for b2g, but this bug still applies.
Depends on: 870876
Whiteboard: [mozbase] → [mozbase][mozrunner]
Mass closing bugs with no activity in 2+ years. If this bug is important to you, please re-open.
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.