Last Comment Bug 749011 - Add support for running mochitest-plain on B2G
: Add support for running mochitest-plain on B2G
Status: RESOLVED FIXED
:
Product: Testing
Classification: Components
Component: Mochitest (show other bugs)
: unspecified
: All All
: -- normal (vote)
: mozilla15
Assigned To: Jonathan Griffin (:jgriffin)
:
:
Mentors:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2012-04-25 16:08 PDT by Jonathan Griffin (:jgriffin)
Modified: 2012-04-29 14:18 PDT (History)
3 users (show)
ryanvm: in‑testsuite+
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---


Attachments
b2g mochitest automation support (28.29 KB, patch)
2012-04-26 18:44 PDT, Jonathan Griffin (:jgriffin)
jmaher: review+
Details | Diff | Splinter Review

Description Jonathan Griffin (:jgriffin) 2012-04-25 16:08:54 PDT
We want to be able to run mochitest-plain on B2G.  To do that, we'll need B2G versions of remoteautomation.py and runtestsremote.py, among other things.
Comment 1 Joel Maher ( :jmaher) 2012-04-26 10:58:59 PDT
we don't run ssltunnel tests remotely or websocket tests remotely.  Currently we don't run mochitest-chrome and for browser-chrome everything is bundled in an extension and run off the device only.

Also to consider is the host machine running the tests and serving the files via http.  

If our core changes to the system can happen in devicemanager*.py, then we don't have to change much in our harnesses.  For Robocop I had to change harnesses and devicemanager*.py, I expect similar stuff here as well.
Comment 2 Jonathan Griffin (:jgriffin) 2012-04-26 18:12:01 PDT
(In reply to Joel Maher (:jmaher) from comment #1)
> we don't run ssltunnel tests remotely or websocket tests remotely. 
> Currently we don't run mochitest-chrome and for browser-chrome everything is
> bundled in an extension and run off the device only.
> 
> Also to consider is the host machine running the tests and serving the files
> via http.  
> 
> If our core changes to the system can happen in devicemanager*.py, then we
> don't have to change much in our harnesses.  For Robocop I had to change
> harnesses and devicemanager*.py, I expect similar stuff here as well.

We won't run browser-chrome tests either, since we don't have any of the same chrome as Firefox.  I'm not sure whether it's worth trying to get the mochitest-chrome tests working or not; I'm guessing not if you're not using them for Fennec.

I'm attaching copies of b2gautomation.py (modeled after remoteautomation.py) and runtestsb2g.py (modeled after runtestsremote.py).  This works well locally for mochitest-plain, using the 'android.json' manifest of tests that have been vetted for Fennec.

Chunk 1/8 has these results:

I/GeckoDump( 2611): 21232 INFO TEST-START | Shutdown
I/GeckoDump( 2611): 21233 INFO Passed: 20587
I/GeckoDump( 2611): 21234 INFO Failed: 5
I/GeckoDump( 2611): 21235 INFO Todo:   211
I/GeckoDump( 2611): 21236 INFO SimpleTest FINISHED

The current code works well enough that it puts me in a position to be able to start working on the continuous integration bits, which is up next.
Comment 3 Jonathan Griffin (:jgriffin) 2012-04-26 18:44:45 PDT
Created attachment 618888 [details] [diff] [review]
b2g mochitest automation support
Comment 4 Joel Maher ( :jmaher) 2012-04-26 19:12:03 PDT
Comment on attachment 618888 [details] [diff] [review]
b2g mochitest automation support

Review of attachment 618888 [details] [diff] [review]:
-----------------------------------------------------------------

Not too many nits, but a few.  As I mentioned in IRC, there is a lot of duplicated code.  I can't see an easy way to reuse the majority of the code without ugly hacks.  My goal is that when we are running these harnesses with mozbase, that can shed light on a better solution for reusing code.

::: build/mobile/b2gautomation.py
@@ +33,5 @@
> +# and other provisions required by the GPL or the LGPL. If you do not delete
> +# the provisions above, a recipient may use your version of this file under
> +# the terms of any one of the MPL, the GPL or the LGPL.
> +#
> +# ***** END LICENSE BLOCK *****

MPL 2.0?  I am not a contributor to this, except by reputation!

@@ +94,5 @@
> +        if crashreporter:
> +            env['MOZ_CRASHREPORTER_NO_REPORT'] = '1'
> +            env['MOZ_CRASHREPORTER'] = '1'
> +        else:
> +            env['MOZ_CRASHREPORTER_DISABLE'] = '1'

are these environment variables used in b2g?

@@ +119,5 @@
> +            args.remove('-foreground')
> +        except:
> +            pass
> +#TODO: figure out which platform require NO_EM_RESTART
> +#        return app, ['--environ:NO_EM_RESTART=1'] + args

since this is a unique file, you could remove this TODO.

@@ +156,5 @@
> +        serial = serial or self._devicemanager.deviceSerial
> +        status = 'unknown'
> +
> +        for line in self._devicemanager.runCmd(['devices']).stdout.readlines():
> +            result =  re.match('(.*?)\t(.*)', line)

this looks very blanket and potentially problematic.  No suggestios for something better.

::: testing/mochitest/runtestsb2g.py
@@ +32,5 @@
> +# and other provisions required by the GPL or the LGPL. If you do not delete
> +# the provisions above, a recipient may use your version of this file under
> +# the terms of any one of the MPL, the GPL or the LGPL.
> +#
> +# ***** END LICENSE BLOCK *****

MPL 2.0
Comment 5 Jonathan Griffin (:jgriffin) 2012-04-27 11:49:23 PDT
I addressed the review comments and pushed as http://hg.mozilla.org/integration/mozilla-inbound/rev/acd7146a0484
Comment 6 Ryan VanderMeulen [:RyanVM] 2012-04-29 14:18:07 PDT
http://hg.mozilla.org/mozilla-central/rev/acd7146a0484

Note You need to log in before you can comment on or make changes to this bug.