762744 - Add support for webapp run time mini-mochitest suite

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P3)

Description

[cmtalbert]

Bug 733631 is building a unittest system for webapps. The code is based on mochitest, so as far as buildbot is concerned this is adding two more steps calling mochitest's runtests.py file with different arguments.

They are still ensuring the tests work properly, but you should be able to use the following command lines on Windows, Linux and Mac OS's. (Note, there is no mobile support for this at this time).

= webapps Content test =
python mochitest/runtests.py --appname=<appname> --utility-path=bin --extra-profile-file=bin/plugins --certificate-path=certs --autorun --close-when-done --console-level=INFO --symbols-path=<symbol_path> --webapprt-content

Use the normal content mochitest log parser for the webapp content tests in order to determine pass/fail status.

= webapps Chrome test =
python mochitest/runtests.py --appname=<appname> --utility-path=bin --extra-profile-file=bin/plugins --certificate-path=certs --autorun --close-when-done --console-level=INFO --symbols-path=<symbol_path> --webapprt-chrome

Use the normal browser-chrome mochitest log parser for the webapp chrome tests in order to determine their pass/fail status.

= Trees =
We need these tests on the following trees:
* Mozilla-central
* Mozilla-inbound
* Try

= Further Changes to downstream tools =
Once we have buildbot running these jobs, we'll need to make the corresponding changes to tbpl, and orange factor's log parser to accept the new test framework. Edmorley and Jgriffin I'll let both of you file the corresponding bugs for that work. Armenzg, we'll also need to modify the try chooser to be able to accept these tests and conditionally turn them on/off.

Comment 1

[(not currently active) Ted Mielczarek]

I think we should probably just stick these into mochitest-other.

Comment 2

[Jonathan Griffin (:jgriffin)]

(In reply to Ted Mielczarek [:ted] from comment #1)
> I think we should probably just stick these into mochitest-other.

If we stick them there, we get orange factor and TBPL support for free!

Comment 3

[Drew Willcoxon :adw]

Hey guys, what's the status here?  Let me know if I can help.  A few things:

* These tests only work on OS X for now.  Bug 770771 and bug
  770772 track Windows and Linux.
* This probably goes without saying, but I'd like to do some
  realistic trial runs before they're turned on "for real" to
  make sure they actually work in the buildbot environment.
* The runtests.py arguments have changed a little since this bug
  was filed.  See the webapprt-test-{content,chrome} targets for
  current syntax:
  http://mxr.mozilla.org/mozilla-central/source/testing/testsuite-targets.mk#132

Comment 4

[cmtalbert]

Catlee, what would it take to get this onto try? That's really all we need here in the beginning.

Comment 5

[Chris AtLee [:catlee]]

not too bad, I don't think...first guess would be to add it here:
http://hg.mozilla.org/build/buildbot-configs/file/default/mozilla-tests/config.py#l1163

Comment 6

[[e-mail] (:bhearsum)]

Still wanted?

Comment 7

[Jonathan Griffin (:jgriffin)]

This has already been done actually.