Last Comment Bug 417516 - add top level Makefile targets to run test suites
: add top level Makefile targets to run test suites
Status: RESOLVED FIXED
: dev-doc-complete
Product: Core
Classification: Components
Component: Build Config (show other bugs)
: Trunk
: All All
: -- enhancement (vote)
: mozilla1.9.1b1
Assigned To: Ted Mielczarek [:ted.mielczarek]
:
:
Mentors:
Depends on: 443090
Blocks: 447664 468913 469581 479225
  Show dependency treegraph
 
Reported: 2008-02-14 08:57 PST by Ted Mielczarek [:ted.mielczarek]
Modified: 2012-03-12 15:58 PDT (History)
13 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
add "mochitest" target, and targets for each mochitest variant (1.11 KB, patch)
2008-07-01 10:30 PDT, Ted Mielczarek [:ted.mielczarek]
no flags Details | Diff | Splinter Review
somewhat more complete (3.21 KB, patch)
2008-07-01 12:21 PDT, Ted Mielczarek [:ted.mielczarek]
no flags Details | Diff | Splinter Review
handles mochitest correctly (4.54 KB, patch)
2008-07-10 08:43 PDT, Ted Mielczarek [:ted.mielczarek]
benjamin: review+
Details | Diff | Splinter Review
mochitest targets, as checked in (the first time) (4.70 KB, patch)
2008-07-23 12:04 PDT, Ted Mielczarek [:ted.mielczarek]
no flags Details | Diff | Splinter Review
[checked in] fixed the bug in runtests.py (1.86 KB, patch)
2008-07-25 08:27 PDT, Ted Mielczarek [:ted.mielczarek]
no flags Details | Diff | Splinter Review

Description Ted Mielczarek [:ted.mielczarek] 2008-02-14 08:57:28 PST
Right now running mochitest or reftest requires knowing an arcane commandline.  We should just have "mochitest" and "reftest" targets in the top level makefile.
Comment 1 Ted Mielczarek [:ted.mielczarek] 2008-07-01 10:30:19 PDT
Created attachment 327627 [details] [diff] [review]
add "mochitest" target, and targets for each mochitest variant
Comment 2 Ted Mielczarek [:ted.mielczarek] 2008-07-01 10:31:43 PDT
Comment on attachment 327627 [details] [diff] [review]
add "mochitest" target, and targets for each mochitest variant

Ack, hit enter too early. This is a rough draft of a patch that adds a "mochitest" target, to run all the mochitest variants, as well as a target per-variant. It's not terribly useful yet, since it won't error on failure, I don't think.
Comment 3 Ted Mielczarek [:ted.mielczarek] 2008-07-01 12:21:47 PDT
Created attachment 327654 [details] [diff] [review]
somewhat more complete

This works for the basic mochitest case. browser-chrome output is different, so it won't work there. I haven't tried the a11y tests yet.

We need to get our test suites on the same page. :-/
Comment 4 Ted Mielczarek [:ted.mielczarek] 2008-07-10 08:43:57 PDT
Created attachment 328898 [details] [diff] [review]
handles mochitest correctly

Well now that I've done a bunch of work in bug 443090, this is easier! The runtests.py changes are because I didn't fix that problem for browser-chrome when I fixed it for regular mochitest, apparently.
Comment 5 Ted Mielczarek [:ted.mielczarek] 2008-07-17 11:16:26 PDT
Comment on attachment 328898 [details] [diff] [review]
handles mochitest correctly

I think this is good to go now.
Comment 6 Benjamin Smedberg [:bsmedberg] 2008-07-22 16:04:53 PDT
Comment on attachment 328898 [details] [diff] [review]
handles mochitest correctly

Are you sure you want to have all of these targets for all products? (aren't the browser-chrome tests Firefox-specific?) If you are, r=me... otherwise maybe we should put that part in browser/build.mk
Comment 7 Ted Mielczarek [:ted.mielczarek] 2008-07-22 16:40:09 PDT
Good call, I'll move mochitest-browser to browser/build.mk. The other stuff won't work on a non-browser app, but I do want Firefox and SeaMonkey to both have access.
Comment 8 Ted Mielczarek [:ted.mielczarek] 2008-07-23 12:04:07 PDT
Created attachment 330976 [details] [diff] [review]
mochitest targets, as checked in (the first time)

Made some tweaks and checked this in. Leaving the bug open to handle reftest.
Comment 9 Shawn Wilsher :sdwilsh 2008-07-23 23:01:07 PDT
Are these documented somewhere (how to use them)?
Comment 10 Ted Mielczarek [:ted.mielczarek] 2008-07-24 04:27:36 PDT
No, I didn't get around to that yet. Also apparently I got backed out last night.
Comment 11 Ted Mielczarek [:ted.mielczarek] 2008-07-24 14:08:45 PDT
vlad found a bug in this, if you run runtests.py without --log-file, it breaks mochitest. That might be what was causing problems yesterday. (Oops!)
Comment 12 Ted Mielczarek [:ted.mielczarek] 2008-07-25 08:27:29 PDT
Created attachment 331309 [details] [diff] [review]
[checked in] fixed the bug in runtests.py

This fixes that stupid oversight in runtests.py. I'll push this when the tree goes green again...
Comment 13 Ted Mielczarek [:ted.mielczarek] 2008-07-27 10:07:17 PDT
Comment on attachment 331309 [details] [diff] [review]
[checked in] fixed the bug in runtests.py

Pushed to mozilla-central:
http://hg.mozilla.org/mozilla-central/index.cgi/rev/409570e09968
Comment 14 Justin Wood (:Callek) 2008-08-24 19:27:04 PDT
ted, this resolved or should it stay open?
Comment 15 Arpad Borsos [:Swatinem] 2008-08-25 01:45:58 PDT
So far only "make mochitest" is supported. Something like "make reftest" is still missing.
Comment 16 Nickolay_Ponomarev 2008-12-07 11:35:53 PST
Note, documentation should be updated on at least:
https://developer.mozilla.org/En/Mochitest (I partially updated this, but deki is too frustrating for me)
https://developer.mozilla.org/en/Chrome_tests
https://developer.mozilla.org/en/Browser_chrome_tests

Ideally, there should be a single document on running all the kinds of tests we have (aka what to do before checking in).
Comment 17 Ted Mielczarek [:ted.mielczarek] 2008-12-10 12:45:14 PST
Waldo filed bug 468913 on reftest, so might as well close this.
Comment 18 Eric Shepherd [:sheppy] 2009-02-17 14:31:23 PST
Someone has already updated https://developer.mozilla.org/En/Mochitest to cover this.

Note You need to log in before you can comment on or make changes to this bug.