Right now running mochitest or reftest requires knowing an arcane commandline. We should just have "mochitest" and "reftest" targets in the top level makefile.
Created attachment 327627 [details] [diff] [review] add "mochitest" target, and targets for each mochitest variant
Comment on attachment 327627 [details] [diff] [review] add "mochitest" target, and targets for each mochitest variant Ack, hit enter too early. This is a rough draft of a patch that adds a "mochitest" target, to run all the mochitest variants, as well as a target per-variant. It's not terribly useful yet, since it won't error on failure, I don't think.
Created attachment 327654 [details] [diff] [review] somewhat more complete This works for the basic mochitest case. browser-chrome output is different, so it won't work there. I haven't tried the a11y tests yet. We need to get our test suites on the same page. :-/
Created attachment 328898 [details] [diff] [review] handles mochitest correctly Well now that I've done a bunch of work in bug 443090, this is easier! The runtests.py changes are because I didn't fix that problem for browser-chrome when I fixed it for regular mochitest, apparently.
Comment on attachment 328898 [details] [diff] [review] handles mochitest correctly I think this is good to go now.
Comment on attachment 328898 [details] [diff] [review] handles mochitest correctly Are you sure you want to have all of these targets for all products? (aren't the browser-chrome tests Firefox-specific?) If you are, r=me... otherwise maybe we should put that part in browser/build.mk
Good call, I'll move mochitest-browser to browser/build.mk. The other stuff won't work on a non-browser app, but I do want Firefox and SeaMonkey to both have access.
Created attachment 330976 [details] [diff] [review] mochitest targets, as checked in (the first time) Made some tweaks and checked this in. Leaving the bug open to handle reftest.
Are these documented somewhere (how to use them)?
No, I didn't get around to that yet. Also apparently I got backed out last night.
vlad found a bug in this, if you run runtests.py without --log-file, it breaks mochitest. That might be what was causing problems yesterday. (Oops!)
Created attachment 331309 [details] [diff] [review] [checked in] fixed the bug in runtests.py This fixes that stupid oversight in runtests.py. I'll push this when the tree goes green again...
Comment on attachment 331309 [details] [diff] [review] [checked in] fixed the bug in runtests.py Pushed to mozilla-central: http://hg.mozilla.org/mozilla-central/index.cgi/rev/409570e09968
ted, this resolved or should it stay open?
So far only "make mochitest" is supported. Something like "make reftest" is still missing.
Note, documentation should be updated on at least: https://developer.mozilla.org/En/Mochitest (I partially updated this, but deki is too frustrating for me) https://developer.mozilla.org/en/Chrome_tests https://developer.mozilla.org/en/Browser_chrome_tests Ideally, there should be a single document on running all the kinds of tests we have (aka what to do before checking in).
Waldo filed bug 468913 on reftest, so might as well close this.
Someone has already updated https://developer.mozilla.org/En/Mochitest to cover this.