496580 - Make check in js/src should run the js test suite

Status: RESOLVED DUPLICATE of bug 469718

(Core :: JavaScript Engine, defect, P1)

Description

[Jason Orendorff [:jorendorff]]

`make check` in an objdir for js/src should run the regression test suite in js/tests.

The catch: Right now, given a snapshot of the trunk, you can't just run all the tests and tell if anything is wrong. You need either a baseline js shell (as in bug 478437 comment 30), or an externally maintained list of expected failures (as in <http://hg.mozilla.org/users/jorendorff_mozilla.com/scripts/file/3eda1f592c0c/js-check>). Either way you need a script that isn't checked in.

(Why do we have tests that are expected to fail?  To reduce the cost of maintenance, the whole js/tests directory is kept identical in all branches.  Some tests pass in some branches and not in others.  Some tests only pass in Rhino.  Some tests only pass if LiveConnect is present.  Having expected failures is teh lame, but this bug should not block on that.)

(Related: Bug 473989 asks for "make check" to run trace-test.js.  It can do both.)

Comment 1

[Robert Sayre]

Create a list that excludes liveconnect and check it in?

Comment 2

[Graydon Hoare :graydon]

(In reply to comment #0)

> (Related: Bug 473989 asks for "make check" to run trace-test.js.  It can do
> both.)

Yes please. And v8 and ss. And as many other suites as we can dig up. Many of the compute-intensive demos we've seen on the web are good candidates. I have a list that I keep meaning to convert to shell cases.