547111 - Build-time check that all directories in js/src/tests have jstests.list, all such directories are listed in their parents, no js file is omitted, etc.

Status: RESOLVED INCOMPLETE

(Core :: JavaScript Engine, defect)

Description

[Jeff Walden [:Waldo]]

We seem to have perennial problems forgetting to add test files, and new test directories, to manifests.  We should put some sanity checks for all this in the build process so we don't have to manually track down these mistakes in the future.

Comment 1

[David Mandelin [:dmandelin]]

The '-c' option of jstests.py should do most of this. Just need to make tinderbox output for it and add it to make check.

Comment 2

[Jeff Walden [:Waldo]]

I'd prefer if this sanity check weren't a "check" thing, rather just part of the "unnamed" Makefile target -- the earlier you find out about something like this, the better, especially if you write a really big patch that you think you've finished completely but then find the omitted test reveals some critical flaw that requires many changes to fix.

Comment 3

[Igor Bukanov]

(In reply to comment #0)
> We seem to have perennial problems forgetting to add test files, and new test
> directories, to manifests.

BTW, why it was decided that the manifest must explicitly list all test files rather than including any js file or sub-directory by default and using explicit skip commands in manifest for entries that are not tests?

Comment 4

[Bob Clary [:bc] (inactive)]

The reftest framework was leveraged thus bring the same manifest requirements.