563077 - jstests.py always skips tests marked as "random"

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect)

Description

[Jesse Ruderman]

Attachment: patch

Even if I don't pass -d to jstests.py, it skips the tests marked as "random".

Comment 1

[Jesse Ruderman]

Attachment: patch v2

Also, report crashes in tests marked as random!  This brings jstests.py in line with the other manifest-based test suites, where "skip" is used for crashing tests and "random" is used for tests where we want to ignore everything bug crashes.

Comment 2

[Jesse Ruderman]

One problem with this is that tests.py sets the status to CRASH when rc == 3.  I think this should be considered FAIL, not CRASH.

Comment 3

[David Mandelin [:dmandelin]]

Comment on attachment 442851 [details] [diff] [review]
patch v2

I get a few new failures with this patch. Can you check them out? We might just need to update the manifest, or it could be the rc == 3 thing you mentioned.

[2793|   0| 167] 100% ===============================================>|  199.7s
REGRESSIONS
    ecma_3\Number\15.7.4.6-1.js
    ecma_3\Number\15.7.4.7-1.js
    js1_6\extensions\regress-456826.js
    js1_8_5\regress\regress-555246-1.js
FAIL

Comment 4

[Jesse Ruderman]

Attachment: patch v3

With these additional changes, I get PASS on 64-bit Linux.

While I was changing tests.py, I sneaked in a change to make printed failures count as failures even if there's a nonzero rc.  This doesn't affect any current tests.

I sent this to try server, which was dumb because try server doesn't run jstests.

Comment 5

[David Mandelin [:dmandelin]]

Comment on attachment 442970 [details] [diff] [review]
patch v3

Runs cleanly on Win/ia32 as well. I also like the cleanup of rc checking.

Comment 6

[Jesse Ruderman]

http://hg.mozilla.org/tracemonkey/rev/4c3f56a1610c

Comment 7

[Robert Sayre]

http://hg.mozilla.org/mozilla-central/rev/4c3f56a1610c