685751 - jstests.py should match jstestbrowser

Status: RESOLVED INCOMPLETE

(Core :: JavaScript Engine, defect)

Description

[Brian Hackett [Laid off!]]

Several tests which jstests.py runs will mark tests as failing even when jstestbrowser does not run them.  Examples are js1_8/extensions/regress-422269.js and js1_8_5/extensions/findReferences-*.  These tests get no automated coverage and are either flaky and intermittently fail, or always fail.  When someone then tries to run jstests in the shell, they see these failures and need to investigate whether their patch caused the problem (unless they know from experience which test failures to ignore).  This behavior is confusing and wastes time and effort.

The behavior of running jstests.py should mark things as failures in the same way as jstestbrowser on tinderbox.  Tests skipped by the browser should be fixed, removed, moved into jit-tests (which gets automated shell test coverage), or clearly marked as ignorable by jstests.py.

Comment 1

[Bryan Thrall [:bthrall]]

Neither extensions/regress-422269.js nor extensions/findReferences-* exist any more, and jstests is not reporting tests as failing.