Open Bug 685751 Opened 9 years ago Updated 6 years ago
.py should match jstestbrowser
Several tests which jstests.py runs will mark tests as failing even when jstestbrowser does not run them. Examples are js1_8/extensions/regress-422269.js and js1_8_5/extensions/findReferences-*. These tests get no automated coverage and are either flaky and intermittently fail, or always fail. When someone then tries to run jstests in the shell, they see these failures and need to investigate whether their patch caused the problem (unless they know from experience which test failures to ignore). This behavior is confusing and wastes time and effort. The behavior of running jstests.py should mark things as failures in the same way as jstestbrowser on tinderbox. Tests skipped by the browser should be fixed, removed, moved into jit-tests (which gets automated shell test coverage), or clearly marked as ignorable by jstests.py.
You need to log in before you can comment on or make changes to this bug.