bugzilla.mozilla.org has resumed normal operation. Attachments prior to 2014 will be unavailable for a few days. This is tracked in Bug 1475801.
Please report any other irregularities here.

Simple test runner for pages that cause crashes/assertions/leaks

RESOLVED DUPLICATE of bug 397725

Status

Testing
Reftest
RESOLVED DUPLICATE of bug 397725
11 years ago
10 years ago

People

(Reporter: kinetik, Unassigned)

Tracking

Trunk
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

11 years ago
We need a test runner that quickly runs through a potentially large collection of simple tests ensuring that they do not cause crashes, assertions, or leaks.  Attached is a first pass at a test runner to do this.  It will run through around 10,000 tests in 2 minutes on my laptop.  It's mostly just a minimized clone of the drive from the existing MochiTest code, so I've left the license attribution as it was in the original code.  Perhaps I should've shared more of the code with MochiTest rather than copying it, but I'm not sure how much farther this test runner will evolve away from MochiTest.

The URL of each test file is printed to stdout before it is loaded.  There is no special test environment provided to the tests when they are run--the intention is that they are standalone tests that would work as if they were loaded into a normal browser instance.

For adding a test, it works very much like MochiTest--add a new test file to your component, add the file to the list of _CRASHTEST_FILES in Makefile.in (or clone the rule if adding a test to a component without any existing tests).  The tests are linked into @OBJDIR@/_tests/crashtests/tests/@COMPONENT_PATH@ during the build.  Building this directory can be disabled at configure time with --disable-crashtests.

To run the testsuite, cd into @OBJDIR@/_tests/crashtests and run |perl runtests.pl|, specify --help to get a list of help.  The interesting options are --autorun (starts tests immediately after initialization), --close-when-done (close browser after tests have run), and --wait-for-render (alters the next-page driver so that we wait long enough for each page to draw rather than moving on as soon as DOMContentLoaded has fired).
sayrer, how should we proceed here? You want to review it?
This looks like a duplicate of bug 397725...

Comment 4

11 years ago
Getting something that works is, of course, a good thing, but there's a lot of similarity between what's here and what's in testing/mochitest, of course.  Would it be possible to move the files back into testing/mochitest to avoid exact duplicates where possible, and when the files do differ, naming the crashing one *-crash.*?  I'd just like to make it easier to consolidate the two in the future.  (I don't think there'll be that much drift between mochitest and crashtest, to be honest.)

Also, I'd prefer not adding an option for --disable-crashtests.  The added code that's downloaded is negligible, and I don't think we should make it possible for people to think of tests as less important than the code they exercise.  Fewer options for configuring a build also mean less cognitive overhead in setting up a build, too.

As for dup, yeah, probably, but we have a patch here, so I'm fine with just using this one.
(In reply to comment #3)
> This looks like a duplicate of bug 397725...

It certainly *feels* like a dupe, but it has a patch in it...

Looking briefly, it looks like a copy of Sayrer's and Gavin's tools to run mochitests in browser chrome. CC'ing Jonas to ask if this would be sufficient for his requirements in bug 397725.
(Reporter)

Comment 7

11 years ago
Yup, this is a dupe of bug 397725.  I wasn't aware of that bug when I was working on this.  I'll look at finishing up the manifest work in the other bug unless someone beats me to it.
Status: NEW → RESOLVED
Last Resolved: 11 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 397725
Thanks Matthew, sorry about the duplicated work.
Component: Testing → Reftest
Product: Core → Testing
QA Contact: testing → reftest
You need to log in before you can comment on or make changes to this bug.