Closed Bug 362433 Opened 14 years ago Closed 12 years ago

xpcshell-tests: add dump() to each do_check_*(), at least

Categories

(Testing :: XPCShell Harness, enhancement)

enhancement
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED
mozilla1.9.2a1

People

(Reporter: moco, Assigned: sgautherie)

References

(Blocks 1 open bug)

Details

(Keywords: autotest-issue, verified1.9.1, Whiteboard: [fixed1.9.1b99])

Attachments

(1 file)

[RFE] option to "make check" be more verbose

When I run the tests, and they pass I get something like this:

sh-3.1$ pwd
/cygdrive/c/builds/trunk/mozilla
sh-3.1$ cd ff-debug/browser/components/places/tests/
sh-3.1$ make check
../../../../dist/bin/test_all.sh ../../../../dist/bin/test_places
../../../../dist/bin/test_places/test_bookmarks.js: PASS 
../../../../dist/bin/test_places/test_livemarks.js: PASS 

Is there a way we could add a "make checkverbose" target so that we'd see more of what's going on?  I was thinking about http://lxr.mozilla.org/seamonkey/source/tools/test-harness/xpcshell-simple/head.js and doing a dump() on all do_check_*() functions.

robert, what do you think?
I think that's fine if-wanted, but the tests themselves will have to be rewritten to handle the extra verbosity. An integer value for verbosity a la xUnit might be nice. e.g., 0 = terse, 1 = standard 2+ = verbose ?
(In reply to comment #1)
> I think that's fine if-wanted, but the tests themselves will have to be
> rewritten to handle the extra verbosity. An integer value for verbosity a la
> xUnit might be nice. e.g., 0 = terse, 1 = standard 2+ = verbose ?
> 

Let's use the same thing mochikit does-- a Log4J approach. DEBUG, INFO, ERROR, etc. Test failures are logged at ERROR level.
make check output on the mac is different from the other platforms. If I run make -k check in my objdir, I get notice that deps are being updated, but it doesn't tell me where. Linux and Windows both report which directories are being entered and left.
OS: Windows XP → All
Hardware: PC → All
Component: Testing → TUnit
Product: Core → Testing
QA Contact: testing → tunit
Version: Trunk → unspecified
(In reply to comment #0)
> http://lxr.mozilla.org/seamonkey/source/tools/test-harness/xpcshell-simple/head.js
> doing a dump() on all do_check_*() functions.

Just had this idea too while modifying a test ;-)

(In reply to comment #1)
> I think that's fine if-wanted, but the tests themselves will have to be
> rewritten to handle the extra verbosity.

Let's morph this bug:
add dump()s to what we have now (only), which will go to the individual log files,
then leave the rest to bug 468196.

(In reply to comment #3)
> make check output on the mac is different from the other platforms. If I run
> make -k check in my objdir, I get notice that deps are being updated, but it
> doesn't tell me where. Linux and Windows both report which directories are
> being entered and left.

That's unrelated to this bug, is it?
Severity: normal → enhancement
Summary: [RFE] option to "make check" be more verbose → xpcshell-tests: add dump() to each do_check_*(), at least
Version: unspecified → Trunk
The output is still a little raw, but this is a major enhancement yet.


[Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.2a1pre) Gecko/20090531 Minefield/3.6a1pre] (mozilla-central-win32-unittest/1243808976) (W2Ksp4)
(http://hg.mozilla.org/mozilla-central/rev/ab395a1916be)

628 TEST-PASS = 579 with checks + 49 without checks.
Assignee: nobody → sgautherie.bz
Status: NEW → ASSIGNED
Attachment #381133 - Flags: review?(rcampbell)
Blocks: 468196
Comment on attachment 381133 [details] [diff] [review]
(Av1) Log check results
[Checkin: See comment 8 & 9]

I think this looked pretty good on a quick pass.

coop, do you have time to give this a once-over?
Attachment #381133 - Flags: review?(rcampbell)
Attachment #381133 - Flags: review?(ccooper)
Attachment #381133 - Flags: review+
Attachment #381133 - Flags: review?(ccooper) → review+
Comment on attachment 381133 [details] [diff] [review]
(Av1) Log check results
[Checkin: See comment 8 & 9]

> function do_test_pending() {
>-  dump("*** test pending\n");
>-  _tests_pending++;
>+  dump("TEST-INFO | (xpcshell/head.js) | test " + (++_tests_pending) +
>+         " pending\n");
> }

My only concern is incrementing _tests_pending while we're concatenating. I'd prefer that as two distinct statements.
Blocks: 496443
Comment on attachment 381133 [details] [diff] [review]
(Av1) Log check results
[Checkin: See comment 8 & 9]


http://hg.mozilla.org/mozilla-central/rev/07971f68e728
(Av1a) Log check results
Attachment #381133 - Attachment description: (Av1) Log check results → (Av1) Log check results [Checkin: See comment 8]
Status: ASSIGNED → RESOLVED
Closed: 12 years ago
Flags: in-testsuite-
Keywords: autotest-issue
Resolution: --- → FIXED
Target Milestone: --- → mozilla1.9.2a1
Attachment #381133 - Attachment description: (Av1) Log check results [Checkin: See comment 8] → (Av1) Log check results [Checkin: See comment 8 & 9]
Comment on attachment 381133 [details] [diff] [review]
(Av1) Log check results
[Checkin: See comment 8 & 9]


http://hg.mozilla.org/releases/mozilla-1.9.1/rev/3bab31a7104f

after fixing context for
{
patching file testing/xpcshell/head.js
Hunk #2 FAILED at 40
1 out of 5 hunks FAILED
patching file testing/xpcshell/runxpcshelltests.py
Hunk #1 FAILED at 14
Hunk #2 FAILED at 208
2 out of 2 hunks FAILED
}
Keywords: fixed1.9.1
Whiteboard: [fixed1.9.1rc1]
Blocks: 495776
verified1.9.1, per bug 496776 comment 4.
Whiteboard: [fixed1.9.1rc1] → [fixed1.9.1b99]
Blocks: 384339
No longer blocks: 384339
You need to log in before you can comment on or make changes to this bug.