347823 - prepend "FAILED:" to TestVersionComparator output if a test fails

Status: RESOLVED INCOMPLETE

(Core :: XPCOM, defect)

Description

[Dave Liebreich [:davel]]

allow tinderbox to detect failure of "make check" target by scanning the output for the string "FAIL"

Comment 1

[Dave Liebreich [:davel]]

Attachment: patch to modify failure output

will submit for review once bug 347820 is further along

Comment 2

[Benjamin Smedberg]

Comment on attachment 232646 [details] [diff] [review]
patch to modify failure output

You are using the exit code, correct? Not that I'm against an extra FAILED, but I don't think we want to go around grepping output unless there's no other choice.

Comment 3

[Jeff Walden [:Waldo]]

(In reply to comment #2)
> You are using the exit code, correct?

No, and it can bite us (see bug 353010).

Comment 4

[Dave Liebreich [:davel]]

(In reply to comment #3)
> (In reply to comment #2)
> > You are using the exit code, correct?
> 
> No, and it can bite us (see bug 353010).
> 

this program is not invoked by test_all.sh, so I'm not sure the bug you mention applies.

Comment 5

[Jeff Walden [:Waldo]]

(In reply to comment #4)
> this program is not invoked by test_all.sh, so I'm not sure the bug you mention
> applies.

Ah, indeed -- I should have noticed that, sorry.

Comment 6

[Dave Liebreich [:davel]]

I'm holding off on committing this change.  This program is not designed to run in its own build tree; instead, it needs to be copied to the app dir of a different version and run there.

(I don't recall whether the new version needs to run in old releases, or the versions from old releases need to run in the new build - dougt has details)

The main motivation for this change was to make it easy to find all failures in a "make check" log by searching for the string FAIL.

As run now, this program will never fail.  More effort is needed to figure out how best to run this test in a meaningful way.