prepend "FAILED:" to TestVersionComparator output if a test fails

RESOLVED INCOMPLETE

Status

()

Core
XPCOM
RESOLVED INCOMPLETE
11 years ago
5 years ago

People

(Reporter: davel, Unassigned)

Tracking

Trunk
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

11 years ago
allow tinderbox to detect failure of "make check" target by scanning the output for the string "FAIL"
(Reporter)

Comment 1

11 years ago
Created attachment 232646 [details] [diff] [review]
patch to modify failure output

will submit for review once bug 347820 is further along
(Reporter)

Updated

11 years ago
Attachment #232646 - Flags: review?(dougt)
(Reporter)

Updated

11 years ago
Assignee: davel → nobody

Comment 2

11 years ago
Comment on attachment 232646 [details] [diff] [review]
patch to modify failure output

You are using the exit code, correct? Not that I'm against an extra FAILED, but I don't think we want to go around grepping output unless there's no other choice.

Updated

11 years ago
Attachment #232646 - Flags: review?(dougt) → review+
(In reply to comment #2)
> You are using the exit code, correct?

No, and it can bite us (see bug 353010).
(Reporter)

Comment 4

11 years ago
(In reply to comment #3)
> (In reply to comment #2)
> > You are using the exit code, correct?
> 
> No, and it can bite us (see bug 353010).
> 

this program is not invoked by test_all.sh, so I'm not sure the bug you mention applies.
(In reply to comment #4)
> this program is not invoked by test_all.sh, so I'm not sure the bug you mention
> applies.

Ah, indeed -- I should have noticed that, sorry.
(Reporter)

Comment 6

11 years ago
I'm holding off on committing this change.  This program is not designed to run in its own build tree; instead, it needs to be copied to the app dir of a different version and run there.

(I don't recall whether the new version needs to run in old releases, or the versions from old releases need to run in the new build - dougt has details)

The main motivation for this change was to make it easy to find all failures in a "make check" log by searching for the string FAIL.

As run now, this program will never fail.  More effort is needed to figure out how best to run this test in a meaningful way.

Updated

5 years ago
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.