Closed Bug 775102 Opened 12 years ago Closed 12 years ago

Modify FennecTalosAssert so that assertion failures make the tests orange

Categories

(Testing :: Talos, defect)

All
Android
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED
mozilla17

People

(Reporter: kats, Assigned: kats)

Details

Attachments

(1 file)

See bug 775071. Right now the function in FennecTalosAssert are a no-op and can lead to missed failures or useless tests. This should be fixed.
Component: General → Talos
Product: Firefox for Android → Testing
QA Contact: general
Unassigning from myself for now. I don't know where the code is that determines whether or not a talos job is considered a failure, therefore I don't know what to put in FennecTalosAssert to trigger a job to fail. Apparently printing "error" is not enough because that didn't work; the talos run of rck3 at https://tbpl.mozilla.org/?tree=Try&rev=f307976a7ae6 has "Error" and stack traces in the output but still got marked as green.

I can pick this bug up later if nobody else gets to it first.
Assignee: bugmail.mozilla → nobody
We should try "__FAIL<something>__FAIL":

http://mxr.mozilla.org/build-central/source/talos/talos/results.py#235
http://mxr.mozilla.org/build-central/source/pageloader/chrome/pageloader.js#317

...but I'm not sure that will work if __start_report is also present.
Attached patch PatchSplinter Review
Thanks, gbrown. Patch based on that is attached, pushed to try at https://tbpl.mozilla.org/?tree=Try&rev=72c8fbfe2a71
(In reply to Geoff Brown [:gbrown] from comment #2)
> ...but I'm not sure that will work if __start_report is also present.

Right, if __start_report is present then that takes priority and __FAIL is ignored. Relevant try runs:

https://tbpl.mozilla.org/?tree=Try&rev=a978d1253723 (this has both __start_report and __FAIL, gets marked green)
https://tbpl.mozilla.org/?tree=Try&rev=72c8fbfe2a71 (this has only __FAIL, gets marked red)
Attachment #643838 - Flags: review?(gbrown)
Filed bug 775610 for making __FAIL always fail the test.
Assignee: nobody → bugmail.mozilla
Comment on attachment 643838 [details] [diff] [review]
Patch

Review of attachment 643838 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks -- we should have done this long ago!
Attachment #643838 - Flags: review?(gbrown) → review+
https://hg.mozilla.org/mozilla-central/rev/27b008d93185
Status: ASSIGNED → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla17
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: