Closed
Bug 925873
Opened 9 years ago
Closed 9 years ago
Make ASan failures something tbpl will recognize as an error, and an error in a particular test
Categories
(Core :: General, defect)
Core
General
Tracking
()
RESOLVED
FIXED
mozilla27
People
(Reporter: philor, Assigned: decoder)
References
(Blocks 1 open bug)
Details
Attachments
(1 file)
1.31 KB,
patch
|
ted
:
review+
|
Details | Diff | Splinter Review |
Things that tbpl recognizes so they can be starred are logged in the form "{possible noise from mozharness} TEST-UNEXPECTED-FAIL | this/is/the/test_filename | what failed was this". ASan failures are logged in the form "{noise from mozharness} ==2325==ERROR: AddressSanitizer: heap-use-after-free on address 0x60600027e0f8 at pc 0x7f104e3c04fd bp 0x7fff9baf4d50 sp 0x7fff9baf4d48". That means that for a failure like bug 921622, we have to know to open a full log like https://tbpl.mozilla.org/php/getParsedLog.php?id=28903669&tree=Fx-Team, scroll down to the bottom and then back up to the end of the runtests step, find the failure, find the test it was in, search for the bug, copy-paste the link to the log, but for a failure like https://tbpl.mozilla.org/php/getParsedLog.php?id=28987951&tree=Mozilla-Inbound where it was in the first of the three suites in mochitest-other, we have to remember (assuming this is true of all of them) that "AddressSanitizer:" is in ASan failure messages to even find the error.
Comment 1•9 years ago
|
||
Thank you for filing this Phil, guess this answers my/nigelb's question from earlier :-)
Reporter | ||
Comment 2•9 years ago
|
||
Also decidedly hard to find: https://tbpl.mozilla.org/php/getParsedLog.php?id=29393777&tree=Profiling contains two ASan failures, one OOP (findable with ==2357==ERROR:) which didn't even stop the run or affect the success/fail count or mark that part of the suite as having failed, and a startup failure in a11y, (==2676==ERROR:) which did at least wind up calling that part of the suite a T-FAIL.
Comment 3•9 years ago
|
||
decoder, may you take a look at this? :-)
Blocks: asan-tests
Flags: needinfo?(choller)
Assignee | ||
Comment 4•9 years ago
|
||
First of all: Is this failing on all ASan traces you see now? If it is, then a change in automation must have regressed it. I am looking at https://tbpl.mozilla.org/php/getParsedLog.php?id=28903669&tree=Fx-Team now and that has the same way of failing.
Flags: needinfo?(choller)
Assignee | ||
Comment 5•9 years ago
|
||
Fwiw, I have a patch that should at least fix the fx-team issue and one of the traces in the earlier profiling failure. It is still unclear to me, when this started happening because I remember that we once had TBPL starrable failures on mochitests. After talking with ted, we came to the conclusion that the mochitest framework doesn't emit a TEST-UNEXPECTED-FAIL for a non-zero exit code (which ASan will use). But any other reason that would terminate the browser without producing a minidump would cause the same failure. I'm testing the patch for fixing that behavior right now on try.
Assignee: nobody → choller
Blocks: asan-maintenance
Comment 6•9 years ago
|
||
Great! Thank you :-)
Assignee | ||
Comment 7•9 years ago
|
||
This patches solves the problem, confirmed on try with a failing mochitest.
Attachment #819748 -
Flags: review?(ted)
Updated•9 years ago
|
Attachment #819748 -
Flags: review?(ted) → review+
Assignee | ||
Comment 8•9 years ago
|
||
Thanks for the quick review. https://hg.mozilla.org/integration/mozilla-inbound/rev/654ffd9d6b09
Status: NEW → ASSIGNED
Comment 9•9 years ago
|
||
Sheriffs: Note this will change (well, lengthen) the signature of annotated summaries on TBPL for all mochitest failures where the return code != 0 (and not just ASan). So think this includes crashes etc. Not necessarily a bad thing, but just so you are aware :-)
Assignee | ||
Comment 10•9 years ago
|
||
(In reply to Ed Morley (Mostly away until 21st October) [:edmorley UTC+1] from comment #9) > Sheriffs: Note this will change (well, lengthen) the signature of annotated > summaries on TBPL for all mochitest failures where the return code != 0 (and > not just ASan). So think this includes crashes etc. Not necessarily a bad > thing, but just so you are aware :-) If this causes you any trouble, please let me know :)
https://hg.mozilla.org/mozilla-central/rev/654ffd9d6b09
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla27
You need to log in
before you can comment on or make changes to this bug.
Description
•