1375341 - Add support for ASan LeakSanitizer bug suggestions

Status: RESOLVED FIXED

(Tree Management :: Treeherder: Data Ingestion, enhancement, P2)

Description

[Phil Ringnalda (:philor)]

ASan (LSan, let's call the whole thing off) leak messages are of the form "TEST-UNEXPECTED-FAIL | LeakSanitizer | leak at Object1, Object2, Object3".

For an astonishingly long time, we apparently got away with (if you ignore massive misstarring) treating that as a failure in the test with the filename LeakSanitizer, and suggesting every open bug in that test filename.

Then recently we got a rash of new leaks, which have driven us up well over 20 kw:intermittent-failure LeakSanitzer results, so now they all have the suggestion "Exceeded max (20) bug suggestions, most of which are likely false positives."

For non-LSan, leak failure messages are of the form "leakcheck | .*\d+ bytes leaked (Object-1, Object-2, Object-3, ...)" and we have a regex to make anything matching \d+ bytes leaked \(.+\) search for the stuff in the parens in https://github.com/mozilla/treeherder/blob/37cf7fa970a0d641b4b6cae58b123db85e67ec52/treeherder/model/error_summary.py#L107.

We can't exactly do that, since that's already a bit sketchy, not checking for the filename leakcheck before matching, and matching any summary including "leak at" would be far too sketchy, but in the same neighborhood we could reasonably check for test_name_or_path == "LeakSanitizer" and then set search_term to a match on leak at (.+).

Comment 1

[Ed Morley [:emorley]]

Thank you for filing :-)

Adjusting summary, since this isn't a Treeherder regression, but a case of "someone added a new format test which requires additional special-casing, but a Treeherder bug wasn't filed".

For whomever fixes this, the regex here needs to be updated:
https://github.com/mozilla/treeherder/blob/dac80b1fa492481cf2d31c83d8ff0c0a3113f56a/treeherder/model/error_summary.py#L13

I'd update the comment here:
https://github.com/mozilla/treeherder/blob/dac80b1fa492481cf2d31c83d8ff0c0a3113f56a/treeherder/model/error_summary.py#L106-L107

And then add a testcase here:
https://github.com/mozilla/treeherder/blob/dac80b1fa492481cf2d31c83d8ff0c0a3113f56a/tests/model/test_error_summary.py#L85-L104

Comment 2

[GitHub Autolander Bot]

Attachment: [treeherder] KWierso:lsanfix > mozilla:master

Comment 3

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Comment on attachment 8880280 [details] [review]
[treeherder] KWierso:lsanfix > mozilla:master

How about this?

Comment 4

[Ed Morley [:emorley]]

Comment on attachment 8880280 [details] [review]
[treeherder] KWierso:lsanfix > mozilla:master

The tests are failing. I think the check should be built into the existing one, rather than adding an extra conditional after the fallback mechanism.

Comment 5

[Ed Morley [:emorley]]

(In reply to Ed Morley [:emorley] from comment #4)
> The tests are failing.

Sorry this part is due to breakage caused by Travis's rollout yesterday, that didn't appear when testing (they didn't provide a way to test the container builds, only the sudo-enabled builds), for which I've filed:
https://github.com/travis-ci/travis-ci/issues/7951

Comment 6

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Comment on attachment 8880280 [details] [review]
[treeherder] KWierso:lsanfix > mozilla:master

Moved it up as requested.

Comment 7

[Ed Morley [:emorley]]

Comment on attachment 8880280 [details] [review]
[treeherder] KWierso:lsanfix > mozilla:master

Ah the new regex needs to be built into the old one, not moved higher up the file :-)

(In reply to Ed Morley [:emorley] from comment #4)
> I think the check should be built into the existing one

Comment 8

[Ed Morley [:emorley]]

Comment on attachment 8880280 [details] [review]
[treeherder] KWierso:lsanfix > mozilla:master

Many thanks!

Comment 9

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/3e9e7443cb021ab5e92bb7c0cda98e080a17a834
Bug 1375341 - Add support for leaksanitizer errors (#2587) r=emorley