911237 - TBPL support for ASAN tests

Status: RESOLVED FIXED

(Tree Management Graveyard :: TBPL, defect)

Description

[Chris AtLee [:catlee]]

We're running tests for asan builds on a few branches now, so need them to show up properly on TBPL.

The builders are called e.g. "Ubuntu ASAN VM 12.04 x64 cedar opt test mochitest-other"

Comment 1

[Ryan VanderMeulen [:RyanVM]]

We're still running debug ASAN builds. Do we want to be doing that? It affects what I want to do here.

Comment 2

[Ryan VanderMeulen [:RyanVM]]

Attachment: create new ASAN flavor

Creates a new flavor for ASAN builds. The builds + tests will show on their own line similar to how opt, debug, and pgo do currently. Note that this only really supports opt builds correctly at the moment. My understanding is that we only be running opt builds on m-c. However, I do see debug builds running on cedar.

Decoder, do we need to worry about supporting debug ASAN or is opt sufficient?

Comment 3

[Ryan VanderMeulen [:RyanVM]]

Correction, the debug builds run on m-c as well.

Comment 4

[Ed Morley [:emorley]]

(Waiting on the needinfo before reviewing :-))

Comment 5

[Christian Holler (:decoder)]

The tests are only run on opt, and we do not need to care about tests on debug (also not in the future most likely, because it's too slow and doesn't make much sense in my opinion).

We do however have debug *builds* on m-c and that's also pretty important to keep.

Comment 6

[Ed Morley [:emorley]]

(In reply to Christian Holler (:decoder) from comment #5)
> We do however have debug *builds* on m-c and that's also pretty important to
> keep.

Yeah comment 2 was meaning the debug builds - out of curiosity, what value do they add? :-)

Comment 7

[Ryan VanderMeulen [:RyanVM]]

Attachment: create new ASAN flavor

Well, I feel dirty.

This works, but it's not as elegant as I was hoping. This:
1.) Creates a new ASan flavor
2.) Add new symbols for ASan debug/opt builds/nightlies

If we didn't have to support debug builds, we could just add the flavor and get rid of all the extra symbols. If we ran tests on debug builds, we could just add a debug ASan flavor to cover it all. However, since this is the world we live in, I think this solution makes the most sense.

Comment 8

[Christian Holler (:decoder)]

(In reply to Ed Morley [:edmorley UTC+1] from comment #6)
> (In reply to Christian Holler (:decoder) from comment #5)
> > We do however have debug *builds* on m-c and that's also pretty important to
> > keep.
> 
> Yeah comment 2 was meaning the debug builds - out of curiosity, what value
> do they add? :-)

The main reason for having them is that they use less inlining and therefore can produce more meaningful traces if the opt trace is not clear.

Comment 9

[Christian Holler (:decoder)]

Fwiw, I would be ok with just showing ASan opt on TBPL and leave the debug build in the hidden job, if it's too much of a hassle to get both :)

Comment 10

[Ryan VanderMeulen [:RyanVM]]

It really doesn't matter whether it's visible by default or not. We still need to display it in a sensible way when it is shown.

Comment 11

[Christian Holler (:decoder)]

What I meant is that it could remain part of this job:

https://tbpl.mozilla.org/?jobname=asan&showall=1

But maybe I am misunderstanding something here.

Comment 12

[Ryan VanderMeulen [:RyanVM]]

Attachment: as attached

The patch as attached creates a new "Linux64 ASAN" line where all builds + tests are displayed rather than intermingling them on other lines.

Mainly I'm concerned about the visual consistency of displaying all of the opt jobs in a line by themselves while putting the debug build on a different line. Default visibility doesn't really impact that one way or the other. I think showing both Opt and Debug like the attached picture makes the most sense.

And if we some day want to run tests on debug builds, we can just remove the "Bo/Bd" special casing and add another "Linux64 ASAN Debug" flavor instead.

Comment 13

[Ed Morley [:emorley]]

Comment on attachment 798125 [details] [diff] [review]
create new ASAN flavor

Review of attachment 798125 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good - only tweak I might suggest is to swap opt and debug around (just in config.js), so it displays as Bo Bd (ie same order as the opt/debug rows do themselves :-)

Comment 14

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/webtools/tbpl/rev/4ae092ddb897

Comment 15

[Ed Morley [:emorley]]

In production :-)

Comment 16

[Christian Holler (:decoder)]

I can see the builds here

https://tbpl.mozilla.org/?showall=1

but not the tests. Am I missing something?

Comment 17

[Ryan VanderMeulen [:RyanVM]]

They're running on Cedar, not m-c.

Comment 18

[Ed Morley [:emorley]]

This bug was just to get TBPL to identify the ASan tests properly, which it now does:
https://tbpl.mozilla.org/?tree=Cedar&showall=1&jobname=asan

However, it looks like there was a problem with the patches in bug 831491 to get the jobs to run on other trees, since they have been checked in (and more importantly merged to production and presumably reconfiged) but the jobs aren't appearing.