1452126 - Windows ASan tests aren't linked to their build tasks

Status: RESOLVED WORKSFORME

(Firefox Build System :: Task Configuration, task)

Description

[(Away)]

This try push included `-p win64-asan` and `-u all` but the asan tests didn't get scheduled automatically. I had to trigger these tests by hand: https://treeherder.mozilla.org/#/jobs?repo=try&revision=3be8681b6093c7a2239bcdb38936d8caa2b50e1a&filter-tier=1&filter-tier=2&filter-tier=3&filter-searchStr=asan

And, even when I added them by hand, it looks like they only apply to opt builds? Is there a way to get debug ones too?

I'd appreciate some pointers on how to investigate this.

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

Have a look in https://dxr.mozilla.org/mozilla-central/source/taskcluster/ci/test/ -- there's a bunch of asan-specific configuration in there.  I don't recall the details of all of it, or the rationale.  It looks like a lot of it is tier-3, so maybe you're not seeing those?

Comment 2

[(Away)]

My treeherder is pretty much permanently on &filter-tier=1&filter-tier=2&filter-tier=3 nowadays.

Comment 3

[(Away)]

Perhaps this is a clue: https://searchfox.org/mozilla-central/rev/d0413b711da4dac72b237d0895daba2537f1514c/taskcluster/taskgraph/transforms/tests.py#533

It might explain the asymmetry in having opt test coverage but not debug. Well, if Linux doesn't run debug either, then I guess I won't worry about it.

Comment 4

[(Away)]

I noticed that my ASan pushes still get TV tasks, and looking into their differences versus regular mochitests I found that TV's run-on-projects says ['trunk', 'try'] where mochitests currently say built-projects.

So this push https://hg.mozilla.org/try/rev/2e03a223a26d66219f4252b49c02fa841824004c managed to get mochitests scheduled.

I know it's not a proper fix, but does it offer any clues into where the problem might be?

Comment 5

[Dustin J. Mitchell [:dustin] (he/him)]

So, "built-projects" means "set run-on-projects to the same as the build task for the installer this test would use".  For try, run-on-projects only applies to "all" (the default) for -p and -u and -t.

The win64-asan build probably doesn't run on try by default, which means that its run-on-projects doesn't contain 'try'.  The mochitests inherit that, so even though you selected the build with `-p win64-asan`, the `-u all` did not select the tests.  I think you could fix this with -u all,mochitest[asan] or something like that?

That said, this is one of the thousands of corner cases where try syntax just isn't expressive enough.  `mach try fuzzy` might be a better choice for this situation.  There, you can select all of the win64 asan test jobs that you're interested in, and any necessary builds will be automatically included.  All of this run-on-projects mumbo-jumbo is entirely avoided.

Comment 6

[(Away)]

(In reply to Dustin J. Mitchell [:dustin] pronoun: he from comment #5)
> The win64-asan build probably doesn't run on try by default, which means
> that its run-on-projects doesn't contain 'try'.  The mochitests inherit
> that, so even though you selected the build with `-p win64-asan`, the `-u
> all` did not select the tests.  I think you could fix this with -u
> all,mochitest[asan] or something like that?

This was the key connection that I wasn't making. Thanks!

`-u all[windows10-64-asan]` will let me get by for now. 

And I assume that if/when I make WinAsan be tier-1 and running by default, this problem will just go away.