Closed Bug 1296088 Opened 5 years ago Closed 5 years ago
-p all -u all[x64] try syntax no longer triggers Linux asan tests
'-b do -p all -u all[x64] -t none' has been my goto try syntax for many of the changes that I make, as I know that my changes tend to not be platform-specific. One of the nice things about this syntax is that I get Linux ASan tests to run without having to explicitly pick them, cf. this try push from two weeks ago: https://treeherder.mozilla.org/#/jobs?repo=try&revision=bef1c2a79998 However, a try push from yesterday, with the same syntax, did not trigger any ASan tests: https://treeherder.mozilla.org/#/jobs?repo=try&revision=b4f831f8a347
Component: Tools → Task Configuration
Product: Release Engineering → Taskcluster
QA Contact: hwine
asan builds recently moved to taskcluster, this may be related
I looked at the try_parser.py code and my understanding is that it only looks at the lists of buildbot jobs, is that correct Reading this https://ahal.ca/blog/2015/try-syntax/ and https://wiki.mozilla.org/ReleaseEngineering/TryServer Going to ask Armen for more info since he added content to the TryServer page about scheduling taskcluster jobs
Will ask ahal instead since he wrote the blog post on try syntax on tc and armen is away. Ahal, do you know the state of triggering jobs via try when they are not available on buildbot, but taskcluster only?
Flags: needinfo?(armenzg) → needinfo?(ahalberstadt)
There's a separate try_parser for taskcluster that lives here: https://dxr.mozilla.org/mozilla-central/source/taskcluster/taskgraph/try_option_syntax.py I believe the problem is that this taskcluster try parser doesn't support the square bracket notation. I think dustin's done the most work here, he might even already have a bug on it somewhere. I should note that we're going to be looking into ways of improving how task selection on try works in the nearish future. Though this probably means looking at replacements for try syntax, rather than improving it.
Flags: needinfo?(ahalberstadt) → needinfo?(dustin)
It does support it. In the original parser, the contents of square brackets are substring-matched against of the builder name, but of course taskcluster doesn't have builder names. There is a mapping ("UNITTEST_PLATFORM_PRETTY_NAMES") in that file which is meant to replace that substring matching. It was generated based on a few hundred try syntaxes, but perhaps that sample was taken before ASAN was a commonly-run thing. Anyway, modifying that list should help.
So asan jobs are included in UNITTEST_PLATFORM_PRETTY_NAMES. How are jobs parsed as buildbot or tc and parsed by the appropriate parsers? I ran a try run with mach try this morning and it didn't run the associated asan tests. I couldn't find where it https://dxr.mozilla.org/mozilla-central/source/taskcluster/taskgraph/try_option_syntax.py was called https://treeherder.mozilla.org/#/jobs?repo=try&revision=8555716c8f330bac9b933d33e1c39e1813404928
> So asan jobs are included in UNITTEST_PLATFORM_PRETTY_NAMES. It doesn't look like your try push actually changed this? The try push has no changes at all, in fact.
Jobs aren't parsed as Buildbot or TC -- both parsers run over all pushes. They just schedule a mostly-disjoint set of "builds"/"tasks". I may be misunderstanding your comment.. I can't really make a sentence out of the last bit..
I didn't need to change anything in UNITTEST_PLATFORM_PRETTY_NAMES since it already included asan. I was just trying to get the asan tests to run and they didn't. Same result here https://treeherder.mozilla.org/#/jobs?repo=try&revision=64d66fa0b403e4a29d262b356ba76162e50f4fa1 How are the parsers deployed to run over the pushes? I'm not familiar with that part of our infrastructure.
tested this patch on try and it triggered the asan tests. https://treeherder.mozilla.org/#/jobs?repo=try&revision=c98c8de6d672
Attachment #8784002 - Flags: review?(dustin)
Attachment #8784002 - Flags: review?(dustin) → review?(aselagea)
Comment on attachment 8784002 [details] [diff] [review] bug1296088.patch Looks good.
Attachment #8784002 - Flags: review?(aselagea) → review+
Pushed by firstname.lastname@example.org: https://hg.mozilla.org/integration/mozilla-inbound/rev/4a0dd5baf1a2 p all -u all[x64] try syntax no longer triggers Linux asan tests r=aselagea DONTBUILD
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla51
Sorry, what info did you need from me here?
Dustin: I originally asked you for review but switched to to Alin. No info needed now.
You need to log in before you can comment on or make changes to this bug.