1062228 - ASAN tests are displayed on the opt row

Status: RESOLVED FIXED

(Tree Management :: Treeherder, defect, P1)

Description

[Ed Morley [:emorley]]

ie:
https://treeherder.mozilla.org/ui/#/jobs?searchQuery=asan

Is due to:
BUILD_TYPE_BUILDERNAME = [
    {
        'type': 'pgo',
        'regex': re.compile('.+ pgo(?:[ ]|-).+'),
    },
    {
        'type': 'asan',
        'regex': re.compile('.+ asan .+'),
    },
    {
        'type': 'debug',
        'regex': re.compile('(?:debug|leak)', re.IGNORECASE),
    }
    # defaults to "opt" if not found
]

...

def extract_build_type(source_string):
    output = 'opt'
    for build_type in BUILD_TYPE_BUILDERNAME:
        if build_type["regex"].search(source_string, re.IGNORECASE):
            output = build_type["type"]
            return output
    return output

---

.search() doesn't accept re options such as re.IGNORECASE - that needs to be specified as part of the re.compile()

Comment 1

[Ed Morley [:emorley]]

So the reason this occurs is that for:
>         if build_type["regex"].search(source_string, re.IGNORECASE):

We're using the regexobject .search() method, which doesn't take the flags argument:
https://docs.python.org/2/library/re.html#re.RegexObject.search
> search(string[, pos[, endpos]])

Unlike the re module .search() function:
https://docs.python.org/2/library/re.html#re.search
> re.search(pattern, string, flags=0)

To make things worse, re.IGNORECASE comes from:
http://hg.python.org/releasing/2.7.7/file/4b38a5a36536/Lib/sre_constants.py#l208
> SRE_FLAG_IGNORECASE = 2 # case insensitive

And so we don't get an exception since the flag is interpreted as pos=2.

Comment 2

[Ed Morley [:emorley]]

Attachment: service PR198 - Set re.IGNORECASE correctly for the build_type regex

Only the re module function .search() accepts the flags argument - for the regexobject search() method it actually gets interpreted as the 'pos' parameter, since |re.IGNORECASE == 2|.

With this change we now correctly identify tests run on ASAN builds as buildtype 'asan'.
Updates the test that was testing we were wrong, doh.

Comment 3

[Ed Morley [:emorley]]

Comment on attachment 8483751 [details] [review]
service PR198 - Set re.IGNORECASE correctly for the build_type regex

Adding jeads as a reviewer to spread the load a bit, whichever of you gets to this first can cancel the other person's review.

> Created attachment 8483751 [details] [review]
> service PR198 - Set re.IGNORECASE correctly for the build_type regex
> 
> Only the re module function .search() accepts the flags argument - for the
> regexobject search() method it actually gets interpreted as the 'pos'
> parameter, since |re.IGNORECASE == 2|.
> 
> With this change we now correctly identify tests run on ASAN builds as
> buildtype 'asan'.
> Updates the test that was testing we were wrong, doh.

Comment 4

[Ed Morley [:emorley]]

https://github.com/mozilla/treeherder-service/commit/426efbecd14a1c427b54918a5dc56670d13575e7