816971 - TBPL should recognise '^[A-Za-z]+Error:' and '^BaseException:' to catch most python exceptions

Status: RESOLVED FIXED

(Tree Management Graveyard :: TBPL, defect)

Description

[Ed Morley [:emorley]]

We already do:
{
       || preg_match("/^DMError:/", $line) // . . . . . . . . . .  . . . . . . DeviceManager
       || preg_match("/^talosError:/", $line)  // . . . . . . . . . .  . . . . Talos
}

But this misses IOError/KeyError/various other common built-in or custom exceptions we see in on TBPL.

Instead, how about:
"/^[A-Za-z]+Error:/"

...I'll need to test carefully locally to ensure there aren't any false positives, but I'm hoping we can solve many problems in one with this.

Comment 1

[Ed Morley [:emorley]]

Attachment: Patch v1

The new regex meant I was able to catch bug 817349 :-)

No obvious false-parsing-positives that I can find when testing locally; next step testing on tbpl-dev with a larger dataset than is viable than when waiting for the local VM to download logs.

Comment 2

[Jeff Hammel]

Comment on attachment 687474 [details] [diff] [review]
Patch v1

Looks good to me.  I also am concerned about false positives but will trust your testing.

Comment 3

[Ed Morley [:emorley]]

Ta :-)

https://hg.mozilla.org/users/mstange_themasta.com/tinderboxpushlog/rev/58479c0656f2

(Will check tbpl-dev logs before pushing to prod)

Comment 4

[Ed Morley [:emorley]]

All the suites I checked were fine, other than jsreftest:
eg
https://tbpl-dev.allizom.org/php/getParsedLog.php?id=17559472&tree=Mozilla-Inbound

{
TypeError: invalid XML name 0
ReferenceError: foo is not defined
TypeError: anonymous function does not always return a value
TypeError: anonymous function does not always return a value
SyntaxError: return not in function
SyntaxError: function statement requires a name
InternalError: too much recursion
ReferenceError: a is not defined | undefined | 45
SyntaxError: property name a appears more than once in object literal
SyntaxError: property name 1 appears more than once in object literal
TypeError: can't redefine non-configurable property '5'
TypeError: variable v redeclares argument
TypeError: Value does not implement interface Node.
TypeError: z is not a function
SyntaxError: return not in function
TypeError: 4 is not a function
TypeError: 6 is not a function
TypeError: 3.a is undefined
ReferenceError: d is not defined
ReferenceError: d is not defined
TypeError: e is not a function
TypeError: null has no properties
TypeError: (intermediate value)[1] is undefined
TypeError: already executing generator
TypeError: already executing generator
TypeError: already executing generator
SyntaxError: let declaration not directly within block
SyntaxError: let declaration not directly within block
SyntaxError: let declaration not directly within block
SyntaxError: let declaration not directly within block
ReferenceError: d is not defined
TypeError: missing argument 1 when calling function watch
TypeError: /a/g is not a function
TypeError: (intermediate value).__proto__ is not a function
TypeError: null has no properties
}

I haven't checked yet whether these are errors that are expected or not. (jsreftest run was green)

Comment 5

[Ed Morley [:emorley]]

Backed out for now so as to not block up tbpl-dev's next push to prod:
https://hg.mozilla.org/users/mstange_themasta.com/tinderboxpushlog/rev/f745614241f0

I'll take another look at this soon.

Comment 6

[Ed Morley [:emorley]]

We don't seem to have many failures that aren't already matched at the moment; I'll come back to this if we find the few Exceptions we already support aren't sufficient.

Comment 7

[Ed Morley [:emorley]]

I think the correct solution here is to get the JS tests print() to prefix the output with TEST-INFO | 

http://mxr.mozilla.org/mozilla-central/source/js/src/tests/browser.js#33

Comment 8

[Ed Morley [:emorley]]

I've pinged bug 829684, if we can get that resolved, then this can reland :-)

Comment 9

[Ed Morley [:emorley]]

I'm just going to reland this; testing in Vagrant showed the false positives are mostly gone - and the ones that remain won't even be hit until we start dealing with the mozharness line prefix (since the rules here use '^'). The latter does mean we won't see the full benefit of this immediately on jobs that use the mozharness logging format, but there are others that it will still help (eg bug 845525).

I've also added '^BaseException' to catch bug 835588 and similar.

Comment 10

[Ed Morley [:emorley]]

Attachment: Patch v2

Comment 11

[Ed Morley [:emorley]]

https://hg.mozilla.org/webtools/tbpl/rev/8538e6d7fb63

Comment 12

[Ed Morley [:emorley]]

In production :-)