Closed Bug 807707 Opened 9 years ago Closed 9 years ago

If a filename can't be found in a log failure line, search for bug suggestions using the whole line instead of giving up

Categories

(Tree Management Graveyard :: TBPL, defect)

defect
Not set
major

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: emorley, Assigned: emorley)

References

(Blocks 1 open bug)

Details

(Whiteboard: [sheriff-want])

Attachments

(5 files)

ie we currently do:

>   protected function processLine(&$annotatedLines, $line) {
>     $annotatedLines[] = htmlspecialchars($line);
> 
>     $tokens = preg_split("/\s\\|\s/", $line);
>     if (count($tokens) < 3)
>       return;

Instead of returning, let's just use the whole line, so we can star talosError and more.

#107 of the "why didn't we do this sooner" list (albeit I guess until recently we didn't have annotated summary support for talosError, but still).
Blocks: 788518, 778688
...since we'll be searching for more than just filenames now.
Attachment #677488 - Flags: review?(arpad.borsos)
Diff looks noisier than it actually is.

Only real change is that we replace the return with '$searchTerm = trim($line);'
Attachment #677490 - Flags: review?(arpad.borsos)
* Trim $searchTerm for both filename & whole line codepaths
* Bail early if post-trim the search term is empty
* Cap the search term at 200 characters, because:
Bugzilla's max summary length is 256 characters, so once "Intermittent " and platform/suite information is prefixed, it often only leaves ~200 characters for the failure, so anything more will be truncated in the summary. Capping the length (a) prevents crazy length bzapi requests, and (b) might mean we still match the truncated failure in the summary.
Attachment #677494 - Flags: review?(arpad.borsos)
Attached image Screenshot
huzzah!
Attachment #677488 - Flags: review?(arpad.borsos) → review+
Attachment #677490 - Flags: review?(arpad.borsos) → review+
Attachment #677494 - Flags: review?(arpad.borsos) → review+
Attached file bzapi calls
A quick glance at the bzapi requests being made with this patch shows that we now search for a few extra things that are false positives. However many of them will require switching to a regex-based blacklist, so I think we should defer to another bug, given we already search for stupid things (a la bug 788518).

eg:
{
PGOCVT : fatal error PG0001: An unexpected internal error was detected in source file 'f:\dd\vctools\compiler\utc\src\tools\pogo\cvtpgd\cvtpgd.cpp', line 800.
Thread 12 (crashed)
The web-page 'stop build' button was pressed by 'nchen@mozilla.com': Cancelled via self-serve
abort: HTTP Error 404: Not Found
buildbot.slave.commands.TimeoutError: command timed out: 1200 seconds without output, attempting to kill
buildbot.slave.commands.TimeoutError: command timed out: 2400 seconds without output, killing pid 34848
make[5]: *** [BaselineFrameInfo.o] Error 1
}

We'll also need to deal with mozharness' prefixing:

{
09:15:44     INFO -  TEST-UNEXPECTED-FAIL : TimeoutException: socket.timeout
09:16:03     INFO -  TEST-UNEXPECTED-FAIL : TimeoutException: socket.timeout
}

On the plus side, we now catch all of the following (and this is just from the last hour or two of running TBPL locally):

{
talosError: "Could not find report in browser output: [('tsformat', ('__start_report', '__end_report')), ('tpformat', ('__start_tp_report', '__end_tp_report'))] [browser_output.txt]"
talosError: 'browser non-zero return code (1) [browser_output.txt]'
talosError: 'crash during run (stack found)'
talosError: 'failed to initialize browser'
talosError: 'initialization timed out'
talosError: 'stack found after process termination (org.mozilla.fennec: terminated by testAgent.killProcess, plugin-container: terminated by testAgent.killProcess, crashreporter: terminated by testAge
talosError: Unable to proceed with missing counter 'tp5n_main_startup_fileio'
TEST-UNEXPECTED-FAIL | Disconnect Error: Application unexpectedly closed
TEST-UNEXPECTED-FAIL | | exception while running reftests
}
Depends on: 807786
With this patch we can now star all of:
https://bugzil.la/803413,732325,796914,690176,739089,686085,686245,802475,781107,675297,806123,798219,781106,739089,802289,690176,782189
(and that's just open bugs & a very rough search).

:-D
In production.
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
(In reply to Ed Morley [:edmorley UTC+0] from comment #5)
> 09:15:44     INFO -  TEST-UNEXPECTED-FAIL : TimeoutException: socket.timeout
> 09:16:03     INFO -  TEST-UNEXPECTED-FAIL : TimeoutException: socket.timeout

This will soon be fixed in mozharness afaict (bug 804366).
Depends on: 809436
Product: Webtools → Tree Management
Product: Tree Management → Tree Management Graveyard
You need to log in before you can comment on or make changes to this bug.