should have a talos tinderbox error parser

RESOLVED WORKSFORME

Status

Testing
Talos
P3
normal
RESOLVED WORKSFORME
9 years ago
6 years ago

People

(Reporter: dbaron, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [talos])

(Reporter)

Description

9 years ago
In bug 474950 comment 3, ted wrote:

Ideally, we could get the tinderbox error parser to highlight the relevant
failure lines. If need be, we can make an ep_talos.pl to highlight
Talos-specific errors. If they show up in the summary, then it's fairly clear
what happened.

The unit test error parser looks like this:
http://mxr.mozilla.org/mozilla/source/webtools/tinderbox/ep_unittest.pl
I think this is probably the only sane solution to the Talos log problem. If people see something in the log summary, they will pay attention to that and not the bogus disconnection message.

Comment 2

9 years ago
Getting rid of the bogus disconnection message would be nice too ;)
Moving to Future till we can dedicate time to this.
Component: Release Engineering: Talos → Release Engineering: Future

Comment 4

8 years ago
Mass move of bugs from Release Engineering:Future -> Release Engineering. See
http://coop.deadsquid.com/2010/02/kiss-the-future-goodbye/ for more details.
Component: Release Engineering: Future → Release Engineering
Priority: -- → P3
Whiteboard: [talos]

Updated

8 years ago
Duplicate of this bug: 483230
Moving this to Testing:Talos so it can be evaluated.
Component: Release Engineering → Talos
Product: mozilla.org → Testing
QA Contact: release → talos
Version: other → unspecified
Are we still wanting a tinderbox log parser?  Seems that we are quickly moving everything off tinderbox entirely for dashboards...
I'm unable to speak to the wonders of whatever is going to replace tinderbox for storing and serving logs, since years of working on Mozilla stuff has taught me to treat all such promised things that are going to solve all our problems as unicorn-ponies dancing on a rock candy mountain until there's a bug with an 80% patch attached. If there is, I'd be happy to look at it, and say whether it would replace the current need.

An errorparser does three things for you:

* it puts the actual lines from the log that constituted a failure up at the top of the log, linked so you can find them in context
** as a result, tbpl can tell what the error was, can put it in the summary popup that opens when you click on a failing build, and with maybe some tweaking to deal with talos failures maybe not being as strongly associated with test filenames as they are in other suites, could suggest bugs to automatically comment in and star the failure with

* it makes the brief log actually brief, trimming off all the junk and only leaving the context around the failure, so instead of being faced with a whole long log with dozens of steps spewing their environment and outputting nothing or an enormous list of files being unzipped, the log actually shows what went wrong

* it makes the suite author restrict their output to something that can be found - I don't even remember whether there are things other than "FAIL" that mark talos failures, but I do know that the TinderboxPrinting isn't working well. There is currently one fairly common failure on TraceMonkey in dromaeo_sunspider, which I thought for most of the time I was ignoring it was actually two separate failures happening half as often, since the TinderboxPrints were telling me that it was a crash on Linux and a hang on Windows

Those things together are why nearly every bug I file about talos failures, a few months later you ask "is this still happening?" and if I'm in an honest mood I say "nobody has the slightest idea, because talos is annoying to star so everyone just ignores failures in it." In the place where we look for every single other sort of failure to see what happened, with a talos failure it just says "Summary is empty." Unless someone is really bored or really feeling guilty about having maybe caused it, we just figure that means it's a mysterious orange with no output, and ignore it.

Comment 9

6 years ago
This bug no longer seems applicable.  TBPL effectively replaced this.  There has been an ongoing concerted effort to make talos error parsing there sane
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.