If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

tinderbox log highlighter should highlight which build steps failed

RESOLVED WONTFIX

Status

Release Engineering
General
P3
normal
RESOLVED WONTFIX
8 years ago
4 years ago

People

(Reporter: dbaron, Assigned: coop)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [tinderbox])

Attachments

(2 attachments)

(Reporter)

Description

8 years ago
I believe there's a list of regular expressions somewhere that controls which lines get highlighted at the top of a tinderbox log.

I think it would make many logs *much* more comprehensible if the logs highlighted the messages:

... failed
... warnings

I'm not sure whether you can use multi-line regexes or not; if you can, it should be trivial:  just highlight any line ending in " failed" or " warnings" that's one line after the line "======== BuildStep started ========".

If not, I think it's worth tweaking the code that converts the buildbot logs to tinderbox logs so that these messages can be highlighted; this would help people understand all the weird errors by highlighting the part of the build process that had the failure.

Comment 1

8 years ago
Bug 516443 is to be considered as well.
http://mxr.mozilla.org/mozilla/source/webtools/tinderbox/ep_unix.pl
and
http://mxr.mozilla.org/mozilla/source/webtools/tinderbox/ep_unittest.pl

Contain the regexes that are used for normal builds and unittest builds, respectively.
This sounds like a good idea,  I am going to move to future until someone has time to work on this.
Component: Release Engineering → Release Engineering: Future
(Reporter)

Comment 4

8 years ago
This is pretty simple to do.  It just requires a simple modification to the printing code added in:
http://hg.mozilla.org/build/buildbot/rev/f1fda48634d6
and then changes to the files ted mentioned above.  I'll post the patches that should be necessary shortly, but I can't test them.
(Reporter)

Comment 5

8 years ago
Created attachment 407100 [details] [diff] [review]
untested buildbot patch
(Reporter)

Comment 6

8 years ago
Created attachment 407101 [details] [diff] [review]
untested tinderbox patch
(Reporter)

Comment 7

8 years ago
This would really make lots of obscure failures easier to diagnose from the logs.  Any chance I could talk somebody into working on this?
I've hacked the error parser without ever actually setting up a tinderbox. I just have a "test-errorparser.pl" that contains:
require 'ep_unittest.pl';

while(<>) {
    if (has_error($_)) {
        print;
    }
}

I stick that in the webtools/tinderbox dir, run it, and paste in log lines, and it will echo back lines that match the error regex. If you're comfortable with that, then get review from cls, and that's all that's necessary.
Here's a failure log from today where we could've used this:
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1259886360.1259887933.11523.gz

The failure was this, during crashtests:
{
REFTEST TEST-PASS | file:///builds/moz2_slave/mozilla-central-macosx-opt-unittest-everythingelse/build/reftest/tests/dom/base/crashtests/371124-2.html | (LOAD ONLY)
process killed by signal 30
program finished with exit code -1
elapsedTime=21.912001
TinderboxPrint: crashtest<br/><em class="testfail">T-FAIL</em>
Unknown Error: command finished with exit code: -1
=== Output ended ===
======== BuildStep ended ========
}
instead of making buildbot-side changes, could you a regex like "program finished with exit code [^0]"
(Reporter)

Comment 11

8 years ago
(In reply to comment #10)
> instead of making buildbot-side changes, could you a regex like "program
> finished with exit code [^0]"

I want to catch *all* the cases of build step failure (does that really catch all of them), and I want it to highlight a useful description of what failed.
(Reporter)

Comment 12

8 years ago
And, to be clear:  the buildbot changes are just slightly modifying what's already one of our local patches to buildbot.

Updated

8 years ago
Duplicate of this bug: 513929

Updated

8 years ago
Duplicate of this bug: 516443

Updated

8 years ago
Duplicate of this bug: 457855
(Assignee)

Comment 16

8 years ago
Mass move of bugs from Release Engineering:Future -> Release Engineering. See
http://coop.deadsquid.com/2010/02/kiss-the-future-goodbye/ for more details.
Component: Release Engineering: Future → Release Engineering
Priority: -- → P3
(Assignee)

Updated

8 years ago
Whiteboard: [tinderbox]
(Assignee)

Updated

7 years ago
Assignee: nobody → ccooper
(Assignee)

Updated

7 years ago
Whiteboard: [tinderbox] → [tinderbox][triagefollowup]
(Assignee)

Comment 17

7 years ago
dbaron: would you still like this? 

We (releng) would like to stop propping up tinderbox in an effort to get people to switch to TBPL/BBPL. If we can accomplish the same thing with a buildbot-only patch, we'd certainly prefer that.
Whiteboard: [tinderbox][triagefollowup] → [tinderbox]
(Assignee)

Comment 18

7 years ago
Based on the discussions at the "How do we kill tinderbox?" and "War on Orange" meetings at the all-hands, it looks like the Orange Factor tool will be replacing the tinderbox log store soon (matter of weeks?).

Let's file Orange Factor bugs to make sure log parsing meets our current requirements.
Status: NEW → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → WONTFIX
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.