Investigate intermittent talos failures mozilla-central

RESOLVED FIXED

Status

Release Engineering
General
RESOLVED FIXED
9 years ago
4 years ago

People

(Reporter: nthomas, Assigned: nthomas)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Assignee)

Description

9 years ago
Particularly on Mac, but also on windows, there have been alot of Ts and Tgfx failures on Talos as the watchdog thread patch gets landed and backed out (bug 453157). We should figure out what's going on.
(Assignee)

Comment 1

9 years ago
Mac Talos data:

build start - slave - clobber/dep - revision (watchdog in/out) - talos results
22/11 11:27 - moz2-darwin9-slave02 - dep - flaf606... (out) - 5x talos Ts fail
22/11 09:27 - bm-xserve18 - dep - 5d3767... (out) - 5x talos Ts fail
22/11 07:36 - bm-xserve17 - clobber - 5d3767.. (out) - 5x talos green
22/11 05:27 - bm-xserve18 - dep - 04cecb... (in) - 2 green, 2 Ts fail, 1 Tp crash
22/11 03:27 - moz2-darwin9-slave02 - dep - 04cecb... (in) - 2 green, 1 Tgfx fail
22/11 01:27 - bm-xserve18 - dep - 04cecb... (in) - 3 green, 2 Tgfx fail

Working hypothesis - from the 07:36 build, a clobber is required to make the patch apply/unapply properly (something about the patch doesn't get rebuilt properly). So I've clobbered the the builds on bm-xserve16,17,18,19,moz2-darwin9-slave02 (both optimised and leak test).
(Assignee)

Comment 2

9 years ago
Also clobbered the build (but not leak) on moz2-win32-slave1,02,17,18, since those have all produced builds that Talos barfed on in the last few hours.
(Assignee)

Comment 3

9 years ago
Ok, all is green now (except for random unit test failures).

I'm not sure what is causing the Mac builds to be broken when bug 453157 lands and is backed out, except for maybe the start of build clean up not working as well there. The build system does 
 make  -C obj-firefox/ppc
 rm -f -rf ./dist/sdk
 rm -f -rf ./dist/include
 rm -f -rf ./dist/private
 rm -f -rf ./dist/public
 rm -f -rf ./dist/bin/components
 rm -f -rf _tests
and the same for i386 in it's turn. The universal build is created elsewhere though. bug 455940 is related.

If you take a universal mac build without the patch from bug 453157, are any steps necessary for the patch to build cleanly ? Bear in mind that "make package" in <objdir>/ppc/ will give you a dmg that may differ slightly from what is in dist/bin.
Blocks: 453157
(Assignee)

Updated

9 years ago
Status: ASSIGNED → RESOLVED
Last Resolved: 9 years ago
Resolution: --- → FIXED

Updated

9 years ago
Blocks: 466399
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.