mochitest-browser-chrome randomly turns tree orange without reporting any failures

RESOLVED WORKSFORME

Status

Testing
Mochitest
--
major
RESOLVED WORKSFORME
8 years ago
7 years ago

People

(Reporter: dbaron, Unassigned)

Tracking

({helpwanted})

Trunk
helpwanted
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1256784432.1256787026.4781.gz&fulltext=1
OS X 10.5.2 mozilla-central test everythingelse on 2009/10/28 19:47:12  


In this log, mochitest-browser-chrome reported buildbot status "warnings", but there are no failures logged.

It's not clear to me whether this is a build-automation-side bug or a runtests.py bug; I'm not sure how the status is set.
Most probably due to bug 473506 "again": I'm trying to check this locally...
Blocks: 473506
I copied the whole mochitest-browser-chrome log and locally ran
{{
  log = open("steps__in.log", "r")
  failIdent = r"^\t(Fail|Failed): 0"
  if not re.search(failIdent, log.read(), re.MULTILINE) or \
     re.search("^TEST-UNEXPECTED-", log.read(), re.MULTILINE):
      print "WARNINGS"
  else:
      print "PASS"
  log.close()
}}
which passes.

I have no idea about what is going wrong on the box :-(
Keywords: helpwanted
[
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1256788103.1256790593.11034.gz&fulltext=1
OS X 10.5.2 mozilla-central test opt everythingelse on 2009/10/28 20:48:23

WARNING | automationutils.processLeakLog() | refcount logging is off, so leaks can't be detected!
]
is green.
The test details have a "few" differences though both builds reports '3885/0/5'.
Neither this nor the 'no refcount' warning should make a difference.

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1256786847.1256789290.29506.gz&fulltext=1
Linux mozilla-central test opt everythingelse on 2009/10/28 20:27:27
is green too.

Like comment 2, this would mean bug 473506 might not be involved after all.
Yet, I have even less idea what the cause could be:
over to RelEng, unless someone else has an idea.
Severity: normal → major

Comment 4

8 years ago
I ran the same script Serge did locally (on Mac) and it passed for me as well.

I've backed out the offending patch in bug 473506 and reconfig-ed the production-masters. We'll try to figure this out in staging.

Comment 5

8 years ago
(In reply to comment #4) 
> I've backed out the offending patch in bug 473506 and reconfig-ed the
> production-masters. We'll try to figure this out in staging.

Er, I meant bug 473259.
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1256793537.1256795953.4162.gz
OS X 10.5.2 mozilla-central test everythingelse on 2009/10/28 22:18:57
is green.

Then, I would be 95% sure it's unrelated to bug 473506.
And not have any clue about what happened on comment 0 build. :-(

***

(In reply to comment #4)
> I've backed out the offending patch in bug 473506 and reconfig-ed the
> production-masters.

I didn't want that (yet)...

> We'll try to figure this out in staging.

That's more like what I expected (ftb).
This looks like another instance of this:
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1256797391.1256799289.7491.gz
WINNT 5.2 mozilla-central test opt everythingelse on 2009/10/28 23:23:11
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1256796017.1256798109.27384.gz
OS X 10.5.2 mozilla-central test everythingelse on 2009/10/28 23:00:17
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1256796041.1256798139.27702.gz
OS X 10.5.2 mozilla-central test everythingelse on 2009/10/28 23:00:41
This looks like it's happening on the debug unit tester on Windows too:

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox-Unittest/1256819432.1256823850.30495.gz
WINNT 5.2 mozilla-central test debug everythingelse on 2009/10/29 05:30:32
It seems that http://hg.mozilla.org/mozilla-central/rev/2bf3249fe4bb should fix this.
(In reply to comment #7)
(In reply to comment #8)
(In reply to comment #9)

No, these are a fallout of bug 473259 comment 55 "mess".

(In reply to comment #10)

This additional backout works around the added mess,
but we still have no explanation about comment 0.
No more reports than the initial one.
I have no idea what caused the latter.
But it looks improbable to be bug 473506, which is now landed on all 3 branches (without complains).
No longer blocks: 473506
Is this still a problem?
No.
Status: NEW → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.