Closed Bug 746876 Opened 8 years ago Closed 7 years ago

perma-orange robocop failures in test_bug720538 (Checking page background to the left of the iframe - Color rgba(0,0,255,255) not close enough to expected rgb(0,128,0))

Categories

(Firefox for Android :: General, defect)

14 Branch
All
Android
defect
Not set

Tracking

()

RESOLVED FIXED
Firefox 24
Tracking Status
firefox23 --- fixed
firefox24 --- fixed
blocking-fennec1.0 --- -

People

(Reporter: kats, Assigned: kats)

References

Details

(Keywords: intermittent-failure, regression, Whiteboard: [perma-orange])

Attachments

(3 files)

Blocks: 438871
Whiteboard: [orange]
this might justifyt backing out bug 743800, but the test failure itself will no block the release.

Kats, can you push a back out of 743800 to try and use self serve to get enough runs to be able to prove if that caused the randomness or not
blocking-fennec1.0: ? → -
(In reply to Brad Lassey [:blassey] from comment #24)
> Kats, can you push a back out of 743800 to try and use self serve to get
> enough runs to be able to prove if that caused the randomness or not

Pushed backout to https://tbpl.mozilla.org/?tree=Try&rev=d229c921a6ff
(In reply to Kartikaya Gupta (:kats) from comment #28)
> Pushed backout to https://tbpl.mozilla.org/?tree=Try&rev=d229c921a6ff

Backout does show it going back to green. I don't want to land the backout though because that it does fix a real issue, I'd rather just debug the failure.
This is perma-orange on Aurora (14.0a2) since yesterday's uplift.
Oh, I see it's actually perma-orange on mozilla-central since 743800 landed.  If we are not backing out the bug, we need to disable this test or change it to an expected-fail.
Keywords: regression
Summary: Robocop: failures in test_bug720538 (Checking page background to the left of the iframe - Color rgba(0,0,255,255) not close enough to expected rgb(0,128,0)) → perma-orange robocop failures in test_bug720538 (Checking page background to the left of the iframe - Color rgba(0,0,255,255) not close enough to expected rgb(0,128,0))
Whiteboard: [orange] → [orange][perma-orange]
Version: unspecified → Firefox 14
Attached patch disable testSplinter Review
Yup, it's perma-orange alright.  It looks green about 2% of the time, only because it sometimes manages to exit without running any tests at all:

https://tbpl.mozilla.org/php/getParsedLog.php?id=11008121&tree=Mozilla-Inbound
0 INFO SimpleTest START
1 INFO TEST-START | test_bug720538
INFO | automation.py | Application ran for: 0:00:59.636577
INFO | automation.py | Reading PID log: /tmp/tmpcdUEtlpidlog
getting files in '/mnt/sdcard/tests/profile/minidumps/'
WARNING | automationutils.processLeakLog() | refcount logging is off, so leaks can't be detected!

Let's disable this until we have time to make it pass.
Attachment #618489 - Flags: review?(bugmail.mozilla)
Attachment #618489 - Flags: review?(bugmail.mozilla) → review+
Disabled test on inbound.  (Please leave bug open when landing on m-c.)
https://hg.mozilla.org/integration/mozilla-inbound/rev/a6a22e633e7e
Whiteboard: [orange][perma-orange] → [orange][perma-orange][leave open]
Comment on attachment 618489 [details] [diff] [review]
disable test

[Approval Request Comment]
Regression caused by (bug #): bug 743800

User impact if declined: robocop is perma-orange on aurora

Testing completed (on m-c, etc.): landed on inbound

Risk to taking this patch (and alternatives if risky): test-only change to disable a perma-orange test

String changes made by this patch: none
Attachment #618489 - Flags: approval-mozilla-aurora?
(In reply to Matt Brubeck (:mbrubeck) from comment #298)
> Disabled test on inbound.  (Please leave bug open when landing on m-c.)
> https://hg.mozilla.org/integration/mozilla-inbound/rev/a6a22e633e7e

https://hg.mozilla.org/mozilla-central/rev/a6a22e633e7e