Closed Bug 746876 Opened 14 years ago Closed 13 years ago

perma-orange robocop failures in test_bug720538 (Checking page background to the left of the iframe - Color rgba(0,0,255,255) not close enough to expected rgb(0,128,0))

Categories

(Firefox for Android Graveyard :: General, defect)

Product:
Firefox for Android Graveyard
Component:
General
Version:
14 Branch
Platform:
All
Android
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(firefox23 fixed, firefox24 fixed, blocking-fennec1.0 -)

Status:
RESOLVED FIXED
Milestone:
Firefox 24
Tracking Flags:
Tracking Status
firefox23 --- fixed
firefox24 --- fixed
blocking-fennec1.0 --- -

People

(Reporter: kats, Assigned: kats)

References

Details

(Keywords: intermittent-failure, regression, Whiteboard: [perma-orange])

Attachments

(3 files)

disable test
721 bytes, patch
kats
: review+
akeybl
: approval-mozilla-aurora+
Details | Diff | Splinter Review
Re-enable test
1.96 KB, patch
jmaher
: review+
Details | Diff | Splinter Review
Fix double tap coordinates and re-enable test
2.10 KB, patch
gbrown
: review+
Details | Diff | Splinter Review
Blocks: 438871
Whiteboard: [orange]
this might justifyt backing out bug 743800, but the test failure itself will no block the release. Kats, can you push a back out of 743800 to try and use self serve to get enough runs to be able to prove if that caused the randomness or not
blocking-fennec1.0: ? → -
(In reply to Brad Lassey [:blassey] from comment #24) > Kats, can you push a back out of 743800 to try and use self serve to get > enough runs to be able to prove if that caused the randomness or not Pushed backout to https://tbpl.mozilla.org/?tree=Try&rev=d229c921a6ff
(In reply to Kartikaya Gupta (:kats) from comment #28) > Pushed backout to https://tbpl.mozilla.org/?tree=Try&rev=d229c921a6ff Backout does show it going back to green. I don't want to land the backout though because that it does fix a real issue, I'd rather just debug the failure.
This is perma-orange on Aurora (14.0a2) since yesterday's uplift.
Oh, I see it's actually perma-orange on mozilla-central since 743800 landed. If we are not backing out the bug, we need to disable this test or change it to an expected-fail.
status-firefox14: --- → affected
Keywords: regression
Summary: Robocop: failures in test_bug720538 (Checking page background to the left of the iframe - Color rgba(0,0,255,255) not close enough to expected rgb(0,128,0)) → perma-orange robocop failures in test_bug720538 (Checking page background to the left of the iframe - Color rgba(0,0,255,255) not close enough to expected rgb(0,128,0))
Whiteboard: [orange] → [orange][perma-orange]
Version: unspecified → Firefox 14
Attached patch disable test — — Splinter Review
Yup, it's perma-orange alright. It looks green about 2% of the time, only because it sometimes manages to exit without running any tests at all: https://tbpl.mozilla.org/php/getParsedLog.php?id=11008121&tree=Mozilla-Inbound 0 INFO SimpleTest START 1 INFO TEST-START | test_bug720538 INFO | automation.py | Application ran for: 0:00:59.636577 INFO | automation.py | Reading PID log: /tmp/tmpcdUEtlpidlog getting files in '/mnt/sdcard/tests/profile/minidumps/' WARNING | automationutils.processLeakLog() | refcount logging is off, so leaks can't be detected! Let's disable this until we have time to make it pass.
Attachment #618489 - Flags: review?(bugmail.mozilla)
Attachment #618489 - Flags: review?(bugmail.mozilla) → review+
Disabled test on inbound. (Please leave bug open when landing on m-c.) https://hg.mozilla.org/integration/mozilla-inbound/rev/a6a22e633e7e
Whiteboard: [orange][perma-orange] → [orange][perma-orange][leave open]
Comment on attachment 618489 [details] [diff] [review] disable test [Approval Request Comment] Regression caused by (bug #): bug 743800 User impact if declined: robocop is perma-orange on aurora Testing completed (on m-c, etc.): landed on inbound Risk to taking this patch (and alternatives if risky): test-only change to disable a perma-orange test String changes made by this patch: none
Attachment #618489 - Flags: approval-mozilla-aurora?
(In reply to Matt Brubeck (:mbrubeck) from comment #298) > Disabled test on inbound. (Please leave bug open when landing on m-c.) > https://hg.mozilla.org/integration/mozilla-inbound/rev/a6a22e633e7e https://hg.mozilla.org/mozilla-central/rev/a6a22e633e7e
Comment on attachment 618489 [details] [diff] [review] disable test [Triage Comment] Test-only change in support of Fennec Native. Approved for Aurora 14.
Attachment #618489 - Flags: approval-mozilla-aurora? → approval-mozilla-aurora+
Test disabled on Aurora now: https://hg.mozilla.org/releases/mozilla-aurora/rev/b8d1bce48c21
No longer blocks: 438871
status-firefox14: affected → ---
Whiteboard: [orange][perma-orange][leave open] → [perma-orange][test disabled]
Attached patch Re-enable test — — Splinter Review
This test seems to work fine now. I ran it locally 30+ times and the only failures I saw were the ones in bug 767215 (for which I have a patch on that bug). So, time to re-enable the test.
Attachment #635872 - Flags: review?(jmaher)
Comment on attachment 635872 [details] [diff] [review] Re-enable test Review of attachment 635872 [details] [diff] [review]: ----------------------------------------------------------------- thanks!
Attachment #635872 - Flags: review?(jmaher) → review+
I backed it out again since it looks like the failures are still happening on tbpl. I swear it worked fine when I ran it on the tegra board in the MoTo office. Damn you tegra boards! https://hg.mozilla.org/integration/mozilla-inbound/rev/7a882620cbd3
Target Milestone: Firefox 16 → ---
Just an update: I pushed a try run with this test enabled and it hasn't magically fixed itself after all. https://tbpl.mozilla.org/?tree=Try&rev=7eea5e516dd3 In theory this should be debuggable by pushing logging to tryserver and iterating.
Adding keyword to get this off my ~"has orange in whiteboard but missing intermittent-failure keyword so TBPL can't see it" whine.
Keywords: intermittent-failure
Status: ASSIGNED → NEW
The test seems to be working fine now. Try build at https://tbpl.mozilla.org/?tree=Try&rev=9ed690190559 has a bunch of runs of this test (modified robocop.ini) and it's all green.
Attachment #755762 - Flags: review?(gbrown)
Comment on attachment 755762 [details] [diff] [review] Fix double tap coordinates and re-enable test Review of attachment 755762 [details] [diff] [review]: ----------------------------------------------------------------- Thanks!
Attachment #755762 - Flags: review?(gbrown) → review+
https://hg.mozilla.org/mozilla-central/rev/db6844d14219
Status: NEW → RESOLVED
Closed: 13 years ago
Flags: in-testsuite+
Resolution: --- → FIXED
Whiteboard: [perma-orange][test disabled] → [perma-orange]
Target Milestone: --- → Firefox 24
Product: Firefox for Android → Firefox for Android Graveyard
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: