1330560 - Intermittent test_screenshot.py TestScreenCaptureContent.test_capture_flags | AssertionError: u'iVBORw0KGgoA... (images unexpectedly equal))

Status: RESOLVED INVALID

(Testing :: Marionette Client and Harness, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: cbook [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=68319415&repo=autoland

https://archive.mozilla.org/pub/firefox/tinderbox-builds/autoland-win32-debug/1484199796/autoland_win7_vm-debug_test-marionette-bm138-tests1-windows-build14.txt.gz

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

The images (current vs. reference) are actually equal. Which means there is no border around the textfield and the cursor is also not drawn. If that fails more often I will have to take a look into this feature.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+1]]

This is a permafail for Fennec. So I'm going to disable the test completely for now via bug 1330616.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

This might actually be a focus issue. I just noticed this failure locally while letting the test run in the background. As such the wanted focus in the text field will not appear.

Comment 4

[David Burns :automatedtester]

Closing as intermittent has not been seen in last 45 days

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+1]]

This test has been disabled for Fennec immediately due to a high volume of failures. It hasn't been re-enabled / fixed since then, and as such didn't cause any OF entries.

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Sorry, my fault. The original platform for this failure is Win7 VM and not Android.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+1]]

The test is still disabled, and was most likely also dependent on bug 1335085. I will try to re-enable it in combination with bug 1329424.

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Fixed with the landing of my patch on bug 1329424.

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Actually not fixed yet. We still see this failure. I will observe how often and maybe we have to disable it again.

Comment 11

[Intermittent Failures Robot]

4 failures in 4753 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• mozilla-inbound: 1
• mozilla-central: 2

Platform breakdown:

• windows10-64: 2
• windows10-64-shippable: 1
• windows7-32: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1330560&startday=2019-06-10&endday=2019-06-16&tree=all

Comment 12

[Intermittent Failures Robot]

2 failures in 2581 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 2

Platform breakdown:

• windows10-64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1330560&startday=2019-06-17&endday=2019-06-23&tree=all

Comment 13

[Intermittent Failures Robot]

2 failures in 4409 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• mozilla-central: 1

Platform breakdown:

• windows10-64: 1
• windows10-64-ccov: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1330560&startday=2019-06-24&endday=2019-06-30&tree=all

Comment 14

[Intermittent Failures Robot]

9 failures in 3501 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 2
• autoland: 4
• mozilla-inbound: 3

Platform breakdown:

• windows10-64: 7
• windows10-64-shippable: 1
• windows7-32: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1330560&startday=2019-07-01&endday=2019-07-07&tree=all

Comment 15

[Intermittent Failures Robot]

12 failures in 4710 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 8
• mozilla-central: 4

Platform breakdown:

• windows7-32: 2
• windows10-64: 9
• windows10-64-ccov: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1330560&startday=2019-07-08&endday=2019-07-14&tree=all

Comment 16

[Intermittent Failures Robot]

8 failures in 4298 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 6
• try: 1
• mozilla-inbound: 1

Platform breakdown:

• windows10-64: 8

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1330560&startday=2019-07-15&endday=2019-07-21&tree=all

Comment 17

[Intermittent Failures Robot]

6 failures in 4608 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1
• autoland: 4
• try: 1

Platform breakdown:

• windows10-64: 5
• windows10-64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1330560&startday=2019-07-22&endday=2019-07-28&tree=all

Comment 18

[Henrik Skupin [:whimboo][⌚️UTC+1]]

I'm currently not working on this bug given that it doesn't have high priority.

Comment 19

[Intermittent Failures Robot]

7 failures in 4439 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 2
• autoland: 3
• mozilla-central: 2

Platform breakdown:

• windows7-32: 2
• windows10-64: 4
• windows10-64-ccov: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1330560&startday=2019-07-29&endday=2019-08-04&tree=all

Comment 20

[Intermittent Failures Robot]

5 failures in 4666 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-inbound: 1
• autoland: 4

Platform breakdown:

• windows10-64: 4
• windows7-32: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1330560&startday=2019-08-05&endday=2019-08-11&tree=all

Comment 21

[Henrik Skupin [:whimboo][⌚️UTC+1]]

This seems to be a perma failure with my patches from bug 1559592 applied.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=1e793f25d3dbc21e12d70fa7aa26147ae540fa6e&searchStr=marionette%2Cwin

Now that this is reproducible, I could figure out what's wrong.

Comment 22

[Henrik Skupin [:whimboo][⌚️UTC+1]]

I would suggest that we skip the test again for now, means reverting the code removal from:
https://hg.mozilla.org/mozilla-central/rev/8eb5ac3760e7

Once my patch from bug 1559592 landed we can then investigate.

Comment 23

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: Bug 1330560 - [marionette] Temporarily skip TestScreenCaptureContent.test_capture_flags on Windows. r=#webdriver

Temporarily skip on Windows to get the new screenshot API landed.
Given that this change makes this failure permanent it should be
easier for investigation after its landing.

Comment 24

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3b0c482b4235
[marionette] Temporarily skip TestScreenCaptureContent.test_capture_flags on Windows. r=webdriver-reviewers,jgraham

Comment 25

[Alexandru Michis [:malexandru]]

https://hg.mozilla.org/mozilla-central/rev/3b0c482b4235

Comment 26

[Intermittent Failures Robot]

25 failures in 4839 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2
• try: 23

Platform breakdown:

• windows10-64: 8
• windows10-64-shippable: 1
• linux64: 5
• android-em-7-0-x86_64: 2
• windows7-32: 7
• windows10-aarch64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1330560&startday=2019-08-12&endday=2019-08-18&tree=all

Comment 27

[Henrik Skupin [:whimboo][⌚️UTC+1]]

The problem here is that we are trying to capture the caret but given that it is blinking the test can intermittently fail. We are currently discussing this over on bug 1559592, and maybe we should just delete this test.

Comment 28

[Henrik Skupin [:whimboo][⌚️UTC+1]]

This test does no longer exist since the patch on bug 1559592 has been landed.

Comment 29

[BMO Automation]

Moving bug to Testing::Marionette Client and Harness component per bug 1815831.