1413821 - Permafail test_click.py TestClick.test_inclusive_descendant, TestLegacyClick.test_inclusive_descendant | AssertionError: -1 == -1

Status: RESOLVED WONTFIX

(Testing :: Marionette Client and Harness, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: hskupin [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=141450131&repo=autoland

https://queue.taskcluster.net/v1/task/ShuG2OcQRya5yWTrUrb4cw/runs/0/artifacts/public/logs/live_backing.log

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

[task 2017-11-01T21:01:16.870Z] 21:01:16  WARNING -  TEST-UNEXPECTED-FAIL | test_click.py TestClick.test_inclusive_descendant | AssertionError: -1 == -1
[task 2017-11-01T21:01:16.871Z] 21:01:16     INFO -  Traceback (most recent call last):
[task 2017-11-01T21:01:16.871Z] 21:01:16     INFO -    File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 156, in run
[task 2017-11-01T21:01:16.872Z] 21:01:16     INFO -      testMethod()
[task 2017-11-01T21:01:16.872Z] 21:01:16     INFO -    File "/builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit/test_click.py", line 143, in test_inclusive_descendant
[task 2017-11-01T21:01:16.872Z] 21:01:16     INFO -      self.assertNotEqual(select.get_property("selectedIndex"), -1)

This is an Android only failure. And as it looks like the issued click does not correctly select any item of the <select> node.

I would temporarily disable the assertion for Android, so it only gets run for desktop builds.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I submitted a try build to:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=a51ee8480c628fd1409d08607be939ec18ad3ad5

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: Android skip patch

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: Android skip patch

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

The last try job shows this test as passing:

> [task 2017-11-02T09:38:12.174Z] 09:38:12     INFO -  TEST-START | test_click.py TestClick.test_inclusive_descendant
> [task 2017-11-02T09:38:22.570Z] 09:38:22     INFO -  TEST-PASS | test_click.py TestClick.test_inclusive_descendant | took 10395ms

I think we should skip the review cycle and land it due to merge day.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/mozilla-central/rev/b5a3b8ef6902
Skip final assertion in test_inclusive_descendant for Android. r=test-fix a=test-fix

Comment 7

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/b5a3b8ef6902

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+2]]

We have to keep this bug open for the real fix. Marking as disabled instead.

Comment 9

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/6cf9fe9ff66d (FIREFOX_57b_RELBRANCH)
https://hg.mozilla.org/releases/mozilla-release/rev/22f767a4ca1d

Comment 10

[Intermittent Failures Robot]

10 failures in 857 pushes (0.012 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 8
* mozilla-central: 2

Platform breakdown:
* android-4-3-armv7-api16: 10

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1413821&startday=2017-10-30&endday=2017-11-05&tree=all

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Lets try to unskip once Mn jobs run on the faster Android 7.0 x86 platform.

Comment 12

[Henrik Skupin [:whimboo][⌚️UTC+2]]

We moved away from the ARM platform and this failure no longer occurs on packet.net with Android 7.0 x86_64.

Comment 13

[BMO Automation]

Moving bug to Testing::Marionette Client and Harness component per bug 1815831.