Intermittent test_tabbar_session_restore_button.py TestTabbarSessionRestoreButton.test_window_resizing | AssertionError: u'visible' != 'hidden'

VERIFIED WORKSFORME

Status

Testing
Firefox UI Tests
P5
normal
VERIFIED WORKSFORME
6 months ago
2 months ago

People

(Reporter: Treeherder Bug Filer, Unassigned)

Tracking

({bulk-close-intermittents, intermittent-failure})

Version 3
bulk-close-intermittents, intermittent-failure
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [stockwell fixed:backout])

Attachments

(1 attachment)

Comment 1

6 months ago
21 failures in 1008 pushes (0.021 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 9
* mozilla-central: 6
* mozilla-inbound: 5
* try: 1

Platform breakdown:
* linux64: 14
* linux32: 7

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1384875&startday=2017-07-24&endday=2017-07-30&tree=all
This started July 27th. Erica, do you know if something has been changed for the button at this time? It's visible when it should be hidden. Maybe it's related to the window size?

[task 2017-07-27T09:35:49.103230Z] 09:35:49     INFO - TEST-UNEXPECTED-FAIL | test_tabbar_session_restore_button.py TestTabbarSessionRestoreButton.test_window_resizing | AssertionError: u'visible' != 'hidden'
[task 2017-07-27T09:35:49.104485Z] 09:35:49     INFO - Traceback (most recent call last):
[task 2017-07-27T09:35:49.108787Z] 09:35:49     INFO -   File "/home/worker/workspace/build/venv/local/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 156, in run
[task 2017-07-27T09:35:49.108847Z] 09:35:49     INFO -     testMethod()
[task 2017-07-27T09:35:49.109270Z] 09:35:49     INFO -   File "/home/worker/workspace/build/tests/firefox-ui/tests/functional/sessionstore/test_tabbar_session_restore_button.py", line 77, in test_window_resizing
[task 2017-07-27T09:35:49.109346Z] 09:35:49     INFO -     self.assertEqual(wrapper.value_of_css_property('visibility'), 'hidden')
Flags: needinfo?(ewright)
Created attachment 8892844 [details]
screenshot.png
Nothing has been changed for the button. It could be window size, if the window does not successfully resize small then the button will show.
Flags: needinfo?(ewright)
Hm, so the failing lines are:

>        self.marionette.set_window_size(335, 335)
>        self.assertEqual(wrapper.value_of_css_property('visibility'), 'hidden')

Given the screenshot for this failure the window is of size 1200 x 1200 and not 335 x 335.

Andreas, is that another instance of a race condition when we return to early from set window size? Will it be fixed by one of your patches?
Flags: needinfo?(ato)
(In reply to Henrik Skupin (:whimboo) from comment #5)

> Hm, so the failing lines are:
> 
> >        self.marionette.set_window_size(335, 335)
> >        self.assertEqual(wrapper.value_of_css_property('visibility'), 'hidden')
> 
> Given the screenshot for this failure the window is of size 1200 x
> 1200 and not 335 x 335.
> 
> Andreas, is that another instance of a race condition when we
> return to early from set window size? Will it be fixed by one of
> your patches?

I think that’s a reasonable assumption considering it only fails
on Linux, where we now window resizing happens asynchronously.

Despite all the changes I have made to increase the determinism
of WebDriver:SetWindowRect, the core of the problem here is that
we can’t rely on window.{outerWidth,outerHeight} when polling
for the window’s size to have changed.  These two DOM properties
are populated with the new size before the resize transition has
completed.

As you can tell from
https://searchfox.org/mozilla-central/rev/f0e4ae5f8c40ba742214e89aba3f554da0b89a33/testing/marionette/driver.js#1345-1457,
we have put into place a large number of guards to prevent this:
it first waits for the "resize" DOM event to fire, then throttles
the event by requesting an animation frame after the main thread is
idle.  Then as a last safeguard it poll-waits for the window’s
size to have changed from its original dimensions.

Unfortunately, I don’t have any more good ideas for how to make
this more reliable.  I should add that this is also an unsolved
problem in GTK and internally in mochitests for window.resizeTo.
Flags: needinfo?(ato)
So it means none of the latest patches as landed in the last days which covered window resizing fixed that yet?
Flags: needinfo?(ato)
(In reply to Henrik Skupin (:whimboo) from comment #7)
> So it means none of the latest patches as landed in the last days which
> covered window resizing fixed that yet?

Apparently not.

I would like to see how this intermittent evolves over the next week or so.
Flags: needinfo?(ato)
Status: NEW → RESOLVED
Last Resolved: 4 months ago
Keywords: bulk-close-intermittents
Resolution: --- → INCOMPLETE

Comment 9

4 months ago
4 failures in 1032 pushes (0.004 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-beta: 4

Platform breakdown:
* linux32-nightly: 3
* linux64-nightly: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1384875&startday=2017-09-11&endday=2017-09-17&tree=all

Comment 10

4 months ago
1 failures in 943 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-beta: 1

Platform breakdown:
* linux32-nightly: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1384875&startday=2017-09-18&endday=2017-09-24&tree=all
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---

Comment 11

3 months ago
1 failures in 947 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-central: 1

Platform breakdown:
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1384875&startday=2017-10-09&endday=2017-10-15&tree=all

Comment 12

3 months ago
21 failures in 116 pushes (0.181 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* autoland: 21

Platform breakdown:
* linux64: 9
* linux32: 5
* windows7-32: 4
* windows10-64: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1384875&startday=2017-11-03&endday=2017-11-03&tree=all

Comment 13

3 months ago
This bug failed 35 times in the last 7 days. Most than half of the fails are happening on Linux, the rest on Windows. The build types are asan, debug, opt and pgo.

Here is a recent log:
https://treeherder.mozilla.org/logviewer.html#?repo=autoland&job_id=142005695&lineNumber=44659

And here is a part of the log:

[task 2017-11-03T17:41:30.266Z] 17:41:30     INFO -  1509730890258	Marionette	TRACE	1 <- [1,47,null,{}]
[task 2017-11-03T17:41:30.267Z] 17:41:30     INFO -  1509730890260	Marionette	TRACE	1 -> [0,48,"takeScreenshot",{"highlights":null,"full":true,"hash":false,"id":null,"scroll":true}]
[task 2017-11-03T17:41:30.326Z] 17:41:30     INFO -  1509730890319	Marionette	TRACE	1 <- [1,48,null,{"value":"iVBORw0KGgoAAAANSUhEUgAABLAAAASwCAYAAADrIbPPAAAgAElEQVR4nOzdy29c54HmYS8b6P8jQBpo9M6zmAF600tvG7PshSEu1bMJtHSAWXmX7hkDI/fCgAQYi ... AAABgTWABAAAAsCawAAAAAFgTWAAAAACsCSwAAAAA1gQWAAAAAGsCCwAAAIA1gQUAAADAmsACAAAAYE1gAQAAALAmsAAAAABYC3l3cyVvUe7QAAAAAElFTkSuQmCC"}]
[task 2017-11-03T17:41:30.342Z] 17:41:30     INFO -  1509730890331	Marionette	TRACE	1 -> [0,49,"setContext",{"value":"chrome"}]
[task 2017-11-03T17:41:30.343Z] 17:41:30     INFO -  1509730890332	Marionette	TRACE	1 <- [1,49,null,{}]
[task 2017-11-03T17:41:30.344Z] 17:41:30     INFO -  1509730890333	Marionette	TRACE	1 -> [0,50,"getContext",{}]
[task 2017-11-03T17:41:30.344Z] 17:41:30     INFO -  1509730890333	Marionette	TRACE	1 <- [1,50,null,{"value":"chrome"}]
[task 2017-11-03T17:41:30.345Z] 17:41:30     INFO -  1509730890334	Marionette	TRACE	1 -> [0,51,"setContext",{"value":"content"}]
[task 2017-11-03T17:41:30.346Z] 17:41:30     INFO -  1509730890334	Marionette	TRACE	1 <- [1,51,null,{}]
[task 2017-11-03T17:41:30.347Z] 17:41:30     INFO -  1509730890335	Marionette	TRACE	1 -> [0,52,"getPageSource",{}]
[task 2017-11-03T17:41:30.357Z] 17:41:30     INFO - TEST-UNEXPECTED-FAIL | test_tabbar_session_restore_button.py TestTabbarSessionRestoreButton.test_window_resizing | AssertionError: u'hidden' != 'visible'
[task 2017-11-03T17:41:30.357Z] 17:41:30     INFO - Traceback (most recent call last):
[task 2017-11-03T17:41:30.358Z] 17:41:30     INFO -   File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 156, in run
[task 2017-11-03T17:41:30.358Z] 17:41:30     INFO -     testMethod()
[task 2017-11-03T17:41:30.358Z] 17:41:30     INFO -   File "/builds/worker/workspace/build/tests/firefox-ui/tests/functional/sessionstore/test_tabbar_session_restore_button.py", line 73, in test_window_resizing
[task 2017-11-03T17:41:30.358Z] 17:41:30     INFO -     self.assertEqual(wrapper.value_of_css_property('visibility'), 'visible')
[task 2017-11-03T17:41:30.358Z] 17:41:30     INFO - TEST-INFO took 9142ms
[task 2017-11-03T17:41:30.358Z] 17:41:30     INFO -  1509730890339	Marionette	TRACE	1 <- [1,52,null,{"value":"<html><head></head><body></body></html>"}]
[task 2017-11-03T17:41:30.359Z] 17:41:30     INFO -  1509730890339	Marionette	TRACE	1 -> [0,53,"setContext",{"value":"chrome"}]
[task 2017-11-03T17:41:30.359Z] 17:41:30     INFO -  1509730890340	Marionette	TRACE	1 <- [1,53,null,{}]
[task 2017-11-03T17:41:30.360Z] 17:41:30     INFO -  1509730890348	Marionette	TRACE	1 -> [0,54,"getContext",{}]
[task 2017-11-03T17:41:30.361Z] 17:41:30     INFO -  1509730890348	Marionette	TRACE	1 <- [1,54,null,{"value":"chrome"}]
[task 2017-11-03T17:41:30.361Z] 17:41:30     INFO -  1509730890350	Marionette	TRACE	1 -> [0,55,"deleteSession",{}]
[task 2017-11-03T17:41:30.362Z] 17:41:30     INFO -  1509730890353	Marionette	TRACE	1 <- [1,55,null,{}]

:gbrown: Hi, can you please take a look at this bug?
Flags: needinfo?(gbrown)
Whiteboard: [stockwell needswork]
It looks like this sudden spike started with the change from bug 1414252. 

https://treeherder.mozilla.org/#/jobs?repo=autoland&filter-searchStr=firefox-ui&tochange=bedc52fb5267646d5689e4450a455daf66e8e2ac&fromchange=4635df9d3bb6919a7480cdd41cb8c4af5e2d4e71

That change was backed out and the test is no longer failing.
Flags: needinfo?(gbrown)
See Also: → bug 1414252

Comment 15

2 months ago
34 failures in 857 pushes (0.04 failures/push) were associated with this bug in the last 7 days.   

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* autoland: 34

Platform breakdown:
* linux64: 13
* windows7-32: 8
* windows10-64: 7
* linux32: 6

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1384875&startday=2017-10-30&endday=2017-11-05&tree=all
Whiteboard: [stockwell needswork] → [stockwell fixed:backout]
There was a single failure about 25 days ago. Since then it didn't happen anymore beside the issue and backout mentioned above. Lets close as WFM.
Status: REOPENED → RESOLVED
Last Resolved: 4 months ago2 months ago
Resolution: --- → WORKSFORME
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.