Re-enable /webdriver/tests/maximize_window/user_prompts.py

RESOLVED FIXED in Firefox 63

Status

defect
P2
normal
RESOLVED FIXED
2 years ago
9 months ago

People

(Reporter: intermittent-bug-filer, Assigned: whimboo)

Tracking

(Blocks 1 bug, {intermittent-failure})

Version 3
mozilla63
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox61 disabled, firefox62 disabled, firefox63 fixed)

Details

Attachments

(1 attachment)

Reporter

Description

2 years ago
treeherder
Filed by: archaeopteryx [at] coole-files.de

https://treeherder.mozilla.org/logviewer.html#?job_id=130787297&repo=autoland

https://queue.taskcluster.net/v1/task/ZDy6UKwcTbGrOI24yh2Zlw/runs/0/artifacts/public/logs/live_backing.log

[task 2017-09-13T20:46:34.738114Z] 20:46:34     INFO - tests/web-platform/tests/webdriver/tests/contexts/maximize_window.py:107: 
[task 2017-09-13T20:46:34.738323Z] 20:46:34     INFO - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[task 2017-09-13T20:46:34.738551Z] 20:46:34     INFO - 
[task 2017-09-13T20:46:34.738616Z] 20:46:34     INFO - response = wdclient.Response(status=200, body={u'value': {u'y': 0, u'width': 1600, u'x': 0, u'height': 1200}})
[task 2017-09-13T20:46:34.738922Z] 20:46:34     INFO - error_code = 'unexpected alert open'
[task 2017-09-13T20:46:34.739027Z] 20:46:34     INFO - 
[task 2017-09-13T20:46:34.739246Z] 20:46:34     INFO -     def assert_error(response, error_code):
[task 2017-09-13T20:46:34.739577Z] 20:46:34     INFO -         """Verify that the provided wdclient.Response instance described a valid
[task 2017-09-13T20:46:34.739686Z] 20:46:34     INFO -         error response as defined by `dfn-send-an-error` and the provided error
[task 2017-09-13T20:46:34.739969Z] 20:46:34     INFO -         code.
[task 2017-09-13T20:46:34.740127Z] 20:46:34     INFO -     
[task 2017-09-13T20:46:34.740385Z] 20:46:34     INFO -         :param response: wdclient.Response instance
[task 2017-09-13T20:46:34.740764Z] 20:46:34     INFO -         :param error_code: string value of the expected "error code"
[task 2017-09-13T20:46:34.740900Z] 20:46:34     INFO -         """
[task 2017-09-13T20:46:34.741000Z] 20:46:34     INFO - >       assert response.status == errors[error_code]
[task 2017-09-13T20:46:34.741454Z] 20:46:34     INFO - E       assert 200 == 500
[task 2017-09-13T20:46:34.741499Z] 20:46:34     INFO - E        +  where 200 = 200
[task 2017-09-13T20:46:34.741663Z] 20:46:34     INFO - E        +    where 200 = wdclient.Response(status=200, body={u'value': {u'y': 0, u'width': 1600, u'x': 0, u'height': 1200}}).status
[task 2017-09-13T20:46:34.741846Z] 20:46:34     INFO - 
[task 2017-09-13T20:46:34.742089Z] 20:46:34     INFO - tests/web-platform/tests/webdriver/tests/support/asserts.py:65: AssertionError
[task 2017-09-13T20:46:34.742322Z] 20:46:34     INFO - TEST-PASS | /webdriver/tests/contexts/maximize_window.py | maximize_window.py::test_fully_exit_fullscreen 
[task 2017-09-13T20:46:34.742665Z] 20:46:34     INFO - TEST-PASS | /webdriver/tests/contexts/maximize_window.py | maximize_window.py::test_restore_the_window 
[task 2017-09-13T20:46:34.743095Z] 20:46:34     INFO - TEST-UNEXPECTED-FAIL | /webdriver/tests/contexts/maximize_window.py | maximize_window.py::test_maximize 
[task 2017-09-13T20:46:34.743156Z] 20:46:34     INFO - session = <webdriver.client.Session object at 0x7f83888cafd0>
[task 2017-09-13T20:46:34.743191Z] 20:46:34     INFO - 
[task 2017-09-13T20:46:34.743503Z] 20:46:34     INFO -     def test_maximize(session):
[task 2017-09-13T20:46:34.743549Z] 20:46:34     INFO -         """
[task 2017-09-13T20:46:34.743910Z] 20:46:34     INFO -         6. Maximize the window of the current browsing context.
[task 2017-09-13T20:46:34.743954Z] 20:46:34     INFO -     
[task 2017-09-13T20:46:34.744358Z] 20:46:34     INFO -         [...]
[task 2017-09-13T20:46:34.744403Z] 20:46:34     INFO -     
[task 2017-09-13T20:46:34.744741Z] 20:46:34     INFO -         To maximize the window, given an operating system level window with an
[task 2017-09-13T20:46:34.744797Z] 20:46:34     INFO -         associated top-level browsing context, run the implementation-specific
[task 2017-09-13T20:46:34.745138Z] 20:46:34     INFO -         steps to transition the operating system level window into the
[task 2017-09-13T20:46:34.745203Z] 20:46:34     INFO -         maximized window state.  If the window manager supports window
[task 2017-09-13T20:46:34.745342Z] 20:46:34     INFO -         resizing but does not have a concept of window maximation, the window
[task 2017-09-13T20:46:34.745827Z] 20:46:34     INFO -         dimensions must be increased to the maximum available size permitted
[task 2017-09-13T20:46:34.745888Z] 20:46:34     INFO -         by the window manager for the current screen.  Return when the window
[task 2017-09-13T20:46:34.746263Z] 20:46:34     INFO -         has completed the transition, or within an implementation-defined
[task 2017-09-13T20:46:34.746309Z] 20:46:34     INFO -         timeout.
[task 2017-09-13T20:46:34.746366Z] 20:46:34     INFO -     
[task 2017-09-13T20:46:34.746781Z] 20:46:34     INFO -         """
[task 2017-09-13T20:46:34.746836Z] 20:46:34     INFO -         before_size = session.window.size
[task 2017-09-13T20:46:34.747148Z] 20:46:34     INFO -     
[task 2017-09-13T20:46:34.747197Z] 20:46:34     INFO -         response = maximize(session)
[task 2017-09-13T20:46:34.747413Z] 20:46:34     INFO -         assert_success(response)
[task 2017-09-13T20:46:34.747640Z] 20:46:34     INFO -     
[task 2017-09-13T20:46:34.750686Z] 20:46:34     INFO - >       assert before_size != session.window.size
[task 2017-09-13T20:46:34.750871Z] 20:46:34     INFO - E       assert (800, 600) != (800, 600)
[task 2017-09-13T20:46:34.751018Z] 20:46:34     INFO - E        +  where (800, 600) = <webdriver.client.Window object at 0x7f83888caa50>.size
[task 2017-09-13T20:46:34.751242Z] 20:46:34     INFO - E        +    where <webdriver.client.Window object at 0x7f83888caa50> = <webdriver.client.Session object at 0x7f83888cafd0>.window
[task 2017-09-13T20:46:34.751418Z] 20:46:34     INFO -
Looks like we return too early from `maximize` here. But cc'ing Andreas given that this is his area of expertise.
Comment hidden (Intermittent Failures Robot)
Comment hidden (Intermittent Failures Robot)
None of the patches landed last week have made the situation better. Andreas, can you please have a look if using the same technique as we did for minimize window would help here?
Flags: needinfo?(ato)
(In reply to Henrik Skupin (:whimboo) from comment #4)

> None of the patches landed last week have made the situation
> better.

That’s right, they targetted WebDriver:MinimizeWindow.

> Andreas, can you please have a look if using the same technique as
> we did for minimize window would help here?

WebDriver:MaximizeWindow is another beast entirely.  There’s as
far as I know no definitive web platform event you can listen for to
determine that a browser was maximised, because the notion of window
maximisation is a window manager specific concept.

You’ve reviewed this code previously, so you know that on
Linux/X11 in WMs that don’t support maximisation we use a
TimedPromise that is explicitly allowed to time out after we’ve
called ChromeWindow.maximize().  On WMs that do support maximisation
we listen for sizemodechange, but it is probable that this isn’t
sufficient to guarantee that the window outerWidth/outerHeight has
had time to change.

So to answer your question, we can’t use the same technique here
because there’s no definitive event to listen for, but we need to
somehow come up with a solution that either prevents Marionette from
returning to the client before the window dimensions have had time
to propagate to web content, or employ a poll-wait condition in the
test expectations.

I will look into this, but not immediately.
Assignee: nobody → ato
Flags: needinfo?(ato)
thanks for working on this :ato!
Whiteboard: [stockwell needswork]
Comment hidden (Intermittent Failures Robot)
Comment hidden (Intermittent Failures Robot)
Comment hidden (Intermittent Failures Robot)
Comment hidden (Intermittent Failures Robot)
Andreas, since you commented last we got a huge increase of this failure. If we cannot fix it soon it will be disabled. Can you please have a look at? Thanks.
Flags: needinfo?(ato)
(In reply to Henrik Skupin (:whimboo) from comment #11)

> Andreas, since you commented last we got a huge increase of this
> failure. If we cannot fix it soon it will be disabled. Can you
> please have a look at?  Thanks.

I have other priorities right now, I’m afraid.  If we have to
disable it, we have to disable it.
Flags: needinfo?(ato)

Comment 13

2 years ago
Pushed by jmaher@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/bd5dbfc14551
Disable /webdriver/tests/contexts/maximize_window.py on linux for frequent failures. r=me, a=test-only
please remember to re-enable this test case while working on this bug.
Keywords: leave-open
Whiteboard: [stockwell needswork] → [stockwell disabled]
(In reply to Joel Maher ( :jmaher) (UTC-5) from comment #14)
> please remember to re-enable this test case while working on this bug.

Thanks for the quick turnaround, jmaher!  Will do.
Comment hidden (Intermittent Failures Robot)
Blocks conformance.
Blocks: webdriver
Priority: P5 → P2
Andreas, it looks like you missed to work on this high frequent intermittent failure. While it is disabled in our tree it make us look bad on wpt.fyi because most of the maximize window tests (13/20) fail.

Can you please have a look again so that we can get this fixed and the test re-enabled? Thanks.
Flags: needinfo?(ato)
(In reply to Henrik Skupin (:whimboo) from comment #19)
> because most of the maximize window tests (13/20) fail.

Sorry, that was the pass rate... it's actually 7 out of 20 tests which fail.
(In reply to Henrik Skupin (:whimboo) from comment #19)

> Andreas, it looks like you missed to work on this high frequent
> intermittent failure. While it is disabled in our tree it make
> us look bad on wpt.fyi because most of the maximize window tests
> (13/20) fail.
>
> Can you please have a look again so that we can get this fixed and
> the test re-enabled? Thanks.

I’m afraid I can’t commit to work on this at present.
Flags: needinfo?(ato)
Assignee: ato → nobody
I actually triggered a try build to check how the tests are behaving those days:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=3456d9f877006bacc5756d86064aa42d73c1237d&selectedJob=195475660

By surprise they are all green. I triggered some more wdspec jobs to see if we have it at least still as intermittent. If it turns out that the tests are stable, I will go ahead an re-enable them.
It's still all green. So we can re-enable all the maximize window tests.
Assignee: nobody → hskupin
Status: NEW → ASSIGNED
The summary actually doesn't reflect the current situation anymore given that the tests were refactored a while ago. Lets re-enable the user prompt tests.
Keywords: test-disabled
Summary: Intermittent /webdriver/tests/contexts/maximize_window.py | maximize_window.py::test_maximize → Re-enable /webdriver/tests/maximize_window/user_prompts.py
Attachment #9003729 - Flags: review?(james) → review+

Comment 26

9 months ago
Pushed by hskupin@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/b936d934812f
[wdspec] Re-enable maximize window tests.
Status: ASSIGNED → RESOLVED
Last Resolved: 9 months ago
Resolution: --- → FIXED
Whiteboard: [stockwell disabled]
Target Milestone: --- → mozilla63
You need to log in before you can comment on or make changes to this bug.