Closed Bug 1491800 Opened 6 years ago Closed 3 years ago

Intermittent /webdriver/tests/maximize_window/maximize.py | test_maximize - assert (800, 600) != (800, 600)

Categories

(Remote Protocol :: Marionette, defect, P5)

Product:
Remote Protocol
Component:
Marionette
Version:
Version 3
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: intermittent-bug-filer, Unassigned)

References

(Blocks 1 open bug, Regression)

Details

(Keywords: intermittent-failure, regression)

Filed by: ebalazs [at] mozilla.com https://treeherder.mozilla.org/logviewer.html#?job_id=199637786&repo=mozilla-inbound https://queue.taskcluster.net/v1/task/Hmq1jWNyTtyCLSn4MVTeHw/runs/0/artifacts/public/logs/live_backing.log > assert before_size != session.window.size [task 2018-09-17T10:40:37.179Z] 10:40:37 INFO - STDOUT: E assert (800, 600) != (800, 600) [task 2018-09-17T10:40:37.179Z] 10:40:37 INFO - STDOUT: E + where (800, 600) = <webdriver.client.Window object at 0x7fbab4a9c650>.size [task 2018-09-17T10:40:37.179Z] 10:40:37 INFO - STDOUT: E + where <webdriver.client.Window object at 0x7fbab4a9c650> = <Session e1c4b659-8146-4002-893b-e8fc32f244ae>.window [task 2018-09-17T10:40:37.180Z] 10:40:37 INFO - STDOUT: before_size = (800, 600) [task 2018-09-17T10:40:37.180Z] 10:40:37 INFO - STDOUT: response = <Responsetatus=200 body={"value": {"y": 0, "x": 0, "height": 600, "width": 800}}> [task 2018-09-17T10:40:37.181Z] 10:40:37 INFO - STDOUT: session = <Session e1c4b659-8146-4002-893b-e8fc32f244ae> [task 2018-09-17T10:40:37.182Z] 10:40:37 INFO - STDOUT: value = {'height': 600, 'width': 800, 'x': 0, 'y': 0} [task 2018-09-17T10:40:37.183Z] 10:40:37 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/maximize_window/maximize.py [task 2018-09-17T10:40:37.184Z] 10:40:37 INFO - STDOUT: :69: AssertionError [task 2018-09-17T10:40:37.184Z] 10:40:37 INFO - STDOUT: =============================== warnings summary =============================== [task 2018-09-17T10:40:37.184Z] 10:40:37 INFO - STDOUT: <undetermined location> [task 2018-09-17T10:40:37.185Z] 10:40:37 INFO - STDOUT: Module already imported so cannot be rewritten: mozlog [task 2018-09-17T10:40:37.186Z] 10:40:37 INFO - STDOUT: -- Docs: http://doc.pytest.org/en/latest/warnings.html [task 2018-09-17T10:40:37.188Z] 10:40:37 INFO - STDOUT: =============== 2 failed, 4 passed, 1 warnings in 17.64 seconds ================ [task 2018-09-17T10:40:37.190Z] 10:40:37 INFO - [task 2018-09-17T10:40:37.191Z] 10:40:37 INFO - TEST-PASS | /webdriver/tests/maximize_window/maximize.py | test_no_browsing_context [task 2018-09-17T10:40:37.191Z] 10:40:37 INFO - TEST-PASS | /webdriver/tests/maximize_window/maximize.py | test_fully_exit_fullscreen [task 2018-09-17T10:40:37.192Z] 10:40:37 INFO - TEST-PASS | /webdriver/tests/maximize_window/maximize.py | test_restore_the_window [task 2018-09-17T10:40:37.192Z] 10:40:37 INFO - TEST-UNEXPECTED-FAIL | /webdriver/tests/maximize_window/maximize.py | test_maximize - assert (800, 600) != (800, 600) [task 2018-09-17T10:40:37.193Z] 10:40:37 INFO - session = <Session e1c4b659-8146-4002-893b-e8fc32f244ae> [task 2018-09-17T10:40:37.193Z] 10:40:37 INFO - [task 2018-09-17T10:40:37.194Z] 10:40:37 INFO - def test_maximize(session): [task 2018-09-17T10:40:37.194Z] 10:40:37 INFO - before_size = session.window.size [task 2018-09-17T10:40:37.194Z] 10:40:37 INFO - [task 2018-09-17T10:40:37.194Z] 10:40:37 INFO - response = maximize(session) [task 2018-09-17T10:40:37.194Z] 10:40:37 INFO - assert_success(response) [task 2018-09-17T10:40:37.195Z] 10:40:37 INFO - [task 2018-09-17T10:40:37.195Z] 10:40:37 INFO - > assert before_size != session.window.size [task 2018-09-17T10:40:37.195Z] 10:40:37 INFO - E assert (800, 600) != (800, 600) [task 2018-09-17T10:40:37.196Z] 10:40:37 INFO - E + where (800, 600) = <webdriver.client.Window object at 0x7fbab4a9c650>.size [task 2018-09-17T10:40:37.196Z] 10:40:37 INFO - E + where <webdriver.client.Window object at 0x7fbab4a9c650> = <Session e1c4b659-8146-4002-893b-e8fc32f244ae>.window [task 2018-09-17T10:40:37.196Z] 10:40:37 INFO - [task 2018-09-17T10:40:37.196Z] 10:40:37 INFO - before_size = (800, 600) [task 2018-09-17T10:40:37.197Z] 10:40:37 INFO - response = <Responsetatus=200 body={"value": {"y": 0, "x": 0, "height": 600, "width": 800}}> [task 2018-09-17T10:40:37.197Z] 10:40:37 INFO - session = <Session e1c4b659-8146-4002-893b-e8fc32f244ae> [task 2018-09-17T10:40:37.197Z] 10:40:37 INFO - [task 2018-09-17T10:40:37.197Z] 10:40:37 INFO - tests/web-platform/tests/webdriver/tests/maximize_window/maximize.py:48: AssertionError [task 2018-09-17T10:40:37.198Z] 10:40:37 INFO - [task 2018-09-17T10:40:37.198Z] 10:40:37 INFO - TEST-UNEXPECTED-FAIL | /webdriver/tests/maximize_window/maximize.py | test_payload - assert (800, 600) != (800, 600) [task 2018-09-17T10:40:37.199Z] 10:40:37 INFO - session = <Session e1c4b659-8146-4002-893b-e8fc32f244ae> [task 2018-09-17T10:40:37.199Z] 10:40:37 INFO - [task 2018-09-17T10:40:37.199Z] 10:40:37 INFO - def test_payload(session): [task 2018-09-17T10:40:37.199Z] 10:40:37 INFO - before_size = session.window.size [task 2018-09-17T10:40:37.200Z] 10:40:37 INFO - [task 2018-09-17T10:40:37.200Z] 10:40:37 INFO - response = maximize(session) [task 2018-09-17T10:40:37.200Z] 10:40:37 INFO - [task 2018-09-17T10:40:37.201Z] 10:40:37 INFO - assert response.status == 200 [task 2018-09-17T10:40:37.201Z] 10:40:37 INFO - assert isinstance(response.body["value"], dict) [task 2018-09-17T10:40:37.201Z] 10:40:37 INFO - [task 2018-09-17T10:40:37.201Z] 10:40:37 INFO - value = response.body["value"] [task 2018-09-17T10:40:37.201Z] 10:40:37 INFO - assert "width" in value [task 2018-09-17T10:40:37.201Z] 10:40:37 INFO - assert "height" in value [task 2018-09-17T10:40:37.202Z] 10:40:37 INFO - assert "x" in value [task 2018-09-17T10:40:37.202Z] 10:40:37 INFO - assert "y" in value [task 2018-09-17T10:40:37.202Z] 10:40:37 INFO - assert isinstance(value["width"], int) [task 2018-09-17T10:40:37.202Z] 10:40:37 INFO - assert isinstance(value["height"], int) [task 2018-09-17T10:40:37.203Z] 10:40:37 INFO - assert isinstance(value["x"], int) [task 2018-09-17T10:40:37.203Z] 10:40:37 INFO - assert isinstance(value["y"], int) [task 2018-09-17T10:40:37.204Z] 10:40:37 INFO - [task 2018-09-17T10:40:37.204Z] 10:40:37 INFO - > assert before_size != session.window.size [task 2018-09-17T10:40:37.204Z] 10:40:37 INFO - E assert (800, 600) != (800, 600) [task 2018-09-17T10:40:37.205Z] 10:40:37 INFO - E + where (800, 600) = <webdriver.client.Window object at 0x7fbab4a9c650>.size [task 2018-09-17T10:40:37.205Z] 10:40:37 INFO - E + where <webdriver.client.Window object at 0x7fbab4a9c650> = <Session e1c4b659-8146-4002-893b-e8fc32f244ae>.window [task 2018-09-17T10:40:37.206Z] 10:40:37 INFO - [task 2018-09-17T10:40:37.206Z] 10:40:37 INFO - before_size = (800, 600) [task 2018-09-17T10:40:37.206Z] 10:40:37 INFO - response = <Responsetatus=200 body={"value": {"y": 0, "x": 0, "height": 600, "width": 800}}> [task 2018-09-17T10:40:37.207Z] 10:40:37 INFO - session = <Session e1c4b659-8146-4002-893b-e8fc32f244ae> [task 2018-09-17T10:40:37.207Z] 10:40:37 INFO - value = {'height': 600, 'width': 800, 'x': 0, 'y': 0} [task 2018-09-17T10:40:37.208Z] 10:40:37 INFO - [task 2018-09-17T10:40:37.208Z] 10:40:37 INFO - tests/web-platform/tests/webdriver/tests/maximize_window/maximize.py:69: AssertionError [task 2018-09-17T10:40:37.209Z] 10:40:37 INFO - . [task 2018-09-17T10:40:37.209Z] 10:40:37 INFO - TEST-OK | /webdriver/tests/maximize_window/maximize.py | took 17857ms [task 2018-09-17T10:40:37.246Z] 10:40:37 WARNING - u'runner_teardown': () [task 2018-09-17T10:40:37.247Z] 10:40:37 INFO - Starting runner [task 2018-09-17T10:40:37.270Z] 10:40:37 INFO - PID 14279 | 1537180837265 webdriver::httpapi DEBUG Creating routes [task 2018-09-17T10:40:37.273Z] 10:40:37 INFO - PID 14279 | 1537180837270 geckodriver DEBUG Listening on 127.0.0.1:4444 [task 2018-09-17T10:40:37.773Z] 10:40:37 INFO - WebDriver HTTP server listening at http://127.0.0.1:4444/ [task 2018-09-17T10:40:37.774Z] 10:40:37 INFO - TEST-START | /webdriver/tests/maximize_window/user_prompts.py [task 2018-09-17T10:40:37.976Z] 10:40:37 INFO - STDOUT: ============================= test session starts ============================== [task 2018-09-17T10:40:37.976Z] 10:40:37 INFO - STDOUT: platform linux2 -- Python 2.7.12, pytest-unknown, py-1.5.2, pluggy-0.5.3.dev -- /builds/worker/workspace/build/venv/bin/python [task 2018-09-17T10:40:37.976Z] 10:40:37 INFO - STDOUT: rootdir: /builds/worker/workspace/build, inifile: [task 2018-09-17T10:40:37.977Z] 10:40:37 INFO - STDOUT: collecting ... [task 2018-09-17T10:40:38.065Z] 10:40:38 INFO - STDOUT: collected 18 items [task 2018-09-17T10:40:38.067Z] 10:40:38 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/maximize_window/user_prompts.py::test_accept[capabilities0-alert-None] STDOUT: =============================== warnings summary =============================== [task 2018-09-17T10:46:48.353Z] 10:46:48 INFO - STDOUT: <undetermined location> [task 2018-09-17T10:46:48.353Z] 10:46:48 INFO - STDOUT: Module already imported so cannot be rewritten: mozlog [task 2018-09-17T10:46:48.353Z] 10:46:48 INFO - STDOUT: -- Docs: http://doc.pytest.org/en/latest/warnings.html [task 2018-09-17T10:46:48.354Z] 10:46:48 INFO - STDOUT: ===================== 3 passed, 1 warnings in 0.53 seconds ===================== [task 2018-09-17T10:46:48.403Z] 10:46:48 INFO - ... [task 2018-09-17T10:46:48.403Z] 10:46:48 INFO - TEST-OK | /webdriver/tests/switch_to_parent_frame/switch.py | took 636ms [task 2018-09-17T10:46:48.403Z] 10:46:48 INFO - No more tests [task 2018-09-17T10:46:48.469Z] 10:46:48 WARNING - u'runner_teardown': () [task 2018-09-17T10:46:48.471Z] 10:46:48 INFO - Got 2 unexpected results [task 2018-09-17T10:46:48.471Z] 10:46:48 INFO - SUITE-END | took 768s [task 2018-09-17T10:46:48.532Z] 10:46:48 INFO - Closing logging queue [task 2018-09-17T10:46:48.533Z] 10:46:48 INFO - queue closed [task 2018-09-17T10:46:48.981Z] 10:46:48 ERROR - Return code: 1
The call to `WebDriver:MaximizeWindow` in Marionette returns immediately with the window size not being changed: > INFO - PID 14042 | 1537180824223 webdriver::server DEBUG <- 200 OK {"value":{"x":0,"y":0,"width":800,"height":600}} > INFO - PID 14042 | 1537180824225 webdriver::server DEBUG -> POST /session/e1c4b659-8146-4002-893b-e8fc32f244ae/window/maximize {} > INFO - PID 14042 | 1537180824226 Marionette TRACE 0 -> [0,66,"WebDriver:MaximizeWindow",{}] > INFO - PID 14042 | 1537180826239 Marionette TRACE 0 <- [1,66,null,{"x":0,"y":0,"width":800,"height":600,"state":"maximized"}] Same I can actually also see for other tests like fullscreen: > INFO - PID 14042 | 1537180823823 Marionette TRACE 0 -> [0,28,"WebDriver:FullscreenWindow",{}] > INFO - PID 14042 | 1537180823829 Marionette TRACE 0 <- [1,28,null,{"x":0,"y":0,"width":800,"height":600,"state":"fullscreen"}] As such I assume something is wrong with the screen resolution and it's 800x600 only? Andreas, I think it would make sense to only check for the maximized state in the affected test, and skip checking the action window size. https://dxr.mozilla.org/mozilla-central/rev/7ed950e60f3c1f8a47c117c04124d31e94a66e32/testing/web-platform/tests/webdriver/tests/maximize_window/maximize.py#48
Flags: needinfo?(ato)
We used to check the state field, but we had to yank it from WebDriver Level 1. You’ll note from the Window Rect JSON serialisation that it is not included: https://w3c.github.io/webdriver/#dfn-json-serialization-of-window-rect We should now be able to revive window state and since we already implement it, it should be easy to revert the tests to use that.
Flags: needinfo?(ato)
Please note that this is a high frequent failure for wpt.fyi test results but currently hidden due to bug 1491971 which causes permanent timeouts.
(In reply to Andreas Tolfsen ❲:ato❳ from comment #2) > We used to check the state field, but we had to yank it from WebDriver > Level 1. You’ll note from the Window Rect JSON serialisation that > it is not included: > https://w3c.github.io/webdriver/#dfn-json-serialization-of-window-rect > > We should now be able to revive window state and since we already > implement it, it should be easy to revert the tests to use that. Just to wrap-up from TPAC, this is still what we want? Even that some browsers don't have a way to tell if they are in eg maximized state.
Flags: needinfo?(ato)
You summarised it correctly. We decided not to add the window state back for those reasons.
Flags: needinfo?(ato)
This is still a huge problem in wpt.fyi. I assume that this might be related to the screen size used by those tests, which might be also 800x600, and as such maximize won't cause a different window size. I asked foolip and jugglingmike over on https://github.com/web-platform-tests/wpt/issues/13005#issuecomment-438199267. My proposal would be to first check in all the maximize tests that the window doesn't have the screen size yet, and if it has just reducing it's size before the maximize command.
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → INCOMPLETE
Status: RESOLVED → REOPENED
Keywords: leave-open
Resolution: INCOMPLETE → ---

Sorry for the fuzz. I hope this have been fixed by bug 1589541. Correct me if I'm wrong.

No failures since 4 months. Marking as WFM.

Status: REOPENED → RESOLVED
Closed: 6 years ago → 3 years ago
Keywords: leave-open
Resolution: --- → WORKSFORME
Has Regression Range: --- → yes
Moving bug to Remote Protocol::Marionette component per bug 1815831.
Component: geckodriver → Marionette
Product: Testing → Remote Protocol
You need to log in before you can comment on or make changes to this bug.