Closed
Bug 1763692
Opened 3 years ago
Closed 3 years ago
Intermittent /webdriver/tests/bidi/browsing_context/navigate/navigate.py | test_interactive_simultaneous_navigation - AssertionError: assert 'http://web-platform.test:8000/webdriver/tests/support/inline.py?doc=%3C%21doctype+html%3E%0A%3Cmeta+charset%3DUTF
Categories
(Remote Protocol :: Marionette, defect, P5)
Remote Protocol
Marionette
Tracking
(firefox101 fixed)
RESOLVED
FIXED
101 Branch
| Tracking | Status | |
|---|---|---|
| firefox101 | --- | fixed |
People
(Reporter: intermittent-bug-filer, Assigned: jdescottes)
References
(Regression)
Details
(Keywords: assertion, intermittent-failure, regression)
Attachments
(1 file)
Filed by: abutkovits [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=373751496&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/LU6vout5QRyaGaSjqP-Z8A/runs/0/artifacts/public/logs/live_backing.log
[task 2022-04-07T18:07:04.880Z] 18:07:04 INFO - TEST-PASS | /webdriver/tests/bidi/browsing_context/navigate/navigate.py | test_payload
[task 2022-04-07T18:07:04.880Z] 18:07:04 INFO - TEST-UNEXPECTED-FAIL | /webdriver/tests/bidi/browsing_context/navigate/navigate.py | test_interactive_simultaneous_navigation - AssertionError: assert 'http://web-platform.test:8000/webdriver/tests/support/inline.py?doc=%3C%21doctype+html%3E%0A%3Cmeta+charset%3DUTF-8%3E%0A%3Cdiv%3Efoo%3C%2Fdiv%3E&mime=text%2Fhtml&charset=UTF-8' == 'http://web-platform.test:8000/webdriver/tests/support/inline.py?doc=%3C%21doctype+html%3E%0A%3Cmeta+charset%3DUTF-8%3E%0A%3Ciframe+src%3D%27http%3A%2F%2Fweb-platform.test%3A8000%2Fwebdriver%2Ftests%2Fsupport%2Finline.py%3Fdoc%3D%253C%2521doctype%2Bhtml%253E%250A%253Cmeta%2Bcharset%253DUTF-8%253E%250Aframe1%26mime%3Dtext%252Fhtml%26charset%3DUTF-8%27%3E%3C%2Fiframe%3E%3Ciframe+src%3D%27http%3A%2F%2Fweb-platform.test%3A8000%2Fwebdriver%2Ftests%2Fsupport%2Finline.py%3Fdoc%3D%253C%2521doctype%2Bhtml%253E%250A%253Cmeta%2Bcharset%253DUTF-8%253E%250Aframe2%26mime%3Dtext%252Fhtml%26charset%3DUTF-8%27%3E%3C%2Fiframe%3E&mime=text%2Fhtml&charset=UTF-8'
[task 2022-04-07T18:07:04.880Z] 18:07:04 INFO - bidi_session = <webdriver.bidi.client.BidiSession object at 0x7fe69fe6c390>
[task 2022-04-07T18:07:04.880Z] 18:07:04 INFO - inline = <function inline.<locals>.inline at 0x7fe69feaca70>
[task 2022-04-07T18:07:04.880Z] 18:07:04 INFO - top_context = {'children': [], 'context': 'fa06dc2d-64d1-4d62-b6e1-fda4df39dd45', 'parent': None, 'url': 'about:blank'}
[task 2022-04-07T18:07:04.880Z] 18:07:04 INFO -
[task 2022-04-07T18:07:04.880Z] 18:07:04 INFO - async def test_interactive_simultaneous_navigation(bidi_session, inline, top_context):
[task 2022-04-07T18:07:04.880Z] 18:07:04 INFO - frame1_start_url = inline("frame1")
[task 2022-04-07T18:07:04.880Z] 18:07:04 INFO - frame2_start_url = inline("frame2")
[task 2022-04-07T18:07:04.880Z] 18:07:04 INFO -
[task 2022-04-07T18:07:04.880Z] 18:07:04 INFO - url = inline(
[task 2022-04-07T18:07:04.880Z] 18:07:04 INFO - f"<iframe src='{frame1_start_url}'></iframe><iframe src='{frame2_start_url}'></iframe>"
[task 2022-04-07T18:07:04.880Z] 18:07:04 INFO - )
[task 2022-04-07T18:07:04.880Z] 18:07:04 INFO -
[task 2022-04-07T18:07:04.881Z] 18:07:04 INFO - result = await bidi_session.browsing_context.navigate(
[task 2022-04-07T18:07:04.881Z] 18:07:04 INFO - context=top_context["context"], url=url, wait="complete"
[task 2022-04-07T18:07:04.881Z] 18:07:04 INFO - )
[task 2022-04-07T18:07:04.881Z] 18:07:04 INFO - > assert result["url"] == url
[task 2022-04-07T18:07:04.881Z] 18:07:04 INFO - E AssertionError: assert 'http://web-platform.test:8000/webdriver/tests/support/inline.py?doc=%3C%21doctype+html%3E%0A%3Cmeta+charset%3DUTF-8%3E%0A%3Cdiv%3Efoo%3C%2Fdiv%3E&mime=text%2Fhtml&charset=UTF-8' == 'http://web-platform.test:8000/webdriver/tests/support/inline.py?doc=%3C%21doctype+html%3E%0A%3Cmeta+charset%3DUTF-8%3E%0A%3Ciframe+src%3D%27http%3A%2F%2Fweb-platform.test%3A8000%2Fwebdriver%2Ftests%2Fsupport%2Finline.py%3Fdoc%3D%253C%2521doctype%2Bhtml%253E%250A%253Cmeta%2Bcharset%253DUTF-8%253E%250Aframe1%26mime%3Dtext%252Fhtml%26charset%3DUTF-8%27%3E%3C%2Fiframe%3E%3Ciframe+src%3D%27http%3A%2F%2Fweb-platform.test%3A8000%2Fwebdriver%2Ftests%2Fsupport%2Finline.py%3Fdoc%3D%253C%2521doctype%2Bhtml%253E%250A%253Cmeta%2Bcharset%253DUTF-8%253E%250Aframe2%26mime%3Dtext%252Fhtml%26charset%3DUTF-8%27%3E%3C%2Fiframe%3E&mime=text%2Fhtml&charset=UTF-8'
[task 2022-04-07T18:07:04.881Z] 18:07:04 INFO - E - http://web-platform.test:8000/webdriver/tests/support/inline.py?doc=%3C%21doctype+html%3E%0A%3Cmeta+charset%3DUTF-8%3E%0A%3Ciframe+src%3D%27http%3A%2F%2Fweb-platform.test%3A8000%2Fwebdriver%2Ftests%2Fsupport%2Finline.py%3Fdoc%3D%253C%2521doctype%2Bhtml%253E%250A%253Cmeta%2Bcharset%253DUTF-8%253E%250Aframe1%26mime%3Dtext%252Fhtml%26charset%3DUTF-8%27%3E%3C%2Fiframe%3E%3Ciframe+src%3D%27http%3A%2F%2Fweb-platform.test%3A8000%2Fwebdriver%2Ftests%2Fsupport%2Finline.py%3Fdoc%3D%253C%2521doctype%2Bhtml%253E%250A%253Cmeta%2Bcharset%253DUTF-8%253E%250Aframe2%26mime%3Dtext%252Fhtml%26charset%3DUTF-8%27%3E%3C%2Fiframe%3E&mime=text%2Fhtml&charset=UTF-8
[task 2022-04-07T18:07:04.881Z] 18:07:04 INFO - E + http://web-platform.test:8000/webdriver/tests/support/inline.py?doc=%3C%21doctype+html%3E%0A%3Cmeta+charset%3DUTF-8%3E%0A%3Cdiv%3Efoo%3C%2Fdiv%3E&mime=text%2Fhtml&charset=UTF-8
[task 2022-04-07T18:07:04.881Z] 18:07:04 INFO -
[task 2022-04-07T18:07:04.881Z] 18:07:04 INFO - bidi_session = <webdriver.bidi.client.BidiSession object at 0x7fe69fe6c390>
[task 2022-04-07T18:07:04.881Z] 18:07:04 INFO - frame1_start_url = 'http://web-platform.test:8000/webdriver/tests/support/inline.py?doc=%3C%21doctype+html%3E%0A%3Cmeta+charset%3DUTF-8%3E%0Aframe1&mime=text%2Fhtml&charset=UTF-8'
[task 2022-04-07T18:07:04.881Z] 18:07:04 INFO - frame2_start_url = 'http://web-platform.test:8000/webdriver/tests/support/inline.py?doc=%3C%21doctype+html%3E%0A%3Cmeta+charset%3DUTF-8%3E%0Aframe2&mime=text%2Fhtml&charset=UTF-8'
[task 2022-04-07T18:07:04.881Z] 18:07:04 INFO - inline = <function inline.<locals>.inline at 0x7fe69feaca70>
[task 2022-04-07T18:07:04.881Z] 18:07:04 INFO - result = {'navigation': None,
[task 2022-04-07T18:07:04.881Z] 18:07:04 INFO - 'url': 'http://web-platform.test:8000/webdriver/tests/support/inline.py?doc=%3C%21doctype+html%3E%0A%3Cmeta+charset%3DUTF-8%3E%0A%3Cdiv%3Efoo%3C%2Fdiv%3E&mime=text%2Fhtml&charset=UTF-8'}
[task 2022-04-07T18:07:04.881Z] 18:07:04 INFO - top_context = {'children': [],
[task 2022-04-07T18:07:04.881Z] 18:07:04 INFO - 'context': 'fa06dc2d-64d1-4d62-b6e1-fda4df39dd45',
[task 2022-04-07T18:07:04.881Z] 18:07:04 INFO - 'parent': None,
[task 2022-04-07T18:07:04.881Z] 18:07:04 INFO - 'url': 'about:blank'}
[task 2022-04-07T18:07:04.882Z] 18:07:04 INFO - url = 'http://web-platform.test:8000/webdriver/tests/support/inline.py?doc=%3C%21doctype+html%3E%0A%3Cmeta+charset%3DUTF-8%3E%0A%3Ciframe+src%3D%27http%3A%2F%2Fweb-platform.test%3A8000%2Fwebdriver%2Ftests%2Fsupport%2Finline.py%3Fdoc%3D%253C%2521doctype%2Bhtml%253E%250A%253Cmeta%2Bcharset%253DUTF-8%253E%250Aframe1%26mime%3Dtext%252Fhtml%26charset%3DUTF-8%27%3E%3C%2Fiframe%3E%3Ciframe+src%3D%27http%3A%2F%2Fweb-platform.test%3A8000%2Fwebdriver%2Ftests%2Fsupport%2Finline.py%3Fdoc%3D%253C%2521doctype%2Bhtml%253E%250A%253Cmeta%2Bcharset%253DUTF-8%253E%250Aframe2%26mime%3Dtext%252Fhtml%26charset%3DUTF-8%27%3E%3C%2Fiframe%3E&mime=text%2Fhtml&charset=UTF-8'
[task 2022-04-07T18:07:04.882Z] 18:07:04 INFO -
[task 2022-04-07T18:07:04.882Z] 18:07:04 INFO - tests/web-platform/tests/webdriver/tests/bidi/browsing_context/navigate/navigate.py:28: AssertionError
[task 2022-04-07T18:07:04.882Z] 18:07:04 INFO - .
[task 2022-04-07T18:07:04.882Z] 18:07:04 INFO - TEST-OK | /webdriver/tests/bidi/browsing_context/navigate/navigate.py | took 2335ms
[task 2022-04-07T18:07:04.884Z] 18:07:04 INFO - PID 1106 | 1649354824871 webdriver::server DEBUG -> GET /status
[task 2022-04-07T18:07:04.885Z] 18:07:04 INFO - PID 1106 | 1649354824871 webdriver::server DEBUG <- 200 OK {"value":{"message":"","ready":true}}
[task 2022-04-07T18:07:05.031Z] 18:07:05 INFO - STDOUT: context=top_context["context"], url=url, wait="complete"
[task 2022-04-07T18:07:05.032Z] 18:07:05 INFO - STDOUT: )
[task 2022-04-07T18:07:05.032Z] 18:07:05 INFO - STDOUT: > assert result["url"] == url
[task 2022-04-07T18:07:05.033Z] 18:07:05 INFO - STDOUT: E AssertionError: assert 'http://web-platform.test:8000/webdriver/tests/support/inline.py?doc=%3C%21doctype+html%3E%0A%3Cmeta+charset%3DUTF-8%3E%0A%3Cdiv%3Efoo%3C%2Fdiv%3E&mime=text%2Fhtml&charset=UTF-8' == 'http://web-platform.test:8000/webdriver/tests/support/inline.py?doc=%3C%21doctype+html%3E%0A%3Cmeta+charset%3DUTF-8%3E%0A%3Ciframe+src%3D%27http%3A%2F%2Fweb-platform.test%3A8000%2Fwebdriver%2Ftests%2Fsupport%2Finline.py%3Fdoc%3D%253C%2521doctype%2Bhtml%253E%250A%253Cmeta%2Bcharset%253DUTF-8%253E%250Aframe1%26mime%3Dtext%252Fhtml%26charset%3DUTF-8%27%3E%3C%2Fiframe%3E%3Ciframe+src%3D%27http%3A%2F%2Fweb-platform.test%3A8000%2Fwebdriver%2Ftests%2Fsupport%2Finline.py%3Fdoc%3D%253C%2521doctype%2Bhtml%253E%250A%253Cmeta%2Bcharset%253DUTF-8%253E%250Aframe2%26mime%3Dtext%252Fhtml%26charset%3DUTF-8%27%3E%3C%2Fiframe%3E&mime=text%2Fhtml&charset=UTF-8'
[task 2022-04-07T18:07:05.034Z] 18:07:05 INFO - STDOUT: E - http://web-platform.test:8000/webdriver/tests/support/inline.py?doc=%3C%21doctype+html%3E%0A%3Cmeta+charset%3DUTF-8%3E%0A%3Ciframe+src%3D%27http%3A%2F%2Fweb-platform.test%3A8000%2Fwebdriver%2Ftests%2Fsupport%2Finline.py%3Fdoc%3D%253C%2521doctype%2Bhtml%253E%250A%253Cmeta%2Bcharset%253DUTF-8%253E%250Aframe1%26mime%3Dtext%252Fhtml%26charset%3DUTF-8%27%3E%3C%2Fiframe%3E%3Ciframe+src%3D%27http%3A%2F%2Fweb-platform.test%3A8000%2Fwebdriver%2Ftests%2Fsupport%2Finline.py%3Fdoc%3D%253C%2521doctype%2Bhtml%253E%250A%253Cmeta%2Bcharset%253DUTF-8%253E%250Aframe2%26mime%3Dtext%252Fhtml%26charset%3DUTF-8%27%3E%3C%2Fiframe%3E&mime=text%2Fhtml&charset=UTF-8
[task 2022-04-07T18:07:05.035Z] 18:07:05 INFO - STDOUT: E + http://web-platform.test:8000/webdriver/tests/support/inline.py?doc=%3C%21doctype+html%3E%0A%3Cmeta+charset%3DUTF-8%3E%0A%3Cdiv%3Efoo%3C%2Fdiv%3E&mime=text%2Fhtml&charset=UTF-8
[task 2022-04-07T18:07:05.035Z] 18:07:05 INFO - STDOUT: bidi_session = <webdriver.bidi.client.BidiSession object at 0x7fe69fe6c390>
[task 2022-04-07T18:07:05.035Z] 18:07:05 INFO - STDOUT: frame1_start_url = 'http://web-platform.test:8000/webdriver/tests/support/inline.py?doc=%3C%21doctype+html%3E%0A%3Cmeta+charset%3DUTF-8%3E%0Aframe1&mime=text%2Fhtml&charset=UTF-8'
[task 2022-04-07T18:07:05.036Z] 18:07:05 INFO - STDOUT: frame2_start_url = 'http://web-platform.test:8000/webdriver/tests/support/inline.py?doc=%3C%21doctype+html%3E%0A%3Cmeta+charset%3DUTF-8%3E%0Aframe2&mime=text%2Fhtml&charset=UTF-8'
[task 2022-04-07T18:07:05.036Z] 18:07:05 INFO - STDOUT: inline = <function inline.<locals>.inline at 0x7fe69feaca70>
[task 2022-04-07T18:07:05.037Z] 18:07:05 INFO - STDOUT: result = {'navigation': None,
[task 2022-04-07T18:07:05.037Z] 18:07:05 INFO - 'url': 'http://web-platform.test:8000/webdriver/tests/support/inline.py?doc=%3C%21doctype+html%3E%0A%3Cmeta+charset%3DUTF-8%3E%0A%3Cdiv%3Efoo%3C%2Fdiv%3E&mime=text%2Fhtml&charset=UTF-8'}
[task 2022-04-07T18:07:05.037Z] 18:07:05 INFO - STDOUT: top_context = {'children': [],
[task 2022-04-07T18:07:05.037Z] 18:07:05 INFO - 'context': 'fa06dc2d-64d1-4d62-b6e1-fda4df39dd45',
[task 2022-04-07T18:07:05.038Z] 18:07:05 INFO - 'parent': None,
[task 2022-04-07T18:07:05.038Z] 18:07:05 INFO - 'url': 'about:blank'}
[task 2022-04-07T18:07:05.038Z] 18:07:05 INFO - STDOUT: url = 'http://web-platform.test:8000/webdriver/tests/support/inline.py?doc=%3C%21doctype+html%3E%0A%3Cmeta+charset%3DUTF-8%3E%0A%3Ciframe+src%3D%27http%3A%2F%2Fweb-platform.test%3A8000%2Fwebdriver%2Ftests%2Fsupport%2Finline.py%3Fdoc%3D%253C%2521doctype%2Bhtml%253E%250A%253Cmeta%2Bcharset%253DUTF-8%253E%250Aframe1%26mime%3Dtext%252Fhtml%26charset%3DUTF-8%27%3E%3C%2Fiframe%3E%3Ciframe+src%3D%27http%3A%2F%2Fweb-platform.test%3A8000%2Fwebdriver%2Ftests%2Fsupport%2Finline.py%3Fdoc%3D%253C%2521doctype%2Bhtml%253E%250A%253Cmeta%2Bcharset%253DUTF-8%253E%250Aframe2%26mime%3Dtext%252Fhtml%26charset%3DUTF-8%27%3E%3C%2Fiframe%3E&mime=text%2Fhtml&charset=UTF-8'
[task 2022-04-07T18:07:05.039Z] 18:07:05 INFO - STDOUT: tests/web-platform/tests/webdriver/tests/bidi/browsing_context/navigate/navigate.py
[task 2022-04-07T18:07:05.039Z] 18:07:05 INFO - STDOUT: :28: AssertionError
[task 2022-04-07T18:07:05.040Z] 18:07:05 INFO - STDOUT: =========================== short test summary info ============================
[task 2022-04-07T18:07:05.040Z] 18:07:05 INFO - STDOUT: FAILED tests/web-platform/tests/webdriver/tests/bidi/browsing_context/navigate/navigate.py::test_interactive_simultaneous_navigation
|
||
Comment 1•3 years ago
|
||
:jdescottes, since you are the author of the regressor, bug 1730642, could you take a look?
For more information, please visit auto_nag documentation.
Flags: needinfo?(jdescottes)
|
Assignee | |
Comment 2•3 years ago
|
||
It seems we hit the "unload" timeout:
[39] No navigation detected: http://web-platform.test:8000/webdriver/tests/support/inline.py?doc=%3C%21doctype+html%3E%0A%3Cmeta+charset%3DUTF-8%3E%0A%3Cdiv%3Efoo%3C%2Fdiv%3E&mime=text%2Fhtml&charset=UTF-8
We can also see there is a remoteness change detected right before:
Remoteness change detected. Set new top-level browsing context to 39
And that the url returned is inline("<div>foo</div>") which comes from the previous test test_payload.
Finally the top_context (retrieved as a fixture at the beginning of test) points to about:blank.
All this makes me think that we have a race condition between the first test test_payload, which does a navigation with wait=none, and the second test. This relates to Bug 1763108, where we want to look in details at what happens for successive navigations.
Nevertheless, doing a navigation with wait=none followed by one with wait=complete should work. Maybe we should also isolate the tests a bit more and ensure that all pending navigations from a given tests have completed before moving on to the next one.
|
|
Assignee | |
Comment 3•3 years ago
|
||
I can reproduce locally by performing many (hundreds) of wait=none then wait=complete navigations.
Increasing the navigation timeout doesn't help, the call to browsingContext.loadURI seem to have completely no effect. No navigation is started at all.
I noticed that here we are reusing a context variable that could point to an outdated browser (although I think it would only be possible for wait=interactive navigations, which we don't use here). Using webProgress.browsingContext doesn't help either.
The only way I managed to fix that so far is to reject the waitForNavigation promise in case of timeouts, and attempt the navigation another time in this case.
|
|
||
Updated•3 years ago
|
Has Regression Range: --- → yes
| Comment hidden (Intermittent Failures Robot) |
|
||
Comment 5•3 years ago
|
||
(In reply to Julian Descottes [:jdescottes] from comment #2)
Nevertheless, doing a navigation with
wait=nonefollowed by one withwait=completeshould work. Maybe we should also isolate the tests a bit more and ensure that all pending navigations from a given tests have completed before moving on to the next one.
Note that for the Marionette unit tests and some of the WebDriver tests we mainly run navigation tests in their own tab to just prevent such cases. Maybe that is what we should do here as well.
|
|
||
Comment 6•3 years ago
|
||
Also interesting is that this so far is MacOS only.
|
|
Assignee | |
Comment 7•3 years ago
|
||
(In reply to Henrik Skupin (:whimboo) [⌚️UTC+1] from comment #5)
(In reply to Julian Descottes [:jdescottes] from comment #2)
Nevertheless, doing a navigation with
wait=nonefollowed by one withwait=completeshould work. Maybe we should also isolate the tests a bit more and ensure that all pending navigations from a given tests have completed before moving on to the next one.Note that for the Marionette unit tests and some of the WebDriver tests we mainly run navigation tests in their own tab to just prevent such cases. Maybe that is what we should do here as well.
Yep I agree, even though it's interesting to see this edge case, we should not let the intermittent pollute the CI for to long.
Also my fix to retry navigations, although it works locally, did not seem to fix the issue on try.
I'll take the bug and try to run tests in their own tabs
|
|
Assignee | |
Comment 8•3 years ago
|
||
Pushed by jdescottes@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/798d689629e0
[wdspec] Create a dedicated tab for each browsingContext navigate test r=webdriver-reviewers,whimboo
Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/33621 for changes under testing/web-platform/tests
|
||
Comment 11•3 years ago
|
||
| bugherder | ||
Status: ASSIGNED → RESOLVED
Closed: 3 years ago
status-firefox101:
--- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 101 Branch
Upstream PR merged by moz-wptsync-bot
|
|
||
Updated•3 years ago
|
Keywords: regression
| Comment hidden (Intermittent Failures Robot) |
|
||
Comment 14•2 years ago
|
||
Moving bug to Remote Protocol::Marionette component per bug 1815831.
Component: geckodriver → Marionette
Product: Testing → Remote Protocol
You need to log in
before you can comment on or make changes to this bug.
Description
•