1429391 - Intermittent /webdriver/tests/get_current_url/get.py | test_get_current_url_after_modified_location - AssertionError: assert 'file:///' == 'about:blank#wd_test_modification'

Reporter

Description

•

8 years ago

treeherder

Filed by: hskupin [at] mozilla.com https://treeherder.mozilla.org/logviewer.html#?job_id=155307885&repo=mozilla-inbound https://queue.taskcluster.net/v1/task/aAhABBj4TA21UqmPbJpnyQ/runs/0/artifacts/public/logs/live_backing.log

Henrik Skupin [:whimboo][⌚️UTC+2] (away 07/21 - 08/08)

Assignee

Comment 1

•

8 years ago

[task 2018-01-10T11:40:58.412Z] 11:40:58 INFO - session = <webdriver.client.Session object at 0x7fb987f980d0> [task 2018-01-10T11:40:58.413Z] 11:40:58 INFO - [task 2018-01-10T11:40:58.414Z] 11:40:58 INFO - def test_get_current_url_after_modified_location(session): [task 2018-01-10T11:40:58.414Z] 11:40:58 INFO - start = session.transport.send("GET", "session/%s/url" % session.session_id) [task 2018-01-10T11:40:58.415Z] 11:40:58 INFO - session.execute_script("window.location.href = 'about:blank#wd_test_modification'") [task 2018-01-10T11:40:58.416Z] 11:40:58 INFO - wait(session, [task 2018-01-10T11:40:58.417Z] 11:40:58 INFO - lambda s: s.transport.send("GET", "session/%s/url" % session.session_id) != start.body["value"], [task 2018-01-10T11:40:58.417Z] 11:40:58 INFO - "URL did not change") [task 2018-01-10T11:40:58.418Z] 11:40:58 INFO - result = session.transport.send("GET", "session/%s/url" % session.session_id) [task 2018-01-10T11:40:58.418Z] 11:40:58 INFO - [task 2018-01-10T11:40:58.419Z] 11:40:58 INFO - > assert_success(result, "about:blank#wd_test_modification") [task 2018-01-10T11:40:58.420Z] 11:40:58 INFO - [task 2018-01-10T11:40:58.420Z] 11:40:58 INFO - result = <Responsetatus=200 body={"value": "file:///"}> [task 2018-01-10T11:40:58.421Z] 11:40:58 INFO - session = <webdriver.client.Session object at 0x7fb987f980d0> [task 2018-01-10T11:40:58.422Z] 11:40:58 INFO - start = <Responsetatus=200 body={"value": "file:///"}> [task 2018-01-10T11:40:58.422Z] 11:40:58 INFO - [task 2018-01-10T11:40:58.423Z] 11:40:58 INFO - tests/web-platform/tests/webdriver/tests/navigation/current_url.py:86: [task 2018-01-10T11:40:58.424Z] 11:40:58 INFO - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [task 2018-01-10T11:40:58.424Z] 11:40:58 INFO - [task 2018-01-10T11:40:58.425Z] 11:40:58 INFO - response = <Responsetatus=200 body={"value": "file:///"}> [task 2018-01-10T11:40:58.425Z] 11:40:58 INFO - value = 'about:blank#wd_test_modification' [task 2018-01-10T11:40:58.426Z] 11:40:58 INFO - [task 2018-01-10T11:40:58.426Z] 11:40:58 INFO - def assert_success(response, value=None): [task 2018-01-10T11:40:58.426Z] 11:40:58 INFO - """ [task 2018-01-10T11:40:58.427Z] 11:40:58 INFO - Verify that the provided webdriver.Response instance described [task 2018-01-10T11:40:58.427Z] 11:40:58 INFO - a valid error response as defined by `dfn-send-an-error` and [task 2018-01-10T11:40:58.427Z] 11:40:58 INFO - the provided error code. [task 2018-01-10T11:40:58.428Z] 11:40:58 INFO - [task 2018-01-10T11:40:58.428Z] 11:40:58 INFO - :param response: ``webdriver.Response`` instance. [task 2018-01-10T11:40:58.428Z] 11:40:58 INFO - :param value: Expected value of the response body, if any. [task 2018-01-10T11:40:58.429Z] 11:40:58 INFO - """ [task 2018-01-10T11:40:58.429Z] 11:40:58 INFO - assert response.status == 200, str(response.error) [task 2018-01-10T11:40:58.429Z] 11:40:58 INFO - [task 2018-01-10T11:40:58.430Z] 11:40:58 INFO - if value is not None: [task 2018-01-10T11:40:58.430Z] 11:40:58 INFO - > assert response.body["value"] == value [task 2018-01-10T11:40:58.430Z] 11:40:58 INFO - E AssertionError [task 2018-01-10T11:40:58.430Z] 11:40:58 INFO - [task 2018-01-10T11:40:58.431Z] 11:40:58 INFO - response = <Responsetatus=200 body={"value": "file:///"}> [task 2018-01-10T11:40:58.431Z] 11:40:58 INFO - value = 'about:blank#wd_test_modification' [task 2018-01-10T11:40:58.431Z] 11:40:58 INFO - [task 2018-01-10T11:40:58.432Z] 11:40:58 INFO - tests/web-platform/tests/webdriver/tests/support/asserts.py:86: AssertionError [task 2018-01-10T11:40:58.432Z] 11:40:58 INFO - ..

Comment hidden (Intermittent Failures Robot)

Eliza Balazs [:ebalazs_]

Comment 3

•

8 years ago

We have 33 failures in the last 7 days. They occur mostly on Linux (debug and opt) and the rest of them on Linux x64 (opt and debug) and linux64-ccov (opt). Recent log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-central&job_id=157294056&lineNumber=9260 relavant part of it: [task 2018-01-18T23:15:25.400Z] 23:15:25 INFO - TEST-PASS | /webdriver/tests/navigation/current_url.py | test_set_malformed_url 9260 [task 2018-01-18T23:15:25.401Z] 23:15:25 INFO - TEST-UNEXPECTED-FAIL | /webdriver/tests/navigation/current_url.py | test_get_current_url_after_modified_location 9261 [task 2018-01-18T23:15:25.401Z] 23:15:25 INFO - session = <webdriver.client.Session object at 0x7fb48e0f1650> 9262 [task 2018-01-18T23:15:25.402Z] 23:15:25 INFO - 9263 [task 2018-01-18T23:15:25.402Z] 23:15:25 INFO - def test_get_current_url_after_modified_location(session): 9264 [task 2018-01-18T23:15:25.402Z] 23:15:25 INFO - start = session.transport.send("GET", "session/%s/url" % session.session_id) 9265 [task 2018-01-18T23:15:25.403Z] 23:15:25 INFO - session.execute_script("window.location.href = 'about:blank#wd_test_modification'") 9266 [task 2018-01-18T23:15:25.403Z] 23:15:25 INFO - wait(session, 9267 [task 2018-01-18T23:15:25.404Z] 23:15:25 INFO - lambda s: s.transport.send("GET", "session/%s/url" % session.session_id) != start.body["value"], 9268 [task 2018-01-18T23:15:25.404Z] 23:15:25 INFO - "URL did not change") 9269 [task 2018-01-18T23:15:25.404Z] 23:15:25 INFO - result = session.transport.send("GET", "session/%s/url" % session.session_id) :whimboo, can you please have a look at this?

Flags: needinfo?(hskupin)

Whiteboard: [stockwell needswork]

Comment hidden (Intermittent Failures Robot)

Cristina Coroiu [:ccoroiu]

Comment 6

•

8 years ago

We have 33 failures in the last 7 days. For the failure pattern see Comment 3 Recent failure log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=159378918&lineNumber=3685 :whimboo As you are the triage owner of this component, can you please take a look into this?

Comment hidden (Intermittent Failures Robot)

Henrik Skupin [:whimboo][⌚️UTC+2] (away 07/21 - 08/08)

Assignee

Comment 8

•

8 years ago

Since my fix for assert introspection on bug 1433390 has been landed we see better failure messages: [task 2018-02-03T20:17:47.339Z] 20:17:47 INFO - > assert response.body["value"] == value [task 2018-02-03T20:17:47.339Z] 20:17:47 INFO - E AssertionError: assert 'file:///' == 'about:blank#wd_test_modification' [task 2018-02-03T20:17:47.339Z] 20:17:47 INFO - E - file:/// [task 2018-02-03T20:17:47.339Z] 20:17:47 INFO - E + about:blank#wd_test_modification [ I really wonder why a second call to `getURL` right after the call to `wait` returns the original value. David, you landed an improvement for this test already on bug 1407383, but it doesn't seem to have helped. Could you have a look at this test failure again?

Depends on: 1407383

Flags: needinfo?(hskupin) → needinfo?(dburns)

Comment hidden (Intermittent Failures Robot)

Noemi Erli[:noemi_erli]

Comment 10

•

7 years ago

There are 35 failures in the past 7 days. Platforms: - Linux/Linux x64 all debug - linux64-ccov opt Recent log failure: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-central&job_id=162504639&lineNumber=9536 Relevant part of the log: [task 2018-02-15T20:41:57.098Z] 20:41:57 INFO - TEST-PASS | /webdriver/tests/navigation/current_url.py | test_get_current_url_no_browsing_context [task 2018-02-15T20:41:57.098Z] 20:41:57 INFO - TEST-PASS | /webdriver/tests/navigation/current_url.py | test_get_current_url_alert_prompt [task 2018-02-15T20:41:57.098Z] 20:41:57 INFO - TEST-PASS | /webdriver/tests/navigation/current_url.py | test_get_current_url_matches_location [task 2018-02-15T20:41:57.099Z] 20:41:57 INFO - TEST-PASS | /webdriver/tests/navigation/current_url.py | test_get_current_url_payload [task 2018-02-15T20:41:57.099Z] 20:41:57 INFO - TEST-PASS | /webdriver/tests/navigation/current_url.py | test_get_current_url_special_pages [task 2018-02-15T20:41:57.100Z] 20:41:57 INFO - TEST-PASS | /webdriver/tests/navigation/current_url.py | test_get_current_url_file_protocol [task 2018-02-15T20:41:57.100Z] 20:41:57 INFO - TEST-PASS | /webdriver/tests/navigation/current_url.py | test_set_malformed_url [task 2018-02-15T20:41:57.100Z] 20:41:57 INFO - TEST-UNEXPECTED-FAIL | /webdriver/tests/navigation/current_url.py | test_get_current_url_after_modified_location [task 2018-02-15T20:41:57.101Z] 20:41:57 INFO - session = <webdriver.client.Session object at 0x7fb93fc85050> [task 2018-02-15T20:41:57.101Z] 20:41:57 INFO - [task 2018-02-15T20:41:57.102Z] 20:41:57 INFO - def test_get_current_url_after_modified_location(session): [task 2018-02-15T20:41:57.102Z] 20:41:57 INFO - start = session.transport.send("GET", "session/%s/url" % session.session_id) [task 2018-02-15T20:41:57.102Z] 20:41:57 INFO - session.execute_script("window.location.href = 'about:blank#wd_test_modification'") [task 2018-02-15T20:41:57.103Z] 20:41:57 INFO - wait(session, [task 2018-02-15T20:41:57.103Z] 20:41:57 INFO - lambda s: s.transport.send("GET", "session/%s/url" % session.session_id) != start.body["value"], [task 2018-02-15T20:41:57.104Z] 20:41:57 INFO - "URL did not change") [task 2018-02-15T20:41:57.104Z] 20:41:57 INFO - result = session.transport.send("GET", "session/%s/url" % session.session_id) [task 2018-02-15T20:41:57.104Z] 20:41:57 INFO - [task 2018-02-15T20:41:57.104Z] 20:41:57 INFO - > assert_success(result, "about:blank#wd_test_modification") [task 2018-02-15T20:41:57.105Z] 20:41:57 INFO - [task 2018-02-15T20:41:57.105Z] 20:41:57 INFO - result = <Responsetatus=200 body={"value": "file:///"}> [task 2018-02-15T20:41:57.105Z] 20:41:57 INFO - session = <webdriver.client.Session object at 0x7fb93fc85050> [task 2018-02-15T20:41:57.106Z] 20:41:57 INFO - start = <Responsetatus=200 body={"value": "file:///"}> [task 2018-02-15T20:41:57.106Z] 20:41:57 INFO - [task 2018-02-15T20:41:57.106Z] 20:41:57 INFO - tests/web-platform/tests/webdriver/tests/navigation/current_url.py:86: [task 2018-02-15T20:41:57.106Z] 20:41:57 INFO - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ :whimboo :dburns Can you please take a look at this?

Flags: needinfo?(hskupin)

Comment hidden (Intermittent Failures Robot)

Cristina Coroiu [:ccoroiu]

Comment 13

•

7 years ago

In the last 7 days there are 41 failures, since 18th February a decrease can be observed. For failure pattern, please see comment 10 Recent failure log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=164027723&lineNumber=3894 :whimboo, :dburns do you have any updates?

Comment hidden (Intermittent Failures Robot)

Cosmin Sabou [:CosminS]

Comment 17

•

7 years ago

Attached patch Disable on Linux (obsolete) — Details — Splinter Review

Joel, could you please review this patch? Thanks.

Attachment #8955757 - Flags: review?(jmaher)

Joel Maher ( :jmaher ) (UTC -8)

Comment 18

•

7 years ago

Comment on attachment 8955757 [details] [diff] [review] Disable on Linux Review of attachment 8955757 [details] [diff] [review]: ----------------------------------------------------------------- thanks!

Attachment #8955757 - Flags: review?(jmaher) → review+

Joel Maher ( :jmaher ) (UTC -8)

Updated

•

7 years ago

Keywords: checkin-needed, leave-open

Pulsebot

Comment 19

•

7 years ago

Pushed by csabou@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/1a2aa9cd5120 Disable /webdriver/tests/navigation/current_url.py on Linux for frequent failures. r=jmaher

Keywords: checkin-needed

Pulsebot

Comment 20

•

7 years ago

Backout by csabou@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/d50f11db7d64 Backed out changeset 1a2aa9cd5120 for permafailing at web-platform/runtests.py on a CLOSED TREE

Cosmin Sabou [:CosminS]

Comment 21

•

7 years ago

Backed out because of these wpt failures: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&noautoclassify&fromchange=5bd73dd9739efc2521252dd613964c6dca8c7c7e&tochange=d50f11db7d6479c39125a4db75a8a31daeb8fba3&filter-searchStr=wpt8&selectedJob=165759692 Failure log: https://goo.gl/7qaSa9 Joel, could you please shed some light here since I don't know what exactly have I done wrong. Did I edited the wrong manifest? Created this new file in testing/web-platform/meta/webdriver/tests/current_url.py.ini Should I have put the new ini file in testing/web-platform/meta/webdriver/tests/navigation/current_url.py.ini?

Flags: needinfo?(jmaher)

Joel Maher ( :jmaher ) (UTC -8)

Comment 22

•

7 years ago

oh, that is a good point- sorry for missing that in review, it should be in testing/web-platform/meta/webdriver/tests/navigation/current_url.py.ini you can just do: hg mv testing/web-platform/meta/webdriver/tests/current_url.py.ini testing/web-platform/meta/webdriver/tests/navigation/current_url.py.ini

Flags: needinfo?(jmaher)

Cosmin Sabou [:CosminS]

Comment 23

•

7 years ago

Attached patch Disable on Linux — Details — Splinter Review

Done. The thing is that the other 2 patches have failed too (bug 1437081, bug 1398179). I'm looking now over them.

Attachment #8955757 - Attachment is obsolete: true

Attachment #8955777 - Flags: review?(jmaher)

Comment hidden (Intermittent Failures Robot)

Joel Maher ( :jmaher ) (UTC -8)

Comment 25

•

7 years ago

Comment on attachment 8955777 [details] [diff] [review] Disable on Linux Review of attachment 8955777 [details] [diff] [review]: ----------------------------------------------------------------- lets give this a try

Attachment #8955777 - Flags: review?(jmaher) → review+

Pulsebot

Comment 26

•

7 years ago

Pushed by csabou@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/ce01463870bd Disable /webdriver/tests/navigation/current_url.py on Linux for frequent failures. r=jmaher

Joel Maher ( :jmaher ) (UTC -8)

Updated

•

7 years ago

Whiteboard: [stockwell disable-recommended] → [stockwell disabled]

Henrik Skupin [:whimboo][⌚️UTC+2] (away 07/21 - 08/08)

Assignee

Updated

•

7 years ago

status-firefox60: --- → disabled

Flags: needinfo?(hskupin)

Keywords: leave-open → test-disabled

Priority: P5 → P2

Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)

Comment 27

•

7 years ago

bugherder

https://hg.mozilla.org/mozilla-central/rev/ce01463870bd

Status: NEW → RESOLVED

Closed: 7 years ago

Resolution: --- → FIXED

Target Milestone: --- → mozilla60

Henrik Skupin [:whimboo][⌚️UTC+2] (away 07/21 - 08/08)

Assignee

Updated

•

7 years ago

Status: RESOLVED → REOPENED

Resolution: FIXED → ---

Comment hidden (mozreview-request)

David Burns :automatedtester

Updated

•

7 years ago

Flags: needinfo?(dburns)

Comment hidden (Intermittent Failures Robot)

Henrik Skupin [:whimboo][⌚️UTC+2] (away 07/21 - 08/08)

Assignee

Updated

•

7 years ago

status-firefox61: --- → disabled

Summary: Intermittent /webdriver/tests/navigation/current_url.py | test_get_current_url_after_modified_location → Intermittent /webdriver/tests/navigation/current_url.py | test_get_current_url_after_modified_location - AssertionError: assert 'file:///' == 'about:blank#wd_test_modification'

Henrik Skupin [:whimboo][⌚️UTC+2] (away 07/21 - 08/08)

Assignee

Comment 34

•

7 years ago

David, you haven't updated the bug for a while. Should I take care of this disabled test?

Flags: needinfo?(dburns)

Henrik Skupin [:whimboo][⌚️UTC+2] (away 07/21 - 08/08)

Assignee

Comment 35

•

7 years ago

The problematic code is here: https://dxr.mozilla.org/mozilla-central/rev/28db2c96ac695ce77c532901f5431a18a1d44374/testing/web-platform/tests/webdriver/tests/navigation/current_url.py#79-83 `start` is the result of `session.transport.send()`, and inside wait we compare a different result of such a call with `.body["value"]`. That means this is always False and as such we return immediately. In case it's too fast the next assertion will fail. I will get this fixed as part of my patch on bug 1455568.

Assignee: nobody → hskupin

Status: REOPENED → ASSIGNED

Depends on: 1455568
No longer depends on: 1407383

Flags: needinfo?(dburns)

Henrik Skupin [:whimboo][⌚️UTC+2] (away 07/21 - 08/08)

Assignee

Comment 36

•

7 years ago

Bug 1455568 fixed it.

Status: ASSIGNED → RESOLVED

Closed: 7 years ago → 7 years ago

status-firefox62: --- → fixed

Resolution: --- → FIXED

Target Milestone: mozilla60 → mozilla62

Henrik Skupin [:whimboo][⌚️UTC+2] (away 07/21 - 08/08)

Assignee

Updated

•

7 years ago

Keywords: test-disabled

Henrik Skupin [:whimboo][⌚️UTC+2] (away 07/21 - 08/08)

Assignee

Comment 37

•

7 years ago

Sorry, I actually missed to remove the manifest file. Patch upcoming.

Status: RESOLVED → REOPENED

status-firefox62: fixed → disabled

Resolution: FIXED → ---

Henrik Skupin [:whimboo][⌚️UTC+2] (away 07/21 - 08/08)

Assignee

Updated

•

7 years ago

Summary: Intermittent /webdriver/tests/navigation/current_url.py | test_get_current_url_after_modified_location - AssertionError: assert 'file:///' == 'about:blank#wd_test_modification' → Intermittent /webdriver/tests/get_current_url/get.py | test_get_current_url_after_modified_location - AssertionError: assert 'file:///' == 'about:blank#wd_test_modification'

Comment hidden (mozreview-request)

Henrik Skupin [:whimboo][⌚️UTC+2] (away 07/21 - 08/08)

Assignee

Updated

•

7 years ago

Attachment #8974273 - Flags: review?(ato)

Andreas Tolfsen ❲:ato❳

Comment 39

•

7 years ago

mozreview-review

Comment on attachment 8974273 [details] Bug 1429391 - [wdspec] Re-enable /webdriver/tests/get_current_url/get.py. https://reviewboard.mozilla.org/r/242584/#review248520

Attachment #8974273 - Flags: review?(ato) → review+

Henrik Skupin [:whimboo][⌚️UTC+2] (away 07/21 - 08/08)

Assignee

Comment 40

•

7 years ago

mozreview-review-reply

Comment on attachment 8974273 [details] Bug 1429391 - [wdspec] Re-enable /webdriver/tests/get_current_url/get.py. https://reviewboard.mozilla.org/r/242584/#review248520 I noticed that the MANIFEST file contains more changes as wanted. I will update the patch so that only the necessary pieces are getting updated.

Comment hidden (mozreview-request)

Pulsebot

Comment 42

•

7 years ago

Pushed by hskupin@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/4614964298a8 [wdspec] Re-enable /webdriver/tests/get_current_url/get.py. r=ato

Cosmin Sabou [:CosminS]

Comment 43

•

7 years ago

bugherder

https://hg.mozilla.org/mozilla-central/rev/4614964298a8

Status: REOPENED → RESOLVED

Closed: 7 years ago → 7 years ago

Resolution: --- → FIXED

BMO Automation

Comment 44

•

2 years ago

Moving bug to Remote Protocol::Marionette component per bug 1815831.

Component: geckodriver → Marionette

Product: Testing → Remote Protocol

Disable on Linux 7 years ago Cosmin Sabou [:CosminS] 584 bytes, patch	jmaher : review+	Details \| Diff \| Splinter Review
Disable on Linux 7 years ago Cosmin Sabou [:CosminS] 576 bytes, patch	jmaher : review+	Details \| Diff \| Splinter Review
Bug 1429391: Correct variable usage in lambda 7 years ago David Burns :automatedtester 59 bytes, text/x-review-board-request		Details
Bug 1429391: Re-Enable current_url wdspec tests 7 years ago David Burns :automatedtester 59 bytes, text/x-review-board-request		Details
adding some logging 7 years ago David Burns :automatedtester 59 bytes, text/x-review-board-request		Details
Bug 1429391 - [wdspec] Re-enable /webdriver/tests/get_current_url/get.py. 7 years ago Henrik Skupin [:whimboo][⌚️UTC+2] (away 07/21 - 08/08) 59 bytes, text/x-review-board-request	ato : review+	Details